Oct 28 2005

Suspect Rounding In .Net

Published by at 11:28 am under .NET

This is an old article and the information contained within it may be out of date, not reflect my current views and/or contain broken links. If you feel this article is still valid and requires updating, you can use the contact form to let me know. However, I make no guarantee that it will get updated.

For the past week (might be a slight exageration) I have been working on two rather complex and long winded calculations for a certain project that I am working on. I have been given a set of data to run through the calculations and a list of expected results. Unfortunately I have been receiving some rather unexpected rounding results and as soon as I manage to fix the rounding for some of the dataset it seems to throw out the results for other items in the dataset.

Unfortunately I am unable to step through the code as it is running the calculation and (until late on Friday) I was only able to output one result value. As a result I debugging has been a rather slow and laborious process. On Friday I implemented a debugging system that let me not only output a final result but also log various values throughout the calculation process.

Today I finally I managed to track down the round problem to the .Net Math.Floor() function.

From the MSDN library:

Returns the largest whole number less than or equal to the specified number.

[Visual Basic]
Public Shared Function Floor(ByVal d As Double) As Double

A number.

Return Value
The largest whole number less than or equal to d. If d is equal to NaN, NegativeInfinity, or PositiveInfinity, then that value is returne

The code that was causing me problems was the following:

Public Function Truncate(number As Double, places As Integer) As Double
    Dim temp As Integer = Math.Floor(number * Math.Pow(10,places))
    Return temp / Math.Pow(10,places)
End Function

What this code does, is take a floating point number and truncate it to a user specified number of decimal places. For example Truncate(125.12542, 2) should give a result of 125.12. It should also be possible to truncate to 0 decimal places using, for example, Truncate(125.12542, 0) should give a result of 125.

The trouble I was getting, however, was that I was calling Truncate(1001.010, 2) and I was getting a result of 1001.00 when the result should have been 1001.01. As you can imagine, when you are working with financial calculations even a penny out is far from acceptable…. all calculations have to be exact. So this problem had to be resolved.

I’m not about to give you a run down of the whole debugging process that I had to go through as it took several days and a lot of caffine and stress. But in the end I tracked to the problem to being the fact that I had used the Double vairable type. For some reason, which I’m still not entirely aware of, the is some odd rounding when rounding Doubles. These issues seem to disappear if you use variables of type Decimal. So I spent a whole day going through my code replacing all instances of Double with Decimal.

Now that I have completed that all my code is now producing the correct results. If only Microsoft had made these issues more clear in their documentation I might have saved myself several days work and a lot of stress.

2 responses so far