Oct 28 2005
Suspect Rounding In .Net
For the past week (might be a slight exageration) I have been working on two rather complex and long winded calculations for a certain project that I am working on. I have been given a set of data to run through the calculations and a list of expected results. Unfortunately I have been receiving some rather unexpected rounding results and as soon as I manage to fix the rounding for some of the dataset it seems to throw out the results for other items in the dataset.
Unfortunately I am unable to step through the code as it is running the calculation and (until late on Friday) I was only able to output one result value. As a result I debugging has been a rather slow and laborious process. On Friday I implemented a debugging system that let me not only output a final result but also log various values throughout the calculation process.
Today I finally I managed to track down the round problem to the .Net Math.Floor() function.
From the MSDN library:
Returns the largest whole number less than or equal to the specified number.
[Visual Basic]
Public Shared Function Floor(ByVal d As Double) As DoubleParameters
d
A number.Return Value
The largest whole number less than or equal to d. If d is equal to NaN, NegativeInfinity, or PositiveInfinity, then that value is returne
The code that was causing me problems was the following:
Public Function Truncate(number As Double, places As Integer) As Double
Dim temp As Integer = Math.Floor(number * Math.Pow(10,places))
Return temp / Math.Pow(10,places)
End Function
What this code does, is take a floating point number and truncate it to a user specified number of decimal places. For example Truncate(125.12542, 2) should give a result of 125.12. It should also be possible to truncate to 0 decimal places using, for example, Truncate(125.12542, 0) should give a result of 125.
The trouble I was getting, however, was that I was calling Truncate(1001.010, 2) and I was getting a result of 1001.00 when the result should have been 1001.01. As you can imagine, when you are working with financial calculations even a penny out is far from acceptable…. all calculations have to be exact. So this problem had to be resolved.
I’m not about to give you a run down of the whole debugging process that I had to go through as it took several days and a lot of caffine and stress. But in the end I tracked to the problem to being the fact that I had used the Double vairable type. For some reason, which I’m still not entirely aware of, the is some odd rounding when rounding Doubles. These issues seem to disappear if you use variables of type Decimal. So I spent a whole day going through my code replacing all instances of Double with Decimal.
Now that I have completed that all my code is now producing the correct results. If only Microsoft had made these issues more clear in their documentation I might have saved myself several days work and a lot of stress.
[…] 7:26 pm I’ve spouted on about this previously on my blog but I thought it was about time I posted some code that demonstrates some of the problems we’re up against. The problem seems to be that under certain circumstances the .Net Double data type loose its precision and gives you a number that you weren’t expecting. The chances are that the error will be out by 0.00000000000001, but that might be enough to cause an error. […]
The problem with the Double value is that it is a binary representation (0’s and 1’s).
Binary representations cannot represent every possible decimal value.
It’s very important to understand that in the same way that you can’t represent a third exactly in a (finite) decimal expansion, there are lots of numbers which look simple in decimal, but which have long or infinite expansions in a binary expansion. This means that (for instance) a binary floating point variable can’t have the exact value of decimal 0.1.
See “http://www.yoda.arachsys.com/csharp/floatingpoint.html” for more info
If you need to guaranty precision use decimal. Microsoft documentation state “The Decimal value type is appropriate for financial calculations requiring large numbers of significant integral and fractional digits and no round-off errors.”