I had these values in variables as double.
(6.00 + 254.21) = 260.21
Is acts as False in the code.
I put it in the Debug viewer. It also says False.
If I add & " " to both sides it says true.
Doubles are not exact numbers, they are floating point numbers. http://en.wikipedia.org/wiki/Floating_point
Double Data Type (Visual Basic)
"Precision. When you work with floating-point numbers, remember that they do not always have a precise representation in memory. This could lead to unexpected results from certain operations, such as value comparison"
Troubleshooting Data Types (Visual Basic)
To compare floating-point quantities
Calculate the absolute value of their difference by using the Abs method of the Math class in the System namespace.
Determine an acceptable maximum difference, such that you can consider the two quantities to be equal for practical purposes if their difference is no larger.
Compare the absolute value of the difference to the acceptable difference.
Last edited by NoUse4aName; 11-18-2013 at 07:22 AM.
Originally Posted by bklabel1
In that case you converted the double to string.
I tried reading the explanation in Troubleshhting Data Types ( VB ) above. They lost me when they said:
For example, 0.5 (= 1/2) and 0.3125 (= 5/16) can be held as precise values, whereas 0.2 (= 1/5) and 0.3 (= 3/10) can be only approximations.
I am happy that I can do the comparison after changing them to a string. I cannot think of a case where this will break.
1/2 and 5/16 are binary fractions where 1/5 and 3/10 are not.
The Floating-Point Guide - Binary Fractions
This is all the theory stuff they teach up front in a computer science class that many people forget about until you encounter these problems.
For your example values anyway you would want to use a currency or decimal data type rather than double.
Originally Posted by NoUse4aName
That doesn't seem right. In most languages, any non-zero/not null value is considered true.
In practice, I wouldn't do boolean expressions directly on a float or double. It makes the code undreadable for beginners looking at the code at the first time.
It's always better to assign it to intermediate variables for clarity.
boolShouldDoAdditionalProcessing = (floatA + floatB) <> 0
if boolShouldDoAdditionalProcessing then
Yeah but what's that got to do with this?
Originally Posted by dlai
The article was pretty confusing. What I took away from it has to do with the bottom (denominator) if the binary number was written as a fraction. If the bottom number is 2, 4, 8, 16 , 32 and so on, the computer can represent the number exactly. Other wise it has to do some rounding.
I should look at the code and find out where and why it started using doubles. Its not all my code.
David and NoUse4aName...I appreciate you assistance.
Here ya go Kevin:
Decimal Data Type (Visual Basic)
I don't think in Var types as a baseline and just looked at it as a math problem to solve with vb
So I found the result just searching "vb adding decimals". Hopefully it helps, and is probably more accurate than doing the String compare which I think you ended up with.
And for further reading about rounding, in case its needed:
How To Implement Custom Rounding Procedures