Thanks:  0
Likes:  0
Dislikes:  0

1. ## Mathematics

I had these values in variables as double.

(6.00 + 254.21) = 260.21

Is acts as False in the code.

I put it in the Debug viewer. It also says False.

Why?

2. If I add & " " to both sides it says true.

3. Doubles are not exact numbers, they are floating point numbers. http://en.wikipedia.org/wiki/Floating_point

Double Data Type (Visual Basic)
"Precision. When you work with floating-point numbers, remember that they do not always have a precise representation in memory. This could lead to unexpected results from certain operations, such as value comparison"

Troubleshooting Data Types (Visual Basic)
To compare floating-point quantities
Calculate the absolute value of their difference by using the Abs method of the Math class in the System namespace.
Determine an acceptable maximum difference, such that you can consider the two quantities to be equal for practical purposes if their difference is no larger.
Compare the absolute value of the difference to the acceptable difference.

4. Originally Posted by bklabel1
If I add & " " to both sides it says true.

In that case you converted the double to string.

5. I tried reading the explanation in Troubleshhting Data Types ( VB ) above. They lost me when they said:

For example, 0.5 (= 1/2) and 0.3125 (= 5/16) can be held as precise values, whereas 0.2 (= 1/5) and 0.3 (= 3/10) can be only approximations.

I am happy that I can do the comparison after changing them to a string. I cannot think of a case where this will break.

Thanks,

Kevin

6. 1/2 and 5/16 are binary fractions where 1/5 and 3/10 are not.
The Floating-Point Guide - Binary Fractions

This is all the theory stuff they teach up front in a computer science class that many people forget about until you encounter these problems.

For your example values anyway you would want to use a currency or decimal data type rather than double.

7. Originally Posted by NoUse4aName
1/2 and 5/16 are binary fractions where 1/5 and 3/10 are not.
The Floating-Point Guide - Binary Fractions

This is all the theory stuff they teach up front in a computer science class that many people forget about until you encounter these problems.

For your example values anyway you would want to use a currency or decimal data type rather than double.

That doesn't seem right. In most languages, any non-zero/not null value is considered true.

In practice, I wouldn't do boolean expressions directly on a float or double. It makes the code undreadable for beginners looking at the code at the first time.
It's always better to assign it to intermediate variables for clarity.

boolShouldDoAdditionalProcessing = (floatA + floatB) <> 0
...
endif

8. Originally Posted by dlai
That doesn't seem right. In most languages, any non-zero/not null value is considered true.
Yeah but what's that got to do with this?

9. The article was pretty confusing. What I took away from it has to do with the bottom (denominator) if the binary number was written as a fraction. If the bottom number is 2, 4, 8, 16 , 32 and so on, the computer can represent the number exactly. Other wise it has to do some rounding.

I should look at the code and find out where and why it started using doubles. Its not all my code.

David and NoUse4aName...I appreciate you assistance.

Thanks,

Kevin

10. Here ya go Kevin:
Decimal Data Type (Visual Basic)

I don't think in Var types as a baseline and just looked at it as a math problem to solve with vb

So I found the result just searching "vb adding decimals". Hopefully it helps, and is probably more accurate than doing the String compare which I think you ended up with.

How To Implement Custom Rounding Procedures

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 06:02 AM.