Thanks:  0
Likes:  0
Dislikes:  0

# Thread: Rounding problem with Str

1. ## Rounding problem with Str

A compatriot pointed this problem out to me and I thought I should bring it to the attention of the community at large. Just in case any of you are using Str to do numerical rounding in financial applications.

Seems that Str screws up quite often when the number you're rounding has a 5 as the number after your rounding position. Should round up, but sometimes rounds down.

</font><blockquote><font size="1" face="Verdana, Arial, Helvetica">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">// 349 * 0.175 = 61.075
// so the two things below SHOULD print the same thing
Print(Str(349 * 0.175, NULL, 2)) // prints 61.07
Print(Str(61.075, NULL, 2)) // prints 61.08

Print (Str (136.3455, NULL, 3)) // prints 136.345</pre><hr /></blockquote><font size="2" face="Verdana, Arial, Helvetica">Anyway, it's a problem. Logged with Segue (but if anyone else wants to also log it, the more the merrier).

Attached is a Round function that works correctly if you need it.

Rounding Function

- Dave

2. ## Re: Rounding problem with Str

Just for kicks, here was Segue's reply:

This function is working as designed. The commentary below explains the rationale behind this theory.
"There are implications of the binary representation of decimal floating-point numbers. That is: there is not necessarily an exact binary representation for any particular decimal floating-point number, and the rounding behavior of any output routine is based on the binary representation, not on the decimal
representation. If it happens that the nearest binary approximation to 136.3455 is actually
less than 136.3455, then when it's rounded to three places, it'll come out as 136.3455--where
136.3455 comes out as 136.3456. The standard C libraries we use for low-level I/O perform this computation and similar behavior are most likely in other applications on the same platform."
Therefore in light of this, if silktest executes the following lines of code:
[ ] Print (Str (136.4455, NULL, 3)) // prints 136.446
[ ] Print (Str (136.3456, NULL, 3)) // prints 136.346
[ ] Print (Str (136.3455, NULL, 3)) // prints
136.345

You will see this theory in action."

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•
Search Engine Optimisation provided by DragonByte SEO v2.0.40 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.