SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 2 of 2
  1. #1
    Senior Member
    Join Date
    Jul 2001
    Location
    Elizabeth, CO, USA
    Posts
    653
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Rounding problem with Str

    Genties and Ladlemen,
    A compatriot pointed this problem out to me and I thought I should bring it to the attention of the community at large. Just in case any of you are using Str to do numerical rounding in financial applications.

    Seems that Str screws up quite often when the number you're rounding has a 5 as the number after your rounding position. Should round up, but sometimes rounds down.

    </font><blockquote><font size="1" face="Verdana, Arial, Helvetica">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">// 349 * 0.175 = 61.075
    // so the two things below SHOULD print the same thing
    Print(Str(349 * 0.175, NULL, 2)) // prints 61.07
    Print(Str(61.075, NULL, 2)) // prints 61.08

    Print (Str (136.3455, NULL, 3)) // prints 136.345</pre><hr /></blockquote><font size="2" face="Verdana, Arial, Helvetica">Anyway, it's a problem. Logged with Segue (but if anyone else wants to also log it, the more the merrier).

    Attached is a Round function that works correctly if you need it.

    Rounding Function

    - Dave
    WaltzingRhino.com
    A concerned Borland customer, a fly in the ointment, a wrench in the works.

  2. #2
    Junior Member
    Join Date
    Jul 2003
    Location
    Framingham
    Posts
    1
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Rounding problem with Str

    Just for kicks, here was Segue's reply:

    This function is working as designed. The commentary below explains the rationale behind this theory.
    "There are implications of the binary representation of decimal floating-point numbers. That is: there is not necessarily an exact binary representation for any particular decimal floating-point number, and the rounding behavior of any output routine is based on the binary representation, not on the decimal
    representation. If it happens that the nearest binary approximation to 136.3455 is actually
    less than 136.3455, then when it's rounded to three places, it'll come out as 136.3455--where
    136.3455 comes out as 136.3456. The standard C libraries we use for low-level I/O perform this computation and similar behavior are most likely in other applications on the same platform."
    Therefore in light of this, if silktest executes the following lines of code:
    [ ] Print (Str (136.4455, NULL, 3)) // prints 136.446
    [ ] Print (Str (136.3456, NULL, 3)) // prints 136.346
    [ ] Print (Str (136.3455, NULL, 3)) // prints
    136.345

    You will see this theory in action."

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 11.54%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 08:33 AM.

Copyright BetaSoft Inc.