SPONSORS:






User Tag List

Results 1 to 7 of 7
  1. #1
    Senior Member
    Join Date
    Jul 1999
    Location
    Bellingham, WA USA
    Posts
    1,323
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Limitation in Val( )

    There appears to be a limitation on the size of the string one can pass to the Val( ) function. Is this a known issue? Is there a work around? This is a pretty serious limitation.

    Example:
    <code>
    // Good behavior...
    String sValToPass = 999999999

    Number n

    n = Val(sValToPass )

    Print("{n}") //Prints 999999999

    // Bad behavior...
    sValToPass = 9999999999

    n = Val(sValToPass )

    Print("{n}") //Prints 1410065406
    </code>

    ------------------

  2. #2
    Senior Member
    Join Date
    Jul 1999
    Location
    Bellingham, WA USA
    Posts
    1,323
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Limitation in Val( )

    New information:

    Further investigation shows that the limitation is actually coming from the return DATA TYPE Number.

    The valid range of NUMBER is -2,147,483,648 to 2,147,383,647 (i.e. -(2 to the power 31) to (2 to the power 31) -1)

    When you try to convert 9,999,999,999 to number you get 1410065406 because 9,999,999,999 - 2,147,483,648 *4 + 1 = 1,410,065,407

    So, do we have any solutions? I can't believe that no one has run into this before. We're considering overloading the Silk functionality, either in 4Test, or doing a dll, but if there is a process out there already, I'd really rather avoid this if possible.

    ------------------

  3. #3
    Member
    Join Date
    Oct 2000
    Location
    Bothell, WA, USA
    Posts
    54
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Limitation in Val( )

    According to SilkTest Help, it looks like NUMBER is supposed to go up to 1.79E308, but it seems to only go up to the INTEGER maximum. Looks like Segue has an issue to address.

    Data type...Range of values
    INTEGER.....Minimum: 2147483648
    ............Maximum: 2147483647
    REAL........Machine-dependent, though on most systems a REAL is 8 bytes.
    ............Range on x86 architectures: the value zero and the range 2.23E308 to 1.79E308, positive and negative
    NUMBER......Covers the range of both INTEGER and REAL.

    ------------------


    [This message has been edited by dano (edited 10-02-2001).]

  4. #4
    Senior Member
    Join Date
    Aug 1999
    Location
    Cambridge, UK
    Posts
    470
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Limitation in Val( )

    For any situation where an integral value is (or could be) greater than the INTEGER maximum, it is easiest to just the REAL datatype I find; if you're already using NUMBER, then I don't think there's any greater overhead in terms of memory or suchlike.

    ------------------

  5. #5
    Senior Member
    Join Date
    Jul 1999
    Location
    Bellingham, WA USA
    Posts
    1,323
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Limitation in Val( )

    I get the exact same results with the REAL datatype.

    ------------------

  6. #6
    Senior Member
    Join Date
    Aug 1999
    Location
    Cambridge, UK
    Posts
    470
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Limitation in Val( )

    Oh yeah you do; that's absolutely shocking. In past situations where I've used a real to hold a large integral value, I've never had to do it using Val() I suppose.

    I'd imagine it's possible to write your own function which is able to spot strings which are likely to cause this bug to occur, and work around the problem. It might be a bit slow perhaps, but it could work. You'd have to count the number of digits in the string, taking into account minus signs and decimal points, and then go through each digit and perform Val on it individually, and multiply the result by ten to the power of it's position in the string.

    Actually, an easier way of doing it is to check whether there's a decimal point in the string; if there isn't, then just add ".0" on the end (giving you e.g. "9999999999.0"). Val() will treat this as a real and give you the right answer.

    ------------------

  7. #7
    Senior Member
    Join Date
    Jul 1999
    Location
    Bellingham, WA USA
    Posts
    1,323
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Limitation in Val( )

    This solution only works to certain extent.
    If the number we are working with is greater than 2 to the power 53 (like below), the the solution of adding the decimal will not work correctly.
    Consider this :

    [ ] String s = "9007199254740993.0"
    [ ] Print(Val(s))

    One of the testers on my team has come up this the attached as an intermediate solution. He has defined a new data type "long number" and put together some methods of doing basic calculations with it.


    ------------------

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.0.9 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Questions / Answers Form provided by vBAnswers (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominatevBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 05:55 AM.

Copyright BetaSoft Inc.