Thanks:  0
Likes:  0
Dislikes:  0

# Thread: Number of decimals in Real values

1. ## Number of decimals in Real values

Whenever I convert a string to real or get a real value from an AUT, Silk Test adds additional zeros? E.g. 1.1434 becomes 1.143400? Is there a way to avoid this?
It's important the correct number of decimals are kept even though the value is e.g. "1.1340". So 1.134000 must become 1.1340.

Notice! The number of decimals may wary

Thanks
Christian

2. ## Re: Number of decimals in Real values

No, there's no way of avoiding this when converting to floating point values. Floating point numbers aren't stored as a sequence of digits ("1.1340"), so it doesn't really make sense to say they either have or don't have trailing zeroes. 1.1340 is exactly the same value as 1.134000, the floating point representations of each will be the same.
Sounds like what you're after is some kind of way of representing a number as a value plus a record of the known accuracy of that value. I'd imagine that such things exist, but I haven't heard of silktest having an implementation of it.

3. ## Re: Number of decimals in Real values

Yo yo ... yo I am assuming that you want to convert this to a number because you need to do some arithmetic on the values that you have in the string data type (probably did something like getText())? If my assumption is correct you can do something like the following. If its not correct then why are you converting it to a number? Just do string comparisions, no need to go through and convert to a number.

</font><blockquote><font size="1" face="Verdana, Arial, Helvetica">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">STRING sProductValue,sNewProductValue
NUMBER nProductValue,nTax

sProductValue = &quot;1.1340&quot;
nProductValue = Val(sProductValue)

//do math stuff here like
nTax = 12.50
nProductValue = nProductValue + nTax

sExpectedProductValue = Str(nProductValue,10,4)
sActualProductValue = WhatereverApp.tfTotalPrice.GetText()

Verify(sActualProductValue,sExpectedProductValue)</pre><hr /></blockquote><font size="2" face="Verdana, Arial, Helvetica">Just an example, not sure it compiles, but take note of the Str function. It has decimal stuff you can use, look it up in help. The thing is if you don't know what the number of significant digits are then this may be a bit more difficutlt (ah just do some string manipulation stuff to figure it out). You know like using Avogadros number for calculations and stuff like that may be a problem.

Ok Later Homie, keep it real....

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.