Thanks:  0
Likes:  0
Dislikes:  0

1. ## No Brainer

Ok, I'm sure this is an easy fix, but I can't seem to get it to work. Here's what I'm trying to do: compare 2 values. And here's how I'm doing it:

if (aOut[3]-aOut[4]-aOut[5] == aOut[6])

This always returns O even when I know it should return 1. When I view the left side of the statement using the watch list winrunner tells me it equals 101.36

if I do this: if (101.36 == aOut[6]), it returns 1.

if I do this: tmp = aOut[3]-aOut[4]-aOut[5], tmp equals 101.36 (via the watch list)

but if I try either tmp == aOut[6] or 101.36 == tmp, both return 0.

Is there something I am doing wrong?

------------------

2. ## Re: No Brainer

My best guess is it's a result of floating-point math rounding errors. Since WR treats variables as strings or numbers as appropriate and does not have integer types and such, it can be a bit troublesome like that sometimes. If it's acceptable to limit the precision to some arbitrary amount, you might have better luck with something like this:

# round to 2 decimals:
leftSide = sprintf("%.2f", aOut[3] - aOut[4] - aOut[5]);
rightSide = sprintf("%.2f", aOut[6]);
if (leftSide == rightSide)
{
# do something...
}

------------------
Charles Reace (charlesDOTreaceATverizonDOTnet)
"Forward!" he cried from the rear, and the front ranks died.
The generals sat, and the lines on the map moved from side to side.

(Roger Waters, "Us and Them")

3. ## Re: No Brainer

That doesn't work either. Left side and right side are equal but when I compare them, it still returns 0.

------------------

4. ## Re: No Brainer

Correction, I had a typo in there. It worked. Thanks a lot. Wish I would have posted sooner. Could've saved myself a lot of time.

Thanks again...

M

------------------

5. ## Re: No Brainer

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by mana22:
Correction, I had a typo in there. It worked. Thanks a lot. Wish I would have posted sooner. Could've saved myself a lot of time.

Thanks again...

M

<HR></BLOCKQUOTE>
Glad you got it working. Some day I'll need you to return the favor, I'm sure.

------------------
Charles Reace (charlesDOTreaceATverizonDOTnet)
"Forward!" he cried from the rear, and the front ranks died.
The generals sat, and the lines on the map moved from side to side.

(Roger Waters, "Us and Them")

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.