Comparing variables in QTP
I'm fairly new to QTP so might be mssing something obvious.
I am trying compare 2 values when they are stored as variables, if they match then the step should pass. I grab the values from the screen, convert them to double data type, calculate and then compare. At the moment the 2 variables match in value, length and datatype but when I compare them in an If statement, the step always fails.
'This is the value caculated by the system which I have converted to data type double
systemDealValueForCalculation = CDbl(systemDealValueForCalculation)
'using the other values on the screen, I do my own calculation and convert this to data type double.
myCalculatedDealValue = CDbl((varQuantity * varSharePriceForCalulation) + varCommissionForCalculation + varLevyForCalculation + varStampDutyForCalculation + varComplianceChargeForCalculation)
'this is part of my debugging and shows the value, length and datatype for my 2 variables that I am comparing. The value, length and datatype are same for both variables.
varlen = len(myCalculatedDealValue)
varSlen = len(systemDealValueForCalculation)
'this compares the 2 variables and always fails
If systemDealValueForCalculation = myCalculatedDealValue Then
Reporter.ReportEvent micPass, "Check Order", "The system calculated the Deal Value as '£" & systemDealValueForCalculation & "' which is correct"
Reporter.ReportEvent micFail, "Check Order", "The system calculated the Deal Value as '£" & systemDealValueForCalculation & "' which is not correct. '£" & myCalculatedDealValue & "' is the expected value"
Any help is much appreciated.
It looks like you've been pretty thorough in your checks but maybe try something like this:
print "x" & myCalculatedDealValue & "x"
print "x" & systemDealValueForCalculation & "x"
print myCalculatedDealValue = systemDealValueForCalculation
also, look for any typos in the variable names in your code. maybe set option explicit on for one run, if you can, to make sure you've got no problems with variable names?
Looks like you're just getting into floating point math fun.
Fun with Floating Point Arithmetic, Part Four - Fabulous Adventures In Coding - Site Home - MSDN Blogs
Try using CCur to convert to Currency data type rather than a double it's specifically designed to handle monetary values. Either that or you'll have to do your own significant digit manipulation.
Thanks for this! Using CCur worked.
I read the article from the link but I still don't quite understand how it applies to my situation as I converted both variables to double precision. "systemDealValueForCalculation" is grabbed from the screen and converted to double. "myCalculatedDealValue" is caculated from grabbing other numbers from the screen, adding and multiplying, then converting the result to double precision.
Here's a better article specifically on why you don't want to do a direct compare with doubles. The values in memory are only exact if it is a binary fraction.
Troubleshooting Data Types (Visual Basic)
Makes you wonder why they didn't just make = an invalid operator for doubles.
Last edited by NoUse4aName; 09-30-2014 at 07:59 AM.