We already asked as similar question in the compuware discussion group, but did not get an useful answer. So I gonna ask you guys...
The test scripts we use are written in Java (implementing EasyScript) and we use QALoad 05.02.00 Build 235 to run them. The scripts run fine so far.
Here comes our problem; we have certain transactions within a test run that fail because of business errors. By way of example, a customer is not allowed to conclude a certain contract. Maybe because of the region the customer lives in or some other tricky reason the test can not know about. How can we get information about failed tests in the report? I can see two options, but neither of them work.
- The first approach was to throw an EasyScriptException so that the transaction fails. But throwing an exception kills the whole test run. Setting the error handling in the conductor to "Continue Transaction" does not change this behavior. The third option "Restart Transaction" in not available for the middleware type Java.
- My other ideas was to swallow the exception in the code and use the method REPORT_FAILED() from the Output class to report the failure. According to the API this does only inform the conductor, which is what it does. So this does not change the reported success/failed counter in the Analyze application.
How can i tag a transaction failed without shutting down the test run? I need some mechanism to tell the guy that will run and analyze the test, whether 10 or 999 out of 1000 transactions failed. The way our tests are implemented at the moment the report states 1000 transactions successful even though some of them failed due to business errors.
Are you using the Datapool? If so add the data where in you can complete the whole Transaction.
Validate the data with one user in Conductor before running the actual Test.
There might be some Server generated Numbers or Strings that you need to handel using the DO_GetUniqueString command.
Record Two Scripts as much as exactly as possible (Data, Checkpoints) and on WorkBench>>Tools>>Use Windiff and compare the Two Files --- You will know if there are any different numbers or Strings on Scripts - If there are than you need to Use the above command and handel the same.
As an idea, try using a SCRIPT_MESSAGE command whenever you have an error. You can also create a CUSTOM_COUNTER as well, and track them.
Basically, instead of using the QALoad Error counts and logic, you can implement your own. Track the number of errors with the custom counter, and output a message with script message, and you should be able to manage whatever error handling is required so the transaction restarts. The beauty of the Java middleware is you aren't bogged down with a recorded script - BUT you have to manage everything yourself.
I have implemented a CUSTOM_COUNTER to count the failed transactions. That is already an improvement. Alas that way the failed transactions are still included in the report. With this solution we leastwise have some evidence about the quality of the reports.