Need help getting TeamCity to report the results of my JMeter tests correctly
I am attempting to run JMeter tests from TeamCity and am not getting it to report correctly.
I have reviews these two articles:
As well as reviewing the JMeter plugin for TeamCity located here: https://github.com/jtorgan/jmeter_pl...readme.txt#L36
Currently my tests are running from TeamCity and are creating a logfile (.jtl), however the results are not being reported correctly in TC. I have a POST that is sending back a 500 and in JMeter fails. However, in TeamCity it reports as a pass. Let me also say that this test plan has 7 tests. It only runs 4 then stops. The 4th one fails in JMeter, TC reports it as a pass, but does not continue to run the remaining 3 tests. I believe this is due to the 4th test actually failing and it stopping on the fail.
If I go into TeamCity > Build Features where I have Performance Metrics Calculation and place a check in Assertions in Format Settings, all my tests fail. If I remove the check in Assertions, then all my tests pass. Placing and removing Response Assertion in JMeter does not seem to affect this. I also have placed and removed Generate Summary Results and Assertion Results in order to see if I can get it to report correctly. So far none of these changes have been able to show the correct results from TC which should be 5 passes and 2 fails. (my POST and GET)
Does anyone have any insight as to what I can do in order to get TeamCity to report the results of my JMEter tests correctly?
Thanks in advance
teamcity has special message types you can have your scripts output, and it will be processed by teamcity. With it you can use it to create test results and fail a test.
Check out How to Run JMeter Tests with TeamCity for Continuous Integration article, it covers handling non "200" HTTP response codes.
You can also consider running your JMeter test using Taurus tool as a wrapper, in that case you will be able to specify flexible Pass/Fail Criteria, moreover Taurus will return non-zero exit code in case of i.e. assertion failure so you can fail your Teamcity build basing on the script exit code.