Load Test tool comparisons
User Jayan Kandathil (firstname.lastname@example.org) posted:
Has anyone done any comparative evaluations of Performance/Test Studio
(Rational), LoadRunner (Mercury) and SilkPerformer (Segue) recently
If yes, could you please reply as to what the result was?
How do any of them compare to e-Load (Empirix) and QALoad (Compuware)?
Anecdotal information will also be useful to me.
I'm currently in the middle of doing the same. Any previous results would
Re: Load Test tool comparisons
User (Steve_Palmo@ibi.com.nospam) posted:
loadtest was cheaptest and most user-friendly plus we had heavy investment
in rational robot.
seque never came for a demo, qaload failed miserably in our environment and
was ridiculously expensive (at the time double the amount as rational
hope this helps.
Re: Load Test tool comparisons
User (email@example.com) posted:
Yes, I have some useful info for you.
Rational's web site has a link to one done by an independent party. I was
not very impressed by the way it was structured, but it does provied some
Sorry I don't have the URL for you. Go to www.rational.com and find the
page for this product and click on the links on the left side of the page.
We use Rational Robot (Team Test with stand-alone and distributed testing
licenses) and Empirix E-Test. Also have WinRunner and VisualTest, but
don't use them.
I use Robot all the time to automate testing of stand-alone, client/server
and web apps (and combinations of the three), including VB 3.0 and 5.0,
COM/DCOM, Sybase, Crystal Reports (VB and Web versions), Excel and VBA,
Outlook, Borland Delphi 1.0 and 5.0, Borland C++ (really old version),
HTML and ASP and in IE5, MS C++ and Java Applets in IE5. Have not yet
used its distributed testing capabilities, but plan to do so the next time
I have a web site to load test (this is usually done by a coworker).
I recently tried to use E-Test 5.01 and 6.0 to load test an extranet site,
with disastrous results.
I was able to script the scenarios and run them individually from E-Tester
(single fat client), but was unable to run any of them from E-Load a
variety of reasons:
- Change to firewall now requires authentication to load page content from
web server outside of our firewall. Not a problem for thick click, but
thin client does not contain IE authentication code that passes uname/pwd,
so page context does not get loaded. This causes validation test to fail
(as I recall, test still aborted even after disabling validation test).
- Both E-Tester and E-Load hang when click on a link opens a separate
- Upgraded to E-Test version 6.0. It was even worse. Does not support
Windows 95 (oops!), which was on all of our test clients. Also, E-Load
hung/crashed every time I tried to run a script with any number of virtual
users (tried thick and thin client).
After exhausting every combination of local/remote fat/thin clients, I ran
out of time. Had to abandon E-Test and do the load test using a
combination of manual input and "regular" Robot scripts (did not do
distributed test using virtual users).
Robot takes more work to script, but seems to be a lot more
dependable/flexible than E-Test. I don't like E-Test's "visual scripting"
model. The only control Robot it was not able to work with was a
Macromedia Flash movie, which is _not_ part of our standard tool set
(E-Test couldn't deal with it either).
I have not had time yet to compare Robot's Load reporting features to
E-Test's, which are purported to be very accurate/informative (I wouldn't
know, since it didn't work for me).