We have IBM Rational ROBOT and I agree with Scott Barber that Rational's VU is the best of the best for performance testing. But all the testing is controlled by Rational TestManager and TestManager will not run in CITRIX Terminal Server environment. Any one have experience with a Performance testing tool that works in the CITRIX environment
Robot works fine in a citrix environment. You just have to use a windows box as a master station on the same subnet as the citrix client machine, then set up network recording to record all traffic to and from that machine.
I guess I don’t understand your answer to my question. CITRIX is a thin-client product that utilizes Windows Terminal Server to deliver the desktop image to the Windows Terminals. I just talked to Toni at IBM Rational again this morning. He is the Head Performance Escalation Engineer. He reaffirmed that there is no thin client technology available from IBM RATIONAL for performance testing. He reconfirmed that Robot VU will not run in Citrix.
I tried it again myself just now to be sure that I was not missing what you were saying. When I open ROBOT in CITRIX the VU record option is grayed out. If you bring ROBOT up outside of CITRIX on the same work station, it can see the CITIRIX window on the screen but it is just a picture You can’t do anything with it as far as recording in the application running in CITRIX with a workstation outside of the CITRIX environment.
So maybe I am not understanding what you are saying in your post. If you have time I would appreciate further clarification on what you mean by "WINDOW BOX AS A MASTER STATION ON THE SAME SUBNET AS THE CITRIX CLIENT MACHINE".
If I can get ROBOT to do VU recording in CITRIX by what ever means, that will save our company thousands of dollars because we will not have to buy a second tool to do performance testing in CITRIX.
Thanks for taking the time to help me with this.
PS: Also.... I found this on the Internet this moring at: http://www.mercuryinteractive.com/cg...r_friendly.cgi
Partner Solution Description
Mercury and Citrix, have announced an expanded relationship whereby the two companies will combine Citrix technology with Mercury Interactive's LoadRunner® and Topaz™ to enable organizations to emulate, test and monitor the performance of applications deployed with Citrix® MetaFrame®. Specifically, the two companies have jointly developed technology to allow LoadRunner and Topaz to emulate, test and monitor Citrix-based application serving environments. The alliance enables Mercury Interactive and Citrix customers to optimize the performance and scalability of their Citrix-based applications and ensure availability, reliability and performance of their deployed systems.
In addition, Mercury has become a Premier Member of Citrix Business Alliance (CBA) program in a newly created category for Testing and Performance Management and Citrix is now a Validated Integration Partner (VIP) in Mercury Interactive's Open Architecture Program (OAP).
So maybe I answered my own question if Rational Robot VU will not work
Thank you. That is almost amazing. Our software is written in PROGRESS language. I went to the Tevron web page and found this:
February 19, 2003 - Tevron LLC introduces support across its CitraTest family of automated testing solutions for all Progress Software based applications.
We will for sure look into this and I will let you know what we find.
Thanks for the valuable tip
[ 09-26-2003, 06:24 AM: Message edited by: Cordell V ]
Take a look at CitraTest from Tevron. We are partnered with them and using it successfully to test Citrix environments. It is really slick.
We're a Rational shop as well and we need to test Citrix. At last month's RUC, I was told from a fairly reputable source that Tevron CitraTest VU was a pretty good tool for performance testing Citrix. We haven't had a chance to look at it yet--if you do, please pass on whatever you find out.
Just to clarify, my solution does not involve Rational being installed on the Citrix machine. I don't understand why the tech support guy hasn't been able to assist you with this. Ask about recording remotely via Network or Proxy recording. It sounds to me like there is a communication issue here. Either I'm missing something, the tech rep is missing something or... something.
Note, I'm not saying this is an easy solution - just that it's possible under the conditions that I think you are describing.
I think I see how Scott and Cordell may be talking at cross purposes here.
It sounds to me as if Cordell is wanting to emulate multiple Citrix client sessions to place load on a citrix server, whilst Scott is talking about recording the traffic *from* the Citrix server (which is effectively an application client) to a back-end server, to allow emulation of the load on the back-end system.
Is that right - or have I just added even more confusion to the mix?
I'll also throw in one more option in response to (my interpretation of) the original question. It is possible to run multiple Citrix client sessions from a single PC, driven by a GUI testing tool. So if one PC can run <n> sessions, <m> PCs can run <n>*<m> sessions. If the target is only to test a single Citrix server to capacity that may not be an unmanageable number of test PCs.
We did this successfully on a project I worked on recently using WinRunner - but I've seen the same technique used successfully with Robot in other environments.
The tricky parts come in scripting the scheduling of the indiviual sessions, allowing for randomised delays, etc, etc - but if you keep it simple, a viable test harness can be built with a couple of weeks effort from a good developer.
The key thing is not to try too many sessions from a single PC - though as Citrix is a very thin client, it may be possible to get away with quite a few. We ran 4 sessions per PC from 10 PCs to reach our target of 40 sessions per Citrix server - but that was with a pretty intensive application - your mileage *will* vary.
Having said all that - CitraTest does sound a good option if you can justify the purchase.
Thanks for your suggestions. There are a couple of issues that I should have better outlined in my original question.
1. Rational Robot VU will not run in the Citrix environment at all. Scott is trying to help me see how it can run outside of Citrix and still test Citrix but I have not gotten the concept yet. Trevron's tool actually does that using OCR... we are going to test it next week. But as for running Robot in Citrix, the VU recording button is grayed out when you bring Robot up in Citrix. So if it were to work it has to be done someway like Scott is suggesting where the controlling PC is outside the Citrix environment and you are just monitoring the API commands going back and forth to the server (which is what VU testing is anyway)
2. Normally you control the Robot performance testing through Rational TestManager once the VU scripts are written. TestManager will not run in CITRIX except as a log. So again as Scott suggested the controlling or master PC has to be outside the Citrix environment. That is why Rational is saying that there is no known solution to performance testing in the thin-client environment right now. BY ANYONE... Even Trevon runs outside Citrix.
3. To bring up multiple sessions of Citrix, you have to have a Rational License for each session. We only have 3 user licenses and a 5 virtual tester license. We will have to buy a 500 virtual tester license from someone to do this.
4. We have about 300 users at any one time across the state from all the school districts logged in and doing some different task. So we need to emulate 300 users doing different tasks, some of them just logged on sitting idle during class.
You said: So if one PC can run <n> sessions, <m> PCs can run <n>*<m> sessions.
If all of those sessions are running the appliction then you are going to need to have a tool like Robot to automate it and have them all running at the same time. In that case then, that is more of a license issue than anything else. Each session running would need a Robot license.
At my previous employment we had unlimited Robot licenses so we did that same thing you are suggesting. We brought up 10 computers with 10 sessions each running Robot staged to enter a time sheet in the application under test. They all were waiting, polling a control file waiting for the GO. When we put data in control file saying GO they all hit the server at the same time and brought the CIRITX farm to its knees. That is what we were trying to demonstrate to the management. We told them that if they installed the new software and let 1200 people all enter their time card at the same time on Friday after noon at 4:45 p.m. that the server could not handle it. To do that, we used 1000 virtual licenses. A 500 Virtual Test licenses from Rational is about $30,000 and a 500 user license from Trevon is $60,000 so this is no small project to undertake.
Lots of variables.
Hope that makes things a little clearer here.
I am hoping I can learn what Scott is suggesting but Rational is saying it is not going to be a reliable way to test and they can not support it at this time doing it that way. So it looks like we are going with Trevon or someone else for now.
[ 09-26-2003, 06:33 AM: Message edited by: Cordell V ]
OK, I understand a bit better what you're trying to do. I'll just have a go at restating the options as I see them - I looked into this quite carefully for the Citrix project I mentioned, so I'm fairly confident about this.
Firstly, it's important to know whether you are trying to test the entire system - including the Citrix server (or probably Citrix server farm), or whether your main concern is the back-end servers being accessed by the applications on the Citrix servers.
1) If the focus is the back-end systems, then using Rational TestManager from a standalone (non-citrix) PC to capture the traffic between the Citrix servers and the back-end, using the network or proxy recording option, should allow you to create VU scripts that you can replay against the back-end servers (assuming that the Citrix application to back-end server communication uses a protocol supported by Rational). I think this is what Scott was suggesting.
The VU scripts that you generate like this should be identical to the scripts that you *would* get if you *could* run a Robot VU recording session with Robot on the actual Citrix server (ie if the VU button wasn't greyed out). Both approaches would see the traffic between the application on the Citrix box and the back-end servers.
2) If you also want to test the behaviour of the Citrix servers under load, you will need to emulate the Citrix ICA client sessions, in order to generate the load on the Citrix servers.
In principle it ought to be possible to emulate the ICA sessions with some sort of virtual user approach. However ICA is not a supported protocol for Rational VU - and is far too complex a protocol to think about doing at the socket level with the Rational tools (IMHO). I recall having heard of a tool from Citrix themselves that does this - though I never managed to get any details on it.
assumin that there is no way to emulate multiple ICA sessions with a VU approach, the only option is to find a way to run enough copies of the Citrix ICA client to produce the load that you need. It sounds as if the Tevron tool is specifically designed to do that.
An alternative is to drive the ICA client sessions with a GUI testing tool like Rational Robot or Mercury WinRunner. Two main problems with this:
a) Robot and WinRunner are designed to emulate a single user from a single PC. For a large test, the logistics of getting enough PCs, desk space to put them on, IP address space, licences, etc, etc - pretty quickly make it impractcal - I generally reckon about 20 - 30 PCs is about the limit.
Running multiple ICA sessions from a single PC is just a way of raising the bar in terms of numbers that can be done in this way. It means less physical PCs and less licences (one Robot instance/licence can run <n> ICA sessions).
The downside of this approach is much more complex test scripts and less use of the inbuilt timing and reporting facilities that TestManager offers.
b) The Citrix ICA client session does not expose the application's underlying GUI objects (drop-down list boxes, etc) to the GUI recording tool. This means that the recorded script is pretty dumb - keystrokes, mouseclicks at pixel coordinates and recognition of state by looking at a bitmap of the screen.
These limitations would make it almost completely useless for functional testing - but for performance testing with a limited set of transactions, some clever tricks and simplifying assumptions in the scripts and a set of identically configured PCs (screen resolution, colour depth, etc!!!) it *can* be done. We did it successfully on a *very* complex application with a *very* rich GUI.
If anyone knows of any other options I'd love to hear about them.
Hope this is some use.