Performance Testing and File Veification
I am testing a report generating webapplication with Silkperformer wich generates:
- PDFs and displays them via AcroIEhelper.dll and/or
- Excel-Documents (displayed in a separate Excel-Window)
On top of Performance Testing i'd like to verifiy the Report-contents.
Does anybody know how i can save the report documents or even better the document contents in ascii-format with SilkPerformer?
I'd like to compare the contents after the SilkPerformer run with a separate difftool (because of different timestamps and environment-information within the report text)
Any other suggestions?
Thanks in advance!
Re: Performance Testing and File Veification
Sorry, performance tools operate at the protocol level, underneath the browser level which negates add-ins for the loading of Acrobat and Excel, which if hundreds of sessions were to attempt to load these items on a PC you would see lots of smoke coming from your generator.
Ask yourself, are you co-mingling a performance test and a functional test. You should have validated that these business processes are already stable and functionally correct for a single user. If you have not, then you are too early to performance test these business processes: Go directly to a hold state, do not pass GO, do not collect your consulting fee.
You can use correlation mechanisms to capture the returned data between start and end of file markers for either Acrobat or Excel formatted files. You can then use standard language mechanisms for file i/o to write the items to disk. You will need to write your own diff code to compare the files byte by byte or use a system call to reference an existing utility and then open a log file to determine the results. All the time you do this you will be
* increasing dramatically the memory usage of each single virtual user
* increasing dramatically the CPU usage for each virtual user due to
- Disk I/O processing. This is a Kernel level process which much be handled instantly. You have to steal CPU from application layer processes (aka virtual users) to handle this immediate I/O request
- String comparisons. It doesn't matter what language you use, string content verifications are CPU intensive.
But should you? What is the payback? What question are you trying to answer by doing this? IS the question a functional one or a performance one? The last question getting to the heart of the matter, are you comingling your performance and functional testing?
Ask yourself, would it be sufficient to simply understand that the appropriate beginning and end of file markers for Acrobat and Excel file types appear in my returned data stream? This way I can suggest that what appears to be a properly formatted file is returned. If you absolutely must have verification, consider a single PC executing a graphical virtual user (SilkTest) to answer your data validation question.
Replace ineffective offshore contracts, LoadRunnerByTheHour
. Starting @ $19.95/hr USD.
Put us to the test, skilled expertise is less expensive than you might imagine.
Twitter: @LoadRunnerBTH @PerfBytes