I am working on testing software designed to handle a variety of test files such as Word, Excel, Acrobat, txt, and more. The program is designed to extract every readable property available to each file. I understand that it would be simple to perform a positive test (simply throw a large volume of data at the program). The application may crash on a given file during this test, revealing a defect.
However how would I identify files that are handled properly by the app, but unable to extract certain properties? I need help in forming a robust and reliable test plan for an app like this.
You can figure out this problem if divide in pieces then combine it.
First write the test cases for file type - word, excel etc with certain maximum text.
Then test the application with single file type and a file.
After that single file type and mulitilpe file types.
Then test with multiple files type and a single file of each type.
Then test the application with muliple files types and multiple files.
I hope it helps you.
I would suggest you establish a baseline of files by producing each file type and reviewing them for defects of any sort. Then use those baseline files as a comparison for a file comparison tool. This will give you a good start towards what you appear to require.
Success is the ability to go from one failure to another with no loss of enthusiasm.
~ Winston Churchill ~
Do you know what the "readable properties" are for each type of file? If not, that would be step 1.
Then, I would build valid files of each type, probably 3 for each - maximum data, "normal", and minimum. Every "readable property" for each type of file would be represented in these tests. I would run these through and validate everything was extracted. This would form my baseline. Validation can be done through manual inspection or (more efficiently) through a file compare tool. I would certainly plan on using a file compare tool for regression testing.
I would then begin to build the layers on top of that to validate error routines, performance/stress/load, etc.