Thread: Writing test cases - orthogonal arrays

1. Writing test cases - orthogonal arrays

My company still struggles with developing test cases. We can build massive arrays to find combinations of scenarios, but still have trouble selecting and condensing into a reasonable number of functional tests.

I am somewhat familiar with orthogonal arrays, and am looking for more resources and tools to develop better test cases.

I know this seems like a basic question, but I don't see a lot of discussion around this fundamental topic.

Thanks !

2. Re: Writing test cases - orthogonal arrays

Orthogonal arrays can be built manually. Unfortunately, such an approach can be very time consuming for anything more than the most simple of arrays. For those interested in pursuing such an approach Hedayat et al‘s book "Orthogonal Arrays: Theory and Applications" provides a good starting point.

Alternatively, tools such as “AETG web” from Telcordia/ARGreenhouse (argreenhouse.com) that use an orthogonal array (or “pairwise” or "combinatorial method") strategy can be used to create an optimal set of test cases which can then be used for manual testing or can be used to drive an automated testing tool.

Other mathematical approaches which aim to make sense of multivariate data in a systematic manner include factor analysis and cluster analysis.

Hope this helps?
Steve

3. Re: Writing test cases - orthogonal arrays

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by jhertzfeld:
My company still struggles with developing test cases. We can build massive arrays to find combinations of scenarios, but still have trouble selecting and condensing into a reasonable number of functional tests.

I am somewhat familiar with orthogonal arrays, and am looking for more resources and tools to develop better test cases.

I know this seems like a basic question, but I don't see a lot of discussion around this fundamental topic.

Thanks !
<HR></BLOCKQUOTE>

Prof. Owen at Stanford has developed programs that construct and manipulate orthogonal arrays. According to him, they may be freely used and shared. His code comes with no warranty of any kind. The Semiconductor Research Corporation and
the National Science Foundation for supported his work.

Go to this url: lib.stat.cmu.edu then select DESIGNS and then pick Owen to download the C code.
(http://lib.stat.cmu.edu/designs/owen.html).

HTH,
Elfriede

------------------
Elfriede Dustin
Author (with Rashka, Paul)of book "Automated Software Testing", July ‘99
Author (with Rashka, McDiarmid) of book "Quality Web Systems: Performance, Security & Usability", July ‘01
http://www.autotestco.com

4. Re: Writing test cases - orthogonal arrays

I looked at the link at it is not that straight forward to understand however I have used this method back in my mechanical engineering days and if you look for books by Taguchi he gives very good explanations on reducing test runs by using these arrays his own experimental design book is very good and easier to understand

5. Re: Writing test cases - orthogonal arrays

OA can help you significantly reduce the number of test cases . let's look at four
parameters like A, B, C and D, each of them
with three 'levels', such as 1, 2 , 3 etc etc . if you are using orthogonal arrays, you're looking at an array where the columns
are mutually orthogonal. for any pair of columns, all combinations of factor levels occur, and they occur an equal number of
times. this would be called an 'L9' design with 9 indicating the nine rows, configurations or in our case, prototypes to be tested.this would mean that nine experiments are to be carried out to
study four variables at three levels. The number of columns of an array represents the maximum number of parameters that can be studied using that array. . in our case it would reduce 3raised4 (81) to nine test cases. now this more of orthogonal arrays. we had used taguchi method and crudely the steps for implementing taguchi's method would be--

figure out the characteristic to be optimized ----> identify the noise and
test conditions-----> identify control factors and their levels----> design
the matrix------>conduct the exp----> analyze the data and optimize
levels------> predict performance at these levels.

of these most are self explanatory except data analysis- for this taguchi uses a signal/noise ratio simply put s/n ratio
is the ratio of the mean (signal) to the standard deviation (noise). as is obvious larger the s/n the better it is.

we do not have enough data currently to claim that this succeded completely. for more reference you could look at the site/ book

[URL=http://www.omega.com/bobi/productpage.asp?id=ME-1001]
this is a good book on using Robust Design methodologies

This is a brief but good article on planning tests using taguchi's method and OAs http://www.stsc.hill.af.mil/CrossTal...t/planning.asp

[This message has been edited by sundara (edited 03-03-2001).]

6. Re: Writing test cases - orthogonal arrays

Just a point to remember about orthogonal arrays. The purpose of using this approach was to reduce the number of tests to find the OPTIMUM setting for a system. E.g if u had an engineering plant and wanted to know what the best temperature, humidity, mix, blah etc would produce the best engineered product (lets call this the factor x u are looking for). Then rather than test every variable lets call theses variables (a, b, c) these are termed as 'factors' also. We can also reduce the number of levels for these factors (levels being the given temperatures for example).
I have used this method extensively in the Engineering industry using a popular approach called the Taguchi method which is all part of a larger field called the design of experiments. The purpose was always to find out what the optimum setting was for a system using the least amount of tests and not just for the reducing the amounts of tests you did but to find the best setting. In this case this methodology is better suited for performance tests where you are trying to find the best settings for given variables/parameters or factors.

E.g. to conclude to get X(best) = A (at right level), B (at right level) and C(at right level).

Thats what the experiment was designed to do.
Hope this helps

7. Re: Writing test cases - orthogonal arrays

I've tried to use pairwise testing several times in my testing career and have seldom found it useful. You need to be sure that each parameter actually has effect on the other parameters or else you might as well do simple path coverage on each parameter. For instance, when first learning pairwise testing, I was taught with an example of a contact's name and using pairwise testing with first, last name and address fields and using different types of data (alpha, numeric, special characters). Pairwise testing proved to be absolutely useless here - data in the first name field had no bearing on data in the last name field. The only time I have found it useful is for coming up with hardware/software configurations on which to test the software (especially when testing installations).

Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.