Orthogonal array testing

Orthogonal array testing is a black box testing technique that is a systematic, statistical way of software testing.[1][2] It is used when the number of inputs to the system is relatively small, but too large to allow for exhaustive testing of every possible input to the systems.[1] It is particularly effective in finding errors associated with faulty logic within computer software systems.[1] Orthogonal arrays can be applied in user interface testing, system testing, regression testing, configuration testing and performance testing. The permutations of factor levels comprising a single treatment are so chosen that their responses are uncorrelated and therefore each treatment gives a unique piece of information. The net effects of organizing the experiment in such treatments is that the same piece of information is gathered in the minimum number of experiments.

Background

Orthogonal vector

Orthogonal vectors exhibit orthogonality. Orthogonal vectors exhibit the following properties:

  • Each of the vectors conveys information different from that of any other vector in the sequence, i.e., each vector conveys unique information therefore avoiding redundancy.
  • On a linear addition, the signals may be separated easily.
  • Each of the vectors is statistically independent of the others, i.e., the correlation between them is nil.
  • When linearly added, the resultant is the arithmetic sum of the individual components.

Technique

Consider a system which has three parameters {country; product; sales person} and each of them has three values. To test all the possible combinations of these parameters (i.e. exhaustive testing) we will need a set of 33 = 27 test cases. But instead of testing the system for each combination of parameters, we can use an orthogonal array to select only a subset of these combinations. Using orthogonal array testing, we can maximize the test coverage while minimizing the number of test cases to consider.[3] We here assume that the pair that maximizes interaction between the parameters will have more defects and that the technique works.[3]

Orthogonal array
Test case ↓ Country Product Salesperson
TC-1 DENotebookCharlie
TC-2 DEDesktopBob
TC-3 DEMouseAlice
TC-4 USNotebookBob
TC-5 USDesktopAlice
TC-6 USMouseCharlie
TC-7 GBNotebookAlice
TC-8 GBDesktopCharlie
TC-9 GBMouseBob

Given that assumption, the table shows the set of nine combination of parameters which are sufficient to catch the fault, considering the interaction of the input parameters, which is very effective and economical. The array is orthogonal, because all possible pair-wise combinations between parameters occurs only once.[3]
The given L9 Orthogonal Array assess result of test cases as follows:[1]
Single-mode faults – Single mode faults occur only due to one parameter. For example, in above Orthogonal array if test cases TF-7, TF-8 and TF-9 show error, we can expect that value {Great Britain} of parameter {Country} is causing the error. Likewise we can detect as well as isolate the error.
Double-mode fault – Double mode fault is caused by the two specific parameters values interacting together. Such an interaction is a harmful interaction between interacting parameters.
Multimode faults – If more than two interacting components produce the consistent erroneous output, then it is a multimode fault. Orthogonal array detects the multimode faults.

Refer the following link for more details: http://www.51testing.com/ddimg/uploadsoft/20090113/OATSEN.pdf

Benefits

  • Testing cycle time is reduced and analysis is simpler.
  • Test cases are balanced, so it's straightforward to isolate defects and assess performance. This provides a significant cost savings over pair-wise testing.

References

  1. 1 2 3 4 Pressman, Roger S (2005). Software Engineering: A Practitioner's Approach (6th ed.). McGraw-Hill. ISBN 0-07-285318-2.
  2. ↑ Phadke, Madhav S. "Planning Efficient Software Tests". Phadke Associates, Inc. Numerous articles on utilizing Orthogonal Arrays for Software and System Testing.
  3. 1 2 3 Dustin, Elfriede. "Orthogonally Speaking" (PDF). (subscription required)
  • Rao, Calyampudi Radhakrishna (2009). "Orthogonal arrays". Scholarpedia. 4 (7): 9076. doi:10.4249/scholarpedia.9076.
  • Delius, Gustav W (May 2004). "Orthogonal Arrays (Taguchi Designs)". University of York.
  • Kuhfeld, Warren F. "Orthogonal Arrays". SAS Institute Inc. SAS provides a catalog of over 117,000 orthogonal arrays.
  • Phadke, Madhav S. "Planning Efficient Software Tests". Phadke Associates, Inc. Numerous articles on utilizing Orthogonal Arrays for Software and System Testing.
  • "rdExpert Software for Orthogonal Array Testing". Phadke Associates, Inc. Commercial toolset for Orthogonal Array Testing.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.