The Test Management Guide Software Test Guide RSS Feed
*
Accessibility

Background

BS7925-2 Annex B - Guidelines for Testing and Test Measurement

Condition Testing



Annex B   
(informative)

Guidelines for Testing Techniques and Test Measurement

B.9 / B.10 / B.11   Condition Testing

Introduction

Branch Condition Testing, Branch Condition Combination Testing, and Modified Condition Decision Testing are closely related, as are the associated coverage measures. For convenience, these test case design and test measurement techniques are collectively referred to as condition testing.

Condition testing is based upon an analysis of the conditional control flow within the component and is therefore a form of structural testing.

Example

Consider the following fragment of code:

 

if A or (B and C) then

            do_something;

else

            do_something_else;

end if;

 

The Boolean operands within the decision condition are A, B and C. These may themselves be comprised of complex expressions involving relational operators. For example, the Boolean operand A could be an expression such as X>=Y. However, for the sake of clarity, the following examples regard A, B and C as simple Boolean operands.

Branch Condition Testing and Coverage

Branch Condition Coverage would require Boolean operand A to be evaluated both TRUE and FALSE, Boolean operand B to be evaluated both TRUE and FALSE, and Boolean operand C to be evaluated both TRUE and FALSE.

Branch Condition Coverage may therefore be achieved with the following set of test inputs (note that there are alternative sets of test inputs which will also achieve Branch Condition Coverage):

 

Case

A

B

C

1

FALSE

FALSE

FALSE

2

TRUE

TRUE

TRUE

 

Branch Condition Coverage can often be achieved with just two test cases, irrespective of the number of actual Boolean operands comprising the condition.

A further weakness of Branch Condition Coverage is that it can often be achieved without testing both the TRUE and FALSE branches of the decision. For example, the following alternative set of test inputs achieve Branch Condition Coverage, but only test the TRUE outcome of the overall Boolean condition. Thus Branch Condition Coverage will not necessarily subsume Branch Coverage.

Case

A

B

C

1

TRUE

FALSE

FALSE

2

FALSE

TRUE

TRUE

 

Branch Condition Combination Testing and Coverage

Branch Condition Combination Coverage would require all combinations of Boolean operands A, B and C to be evaluated. For the example condition, Branch Condition Combination Coverage can only be achieved with the following set of test inputs:

Case

A

B

C

1

FALSE

FALSE

FALSE

2

TRUE

FALSE

FALSE

3

FALSE

TRUE

FALSE

4

FALSE

FALSE

TRUE

5

TRUE

TRUE

FALSE

6

FALSE

TRUE

TRUE

7

TRUE

FALSE

TRUE

8

TRUE

TRUE

TRUE

 

Branch Condition Combination Coverage is very thorough, requiring 2n test cases to achieve 100% coverage of a condition containing n Boolean operands. This rapidly becomes unachievable for more complex conditions.

Modified Condition Decision Testing and Coverage

Modified Condition Decision Coverage (MCDC) is a pragmatic compromise which requires fewer test cases than Branch Condition Combination Coverage. It is widely used in the development of avionics software, as required by RTCA/DO-178B.

Modified Condition Decision Coverage requires test cases to show that each Boolean operand (A, B and C) can independently affect the outcome of the decision. This is less than all the combinations (as required by Branch Condition Combination Coverage).

For the example decision condition [A  or (B and C)], we first require a pair of test cases where changing the state of A will change the outcome, but B and C remain constant, i.e. that A can independently affect the outcome of the condition:

 

Case

A

B

C

Outcome

A1

FALSE

FALSE

TRUE

FALSE

A2

TRUE

FALSE

TRUE

TRUE

 

Similarly for B, we require a pair of test cases which show that B can independently affect the outcome, with A and C remaining constant:

 

Case

A

B

C

Outcome

B1

FALSE

FALSE

TRUE

FALSE

B2

FALSE

TRUE

TRUE

TRUE

 

Finally for C we require a pair of test cases which show that C can independently affect the outcome, with A and B remaining constant:

 

Case

A

B

C

Outcome

C1

FALSE

TRUE

TRUE

TRUE

C2

FALSE

TRUE

FALSE

FALSE

 

Having created these pairs of test cases for each operand separately, it can be seen that test cases A1 and B1 are the same, and that test cases B2 and C1 are the same. The overall set of test cases to provide 100% MCDC of the example expression is consequently:

 

Case

A

B

C

Outcome

1  (A1, &B1)

FALSE

FALSE

TRUE

FALSE

2  (A2)

TRUE

FALSE

TRUE

TRUE

3  (B2,&C1)

FALSE

TRUE

TRUE

TRUE

4  (C2)

FALSE

TRUE

FALSE

FALSE

 

In summary:

A is shown to independently affect the outcome of the decision condition by test cases 1 and 2;

B is shown to independently affect the outcome of the decision condition by test cases 1 and 3;

C is shown to independently affect the outcome of the decision condition by test cases 3 and 4.

 

Note that there may be alternative solutions to achieving MCDC. For example, A could have been shown to independently affect the outcome of the condition by the following pair of test cases:

 

Case

A

B

C

Outcome

A3

FALSE

TRUE

FALSE

FALSE

A4

TRUE

TRUE

FALSE

TRUE

 

Test case A3 is the same as test case C2 (or 4) above, but test case A4 is one which has not been previously used. However, as MCDC has already been achieved, test case A4 is not required for coverage purposes.

To achieve 100% Modified Condition Decision Coverage requires a minimum of n+1 test cases, and a maximum of 2n test cases, where n is the number of Boolean operands within the decision condition. In contrast, Branch Condition Combination Coverage requires n2 test cases. MCDC is therefore a practical compromise with Branch Condition Combination Coverage where condition expressions involve more than just a few Boolean operands.

Other Boolean Expressions

One weakness of these condition testing and test measurement techniques is that they are vulnerable to the placement of Boolean expressions which control decisions being placed outside of the actual decision condition. For example:

FLAG := A or (B and C);

if FLAG then

            do_something;

else

            do_something_else;

end if;

 

To combat this vulnerability, a practical variation of these condition testing and condition coverage techniques is to design tests for all Boolean expressions, not just those used directly in control flow decisions.

Optimised Expressions

Some programming languages and compilers short circuit the evaluation of Boolean operators.

For example, the C and C++ languages always short circuit the Boolean "and" (&&) and "or" (||) operators, and the Ada language provides special short circuit operators and then and or else.  With these examples, when the outcome of a Boolean operator can be determined from the first operand, then the second operand will not be evaluated.

The consequence is that it will be infeasible to show coverage of one value of the second operand. For a short circuited "and" operator, the feasible combinations are True:True, True:False and False:X, where X is unknown. For a short circuited "or" operator, the feasible combinations are False:False, False:True and True:X.

Other languages and compilers may short circuit the evaluation of Boolean operators in any order.  In this case, the feasible combinations are not known.  The degree of short circuit optimisation of Boolean operators may depend upon compiler switches or may be outside the user's control.

Short circuited control forms present no obstacle to Branch Condition Coverage or Modified Condition Decision Coverage, but they do obstruct the measurement of Branch Condition Combination Coverage. There are situations where it is possible to design test cases which should achieve 100% coverage (from a theoretical point of view), but where it is not possible to actually measure that 100% coverage has been achieved.

Other Branches and Decisions

The above description of condition testing techniques and condition coverage measures is given in terms of branches or decisions which are controlled by Boolean conditions. Other branches and decisions, such as multi-way branches (implemented by "case", "switch" or "computed goto" statements), and counting loops (implemented by "for" or "do" loops) do not use Boolean conditions, and are therefore not addressed by the descriptions.

There are two options available.

The first option is to assume that the branch or decision is actually implemented as an equivalent set of Boolean conditions (irrespective of how the design is coded), to design test cases which would exercise these conditions using one of the condition testing techniques, and to measure coverage as if the equivalent Boolean conditions were actually coded.

The second option is to only use a condition testing test case design technique and a coverage measure as a supplement to branch testing and branch or decision coverage. Branch testing will address all simple decisions, multi-way decisions, and all loops. Condition testing will then address the decisions which include Boolean conditions.

In practice, a set of test cases which achieves 100% coverage by one of these options will also achieve 100% coverage by the other option. However, lower levels of coverage cannot be compared between the two options.



© RuleWorks - All Rights Reserved - Policy - - Sitemap