Guidelines for Testing Techniques and Test Measurement
B.9 / B.10 / B.11 Condition Testing
Introduction
Branch
Condition Testing, Branch Condition Combination Testing, and Modified Condition
Decision Testing are closely related, as are the associated coverage measures.
For convenience, these test case design and test measurement techniques are
collectively referred to as condition testing.
Condition testing is based upon an
analysis of the conditional control flow within the component and is therefore
a form of structural testing.
Example
Consider
the following fragment of code:
if
A or (B and C) then
do_something;
else
do_something_else;
end
if;
The Boolean operands within the
decision condition are A, B and C. These may themselves be comprised of complex
expressions involving relational operators. For example, the Boolean operand A
could be an expression such as X>=Y. However, for the sake of clarity, the
following examples regard A, B and C as simple Boolean operands.
Branch
Condition Testing and Coverage
Branch Condition Coverage would require
Boolean operand A to be evaluated both TRUE and FALSE, Boolean operand B to be
evaluated both TRUE and FALSE, and Boolean operand C to be evaluated both TRUE
and FALSE.
Branch Condition Coverage may therefore
be achieved with the following set of test inputs (note that there are
alternative sets of test inputs which will also achieve Branch Condition
Coverage):
|
Case
|
A
|
B
|
C
|
|
1
|
FALSE
|
FALSE
|
FALSE
|
|
2
|
TRUE
|
TRUE
|
TRUE
|
Branch Condition Coverage can often be
achieved with just two test cases, irrespective of the number of actual Boolean
operands comprising the condition.
A further weakness of Branch Condition
Coverage is that it can often be achieved without testing both the TRUE and
FALSE branches of the decision. For example, the following alternative set of
test inputs achieve Branch Condition Coverage, but only test the TRUE outcome
of the overall Boolean condition. Thus Branch Condition Coverage will not
necessarily subsume Branch Coverage.
|
Case
|
A
|
B
|
C
|
|
1
|
TRUE
|
FALSE
|
FALSE
|
|
2
|
FALSE
|
TRUE
|
TRUE
|
Branch
Condition Combination Testing and Coverage
Branch
Condition Combination Coverage would require all combinations of Boolean
operands A, B and C to be evaluated. For the example condition, Branch
Condition Combination Coverage can only be achieved with the following set of
test inputs:
|
Case
|
A
|
B
|
C
|
|
1
|
FALSE
|
FALSE
|
FALSE
|
|
2
|
TRUE
|
FALSE
|
FALSE
|
|
3
|
FALSE
|
TRUE
|
FALSE
|
|
4
|
FALSE
|
FALSE
|
TRUE
|
|
5
|
TRUE
|
TRUE
|
FALSE
|
|
6
|
FALSE
|
TRUE
|
TRUE
|
|
7
|
TRUE
|
FALSE
|
TRUE
|
|
8
|
TRUE
|
TRUE
|
TRUE
|
Branch Condition Combination Coverage
is very thorough, requiring 2n test cases to achieve 100% coverage
of a condition containing n Boolean operands. This rapidly becomes unachievable
for more complex conditions.
Modified
Condition Decision Testing and Coverage
Modified
Condition Decision Coverage (MCDC) is a pragmatic compromise which requires
fewer test cases than Branch Condition Combination Coverage. It is widely used
in the development of avionics software, as required by RTCA/DO-178B.
Modified
Condition Decision Coverage requires test cases to show that each Boolean
operand (A, B and C) can independently affect the outcome of the decision. This
is less than all the combinations (as required by Branch Condition Combination
Coverage).
For
the example decision condition [A or (B
and C)], we first require a pair of test cases where changing the state of A
will change the outcome, but B and C remain constant, i.e. that A can
independently affect the outcome of the condition:
|
Case
|
A
|
B
|
C
|
Outcome
|
|
A1
|
FALSE
|
FALSE
|
TRUE
|
FALSE
|
|
A2
|
TRUE
|
FALSE
|
TRUE
|
TRUE
|
Similarly
for B, we require a pair of test cases which show that B can independently
affect the outcome, with A and C remaining constant:
|
Case
|
A
|
B
|
C
|
Outcome
|
|
B1
|
FALSE
|
FALSE
|
TRUE
|
FALSE
|
|
B2
|
FALSE
|
TRUE
|
TRUE
|
TRUE
|
Finally
for C we require a pair of test cases which show that C can independently
affect the outcome, with A and B remaining constant:
|
Case
|
A
|
B
|
C
|
Outcome
|
|
C1
|
FALSE
|
TRUE
|
TRUE
|
TRUE
|
|
C2
|
FALSE
|
TRUE
|
FALSE
|
FALSE
|
Having
created these pairs of test cases for each operand separately, it can be seen
that test cases A1 and B1 are the same, and that test cases B2 and C1 are the
same. The overall set of test cases to provide 100% MCDC of the example
expression is consequently:
|
Case
|
A
|
B
|
C
|
Outcome
|
|
1 (A1, &B1)
|
FALSE
|
FALSE
|
TRUE
|
FALSE
|
|
2 (A2)
|
TRUE
|
FALSE
|
TRUE
|
TRUE
|
|
3 (B2,&C1)
|
FALSE
|
TRUE
|
TRUE
|
TRUE
|
|
4 (C2)
|
FALSE
|
TRUE
|
FALSE
|
FALSE
|
In
summary:
A
is shown to independently affect the outcome of the decision condition by test
cases 1 and 2;
B
is shown to independently affect the outcome of the decision condition by test
cases 1 and 3;
C
is shown to independently affect the outcome of the decision condition by test
cases 3 and 4.
Note
that there may be alternative solutions to achieving MCDC. For example, A could
have been shown to independently affect the outcome of the condition by the
following pair of test cases:
|
Case
|
A
|
B
|
C
|
Outcome
|
|
A3
|
FALSE
|
TRUE
|
FALSE
|
FALSE
|
|
A4
|
TRUE
|
TRUE
|
FALSE
|
TRUE
|
Test
case A3 is the same as test case C2 (or 4) above, but test case A4 is one which
has not been previously used. However, as MCDC has already been achieved, test
case A4 is not required for coverage purposes.
To
achieve 100% Modified Condition Decision Coverage requires a minimum of n+1
test cases, and a maximum of 2n test cases, where n is the number of Boolean
operands within the decision condition. In contrast, Branch Condition
Combination Coverage requires n2 test cases. MCDC is therefore a
practical compromise with Branch Condition Combination Coverage where condition
expressions involve more than just a few Boolean operands.
Other
Boolean Expressions
One
weakness of these condition testing and test measurement techniques is that
they are vulnerable to the placement of Boolean expressions which control
decisions being placed outside of the actual decision condition. For example:
FLAG := A or (B and
C);
if FLAG then
do_something;
else
do_something_else;
end if;
To combat this vulnerability, a
practical variation of these condition testing and condition coverage
techniques is to design tests for all Boolean expressions, not just those used
directly in control flow decisions.
Optimised
Expressions
Some programming languages and
compilers short circuit the evaluation of Boolean operators.
For example, the C and C++ languages
always short circuit the Boolean "and" (&&) and
"or" (||) operators, and the Ada language provides special short
circuit operators and then and or else. With these examples, when the outcome of a Boolean operator can
be determined from the first operand, then the second operand will not be
evaluated.
The consequence is that it will be
infeasible to show coverage of one value of the second operand. For a short
circuited "and" operator, the feasible combinations are True:True,
True:False and False:X, where X is unknown. For a short circuited
"or" operator, the feasible combinations are False:False, False:True
and True:X.
Other languages and compilers may short
circuit the evaluation of Boolean operators in any order. In this case, the feasible combinations are
not known. The degree of short circuit
optimisation of Boolean operators may depend upon compiler switches or may be
outside the user's control.
Short circuited control forms present
no obstacle to Branch Condition Coverage or Modified Condition Decision
Coverage, but they do obstruct the measurement of Branch Condition Combination
Coverage. There are situations where it is possible to design test cases which
should achieve 100% coverage (from a theoretical point of view), but where it
is not possible to actually measure that 100% coverage has been achieved.
Other
Branches and Decisions
The above description of condition
testing techniques and condition coverage measures is given in terms of
branches or decisions which are controlled by Boolean conditions. Other
branches and decisions, such as multi-way branches (implemented by
"case", "switch" or "computed goto" statements),
and counting loops (implemented by "for" or "do" loops) do
not use Boolean conditions, and are therefore not addressed by the
descriptions.
There are two options available.
The first option is to assume that the
branch or decision is actually implemented as an equivalent set of Boolean
conditions (irrespective of how the design is coded), to design test cases
which would exercise these conditions using one of the condition testing
techniques, and to measure coverage as if the equivalent Boolean conditions were
actually coded.
The second option is to only use a
condition testing test case design technique and a coverage measure as a
supplement to branch testing and branch or decision coverage. Branch testing
will address all simple decisions, multi-way decisions, and all loops.
Condition testing will then address the decisions which include Boolean
conditions.
In practice, a set of test cases which
achieves 100% coverage by one of these options will also achieve 100% coverage
by the other option. However, lower levels of coverage cannot be compared
between the two options.
