Guidelines for Testing Techniques and Test Measurement
B.5 Syntax Testing
Introduction
This black box technique is based upon
an analysis of the specification of the component to model its behaviour by
means of a description of the input via its syntax. We illustrate the technique by means of a worked example. The technique is only effective to the
extent that the syntax as defined corresponds to the required syntax.
Example
Consider a component that simply checks
whether an input float_in conforms to
the syntax of a floating point number, float (defined below). The component outputs check_res, which takes the form 'valid' or 'invalid' dependent on
the result of its check.
Here
is a representation of the syntax for the floating point number, float in
Backus Naur Form (BNF) :
float = int "e" int.
int = ["+"|"-"] nat.
nat = {dig}.
dig = "0"|"1"|"2"|"3"|"4"|"5"|"6"|"7"|"8"|"9".
Terminals are shown in quotation marks; these are the most
elementary parts of the syntax - the actual characters that make up the input
to the component. | separates
alternatives. [] surrounds an optional item, that is, one for which nothing is
an alternative. {} surrounds an item
which may be iterated one or more times.
B.5.1 Test Cases with Valid Syntax
The
first step is to derive the options from the syntax. Each option is followed by a label, in the form [opt_1], [opt_2],
etc., to enable it to be identified later.
float has no options.
int has three options: nat [opt_1],
"+" nat [opt_2] and "-" nat [opt_3].
nat has two options: a single digit
number [opt_4] and a multiple digit number [opt_5].
dig has ten options: one for each digit
[opt_6 to opt_15].
There are thus fifteen options to be
covered. The next step is to construct
test cases to cover the options.
The
following test cases cover them all:
|
test case
|
float_in
|
option(s)
executed
|
check_res
|
|
1
|
3e2
|
opt_1
|
'valid'
|
|
2
|
+2e+5
|
opt_2
|
'valid'
|
|
3
|
-6e-7
|
opt_3
|
'valid'
|
|
4
|
6e-2
|
opt_4
|
'valid'
|
|
5
|
1234567890e3
|
opt_5
|
'valid'
|
|
6
|
0e0
|
opt_6
|
'valid'
|
|
7
|
1e1
|
opt_7
|
'valid'
|
|
8
|
2e2
|
opt_8
|
'valid'
|
|
9
|
3e3
|
opt_9
|
'valid'
|
|
10
|
4e4
|
opt_10
|
'valid'
|
|
11
|
5e5
|
opt_11
|
'valid'
|
|
12
|
6e6
|
opt_12
|
'valid'
|
|
13
|
7e7
|
opt_13
|
'valid'
|
|
14
|
8e8
|
opt_14
|
'valid'
|
|
15
|
9e9
|
opt_15
|
'valid'
|
This is by no means a minimal test set
to exercise the 15 options (it can be reduced to just three test cases, for
example, 2, 3 and 5 above), and some test cases will exercise more options than
the single one listed in the 'options executed' column. Each option has been treated separately here
to aid understanding of their derivation.
This approach may also contribute to the ease with which the causes of
faults are located.
B.5.2 Test Cases with Invalid Syntax
The
first step is to construct a checklist of generic mutations. A possible checklist is:
m1. introduce an invalid value for an element;
m2. substitute an element with another defined
element;
m3. miss out a defined element;
m4. add an extra element.
These
generic mutations are applied to the individual elements of the syntax to yield
specific mutations. The elements,
represented in the form el_1, el_2, etc., of the syntax for float can be
identified from the BNF representation as shown below:
float = int "e" int. el_1
=
el_2 el_3 el_4.
int = ["+"|"-"]
nat. el_5
=
el_6 el_7.
nat = {dig}. el_8 = el_9.
dig = "0"|"1"|"2"|"3"|"4"|"5"|"6"|"7"|"8"|"9". el_10 = el_11.
["+"|"-"] has been treated as a single
element because the mutation of optional items separately does not create test
cases with invalid syntax (using these generic mutations).
The
next step is to construct test cases to cover the mutations:
|
test case
|
float_in
|
mutation
|
element
|
check_res
|
|
1
|
xe0
|
m1
|
x for el_2
|
'invalid'
|
|
2
|
0x0
|
m1
|
x for el_3
|
'invalid'
|
|
3
|
0ex
|
m1
|
x for el_4
|
'invalid'
|
|
4
|
x0e0
|
m1
|
x for el_6
|
'invalid'
|
|
5
|
+xe0
|
m1
|
x for el_7
|
'invalid'
|
|
6
|
ee0
|
m2
|
el_3 for
el_2
|
'invalid'
|
|
7
|
+e0
|
m2
|
el_6 for
el_2
|
'invalid'
|
|
8
|
000
|
m2
|
el_2 for
el_3
|
'invalid'
|
|
9
|
0+0
|
m2
|
el_6 for
el_3
|
'invalid'
|
|
10
|
0ee
|
m2
|
el_3 for
el_4
|
'invalid'
|
|
11
|
0e+
|
m2
|
el_6 for
el_4
|
'invalid'
|
|
12
|
e0e0
|
m2
|
el_3 for
el_6
|
'invalid'
|
|
13
|
+ee0
|
m2
|
el_3 for
el_7
|
'invalid'
|
|
14
|
++e0
|
m2
|
el_6 for
el_7
|
'invalid'
|
|
15
|
e0
|
m3
|
el_2
|
'invalid'
|
|
16
|
00
|
m3
|
el_3
|
'invalid'
|
|
17
|
0e
|
m3
|
el_4
|
'invalid'
|
|
18
|
y0e0
|
m4
|
y in el_1
|
'invalid'
|
|
19
|
0ye0
|
m4
|
y in el_1
|
'invalid'
|
|
20
|
0ey0
|
m4
|
y in el_1
|
'invalid'
|
|
21
|
0e0y
|
m4
|
y in el_1
|
'invalid'
|
|
22
|
y+0e0
|
m4
|
y in el_5
|
'invalid'
|
|
23
|
+y0e0
|
m4
|
y in el_5
|
'invalid'
|
|
24
|
+0yeo
|
m4
|
y in el_5
|
'invalid'
|
Some of the mutations are
indistinguishable from correctly formed expansions and these have been
discarded. For example, the generic
mutation m2 (el_2 for el_4) generates correct syntax as m2 is "substitute an
element with another defined element" and el_2 and el_4 are the same (int).
Some of the remaining mutations are
indistinguishable from each other and these are covered by a single test case. For example, applying the generic mutation m1
("introduce an invalid value for an element") by replacing el_4, which should
be an integer, with "+" creates the form "0e+". This is the same input as generated for test case 11 above.
Many more test cases can be created by
making different choices when using single mutations, or combining mutations.
B.5.3 Syntax Test Coverage
No test coverage measures are defined
for syntax testing. Any measures of
coverage would be based on the rules for generating valid syntax options and
the checklist for generating test cases with invalid syntax. Neither the rules nor the checklist are
definitive and so any syntax test coverage measures based on a particular set
will be specific to that set of rules and checklist.
