The Test Management Guide Software Test Guide RSS Feed
*
Accessibility

Background

BS7925-2 Annex B - Guidelines for Testing and Test Measurement

Syntax Testing



Annex B   
(informative)

Guidelines for Testing Techniques and Test Measurement

B.5    Syntax Testing

Introduction

This black box technique is based upon an analysis of the specification of the component to model its behaviour by means of a description of the input via its syntax.  We illustrate the technique by means of a worked example.  The technique is only effective to the extent that the syntax as defined corresponds to the required syntax.

Example

Consider a component that simply checks whether an input float_in conforms to the syntax of a floating point number, float (defined below).  The component outputs check_res, which takes the form 'valid' or 'invalid' dependent on the result of its check.

Here is a representation of the syntax for the floating point number, float in Backus Naur Form (BNF) :

float   =  int "e" int.

int      =  ["+"|"-"] nat.

nat     =  {dig}.

dig     =  "0"|"1"|"2"|"3"|"4"|"5"|"6"|"7"|"8"|"9".

 

Terminals are shown in quotation marks; these are the most elementary parts of the syntax - the actual characters that make up the input to the component.  | separates alternatives. [] surrounds an optional item, that is, one for which nothing is an alternative.  {} surrounds an item which may be iterated one or more times.

B.5.1 Test Cases with Valid Syntax

The first step is to derive the options from the syntax.  Each option is followed by a label, in the form [opt_1], [opt_2], etc., to enable it to be identified later.

float has no options.

int has three options: nat [opt_1], "+" nat [opt_2] and "-" nat [opt_3].

nat has two options: a single digit number [opt_4] and a multiple digit number [opt_5].

dig has ten options: one for each digit [opt_6 to opt_15].

 

There are thus fifteen options to be covered.  The next step is to construct test cases to cover the options.

The following test cases cover them all:

test case

float_in

option(s) executed

check_res

1

3e2

opt_1

'valid'

2

+2e+5

opt_2

'valid'

3

-6e-7

opt_3

'valid'

4

6e-2

opt_4

'valid'

5

1234567890e3

opt_5

'valid'

6

0e0

opt_6

'valid'

7

1e1

opt_7

'valid'

8

2e2

opt_8

'valid'

9

3e3

opt_9

'valid'

10

4e4

opt_10

'valid'

11

5e5

opt_11

'valid'

12

6e6

opt_12

'valid'

13

7e7

opt_13

'valid'

14

8e8

opt_14

'valid'

15

9e9

opt_15

'valid'

 

This is by no means a minimal test set to exercise the 15 options (it can be reduced to just three test cases, for example, 2, 3 and 5 above), and some test cases will exercise more options than the single one listed in the 'options executed' column.  Each option has been treated separately here to aid understanding of their derivation.  This approach may also contribute to the ease with which the causes of faults are located.

B.5.2 Test Cases with Invalid Syntax

The first step is to construct a checklist of generic mutations.  A possible checklist is:

m1.     introduce an invalid value for an element;

m2.     substitute an element with another defined element;

m3.     miss out a defined element;

m4.     add an extra element.

 

These generic mutations are applied to the individual elements of the syntax to yield specific mutations.  The elements, represented in the form el_1, el_2, etc., of the syntax for float can be identified from the BNF representation as shown below:

float      =    int "e" int.                                                    el_1      =   el_2 el_3 el_4.

int         =    ["+"|"-"] nat.                                     el_5      =   el_6 el_7.

nat        =    {dig}.                                                           el_8      =   el_9.

dig        =    "0"|"1"|"2"|"3"|"4"|"5"|"6"|"7"|"8"|"9".                el_10    =   el_11.

 

["+"|"-"] has been treated as a single element because the mutation of optional items separately does not create test cases with invalid syntax (using these generic mutations).

The next step is to construct test cases to cover the mutations:

test case

float_in

mutation

element

check_res

1

xe0

m1

x for el_2

'invalid'

2

0x0

m1

x for el_3

'invalid'

3

0ex

m1

x for el_4

'invalid'

4

x0e0

m1

x for el_6

'invalid'

5

+xe0

m1

x for el_7

'invalid'

6

ee0

m2

el_3 for el_2

'invalid'

7

+e0

m2

el_6 for el_2

'invalid'

8

000

m2

el_2 for el_3

'invalid'

9

0+0

m2

el_6 for el_3

'invalid'

10

0ee

m2

el_3 for el_4

'invalid'

11

0e+

m2

el_6 for el_4

'invalid'

12

e0e0

m2

el_3 for el_6

'invalid'

13

+ee0

m2

el_3 for el_7

'invalid'

14

++e0

m2

el_6 for el_7

'invalid'

15

e0

m3

el_2

'invalid'

16

00

m3

el_3

'invalid'

17

0e

m3

el_4

'invalid'

18

y0e0

m4

y in el_1

'invalid'

19

0ye0

m4

y in el_1

'invalid'

20

0ey0

m4

y in el_1

'invalid'

21

0e0y

m4

y in el_1

'invalid'

22

y+0e0

m4

y in el_5

'invalid'

23

+y0e0

m4

y in el_5

'invalid'

24

+0yeo

m4

y in el_5

'invalid'

 

Some of the mutations are indistinguishable from correctly formed expansions and these have been discarded.  For example, the generic mutation m2 (el_2 for el_4) generates correct syntax as m2 is "substitute an element with another defined element" and el_2 and el_4 are the same (int).

Some of the remaining mutations are indistinguishable from each other and these are covered by a single test case. For example, applying the generic mutation m1 ("introduce an invalid value for an element") by replacing el_4, which should be an integer, with "+" creates the form "0e+".  This is the same input as generated for test case 11 above.

Many more test cases can be created by making different choices when using single mutations, or combining mutations.

B.5.3 Syntax Test Coverage

No test coverage measures are defined for syntax testing.  Any measures of coverage would be based on the rules for generating valid syntax options and the checklist for generating test cases with invalid syntax.  Neither the rules nor the checklist are definitive and so any syntax test coverage measures based on a particular set will be specific to that set of rules and checklist.

© RuleWorks - All Rights Reserved - Policy - - Sitemap