Classification Tree Method

Quick Summary

• The Classification Tree Method (CTM) is a structured black-box test design technique used to discover and document the SUT test parameters and their values, using a graphical representation (a tree).

• Developers can use CTM in white-box testing: after the Classification Tree is drawn, it can be used to write better unit or other automated tests

• CTM can be combined with EP & BVA

• CTM output can be used in combinatorial testing, for example, to create Pairwise tests.

A simple example

Suppose the SUT is a Password Validator, and the only requirement is that the password must be 10 characters or longer. Then the Classification Tree (CT) could look like this:

A trivial tree for trivial requirements (so far). The following can be observed or deduced:

• The top-level node represents the SUT chosen

• The tree may have many levels.

• Leaf nodes may either represent concrete values ("abcdefgh") or entire partitions ("< 10")

  • Thus, CTM can be used together with the EP & BVA technique

The Tree is then used to create test cases.

Input Length	Expected result
7	Rejected
15	Accepted

The CTM itself does not provide a way to pick valuable test data.

Additional techniques and knowledge of special values are required.

Here is a better test case table (far from exhaustive)

Input Length	Expected result	Comment
0	Rejected	Special value
9	Rejected	Border value
10	Accepted	Border value
11	Accepted	Border value
Extremely long	(check business or technical constraints)	Challenges constraints of backend storage systems
Special chars (blank spaces, emojis, non-Latin chars)	(check business or technical constraints)	Challenges string parsing correctness

A more advanced example

Requirements: a password must be at least 10 characters long, at least one uppercase character, at least one number, and one of the following special characters: !@#$%^&.

The tree might now look like this:

This is not the only way to create a CT for the stated requirements. While some CT syntax rules exist, there is no single correct way to map requirements to a CT - it is not a mechanical task and requires knowledge and ingenuity.

CTM Tips

1. Tightly scope your SUT. If you have a large piece of software, break it up into small pieces and create a CTM for each, or where applicable. Broadly scoped SUTs lead to huge, unwieldy CTs.

   • Example: Instead of creating a CT for "Text Editor", create one CT for "Search Text and Replace", another for "Table Operations", etc.

2. Avoid overlapping concepts and classifications (tree nodes). They result in redundancy when test cases are created.

   • Example: User Type = Admin / Guest and Permissions = Full / Limited

3. Keep the tree balanced

   1. One classification (node) has 10 classes (subnodes), others have 2-3.

   2. This might mean overthinking one aspect and underestimating another.

4. Separate input, state, and environment. The most common approach is to create separate trees or use different techniques.

   1. You may create an input (data) CT, and then handpick a state (logged in, locked, expired) and environment (desktop, mobile, web) for testing.

Generating test cases from the CT

Given a CT, one can generate test cases:

1. Manually (create and fill a table with cases)

2. Using a tool such as Testona

3. Prepare a model to feed into a Pairwise tool to generate a Covering Array with value pairs, triplets, or other, where each row is a test case.

Testona example

Testona is a proprietary tool that allows creating a CT and generating Pairwise (and other) test cases from it. In the screenshot, Test Case 1 should read as "Use a Super User, with Chrome, and perform a Delete operation."

Source: testona features

Testona serves as a great visual example of how a CT can be used to create test cases. But the same can be achieved with UI-less pairwise tools.

PICT example

The core strength of a CTM is that it helps visualize the input or coverage space of the SUT.

However, you can draw a CT by hand or using any free Mind Mapping tool, and then type out the input data into a text model to be fed into PICT or a similar tool.

In other words, though CTM was originally developed as a black-box technique, it can be used as a white-box technique by developers to create better unit or higher-level tests.

The following input_model.txt can be created:

LongerThan10Chars: yes, no
HasUpperCase: yes, no
HasNumber: yes, no
HasSpecialChar: yes, no

And this is the Pairwise output when given to the PICT tool:

LongerThan10Chars HasUpperCase    HasNumber       HasSpecialChar
no                  yes             yes                 yes
no                  no              no                  no
yes                 no              yes                 no
yes                 yes             no                  no
yes                 no              no                  yes

Finally, given a custom function such as

String generatePassword(
        boolean isLongerThan10,
        boolean hasUpperCase,
        boolean hasNumber,
        boolean hasSpecialChar)

Invoking it with all the PICT output data, we get the final test data:

Test 1: 8u@X$J             -> Rejected
Test 2: ylpvcz             -> Rejected
Test 3: 22nxpbscld8gj7x    -> Rejected
Test 4: ZMwFOaoeomNyq      -> Rejected  
Test 5: hquxj$h*d%dzu      -> Rejected 

In addition to missing edge case scenarios previously described, notice that there is not a single test resulting in the acceptance of the password.

Refining the test design

The final test data above won't necessarily cover all special characters as stated in the requirements. This indicates a flaw in either the CT constructed or the in input_model.txt , highlighting that CTM and Pairwise input require careful thought and design.

In this instance, we only have a relatively small set of special characters !@#$%^&, but dedicating a leaf node to each of these would already require a lot of manual work, creating an unnecessarily large tree.

The tree is for modeling rule presence, not data variety.

We should thus consider changing the input_model.txt and the password-generating method.

LongerThan10Chars: yes, no
HasUpperCase: yes, no
HasNumber: yes, no
Special Char: !,@,#,$,%,^,&        # changed line

For greater thoroughness still, you could change HasNumber: yes, no to Number: 0,1,2,3,4,5,6,7,8,9

And this is the new Pairwise output that can be used to produce actual test data:

LongerThan10Chars HasUpperCase  HasNumber  Special Char
yes                 yes     			yes     		!
no                  no      			no      		$
yes                 no      			no      		^
no                  yes     			no      		#
no                  no      			yes     		^
no                  yes     			yes     		@
yes                 yes     			yes     		&
no                  no      			no      		!
no                  yes     			no      		%
no                  no      			no      		&
yes                 no      			yes     		#
yes                 yes     			yes     		$
yes                 no      			no      		@
yes                 yes     			yes     		^
yes                 no      			yes     		%

The above test data set may be adjusted further. Among other things, we should ensure that it contains test cases proving that only ONE characteristic results in a rejection, i.e., only "too short" leads to a rejection, only "no upper case char" results in a rejection, etc.

See the Elementary Comparison technique.

Conclusion

CTM may serve as a great, visual starting point to map SUT characteristics and test-relevant dimensions, but it only serves as a starting point.

Additional knowledge and techniques, such as test data generation strategies and combinatorial selection heuristics (EP & BVA, Special Values, Elementary Comparison, Pairwise) are needed to ensure meaningful coverage and effective fault detection.

References

1. Testona YouTube Tutorial