3582296416

3 Method

3.1    Metrics used in evaluation

lt łs ahvays a daunting task to discuss measunng of quality. In order to do so. we need to establish some definitions as well as some metrics. We have found inspiration for doing this in [Pfaller 2008].

3.1.1    What are we comparing

The object of measurement is in this case the test suites that have been generated from the two different approaches.

3.1.2    Relative quality

As this is a comparison of two approaches to test case generation against a very smali specification and therefore also a very smali unit of production codę it should be noted that the quality we measure here is by naturę onfy relat)ve. Against other kind of production codę or other specifications results may be different.

3.1.3    Defects detected

The ability to detect defects in the production codę is of course the most important property of a test suitę.

3.1.4    Time spent on test case generation

This metric is concerned with the time that is spent with the test case creation. In the case of the traditional approach this includes the equivalence class partitiomng. boundary value analysts and the writing of the concrete test cases.

In case of the automatic test case generation tool we decided not to measure time to learn usmg the tool as mastering the EQ+BV analysis also has a leaming curve.

The time metric is not directly related to the quality of the test cases but it is needed to evaluate the two methods against each other.