00046 €c2509350e21c3447d8db6068221f3a

45

A Rule-Based Approach to Multiple Statistical Test Analysis

histograms. FigurÄ™ 11 compares the system's and the baseline's cumulative freÄ…uencies. The graph shows that the early baseline advantage is smali compared to the magnitude of the later system edge.

The accompanying table summarizes the information contained in the cumulative chart. Cumulative totals and percents are shown for each five observation interval, starting from the process shift. The baseline jumps to an advantage during the five extra observations the system takes to reach a decision. The baseline accumulates 16 correct decisions before the system is allowed to counter. The system still lags slightly behind, 207 to 215 detections, after the 30th observation. By the end of the next bÅ‚ock, the system has dramatically reversed the leads and is now 102 detections-or 9.5% of the total detections--ahead. At the end of forty runs, or twenty past the process change, the system has correctly classified 121 morÄ™ observations than the baseline~an 11.3% gap. The system stays slightly ahead and still maintains an eleven correct detection lead after the sixtieth observation. The finaÅ‚ eleven observation difference represents the accuracy advantage the system carries over the baseline.

Ali detections classified as misses—those that did not receive a detection notification—were given a very conservative arbitrary value of 61. With this, the average run lengths were calculated for the two methods. For the system, the average defect number of a correct detection was 35.90, or 15.90 readings past the process shift. For the baseline, the average was 37.65, or 17.65 observations past the shift. Therefore it is easy to calculate the difference as 1.75 fewer observations for the system to reach a slightly better decision than the baseline. The actual detection values of the misses would likely be much larger than 61, making the system's comparison look much better.

Continuous Drift Case

In both of the performance measurements, the system outperformed the baseline method. As shown in FigurÄ™ 10, the system barely beat the baseline 95.46% to 95.30% in accuracy. The system resulted in slightly better detection decisions being reached than the baseline. MorÄ™ pronounced though, was that the system also provided for better performance in the second area of speed of detection.

The run lengths required to reach a correct decision again was used as the metric to measure detection speed. FigurÄ™ 12 displays the cumulative correct decision run lengths for the system and baseline. The continuous drift is not considered detectable until the 30th observation, therefore only readings received after that point are considered. As with the shift scenario, the system