Dynamic testing of knowledge bases using the heuristic testing approach

摘要

How to develop knowledge-based and expert systems today is becoming more and more well understood; how to test these systems still poses some challenges. There has been considerable progress in developing techniques for static testing of these systems, checking for problems via formal examination methods; but there has been almost no work on dynamic testing, testing the systems under operating conditions. A novel approach for the dynamic testing of expert system rule bases is presented. This approach, Heuristic Testing, is based on the idea of first testing systems for disastrous safety and integrity problems before testing for primary functions and other classes of problems, and a prioritized series of 10 classes of faults are identified. The Heuristic Testing approach is intended to assure software reliability rather than simply find defects; the reliability is based on the 10 fault clones called compotent reliability. General procedures for conceptualizing and generating test cases were developed for all fault classes, including a Generic Testing Method for generating key test-case values. One of the classes, error-metric, illustrates how complexity-metrics, now used for predicting conventional software problems, could be developed for expert system rule bases. Two key themes are automation (automatically generating test cases) and fix-as-you-go testing (fixing a problem before continuing to test). The overall approach may be generalizable to static rule base testing, to testing of other expert system components, to testing of other nonconventional systems such as neural network and object-oriented systems, and even to conventional software.