| | |
Summary: General Test Result Checking
with Log File Analysis
James H. Andrews, Member, IEEE, and Yingjun Zhang
Abstract---We describe and apply a lightweight formal method for checking test results. The method assumes that the software under
test writes a text log file; this log file is then analyzed by a program to see if it reveals failures. We suggest a statemachinebased
formalism for specifying the log file analyzer programs and describe a language and implementation based on that formalism. We
report on empirical studies of the application of log file analysis to random testing of units. We describe the results of experiments done
to compare the performance and effectiveness of random unit testing with coverage checking and log file analysis to other unit testing
procedures. The experiments suggest that writing a formal log file analyzer and using random testing is competitive with other formal
and informal methods for unit testing.
Index Terms---Testing, specification, safety verification, lightweight formal methods, test oracles, unit testing, log file analysis
#
1 INTRODUCTION
T
ESTING software involves three main activities: choosing
the test cases, running them on the software under test
(SUT), and evaluating the results of the tests to see whether
any failures have occurred. This last activity, evaluating test
results, is generally done either by human inspection or by
checking whether the results are the same as others
|
