skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Statistical Considerations in Designing Tests of Mine Detection Systems: II - Measures Related to the False Alarm Rate

Abstract

The rate at which a mine detection system falsely identifies man-made or natural clutter objects as mines is referred to as the system's false alarm rate (FAR). Generally expressed as a rate per unit area or time, the FAR is one of the primary metrics used to gauge system performance. In this report, an overview is given of statistical methods appropriate for the analysis of data relating to FAR. Techniques are presented for determining a suitable size for the clutter collection area, for summarizing the performance of a single sensor, and for comparing different sensors. For readers requiring more thorough coverage of the topics discussed, references to the statistical literature are provided. A companion report addresses statistical issues related to the estimation of mine detection probabilities.

Authors:
Publication Date:
Research Org.:
Sandia National Laboratories, Albuquerque, NM, and Livermore, CA
Sponsoring Org.:
USDOE
OSTI Identifier:
1032
Report Number(s):
SAND98-1769/2
ON: DE00001032
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEFENSE; Military Equipment; Chemical Explosives; Detection

Citation Formats

Simonson, K.M. Statistical Considerations in Designing Tests of Mine Detection Systems: II - Measures Related to the False Alarm Rate. United States: N. p., 1998. Web. doi:10.2172/1032.
Simonson, K.M. Statistical Considerations in Designing Tests of Mine Detection Systems: II - Measures Related to the False Alarm Rate. United States. doi:10.2172/1032.
Simonson, K.M. 1998. "Statistical Considerations in Designing Tests of Mine Detection Systems: II - Measures Related to the False Alarm Rate". United States. doi:10.2172/1032. https://www.osti.gov/servlets/purl/1032.
@article{osti_1032,
title = {Statistical Considerations in Designing Tests of Mine Detection Systems: II - Measures Related to the False Alarm Rate},
author = {Simonson, K.M.},
abstractNote = {The rate at which a mine detection system falsely identifies man-made or natural clutter objects as mines is referred to as the system's false alarm rate (FAR). Generally expressed as a rate per unit area or time, the FAR is one of the primary metrics used to gauge system performance. In this report, an overview is given of statistical methods appropriate for the analysis of data relating to FAR. Techniques are presented for determining a suitable size for the clutter collection area, for summarizing the performance of a single sensor, and for comparing different sensors. For readers requiring more thorough coverage of the topics discussed, references to the statistical literature are provided. A companion report addresses statistical issues related to the estimation of mine detection probabilities.},
doi = {10.2172/1032},
journal = {},
number = ,
volume = ,
place = {United States},
year = 1998,
month = 8
}

Technical Report:

Save / Share:
  • One of the primary metrics used to gauge the performance of mine detection systems is PD, the probability of detecting an actual mine that is encountered by the sensor. In this report statistical issues and techniques that are relevant to the estimation of PD are discussed. Appropriate methods are presented for summarizing the performance of a single detection system, for comparing different systems, and for determining the sample size (number of target mines) required for a desired degree of precision. References are provided to pertinent sources within the extensive literature on statistical estimation and experimental design. A companion report addressesmore » the estimation of detection system false alarm rates.« less
  • The U.S. Department of Energy rule for Occupational Radiation Protection (10 CFR Part 835, December 1993) and the DOE Radiological Control Manual (the RCM) (DOE/EH-0256T, Rev. 1, April 1994) require the use of continuous air monitors (CAMs) in normally occupied areas where an individual is likely to be exposed to a concentration of airborne radioactivity exceeding the derived air concentration (DAC) or where there is a need to alert potentially exposed individuals to unexpected increases in airborne radioactivity levels. The DAC is the airborne concentration that equals the annual limit on intake divided by the volume of air breathed bymore » an average worker for a working year of 2000 h (assuming a breathing volume of 2400 m{sup 3}). It is equivalent to the airborne concentration to which a worker could be exposed for an entire working year (2000 h) without exceeding the allowable annual limit on intake. The rule and the RCM further require that real-time air monitors have an alarm capability and sufficient sensitivity to alert potentially exposed individuals that immediate action is necessary in order to minimize or terminate inhalation exposures. The RCM also recommends that real-time air monitors should be capable of measuring 1 DAC when averaged over 8 h (8 DAC-h) under laboratory conditions. In response to these recommendations, we are developing procedures for determining the basic sensitivity of alpha CAMs under laboratory conditions and for documenting practical alarm set points for routine use of CAMs under a range of radon and thoron concentrations.« less
  • In this project, the basic problem is to automatically separate test samples into one of two categories: clean or corrupt. This type of classification problem is known as a two-class classification problem or detection problem. In what follows, we refer to clean examples as negative examples and corrupt examples as positive examples. In a detection problem, a classifier decision on any one sample can be grouped into one of four decision categories: true negative, true positive, false negative and false positive. These four categories are illustrated by Table 1. True negatives and true positives are cases where the classifier hasmore » made the correct decision. False positives are cases where the classifier decides positive when the true nature of the sample was negative, and false negatives are cases where the classifier decides negative when the sample was actually positive. To evaluate the performance of a classifier, we run the classifier on all the samples of a data set and then count all the instances of true negatives, true positives, false negatives, and false positives. All of the performance metrics in this report are then formed from a combination of these four basic decision categories.« less
  • Median filtering reduces speckle in synthetic aperture radar (SAR) imagery while preserving edges, at the expense of coarsening the resolution, by replacing the center pixel of a sliding window by the median value. For shadow detection, this approach helps distinguish shadows from clutter more easily, while preserving shadow shape delineations. However, the nonlinear operation alters the shadow and clutter distributions and statistics, which must be taken into consideration when computing probability of detection and false alarm metrics. Depending on system parameters, median filtering can improve probability of detection and false alarm by orders of magnitude. Herein, we examine shadow probabilitymore » of detection and false alarm in a homogeneous, ideal clutter background after median filter post-processing. Some comments on multi-look processing effects with and without median filtering are also made.« less
  • Recovery of spores from environmental surfaces is known to vary due to sampling methodology, techniques, spore size and characteristics, surface materials, and environmental conditions. A series of tests were performed to evaluate a new, validated sponge-wipe method. Specific factors evaluated were the effects of contaminant concentrations and surface materials on recovery efficiency (RE), false negative rate (FNR), limit of detection (LOD) - and the uncertainties of these quantities. Ceramic tile and stainless steel had the highest mean RE values (48.9 and 48.1%, respectively). Faux leather, vinyl tile, and painted wood had mean RE values of 30.3, 25.6, and 25.5, respectively,more » while plastic had the lowest mean RE (9.8%). Results show a roughly linear dependence of surface roughness on RE, where the smoothest surfaces have the highest mean RE values. REs were not influenced by the low spore concentrations tested (3 x 10{sup -3} to 1.86 CFU/cm{sup 2}). The FNR data were consistent with RE data, showing a trend of smoother surfaces resulting in higher REs and lower FNRs. Stainless steel generally had the lowest mean FNR (0.123) and plastic had the highest mean FNR (0.479). The LOD{sub 90} varied with surface material, from 0.015 CFU/cm{sup 2} on stainless steel up to 0.039 on plastic. Selecting sampling locations on the basis of surface roughness and using roughness to interpret spore recovery data can improve sampling. Further, FNR values, calculated as a function of concentration and surface material, can be used pre-sampling to calculate the numbers of samples for statistical sampling plans with desired performance, and post-sampling to calculate the confidence in characterization and clearance decisions.« less