Test for the statistical significance of differences between ROC curves. [For analysis of nuclear medical diagnostic data]

Conference · Mon Jan 01 00:00:00 EST 1979

OSTI ID:5793646

Metz, C. E.; Kronman, H. B.

A test for the statistical significance of observed differences between two measured Receiver Operating Characteristic (ROC) curves has been designed and evaluated. The set of observer response data for each ROC curve is assumed to be independent and to arise from a ROC curve having a form which, in the absence of statistical fluctuations in the response data, graphs as a straight line on double normal-deviate axes. To test the significance of an apparent difference between two measured ROC curves, maximum likelihood estimates of the two parameters of each curve and the associated parameter variances and covariance are calculated from the corresponding set of observer response data. An approximate Chi-square statistic with two degrees of freedom is then constructed from the differences between the parameters estimated for each ROC curve and from the variances and covariances of these estimates. This statistic is known to be truly Chi-square distributed only in the limit of large numbers of trials in the observer performance experiments. Performance of the statistic for data arising from a limited number of experimental trials was evaluated. Independent sets of rating scale data arising from the same underlying ROC curve were paired, and the fraction of differences found (falsely) significant was compared to the significance level, ..cap alpha.., used with the test. Although test performance was found to be somewhat dependent on both the number of trials in the data and the position of the underlying ROC curve in the ROC space, the results for various significance levels showed the test to be reliable under practical experimental conditions.

• Research Organization: Chicago Univ., IL (USA). Dept. of Radiology; Franklin McLean Memorial Research Inst., Chicago, IL (USA)
• Sponsoring Organization: USDOE
• DOE Contract Number: EY-76-C-02-0069
• OSTI ID: 5793646
• Report Number(s): CONF-790769-1; TRN: 79-021617
• Resource Relation: Conference: International conference on information processing in medical imaging, Paris, France, 2 Jul 1979
• Country of Publication: United States
• Language: English

Related Subjects

62 RADIOLOGY AND NUCLEAR MEDICINE
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE
SCINTISCANNING
DATA ANALYSIS
EXPERIMENTAL DATA
GRAPHS
ISOLATED VALUES
NUCLEAR MEDICINE
NUMERICAL SOLUTION
STATISTICS
COUNTING TECHNIQUES
DATA
DATA FORMS
DIAGNOSTIC TECHNIQUES
INFORMATION
MATHEMATICS
MEDICINE
NUMERICAL DATA
RADIOISOTOPE SCANNING
550601* - Medicine- Unsealed Radionuclides in Diagnostics
990200 - Mathematics & Computers