Comparison of free-response and ROC analyses in modality comparison studies involving lesion localization

Dev Prasad Chakraborty

doi:10.1117/12.462680

12 April 2002 Comparison of free-response and ROC analyses in modality comparison studies involving lesion localization

Dev Prasad Chakraborty

Proceedings Volume 4686, Medical Imaging 2002: Image Perception, Observer Performance, and Technology Assessment; (2002) https://doi.org/10.1117/12.462680
Event: Medical Imaging 2002, 2002, San Diego, California, United States

Abstract

It is possible that neglect of location information in lesion detection studies analyzed by the Receiver Operating Characteristic (ROC) method can compromise power. The Alternative Free-response ROC (AFROC) analysis considers the location information but its usage has been discouraged, since it neglects intra-image correlations. This study compared the statistical power of ROC and AFROC methodologies using simulations. A model including intra-image correlations was developed to describe the decision variable sampling and was used to simulate data for ROC and AFROC analysis. Five readers and 200 cases, half of which contained one signal, were simulated for each trial. Two hundred trials were run, equally split between the Null Hypothesis (NH) and the Alternative Hypothesis (AH). The ratings were analyzed by the Dorfman-Berbaum-Metz (DBM) method and the separation of the NH/AH distributions was calculated. It was found that the AFROC method yielded higher power than ROC. The separation of the NH and AH distributions were larger by a factor of 1.6, irrespective of the presence or absence of intra-image correlations. The effect of the incorrect localizations occurring in ROC analysis of localization data is believed to be the major reason for the enhanced power of the AFROC method, and greater use of AFROC methodology is warranted.

Citation Download Citation

Dev Prasad Chakraborty "Comparison of free-response and ROC analyses in modality comparison studies involving lesion localization", Proc. SPIE 4686, Medical Imaging 2002: Image Perception, Observer Performance, and Technology Assessment, (12 April 2002); https://doi.org/10.1117/12.462680

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available