Special Section on Pioneers in Medical Imaging: Honoring the Memory of Robert F. Wagner

Nonparametric estimation receiver operating characteristic analysis for performance evaluation on combined detection and estimation tasks

[+] Author Affiliations
Adam Wunderlich

U.S. Food and Drug Administration, Center for Devices and Radiological Health, Silver Spring, Maryland, United States

Bart Goossens

Ghent University, Department of Telecommunications and Information Processing, TELIN-IPI-iMinds, Ghent, Belgium

J. Med. Imag. 1(3), 031002 (Aug 26, 2014). doi:10.1117/1.JMI.1.3.031002
History: Received April 2, 2014; Revised July 2, 2014; Accepted July 7, 2014
Text Size: A A A

Abstract.  In an effort to generalize task-based assessment beyond traditional signal detection, there is a growing interest in performance evaluation for combined detection and estimation tasks, in which signal parameters, such as size, orientation, and contrast are unknown and must be estimated. One motivation for studying such tasks is their rich complexity, which offers potential advantages for imaging system optimization. To evaluate observer performance on combined detection and estimation tasks, Clarkson introduced the estimation receiver operating characteristic (EROC) curve and the area under the EROC curve as a summary figure of merit. This work provides practical tools for EROC analysis of experimental data. In particular, we propose nonparametric estimators for the EROC curve, the area under the EROC curve, and for the variance/covariance matrix of a vector of correlated EROC area estimates. In addition, we show that reliable confidence intervals can be obtained for EROC area, and we validate these intervals with Monte Carlo simulation. Application of our methodology is illustrated with an example comparing magnetic resonance imaging k-space sampling trajectories. MATLAB® software implementing the EROC analysis estimators described in this work is publicly available at http://code.google.com/p/iqmodelo/.

Figures in this Article
© 2014 Society of Photo-Optical Instrumentation Engineers

Citation

Adam Wunderlich and Bart Goossens
"Nonparametric estimation receiver operating characteristic analysis for performance evaluation on combined detection and estimation tasks", J. Med. Imag. 1(3), 031002 (Aug 26, 2014). ; http://dx.doi.org/10.1117/1.JMI.1.3.031002


Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.