Image Processing

On the evaluation of segmentation editing tools

[+] Author Affiliations
Frank Heckel

Fraunhofer MEVIS, Universitaetsallee 29, 28357 Bremen, Germany

University of Leipzig, Innovation Center Computer Assisted Surgery, Semmelweisstraße 14, 04103 Leipzig, Germany

Jan H. Moltz

Fraunhofer MEVIS, Universitaetsallee 29, 28357 Bremen, Germany

Hans Meine

Fraunhofer MEVIS, Universitaetsallee 29, 28357 Bremen, Germany

Benjamin Geisler, Horst K. Hahn

Fraunhofer MEVIS, Universitaetsallee 29, 28357 Bremen, Germany

Andreas Kießling

Philipps-University Marburg, Department of Diagnostic Radiology, Baldingerstrasse, 35043 Marburg, Germany

Melvin D’Anastasi

University Hospital of Munich, Department of Clinical Radiology, Marchioninistrasse 15, 81377 Munich, Germany

Daniel Pinto dos Santos, Ashok Joseph Theruvath

University Hospital Mainz, Department of Diagnostic and Interventional Radiology, Langenbeckstrasse 1, 55131 Mainz, Germany

J. Med. Imag. 1(3), 034005 (Nov 14, 2014). doi:10.1117/1.JMI.1.3.034005
History: Received March 31, 2014; Revised September 10, 2014; Accepted October 14, 2014
Text Size: A A A

Abstract.  Efficient segmentation editing tools are important components in the segmentation process, as no automatic methods exist that always generate sufficient results. Evaluating segmentation editing algorithms is challenging, because their quality depends on the user’s subjective impression. So far, no established methods for an objective, comprehensive evaluation of such tools exist and, particularly, intermediate segmentation results are not taken into account. We discuss the evaluation of editing algorithms in the context of tumor segmentation in computed tomography. We propose a rating scheme to qualitatively measure the accuracy and efficiency of editing tools in user studies. In order to objectively summarize the overall quality, we propose two scores based on the subjective rating and the quantified segmentation quality over time. Finally, a simulation-based evaluation approach is discussed, which allows a more reproducible evaluation without the need for human input. This automated evaluation complements user studies, allowing a more convincing evaluation, particularly during development, where frequent user studies are not possible. The proposed methods have been used to evaluate two dedicated editing algorithms on 131 representative tumor segmentations. We show how the comparison of editing algorithms benefits from the proposed methods. Our results also show the correlation of the suggested quality score with the qualitative ratings.

© 2014 Society of Photo-Optical Instrumentation Engineers

Citation

Frank Heckel ; Jan H. Moltz ; Hans Meine ; Benjamin Geisler ; Andreas Kießling, et al.
"On the evaluation of segmentation editing tools", J. Med. Imag. 1(3), 034005 (Nov 14, 2014). ; http://dx.doi.org/10.1117/1.JMI.1.3.034005


Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.