|
In recent years, deep convolutional neural networks (CNNs) have shown tremendous success in solving many biomedical tasks. However, the development of deep convolutional networks requires access to large quantities of high-quality annotated images for training and evaluation. As image annotation is a tedious task for biomedical experts, recruiting non-expert crowd workers is an economical and efficient way to provide a rich dataset of annotated images. We present an approach to improve the accuracy of segmenting nuclei in Hematoxylin and Eosin (H&E) slides by hiring crowd workers. We first present a crowdsourcing framework that enables fast and efficient acquisition of nuclei-segmented masks from the crowd by providing manual and semi-automatic annotation methods. Then, we present CrowdDeep, a novel technique to improve the accuracy of deep learning models trained on expert annotation by efficiently hiring crowd-annotated data. CrowdDeep consists of two sub-networks: Crowd-Subnet, and Expert-Subnet. The Crowd-Subnet is trained on the crowd-annotated images to extract crowd-related features from the crowd-annotated masks, while the Expert-Subnet is trained on the expert-driven annotations to extract expert-related features from the expert-annotated masks. Then, it calculates the final segmentation mask from the generated segmentation masks by two sub-networks. The results show that CrowdDeep outperforms a CNN model trained on solely expert-derived annotations in terms of F1-Score, IOU, and Pixel Accuracy. This approach is multi-organ and generalizes across different organs, staining, and disease states and is easily expandable by crowdsourcing images with an assortment of nuclei shapes and sizes from any desirable body tissue.
|
