Significant amount of effort has been invested in improving the quality of breast imaging modalities (for example, mammography) to increase the accuracy of breast cancer detection. Despite that, about 4-34% of cancers are still missed during mammographic examination of cancer of the breast. This indicates the need to explore a) The features of the lesions that are missed, and b) Whether the features of missed cancers contribute to why some cancers are not ‘looked at’ (search error) whereas others are ‘looked at’ but still not reported. In this visual search study, we perform feature analysis of all lesions that were missed by at least one participating radiologist. We focus on features extracted by means of Grey Level Co-occurrence Matrix properties, textural properties using Gabor filters, statistical information extraction using 2nd and higher-order (3rd and 4th) spectral analysis and also spatial-temporal attributes of radiologists’ visual search behaviour. We perform Analysis of Variance (ANOVA) on these features to explore the differences in features for cancers that were missed due to a) search, b) perception and c) decision making errors. Using these features, we trained Support Vector Machine, Gradient Boosting and stochastic gradient decent classifiers to determine the type of missed cancer (search, perception and decision making). We compared these feature-based models with a model trained using deep convolution neural network that learns features by itself. We determined whether deep learning or traditional machine learning performs best in this task.
Visual search, the process of detecting and identifying objects using eye movements (saccades) and foveal vision, has been studied for identification of root causes of errors in the interpretation of mammograms. The aim of this study is to model visual search behavior of radiologists and their interpretation of mammograms using deep machine learning approaches. Our model is based on a deep convolutional neural network, a biologically inspired multilayer perceptron that simulates the visual cortex and is reinforced with transfer learning techniques. Eye-tracking data were obtained from eight radiologists (of varying experience levels in reading mammograms) reviewing 120 two-view digital mammography cases (59 cancers), and it has been used to train the model, which was pretrained with the ImageNet dataset for transfer learning. Areas of the mammogram that received direct (foveally fixated), indirect (peripherally fixated), or no (never fixated) visual attention were extracted from radiologists’ visual search maps (obtained by a head mounted eye-tracking device). These areas along with the radiologists’ assessment (including confidence in the assessment) of the presence of suspected malignancy were used to model: (1) radiologists’ decision, (2) radiologists’ confidence in such decisions, and (3) the attentional level (i.e., foveal, peripheral, or none) in an area of the mammogram. Our results indicate high accuracy and low misclassification in modeling such behaviors.
Visual search, the process of detecting and identifying objects using the eye movements (saccades) and the foveal vision, has been studied for identification of root causes of errors in the interpretation of mammography. The aim of this study is to model visual search behaviour of radiologists and their interpretation of mammograms using deep machine learning approaches. Our model is based on a deep convolutional neural network, a biologically-inspired multilayer perceptron that simulates the visual cortex, and is reinforced with transfer learning techniques.
Eye tracking data obtained from 8 radiologists (of varying experience levels in reading mammograms) reviewing 120 two-view digital mammography cases (59 cancers) have been used to train the model, which was pre-trained with the ImageNet dataset for transfer learning. Areas of the mammogram that received direct (foveally fixated), indirect (peripherally fixated) or no (never fixated) visual attention were extracted from radiologists’ visual search maps (obtained by a head mounted eye tracking device). These areas, along with the radiologists’ assessment (including confidence of the assessment) of suspected malignancy were used to model: 1) Radiologists’ decision; 2) Radiologists’ confidence on such decision; and 3) The attentional level (i.e. foveal, peripheral or none) obtained by an area of the mammogram. Our results indicate high accuracy and low misclassification in modelling such behaviours.
KEYWORDS: Digital breast tomosynthesis, Cancer, Breast cancer, Breast, Statistical analysis, Medical imaging, Clinical trials, Receivers, Data modeling, Data conversion
The rapid evolution in medical imaging has led to an increased number of recurrent trials, primarily to ensure that the efficacy of new imaging techniques is known. The cost associated with time and resources in conducting such trials is usually high. The recruitment of participants, in a medium to large reader study, is often very challenging as the demanding number of cases discourages involvement with the trial. We aim to evaluate the efficacy of Digital Breast Tomosynthesis (DBT) in a recall assessment clinic in Australia in a prospective multi-reader-multi-case (MRMC) trial. Conducting such a study with the more commonly used fully crossed MRMC study design would require more cases and more cases read per reader, which was not viable in our setting. With an aim to perform a cost effective yet statistically efficient clinical trial, we evaluated alternative study designs, particularly the alternative split-plot MRMC study design and compared and contrasted it with more commonly used fully crossed MRMC study design. Our results suggest that ‘split-plot’, an alternative MRMC study design, could be very beneficial for medium to large clinical trials and the cost associated with conducting such trials can be greatly reduced without adversely effecting the variance of the study. We have also noted an inverse dependency between number of required readers and cases to achieve a target variance. This suggests that split-plot could also be very beneficial for studies that focus on cases that are hard to procure or readers that are hard to recruit. We believe that our results may be relevant to other researchers seeking to design a medium to large clinical trials.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.