ER, PR (estrogen, progesterone receptor), and HER2 (human epidermal growth factor receptor 2) status are assessed using immunohistochemistry and reported in standard clinical workflows as they provide valuable information to help treatment planning. The protein Ki67 has also been suggested as a prognostic biomarker but is not routinely evaluated clinically due to insufficient quality assurance. The routine pathological practice usually relies on small biopsies, such that the reduction in consumption is necessary to save materials for special assays. For this purpose, we developed and validated an automatic system for segmenting and identifying the (ER, PR, HER2, Ki67) positive cells from hæmatoxylin and eosin (H&E) stained tissue sections using multiplexed immunofluorescence (MxIF) images at cellular level as a reference standard. In this study, we used 100 tissue-microarray cores sampled from 56 cases of invasive breast cancer. For ER, we extracted cell nucleus images (HoverNet) from the H&E images and assigned each cell nucleus as ER positive vs. negative based on the corresponding MxIF signals (whole cell segmentation with DeepCSeg) upon H&E to MxIF image registration. We trained a Res-Net 18 and validated the model on a separate test-set for classifying the cells as positive vs. negative for ER, and performed the same experiment for the other three markers. We obtained area-under-the- receiver-operating-characteristic-curves (AUCs) of 0.82 (ER), 0.85 (PR), 0.75 (HER2), 0.82 (Ki67) respectively. Our study demonstrates the feasibility of using machine learning to identify molecular status at cellular level directly from the H&E slides.
Pathologists regularly use ink markings on histopathology slides to highlight specific areas of interest or orientation, making it an integral part of the workflow. Unfortunately, digitization of these ink-annotated slides hinders any computer-aided analyses, particularly deep learning algorithms, which require clean data free from artifacts. We propose a methodology that can identify and remove the ink markings for the purpose of computational analyses. We propose a two-stage network with a binary classifier for ink filtering and Pix2Pix for ink removal. We trained our network by artificially generating pseudo ink markings using only clean slides, requiring no manual annotation or curation of data. Furthermore, we demonstrate our algorithm’s efficacy over an independent dataset of H&E stained breast carcinoma slides scanned before and after the removal of pen markings. Our quantitative analysis shows promising results, achieving 98.7% accuracy for the binary classifier. For Pix2Pix, we observed a 65.6% increase in structure similarity index, a 21.3% increase in peak signal-to-noise ratio, and a 30% increase in visual information fidelity. As only clean slides are required for training, the pipeline can be adapted to multiple colors of ink markings or new domains, making it easy to deploy over different sets of histopathology slides. Code and trained models are available at: https://github.com/Vishwesh4/Ink-WSI.
Cytometry plays essential roles in immunology and oncology. Recent advancements in cellular imaging allow more detailed characterization of cells by labeling each cell with multiple protein markers. The increase of dimensionality makes manual analysis challenging. Clustering algorithms provide a means for phenotyping high-dimensional cell populations in an unsupervised manner for downstream analysis. The choice and usability of the methods are critical in practice. Literature provided comprehensive studies on those topics using publicly available flow cytometry data, which validated cell phenotypes by those methods against manual gated cell populations. In order to extend the knowledge for identification of cell phenotypes including unknown cell populations in our dataset, we conducted an exploratory study using clinical relevant tissue types as reference standard. Using our in-house database of multiplexed immunofluorescence images of breast cancer tissue microarrays (TMAs), we experimented with two commonly used algorithms (PhenoGraph and FlowSOM). Our pipeline includes: 1) cell phenotyping using Phenograph/FlowSOM; 2) clustering TMA cores into four groups using the percentage of each cell phenotypes with the algorithms (PhenoGraph/Spectral/K-means); 3) comparing the tissue groups to clinically relevant subtypes that were manually assigned based on the immunohistochemistry scores of serial sections. We experimented with different hyperparameter settings and input markers. Cell phenotypes using Phenograph with 10 markers and tissue clustering using Spectral yielded the highest mean F-measure (average over four tissue subtypes) of 0.71. In general, our results showed that cell phenotypes by Phenograph yielded better performance with larger variations than FlowSOM, which gives very consistent results.
Purpose: Automatic cancer detection on radical prostatectomy (RP) sections facilitates graphical and quantitative surgical pathology reporting, which can potentially benefit postsurgery follow-up care and treatment planning. It can also support imaging validation studies using a histologic reference standard and pathology research studies. This problem is challenging due to the large sizes of digital histopathology whole-mount whole-slide images (WSIs) of RP sections and staining variability across different WSIs.
Approach: We proposed a calibration-free adaptive thresholding algorithm, which compensates for staining variability and yields consistent tissue component maps (TCMs) of the nuclei, lumina, and other tissues. We used and compared three machine learning methods for classifying each cancer versus noncancer region of interest (ROI) throughout each WSI: (1) conventional machine learning methods and 14 texture features extracted from TCMs, (2) transfer learning with pretrained AlexNet fine-tuned by TCM ROIs, and (3) transfer learning with pretrained AlexNet fine-tuned with raw image ROIs.
Results: The three methods yielded areas under the receiver operating characteristic curve of 0.96, 0.98, and 0.98, respectively, in leave-one-patient-out cross validation using 1.3 million ROIs from 286 mid-gland whole-mount WSIs from 68 patients.
Conclusion: Transfer learning with the use of TCMs demonstrated state-of-the-art overall performance and is more stable with respect to sample size across different tissue types. For the tissue types involving Gleason 5 (most aggressive) cancer, it achieved the best performance compared to the other tested methods. This tool can be translated to clinical workflow to assist graphical and quantitative pathology reporting for surgical specimens upon further multicenter validation.
Automatic cancer sub-grading of radical prostatectomy (RP) specimens can support clinical studies seeking the prognostic indications of the sub-grades, and potentially benefits patient risk management and treatment planning. We developed and validated an automatic system which classifies each of nine subgrades (i.e. 4 sub-grades of Gleason grade 3, 3 sub-grades of Gleason grade 4, benign intervening, and other cancerous tissue) on digital histopathology whole-slide images (WSIs). The system was cross-validated against expert-drawn contours on a 25-patient data set comprising 92 mid-gland WSIs of RP specimens. The system used a transfer learning technique by fine-tuning AlexNet to classify each cancerous region of interest (ROI) according to sub-grade. We used leave-one-WSI-out cross-validation to measure classifier performance. The system yielded an area under the receiver-operating characteristic curve (AUC) higher than 0.8 for sub-grades of small fused Gleason 4 (G4), intermediate G3, and other cancerous tissue (AUC of 0.976); and AUCs higher than 0.7 for sub-grades of sparse G3, large cribriform G4, and desmoplastic G3.
Automatic cancer grading and high-grade cancer detection for radical prostatectomy (RP) specimens can benefit pathological assessment for prognosis and post-surgery treatment decision making. We developed and validated an automatic system which grades cancerous tissue as high-grade (Gleason grade 4 and higher) vs. low-grade (Gleason grade 3) on digital histopathology whole-slide images (WSIs). We combined this grading system with our previouslyreported cancer detection system to build a high-grade cancer detection system which automatically finds high-grade cancerous foci on WSIs. The system was tuned on a 3-patient data set and cross-validated against expert-drawn contours on a separate 68-patient data set comprising 286 mid-gland whole-slide images of RP specimens. The system uses machine learning techniques to classify each region of interest (ROI) on the slide as cancer or non-cancer and each cancerous ROI as high-grade or low-grade cancer. We used leave-one-patient-out cross-validation to measure the performance of cancer grading for classified ROIs with three different classifiers and the performance of the high-grade cancer detection system on a per tumor focus basis. The best performing (Fisher) classifier yielded an area under the receiver-operating characteristic curve of 0.87 for cancer grading. The system yielded error rates of 19.5% and 23.4% for pure high-grade (Gleason 4+4, 5+5) and high-grade (Gleason Score ≥ 7) cancer detection, respectively. The system demonstrated potential for practical computation speeds. Upon successful multi-centre validation, this system has the potential to assist the pathologist to find high-grade cancer more efficiently, which benefits the selection and guidance of adjuvant therapy and prognosis post RP.
Automatic localization of cancer on whole-slide histology images from radical prostatectomy specimens would support quantitative, graphical pathology reporting and research studies validating in vivo imaging against gold-standard histopathology. There is an unmet need for such a system that is robust to staining variability, is sufficiently fast and parallelizable as to be integrated into the clinical pathology workflow, and is validated using whole-slide images. We developed and validated such a system, with tuning occurring on an 8-patient data set and cross-validation occurring on a separate 41-patient data set comprising 703,745 480μm × 480μm sub-images from 166 whole-slide images. Our system computes tissue component maps from pixel data using a technique that is robust to staining variability, showing the loci of nuclei, luminal areas, and areas containing other tissue including stroma. Our system then computes first- and second-order texture features from the tissue component maps and uses machine learning techniques to classify each sub-image on the slide as cancer or non-cancer. The system was validated against expert-drawn contours that were verified by a genitourinary pathologist. We used leave-one-patient-out, 5-fold, and 2-fold cross-validation to measure performance with three different classifiers. The best performing support vector machine classifier yielded an area under the receiver operating characteristic curve of 0.95 from leave-one-out cross-validation. The system demonstrated potential for practically useful computation speeds, with further optimization and parallelization of the implementation. Upon successful multi-centre validation, this system has the potential to enable quantitative surgical pathology reporting and accelerate imaging validation studies using histopathologic reference standards.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.