This PDF file contains the front matter associated with SPIE Proceedings Volume 11320, including the Title Page, Copyright information, Table of Contents, Author and Conference Committee lists

Lung adenocarcinoma (LUAD), the most common type of lung cancer, has an average 5-year survival rate of 15%. In LUAD, interaction between tumor and immune cells has been shown to be highly associated with the likelihood of disease progression and metastases. We have previously demonstrated the association between spatial architecture and arrangement of tumor-infiltrating lymphocytes (TILs) with likelihood of recurrence in early stage NSCLC. Recently, gene set enrichment analysis-derived immune scores have been found to be prognostic of outcome. However, this requires transcriptomics techniques as a precursor, which involves mechanical disruption of cells and tissues. In this work (N = 170), we extracted graph-based histomorphometric features on segmented nuclei from digitized H and E biopsy images and then performed principal component analysis (PCA) to select the most representative tiles from each patient. We then identified TILs and quantitative histomorphometric attributes of different nuclei groups (all-nuclei, TILs, non-TILs) prognostic of overall patient survival (OS) and further investigated their associations with immune scores and biological pathways implicated immune response using gene-set enrichment analysis (GSEA). We found TIL-compactness (a set of TIL density features) derived risk scores were prognostic of OS (Hazard Ratio (HR) = 3.26, p = 0.012, C-index = 0.634). The median immune score (IS) in the cohort was used as a threshold to divide the cases into low and high IS expression groups. The TIL compactness measures prognostic of OS were also statistically significantly correlated with the IS and biological pathways related to immune response (Immune System Process, Immune Response, Adaptive Immune Response, and Humoral Immune Response Mediated by Circulating Immunoglobulin).

Allogenic hematopoietic stem cell transplant (HCT) is a curative therapy for acute myeloid leukemia (AML). Relapse after HCT is the most common cause of treatment failure and is associated with poor prognosis. Early identification of which patients are at elevated risk of relapse may justify use of aggressive post-HCT treatment options, potentially preventing relapse and treatment failure. In this study, our goal was to predict relapse after HCT in AML patients using quantitative features extracted from digitized Wright-Giemsa stained posttransplant aspirate smears. We collected 39 aspirate specimens from a cohort of 39 AML patients after HCT, of which 25 experienced relapse, while 14 did not. Our approach comprised the following main steps. First, a deep learning model was developed to segment myeloblasts, a cell type in bone marrow that accumulates and characterizes AML. A total of 161 texture and shape descriptors were then extracted from these segmented myeloblasts. The top eight predictive features were identified using a Wilcoxon rank sum test over 100 iterations of 3-fold cross validation. A model was subsequently built employing these features and yielded an average area under the receiver operating characteristic curve of 0.80±0.05 in cross validation. The top eight features include four Haralick texture features and four fractal dimension features. The texture features appear to characterize chromatin patterns in myeloblasts while the fractal features quantify morphological irregularity and complexity of myeloblasts, in alignment with findings previously reported for AML patients post-treatment.

Immunotherapy is a novel anti-cancer treatment that shows significant improvements in outcomes for lung cancer patients. However, this treatment has the potential for substantial side effects in a minority of patients, and many lung cancer patients do not benefit from it. Programmed-death ligand-1 expression in tumour cells is currently the main biomarker used to identify those who might benefit but it is not very accurate. Tumour mutational burden (TMB) is a promising alternative, with lung cancers having more than 10 mutations/megabase being more likely to respond to immunotherapy. However, the cost and time it takes to obtain TMB makes it difficult to implement in the clinic. In this study, we used the deep learning technique of transfer learning with Alexnet to obtain a model that can estimate whether a cancer is highly mutated or not based on digitized hematoxylin and eosin histology slides that are routinely obtained from surgical resection of squamous cell carcinoma. The system was developed using images from 20 patients obtained through The Cancer Genome Atlas, five of which were reserved for validation. On this validation set, the system had an area under the receiver operator characteristic curve of 0.80, error rate of 24%, false negative rate of 26%, and false positive rate of 22%. This motivates additional work in this direction to build a system that can be used in the future to inform physicians as to which patients with squamous lung carcinoma would benefit from immunotherapy.

Identification of bladder layers is a necessary prerequisite to bladder cancer diagnosis and prognosis. We present a method of multi-class image segmentation, which can recognize Urothelium, Lamina Propria, Muscularis Propria, and Muscularis Mucosa regions from images of H&E-stained slides of bladder biopsies. This method can also recognize the difference between these layers, and regions of red blood cells, cauterized tissue, and inflamed tissue at pixel level. The segmentation is done using the U-Net deep learning paradigm, which consists of a combination of convolutional neural layers and upscaling layers to encode features from an image. The most optimal model for this task was found by training four different weight initializers and three different U-Nets of varying size and dropout on 39 whole slide images of T1 bladder biopsies. The model was visually evaluated by an experienced pathologist on an independent set of 15 slides. The pathologist gave an average score of 8.93 out of 10 for the segmentation accuracy. It only took 23 mins for the pathologist to review 15 slides. Our preliminary findings suggest that predictions of our model can minimize the time needed by pathologists to review the slides. Moreover, the method has the potential to identify the bladder layers accurately and hence can assist the pathologist with the diagnosis of T1 bladder cancer.

Generative modeling using GANs has gained traction in machine learning literature, as training does not require labeled datasets. This is perfect for applications in biological datasets, where large labeled datasets are often difficult and expensive to acquire. However, generative models offer no easy way to encode real images into feature-sets, something that is desirable for network explainability and may yield potentially informative image features. For this reason, we test a VAE-GAN architecture for label-free modeling of glomerular structural features. We show that this network can generate realistic looking synthetic images, and be used to interpolate between images. To prove the biological relevance of the network encodings, we classify small-labeled sets of encoded glomeruli by biopsy Tervaert class and for the presence of sclerosis, obtaining a Cohen’s kappa values of 0.87 and 0.78 respectfully.

There is a recent and increasing trend in the incidence of pancreatic neuroendocrine tumors (PNETs). Ki-67 proliferative index is required for routine pathologic evaluation of PNETs. This index has been found to be a consistent prognostic factor to assess clinical/prognostic outcome of PNETs. Unfortunately, we still lack a standardized method to reliably obtain the Ki-67 proliferative index. As part of a large study to standardize this index, here we present an accurate, easy-to- use, reproducible method to identify tumor nuclei and hotspots within PNETs. We modified the U-Net image segmentation architecture to identify tumor positive and negative nuclei. We also introduced the concept of local depth for identification of hotspots. On an independent test set of 8 whole slide images, the modified U-Net achieved a sensitivity of 96.2% and specificity of 93.3%. The hotspot detection framework resulted in a dice coefficient of 0.81. The method has the potential to not only facilitate the detection of tumor nuclei, but can be adapted to reproduce hotspots by pathologists.

There is a need for computer-aided diagnosis (CAD) systems to relieve the workload on pathologists. This seems to be especially important for intraoperative diagnosis during surgery, for which diagnostic time is very limited. This paper presents preliminary results of intraoperative pixel-based CAD of colon cancer metastasis in a liver from phase-contrast images of unstained frozen sections. In particular, two deep learning networks: the U-net and the structured autoencoder for deep subspace clustering, were trained on eighteen phase-contrast images belonging to five patients and tested on eight images belonging to three patients. Spectrum angle mapper was also used in comparative performance analysis. The best result achieved by the U-net yielded balanced accuracy of 83.70%±8%, sensitivity of 94.50%±8%, specificity of 72.9%±8% and Dice coefficient of 45.20%±25.4%. However, factors such as absence of tissue fixation and ethanol-induced dehydration, melting of the specimen under the microscope and/or frozen crystals in the specimen cause variations in quality of phase-contrast images of unstained frozen sections. This, in return, affects reproducibility of diagnostic performance.

Development of computer-aided diagnosis (CAD) systems is motivated by reduction of the workload on the pathologist that is increasing steadily. Among approaches upon which CAD-based systems are built, deep learning (DL) methods seem to be well suited for image analysis in digital pathology. However, DL networks include a large number of parameters and that requires a large annotated training dataset. Unfortunately, probably the biggest problem in digital pathology using machine learning methods is a small number of annotated images. That is especially true in intraoperative tissue analysis which coincides with the topic of the present paper: intraoperative CAD-based diagnosis of metastasis of colon cancer in a liver from hematoxylin-eosin (H and E) stained frozen section. To cope with the insufficiency of training images we adopt a transfer learning approach using the Nested UNet architecture. For better diagnostic performance, the trained model predicted pixels multiple times for different striding levels using the sliding window strategy. Threshold optimization using balanced accuracy score showed the validity of such an approach as balanced accuracy has increased significantly. When compared to often used UNet with VGG16 backbone, Nested UNet model with DenseNet201 backbone performs better on our dataset for both balanced accuracy metric and F1 score.

Bone marrow biopsies play a central role in hematopathology for diagnosing a variety of diseases, staging lymphomas or performing follow-up progression. Tasks performed while examining biopsies include counting cells and estimating the ratio of various hematopoietic lineages. Inter- and intra-observer variability between hematopathologists in the outcome of these tasks has been shown to be significant, which could result in multiple pathologists diagnosing some patients differently. To that end, this paper presents a fully-convolutional neural network (FCNN) architecture to segment six classes in bone marrow trephine biopsies, which could assist hematopathologists in identifying and delineating cells, thus reducing inter- and intra-observer variability. Additionally, to show an application of the neural network to a clinically relevant task, the output of the network is used to train a classifier capable of distinguishing between normocellular and aplastic bone marrow. Results indicate the network is successfully capable of segmenting cells with an average detection rate of 83%. The classifier for distinguishing normocellular/aplastic bone marrow reaches an AUC of 0.990, showing that is capable of automatically identifying aplasia.

Squamous cell carcinoma (SCC) comprises over 90 percent of tumors in the head and neck. The diagnosis process involves performing surgical resection of tissue and creating histological slides from the removed tissue. Pathologists detect SCC in histology slides, and may fail to correctly identify tumor regions within the slides. In this study, a dataset of patches extracted from 200 digitized histological images from 84 head and neck SCC patients was used to train, validate and test the segmentation performance of a fully-convolutional U-Net architecture. The neural network achieved a pixel-level segmentation AUC of 0.89 on the testing group. The average segmentation time for whole slide images was 72 seconds. The training, validation, and testing process in this experiment produces a model that has the potential to help segment SCC images in histological images with improved speed and accuracy compared to the manual segmentation process performed by pathologists.

When diagnosing and reporting lung adenocarcinoma (LAC), pathologists currently include an assessment of histologic tumor growth patterns because the predominant growth pattern has been reported to impact prognosis. However, the subjective nature of manual slide evaluation contributes to suboptimal inter-pathologist variability in tumor growth pattern assessment. We applied a deep learning approach to identify and automatically delineate areas of four tumor growth patterns (solid, acinar, micropapillary, and cribriform) and non-tumor areas in whole slide images (WSI) from resected LAC specimens. We trained a DenseNet model using patches from 109 slides collected at two institutions. The model was tested using 56 WSIs including 20 that were collected at a third institution. Using the same slide set, the concordance between the DenseNet model and an experienced pathologist (blinded to the DenseNet results) in determining the predominant tumor growth pattern was substantial (kappa score = 0.603). Using a subset of 36 test slides that were manually annotated for tumor growth patterns, we also measured the F1-score for each growth pattern: 0.95 (solid), 0.78 (acinar), 0.76 (micropapillary), 0.28 (cribriform) and 0.97 (non-tumor). Our results suggest that DenseNet assessment of WSIs with solid, acinar, and micropapillary predominant tumor growth is more robust than for the WSIs with predominant cribriform growth which are less frequently encountered.

Individual factors that result in susceptibility and morbidity due to Mycobacterium tuberculosis (M.tb) infection are not clearly defined in humans or in animal models of disease. This leads to challenges for example, differentiating life-long control of latent Mycobacterium tuberculosis infection from active tuberculosis disease and accurately predicting who will develop active tuberculosis. As a part of a larger study to identify biomarkers to differentiate these states and predict long-term clinical outcomes, here we present a deep learning approach to segment Mycobacterium tuberculosis bacilli clusters from acid-fast stained lung sections from experimentally infected mice. In a 4-fold cross-validation of 178 slides, our method demonstrates ability to segment bacilli with median dice coefficient of 0.8. In tandem with improved segmentation and cluster analysis association with specific anatomical regions, our method will be able to differentiate between clinical states of M.tb infection.

Generative adversarial networks (GANs) have received immense attention in the field of machine learning for their potential to learn high-dimensional and real data distribution. These methods do not rely on any assumptions about the data distribution of the input sample and can generate real-like samples from latent vector space based on unsupervised learning. In the medical field, particularly, in digital pathology expert annotation and availability of a large set of training data is costly and the study of manifestations of various diseases is based on visual examination of stained slides. In clinical practice, various staining information is required to improve the pathological diagnosis process. But when the sampled tissue to be examined is limited, the final diagnosis made by the pathologist is based on limited stain styles. These limitations can be overcome by studying the usability and reliability of generative models in the field of digital pathology. To understand the usability of the generative models, we synthesize in an unsupervised manner, high resolution renal microanatomical structures like renal glomerulus in thin tissue histology images using state-of-art architectures like Deep Convolutional Generative Adversarial Network (DCGAN) and Enhanced Super- Resolution Generative Adversarial Network (ESRGAN). Successful generation of such structures will lead to obtaining a large set of labeled data for further developing supervised algorithms for disease classification and understanding progression. Our study suggests while GAN is able to attain formalin fixed and paraffin embedded tissue image quality, GAN requires further prior knowledge as input to model intrinsic micro-anatomical details, such as capillary wall, urinary pole, nuclei placement, suggesting developing semi-supervised architectures by using these above details as prior information. Also, the generative models can be used to create an artificial effect of staining without physically tampering the histopathological slide. To demonstrate this, we use a CycleGAN network to transform Hematoxylin and eosin (H&E) stain to Periodic acid-Schiff (PAS) stain and Jones methenamine silver (JMS) stain to PAS stain. In this way GAN can be employed to translate different renal pathology stain styles when the relevant staining information is not available in the clinical settings.

Glandular features play an important role in the evaluation of prostate cancer. There has been significant interest in the use of 2D pathomics (feature extraction) approaches for detection, diagnosis, and characterization of prostate cancer on digitized tissue slide images. With the development of 3D microscopy techniques, such as open-top light-sheet (OTLS), there is an opportunity for rapid 3D imaging of large tissue specimens such as whole biopsies. In this study, we sought to investigate whether 3D features of gland morphology, namely volume and surface curvature, from OTLS images offer superior discrimination between malignant and benign glands compared to the traditional 2D gland features, namely area and curvature, alone. In this study, a cohort of 8 de-identified fresh prostate biopsies comprehensively imaged in 3D via the OTLS platform. A total of 367 glands were segmented from these images, of which 79 were identified as benign and 288 were identified as malignant. Glands were segmented using a 3D watershed algorithm followed by post-processing steps to filter out falsepositive regions. The 2D and 3D features were compared quantitatively and qualitatively. Our experiments demonstrated that a model using 3D features outperformed one using 2D features in differentiating benign and malignant glands. In 3D, both features, gland volume (p = 1.45 × 10⁻³) and surface curvature (p = 3.2 × 10⁻³), were found to be informative whereas in 2D, only gland area (p = 9 × 10⁻¹⁸) was found to be discriminating (p = 0.79 for 2D curvature). Notable visual and quantitative differences between 3D benign/malignant glands encourage the development of additional more sophisticated features in the future.

Automatic cancer sub-grading of radical prostatectomy (RP) specimens can support clinical studies seeking the prognostic indications of the sub-grades, and potentially benefits patient risk management and treatment planning. We developed and validated an automatic system which classifies each of nine subgrades (i.e. 4 sub-grades of Gleason grade 3, 3 sub-grades of Gleason grade 4, benign intervening, and other cancerous tissue) on digital histopathology whole-slide images (WSIs). The system was cross-validated against expert-drawn contours on a 25-patient data set comprising 92 mid-gland WSIs of RP specimens. The system used a transfer learning technique by fine-tuning AlexNet to classify each cancerous region of interest (ROI) according to sub-grade. We used leave-one-WSI-out cross-validation to measure classifier performance. The system yielded an area under the receiver-operating characteristic curve (AUC) higher than 0.8 for sub-grades of small fused Gleason 4 (G4), intermediate G3, and other cancerous tissue (AUC of 0.976); and AUCs higher than 0.7 for sub-grades of sparse G3, large cribriform G4, and desmoplastic G3.

Bile duct cancer, or cholangiocarcinoma, is a rare and aggressive form of cancer, with approximately 2,500 new diagnoses each year in the United States. Biliary adenocarcinoma comprise of 90% of reported cholangiocarcinoma cases and often presents symptoms in later stages and is therefore associated with poor patient outcomes. Survival rates of cholangiocarcinoma when diagnosed at late stage are 6%, but can be improved to 15% when diagnosed early. Many previous studies have investigated the role of quantitative histomorphometric (QH) features on routinely acquired hematoxylin and eosin (H&E) slides of pancreatobiliary biopsies to help identify malignancies in the lining of the biliary tract. The biopsy procedure carries a high risk of strictures and peritonitis. An alternative to biopsy is a bile duct brushing (BDB), which exclusively contain epithelial cells fixed in cytology slides and have lower complication rates. The goal of this proof of concept study was to evaluate the role of image texture features to differentiate benign clusters and adenocarcinoma on digitized BDB specimens. Textural descriptors are able to characterize local regions of interest within malignant lesions by computing spatial arrangements and statistics of color intensities. Typical features often used in characterizing image texture include Gabor, Law, Haralick, and Co-occurrence of Local Anisotropic Gradient Orientations (CoLlAGe) features. These features are able to capture subtle differences in malignant and benign phenotypes caused by a multitude of phenotypic changes, including nuclear overcrowding, disorientation, and disorderly chromatin distribution. Pixel-wise texture features from each family were computed on whole slide images (WSIs) of BDBs corresponding to 59 unique patients of which 31 were from patients pathologically confirmed as having biliary adenocarcinoma through next generation sequencing, and the remaining 28 were confirmed as benign. Each WSI was magnified at 11X, 5.5X, and 2.25X before features were extracted. A total of 445 clusters were annotated across all slide images, of which 275 were identified as malignant by an expert cytopathologist, and used as the ground truth for our machine learning models. The pixel-wise feature distributions were summarized using mean, median, standard deviation, skewness, and kurtosis. The top five statistical texture features were selected using Wilcoxon ranksum, two-sided t-test, and maximum relevance minimum redundancy tests in 3-fold patient-wise cross validation and used to train three different machine learning classifiers. A linear discriminant classifier was trained with the mean Haralick information measure 2, standard deviation and median of CoLlAGe correlation, and mean of 2D Gabor response with {0,4}Hz frequencies and orientations f2 ; 58 g radian orientations, all selected by the mRMR test. We obtained an AUC of 0:83 ±0:01 using 3-fold cross-validation on 39 patients in the training set and 20 patients in the testing set.

Subjective interpretation of histology slides forms the basis of cancer diagnosis, prognosis, and therapeutic response prediction. Deep learning models can potentially help serve as an efficient, unbiased tool for this task if trained on large amounts of labeled data. However, labeled medical data, such as small regions of interests, are often costly to curate. In this work, we propose a flexible, semi-supervised framework for histopathological classification that first uses Contrastive Predictive Coding (CPC) to learn semantic features in an unsupervised manner and then use an attention-based multiple Instance Learning (MIL) for classification without requiring patch-level annotations.

Deep learning-based approaches have shown highly successful performance in the categorization of digitized biopsy samples. The commonly used setting in these approaches is to employ convolutional neural networks for classification of data sets consisting of images all having the same size. However, the clinical practice in breast histopathology necessitates multi-class categorization of regions of interest (ROI) in biopsy samples where these regions can have arbitrary shapes and sizes. The typical solution to this problem is to aggregate the classification results of fixed-sized patches cropped from these images to obtain image-level classification scores. Another limitation of these approaches is the independent processing of individual patches where the rich contextual information in the complex tissue structures has not yet been sufficiently exploited. We propose a generic methodology to incorporate local inter-patch context through a graph convolution network (GCN) that admits a graph-based ROI representation. The proposed GCN model aims to propagate information over neighboring patches in a progressive manner towards classifying the whole ROI into a diagnostic class. The experiments using a challenging data set for a 4-class ROI-level classification task and comparisons with several baseline approaches show that the proposed model that incorporates the spatial context by using graph convolutional layers performs better than commonly used fusion rules.

Prostate cancer (PCa) is one of the most frequent cancers in men. Its grading is required before initiating its treatment. The Gleason Score (GS) aims at describing and measuring the regularity in gland patterns observed by a pathologist on the microscopic or digital images of prostate biopsies and prostatectomies. Deep Learning based (DL) models are the state-of-the-art computer vision techniques for Gleason grading, learning high-level features with high classification power. However, for obtaining robust models with clinical-grade performance, a large number of local annotations are needed. Previous research showed that it is feasible to detect low and high-grade PCa from digitized tissue slides relying only on the less expensive report{level (weakly) supervised labels, thus global rather than local labels. Despite this, few articles focus on classifying the finer-grained GS classes with weakly supervised models. The objective of this paper is to compare weakly supervised strategies for classification of the five classes of the GS from the whole slide image, using the global diagnostic label from the pathology reports as the only source of supervision. We compare different models trained on handcrafted features, shallow and deep learning representations. The training and evaluation are done on the publicly available TCGA-PRAD dataset, comprising of 341 whole slide images of radical prostatectomies, where small patches are extracted within tissue areas and assigned the global report label as ground truth. Our results show that DL networks and class-wise data augmentation outperform other strategies and their combinations, reaching a kappa score of κ = 0:44, which could be further improved with a larger dataset or combining both strong and weakly supervised models.

Tumor cell populations in histopathology exhibit enormous heterogeneity in phenotypic traits such as uncontrolled cellular and microvascular proliferation, nuclear atypia, recurrence and therapy response. However, there is a limited quantitative understanding of how the molecular genotype correspond with the morphological phenotype in cancer. In this work, we develop a deep learning algorithm that learns to map molecular profiles to histopathological patterns. In our preliminary results, we are able to generate high-quality, realistic tissue samples, and demonstrate that by attenuating the mutation of status of few genes, we are able to guide the histopathology tissue image synthesis to exhibit different phenotypes.

Despite holding enormous potential in elucidating the tumor microenvironment and its phenotypic morphological heterogeneity, whole-slide image slides are underutilized in the analysis of survival outcomes and biomarker discovery, with very few methods developed that seek to integrate transcriptome profiles with histopathology data. In this work, we propose to fuse of molecular and histology features using artificial intelligence, and train an end-to-end multimodal deep neural network for survival outcome prediction. Our research establishes insight and theory on how to combine multimodal biomedical data, which will be integral for other problems in medicine with heterogenous data sources.

Spatial arrangement of cells of various types, such as tumor infiltrating lymphocytes and the advancing edge of a tumor, are important features for detecting and characterizing cancers. However, convolutional neural networks (CNNs) do not explicitly extract intricate features of the spatial arrangements of the cells from histopathology images. In this work, we propose to classify cancers using graph convolutional networks (GCNs) by modeling a tissue section as a multi-attributed multi-relational spatial graph of its constituent cells. Cells are detected using their nuclei in H and E stained tissue image, and each cell’s appearance is captured as a multi-attributed high-dimensional vertex feature. The spatial relations between neighboring cells are captured as edge features based on their distances in a multi-relational graph. We demonstrate the utility of this approach by obtaining classification accuracy that is competitive with CNNs, specifically, Inception-v3, on two tasks – cancerous versus non-cancerous and in situ versus invasive – on the BACH breast cancer dataset.

Ki-67 index is an important diagnostic factor in gastrointestinal neuroendocrine tumor (GI-NET). The current gold standard for grading GI-NETs involves the visual screening of histopathologically stained tissues, for hot-spots containing high amounts of proliferating tumor cells (stained with Ki-67 antibody). Subsequently, the Ki-67 index, i.e. the percentage of proliferating tumor cells within the hot-spot is manually obtained. To automate this subjective and time consuming process, we have developed an integrated pipeline, termed SKIE (synaptophysin-Ki-67 index estimator), combining double-immunohistochemical (IHC) staining for synaptophysin (stains tumor) and Ki-67, with whole slide image (WSI) analysis. The Ki-67 index for 50 human GI-NET WSIs were estimated by SKIE and compared with three pathologists’ assessment, and the gold standard (exhaustive counting by a fourth pathologist) based on the double-stained image. All four pathologists unanimously graded 38 WSIs, among which, SKIE achieved 94.74% accuracy. One discrepant case was attributed to staining inconsistencies and the other to SKIE selecting a better hot-spot. The remaining 12 WSIs had discrepant grades among pathologists, and hence, the gold standard was chosen for comparison, wherein, 10 WSI grades matched with that of the gold standard, and SKIE assigned a lower and higher grade to two cases. Overall, SKIE agreed with the gold standard with a substantial linear weighted Cohen’s kappa κ = 0.622 with CI [0.286, 0.958]. We further expanded our method to deep-SKIE, wherein, a deep convolutional neural network (DCNN) was trained and validated using 13,736 hotspot-sized tiles from 40 WSIs, each categorized into one of four classes (background, non-tumor, tumor grade 1, tumor grade 2) by SKIE and tested on 9 WSIs. Deep-SKIE achieved an accuracy of 91.63% with near-perfect agreement (κ = 0.88 with CI [0.87, 0.89]) with the gold standard.

The presence of tumor-infiltrating lymphocytes (TILs) is correlated with outcome and prognosis in epithelial ovarian cancer (EOC). In this study, automated image analysis was used to analyze the association between overall survival (OS) and TIL spatial arrangement and density in a multi-site cohort of 102 EOC patients who received adjuvant chemotherapy following debulking surgery. Features of the spatial arrangement of TILs (SpaTIL) were used to quantify the spatial co-localization of TILs and tumor cells on digitized pathology slides of the malignant neoplasm of excised specimens. A multivariable Cox regression model of SpaTIL features was fit on the n₁ = 51 patient training set and was evaluated in the n₂ = 51 patient validation set. The SpaTIL signature was significantly associated with OS, both in the training set (hazard ratio (HR) = 2.81, 95% confidence interval (CI) = 1.33 − 5.92, and p = 0.003) and the validation set (HR = 2.06, 95% CI = 1.04 − 4.07, and p = 0.008). In addition, fusing our spaTIL risk score and the clinical staging further improved the results of the predictive model (HR = 4.045, 95% CI = 4.11−5.41, and p = 0.0002 in the validation set) and outperformed clinical staging alone. This finding illustrates that a spaTIL risk score is not only able to predict OS independent of clinical data, but also offers prognostic value complementary to current clinical standard-of-care. Patients with longer survival times had significantly higher heterogeneity of non-TIL cluster area, while shorter time survivors had mostly same-sized, evenly-distributed non-TIL clusters and smaller average TIL cluster area. These findings suggest that dispersion of TILs throughout the tumor is associated with better treatment response to post-treatment adjuvant chemotherapy, and therefore longer survival time.

Hematoxylin and Eosin (H&E) are one of the main tissue stains used in histopathology to discriminate between nuclei and extracellular material while performing a visual analysis of the tissue. However, histopathology slides are often characterized by stain color heterogeneity, due to different tissue preparation settings at different pathology institutes. Stain color heterogeneity poses challenges for machine learning-based computational analysis, increasing the difficulty of producing consistent diagnostic results and systems that generalize well. In other words, it is challenging for a deep learning architecture to generalize on stain color heterogeneous data, when the data are acquired at several centers, and particularly if test data are from a center not present in the training data. In this paper, several methods that deal with stain color heterogeneity are compared regarding their capability to solve center-dependent heterogeneity. Systematic and extensive experimentation is performed on a normal versus tumor tissue classification problem. Stain color normalization and augmentation procedures are used while training a convolutional neural networks (CNN) to generalize on unseen data from several centers. The performance is compared on an internal test set (test data from the same pathology institutes as the training set) and an external test set (test data from institutes not included in the training set). This also allows to measure generalization performance. An improved performance is observed when the predictions of the two best-performed stain color normalization methods with augmentation are aggregated. An average AUC and F1-score on external test are observed as 0:892±0:021 and 0:817±0:032 compared to the baseline 0:860±0:027 and 0:772 ± 0:024 respectively.

Hematoxylin and Eosin (H&E) is a widely-used stain for diagnosis and prognosis in clinical pathology; it is a non-specific stain, binding to all cell types. Immunofluorescent (IF) staining is highly specific, binding only to targeted proteins in a sample to identify specific cellular and sub-cellular structures. IF images are costlier and more technically difficult compared with H&E, so are rarely used in routine clinical workup, but can be used for identification of diagnostically significant cell types. In previous work, we used registered IF and H&E images to generate class labels for training a deep learning H&E segmentation algorithm. In this work, we leverage this dataset to train a Conditional Generative Adversarial Network (cGAN) to generate realistic-looking H&E images from IF images stained for DAPI and ribosomal S6. Using these generated images, we trained a semantic segmentation algorithm to identify nuclei, cytoplasm, and membrane classes by thresholding the original IF stains for use as class labels on the generated H&E. The trained classifier was then used to segment a holdout dataset of real H&E images. We found that the semantic segmentation models trained on the generated H&E images (Dice score: 0.539) performed similarly to models trained on real H&E (Dice score: 0.503), suggesting that cGAN generated samples can be used as a viable training set for deep learning models that are intended to be applied on real H&E data.

Hyperspectral imaging (HSI), which acquires up to hundreds of bands, has been proposed as a promising imaging modality for digitized histology beyond RGB imaging to provide more quantitative information to assist pathologists with disease detection in samples. While digitized RGB histology is quite standardized and easy to acquire, histological HSI often requires custom-made equipment and longer imaging times compared to RGB. In this work, we present a dataset of corresponding RGB digitized histology and histological HSI of breast cancer, and we develop a conditional generative adversarial network (GAN) to artificially synthesize HSI from standard RGB images of normal and cancer cells. The results of the GAN synthesized HSI are promising, showing structural similarity (SSIM) of approximately 80% and mean absolute error (MAE) of 6 to 11%. Further work is needed to establish the ability of generating HSI from RGB images on larger datasets.

In recent years, hyperspectral imaging (HSI) has been shown as a promising imaging modality to assist pathologists in the diagnosis of histological samples. In this work, we present the use of HSI for discriminating between normal and tumor breast cancer cells. Our customized HSI system includes a hyperspectral (HS) push-broom camera, which is attached to a standard microscope, and home-made software system for the control of image acquisition. Our HS microscopic system works in the visible and near-infrared (VNIR) spectral range (400 - 1000 nm). Using this system, 112 HS images were captured from histologic samples of human patients using 20× magnification. Cell-level annotations were made by an expert pathologist in digitized slides and were then registered with the HS images. A deep learning neural network was developed for the HS image classification, which consists of nine 2D convolutional layers. Different experiments were designed to split the data into training, validation and testing sets. In all experiments, the training and the testing set correspond to independent patients. The results show an area under the curve (AUCs) of more than 0.89 for all the experiments. The combination of HSI and deep learning techniques can provide a useful tool to aid pathologists in the automatic detection of cancer cells on digitized pathologic images.

The purpose of this study is to develop hyperspectral imaging (HSI) for automatic detection of head and neck cancer cells on histologic slides. A compact hyperspectral microscopic system is developed in this study. Histologic slides from 15 patients with squamous cell carcinoma (SCC) of the larynx and hypopharynx are imaged with the system. The proposed nuclei segmentation method based on Principle Component Analysis (PCA) can extract most nuclei in the hyperspectral image without extracting other sub-cellular components. Both spectra-based support vector machine (SVM) and patchbased convolutional neural network (CNN) are used for nuclei classification. CNNs were trained with both hyperspectral images and pseudo RGB images of extracted nuclei, in order to evaluate the usefulness of extra information provided by hyperspectral imaging. The average accuracy of spectra-based SVM classification is 68%. The average AUC and average accuracy of the HSI patch-based CNN classification is 93.7 and 82.4%, respectively. The hyperspectral microscopic imaging and classification methods provide an automatic tool to aid pathologists in detecting SCC on histologic slides.

The incidence of esophageal squamous cell carcinoma (ESCC) in China is high and the prognosis is poor. Aimed to evaluate the ESCC, we investigated computerized quantitative analyses on diagnostic computed tomography (CT) and digital histopathological slices. A retrospective study with IRB approval was conducted to assess the prognosis of ESCCs in 158 patients who underwent esophagectomy. Every of them has 3D CT image and pathological hematoxylin-eosin staining tissue slide. We performed quantitative analysis on digital histology slices and diagnostic CT volumes. A total of 125 computerized quantitative features were extracted including 20 pathological image clustering (PIC) features and 105 CT features. Then we selected 45 CT features by correlation analysis and 5 PIC features by univariate survival analysis for preventing overfitting. Cox hazard model with L1 penalization was used for prognostic indexing. We compared three different prognostic indices in order to build a robust model: Model A: using CT features alone; Model B: using PIC features alone; Model C: using both CT and PIC features. For testing the model, we performed leave-one-out cross-validation for testing and we used the index of concordance(C-index) to access the accuracy of the prognostic model. The testing C-index values of Model A, B, C were 0.667, 0.601 and 0.711. The improvement of the final Model C is significant and it’s able to provide effective stratification. Quantitative prognostic modeling with joint information from clinical, histopathology, and diagnostic CT has the ability to stratify the patient well. Our prognostic model has potential to impact the perioperative care in the future clinical practice and improve the quality of individual life.

Systemic lupus erythematosus is a disease in which the immune system attacks host tissues. One organ commonly attacked is the kidney, in which case the resultant acute and chronic damages are called lupus nephritis. The accumulated damage can result in renal failure. The percutaneous renal biopsy is invaluable to the assessment of the disease and its therapeutic response. A large portion of the pathological assessment is done by histological analysis of the biopsied tissue with light microscopy. Computational models can alleviate a portion of expert disagreement by providing unified, reproducible quantifications of digitized image structures. In this work, we perform fully automated whole slide segmentation of glomeruli from Periodic Acid- Schiff (PAS), hematoxylin and eosin, silver, and trichrome stained lupus nephritis biopsies. The automatically extracted PAS glomeruli are quantified by a set of 285 hand-crafted features designed specifically to target glomerular lesions in lupus nephritis. These features are fed in sequence to a recurrent neural network architecture which views multiple glomerular features from a single biopsy, and outputs a continuous diagnostic value representative of classes II-V of the scheme by Weening et al1. On 82 whole slide images taken from 65 patients, compared to renal pathologist annotations and using only the PAS stain, the network achieved a Cohen’s kappa of 0.42 with 95% confidence interval [0.32, 0.52] to render the correct class chosen from II-V, and 0.56, 95% CI [0.43, 0.69] to render an additional class V diagnosis when required.

Diffuse large B-cell lymphoma (DLBCL) is the most common type of B-cell lymphoma. It is characterized by a heterogeneous morphology, genetic changes and clinical behavior. A small specific subgroup of DLBCL, harbouring a MYC gene translocation is associated with worse patient prognosis and outcome. Typically, the MYC translocation is assessed with a molecular test (FISH), that is expensive and time-consuming. Our hypothesis is that genetic changes, such as translocations could be visible as changes in the morphology of an HE-stained specimen. However, it has not proven possible to use morphological criteria for the detection of a MYC translocation in the diagnostic setting due to lack of specificity.

In this paper, we apply a deep learning model to automate detection of the MYC translocations in DLBCL based on HE-stained specimens. The proposed method works at the whole-slide level and was developed based on a multicenter data cohort of 91 patients. All specimens were stained with HE, and the MYC translocation was confirmed using fluorescence in situ hybridization (FISH). The system was evaluated on an additional 66 patients, and obtained AUROC of 0.83 and accuracy of 0.77. The proposed method presents proof of a concept giving insights in the applicability of deep learning methods for detection of a genetic changes in DLBCL. In future work we will evaluate our algorithm for automatic pre-screen of DLBCL specimens to obviate FISH analysis in a large number of patients.

Oral dysplasia is a pre-malignant stage of oral epithelial carcinomas, e.g., oral squamous cell carcinoma, where significant changes in tissue layers and cells can be observed under the microscope. However, malignancy can be reverted or cured using proper medication or surgery if the grade of malignancy is assessed properly. The assessment of correct grade is therefore critical in patient management as it can change the treatment decisions and prognosis for the dysplastic lesion. This assessment is highly challenging due to considerable inter- and intraobserver variability in pathologists’ agreement, which highlights the need for an automated grading system that can predict more accurate and reliable grade. Recent advancements have made it possible for digital pathology (DP) and artificial intelligence (AI) to join forces from the digitization of tissue slides into images and using those images to train and predict more accurate grades using complex AI models. In this regard, we propose a novel morphometric approach exploiting the architectural features in dysplastic lesions i.e., irregular epithelial stratification where we measure the widths of different layers of the epithelium from the boundary layer i.e., keratin projecting inwards to the epithelium and basal layers to the rest of the tissue section from a clinically significant viewpoint.

In the age of modern medicine and artificial intelligence, image analysis and machine learning have revolutionized diagnostic pathology, facilitating the development of computer aided diagnostics (CADs) which circumvent prevalent diagnostic challenges. Although CADs will expedite and improve the precision of clinical workflow, their prognostic potential, when paired with clinical outcome data, remains indeterminate. In high impact renal diseases, such as diabetic nephropathy and lupus nephritis (LN), progression often occurs rapidly and without immediate detection, due to the subtlety of structural changes in transient disease states. In such states, exploration of quantifiable image biomarkers, such as Neutrophil Extracellular Traps (NETs), may reveal alternative progression measures which correlate with clinical data. NETs have been implicated in LN as immunogenic cellular structures, whose occurrence and dysregulation results in excessive tissue damage and lesion manifestation. We propose that renal biopsy NET distribution will function as a discriminate, predictive biomarker in LN, and will supplement existing classification schemes. We have developed a computational pipeline for segmenting NET-like structures in LN biopsies. NET-like structures segmented from our biopsies warrant further study as they appear pathologically distinct, and resemble nonlytic, vital NETs. Examination of corresponding H&E regions predominantly placed NET-like structures in glomeruli, including globally and segmentally sclerosed glomeruli, and tubule lumina. Our work continues to explore NET-like structures in LN biopsies by: 1.) revising detection and analytical methods based on evolving NETs definitions, and 2.) cataloguing NET morphology in order to implement supervised classification of NET-like structures in histopathology images.

Triple-negative breast cancer (TNBC) is an aggressive subtype of breast cancer characterized by a high level of inflammation. In this work, we present a computational method for assessing immune activity across many populations of immune cells. For this analysis, we have a dataset of 18 TNBC biopsies, each stained with a ‘strip-and-reprobe’ approach consisting of multiple rounds of staining and imaging the tissue. Each sample is stained with 8 separate antibody panels, resulting in ~20 different stains per sample, plus tissue autofluorescence images. Biopsy sections are imaged across a ~4mm x 5mm area on a Caliber ID confocal microscope with a pixel size of 221 nm, resulting in image composites of ~27,000x27,000x20 pixels per sample. Using radiomic texture analysis, we have analyzed a subset of these images to determine which texture features are predictive of inflammation. From 3 of these samples, we have selected inflamed and uninflamed regions of interest (ROIs) for three lymphocyte markers: 1) CD4, 2) CD3, and 3) CD8. We have computed 20 texture features, specifically gray level co-occurrence matrix features, for each of these 42 ROIs and their corresponding tissue autofluorescence images. From this analysis, we have found multiple features that successfully differentiate inflamed tissue from uninflamed tissue. These features will be calculated across the full dataset to analyze the overall TNBC immune architecture of these samples as diffuse or compartmentalized, and which cell populations tend to coaggregate.

Diabetic Nephropathy (DN) progression is stratified into several stages with different levels of proteinuria, albuminuria, and physical characteristics as observed by pathologists. These physical changes are primarily visible within a patient’s glomeruli which function as filtration units for blood returning for oxygenation. As DN stage increases, it is possible to observe the thickening of the glomerular basement membrane, expansion of the mesangium, and development of nodular sclerosis. Classification of different stages of DN by pathologists is based on semiqualitative assessments of these characteristics on an individual glomerulus basis. Being able to probabilistically infer stage membership of individual glomeruli based on a combination of easily observable and hidden image features would be an invaluable tool for furthering our understanding of the drivers of DN progression. Markov Particle filters, included in the bnlearn package in R, were used to query a Bayesian Network (BN) constructed using the structural Hill-Climbing algorithm on a set of glomerular features. These features included both traditional characteristics such as glomerular area and number of mesangial nuclei as well as more abstract features derived from Minimum Spanning Trees (MST) to quantify spatial distribution of mesangial nuclei. Our results using images from multiple institutions suggest that these abstract features exercise a variable influence on DN stage membership over the course of disease progression. Further research incorporating clinical data will give nephrologists a “white box” visual of quantitative factors present in DN patients.

This article presents an entire framework for analyzing survival-related gland features in gastric cancer images. This approach builds upon a previous automatic gland detection, which partitions the tissue into a set of primitive objects (glands) from a binarized version of the hematoxylin channel. Next, gland shape and nuclei are characterized using local and contextual features that include relationships between color or texture from glands and nuclei (5:120 features). A mutual information max-relevance-min-redundancy (mRMR) approach selects hundred features that correlate with patient survival "survival vs not survival (first year)". Finally, ten statistically significant features (test t-student, p < 0:05) were used to set a "one-year" survival. Evaluation was carried out in a set of fourteen cases diagnosed with pre-cancerous gastric lesions or cancer, under a leave-one-out scheme. Results showed an accuracy of 78.57% when predicting the patient survival (less or more than a year), using a QDA Linear & Quadratic Discriminant Analysis. This approach suggests there exist morphometric gland differences among cases with gastric related pathology.

ABSTRACT When training a deep learning model, the dataset used is of great importance to make sure that the model learns relevant features of the data and that it will be able to generalize to new data. However, it is typically difficult to produce a dataset without some bias toward any specific feature. Deep learning models used in histopathology have a tendency to overfit to the stain appearance of the training data { if the model is trained on data from one lab only, it will usually not be able to generalize to data from other labs. The standard technique to overcome this problem is to use color augmentation of the training data which, artificially, generates more variations for the network to learn. In this work we instead test the use of a so called domain-adversarial neural network, which is designed to prevent the model from being biased towards features that in reality are irrelevant such as the origin of an image. To test the technique, four datasets from different hospitals for Gleason grading of prostate cancer are used. We achieve state of the art results for these particular datasets, and furthermore for two of our three test datasets the approach outperforms the use of color augmentation.

The color rendering of whole-slide images (WSIs) depends on factors involving the sample, such as tissue type, preparation methods, staining type and staining protocol, as well as equipment, such as the WSI scanner, WSI viewer, and WSI display. Variations in any of these steps may change the color rendering and therefore affect the performance of pathologists in the interpretation of WSIs and the robustness of artificial intelligence algorithms. In the literature, color normalization techniques have been proposed to reduce the color variations. The purpose of this work is to develop an objective approach to characterizing color normalization methods used in digital pathology. We employed color normalization methods to normalize the color rendered by a WSI scanner and then compared the normalized color with the actual scan by that scanner. The normalization errors were evaluated on the pixel level using the CIE color difference ΔE metric that have been shown to correlate with visually perceived differences in human vision. A selected set of 310 patch images of breast tissues scanned by two scanners from the ICPR 2014 MITOS & ATYPIA contest was used. Images from one scanner were color normalized to match the color rendering of the other scanner. Four color normalization methods were compared – Macenko, Reinhard, Vahadane, and StainGAN. Experimental results show that average color differences between two scanners in terms of ΔE were reduced from 16.2 before normalization to the range of [13.7,16.9] after normalization for the Macenko, Reinhard, Vahadane methods, and to 8.3 for the StainGAN method. Apparently the StainGAN method is significantly superior to the other three methods in terms of the ΔE metric. As such, we demonstrated a quantitative method for objectively evaluating color normalization techniques. Future work is needed to explore the relationship of the color fidelity measure and the impact of color normalization on pathologist and AI performance in clinical tasks.

The primary purpose of the kidney, specifically the glomerulus, is filtration. Filtration is accomplished through the glomerular filtration barrier, which consists of the fenestrated endothelium, glomerular basement membrane, and specialized epithelial cells called podocytes. In pathologic states, such as Diabetes Mellitus (DM) and diabetic kidney disease (DKD), variable glomerular conditions result in podocyte injury and depletion, followed by progressive glomerular injury and DKD progression. In this work we quantified glomerulus and podocyte structural changes in histopathology image data derived from a murine model of DM. Using a variety of image processing techniques, we studied changes in podocyte morphology and intra-glomerular distribution across healthy, mild DM, and DM glomeruli. Our feature analysis provided feature trends which we believe are reflective of DKD pathology; while glomerular area peaked in mild DM, average podocyte number and distance from the urinary pole continued to increase throughout DM. Ultimately, this study aims to augment the set of quantifiable image biomarkers used for evaluation of DKD progression in digital pathology, as well as underscore the importance of engineering biologically inspired image features.

We demonstrate an instance segmentation method with Mask R-CNN using a ResNet-101 plus Feature Pyramid Network convolutional backbone to segment and classify T cells and antigen presenting cells (APCs) in multi-channel fluorescence confocal images. This network improves on our previous cell distance mapping (CDM) pipeline, which used a custom 10- layer convolutional neural network for cell segmentation. We have validated Mask R-CNN on two independent datasets of fluorescence confocal images: 1) mouse lymph node tissue, and 2) human lupus nephritis (LuN) biopsies. For dataset 1, mice were injected with fluorescent dendritic cells and two populations of fluorescent T cells. Mask R-CNN improved sensitivity averaged across all cell types from 0.88 to 0.94. Specificity improved from 0.92 to 0.95 across all cell types, and intersection over union score (IOU) improved significantly from 0.82 to 0.86 (p < 0.0001). Human LuN biopsies in dataset 2 were stained with two T cell markers and two APC markers, with separate staining panels to identify different populations of APCs. Mask R-CNN again improved segmentation and classification averaged across all cell types, increasing overall sensitivity from 0.72 to 0.76, specificity from 0.86 to 0.93, and significantly increasing IOU from 0.71 to 0.81 (p < 0.0001). Improved IOU scores are particularly important in CDM to be able to quantify cell shape for identification of functional interactions of immune cells. Mask R-CNN is therefore a superior method for instance segmentation of immune cells in microscopy images for image analysis of cellular function in pathological immune states.

In the field of computer vision and medical imaging, the classification is one of the common approaches apply to solve different problems. There are several Deep Learning (DL) models have been proposed in the last few years and applied for classification tasks and shown significantly better performance compared to existing methods. In particular, the DL methods have been applied and achieved state-of-the-art performance for classification, segmentation, and detection tasks in the field of medical imaging. In the field of digital pathology, very large dimensional images are captured with microscopes which are significantly higher in dimension and different (texture, color, etc.) compared to the images are used for analysis in computer vision tasks. As the computational pathology becomes very promising area of research, it is very important to study the impact of different color spaces for pathological image classification problems. In this paper, we have considered six different color spaces including RGB, CIE, HSV, YCbCr, Lab, and HSL for histopathology tissue classification tasks where the Inception Residual Recurrent Convolutional Neural Network (IRRCNN) model is applied. The KIMIA Path960 database is used for evaluating the model in this implementation. The RGB color space shows the highest testing performance for tissue classification task which is around 0. 73% and 1.13% better compared to the performance of Lab and CIE color spaces respectively and this performance is significantly higher than the testing accuracy of the model in others color spaces. In addition, the highest performance for the proposed model is around 0.5 percent better compared to existing methods. This study will provide a clear guidance in advance for implementing classification tasks in digital pathology.I n the field of computer vision and medical imaging, the classification is one of the common approaches apply to solve different problems. There are several Deep Learning (DL) models have been proposed in the last few years and applied for classification tasks and shown significantly better performance compared to existing methods. In particular, the DL methods have been applied and achieved state-of-the-art performance for classification, segmentation, and detection tasks in the field of medical imaging. In the field of digital pathology, very large dimensional images are captured with microscopes which are significantly higher in dimension and different (texture, color, etc.) compared to the images are used for analysis in computer vision tasks. As the computational pathology becomes very promising area of research, it is very important to study the impact of different color spaces for pathological image classification problems. In this paper, we have considered six different color spaces including RGB, CIE, HSV, YCbCr, Lab, and HSL for histopathology tissue classification tasks where the Inception Residual Recurrent Convolutional Neural Network (IRRCNN) model is applied. The KIMIA Path960 database is used for evaluating the model in this implementation. The RGB color space shows the highest testing performance for tissue classification task which is around 0. 73% and 1.13% better compared to the performance of Lab and CIE color spaces respectively and this performance is significantly higher than the testing accuracy of the model in others color spaces. In addition, the highest performance for the proposed model is around 0.5 percent better compared to existing methods. This study will provide a clear guidance in advance for implementing classification tasks in digital pathology.

Purpose - Ultra-widefield fluorescein angiography (UWFA) images are used to assess retinal, vascular, and choroidal abnormalities in retinal disease. During image acquisition, images are taken in sequential time points, which allows for interrogation of vascular features, as well as other pathologies, such as leakage. Variations in eye positioning, injection, and camera positioning all contribute to variability in image quality. The purpose of this study was to evaluate the feasibility of automated image quality classification and selection using deep learning. Methods - The images for this analysis were composed of 3543 UWFA images obtained during standard UWFA image acquisition. Ground truth image quality was assessed by expert image review, and classified into one of four categories (ungradable, poor, good, or best. 3543 images were used to train the model. A testing set composed of 392 images was used to assess model performance. Results - By expert review of 3935 images, 110 (2.8%) were graded as best, 1042 (26.5%) as good, 1156 (29.4%) as poor and 1627 (41.3%) were ungradable. In the testing set, the automated qualit y assessment system showed an overall accuracy of 88% for recognizing between gradable and ungradable images, and 77% accuracy for four-category classification. The receiver operating characteristic (ROC) curve measuring performance of two-class classification (ungradable and gradable) had an AUC of 0.945. Conclusions – We created a deep learning classification model that automatic classified UWFA images by quality category. The high degree of accuracy provides evidence that this method could be used to enhance the acquisition of angiogram images and speed up clinic workflow. This could result in reduced manual image grading workload, allow quality-based image presentation to clinicians, and provide near-instantaneous feedback on image quality during image acquisition for photographers.