Thanks to reviewers who served JMI in 2021.

Purpose: Compton interactions amount to a significant fraction of the registered counts in a silicon detector. In a Compton interaction, only a part of the photon energy is deposited and a single incident photon can result in multiple counts unless tungsten shielding is used. Deep silicon has proved to be a competitive material for photon-counting CT detectors, but to improve the performance further, one possibility is to use coincidence techniques to identify Compton-scattered photons and reconstruct their incident energies.

Approach: In a detector with no tungsten shielding, incident photons can interact through a series of interactions. Based on the position and energy of each interaction, probability-based methods can be used to estimate the incident photon energy. Here, we present a maximum likelihood estimation framework along with an alternative method to estimate the incident photon energy and position in a silicon detector.

Results: Assuming one incident photon per time frame, we show that the incident photon energy can be estimated with a mean error of −0.07  ±  0.03  keV and an RMS error of 3.36  ±  0.02  keV for a realistic case in which we assume a detector with limited energy and spatial resolution. The interaction position was estimated with a mean error of −2  ±  11  μm in x direction and 7  ±  11  μm in y direction. Corresponding RMS errors of 1.09  ±  0.01 and 1.10  ±  0.01  mm were achieved in x and y, respectively.

Conclusions: The presented results show the potential of using probability-based methods to improve the performance of silicon detectors for CT.

Purpose: To develop a synergistic image reconstruction framework that exploits multicontrast (MC), multicoil, and compressed sensing (CS) redundancies in magnetic resonance imaging (MRI).

Approach: CS, MC acquisition, and parallel imaging (PI) have been individually well developed, but the combination of the three has not been equally well studied, much less the potential benefits of isotropy within such a setting. Inspired by total variation theory, we introduce an isotropic MC image regularizer and attain its full potential by integrating it into compressed MC multicoil MRI. A convex optimization problem is posed to model the new variational framework and a first-order algorithm is developed to solve the problem.

Results: It turns out that the proposed isotropic regularizer outperforms many of the state-of-the-art reconstruction methods not only in terms of rotation-invariance preservation of symmetrical features, but also in suppressing noise or streaking artifacts, which are normally encountered in PI methods at aggressive undersampling rates. Moreover, the new framework significantly prevents intercontrast leakage of contrast-specific details, which seems to be a difficult situation to handle for some variational and low-rank MC reconstruction approaches.

Conclusions: The new framework is a viable option for image reconstruction in fast protocols of MC parallel MRI, potentially reducing patient discomfort in otherwise long and time-consuming scans.

Purpose: Multisource images are interesting in medical imaging. Indeed, multisource images enable the use of complementary information of different sources such as for T1 and T2 modalities in MRI imaging. However, such multisource data can also be subject to redundancy and correlation. The question is how to efficiently fuse the multisource information without reinforcing the redundancy. We propose a method for segmenting multisource images that are statistically correlated.

Approach: The method that we propose is the continuation of a prior work in which we introduce the copula model in hidden Markov fields (HMF). To achieve the multisource segmentations, we use a functional measure of dependency called “copula.” This copula is incorporated in the conditionally random fields (CRF). Contrary to HMF, where we consider a prior knowledge on the hidden states modeled by an HMF, in CRF, there is no prior information and only the distribution of the hidden states conditionally to the observations can be known. This conditional distribution depends on the data and can be modeled by an energy function composed of two terms. The first one groups the voxels having similar intensities in the same class. As for the second term, it encourages a pair of voxels to be in the same class if the difference between their intensities is not too big.

Results: A comparison between HMF and CRF is performed via theory and experimentations using both simulated and real data from BRATS 2013. Moreover, our method is compared with different state-of-the-art methods, which include supervised (convolutional neural networks) and unsupervised (hierarchical MRF). Our unsupervised method gives similar results as decision trees for synthetic images and as convolutional neural networks for real images; both methods are supervised.

Conclusions: We compare two statistical methods using the copula: HMF and CRF to deal with multicorrelated images. We demonstrate the interest of using copula. In both models, the copula considerably improves the results compared with individual segmentations.

Purpose: We investigate the enhancement in terahertz (THz) images of freshly excised breast tumors upon treatment with an optical clearance agent. The hyperspectral imaging and spectral classifications are used to quantitatively demonstrate the image enhancement. Glycerol solution with 60% concentration is applied to excised breast tumor specimens for various time durations to investigate the effectiveness on image enhancement.

Approach: THz reflection spectroscopy is utilized to obtain the absorption coefficient and the index of refraction of untreated and glycerol-treated tissues at each frequency up to 3 THz. Two classifiers, spectral angular mapping (SAM) based on several kernels and Euclidean minimum distance (EMD) are implemented to evaluate the effectiveness of the treatment. The testing raw data is obtained from five breast cancer specimens: two untreated specimens and three specimens treated with glycerol solution for 20, 40, or 60 min. All tumors used in the testing data have healthy tissues adjacent to cancerous ones consistent with the challenge faced in lumpectomy surgeries.

Results: The glycerol-treated tissues showed a decrease in the absorption coefficients compared with untreated tissues, especially as the period of treatment increased. Although the sensitivity metric of the classifier presented higher values in the untreated tissues compared with the treated ones, the specificity and accuracy metrics demonstrated higher values for the treated tissues compared with the untreated ones.

Conclusions: The biocompatible glycerol solution is a potential optical clearance agent in THz imaging while keeping the histopathology imaging intact. The SAM technique provided a good classification of cancerous tissues despite the small amount of cancer in the training data (only 7%). The SAM exponential kernel and EMD presented classification accuracy of ∼80  %   to 85% compared with linear and polynomial kernels that provided accuracy ranging from 70% to 80%. Overall, glycerol treatment provides a potential improvement in cancer classification in freshly excised breast tumors.

Purpose: Dual-contrast protocols are a promising clinical multienergy computed tomography (CT) application for focal liver lesion detection and characterization. One avenue to enable multienergy CT is the introduction of photon-counting detectors (PCD). Although clinical translation is highly desired because of the diagnostic benefits of PCDs, it will still be a decade or more before they are broadly available. In our work, we investigated an alternative solution that can be implemented on widely used conventional CT systems (single source and integrating detector) to perform multimaterial spectral decomposition for dual-contrast imaging.

Approach: We propose to slowly alternate the x-ray tube voltage between 3 kVp levels so each kVp level covers a few degrees of gantry rotation. This leads to the challenge of sparsely sampled projection data in each energy level. Performing the material decomposition (MD) in the sinogram domain is not directly possible as the projection images of the three energy levels are not angularly aligned. In order to overcome this challenge, we developed a convolutional neural network (CNN) framework for sparse sinogram completion (SC) and MD. To evaluate the feasibility of the slow kVp switching scheme, simulation studies of an abdominal phantom, which included liver lesions, were conducted.

Results: The line-integral SC network yielded sinograms with a pixel-wise RMSE  <  0.05 of the line-integrals compared to the ground truth. This provided acceptable image quality up to a switching angle of 9 deg per kVp. The MD network we developed allowed us to differentiate iodine and gadolinium in the sinogram domain. The average relative quantification errors for iodine and gadolinium were below 10%.

Conclusions: We developed a slow triple kVp switching data acquisition scheme and a CNN-based data processing pipeline. Results from a digital phantom validation illustrate the potential for future applications of dual-contrast agent protocols on practically available single-energy CT systems.

Purpose: Existing anomaly detection methods focus on detecting interclass variations while medical image novelty identification is more challenging in the presence of intraclass variations. For example, a model trained with normal chest x-ray and common lung abnormalities is expected to discover and flag idiopathic pulmonary fibrosis, which is a rare lung disease and unseen during training. The nuances of intraclass variations and lack of relevant training data in medical image analysis pose great challenges for existing anomaly detection methods.

Approach: We address the above challenges by proposing a hybrid model—transformation-based embedding learning for novelty detection (TEND), which combines the merits of classifier-based approach and AutoEncoder (AE)-based approach. Training TEND consists of two stages. In the first stage, we learn in-distribution embeddings with an AE via the unsupervised reconstruction. In the second stage, we learn a discriminative classifier to distinguish in-distribution data and the transformed counterparts. Additionally, we propose a margin-aware objective to pull in-distribution data in a hypersphere while pushing away the transformed data. Eventually, the weighted sum of class probability and the distance to margin constitutes the anomaly score.

Results: Extensive experiments are performed on three public medical image datasets with the one-vs-rest setup (namely one class as in-distribution data and the left as intraclass out-of-distribution data) and the rest-vs-one setup. Additional experiments on generated intraclass out-of-distribution data with unused transformations are implemented on the datasets. The quantitative results show competitive performance as compared to the state-of-the-art approaches. Provided qualitative examples further demonstrate the effectiveness of TEND.

Conclusion: Our anomaly detection model TEND can effectively identify the challenging intraclass out-of-distribution medical images in an unsupervised fashion. It can be applied to discover unseen medical image classes and serve as the abnormal data screening for downstream medical tasks. The corresponding code is available at https://github.com/XiaoyuanGuo/TEND_MedicalNoveltyDetection.

Purpose: Recent studies have demonstrated the diagnostic and prognostic values of global glomerulosclerosis (GGS) in IgA nephropathy, aging, and end-stage renal disease. However, the fine-grained quantitative analysis of multiple GGS subtypes (e.g., obsolescent, solidified, and disappearing glomerulosclerosis) is typically a resource extensive manual process. Very few automatic methods, if any, have been developed to bridge this gap for such analytics. We present a holistic pipeline to quantify GGS (with both detection and classification) from a whole slide image in a fully automatic manner. In addition, we conduct the fine-grained classification for the subtypes of GGS. Our study releases the open-source quantitative analytical tool for fine-grained GGS characterization while tackling the technical challenges in unbalanced classification and integrating detection and classification.

Approach: We present a deep learning-based framework to perform fine-grained detection and classification of GGS, with a hierarchical two-stage design. Moreover, we incorporate the state-of-the-art transfer learning techniques to achieve a more generalizable deep learning model for tackling the imbalanced distribution of our dataset. This way, we build a highly efficient WSI-to-results GGS characterization pipeline. Meanwhile, we investigated the largest fine-grained GGS cohort as of yet with 11,462 glomeruli and 10,619 nonglomeruli, which include 7841 globally sclerotic glomeruli of three distinct categories. With these data, we apply deep learning techniques to achieve (1) fine-grained GGS characterization, (2) GGS versus non-GGS classification, and (3) improved glomeruli detection results.

Results: For fine-grained GGS characterization, when pretrained on the larger dataset, our model can achieve a 0.778-macro-F1 score, compared to a 0.746-macro-F1 score when using the regular ImageNet-pretrained weights. On the external dataset, our best model achieves an area under the curve (AUC) score of 0.994 when tasked with differentiating GGS from normal glomeruli. Using our dataset, we are able to build algorithms that allow for fine-grained classification of glomeruli lesions and are robust to distribution shifts.

Conclusion: Our study showed that the proposed methods consistently improve the detection and fine-grained classification performance through both cross validation and external validation. Our code and pretrained models have been released for public use at https://github.com/luyuzhe111/glomeruli.

Purpose: The necessity of image retakes is initially determined on a preview monitor equipped with an operating system; therefore, some image blurring is only noticed later, on a high-resolution monitor. The purpose of this study is to investigate blur detection performance on radiographs via a deep learning approach compared with human observers.

Approach: A total of 99 radiographs (blurry 57, nonblurry 42) were independently observed and rated by six observers using preview and diagnostic liquid crystal displays (LCDs). The deep convolution neural network (DCNN) was trained and tested using ninefold cross-validation. The average areas under the ROC curves (AUCs) were calculated for each observer with LCDs and by stand-alone DCNN for each test session and then statistically tested using a 95% confidence interval.

Results: The average AUCs were 0.955 for stand-alone DCNN and 0.827 and 0.947 for human observers using preview and diagnostic LCDs, respectively. The DCNN revealed a high performance for image motion blur on digital radiographs (sensitivity 94.8%, specificity 96.8%, and accuracy 95.6%), along with the capability to detect a slight motion blur that was overlooked by human observers with a preview LCD. There were no cases of motion blur overlooked by the stand-alone DCNN, of which some were incorrectly recognized as nonblurry by human observers.

Conclusions: The deep learning-based approach was capable of distinguishing slight motion blur that was unnoticeable on a preview LCD, and thus, is expected to aid the human visual system for detecting blurred images in the initial review of digital radiographs.

Purpose: Motion artifacts in magnetic resonance (MR) images mostly undergo subjective evaluation, which is poorly reproducible, time consuming, and costly. Recently, full-reference image quality assessment (FR–IQA) metrics, such as structural similarity (SSIM), have been used, but they require a reference image and hence cannot be used to evaluate clinical images. We developed a convolutional neural network (CNN) model to quantify motion artifacts without using reference images.

Approach: The brain MR images were obtained from an open dataset. The motion-corrupted images were generated retrospectively, and the peak signal-to-noise ratio, cross-correlation coefficient, and SSIM were calculated. The CNN was trained using these images and their FR–IQA metrics to predict the FR–IQA metrics without reference images. Receiver operating characteristic (ROC) curves were created for binary classification, with artifact scores   <  4 indicating the need for rescanning. ROC curve analysis was performed on the binary classification of the real motion images.

Results: The predicted FR–IQA metric having the highest correlation with the subjective evaluation was SSIM, which was able to classify images requiring rescanning with a sensitivity of 89.5%, specificity of 78.2%, and area under the ROC curve (AUC) of 0.930. The real motion artifacts were classified with the AUC of 0.928.

Conclusions: Our CNN model predicts FR–IQA metrics with high accuracy, which enables quantitative assessment of motion artifacts in MR images without reference images. It enables classification of images requiring rescanning with a high AUC, which can improve the workflow of MR imaging examinations.

Purpose: To objectively assess new medical imaging technologies via computer-simulations, it is important to account for the variability in the ensemble of objects to be imaged. This source of variability can be described by stochastic object models (SOMs). It is generally desirable to establish SOMs from experimental imaging measurements acquired by use of a well-characterized imaging system, but this task has remained challenging.

Approach: A generative adversarial network (GAN)-based method that employs AmbientGANs with modern progressive or multiresolution training approaches is proposed. AmbientGANs established using the proposed training procedure are systematically validated in a controlled way using computer-simulated magnetic resonance imaging (MRI) data corresponding to a stylized imaging system. Emulated single-coil experimental MRI data are also employed to demonstrate the methods under less stylized conditions.

Results: The proposed AmbientGAN method can generate clean images when the imaging measurements are contaminated by measurement noise. When the imaging measurement data are incomplete, the proposed AmbientGAN can reliably learn the distribution of the measurement components of the objects.

Conclusions: Both visual examinations and quantitative analyses, including task-specific validations using the Hotelling observer, demonstrated that the proposed AmbientGAN method holds promise to establish realistic SOMs from imaging measurements.

Purpose: The coronary arteries are embedded in a layer of fat known as epicardial adipose tissue (EAT). The EAT influences the development of coronary artery disease (CAD), and increased EAT volume can be indicative of the presence and type of CAD. Identification of EAT using echocardiography is challenging and only sometimes feasible on the free wall of the right ventricle. We investigated the use of spectral analysis of the ultrasound radiofrequency (RF) backscatter for its potential to provide a more complete characterization of the EAT.

Approach: Autoregressive (AR) models facilitated analysis of the short-time signals and allowed tuning of the optimal order of the spectral estimation process. The spectra were normalized using a reference phantom and spectral features were computed from both normalized and non-normalized data. The features were used to train random forests for classification of EAT, myocardium, and blood.

Results: Using an AR order of 15 with the normalized data, a Monte Carlo cross validation yielded accuracies of 87.9% for EAT, 84.8% for myocardium, and 93.3% for blood in a database of 805 regions-of-interest. Youden’s index, the sum of sensitivity, and specificity minus 1 were 0.799, 0.755, and 0.933, respectively.

Conclusions: We demonstrated that spectral analysis of the raw RF signals may facilitate identification of the EAT when it may not otherwise be visible in traditional B-mode images.