Pulmonary sarcoidosis (PS) is an inflammatory interstitial lung disease, causing clusters of inflamed tissue called ‘granulomas’ within the lung. However, PS may mimic other conditions such as malignancy or infection, which often leads to a delayed diagnosis, leading to worsening respiratory function. Chest computed tomography (CT) is used to diagnose PS but with varying specificities, as the appearance of PS and other diffuse lung diseases on chest CT are myriad and considered difficult to diagnose by many radiologists. In this work we used a radiomics-guided ensemble of 3D CNN and Vision Transformers (ViT) features (Rad-CNNViT) to classify between PS and other interstitial lung diseases (o-ILDs). The input to the network was the 3D radiomics map that received the top feature score in a Random Forest (RF)-based feature selection approach. The input map was then fed into an ensemble network of CNN and ViT to capture local and global features respectively for diffuse lung disease classification. Training datasets of PS (n=61) and o-ILD chest CTs (n=154) were used for feature discovery using RF and train the Rad-CNNViT and a radiomics-based machine learning (Rad-ML) framework. On a separate test cohort of PS (n=65) and o-ILD (n=96), Rad-CNNViT ensemble network differentiated PS from o-ILDs with higher AUC=0.89 compared to a Rad-ML approach with AUC=0.77.
Model observers designed to predict human observers in detection tasks are important tools for assessing task-based image quality and optimizing imaging systems, protocol, and reconstruction algorithms. Linear model observers have been widely studied to predict human detection performance, and recently, deep learning model observers (DLMOs) have been developed to improve the prediction accuracy. Most existing DLMOs utilize convolutional neural network (CNN) architectures, which are capable of learning local features while not good at extracting long-distance relations in images. To further improve the performance of CNN-based DLMOs, we investigate a hybrid CNN-Swin Transformer (CNN-SwinT) network as DLMO for PET lesion detection. The hybrid network combines CNN and SwinT encoders, which can capture both local information and global context. We trained the hybrid network on the responses of 8 human observers including 4 radiologists in a two-alternative forced choice (2AFC) experiment with PET images generated by adding simulated lesions to clinical data. We conducted a 9-fold cross-validation experiment to evaluate the proposed hybrid DLMO, compared to conventional linear model observers such as a channelized Hotelling observer (CHO) and a non-prewhitening matched filter (NPWMF). The hybrid CNN-SwinT DLMO predicted human observer responses more accurately than the linear model observers and DLMO with only the CNN encoder. This work demonstrates that the proposed hybrid CNN-SwinT DLMO has the potential as an improved tool for task-based image quality assessment.
Image guidance aids neurosurgeons in making critical clinical decisions of safe maximal resection of diseased tissue. The brain however undergoes significant non-linear structural deformation on account of dura opening and tumor resection. Deformable registration of pre-operative ultrasound to intra-operative ultrasound may be used in mapping of pre-operative planning MRI to intraoperative ultrasound. Such mapping may aid in determining tumor resection margins during surgery. In this work, brain structures visible in pre- and intra-operative 3D ultrasound were used for automatic deformable registration. A Gaussian mixture model was used to automatically segment structures of interest in pre- and intra-operative ultrasound and patch-based normalized cross-correlation was used to establish correspondences between segmented structures. An affine registration based on correspondences was followed by B-spline based deformable registration to register pre- and intra-operative ultrasound. Manually labelled landmarks in pre- and intra-operative ultrasound were used to quantify the mean target registration error. We achieve a mean target registration error of 1.43±0.8 mm when validated with 17 pre- and intra-operative ultrasound image volumes of a public dataset.
This work explores a Generative Adversarial Network (GAN) based approach for hemorrhage detection on color Doppler ultrasound images of blood vessels. Given the challenges of collecting hemorrhage data and the inherent pathology variability, we investigate an unsupervised anomaly detection network which learns a manifold of normal blood flow variability and subsequently identifies anomalous flow patterns that fall outside the learned manifold. As an initial, feasibility study, we collected ultrasound color Doppler images of brachial arteries from 11 healthy volunteers. The images were pre-processed to mask out velocities in surrounding tissues and were subsequently cropped, resized, augmented and normalized. The network was trained on 1530 images from 8 healthy volunteers and tested on 70 images from 2 healthy volunteers. In addition, the network was tested on 6 synthetic images generated to simulate blood flow velocity patterns at the site of hemorrhage. Results show significant (p<0.05) differences in anomaly scores of normal arteries and simulated injured arteries. The residual images, or the reconstruction error maps, show promise in localizing anomalies at pixel level.
KEYWORDS: Image registration, 3D modeling, Ultrasonography, Liver, Motion models, Magnetic resonance imaging, 3D image processing, Computer programming, Data acquisition, 3D acquisition
Image fusion-guided interventions often require planning MR/CT and interventional U/S images to be registered in realtime. Organ motion, patient breathing and inconsistent ultrasound probe positioning during intervention, all contribute to the challenges of real-time 3D deformable registration, where alignment accuracy and computation time are often mutual trade-offs. In this work, we propose a novel framework to align planning and interventional 3D U/S by training patientspecific deep-deformation models (PsDDM) at the planning stage. During intervention, planning 3D U/S volumes are efficiently warped onto the interventional 3D U/S volumes using the trained deep-deformation model, thus enabling the transfer of other modality (planning MR/CT) information in real-time on interventional images. The alignment of planning MR/CT to planning U/S is not time-critical as these can be aligned before the intervention with desired accuracy using any known multimodal deformable registration method. The feasibility of training PsDDM is shown on liver U/S data acquired with a custom-built MR-compatible, hands-free 3D ultrasound probe that allows simultaneous acquisition of planning MR and U/S. Liver U/S volumes exhibit large motion in time due to respiration and therefore serve as a good anatomy to quantify the accuracy of the PsDDM. For quantitative evaluation of the PsDDM, a large vessel bifurcation was manually annotated on 9 U/S volumes that were not used for training the PsDDM but from the same subject. Mean target registration error (TRE) between the centroids was 0.84mm ± 0.39mm, mean Hausdorff distance (HD) was 1.80mm ± 0.29mm and mean surface distance (MSD) was 0.44mm ± 0.06mm for all volumes. In another experiment, the PsDDM was trained using liver volumes from one scanning session, while the model was tested on data from a separate scanning session of the same patient, for which qualitative alignment results were presented.
This paper presents a method based on shape-context and statistical measures to match interventional 2D Trans
Rectal Ultrasound (TRUS) slice during prostate biopsy to a 2D Magnetic Resonance (MR) slice of a pre-acquired
prostate volume. Accurate biopsy tissue sampling requires translation of the MR slice information on the TRUS
guided biopsy slice. However, this translation or fusion requires the knowledge of the spatial position of the
TRUS slice and this is only possible with the use of an electro-magnetic (EM) tracker attached to the TRUS
probe. Since, the use of EM tracker is not common in clinical practice and 3D TRUS is not used during biopsy,
we propose to perform an analysis based on shape and information theory to reach close enough to the actual
MR slice as validated by experts. The Bhattacharyya distance is used to find point correspondences between
shape-context representations of the prostate contours. Thereafter, Chi-square distance is used to find out those
MR slices where the prostates closely match with that of the TRUS slice. Normalized Mutual Information (NMI)
values of the TRUS slice with each of the axial MR slices are computed after rigid alignment and consecutively a
strategic elimination based on a set of rules between the Chi-square distances and the NMI leads to the required
MR slice. We validated our method for TRUS axial slices of 15 patients, of which 11 results matched at least
one experts validation and the remaining 4 are at most one slice away from the expert validations.
KEYWORDS: 3D modeling, Image segmentation, Prostate, Data modeling, Magnetic resonance imaging, Image registration, 3D image processing, Statistical modeling, Affine motion model, Principal component analysis
Real-time fusion of Magnetic Resonance (MR) and Trans Rectal Ultra Sound (TRUS) images aid in the localization
of malignant tissues in TRUS guided prostate biopsy. Registration performed on segmented contours of
the prostate reduces computational complexity and improves the multimodal registration accuracy. However,
accurate and computationally efficient 3D segmentation of the prostate in MR images could be a challenging
task due to inter-patient shape and intensity variability of the prostate gland. In this work, we propose to
use multiple statistical shape and appearance models to segment the prostate in 2D and a global registration
framework to impose shape restriction in 3D. Multiple mean parametric models of the shape and appearance
corresponding to the apex, central and base regions of the prostate gland are derived from principal component
analysis (PCA) of prior shape and intensity information of the prostate from the training data. The estimated
parameters are then modified with the prior knowledge of the optimization space to achieve segmentation in 2D.
The 2D segmented slices are then rigidly registered with the average 3D model produced by affine registration
of the ground truth of the training datasets to minimize pose variations and impose 3D shape restriction. The
proposed method achieves a mean Dice similarity coefficient (DSC) value of 0.88±0.11, and mean Hausdorff
distance (HD) of 3.38±2.81 mm when validated with 15 prostate volumes of a public dataset in leave-one-out
validation framework. The results achieved are better compared to some of the works in the literature.
This paper provides a comparison of spline-based registration methods applied to register interventional Trans Rectal Ultrasound (TRUS) and pre-acquired Magnetic Resonance (MR) prostate images for needle guided prostate biopsy. B-splines and Thin-plate Splines (TPS) are the most prevalent spline-based approaches to achieve deformable registration. Pertaining to the strategic selection of correspondences for the TPS registration, we use an automatic method already proposed in our previous work to generate correspondences in the MR and US prostate images. The method exploits the prostate geometry with the principal components of the segmented prostate as the underlying framework and involves a triangulation approach. The correspondences are generated with successive refinements and Normalized Mutual Information (NMI) is employed to determine the
optimal number of correspondences required to achieve TPS registration. B-spline registration with successive grid refinements are consecutively applied for a significant comparison of the impact of the strategically chosen correspondences on the TPS registration against the uniform B-spline control grids. The experimental results
are validated on 4 patient datasets. Dice Similarity Coefficient (DSC) is used as a measure of the registration accuracy. Average DSC values of 0.97±0.01 and 0.95±0.03 are achieved for the TPS and B-spline registrations respectively. B-spline registration is observed to be more computationally expensive than the TPS registration
with average execution times of 128.09 ± 21.7 seconds and
62.83 ± 32.77 seconds respectively for images with maximum width of 264 pixels and a maximum height of 211 pixels.
This paper proposes a method of quantification of the components underlying the human skin that are supposed
to be responsible for the effective reflectance spectrum of the skin over the visible wavelength. The method is
based on independent component analysis assuming that the epidermal melanin and the dermal haemoglobin
absorbance spectra are independent of each other. The method extracts the source spectra that correspond to the
ideal absorbance spectra of melanin and haemoglobin. The noisy melanin spectrum is fixed using a polynomial
fit and the quantifications associated with it are reestimated. The results produce feasible quantifications of each
source component in the examined skin patch.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.