Three-dimensional lesion segmentation is required for analysis of radiomic features and lesion growth kinetics. In clinical trials, radiologists apply the Response Evaluation Criteria in Solid Tumors (RECIST), by manually annotating the long and short diameters of a lesion on a single 2D axial slice (RECIST slice), where the lesion looks largest. We developed a novel approach that leverages the RECIST annotations to segment lesions in 3D on CT scans. We start with bounding box and center point prompts derived from RECIST long and short diameters on RECIST slice. Iteratively, we perform prompted segmentation using Segment Anything Model (SAM) on off-RECIST slices towards the superior and inferior direction until all slices are segmented. To optimize the performance of SAM, we fine-tuned the mask decoder. In addition, it is crucial to detect where the lesions disappear at the superior and inferior direction to prevent over segmentation. We developed a multi-task framework for lesion existence classification and segmentation and further compared the parallel framework and cascaded framework. We used an internal dataset consisting of 2053 and 200 3D lesions for fine-tuning of SAM decoder and testing, respectively. Baseline SAM, SAM with fine tuning, SAM with parallel multi-task fine tuning, and SAM with cascaded multitask fine tuning have Dice scores of 0.4745±0.2138, 0.7136±0.1277, 0.6985±0.1312, and 0.7239±0.1321, respectively. Our experiments proves that multi-task learning is an effective way for 3D segmentation with SAM, and cascaded framework performs better than parallel framework.
We are creating a cancer imaging and therapy analysis platform (CITAP), featuring image analysis/visualization software and multi-spectral cryo-imaging to support innovations in preclinical cancer research. Cryo-imaging repeatedly sections and tiles microscope images of the tissue block face, providing color anatomy and molecular fluorescence 3D microscopic imaging over vast volumes as large as a whole mouse, with single-metastatic-cell sensitivity. We utilized DenseVNet from NiftyNet for multi-organ segmentation on color anatomy images to further analyze major organs. The proposed algorithm was trained/validated/tested on 70/5/4 color anatomy volumes with manually labeled lung, liver, and spleen. The mean Dice similarity coefficient for lung, liver, and spleen in the test set were 0.89±0.01, 0.92±0.01, and 0.83±0.04. We deem Dice coefficient of <0.9 good for analyzing distribution of metastases. To segment GFP-labeled breast cancer metastases in high resolution green fluorescence images, big and small candidates were segmented using marker-based watershed and multi-scale Laplacian of Gaussian filtering followed by Otsu segmentation respectively. A bounding box around each candidate was classified with a 3D convolutional neural network (CNN). In one test mouse with 226 metastases, CNNbased classification and random forest with hand-crafted features achieved sensitivity/specificity of 0.95/0.89 and 0.92/0.82, respectively. DenseVNet-based organ segmentation allows automatic quantification of GFP-labeled metastases in each organ of interest. In the test mouse with 226 metastases, 78 (1 with size <2mm, 21 with size 0.5mm-2mm, and 56 with size <0.5mm) and 24 (1 with size <2mm, 11 with size 0.5-2mm, and 12 with size <0.5mm) were found in the lung and liver respectively.
In an effort to increase the efficiency and cure rate of nonmelanoma skin cancer (NMSC) excisions, we have developed a point-of-care method of imaging and evaluation of skin cancer margins. We evaluate the skin surgical specimens using a smart, near-infrared probe (6qcNIR) that fluoresces in the presence of cathepsin proteases overexpressed in NMSC. Imaging is done with an inverted, flying-spot fluorescence scanner that reduces scatter, giving a 70% improved step response as compared to a conventional imaging system. We develop a scheme for careful comparison of fluorescent signals to histological annotation, which involves image segmentation, fiducial-based registration, and nonrigid free-form deformation on fluorescence images, corresponding color images, “bread-loafed” tissue images, hematoxylin and eosin (H&E)-stained slides, and pathological annotations. From epidermal landmarks, spatial accuracy in the bulk of the sample is ∼500 μm, which when extrapolated with a linear stretch model, suggests an error at the margin of ∼100 μm, within clinical reporting standards. Cancer annotations on H&E slides are transformed and superimposed on the fluorescence images to generate the final results. Using this methodology, fluorescence cancer signals are generally found to correspond spatially with histological annotations. This method will allow us to accurately analyze molecular probes for imaging skin cancer margins.
KEYWORDS: Tumors, Magnetic resonance imaging, Image segmentation, Signal detection, Signal to noise ratio, Image registration, Green fluorescent protein, 3D image processing, Cancer
We created a cancer imaging and therapy platform (CITP) consisting of software and multi-spectral cryo-imaging to support innovations in preclinical cancer research. Cryo-imaging repeatedly sections and tiles microscope images of the tissue block face, providing anatomical episcopic color and molecular fluorescence, enabling 3D microscopic imaging of the entire mouse with single metastatic cell sensitivity. Our platform allows tumor molecular imaging validation with MRI and cryo images registration, GFP metastatic tumor segmentation and quantitative analysis, all of which are important processes in the CITP visualization/analysis pipeline. Our standard approach to register MRI to the cryo color volume involves preprocess Æ affine Æ B-spline non-rigid 3D mutual information registration. We further developed modified mask registration to allow improved registration quality within the created 3D cuboid mask on the organ of interest. In 3 mice kidneys, standard and mask registration yields Dice index of 84% ± 2% and 90% ± 2%, respectively. To segment big metastases in GFP, we use marker based watershed with intensity thresholding. For small metastases, we apply Laplacian of Gaussian filtering to get candidate metastases and use morphological features and support vector machine to classify the candidates. In a test mouse, sensitivity/specificity for metastases detection was 94.1%/99.82% as compared with manual segmentation of 202 metastases. Quantitative analysis of molecular MR imaging agent CREKA-Gd using Rose SNR in the lung of a test mouse showed that all micro-metastases ≥ 0.25 mm2 were detectable with Rose SNR ≥ 4 and around 36% of micro-metastases < 0.25 mm2 were detectable.
We have developed a point-of-care imaging method for non-melanoma skin cancer surgery whereby excised tissues are imaged with a smart near infrared quenched protease probe (6qcNIR) that fluoresces in the presence of overexpressed cathepsin proteases in basal cell carcinoma (BCC) and squamous cell carcinoma (SCC), and determine if margins are clear of cancer. Here we report our imaging system and our method to validate the detection of skin cancer. We imaged skin samples with an inverted, flying spot fluorescence scanner (LI-COR Odyssey CLx). Scatter in Odyssey system was greatly reduced giving an 80% improvement in the step response as compared to a previously used macroscopic imaging system with imaging of a fluorescence phantom. We developed a validation scheme for careful comparison of fluorescent cancer signal to histology annotation, involving image segmentation, fiducial based registration and non-rigid free-form deformation, using our LI-COR fluorescence images, corresponding color images, bread-loafed tissue images, H&E slides and pathologist annotation. Spatial accuracy in the bulk of the sample was ∼500 μm. Extrapolated with a linear stretch model suggests an error at the margin of <100 μm. Cancer annotations on H&E slides were transformed and superimposed on the probe fluorescence to generate the final result. In general, the fluorescence cancer signal corresponded with histological annotation.
KEYWORDS: Image registration, Magnetic resonance imaging, Imaging systems, 3D image processing, Luminescence, 3D acquisition, Tumors, Green fluorescent protein, Image resolution, Visualization, Cancer
We created a metastasis imaging, analysis platform consisting of software and multi-spectral cryo-imaging system suitable for evaluating emerging imaging agents targeting micro-metastatic tumor. We analyzed CREKA-Gd in MRI, followed by cryo-imaging which repeatedly sectioned and tiled microscope images of the tissue block face, providing anatomical bright field and molecular fluorescence, enabling 3D microscopic imaging of the entire mouse with single metastatic cell sensitivity. To register MRI volumes to the cryo bright field reference, we used our standard mutual information, non-rigid registration which proceeded: preprocess → affine → B-spline non-rigid 3D registration. In this report, we created two modified approaches: mask where we registered locally over a smaller rectangular solid, and sliding organ. Briefly, in sliding organ, we segmented the organ, registered the organ and body volumes separately and combined results. Though sliding organ required manual annotation, it provided the best result as a standard to measure other registration methods. Regularization parameters for standard and mask methods were optimized in a grid search. Evaluations consisted of DICE, and visual scoring of a checkerboard display. Standard had accuracy of 2 voxels in all regions except near the kidney, where there were 5 voxels sliding. After mask and sliding organ correction, kidneys sliding were within 2 voxels, and Dice overlap increased 4%–10% in mask compared to standard. Mask generated comparable results with sliding organ and allowed a semi-automatic process.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.