Emerging uses for extended-reality (XR) head-mounted displays (HMDs) within medical environments include visualizations of medical data across various imaging modalities including radiography, computed tomography, ultrasound, and magnetic resonance images. Rendering medical data in XR environments requires real-time updates to account for user movement within the environment. Unlike stationary 2D medical displays, XR HMDs also require real-time stereoscopic rendering capabilities with high performance graphics processing units. Furthermore, performance depends on the status of added systems including tracking sensor technology, user's input data, and in the case of augmented reality (AR), spatial mapping and image registration. These temporal considerations have implications for the interpretation of medical data. However, methods for the evaluation of their effects on image quality are not yet well defined. The definition of these effects in the context of medical XR devices is at best inconsistent if not completely lacking. In this work, we compare the effects and causes for three classes of XR spatiotemporal characteristics affecting medical image quality: temporal artifacts, luminance artifacts, and spatial mapping artifacts. We describe the XR system components starting from user movement recognized by inertial measurement unit and camera sensors and ending with user perception of the display through the optics of the HMD. We summarize our findings and highlight device performance areas contributing to the different effects.
Color normalization is one of the pre-processing steps employed by many deep learning-based algorithms used for aiding pathology diagnoses with whole-slide images. Due to variability in tissue type, specimen preparation, staining protocol, and scanner performance, whole-slide images acquired from different sources may exhibit pronounced color variability that hinders algorithms from executing effectively. In the literature, numerous methods have been proposed to colornormalize hematoxylin and eosin (H&E)-stained images. However, the objective of color normalization has not been colorimetrically defined or evaluated beyond visual comparison. In this study, a quantitative metric, color normality, was defined to evaluate the degree of color similarity between images involved in a color normalization process. The pixelwise spectral data of eight H&E-stained tissue slides were optically measured as the ground truth to test the Reinhard, Macenko, and Vahadane methods. Principal component analysis was conducted on the spectral data to derive a new color normalization method as the reference. Experiment results show that the H&E color gamut needs to be expressed with three components, but the widely used Macenko and Vahadane methods compressed the three-dimensional color gamut volume into a two-dimensional surface and reduced color gamut volumes by 40% or more. None of the color normalization methods could achieve a color normality of greater than 0.6174 when the image was not self-normalized.
The color rendering of whole-slide images (WSIs) depends on factors involving the sample, such as tissue type, preparation methods, staining type and staining protocol, as well as equipment, such as the WSI scanner, WSI viewer, and WSI display. Variations in any of these steps may change the color rendering and therefore affect the performance of pathologists in the interpretation of WSIs and the robustness of artificial intelligence algorithms. In the literature, color normalization techniques have been proposed to reduce the color variations. The purpose of this work is to develop an objective approach to characterizing color normalization methods used in digital pathology. We employed color normalization methods to normalize the color rendered by a WSI scanner and then compared the normalized color with the actual scan by that scanner. The normalization errors were evaluated on the pixel level using the CIE color difference ΔE metric that have been shown to correlate with visually perceived differences in human vision. A selected set of 310 patch images of breast tissues scanned by two scanners from the ICPR 2014 MITOS & ATYPIA contest was used. Images from one scanner were color normalized to match the color rendering of the other scanner. Four color normalization methods were compared – Macenko, Reinhard, Vahadane, and StainGAN. Experimental results show that average color differences between two scanners in terms of ΔE were reduced from 16.2 before normalization to the range of [13.7,16.9] after normalization for the Macenko, Reinhard, Vahadane methods, and to 8.3 for the StainGAN method. Apparently the StainGAN method is significantly superior to the other three methods in terms of the ΔE metric. As such, we demonstrated a quantitative method for objectively evaluating color normalization techniques. Future work is needed to explore the relationship of the color fidelity measure and the impact of color normalization on pathologist and AI performance in clinical tasks.
KEYWORDS: Pathology, Kidney, Human vision and color perception, Color reproduction, Tissues, Imaging devices, Multispectral imaging, Color difference, Transmittance, Colon, Skin, Optical microscopes, Light sources
The color reproducibility of two whole-slide imaging (WSI) devices was evaluated with biological tissue slides.
Three tissue slides (human colon, skin, and kidney) were used to test a modern and a legacy WSI devices. The
color truth of the tissue slides was obtained using a multispectral imaging system. The output WSI images were
compared with the color truth to calculate the color difference for each pixel. A psychophysical experiment was
also conducted to measure the perceptual color reproducibility (PCR) of the same slides with four subjects. The
experiment results show that the mean color differences of the modern, legacy, and monochrome WSI devices are
10.94±4.19, 22.35±8.99, and 42.74±2.96 ▵E00, while their mean PCRs are 70.35±7.64%, 23.06±14.68%, and
0.91±1.01%, respectively.
With improved diagnostic capabilities and complex optical designs, endoscopic technologies are advancing. As one of the several important optical performance characteristics, geometric distortion can negatively affect size estimation and feature identification related diagnosis. Therefore, a quantitative and simple distortion evaluation method is imperative for both the endoscopic industry and the medical device regulatory agent. However, no such method is available yet. While the image correction techniques are rather mature, they heavily depend on computational power to process multidimensional image data based on complex mathematical model, i.e., difficult to understand. Some commonly used distortion evaluation methods, such as the picture height distortion (DPH) or radial distortion (DRAD), are either too simple to accurately describe the distortion or subject to the error of deriving a reference image. We developed the basic local magnification (ML) method to evaluate endoscope distortion. Based on the method, we also developed ways to calculate DPH and DRAD. The method overcomes the aforementioned limitations, has clear physical meaning in the whole field of view, and can facilitate lesion size estimation during diagnosis. Most importantly, the method can facilitate endoscopic technology to market and potentially be adopted in an international endoscope standard.
Endoscopy is a well-established paradigm in medical imaging, and emerging endoscopic technologies such as high resolution, capsule and disposable endoscopes promise significant improvements in effectiveness, as well as patient safety and acceptance of endoscopy. However, the field lacks practical standardized test methods to evaluate key optical performance characteristics (OPCs), in particular the geometric distortion caused by fisheye lens effects in clinical endoscopic systems. As a result, it has been difficult to evaluate an endoscope’s image quality or assess its changes over time. The goal of this work was to identify optimal techniques for objective, quantitative characterization of distortion that are effective and not burdensome. Specifically, distortion measurements from a commercially available distortion evaluation/correction software package were compared with a custom algorithm based on a local magnification (ML) approach. Measurements were performed using a clinical gastroscope to image square grid targets. Recorded images were analyzed with the ML approach and the commercial software where the results were used to obtain corrected images. Corrected images based on the ML approach and the software were compared. The study showed that the ML method could assess distortion patterns more accurately than the commercial software. Overall, the development of standardized test methods for characterizing distortion and other OPCs will facilitate development, clinical translation, manufacturing quality and assurance of performance during clinical use of endoscopic technologies.
We present a platform for designing and executing studies that compare pathologists interpreting histopathology of whole slide images (WSIs) on a computer display to pathologists interpreting glass slides on an optical microscope. eeDAP is an evaluation environment for digital and analog pathology. The key element in eeDAP is the registration of the WSI to the glass slide. Registration is accomplished through computer control of the microscope stage and a camera mounted on the microscope that acquires real-time images of the microscope field of view (FOV). Registration allows for the evaluation of the same regions of interest (ROIs) in both domains. This can reduce or eliminate disagreements that arise from pathologists interpreting different areas and focuses on the comparison of image quality. We reduced the pathologist interpretation area from an entire glass slide (10 to 30 mm2) to small ROIs (<50 μm2). We also made possible the evaluation of individual cells. We summarize eeDAP’s software and hardware and provide calculations and corresponding images of the microscope FOV and the ROIs extracted from the WSIs. The eeDAP software can be downloaded from the Google code website (project: eeDAP) as a MATLAB source or as a precompiled stand-alone license-free application.
Purpose: The purpose of this work is to present a platform for designing and executing studies that compare pathologists interpreting histopathology of whole slide images (WSI) on a computer display to pathologists interpreting glass slides on an optical microscope. Methods: Here we present eeDAP, an evaluation environment for digital and analog pathology. The key element in eeDAP is the registration of theWSI to the glass slide. Registration is accomplished through computer control of the microscope stage and a camera mounted on the microscope that acquires images of the real time microscope view. Registration allows for the evaluation of the same regions of interest (ROIs) in both domains. This can reduce or eliminate disagreements that arise from pathologists interpreting different areas and focuses the comparison on image quality. Results: We reduced the pathologist interpretation area from an entire glass slide (≈10-30 mm)2 to small ROIs <(50 um)2. We also made possible the evaluation of individual cells. Conclusions: We summarize eeDAP’s software and hardware and provide calculations and corresponding images of the microscope field of view and the ROIs extracted from the WSIs. These calculations help provide a sense of eeDAP’s functionality and operating principles, while the images provide a sense of the look and feel of studies that can be conducted in the digital and analog domains. The eeDAP software can be downloaded from code.google.com (project: eeDAP) as Matlab source or as a precompiled stand-alone license-free application.
Technological advances in endoscopes, such as capsule, ultrathin and disposable devices, promise significant
improvements in safety, clinical effectiveness and patient acceptance. Unfortunately, the industry lacks test methods for
preclinical evaluation of key optical performance characteristics (OPCs) of endoscopic devices that are quantitative,
objective and well-validated. As a result, it is difficult for researchers and developers to compare image quality and
evaluate equivalence to, or improvement upon, prior technologies. While endoscope OPCs include resolution, field of
view, and depth of field, among others, our focus in this paper is geometric image distortion. We reviewed specific test
methods for distortion and then developed an objective, quantitative test method based on well-defined experimental and
data processing steps to evaluate radial distortion in the full field of view of an endoscopic imaging system. Our
measurements and analyses showed that a second-degree polynomial equation could well describe the radial distortion
curve of a traditional endoscope. The distortion evaluation method was effective for correcting the image and can be
used to explain other widely accepted evaluation methods such as picture height distortion. Development of consensus
standards based on promising test methods for image quality assessment, such as the method studied here, will facilitate
clinical implementation of innovative endoscopic devices.
KEYWORDS: Scanners, Color management, Transmittance, Color difference, Light sources, Imaging systems, Digital imaging, Photography, Glasses, Field programmable gate arrays
A new method for assessing color reproducibility of whole-slide imaging (WSI) systems is introduced. A color phantom
is used to evaluate the difference between the input to and the output from a WSI system. The method consists of four components: (a) producing the color phantom, (b) establishing the truth of the color phantom, (c) retrieving the digital display data from the WSI system, and (d) calculating the color difference. The method was applied to a WSI system and used to evaluate the color characteristics with and without color management.
Routine color calibration is imperative for medical applications that rely on color fidelity such as digital pathology,
endoscopy, and colposcopy. However, commercially available products vary greatly in price and performance with no
available evaluation standard. Related studies have used only one or a few displays to evaluate the performance of color
calibration kits. We propose the concept of Virtual Display, a universal display platform that emulates the colorimetric
response of real displays. A wide-color-gamut display driven by an FPGA is used to emulate the colorimetric response of
real display devices. By changing the look-up tables in the FPGA, the virtual display emulates various real displays for
testing different color calibration kits. Our experimental data for 6 real displays show that the virtual display can emulate
real displays reasonably well. The results demonstrate that the proposed virtual display approach is a fast, economical,
and objective method for evaluating the performance of color calibration kits.
We studied the impact of the microscope light source on reader's performance using a microscopic version of the
Farnsworth-Munsell 100 hue test for photographic slide film. Each pair of two adjacent color caps in the original test kit
was reproduced on the film with random order and a 5X objective was used to examine the microscopic color patterns.
The subject's visual task was to determine whether the color pair was in the correct hue order or not. The test was
repeated for both a light-emitting diode lamp and a conventional halogen lamp. In this paper, we discuss the
methodology using preliminary results.
The grayscale resolution of current liquid crystal display technology limits its applications in medical imaging
with wide dynamic range and dense grayscales are required. We propose an approach that dynamically
processes the display image such that the luminance and contrast of the gazed area is optimized. A gazecontingent
interactive display system based on an 8-bit LCD and an eye-tracker was implemented to emulate
the proposed concept for a high-dynamic range display.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.