1.

INTRODUCTION

Medical imaging is at an important crossroad. Computer-based (artificial intelligence (AI), deep learning (DL), machine learning (ML)) tools are increasing being developed for use with practically every type of medical data, especially images (e.g., radiology, pathology, dermatology, ophthalmology). The tasks these computer-based analysis tools are designed to do vary considerably and include but are not limited to lesion detection and classification, organ/feature segmentation, outcomes and survival prediction, feature/lesion measurement (size, count, volume), report generation and analysis, image quality enhancement, more efficient image acquisition, and workflow analysis. The goals vary as well but ultimately focus on improving healthcare outcomes through improved data analytics that the healthcare system (e.g., providers, technologists, physicists, schedulers, administrators) can utilize to improve the efficiency and efficacy of diagnoses, treatments and outcomes. Currently however, there are far more analytic schemes being developed than there are being implemented into clinical practice. What can we do to help accelerate clinical use of these tools to realize their full potential and really impact patient care?

The past 30 years has seen dramatic changes in radiology and pathology as advances and improvements in imaging acquisition, analysis, display and storage have occurred. Additionally, public expectations in response to these changes have changed, contributing to referring clinicians and patients expecting and often demanding expert interpretation of images and other medical data not only in major urban areas, but also in areas that are rural and medically underserved. One consequence of the demand for imaging and sub-specialty interpretation is that radiologists and pathologists more than ever are expected to provide service 24/7, requiring providers to be on-call after hours and on weekends. This has led to the development of protocols and software to enable bidirectional communication between physicians, technologists, imaging managers and patients. This is where AI and related tools can also have an impact in terms of improving the efficiencies of accessing and adding to electronic medical records, peer review interfaces, and dictation systems that eliminate manual interfaces (e.g., paper-based tools, non-voice activated/controlled dictation systems) and other tools that are not well suited to increased work demands.

In many respects radiology paved the way pathology with respect to going digital earlier, thus opening the door to AI, DL and ML development and use earlier. Radiology also has a longer history of conducting observer performance studies than does pathology, but that leads to a bit of a dilemma. Once of the core gold standards for radiology is pathology, assuming that the pathologist can provide the definitive “true” answer.

Numerous tools are available to screen and detect cancer, but tissue biopsy examination by pathologists is still the gold standard for the definitive diagnosis of cancer. This examination is typically performed by cutting thin slices of tissue followed by examination under the microscope, but this examination is far from an exact science and subject to significant variability. It is evident that clinical pathology variability and error is just as high in pathology as it is in radiology. For example, kappa values for assessing tumor grade in breast cancer is ~0.50.¹ A recent analysis on behalf of the International Ki67 working group (IKWG) showed high discordance rates (5-40%) between pathologists.² This is important clinically as Ki67 cutoff values of 20% are FDA recommended as a companion diagnostic for Abemaciclib. A recent ring study also documented poor concordance of 18 pathologists reading 170 breast cancer biopsies stained for HER2.³ Using a 4-point scale, they found only 26% concordance between 0 and 1+ compared with 58% concordance between 2+ and 3+ expression. These data clearly show subjectivity in pathology assessment and highlight the need for tools to assess and control subjectivity among pathologists even with light microscopy.

Advances in technology have led to digital scanning of tissue sections and screen-based examination of digital whole slide images (WSI), with increasing evidence, support, and use for primary diagnoses due to high diagnostic concordance rates.^4,5 Major concerns, however, include loss of quality during image acquisition, artifacts introduced by image handling, compression, and storage,^6,7 and determining the best and most efficient viewing strategies.⁸ WSIs have led to significant advances in AI tools for image segmentation and analysis^9-11 but even with explainable AI schemes “black boxes” are often trained by computers and engineers with minimal input from pathologists. There is significant concern regarding AI “trustworthiness”,^12-14 particularly in difficult cases and in cases where there is an admix of tumor and normal elements. There are few if any provisions for pathologists to understand the basis of the outputs provided, forcing them to seek resolution in situations where there is discrepancy between their perceptions and those of the AI.

One of the first steps to reduce diagnostic variability, improve training methods, and better integrate decision aids such as AI into the clinical routine is to understand the perceptual and cognitive factors underlying medical decision making.^15-17 Radiology studies have used eye-tracking technology for over 50 years to characterize search strategies, the development of expertise, and causes of error and variability as radiologists diagnose radiographic images (hardcopy film and digital softcopy).^18-20 Current efforts in AI development incorporate human observers and eye-tracking to inform steps such as automated image segmentation (Figure 1).^21-23 WSI makes it possible to conduct similar studies in pathology, but they have not included important comparisons to light microscopy since eye-tracking to date has not been readily feasible with traditional light microscopes,^24-32 although one study videotaped pathologists viewing glass slides³³ and we have been investigating tools to capture search patterns using light microscopes (Figure 2).

Figure 1.

Using eye tracking to identify lesion edges for image annotation and segmentation.

Figure 2.

A) Original WSI. B) Search pattern of pathologist examining WSI on computer monitor using eye tracking technology. C) Search pattern of the same pathologist viewing glass slide under light microscope.

1.1

Focus on the task

Creating an AI scheme that can achieve levels of performance equivalent to or even better than its intended users is only half the challenge. The ultimate goal is to translate tools into clinical use so it can aid clinical decision making, improve efficiency, and impact patient care. This requires a very different approach and set of skills. Implementation sciences, human factors, and an understanding of the perceptual and cognitive processes involved in clinical decision making are key to helping ensure the successful translation of AI into clinical use. A good first step to figuring out what types of tools would be of benefit in a particular clinical setting for a given set of medical data and task is to sit down with the stakeholders – the potential users – watch what they do in their daily routine and talk with them to identify their pain points. Where do they think computer-based assistance would help with their daily routine and in what ways? For example, in radiology it is necessary to identify vertebrae numerically and by type (thoracic, lumbar etc.) so they can be readily identified in the report (Figure 3). This task does not take much skill and only a couple of minutes to annotate on an image but by the end of a day of reading cases a radiologist has likely spent 30-60 minutes doing a tedious, repetitive task a computer could readily do automatically. Having an tool that automatically identifies and adds labels to the images relieves the radiologist of the burden and tedium of a task that does not require their advanced skills and gives them the additional 30-60 minutes to read additional cases or engage in other relevant tasks. In pathology a similarly tedious task that is subject to inter and intra-observer variation and takes valuable time that could be better spent is counting (Figure 4) cell nuclei (e.g., Ki-67 images).

Figure 3.

Example of a spine image with vertebra identified and numbered.

Figure 4.

Example of a Ki-67 pathology image in which the task is to count cell nuclei.

Once the goal has been established for the development process that does not mean the early stakeholders are no longer relevant. They should be consulted through the development process to ensure algorithm/scheme development stays on course with the original intent.

2.

SUMMARY

The future of healthcare clearly involves computer-based AI, DL and ML tools throughout the enterprise serving many different roles for various stakeholders. Imaging informatics grew out of and will continue to shape the future of radiology and pathology. Technology development and deployment are critical to improve patient care, health outcomes, and the efficacy and efficiency with which our healthcare systems achieve these goals, but it cannot take place without considering how it will be accepted and integrated in routine daily use by all stakeholders. User-centered methods, human factors, perception and cognition, implementation science, and related research frameworks should be used to help ensure successful translation of these tools into clinical use and can provide metrics with which success can be measured objectively.