1.

INTRODUCTION

IN computed tomography (CT) imaging, different tissue types can be represented by the linear attenuation coefficients (LACs) [1]. Based on the well-established X-ray physics inside the human tissues [2], Fig. 1(a) shows the LACs as a function of the X-ray energy for four important human body tissues, i.e., bone, muscle, fat, and lung, and water as the reference. Fig. 1(b) shows the difference between these tissues in terms of CT values. Fig. 1(c) is a zoomed version of Fig. 1(b) to emphasize the differences among muscle, fat, lung and water. It is clearly seen that different tissue types have their energy responses along the energy axis, which we call tissue-energy response (TER) in this study. The different TERs shall bring more tissue contrast distribution information in a series of monoenergetic images than that of the single image reconstructed from the conventional CT, which uses a wide spectrum and could not take the response into consideration. More tissue contrast distribution information shall enhance tissue characterization and, therefore, improve lesion diagnosis. If the energy-independent fraction of each tissue can be obtained, a series of virtual monoenergetic images (VMIs) is able to be generated by the TER curve. Hence, photon-counting CT (PCCT) [3] and multiple energy spectral CT (MECT) [4-6] are recently developed to reconstruct the energy-independent fractions with multiple energy measurements and then generate a series of VMIs. However, they require the use of expensive photon counting detection technology and/or sophisticated image reconstruction methods. Following the physics behind the TER, this study explores an alternative approach to use the TER to generate a series of VMIs from a conventional CT image to further extract contrast textures for lesion diagnosis. An assumption is made that the conventional CT image is obtained from the effective energy of the energy spectrum used by the conventional CT. For example, if conventional CT operates on a 120kVp X-ray tube voltage, the effective photon energy would be 75keV. With the identified location in the TER curve, a series of VMIs are obtained by the relative factor along the energy axis.

Fig. 1:

Tissue energy response curve. (a) Log visualization for linear attenuation coefficients curve; (b) HU curve; (c) Zoomed HU curve.

With the above VMIs, a machine learning (ML)-based CADx model is proposed to differentiate malignant from benign lesions by exploiting the energy-enhanced material features, which uses automatically extracted features by deep learning (DL) technics. DL-based CADx algorithms, more specifically convolutional neural network (CNN), have achieved noticeable successes in the differentiation of malignant and benign lesions [7-11]. Due to the advanced feature learning power, a multi-channel 3D CNN-based CADx model is developed in this study to help recognize energy-specific features to differentiate malignant lesions from benign ones. The final classification result will reflect the CADx performance with the explored spectral information.

The remainder of this paper is organized as follows. Section II will describe the proposed computer-aided diagnosis framework and the overall workflow. Section III presents the experiment design and results. Discussion and conclusions are drawn in Sections IV and V.

2.

METHODS

An overall illustration of our proposed lesion diagnosis pipeline can be found in Fig.2. Fig.2(a) shows the traditional diagnosis pipeline, the conventional CT data is directly used in the ML-based CADx model. As shown in Fig.2(b), our proposed framework is composed of two components: one is virtual monoenergetic images (VMIs) generation by the use of TER and the other is ML-based CADx model for classification. The details of each are described as follows.

Fig. 2:

Illustration and comparison between traditional and our proposed method for lesion diagnosis. (a) Traditional ML-based CADx pipeline. (b) Our proposed ML-based CADx pipeline with the VMIs generated by the use of TER.

2.1

CT Image-based Virtual Monoenergetic Images Generation by the Use of TER

In a CT image, a linear attenuation coefficient can be represented by R types of tissues, e.g. muscle, fat, lung, bone, and so on. The LAC function μ_j(ε) at the j^th pixel of the image is decomposed as:

where μ_r(ε) denotes linear attenuation coefficient of tissue r at energy ε, as shown in Fig.1(a). Notation f_rj is a unitless tissue fraction that quantifies the contribution of tissue r to attenuation in pixel j. Once the tissue fractions are identified, a series of VMIs can be generated.

• 1) Region of interest (ROI) selection

  For lesion diagnosis, an ROI containing the lesion is firstly selected from the whole CT image. An example of ROI selection from one patient slice for lung nodule diagnosis is shown in Fig. 3.

• 2) Tissue fractions identification

  Due to the polychromatic X-ray source, a CT ROI image μ^E₁ reconstructed by filtered backprojection (FBP) method reflects the attenuation coefficients at the effective energy with the tube voltage E₁ kVp as follows,

  

  where denotes the effective energy the selected tube voltage E₁. To identify the tissue fractions, the CT image is first segmented into four tissue types including lung tissue, fat, bone and muscle by a threshold method. And then, for each tissue region, the tissue fraction f_rj can be obtained by In this work, we assumed when a conventional CT scan at E₁ = 120 kVp. And can be directly found with TER as shown in Fig.1(a).

• 3) Virtual monoenergetic CT images generation

  Based on the identified tissue fractions, a series of VMIs at selected n energies are generated with the corresponding tissue LACs as follows.

  

Fig. 3:

An example of ROI selection from the CT image for lung nodule diagnosis. The display window is [0, 0.35] cm^-1.

In this work, n = 10, the energy values are used by 5, 8, 10, 12, 15, 20, 25, 30, 35, 40 and 45 keV as an example. It is because that as shown in Fig. 1(c), the HU values of different tissues have the maximum differences in this energy range and the contrast features could be efficiently enhanced.

2.2

Machine Learning (ML)-based CADx

4) Deep learning (DL)-based CNN model:

For the DL-based model, a 3D CNN architecture with a multi-channel input is developed, which uses each energy image as one input channel as shown in Fig. 4. First, four convolutional layers are used to extract the features, which are then pooled together with a global average pooling (GAP3D) layer such that the final features have a global receptive field. At last, a fully connected (FC) layer-based classifier is adopted to distinguish the malignant and benign lesions. Specifically, each convolutional layer consists of three operators: 3D convolution (Conv3D), 3D average pooling (Avgpolling3D), and rectified linear unit (ReLU). And the binary cross-entropy loss is utilized to train the model. The details of the CNN model are listed in Table I.

Fig. 4:

The 3D CNN architecture of CADx for lesion diagnosis.

TABLE I:

DETAILS OF NETWORK DESIGN

Layer	Type	Channels	Kernel Size	Padding	Stride	Activation
1	Conv3D	32	(7,7,7)	(1,1,1)	(1,1,1)	ReLU
2	Avgpooling3D	-	(2,2,2)	-	(2,2,2)	-
3	Conv3D	64	(5,5,5)	(1,1,1)	(1,1,1)	ReLU
4	Avgpooling3D	-	(2,2,2)	-	(2,2,2)	-
5	Conv3D	128	(3,3,3)	(1,1,1)	(1,1,1)	ReLU
6	Avgpooling3D	-	(2,2,2)	-	(2,2,2)	-
7	Conv3D	128	(3,3,3)	(1,1,1)	(1,1,1)	ReLU
8	Avgpooling3D	-	(2,2,2)	-	(2,2,2)	-
9	GAP3D	-	-	-	-	-
10	FC	128	-	-	-	-

Based on the above model, the final classification results will show the lesion diagnosis performance with the VMIs generated by the TER.

3.

EXPERIMENTS AND RESULTS

4.

DISCUSSION

This study aims at exploring the energy spectral information from a conventional CT image by using the TER to generate VMIs for lesion diagnosis. The proposed CADx framework with the VMIs achieved improved diagnosis performance than the traditional CADx pipeline with the conventional CT data. As we have demonstrated in this work, the VMIs at different energy bins show significant effectiveness for lesion characterization. How to choose the energy range and energy number of the VMIs to capture the meaningful lesion features is still a promising direction [13]. With the analysis from the above, this work will also have a great potential in guiding the energy selection in PCCT imaging for diagnosis. Meanwhile, to compare the PCCT images from the practical measurements with the VMIs by the use of TER from the conventional CT would be a very interesting research topic in the future. And last but not least, clinical evaluations with more disease data sets are needed to test the robustness of the proposed method.

5.

CONCLUSIONS

In conclusion, we proposed to use tissue-energy response to generate the virtual monoenergetic images from the conventional CT for CADx of lesions. In this framework, each tissue contrast distribution along the energy axis is fully enhanced, which brings richer information to ML-based CADx. Experimental results demonstrated that the VMIs generated with the use of TER from the conventional CT applied to the ML-based CADx obtain better performance than the traditional CADx pipeline in lesion classification.