Classification of Liver lesions by Using CNN in CT Images

Abstract—

Deep learning is a general term for dealing with a complicated neural network with multiple layers. Convolutional Neural Networks (CNN) is a type of deep neural network, and there are many other types. CNNs are popular because they have very useful applications to image recognition and have demonstrated a great achievement in image classification. The idea behind this paper is to propose a framework that is able to classify liver lesions of Metastases and Hemangiomas by using Convolutional neural networks. We used dataset from Jiangsu University hospital with 154 liver lesions. We evaluated the performance of our proposed model and we compared AlexNet with our model to determine the performance and the classification strength. The classification accuracy for our model was 87.75%, an F1 score of 0.90 and an MCC 0.51. We have found our model much better than AlexNet.

Index Terms— Deep learning, Convolutional Neural Networks, Liver lesions, Classification.

1. INTRODUCTION

THE liver is a large, meaty organ that sits on the right side of the belly. Weighing about 3 pounds, the liver is reddish-brown in color and feels rubbery to the touch. Normally you can’t feel the liver, because it’s protected by the rib cage.

Metastasis means that cancer spreads to a different body part from where it started. When this happens, doctors say the cancer has “metastasized.” Your doctor may also call it “metastatic cancer,” “advanced cancer,” or “stage 4 cancer.” But these terms can have different meanings. For example, a cancer that is large but has not spread to another body part can also be called advanced cancer or locally advanced cancer.

Metastases are the plural form of metastasis. Metastases most commonly develop when cancer cells break away from the main tumor and enter the bloodstream or lymphatic system. These systems carry fluids around the body. This means that the cancer cells can travel far from the original tumor and form new tumors when they settle and grow in a different part of the body.

Hemangioma is a noncancerous (benign) mass in the liver. A liver hemangioma is made up of a tangle of blood vessels. Other terms for a liver hemangioma are hepatic hemangioma and cavernous hemangioma. Most cases of liver hemangiomas are discovered during a test or procedure for some other condition. People who have a liver hemangioma rarely experience signs and symptoms and don’t need treatment.

In this paper, we propose a framework to use deep features extracted from Convolutional Neural Networks (CNNs) along with a Softmax as a classifier to build our model for liver lesions classification. In medical imaging tasks, annotations are made by radiologists with expert knowledge on the data and task.

Also, these small lesions cannot be detected by X-rays and are only detectable by a CT scan. Even after the detection, it takes a considerable amount of effort and experience on the part of radiologists to detect and label the nodules as benign or as a probable case of malignancy. Considering the large number of cases encountered by radiologists every day there is a constant pressure on them to analyse a huge amount of data and make a decision as quickly as possible based on the analysis.

Figure1. a. Dataset examples of metastases and hemangiomas liver lesions

Figure1. b. Input lesion ROI examples, ROI extraction process from a 2-D CT slice of the liver.

AlexNet is the name of a convolutional neural network which has had a large impact on the field of machine learning, specifically in the application of deep learning to machine vision. It famously won the 2012 ImageNet LSVRC-2012 competition by a large margin (15.3% VS 26.2% (second place) error rates). The network had a very similar architecture as LeNet by Yann LeCun et al but was deeper, with more filters per layer, and with stacked convolutional layers. It consisted of 11×11, 5×5, 3×3, convolutions, max pooling, dropout, data augmentation, ReLU activations, SGD with momentum. It attached ReLU activations after every convolutional and fully-connected layer. AlexNet was trained for 6 days simultaneously on two Nvidia Geforce GTX 580 GPUs which is the reason for why their network is split into two pipelines (..).

2. RELATED WORK

Convolutional Neural Networks (CNNs) have been demonstrated as very powerful techniques in broad range of tasks and in various fields of studies such as computer vision, language processing, image processing, and medical image analysis.

Recently, several methods have been proposed to classify liver lesions. In…the authors presented methods for generating synthetic medical images

using recently presented deep learning Generative Adversarial Networks (GANs). The classification performance using only classic data augmentation yielded 78.6% sensitivity and 88.4% specificity. By adding the synthetic data augmentation, the results increased to 85.7% sensitivity and 92.4% specificity.

In… the authors developed a method to distinguish cyst from metastases and they tested different models by using Convolutional Neural Networks (CNNs) yielded accuracy of 84%. They used deep learning to classify liver lesions in CT images. In …. Tajbakhsh et al, they sought to answer the following central question in the context of medical image analysis: Can the use of pre-trained deep CNNs with sufficient fine-tuning eliminate the need for training a deep CNN from scratch? To address this question, they considered 4 distinct medical imaging applications in 3 specialties (radiology, cardiology, and gastroenterology) involving classification, detection, and segmentation from 3 different imaging modalities, and investigated how the performance of deep CNNs trained from scratch compared with the pretrained CNNs fine-tuned in a layer-wise manner. In …they described a novel method for automated diagnosis of liver lesions in portalphase computed tomography (CT) images that improves over single-dictionary BoVW methods by using an image patch representation of the interior and boundary regions of the lesions. The classification accuracy for the images from the two institutions was 99% and 88%, respectively, and 93% for a combined dataset.

In this study, we evaluated the performance of a customized, sequential CNN in the task of classifying liver lesions Metastases and Hemangiomas.

The contributions of this work are the following:

a)        We compare our model, with pre-trained model, multi-layered and highly flexible architecture.

b)        We have used the batch normalization to normalize the input layer by adjusting the activations and to reduce overfitting; we have put the batch normalization before the input after every layer and dropouts between the fully-connected layers.

c)        The performance of classifiers is evaluated by two widely-used measures:

1. Matthews Correlation Coefficient (MCC),

2.  F₁ – Score (F1 Score is used to measure a test’s accuracy)

3. MATERIALS AND METHODS

3.1. Dataset

In this work, the dataset contains 154 liver lesions collected from Jiangsu hospital by searching medical records for cases of Metastases and Hemangiomas. The largest metastasis lesion was …mL, while the hemangiomas largest was …mL. The data is available in DICOM formats. In this work, we preprocessed our images by converting them from their DICOM formats to a NumPy arrays. We have chosen the NumPy arrays because it provides a high-performance multidimensional array object, and tools for working with these arrays and NumPy array has advantages as less memory, fast, convenient.

3.2.Proposed model

In this section, we explain each part of our model, is shown in Figure 2. in detail.

Image preprocessing: in this stage, normalization is performed. The images were resizing to 224 x 224-pixel resolutions to suit the input requirements of customized and pre-trained CNNs and normalized to assist in faster convergence.

Figure 2. The proposed model.

Feature extraction: Convolutional Neural Network model is used for feature extraction process.

Convolutional Neural Networks (CNN) belong to a specific category of Neural networks methods.

CNN has not only been able to learn image feature representations automatically, but they have also outperformed many conventional hand-crafted feature techniques (…)

The detailed Explanation of our model (CNN).

Figure 3. Architecture of the CNN for liver lesion    classification.

1. The first convolutional layer performs 5×5 convolutions with a 4×4 stride and with valid padding; followed by a ReLU layer, pooling layer of size 2×2. Batch Normalisation before passing it to the next layer to reduce the over-fitting.
2. The second convolutional layer perform 5X5 convolutions with a 1×1 stride and with valid padding; followed by a ReLU layer, pooling layer of size 2×2. Batch Normalisation before passing it to the next layer to reduce the over-fitting.
3.  The next convolutional layer performs 3×3 convolutions with a 1×1 stride and with valid padding; followed by a ReLU layer. Batch Normalisation before passing it to the next layer to reduce the over-fitting.
4.  The next convolutional layer performs 3×3 convolutions with a 1×1 stride and with valid padding; followed by a ReLU layer. Batch Normalisation before passing it to the next layer to reduce the over-fitting.
5. The next convolutional layer performs 3×3 convolutions with a 1×1 stride and with valid padding; followed by a ReLU layer, pooling layer of size 2×2. Batch Normalisation before passing it to the next layer to reduce the over-fitting.
6. The next layer is a 0.4% dropout layer in this work dropout is used to regularize dense layers.

7.  The final layer of our model is the prediction layer that outputs the predicted value for each image.

The classification: The classifier is needed after feature extraction to find the corresponding label for every test image. The Softmax activation is normally applied to the very last layer in a neural net, instead of using ReLU, sigmoid, tanh, or another activation function. The reason why Softmax is useful is because it converts the output of the last layer in your neural network into what is essentially a probability distribution. In this work we use Softmax classifier; the Softmax layer calculates the normalized exponential function to calculate the output activation function.

                $P\left(c_r|x,\theta \right)=\frac{P\left(x,\theta |c_r\right)p\left(c_r\right)}{\sum _{j=1}^k\mathrm{P}\left(x,\theta |c_j\right)p\left(c_j\right)}=\frac{\mathit{exp}\left(a_{\mathit{r }}\left(x,\theta \right)\right)}{\sum _{j=1}^k\mathit{exp}\left(a_j\left(x,\mathit{ \theta }\right)\right)}$

3.3.Training Procedure

In this stage, we present the training for the models.

Normalization: Mean centering and standard deviation normalization was applied to each image patch.

Train/Validation/Test split: our dataset contains 154 liver lesions. The training set of 84 lesions images, 35 lesions images as a validation set, and the test set contained 35 lesions images.

Batch size: A batch size of 32 images was used for training.

Optimization and Learning Rate Scheduling:                            The Adam optimization algorithm [14] was used with an initial learning rate of 10^–3. This learning rate was reduced by a factor of 2 after 10 epochs without improvements in the validation set accuracy, repeatedly. The allowed minimum learning rate was 10^–10.

Iterations: The initial number of epochs was 10^–3.

Training time on an NVIDIA Titan Xp GPU was 2.45h for our model and 4.25 for AlexNet model.

3.4.AlexNet

AlexNet is a convolutional neural network that is trained on more than a million images from the ImageNet database (). AlexNet is one of the most important and famous of the CNN architectures (.)

The AlexNet architecture consists of five convolutional layers, with three max pooling layers then three fully-connected layers and finally a 1000-way Softmax classifier.

Figure 4. Architecture of AlexNet.

3.5.Evaluation

To evaluate how good is our model, we use the following metrics:

The accuracy, precision, and recall(..).

The accuracy of the models is calculated following

equation:

$\mathit{Accuracy}= \frac{\mathit{tp}+\mathit{tn}}{\mathit{tp}+\mathit{tn}+\mathit{fp}+\mathit{fn}}$

Accuracy is the fraction of predictions our model got right. Accuracy = Number of correct predictions Total number of predictions.

$\mathit{Precision}=\frac{\mathit{tp}}{\left(\mathit{tp}+\mathit{fp}\right)}$

Precision refers to the percentage of your results which are relevant. The precision is the ratio of true positives and the total positive predictions.

$\mathit{Recall}=\frac{\mathit{tp}}{\left(\mathit{tp}+\mathit{fn}\right)}$

Recall refers to the percentage of total relevant results correctly classified by your algorithm. The recall is the ratio of true positives and the total, actual positives.

F1 Score is the combination between precision and recall. The range for F₁ Score is [0, 1].

      $F_{1 = 2* \frac{\mathit{Recall }*\mathit{ Precision}}{\mathit{Recall }+\mathit{ Precision}}}$

The Matthews correlation Coefficient (MCC) has a range of -1 to 1 where -1 indicates a completely wrong binary classifier while 1 indicates a completely correct binary classifier. MCC to measure and maximize the overall accuracy of the classification model.

$\mathit{MCC}=\frac{\left(\mathit{tp}/n\right)–\mathit{sp}}{\sqrt{\mathit{ps}\left(1–s\right)\left(1–p\right)}}=\frac{\left(\mathit{tp}\right)\left(\mathit{tn}\right)–\left(\mathit{fp}\right)\left(\mathit{fn}\right)}{\sqrt{\left(\mathit{tp}+\mathit{fp}\right)\left(\mathit{tp}+\mathit{fn}\right)\left(\mathit{tn}+\mathit{fp}\right)\left(\mathit{tn}+\mathit{fn}\right)}}$

4. RESULTS AND DISCUSSION

In this work, we propose a framework that is able to classify liver lesions of Metastases and Hemangiomas by using Convolutional neural networks.AlexNet is one of the best and a popular CNN that is composed of both stacked and connected layers. Table 1 presents the accuracy and quality metrics for our framework CNN using the liver lesion dataset from Jiangsu hospital and the training process for our model took 2.45 hours. We have an average accuracy of 87.75%.

Table 2 presents the accuracy and quality metrics for the AlexNet model using the liver lesion dataset from Jiangsu hospital and training process took 4.52hours. we have an average accuracy of 86.67%.

Table 3 presents the comparison between AlexNet model and our Model performance. We have found our model higher than that of AlexNet.

Table 4 presents Comparison of AlexNet and the custom CNN classification strength. We can see our model is much better and it is a strong classifier for this dataset. We also see that our model has a significantly lower false positive rate (FPR) than AlexNet. For this case we can see that AlexNet is not a strong classifier for this dataset but it does better with more generalized image identification.

Table1: Accuracy and quality metrics for the CNN classifier.

Table2: Accuracy and quality metrics for the AlexNet.

Table3: Comparison of AlexNet and Our model performance

Accuracy = (tp + tn)/ (tp + tn + f p + fn),

Precision = tp/ (tp + f p),

Recall = tp/ (tp + fn),

FPR = fp/ (tn + f p), and

FPR = fn/ (tp + fn).

Table4: Comparison of AlexNet and our Model classification strength

5. CONCLUSION

In this work, a convolutional neural network model was implemented to classify liver lesions of Metastases and Hemangiomas. The experimental results show that our model is a much stronger classifier and can achieve an accuracy of 87.82%. AlexNet is not best suited to the problem of liver lesion classification. We have found our model achieve better performance than AlexNet.

We have used the python for coding and we used Keras to build our CNN. For future work we need more data for getting higher accuracy result.

6. REFERENCES

• ALEXNET ….G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R.R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012
• Related work
• 1 Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Amitai, Jacob Goldberger,
• and Hayit Greenspan, Member, IEEE
  ”GAN-based Synthetic Medical Image Augmentationfor increased CNN Performancein Liver Lesion Classification”
• 2 K. Yasaka, H. Akai, O. Abe, and S. Kiryu, “Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced ct: a preliminary study,” Radiology, vol. 286, no. 3, pp. 887–896, 2017.
• 3 N. Tajbakhsh, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. B. Kendall, M. B.Gotway, and J. Liang, “Convolutional neural networks for medical imageanalysis: Full training or fine tuning?” IEEE Transactions on MedicalImaging, vol. 35, no. 5, pp. 1299–1312, May 2016.

• 4 I. Diamant, A. Hoogi, C. F. Beaulieu, M. Safdari, E. Klang, M. Amitai, H. Greenspan, and D. L. Rubin, “Improved patch-based automated liver lesion classification by separate analysis of the interior and boundary regions,” IEEE Journal of Biomedical and Health Informatics, vol. 20, no. 6, pp. 1585–1594, Nov 2016.
• CNN K. Nguyen, C. Fookes, A. Ross, and S. Sridharan, “Iris Recognition with Off-the-Shelf CNN
• Features: A Deep Learning Perspective,” IEEE Access, 2017.
• classificationS. Minaee and Y. Wang, “Palmprint Recognition Using Deep Scattering Convolutional Network,”
• arXiv preprint arXiv:1603.09027, 2016
• AlexNET ImageNet. http://www.image-net.org
• accuracy https://developers.google.com/machine-learning/crash-course/classification/accuracy
• adam optimization P.D. Kingma and J. Ba, “Adam: A method for stochastic opti?mization,” arXiv preprint arXiv:1412.6980, 2014.

