Intelligent diagnosis of coronavirus with computed tomography images using a deep learning model

Abstract. The coronavirus (COVID-19) disease appeared as a respiratory system disorder and has triggered pneumonia outbreaks globally. As this COVID-19 disease drastically spread around the world, computed tomography (CT) has helped to diagnose it rapidly. It is imperative to implement a faultless computer-aided model for detecting COVID-19-affected patients through CT images. Therefore, a detail extraction pyramid network (DEPNet) is proposed to predict COVID-19-affected cases from CT images of the COVID-CT-MD dataset. In this study, the COVID-CT-MD dataset is applied to detect the accuracy of the deep learning technique; the dataset has CT scans of 169 patients; among those, 60 patients are COVID-19 positive patients, and 76 cases are normal. These affected patients were clinically verified with the standard hospital. The deep learning-oriented CT diagnosis model is implemented to detect COVID-19-affected patients. The experiment revealed that the proposed model categorized COVID-19 cases from other respiratory-oriented diseases with 99.45% accuracy. Further, this model selected the exact lesion parts, mainly ground-glass opacity, which helped the doctors to diagnose visually.


Introduction
Coronavirus (COVID-19) has dragged the world into endangering conditions, and the World Health Organization (WHO) has categorized the contagious disease as a pandemic. This infection is a kind of virus that severely affects the respiratory tract and has a visible likeness to SARS. 1,2 This virus is a highly infectious disease, and the number of affected cases increases at a tremendous rate. To bring an end to this situation, the entire world is fighting against this virus with the fullest effort. Further, the COVID-19 virus is seriously threatening healthcare systems due to its highly contagious character. Moreover, the researchers are encouraged to battle against COVID-19 by implementing novel techniques in its treatment to eradicate this virus from the younger generation. The effectiveness of the infection assessment is essential to offer instant treatment to COVID-19-affected patients. 3 Based on the statement of the WHO on April 3, 2022, 489 million infected cases have been recorded, and around 6 million deaths have occurred globally. The affected cases have been recorded in nearly the whole world from the United Kingdom, United States, Germany, India, Italy, the Korean peninsula, and Japan. [4][5][6] The above situation motivates us to perform research on automatic detection and timely prediction methods to manage the disease. Even though many kinds of diagnostic tests are available to determine the infected patients, these are not efficient at stopping the disease from spreading. 7 A smart health care system was designed using the tremendous capability of the cloud, mobile computing, and IoT sensors. 8 To store the information about patients and monitor them, several medical services have been moved to IoT-oriented models to enhance the services and benefits of the health care system. 9 In the current scenario, developing technologies, such as real-time smart monitoring, smart protection methods, and services, are not effectively carried out in public places, schools, or hospitals. 10,11 The contribution of assistance with artificial intelligence (AI) has many success stories in the clinical diagnostic area of COVID-19 cases, and it provides a system for remote automatic surveillance. 12 The early prediction of COVID-19 is a significant problem because the infectious area is scattered and small. 13 Computed tomography (CT) is playing an important role in the early prediction and diagnosis of COVID-19. 14 Sometimes the results of radiologists produce higher rates of false positive (FP), 15 which shows that a sophisticated computerized lung CT identification technique is desperately required for exact detection of the affected cases, patient screening, and virus surveillance. For the classification of general images, AI is applied similarly to detect some kind of diseases, and medical images are used in AI by applying classification techniques. 15,16 A convolutional neural network (CNN) 17 is a broadly applied feed-forward network that has several models. It helps to extract the features in the given images. The above models are not efficient at classifying the images of CT because these images are fine-grained and possess very less interclass disparities; hence differentiating them is complicated. 18,19 Lately, some techniques have been implemented in the learning methods of fine-grained images, with a deep learning model for CT images being convincing. 20 Mainly, these techniques are in need annotation only at the image level, and the classification results get higher accuracy by training the models and refining the annotations. 21 The above method has some limitations, and to rectify these limitations, this research contributes the proposed DEPNet technique to predict COVID-19-affected cases using CT scans by applying a classification algorithm.
The remaining sections of this article are organized as follows: Sec. 2 states the related studies to the proposed work in detail, Sec. 3 consists of the explanation of the DEPNet methodology, Sec. 4 describes the performance analysis, and Sec. 5 explains the conclusion of the paper.

Related Study
In Ref. 22, the authors proposed a model by developing a pristine signal algorithm after removing the unwanted data termed a low-cost pervasive sensor. This model is involved in many experiments, such as coughing, breathing, and finding the respiratory tract. Moreover, the model obtained 98.99% respiratory system estimation accuracy and 97.34% accuracy in cough estimation. This model helps to screen patients who are affected by COVID-19 and diagnose and monitor patients on a large scale.
In Ref. 23, Hossain et al. introduced deep learning techniques for evaluating the direction of humans by examining their threshold range. Vedaei et al. 24 implemented a novel intelligent model for categorizing x-ray data into three various groups. In that method, they employed three algorithms with the deep learning CNN architecture. The accuracy of the model was evaluated with the benchmark x-ray image dataset of the affected and nonaffected patients, and it produced three different accuracies for three algorithms. The maximum accuracy was recorded as 90.3%.
In Ref. 25, the authors introduced an AI system for detecting COVID-19-affected cases via their recorded voice model using a smartphone. They designed an AI voice recognizing model by collecting the features of COVID-19-affected cases and obtained an accuracy of 98.49%. Xu et al. 26 introduced a B5G by applying the 5G networks possessing minimal latency and high bandwidth and by determining the x-ray images and CT scans of the patients. Gozes et al. 27 designed a robust model of IoT for social distance monitoring to defend the COVID19 cases. In this COVID-19 period, deep learning-oriented models have been implemented effectively for the analysis and classification of chest CT data. 28,29 Several deep learning models have been introduced for COVID-19 testing, 29,30 observation, 31,32 and detection in the hospital. 33 In Ref. 17, the authors designed a network-based model residual neural network50 (ResNet50), which was pretrained and proved to be robust for predicting the features in images. Further, they added a network-like feature pyramid that helped to collect the details of top-n from every image. 15 To learn the important features in the image, the attention unit was combined based on the extracted features.
In the current research, the COVID-CT-MD dataset was applied; it has chest CT scan images of 169 people. Out of 169, 60 cases were affected by COVID-19, and 76 were healthy people.
To detect the abnormality in the chest CT scan images efficiently, a deep learning model named detail extraction pyramid network (DEPNet) was designed. First, the proposed model predicted the main lesion area at several scales by combining the already trained weakly supervised deep learning network (WSDNet) and the feature pyramid network (FPN). Based on the predicted regions, a WSDNet was used to collect the features in every region and the common features among the regions. Then, the collected attributes were combined with the global features of the original image, and for detecting the image level, the features were applied to the multilayer perceptron. Finally, for patient-level diagnosis, the detection in every chest CT image frame of a single patient was integrated. The proposed model showed 99.45% accuracy for differentiating the COVID-19 patients from healthy people by training and validating the COVID-CT-MD dataset. Moreover, the proposed DEPNet ground-glass opacity and lesion features help the doctors to diagnose visually.

Proposed DEPNet
The proposed method introduced a deep learning-oriented chest CT scan analysis technique to predict the COVID-19-inducing pneumonia and to trace the prime lesions. The fully automated CT diagnosing model was implemented on three important levels. In the first level, the main portion of the lung was extracted, and in the second level, a DEPNet was done to retrieve the image-level detections. In the third level, image-level detections were integrated to obtain each patient-level diagnosis.

Data Preprocessing
3D CT scan images of every patient had nearly 200 images, and the adjacent images were very much identical. Nearly 15 images were selected as representatives who were present at equal distances. The speed of the prediction was increased after removing the redundant images. The Open Source Computer Vision Library package was applied in the research to detect the incomplete images, determine the whole lung images in the CT scan, and remove the boundary of the image. To remove the overfitting problem in the deep learning model, the blank portion of the image was replaced with some other lung portion images. Finally, 60 COVID-19 cases with 565 CT images, 33 bacterial pneumonia cases with 232 slices, and 76 normal people with 634 slices were obtained in the research.

DEPNet
The debate was designed according to the already trained WSDNet that was proved to be efficient in predicting the features of the images. Moreover, the FPN was included to collect the topn information in every image. As shown in Fig. 1, the COVID-CT-MD dataset is given as input to the WSDNet to collect the features, and then these features are the input to dense and pooling layers for collecting the global features. Then, the FPN model extracts the local information in the images.
Then, the feature map and region of scales sized 14 × 14, 7 × 7, and 48 × 48, 96 × 96 were used respectively in small portions because lesion portions were commonly tiny, and applying the bigger size feature map would produce more noise in the model. Thus, the FPN detected the important region with a high score. Aggregations of detected features are performed, and a multilayer perceptron is implemented for data calculation. Finally, the images-level frame detection is performed, and selected frames are integrated to get the output. Figure 1 describes the DEPNet analysis model. Dataset is preprocessed and sent to WSDNet for selecting the appropriate global features. Weekly supervision helps to annotate and label the unknown dataset variables with heuristically pattern identification techniques. After processing the features with multiple WSDNet, feature aggregation is performed. Further, multi layer perceptron helps to classify the features, and images are detected. Finally, image frames are integrated for diagnosing COVID-19 with high accuracy.
Based on the extracted portions, top-n subimages are divided from the real image. Then, the real image is measured to create a fresh image by applying the multiplication of the relative portion of the top-n subimages with the high score, and the area not in the top-n images is set as null. After that, these subimages and the newly created images are given as input to WSDNet to gather important features; WSDNet is depicted in Fig. 2.
Finally, the collected features of the original image were aggregated; top-n sub-images and the created image were converted to a one-dimensional (1D) vector, and it was sent to multiple layer perceptron to detect image-level features. According to the FPN, the proposed model not only produces the patient-level detection for helping medical practitioners but also translates the detection by measuring every pixel of the real image. For every affected case, the detection was done on every image frame, and the image-level measured output was integrated for the patient-level detection. In the proposed method, average pooling was applied to aggregate the image-level measurement of illness of every person for the patient-level detection.
In the proposed technique, the COVID-CT-MD dataset was used to evaluate the DEPNet model. The benchmark dataset was given as input to WSDNet after the dataset was preprocessed to collect the features. After that, those features were forwarded to dense and pooling layers for collecting the global features, and FPN extracted the local information of the images. The collected features of the original image was aggregated as the top-n sub-images. The created images were converted to 1D vector and again images were sent to the multiple layer perceptron to detect the image-level features. Therefore, it not only produces the patient-level detection to help the doctors but also translates the detection by measuring every pixel of the original image. The image-level measured outputs are integrated for patient-level detection.

Experimental Evaluation
In the experimental analysis of this proposed work, n ¼ 3, which means that three subimages are collected in every input image. To prove it practically, a model was implemented with two functions: classifying COVID-19 cases from normal people and dividing COVID-19 people from other disease-affected cases and normal people. There are two final prediction classes of our proposed model: first class-0 (Non-covid class) and second class -1 (covid class). For every function, the patient-level partition method was applied by employing random division with 60%, 30%, and 10% training, testing, and validation, respectively. The training images were applied to train the proposed model, and validation images were employed to enhance the hyperparameters for improved effectiveness. The last enhanced design was separately evaluated on the test images. The precision, accuracy, recall, specificity, and F1 score are estimated using Eqs. (1)-(5).
Step 3: Preprocessed COVID-CT-MD dataset is given as input to the WSDNet to collect the features.
Step 4: Collected features are the input to dense and pooling layer to collect the global features.
Step 5: FPN model extracts the local information of the images.
Step 6: Collected features of the original image are aggregated; top-n sub-images.
Step 7: Created images are converted to 1D vector, and those images are sent to multiple layer perceptron to detect the image-level features.
Step 8: Then, it not only produces the patient-level detection to help the doctors but also translates the detection by measuring every pixel of the real image.
The evaluated metrics are calculated as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 1 1 6 ; 7 2 3 (1) E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 2 ; 1 1 6 ; 6 7 0 (2) E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 3 ; 1 1 6 ; 6 3 7 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 4 ; 1 1 6 ; 6 0 4 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 5 ; 1 1 6 ; 5 7 1 Here true positives (TPs), FPs, true negatives, and false negatives were the values that were calculated area under the receiver operating feature curve (AUC) using the scikit-learn. For the three-way classification function, the proposed work calculated the F1 score, recall, and precision for every type and reported the average by default. Initially, an experiment was done by partitioning 60 COVID-19 cases with 565 CT images, 33 bacterial pneumonia cases with 232 slices, and 76 normal persons with 634 slices. The integration of the image-level detection produced trivial rises in the AUC values at the patient level from 0.95 to 0.99 for validation and individual tests. To prove the efficiency of the implemented DEPNet structure, the DEPNet networks with the existing deep learning work, in particular, VDD16, Dense Net, and Resnet, 15 were compared. As depicted in Table 1 and Fig. 3, DEPNet obtained the maximum AUC value among all of the deep learning models, Visual Geometry Group16 (VGG16) and ResNet scored  Fig. 3 Comparisons of patient-level performances.
nearly the same value, and DenseNet scored a minimal value. When evaluated by F1-score, the other balanced evaluation, defense research establishment network scored the highest and DenseNet had the lowest value, whereas ResNet was insignificantly more than VGG16. Comparison of patient level prediction with the other existing deep learning techniques were done, and the measured values of AUC, precision, recall, specificity, accuracy, and F1-score of the DEPNet are higher than those of the other models. VGG16 and ResNet had nearly similar scores, and DenseNet scored slightly lower than the other models. Figure 4 represents the receiver operating feature curves for comparing with similar benchmark techniques. DEPNet has a maximum TP score at portions of the minimized FP rate (FPR) (FPR < 0.02), which are also considered as very essential regions because there were more cases   of bacterial pneumonia than COVID-19. At a maximum FPR (>0.2), all techniques other than DenseNet exactly predict the COVID-19 cases. Figure 5 shows the result of the proposed DEPNet that correctly predicted the COVID-19 pneumonia patients. Figure 6 depicts the CT images of bacterial pneumonia that were erroneously predicted as COVID-19 cases. Figure 7 shows CT images of COVID-19 cases that were falsely detected as bacterial pneumonia cases.

Conclusion
This proposed technique has explained the usefulness of a deep learning technique to help medical practitioners in predicting COVID-19 cases and determining the feasible lesions in the CT images automatically at the end. This model enables categorizing the COVID-19-affected person rapidly and accurately. In the existing model, the authors did not apply a benchmark dataset for evaluation because it was implemented in the earlier period of the COVID-19 pandemic. But, in this proposed method, there are many benchmark datasets to evaluate the designed techniques. This model was designed with the combination of pretrained WSDNet, which enhanced the accuracy of DEPNet. The performance of DEPNet was compared with VGG16, ResNet, and DenseNet. DEPNet produced 99.45% accuracy, which was very much higher than the other models. a number of journals and conference scientific papers. His current research interests include security, computer networks, data bases, and IoT.
Milos Mravik is an assistant of the Computer Science Department at Singidunum University. He is currently a PhD candidate in the Department of Computer Science. His teaching focuses on performance improvement, machine learning, contemporary technologies, and communication technology. He is certified in administration and database fields by many major IT companies, such as Microsoft, Cisco, IBM, and Juniper. He is the author of one University book. His research interests are computer networks, cloud technologies, IoT, and machine learning.
Dijana Jovanovic received her BSc and MSc degrees from the Department of Informatics at the College of Academic Studies "Dositej" in 2018 and 2019, respectively. Currently, she is working as a research assistant at the College of Academic Studies "Dositej." Since 2019, she has been a PhD student at the Faculty of Computer Science, Megatrend University of Belgrade, Serbia.
Ivana Strumberger started her university career in 2013 as teaching assistant at the Faculty of Computer Science in Belgrade. She received her PhD in computer science from Singidunum University in 2020. Currently, she is working as a teaching assistant at the Faculty of Informatics and Computing, Singidunum University, Belgrade, Serbia. However, she is in the process of being elected for assistant professor. She conducts research in the domain of computer science and her specialty includes swarm intelligence, machine learning, optimization and modeling, cloud computing, computer networks, and distributed computing. She has published around 50 scientific papers in high-quality journals and international conferences indexed in Clarivate Analytics JCR, Scopus, WoS, and IEEExplore. She has also published 10 book chapters in Springer Lecture Notes in Computer Science series. She has also published one book from the domain of cloud computing. She is a regular reviewer of many international state-of-the-art journals with high Clarivate Analytics and WoS impact factor, such as Applied Soft Computing, Journal of Ambient Intelligence and Humanized Computing, Soft Computing, Swarm and Evolutionary Computation, etc. She has been included in prestigious list of Stanford University with best 2% world scientists for the year 2021.
Miodrag Zivkovic received his PhD from the School of Electrical Engineering, University of Belgrade in 2014. Currently, he is working as an associate professor at the Faculty of Informatics and Computing, Singidunum University, Belgrade, Serbia. He is involved in scientific research in the field of computer science and his specialty includes stochastic optimization algorithms, swarm intelligence, human-computer interaction, and artificial intelligence algorithms.
Nebojsa Bacanin received his PhD in computer science from the Faculty of Mathematics, University of Belgrade in 2015. He started his university career in Serbia 15 years ago at the Graduate School of Computer Science in Belgrade. Currently, he is working as a full professor and as a vice-rector for scientific research at Singidunum University, Belgrade, Serbia. He is involved in scientific research in the field of computer science, and his specialty includes stochastic optimization algorithms, swarm intelligence, soft computing, and optimization and modeling, as well as artificial intelligence algorithms, swarm intelligence, machine learning, image processing, and cloud and distributed computing. He has published more than 240 scientific papers in high-quality journals and international conferences indexed in Clarivate Analytics JCR, Scopus, WoS, IEEExplore, and other scientific databases. He has also been included in the prestigious Stanford University career list with 2% best world researchers in the field of artificial intelligence the years 2020 and 2021.