Diagnosis of COVID-19 from X-ray images using deep learning techniques

Abstract In this study, we searched for the latest literature on the use of deep learning applications to combat COVID-19 and these were identified from several search engines including IEEE Xplore, Google Scholar, PubMed, and Scopus. This involved a comprehensive analysis of the studies to identify the challenges associated with the use of deep learning models with a view to highlight the possible future trends in the development of deep learning systems that are efficient and more reliable for the diagnosis of COVID-19 patients. This paper provides information related to the deep learning techniques used to detect COVID-19. This paper discusses the Convolutional Neural Networks’ (CNNs) structure, how to train CNNs, and highlights the different pre-trained models of CNNs that can be used for the detection of COVID-19. This paper explores the latest developments in the diagnosis of COVID-19 using deep learning applications that rely on the use of X-ray images taken from medical imaging samples. A review of the different models developed to facilitate effective diagnosis of COVID-19 provides information regarding the experimental data, the data splitting techniques used, as well as the proposed architecture for detecting COVID-19 and the different evaluation metrics for each model. This paper is a useful resource for medical and technical experts, as it helps them to develop a sound understanding of how deep learning techniques can be harnessed to stop the spread of COVID-19.


Introduction
There has been a dramatic turn of events across the globe since December 2019, when a new virus was detected in Wuhan city in China (The WHO-China Joint Mission on Coronavirus Disease 2020 and World Health Organization (WHO), 2019). The virus spread extremely fast from China to many parts of the world resulting in many infections and loss of many lives (Shuja et al., 2021). The highly infectious disease that emerges from this type of virus was named COVID-19 on 11 February 2020 by the World Health Organization (WHO) (Sohrabi et al., 2020). When the infected people cough or sneeze, their respiratory droplets contaminate surfaces thereby facilitating the spread of the virus . Given the rate at which the disease spread across the world, the WHO declared it as a pandemic on the 11th March in theof same year (W. H. Organization, 2020).
The damage caused by the COVID-19 pandemic is huge, it results in loss of many people's lives (Worldometer, 2020). This catastrophic loss of life has a huge impact to the countries' social and economic status. As a result, all the affected countries have had to implement some COVID-19 restrictions in an effort to control the spread of the virus (Boccaletti et al., 2020). However, COVID-19 restrictions such as lockdowns have caused problems in society. For instance, a study conducted by Aki et al. (Aki et al., 2020) in Nigeria demonstrated that lockdown had some psychological implications where people suffered from general anxiety disorders and stress. In particular, the study revealed that younger and middle-aged people suffered from general anxiety disorder more than older people, while older people suffered more stress than younger and middle-aged people.
Several studies have been conducted globally to find a way to combat the COVID-19 pandemic. This included efforts to develop a quick way to diagnose the disease, how to stop the spread of the virus from infected patients, and how to predict the patterns and trends of the disease in countries. For example, in Ecuador, a mathematical model was developed by Tene et al. (Tene et al., 2021) to help with the analysis of COVID-19 outbreak. The proposed model is shown to provide a simple and less time-consuming way to study the effect of the COVID-19 pandemic in different countries, thereby facilitating speedy decisions against the disease. In the same vein, Junnumtuam et al. (Junnumtuam et al., 2021) developed a Bayesian confidence interval for the coefficient of variation of zero-inflated inflated distribution that was helpful to study the daily COVID −19 cases. In a study carried out by Rahman et al. (Rahman et al., 2020) a system that restricts the growth of COVID-19 by finding people who are not wearing facial masks in a smart city network was developed. This involved ensuring that all public places were monitored with closed-circuit television (CCTV) cameras. A deep learning architecture was trained using a dataset that included images of people with and without masks collected from different sources. It was found that the system achieved 98.7% accuracy on distinguishing people with or without face masks for previously unseen data, hence, quite effective in terms of mitigating the spread of communicable diseases for many countries. In another study, Islam et al. (Islam, Ullah, et al., 2021) conducted an overview of the different technologies that are used to help the infected patients for respiration. Their study highlighted some of the latest technologies for breathing, such as oxygen therapy devices, ventilators and CPAP, providing comparative analysis to make it easier for the selection of more efficient technologies. Other interventions will be discussed later, in particular, studies that involved developing techniques for the detection of COVID-19.
People suffering from COVID-19 may experience mild to severe respiratory problems, and ventilation support might be needed (Singh et al., 2021) . It has been shown that elderly people and those with some underlying chronic diseases are more vulnerable to COVID-19 infection. This underlines the importance of the early detection of COVID-19, which helps to trace contacts directly or indirectly and avoid the spread of the virus. While it is known that the real-time polymerase chain reaction (RT-PCR) test is an effective way to diagnose coronaviruses, these tests are challenging to undertake for every infected person given the short supply of test kits (Ai et al., 2020;Hemdan et al., 2020). The other downside of relying on these tests is that they take long time to produce results that range from a few hours up to 2 days. In addition to the delay in confirming results for the patients, these tests also produce high levels of false-negative results (Basu et al., 2020;Nayak et al., 2021). In light of these limitations, the need for an alternative screening technique that is faster and more reliable cannot be overemphasised (Farooq & Hafeez, 2020).
One of the approaches that can provide a reliable alternative to the PCR screening method is the one that makes use of radiography images. This view has been documented in several research articles available in the Radiology journal, which confirm that COVID-19 can be detected using chest scans . It has been shown through research that there is a distinction between the lungs of a COVID-19 patient and those of an individual free from COVID-19 infection. Research has revealed that people with COVID-19 symptoms present lungs with some visible marks, which look like ground-glass opacities and hazy darkened spots (Fang et al., 2020;Xie et al., 2020). As a result, it is believed that a system that is based on chest radiology can be utilized to detect and provide an effective way to follow-up on COVID-19 related cases.
Arguably, the chest radiology-based system is likely to have more benefits in comparison to the currently used method (A. I. . Some of the advantages, including the production of faster results, can be used to analyse many cases simultaneously and can be made available better than the conventional method test kits. Given the important role played by radiography in today's healthcare system, radiology imaging systems can be easily found in almost every hospital. As a result, this alternative system can be more convenient and accessible. In addition, this alternative system can be beneficial in hospitals struggling to access the testing kits and other resources to be used with the conventional method. As a result, X-ray images and computer tomography (CT) have been utilised to support medical doctors' decision-making to enhance the effectiveness of their interventions. There is evidence in the extant literature that chest radiography imaging system is one of the principal approaches for diagnosing pneumonia worldwide (Jaiswal et al., 2019). To reiterate, the method is considered to be easily accessible, fast, and cheap, hence, it is a widely used clinical method (Narayan Das et al., 2020). Given that the main problem associated with COVID-19 is pulmonary disease, such as pneumonia, CXR imaging is a first-line triage tool for COVID-19 patients (Shamout et al., 2021).
It is known that other imaging options, such as CT produce higher resolution; however, CXR imaging is cheaper and inflicts a lower dose of radiation and it is also easier to obtain without risking the contamination of the imaging equipment and disrupting the radiological services (ACR, 2020). However, if screening is to be done manually, it takes an enormous amount of time and it is possible that several human errors can also be experienced causing unnecessary bias in the interpretation of data. Arguably, automated and more reliable approaches for screening COVID-19 patients are necessary (Chaddad et al., 2021). Although medical imaging techniques including chest CT and CXR provide non-invasive approaches to identify COVID-19, clinicians cannot always afford to identify small changes in the scans or images that are caused by the presence of COVID-19. As a result, it is imperative for more intelligent tools to be developed that can predict COVID-19 infection from medical images.
Artificial Intelligence (AI) and Machine Learning (ML) techniques have been used to provide creative solutions to several computer science problems, which range from bio-informatics to image processing (Shuja et al., 2021). Deep learning is part of the field of ML that has been inspired in its development by the structure of the brain (Narin et al., 2021). These techniques have been used extensively in medical imaging and help to produce credible results from these data sets. Deep learning models have been used successfully in different areas including classification, segmentation, and detection of lesions from medical data. In addition, it has been possible to analyse data obtained from medical imaging techniques, such as magnetic resonance imaging (MRI), CT, and X-ray with the aid of deep learning models. Based on these analyses, it is possible to detect and diagnose diseases such as diabetes mellitus (Yildirim et al., 2019), skin cancer (Dorj et al., 2018;Hosseinzadeh Kassani & Hosseinzadeh Kassani, 2019), brain tumour (Saba et al., 2020) and breast cancer (Ribli et al., 2018;Yu et al., 2021) as well as the detection of pulmonary pneumonia X-ray images (Rajpurkar et al., 2017) and lung segmentation (Gaál et al., 2020).
Given their enhanced performance in comparison to the traditional methods, deep learning methods have been adopted more widely across the world. One distinctive characteristic of deep learning approaches is that the features do not require hand-picking. It is possible to train a new model to learn about the key features in the dataset by altering the parameters and configurations of the deep learning convolutional neural network (CNN) architecture. Prior to the advent of COVID-19, researchers have been using deep learning methods to investigate the field of medical imaging. However, with the arrival of the COVID-19 pandemic, the overall use of deep learning methods has increased quite significantly.
The conduct of this study has been informed by similar studies conducted in the past. For example, Rahman et al. (Rahman et al., 2020) conducted an analysis of several studies and identified four machine learning approaches to combat COVID-19 pandemic. In their study, they discussed the major challenges of the existing systems and outline future trends. They recommend the use of machine learning for proper analysis, screening, tracking, forecasting, and predicting the characteristics and trends of COVID-19. Similarly, Asraf et al.  explored several deep learning approaches that can be used to control the pandemic including medical imaging, disease tracing, analysis of protein structure, drug discovery, and virus severity and infectivity to control the ongoing outbreak. In the same vein, this paper seeks to explore and synthesise the latest developments in the diagnosis of COVID-19 using deep learning approaches that rely on the use of X-ray images taken from medical imaging samples. This paper provides comprehensive information on the potential application of deep learning techniques based on the review of several models that have been developed for use in the diagnosis of COVID-19. Based on a comprehensive literature review, we bring to light different aspects including the type of data used in the experimental work, the techniques used for data splitting, the suggested architecture for detection, as well as the evaluation metrics. It is hoped that a comprehensive review of the extant literature will help to bring to light the challenges of the different existing systems and map out the future trends in the development of a reliable and efficient deep learning system for the detection of COVID-19 using X-ray images. The structure of the paper is as follows: the following section 2 highlights the method used in this study. Section 3 provides an introduction to deep learning to detect COVID −19. Section 4 presents a brief overview of CNN architectures and the main points on available CNN architectures. Section 5 highlights the datasets that are used in the research. Section 6 provides a summary of the research and evaluation, and Section 7, the discussion and conclusion including limitations and recommendations for further studies are presented.

Method
This study is a meta-analysis of studies related to the use of deep learning approaches for the detection of the novel coronavirus . We searched for the latest literature focusing on approaches to combat the novel COVID-19 pandemic with particular emphasis on the use of deep learning techniques in the fight against COVID-19. The search of the literature involved the use of different search engines including Google Scholar, PubMed, Scopus, Cochrane, and IEEE Xplore. The initial search using the keyword "deep learning techniques for COVID-19" in Google Scholar resulted in more than 17,000 articles. With the use of more inclusion and exclusion criteria, for example, publication dates between 2014 and 2021, articles with full texts, and articles published in peer-reviewed journals, it was possible to identify 112 articles that were significant for a comprehensive review of the literature. Figure 1 shows a PRISMA flow diagram, which illustrates the screening and selection of studies reviewed in this paper.

Introduction to deep learning for covid-19 detection
Prior to the discussion of more information around the available methods for COVID-19 detection, it is necessary to have a general understanding of the following concepts: deep learning, deep CNN, and some common CNN architectures. Hence, this section and section 4 will focus on providing an overview of these important concepts as well as other concepts, such as classification, transfer learning, and the key features of some of the available CNN architectures.
In contemporary times, there are three commonly used concepts for describing systems or software that behave in an intelligent manner, and this includes AI, ML, and DL. DL is as much of a part of ML as it is an important part of the broader area of AI. Generally speaking, AI involves the integration of human behaviour and intelligence into systems or machines (Sarker, 2021a). On the other hand, ML involves learning from data or experience and this results in the construction of analytical models (Sarker, 2021b). DL also represents learning methods from data that involve computational approaches achieved through multi-layer neural networks and processing. To clarify, the concept of "Deep" in deep learning methodology describes the concept of multiple levels or stages through which data is processed for constructing a data-driven model. Researchers rely on the use of deep learning methods to detect COVID-19 from CT and CXR images. As indicated earlier, most of the research studies have revealed that deep learning methods produce enhanced performance, for example, in computer vision (Hassaballah & Awad, 2020), and face recognition (Masi et al., 2018), among others. DL systems involve several steps, which include data collection, data preparation, feature extraction and classification, and performance evaluation . To produce a particular model, training data is used, and this is assessed by validation data. The developed model's performance is evaluated using the test data. The key steps in the diagnosis of COVID-19 using the deep learning-based system, are feature extraction and classification. The deep learning technique facilitates the automatic extraction of the feature, carrying out different operations repetitively, and lastly, the classification is done. The newly developed system is then evaluated using several metrics such as accuracy, sensitivity, specificity, precision, and F1-score, among others.

Convolutional neural networks (CNNs) for COVID-19 detection
CNNs and ConvNets constitute one of the major groups of deep learning techniques deployed to identify and classify images. CNNs have been successfully deployed in medical image/video detection and classification (Loey et al., 2020). It is worth noting that several AI systems based on DL, in particular, CNNs, have been proposed for the diagnosis of COVID-19. These techniques have shown great potential in the diagnosis of COVID-19. They produce a high level of accuracy in detecting COVID-19 patients using chest x-ray images. These successful results have been attributed to the fact that deep CNNs do not extract features manually, instead they make use of algorithms that learn about the features in the data set automatically (Lecun et al., 2015).
Several studies have been conducted focusing on the development of CNNs for COVID-19 diagnosis. For instance, EMCNet was developed as an automated detection system for the identification of COVID-19 patients, which helps to evaluate chest X-ray images (Saha et al., 2021). When compared with the other recently developed systems, it was shown that EMCNet performs better with 98.91% accuracy, 100% precision, 97.82% recall, and 98.89% F1-score. This system is effective for the automatic detection of COVID-19 through instant detection and low false negatives. In another study, a deep learning technique based on the combination of a convolutional neural network and long short-term memory (LSTM) was developed for the diagnosis of COVID-19 automatically from X-ray images (M. Z. . This system uses the CNN for deep feature extraction and an LSTM is used for detection using the extracted feature. A total of 4575 X-ray images, which included 1525 images of COVID-19, were used as a dataset in the system. The experimental results revealed that the proposed system achieved an accuracy of 99.4%, AUC of 99.9%, specificity of 99.2%, sensitivity of 99.3%, and F1 score of 98.9%. This system can be further developed when more COVID-19 images are available and there is potential for the system to help doctors to diagnose and treat COVID-19 patients more easily. Another study focused on the development of a system that combines the architecture of convolutional neural network (CNN) and the recurrent neural network (RNN) to diagnose COVID-19 from chest X-ray images . This approach has proven to be a good alternative method to diagnose COVID-19 for the medical staff. In another study conducted in South Korea, some data mining models were developed for the prediction of COVID-19 infected patients' recovery using epidemiological dataset of COVID-19 patients (Muhammad et al., 2020). The developed model is one of the best models compared to all the other existing systems with a 99.85% overall accuracy in predicting the possibility of recovery of the infected patients from COVID-19.
The ability to detect features automatically from data without human input makes CNNs more powerful as compared to the conventional or traditional networks (Sarker, 2021a). The CNN architecture comprises several layers (also referred to as the multi-building blocks), including convolution layers and pooling layers, which are followed by one or more fully connected layers at the end (A. . It is in the convolutional and pooling layers where the feature extraction process is carried out. The classification process takes place in the fully connected layer (Narin et al., 2021). Figure 2 shows an example of a CNN architecture, and the different layers are described fully in the following section.

CNN Structure
Any CNN model consists of a first layer known as the input layer. The width, height, and depth of the input image constitute the input parameters. The input layer is then followed up by convolutional layers. These layers are characterised by the following parameters: number of filters, filter window size, stride, padding, and activation.

The convolution layer
This layer is the main building block of a Convolutional Network and is responsible for a significant part of the computational heavy lifting. This layer is where the mathematical operation of convolution is accomplished between the input image and a filter of a certain size M × N. The convolution layer can have more than one filter. When the filter is placed over the input image, the dot product is taken between the filter and the parts of the input image in line with the size of the filter (M x N). The output is referred to as the Feature map and this provides important information about the image, for example, the corners and edges.
Every feature map goes through an activation function where bias is introduced to create the output. In most cases, the rectilinear unit (ReLU) activation is utilised as the activation function. Figure 3 shows an example of a convolution layer

Pooling layer
The main aim of the pooling layer is to reduce the size of the convolved feature map in order to cut the computational costs. The output dimensions experience an exponential increase as the model size becomes larger due to an increased number of filters in the convolutional layer, which makes it difficult for computers to manage. The commonly used pooling techniques are max pooling, average pooling, global average pooling, or spatial pooling layer (Ozcan, 2020). Another intermediate layer that is used is the dropout layer (Narin et al., 2021) and its key function is to avoid network overfitting and divergence (Hawkins, 2003). The output is flattened to form a single-array feature vector, which is fed to a fully connected layer. Figure 4 shows an example of pooling layers.

Fully connected layer
This is the third layer located at the end of each CNN architecture. Each neuron inside the layer is connected to the rest of the neurons of the previous layer, and this is called the fully connected (FC) approach (Alzubaidi et al., 2021). This layer works as a CNN classifier. The layer is characterised with activation functions including softmax and tanh functions. The number of classes is indicated in this layer, and more information on this will be discussed in section 4.2 which highlights the different types of classification of COVID-19. The loss function is another key parameter, which is important for summarising the prediction errors produced during training and validation.

Classification
The Covid-19 classification, based on the type of sample of the available CXR images, can be divided into two schemes shown below: • Binary classification: this classification only produces two groups, namely: COVID-19 or non-COVID-19.
• Multi-class classification: this scheme involves identifying and separating COVID-19 samples from numerous other pneumonia cases as well as the presence of tuberculosis and normal case findings (Horry et al., 2020;Loey et al., 2020). This scheme is carried out with three and four classes. The three classes are divided into non-COVID-19, COVID-19 and other pneumonia and tuberculosis, and the four classes include non-COVID-19, COVID-19, pneumonia, and tuberculosis (Punn & Agarwal, 2021).

Transfer learning with CNNs
Transfer learning is a technique that involves moving the knowledge mined by a CNN from one particular dataset to address a new but linked task from another new set of data. This strategy makes use of a pre-trained CNN model with its weights that have been trained on a sizeable dataset (Al Hadhrami et al., 2021). The use of a pre-trained CNN model helps to remove the challenge of training a CNN from the initial stages, which calls for large, labelled dataset and an enormous amount of computing resources.
To make use of the full potential of the pre-trained CNN, two strategies are normally used (Apostolopoulos & Mpesiana, 2020) and these are discussed below.
(1) One of these strategies is known as feature extraction through transfer learning. This strategy involves the retention of the initial architecture and the learned weights by the pre-trained model (Huh et al., 2016). In this case, the pre-trained model functions as a feature extractor only and the extracted features are fed into a new network that carries out the classification task. It is important to note that this approach is useful in terms of preventing computational costs that are likely to emerge when training a very deep network has to be done from the initial stages or to retain the important feature extractors trained during the first stage.
(2) Fine-tuning is the second approach, and this involves a more complex procedure whereby optimal results are achieved by applying specific changes to the pre-trained model. The changes can include parameter tuning and making changes to the architecture. This means that the weights of the pre-trained CNN model are kept in some layers and tuned in others. Normally, the first layers maintain their weights since the features derived from these layers are general and applicable to new tasks. The last layers produce specific features that can potentially benefit from the fine-tuning approach given that some adjustments will be made to the specific targeted dataset.
The main difference between transfer learning and fine-tuning is that transfer learning involves the retraining of all layers (Subramanian et al., 2022), whereas fine-tuning includes refining of some layers as well as the retraining of some layers depending on the application. These methods have got a disadvantage that the size of the input image cannot be altered. As a result, if the dimensions of the image of the pretrained model are small and transfer learning has to be carried out on a dataset whose dimensions are large, changing the size of the image is inevitable.
Changing the dimensions of the image, for example, changing the size of the image from large to small is likely to affect the performance of the model. It is, therefore, important to take note of any changes in the size of the images when using transfer learning and fine-tuning methods. Figure 5 depicts an example of the overall architecture of the transfer learning technique for the diagnosis of COVID-19.

Pre-trained model with deep transfer learning
A model that has been trained in areas that are similar to the context of the application is referred to as a pre-trained model. High computing power is required to train a model from the beginning and this process takes a lot of time (Shin et al., 2021). The benefit of a pre-trained model with transfer learning is that it facilitates the speeding up of the convergence with network generalisation (Alom et al., 2019). Most of the pre-trained models that are employed in transfer learning are especially made for large convolutional neural network (CNN). A number of pre-trained models have been utilized to diagnose COVID-19 and these included AlexNet (Krizhevsky et al., 2017), Inception/GoogleNet (Szegedy et al., 2015(Szegedy et al., , 2016, and ResNet (Hemdan et al., 2020), visual geometry group (VGG) (Liu & Deng, 2016;Simonyan & Zisserman, 2015), Xception (Chollet, 2017), DenseNet (Huang et al., 2017), and MobileNet (Howard et al., 2017), among others. These models have the advantage of being used to identify new tasks without undergoing training from the initial stages, and they can also be applied in various areas depending on their learning capabilities. A comparison of the models is provided below.
The AlexNet (Krizhevsky et al., 2017) contains eight layers whose parameters can be learned. The first five are convolutional layers with some of them being followed by max-pooling layers. The rest of the layers are fully connected layers. Each layer, with the exception of the output layer, uses ReLU activation. It was established that the use of ReLU as an activation function increased the speed of the training process by nearly six times. The use of dropout layers helped to prevent the model from overfitting. This is the first CNN model to operate with more than 100 million parameters that has got a 60 MB training set. This was considered to be large for deep learning at that particular time and the input image was 224 × 224. The use of VGGNet (Liu & Deng, 2016;Simonyan & Zisserman, 2015) produced good performance on the imageNet dataset. For the improved image extraction function, this model made use of smaller filters measuring 3 × 3 in comparison to AlexNet's 11 × 11 filters. This deep network architecture consists of two versions, which include VGG16 and VGG19, and these two have different layers and depths. For instance, VGG19 is deeper than VGG16. In terms of the number of parameters, VGG19 possesses a larger number of parameters, and this makes it more expensive to train the network compared with the VGG16.
On the other hand, the design of the inception model (Szegedy et al., 2016) consists of about 9 inception modules that are laid on top of each other with max-pooling layers in-between the modules. It consists of 22 layers (up to 27 including the pooling layers). Global average pooling is used after the last inception module. The versions that were designed later are known as Inception VN, with N representing the version number, for example, inception V1 (GoogleNet) (Szegedy et al., 2015), inception V2, and inception V3 (Szegedy et al., 2016), inception V4 and inceptionResNet (Szegedy et al., 2017). The increasing complexity is evident: for instance, inception V3 network consists of various symmetrical and asymmetrical building blocks, and each block consists of many branches of convolutions, average pooling, max-pooling, concatenated, dropouts, and fully connected layers.
A distinctive feature of the ResNet family (Hemdan et al., 2020) is that they use a residual block, that is, a network-in-network in their architecture. It is constructed in line with the concept of "skip connections" and employs several batch normalisations to help it train hundreds of layers while maintaining the same speed over a long time. The construction of the architecture is based on VGG-19 and it consists of a 34-layer plain network where shortcut and skip connections are added to. Different types exist and Inception-ResNet (Szegedy et al., 2017) is a hybrid architecture that merges the inception and residual blocks. The Dense Convolutional Network (DenseNet) (Huang et al., 2017)'s architecture is designed to facilitate deep learning networks to go deeper, and at the same time ensure that they can be trained efficiently using shorter connections between layers. Each layer is connected to the rest of the layers that are deeper in the network. This design helps to facilitate the maximum flow of information between the network layers. The Xception (Chollet, 2017) design is similar to the architecture of the inception family, which involves the use of blocks with depthwise separable convolutional layers. It has got 71 layers, and the input image size is 229 × 229. Last but not least, MobileNet (Howard et al., 2017) is a type of convolutional neural network that is built for mobile and embedded vision applications. Their modernised architecture makes use of depthwise separable convolutions to construct lightweight deep neural networks that can have low latency for mobile and embedded devices.

Datasets used for detecting COVID-19
The performance of any deep learning model is influenced enormously by the size of data. Given that COVID-19 is a relatively new disease, very few datasets have been availed for public use. Some publicly available datasets used in this research are presented in Table 1.

State of the art techniques for detecting COVID-19
Given that the COVID-19 pandemic is relatively new, limited datasets from a small number of samples are available in the public domain. In this case, the most appropriate strategy that can be used with such limited datasets is either transfer learning or fine-tuning. New CNN architectures can be developed but this requires the use of a broader range of images in each class to enhance the performance.
There are several reviews documented which demonstrate the way this approach has been used in recent times for the diagnosis of COVID-19 Dong et al., 2021;Nayak et al., 2021;Shi et al., 2021;Subramanian et al., 2022). In a study by El-Asnaoui and Chawki (Punn & Agarwal, 2021), deep convolutional neural network architectures for automatic multiclass classification of X-rays were compared with CT images between normal, bacteria, and coronavirus. A CNN was employed which is based on Inception network to detect COVID-19 disease in CT by Wang et al. (S. Wang et al., 2020). Similarly, Ying et al. (Apostolopoulos & Mpesiana, 2020) used a modified version of ResNet-50 pre-trained network to classify CT images into three classes, namely, non-COVID-19, COVID-19, and bacterial pneumonia.
A taxonomy for the classification of COVID-19 diagnosis system using X-ray is presented in this section to facilitate the navigation of the landscape. The applied deep learning methods are placed into two groups shown in Figure 6 namely: pre-trained model with deep transfer learning and the fine-tuning discussed in section 6.1 and the custom deep learning methods presented in section 6.2. In addition, an effort is made to compare the different models, in particular, the comparison will focus on the data splitting technique, the proposed architecture for detection, the number of classes in the classification, and the evaluation metrics (evaluation in section 6.3).

Transfer learning and fine-tuning techniques
The consulted literature indicates that transfer learning is used in most of the studies and pretrained models that are trained on the ImageNet database are employed to execute transfer learning. Firstly, Apostolopoulos et al. (Apostolopoulos & Mpesiana, 2020) conducted a comparative study of the existing deep learning architectures that employ the transfer learning methods such as MobileNet V2, Inception, Xception, Inception ResNet V2, and VGG19. Data were drawn from two different datasets from public medical repositories (Cohen et al., 2020;Mangal et al., 2020; D. S. Kermany et al., 2018). In the first instance, a collection of 1427 X-ray images, which included 224 images with confirmed COVID-19 cases, 714 images with confirmed common bacterial pneumonia, and 504 images of normal conditions, were made. This was followed by the use of another dataset, including 224 images with confirmed COVID-19 cases, 714 images with confirmed bacterial and viral pneumonia, and 504 images with normal conditions. Several other pre-trained convolutional neural network-based models were proposed by Narin et al. (Narin et al., 2021). The models include ResNet50, ResNet101, ResNet152, Inception V3, and Inception-ResNetV2. The images used in this study were taken from an open-source GitHub shared by Cohen (Cohen et al., 2020). The CXR images were from 341 COVID-19 patients. A total of 2800 normal, i.e., healthy chest X-ray images were selected from the "ChestX-ray8" database (X. Wang et al., 2017). An additional 2772 bacterial and 1493 viral pneumonia chest X-ray images from the Kaggle repository known as "Chest X-Ray images (Pneumonia)" were used . The designers used three different binary classifications with four classes including COVID-19, normal (healthy), viral pneumonia, and bacterial pneumonia, and they used a five-fold cross-validation approach. For binary classification, a study by Hemdan et al. (Hemdan et al., 2020) of a new COVIDX-Net framework was designed to automatically identify or confirm COVID-19 in X-ray images based on seven deep learning classifiers, such as VGG19, ResNetV2, DenseNet121, InceptionV3, InceptionResNetV2, Xception, and MobileNetV2. The sample of data (Cohen et al., 2020) comprised of 50 images-25 of these images were from healthy people, and the other half were from COVID-19 patients.
Another deep learning framework developed by Ozturk et al. (Ozturk et al., 2020) was utilised to detect COVID-19 cases. The model is called DarkCovidNet and it employs a DarkNet as a classifier containing 17 convolutional layers. The study used data from two different sources (Cohen et al., 2020;X. Wang et al., 2017). Three CNN models with increased number of layers, namely AlexNet, VGGNet, and ResNet, were utilized to produce the new system proposed by Basu et al.(Basu et al., 2020). Four open-source databases generated up to 305 COVID-19 X-ray images (Cohen et al., 2020). All the X-ray data sources were used to prepare two datasets, namely, Data-A and Data-B. Another deep learning framework for the detection of COVID-19 from CXR images was developed by Minaee et al. (Minaee et al., 2020). They employed the fine-tuning of four pre-trained convolutional models including ResNet18, ResNet50, SqueezeNet and DenseNet-121. A dataset of up to 5000 images referred to as COVID-Xray-5k was developed and the images were drawn from two different datasets, namely: Covid-Chest X-ray-Dataset (Cohen et al., 2020) and ChexPert dataset (Irvin et al., 2019). A board-certified radiologist helped to confirm the COVID-19 labels.
By utilising various pre-trained models, such as Res-Net, Inception-V3, Inception ResNet-V2, DenseNet169, and NASNetLarge, a new model was designed by Punn and Agarwal (Punn & Agarwal, 2021) to be used for binary classification (e.g., normal and COVID-19 cases) as well as for multi-class classification (e.g., COVID-19, pneumonia, and normal cases) from posteroanterior CXR cases. Due to the short supply of the posteroanterior CXR images associated with COVID-19 positive cases (Cohen et al., 2020), the design of the model involved the use of data from several datasets, for example, RSNA (Pan et al., 2019) and NLM(MC) .
Another system for the diagnosis of COVID-19 cases was developed by Sethy and Behra (Sethy et al., 2020). The system made use of 13 CNN pre-trained models useful for feature extraction and for classification, a Support Vector Machine (SVM) was used. A dataset containing X-ray images of COVID-19 patients, pneumonia patients, and healthy people was employed. There were a total of 381 images comprised of 127 confirmed COVID-19 images, 127 confirmed pneumonia images (63 bacterial pneumonia and 64 viral pneumonia), and 127 images from healthy people. The source of the COVID-19 X-ray images was the GitHub repository that was published by Cohen et al. (Cohen et al., 2020). Other sources, for example, Kaggle repository contributed 79 X-ray images (Mangal et al., 2020). The two additional sets of X-ray images were collected from pneumonia and healthy people (D. S. Kermany et al., 2018) and were used to train the classification model to differentiate the COVID-19 from pneumonia patients and healthy people.
Loey et al. (Loey et al., 2020) proposed another model using generative adversarial networks (GAN). This design was inspired by the limited datasets for COVID-19 in CXR images. The system is quite innovative and made use of creative ideas for the diagnosis of COVID-19 using GAN put forward by Ian Goodfellow in 2014 (Goodfellow et al., 2014) as well as some pre-trained models of CNN that are capable of deep transfer learning. The fundamental idea involves the collection of all the possible COVID-19 images in existence and then apply the GAN network to develop more images to facilitate the detection of the coronavirus with a high level of accuracy from the X-ray images. The design made use of a total of 307 images from different datasets and included four different classes, namely, COVID-19, normal, pneumonia bacterial, and pneumonia virus (Cohen et al., 2020;D. S. Kermany et al., 2018).
For the purpose of the current study, three deep transfer models have been selected and these include AlexNet, GoogleNet, and ResNet18. These models were selected because they contain a relatively small number of layers in their architecture which helps to reduce the complexity, the memory consumption, and the execution time for the newly proposed model. A research project conducted by Moutounet-Cartan (Moutounet-Cartan, 2020) examined several deep convolutional neural network architectures, such as VGG16, VGG19, InceptionResNetV2 and InceptionV3 and Xception. All of the models follow a flat multi-layer perceptron and a final 30% dropout. The training and testing of the system included the use of posteroanterior CXR images of a total of 327 healthy individuals, 152 patients, 125 diagnosed with COVID-19, and other types of pneumonia (Cohen et al., 2020;D. S. Kermany et al., 2018). Luz et al. (Luz et al., 2021) conducted an exploration of an efficient convolutional network architecture called EfficientNet. The model is commonly known for its high accuracy and low footprints and is useful for the detection of any abnormalities, which are a result of COVID-19, using chest radiography images. The distinctive feature of this model is that it employs far less parameters, for instance, it uses 5 to 30 times fewer parameters compared to the other pretrained models. A total of 13,569 X-ray images that were partitioned into different classes, namely, healthy, non-COVID-19 pneumonia, and COVID-19 patients and these were used to train the suggested approaches and the other proposed architectures (Cohen et al., 2020;Pan et al., 2019;. Asif (Asif et al., 2020) constructed deep convolutional neural networks (DCNN) based on the InceptionV3 model. The model was designed to facilitate the classification of COVID-19 CXR images into different classes including non-COVID-19, viral pneumonia, and COVID-19. In order to address the problem of inadequate data and training time, the model design involved the application of the transfer learning technique using ImageNet data. This dataset comprised of 864 COVID-19, 1345 viral pneumonia, and 1341 normal chest X-ray images and these were gathered from three different sources (Cohen et al., 2020;T. Rahman et al., 2020).
GS's grid search strategy was combined with three pre-trained CNN models, namely GoogleNet, ResNet18, and ResNet50, to produce a new classification system by Ozcan (Ozcan, 2020). In this case, the GS was utilised for hyperparameter optimisation, and the other pre-trained models were utilised for transfer learning. The data employed in this design were collected from several sources and hospital shares (Cohen et al., 2020) and hospital. Bukhari et al. (Usama Khalid Bukhari et al., 2020) experimented with ResNet-50 CNN architectures using 278 CXR images. The images were collected from public repositories that are published by the University of Montreal and the National Institutes of Health (Cohen et al., 2020;National Institutes of Health, 2022). The images were divided into three classes, namely, normal, pneumonia, and COVID-19. Due to the limited dataset, a transfer learning approach was utilised together with pre-trained models and ImageNet data (Russakovsky et al., 2015). The experiment involved training of only six layers of the pre-trained Resnet-50 model, while the rest of the layers remained unaltered. To prevent overfitting, the design of the model involves the use of several augmentation techniques, including a 50% dropout.

Custom deep learning techniques
The use of custom deep learning techniques helps to develop user-friendly architectural designs that facilitate more consistent and accurate performance because of their ability to align with the selected application. A deep convolutional neural network architecture called CoroNet was proposed by Khan et al. (A. I. Khan et al., 2020). This model is based on the Xception architecture and is useful for the detection of COVID-19 cases using chest X-ray images. It has a dropout layer with two fully connected layers added at the end. This new model has been trained and tested using a dataset developed by collecting COVID-19 and chest pneumonia X-ray images from two open access databases (Cohen et al., 2020). The model is used to classify three different types of pneumonia, namely bacterial pneumonia, viral pneumonia, and COVID-19 pneumonia. Their study also involved a binary classification and a three-class classification version of the newly proposed model.
Another model, called COVID-Net, was developed to detect COVID-19 using chest X-ray images by . This is a residual deep architecture that uses a projectionexpansion-projection-extension (PEP) design pattern with two stages of projections, expansions, a depthwise representation, and an extension to facilitate classification of cases. This system employed three classes instead of four by classifying the bacterial and viral classes into a single group of negative cases. They created COVIDx, an open access benchmark dataset that consists of 13,975 CXR images obtained from five open access data repositories (Cohen et al., 2020;Pan et al., 2019;T. Rahman et al., 2020). The authors performed transfer learning by training the CNN architecture on the ImageNet dataset followed by the COVIDx dataset.
Another model called CVDNet was proposed by Ouchicha et al. (Ouchicha et al., 2020). This is a Deep Convolutional Neural Network (CNN) model and is used to classify COVID-19 infection from non-COVID-19 and to identify other cases of pneumonia from chest X-ray images. The architecture proposed for this model is based on a residual neural network, and it is built using two parallel levels with different kernel sizes to bring together local and global features of the inputs. The model is developed and trained using a dataset that was made public which contains a mix of 219 COVID-19, 1341 normal and 1345 viral pneumonia CXR images (T. Rahman et al., 2020).
There is an alternative modelling framework proposed by Afshar et al. (Afshar et al., 2020). It is based on Capsule Networks, also known as COVID-CAPS. The new network used a combination of four convolutional layers and three capsule layers. The last capsule layer is used for classification, and the rest of the layers are used for feature extraction. One of its celebrated advantages is the ability to handle small datasets, an important feature especially with the rapid spread of COVID-19. The model was developed using the same dataset as the COVID dataset published by ).
An enhanced ResNet-50 CNN architecture called COVIDResNet has been designed by Farooq and Hafeez (Farooq & Hafeez, 2020). Their model presents a three-step technique to fine-tune a pretrained ResNet-50 architecture. By doing this, they enhance the performance of the model by reducing the training time. The design was achieved by continuously resizing the input images to 128x128x3, 224x224x3, and 229x229x3 pixels and fine-tuning the network at each stage. This involved the use of COVIDx dataset after it was made public by the authors of COVID-Net (L. . It comprised of 5941 posteroanterior chest radiography images categorized into four classes including: (1) Normal (no infections), (2) Bacterial (bacterial pneumonia), (3) Viral (non-COVID-19 viral pneumonia), and (4) COVID-19.
Another model proposed by Abbas et al. (Abbas et al., 2021) is called Decompose, Transfer, and Compose (DeTraC), and it consists of three phases. During the first phase, a pre-trained CNN model is trained to extract deep local features from each image. Then, the classdecomposition layer of DeTraC is applied to help simplify the local structure of the distribution of data. The second phase focuses on finalising the training using a sophisticated gradient descent optimisation approach. The third phase involves the use of the class composition layer to refine the final classification of the images. This model is capable of addressing any abnormalities in the image dataset using a class decomposition method to analyse class boundaries. The design of this system employed a total of 196 images collected from two different datasets. In the dataset, 80 samples were from normal patients Jaeger, Karargyris, et al., 2014), 105 samples of COVID-19 were from another source (Cohen et al., 2020), and 11 samples were of SARS.
A concatenated neural network was proposed by Rahimzadeh and Attar (Rahimzadeh & Attar, 2020). Their model is based on Xception and ResNet50V2 networks used to classify the CXR images into three classes including normal, pneumonia, and COVID-19. To enhance the quality of the generated semantic features that are useful for classification, the experiment used inception and residual layers. Two sources of data that were open to public use were used (Cohen et al., 2020;Pan et al., 2019) and they contained 180 images of people infected with COVID-19, 6054 images of people affected with pneumonia, and 8851 images from healthy people. Given the limited number of COVID-19 class images, the researchers came up with an approach that involved training a neural network to deal with an unbalanced dataset.
A new COVID-19 detection AI model, called COVID-Diagnosis-Net, was developed by Ucar and Korkmaz (Ucar & Korkmaz, 2020). This model is based on the deep SqueezeNet model. The SqueezeNet has got a light network design and is therefore, tuned for the diagnosis of COVID-19 using a Bayesian optimisation additive. In this study, a distinctive dataset containing CXR images was selected to develop a rapid diagnosis system (L. . To prepare a special COVID-19 dataset, the study made use of two different open access datasets and these were put together to form the COVID-19 chest X-ray dataset (Cohen et al., 2020) and the Kaggle chest X-ray pneumonia dataset (D. Kermany et al., 2018). One of the distinctive features of the study (L. , was the performance of a multi-scale augmentation process to address the imbalance problem in the proposed dataset. The use of fine-tuned hyperparameters and augmented datasets help the proposed network to perform better than the other available network designs, thereby facilitating the achievement of higher accuracy level in the COVID-19 diagnosis. A study carried out by Mukherjee et al. (Mukherjee et al., 2021) developed a system capable of detecting COVID-19 applying shallow CNNs in chest X-ray images. This model's architecture comprised of less parameters in comparison to other deep learning models. The design of the model was validated using 321 COVID-19 positive chest X-ray images (Cohen et al., 2020). Another set of non-COVID-19, which consists of 5856 cases that were published publicly, was also considered in addition to the other datasets.
The CNN has been used for the extraction of features in hybrid models from chest X-ray images from a single data source and using binary classification by Alqudah et al. (2020). A dataset containing 71 chest X-ray images produced from 48 COVID-19 cases and 23 non-COVID-19 cases by Cohen et al. (Cohen et al., 2020). The use of different machine learning algorithms including CNN, support vector machine (SVM), and random forest (RF) facilitated the best recognition performance and helped to produce a binary classification of COVID-19 or Non-COVID-19. Two outstanding extracted features are used for training and parameter testing. Another model, VGG16, which is based on a modified CNN, was proposed and given the name Corona-Net by Elbishlawi et al. (Elbishlawi et al., 2021). This model employs three-class classification and consists of two phases for the detection of COVID-19, namely: reinitialization and classification phases. The architecture of CORONA-Net is inspired by the absence of networks that are pre-trained on large medical datasets, for example, ImageNet (Krizhevsky et al., 2017) which can use their weights to transfer knowledge from one task to another. The design of the reinitialization phase facilitates the weights of the backbone/encoder network to be more appropriate for medical images. The classification phase starts the training process from the weights learnt from the reinitialization phase as soon as the network has been initialised. The development of this model also relied on the same data set of the COVIDx that was published by .

Evaluation
There exist several metrics that can be used to evaluate the performance of classification models. These include the most commonly used measures such as classification accuracy, sensitivity, specificity, precision, and F1-scores. Several studies have been conducted using different methods. Table 2 summarises the studies highlighting the methods used as well as the evaluation of the effectiveness of the different models. The comparison of the models explores different aspects including the type of data used in experiments and the source of data (single or multiple sources). In addition, the comparative analysis reflects on the proposed architecture for the detection of COVID-19, the number of classes in the classification, as well as the evaluation metrics.
It is evidently clear from the consulted literature that several researchers have started exploring the development of appropriate models for the detection of COVID-19 and other conditions, such as pneumonia. These studies, however, demonstrate that research in this area is still in its infancy stage; hence, the need for more research to be carried out to enhance knowledge and understanding of the best approaches that can be used to design models that work efficiently in the diagnosis of COVID-19. Arguably, there is scope for improvement in the development of deep learning models for the screening of COVID-19 suspected individuals. This, therefore, underscores the importance of the research at hand.

Discussion and Conclusion of the research
This study involves the conduct of a comprehensive review of extant literature focusing on the diagnosis of COVID-19 using the DL networks. Several studies were identified using different search engines, and they were reviewed. There is evidence that many researchers have focused their attention on the development of systems aimed at combating the novel COVID-19 This paper has successfully provided detailed information on several deep learning techniques that can be deployed to detect COVID-19. Apart from providing details of the different models, one of the main contributions of the paper is the discussion of the key concepts, such as artificial intelligence, machine learning, and deep learning. To help develop a better understanding of the applications of deep learning approaches, the paper starts by providing a detailed discussion of the structure and functionality of the different CNN architectures. This paper discusses the CNNs structure and the training of CNN and provides insights into several pre-trained CNN models that have been developed to detect COVID-19. In addition, the paper brings to light some of the latest developments in the application of deep learning techniques, which depend on the use of X-ray images drawn from medical imaging samples to facilitate the detection of COVID-19. The analysis of each model provides informations such as the experimental data used, the data splitting techniques applied and the suggested architecture for the detection of COVID-19 as well as the different evaluation metrics.
This study establishes that research in the use of deep learning applications in the detection of COVID-19 is still in its infancy stage; hence, more studies should be developed to enhance knowledge and understanding of the best approaches that can be used to design models that work efficiently in the diagnosis of COVID-19. Based on the analysis of the studies conducted in the past, it can be seen that there are several challenges that need to be addressed when developing a robust and accurate COVID-19 diagnosis system. For instance, while these innovative deep learning models provide tangible benefits in terms of detecting COVID-19, one big challenge is that they require a large body of data to generate accurate results, otherwise there would be the problem of overfitting. Given that COVID-19 is relatively new, there is limited data to work with. In addition, the type and format of the readily available data can be very different from each other given that different hospitals and agencies apply different approaches for data collection. This, therefore, makes it imperative to employ data fusion approaches to ensure that a single coherent dataset is constructed. In this case, future research should focus on understanding different data fusion approaches.
In line with future research, exploring different data fusion methods for the task is suggested. The success of any deep learning model relies on the quality of the data as well as the performance of the learning algorithms used.
This study revealed that there exist several parameters that can impact the accuracy of the model. For instance, the number of layers, layer specification, number of epochs, the rate of learning, dropout layer, and batch size, among others, in particular, for customised architectures. In addition, the accuracy of the model is also affected by the number of classes in the classification.
An important consideration should be made to reconcile the gap between research and impact. It is evidently clear that several research and ideas for developing new deep learning models exist; however, it is important to focus on what happens at the implementation stage of any proposed model. Reality works differently, hence the need to be cautious when developing models to ensure that the desired results can be achieved. An important recommendation is that it is important to ensure that research is developed in collaboration with the practitioners. To ensure good progress, it is hoped that deep learning experts should take a proactive stance and work together with other professionals including radiologists and other medical experts who can help with advice and ideas on how to develop effective systems for the detection of COVID-19 infections, in particular, during the early stages of the disease or for measuring the level of impact of the infection.