Efficacy of early warning systems in assessing country-level risk exposure to COVID-19

Abstract COVID-19 has evolved as a pandemic causing unprecedented damages and disruptions to all spheres of life including healthcare, transportation, supply chains, education, and economy, among others. Pandemics are very low-probability events associated with deep uncertainty about the timing of such events and ensuing damages. National policy-makers generally rely on a set of risk indices associated with natural disasters and pandemics to assess the country’s vulnerability and strategy formulation for such rare events. This paper explores the efficacy of early warning systems (disasters and epidemics-based risk ratings) in predicting the country-level exposure to COVID-19. Utilizing three real data-sets reflecting the risk exposure of individual countries to disasters, epidemics, and COVID-19, we explore relations among the associated risk dimensions, namely hazard and exposure, vulnerability, and lack of coping capacity. A comprehensive methodology integrating Pearson’s correlation, ANOVA, and Bayesian Belief Networks-based techniques is adopted to explore and triangulate relations among the three risk indices. Results show that the risk ratings associated with epidemic risk and COVID-19 risk are statistically strongly correlated. However, only the vulnerability dimension of epidemic risk significantly influences the two risks.


Introduction
COVID-19 has proved to be a devastating pandemic in a very short period (Cardil and de-Miguel 2020). It has disrupted operations globally across manufacturing and service industries such as logistics, aviation, healthcare, education, food supply chains, among others (Bruinen de Bruin et al. 2020;Haas 2020). Since catastrophic events such as COVID-19 are considered as very rare events involving deep uncertainty about their occurrence timing and severity, national policy-makers have limited choices to adopt proactive measures for pandemics (Haghani et al. 2020). Nonetheless, available tools and techniques in the realm of risk analysis can still help policy-makers understand the uncertainty surrounding such impactful events and make risk-informed decisions (Aven 2012(Aven , 2016(Aven , 2017. Several studies recommend to adopt appropriate risk mitigation strategies after gaining some clarity on the severity of COVID-19 (Haghani et al. 2020;Bruinen de Bruin et al. 2020;Sangiorgio and Parisi 2020).
Several early warning indicators (e.g. risk ratings) (Scherzer et al. 2019;Haque et al. 2020) are introduced for establishing the country-level exposure to rare risk events such as natural disasters and pandemics. In this context, risk is usually conceptualized as a multi-dimensional construct including the hazard and exposure, vulnerability, and lack of coping capacity dimensions (Ryan et al. 2020). INFORM is an international agency pioneering in the assessment of country-level risk exposure to disasters and epidemics (Haque et al. 2020). INFORM's early warning indicators (risk ratings) capture the vulnerability of countries to humanitarian crises and disasters, and epidemics (INFORM 2020a). INFORM has recently published a new risk rating scheme for assessing the exposure of individual countries to COVID-19 (INFORM 2020b).
INFORM conceptualizes disaster and humanitarian crises risk as a three-dimensional construct comprising hazard and exposure (an event that could happen and the associated exposure of that event), vulnerability (the susceptibility of communities to hazards), and lack of coping capacity (lack of resources that can aggravate the impact of hazards) (Haque et al. 2020;Islam et al. 2013). Risk ratings are assigned to countries on a continuous scale of 1-10, with 1 representing the least amount of risk and 10 the maximum amount of risk, based on multi-dimensional factors related to health system, natural disasters, humanitarian crises, infrastructure, development, socio-economic conditions, conflicts and wars, communication, governance, disaster risk reduction, and others (Haque et al. 2020). Similarly, INFORM has assigned risk ratings to countries based on their exposure to epidemics to indicate the preparedness of countries in dealing with epidemics. Recently, INFORM has introduced the COVID-19 risk index, which identifies 'countries at risk from health and humanitarian impacts of COVID-19 that could overwhelm current national response capacity, and therefore lead to a need for additional international assistance' (INFORM 2020b).
Utilizing the INFORM-based framework, Haque et al. (2020) assessed the disaster risk exposure within Bangladesh relative to five discrete levels of risk exposure. Similarly, Halkia et al. (2020) evaluated the efficacy of the Global Conflict Risk Index (GCRI) in predicting global conflicts and investigated the validity of the GCRI model. Noy and Yonson (2018) argue that besides the characteristics of hazards, socio-economic factors significantly influence the vulnerability of communities to disasters. For a comprehensive overview of the literature on disaster risk management, interested readers may consult Ryan et al. (2020) and Rus et al. (2018).
Early warning systems related to disasters and epidemics can significantly reduce the devastating impact of such catastrophic events (Phillips et al. 2020). According to Fearnley and Dixon (2020, p.1): 'It is too late to develop a cross-border, standardized early warning system for the first wave of COVID-19, but it is vital that a forensic analysis on how this crisis emerged includes an assessment of the variable successes in warning systems adopted by countries'. However, it remains unanswered whether existing early warning indicators (risk ratings related to disasters and pandemics) can reasonably predict the country-level exposure to  Several studies have investigated the impact of COVID-19 on businesses and society (Bruinen de Bruin et al. 2020;Haghani et al. 2020;Pluchino et al. 2021;Xuan Tran et al. 2020). Others have explored the assessment of COVID-19 risk and the efficacy of risk mitigation strategies adopted at a national level. For instance, Xuan Tran et al. (2020) investigated the capacity of local authority and community in Vietnam in responding to COVID-19. Sangiorgio and Parisi (2020) introduced a multi-criteria approach for the risk assessment of COVID-19 using Artificial Neural Networks. Bruinen de Bruin et al. (2020) classified COVID-19 risk mitigation strategies into six categories, namely mobility restrictions, socio-economic restrictions, physical distancing, hygiene, communications, and international support mechanisms.
The main limitation of the COVID-19-related research reported so far is its fragmentation across diverse disciplines such as the safety of treatments, financial losses, social safety, food security and reliability of supply chains (Haghani et al. 2020). Further, the efficacy of early warning risk indicators in predicting the actual risk exposure of countries to COVID-19 has not been investigated while holistically covering multi-dimensional factors related to hazard and exposure, vulnerability, and lack of coping capacity. Such an assessment can help policy-makers adopt suitable strategies through a robust risk mitigation process, which is defined as 'an interdisciplinary decision-making process based on information from risk and exposure assessment. It entails consideration of political, socioeconomic, epidemiological, (mental) health and engineering data to compile regulatory options and select the appropriate regulatory, societal, sector or company response to COVID-19' (Bruinen de Bruin et al. 2020, p.4).
In this paper, we aim to address the gaps mentioned above and explore the predictive ability of existing early warning risk indicators in assessing the actual risk exposure of countries to COVID-19. We explore a holistic approach to establish relations among early warning risk indicators and COVID-19 risk ratings such that multiple dimensions of risk associated with disasters and epidemics-based indicators and COVID-19 are integrated in a probabilistic network setting. The findings of this study can provide useful insights to policy-makers for making contingency plans, which must be developed holistically to 'improve prevention, preparedness, mitigation, response and rehabilitation to new emergency events' (Cardil and de-Miguel 2020). The remainder of the paper is organized as follows: The research methodology is described in Section 2. The results are presented in Section 3. We discuss the implications of our study in Section 4 and present conclusions and directions for future research in Section 5.

Materials and methodology
Three data-sets published by INFORM (INFORM 2020a), representing the countrylevel risk exposure of 191 countries associated with disasters, epidemics and COVID-19, were analyzed to test the following hypotheses: Hypothesis 1: The country risk ratings on humanitarian crises and disasters, and epidemics are associated with the prediction of the country-level exposure to COVID-19.

Hypothesis 2:
The ratings of all three risk dimensions (hazard and exposure, vulnerability, and lack of coping capacity) on humanitarian crises and disasters, and epidemics are associated with the prediction of country-level exposure to COVID-19.
Hypothesis 3: Low (high) ratings of the individual risk dimensions on the country-level exposure to humanitarian crises and disasters, epidemics, and COVID-19 are associated with the prediction of low (high) ratings for all three risk categories.
The risk ratings related to humanitarian crises and disasters, and epidemics can be considered as an early warning indicator of the vulnerability of individual countries to an extremely rare event such as COVID-19. Since INFORM has published the country-level ratings for all three risk categories, it was considered appropriate to utilize the same three data-sets. Multi-dimensional factors associated with the three datasets, representing the country-level exposure of countries to individual risk categories, are presented in Table 1. Although there are common factors indicated across the three risk categories, their focus might change depending on the nature of a specific risk. For instance, lack of coping capacity includes a different set of components for COVID-19 risk and epidemic risk (see Table 1) and it specifically focuses on the country-level preparedness and response status for COVID-19 in the case of COVID-19 risk (INFORM 2020b). The statistics for all risk categories and associated risk dimensions are presented in Table 2.
The INFORM model utilizes 54 different indicators to establish the overall risk rating for individual countries (Marin-Ferrer et al. 2017). These indicators, such as the Gender Inequality Index and government efficiency, are developed by reliable international organizations and academic institutes, including World Bank, World Health Organization, Heidelberg Institute for International Conflict Research, and others . We adopted three different statistical techniques, namely Pearson's correlation, oneway Analysis of Variance (ANOVA), and Bayesian Belief Networks (BBNs), to explore the research hypotheses for this study. Correlation analysis and one-way ANOVA were considered appropriate due to the nature of the hypotheses (Dikmen et al. 2018) as there was a need to investigate correlations among the data-sets and establish whether the three risk ratings were significantly influenced by individual risk dimensions. Bayesian Belief Networks were utilized to explore the interaction effects of multiple factors in a network setting (Hanea et al. 2015), which cannot be achieved using one-way ANOVA and correlation analysis (Kabir and Papadopoulos 2019;Sigurdsson et al. 2001).
Using the three data-sets by INFORM, we developed a discrete BBN model in GeNIe 2.0, which is a software for developing and analyzing BBN models. The development of the model involved several sequential steps. First, the data were discretized, and discrete states were established for each variable indicated in Table 2. A uniform-width discretization scheme was adopted for all variables and three states were introduced for each variable, namely low (0-3.33), medium (3.33-6.67), and high (6.67-10). Second, three different algorithms were used to develop the network structure of the model, namely the Bayesian Search, PC, and Greedy Thick Thinning algorithms. For details about the mechanics of BBNs and algorithms used in BBN models, interested readers may consult Jensen and Nielsen (2007), Hossain et al. (2020), Kameshwar et al. (2019), and Zabinski et al. (2018). The third step involved the learning of the parameters for the three models using the three data-sets. Finally, the three models were validated using the cross-validation scheme available in GeNIe.
The prediction accuracy of the three models developed is presented in Table 3. The Greedy Thick Thinning algorithm-based model was selected due to its superior prediction ability (see Figure 1). The model comprises nodes, representing the three risk categories and associated risk dimensions, and arcs reflecting statistical dependencies among interconnected variables (Cox et al. 2018). The probability distribution associated with each variable can be inferred from the model. For instance, 4.3% of the countries included in the data are associated with a high exposure to COVID-19 risk.

Results
The first two hypotheses were tested using Pearson's correlation analysis (see Table  4). All the correlations were found to be statistically significant at a significance level of 0.01, except those highlighted in Table 4. Epidemic risk was found to be strongly correlated (0.91) with COVID-19 risk, whereas disaster risk was moderately correlated (0.39) with COVID-19 risk and epidemic risk (0.40). Therefore, only epidemic risk ratings could reasonably predict the exposure of countries to COVID-19 risk. However, there were some differences noted in the INFORM ratings associated with epidemic risk and COVID-19 risk for certain countries. For instance, China and Saudi Arabia are indicated as medium-risk countries relative to epidemic risk, whereas their rating is low relative to COVID-19 risk. Similarly, Lebanon's risk rating is medium and high relative to epidemic risk and COVID-19 risk, respectively, whereas it is the opposite in the case of Turkey. Hazard and exposure for epidemic risk was moderately correlated (0.60) with that for COVID-19 risk. In contrast, the hazard and exposure-based correlation was found to be non-significant between disaster risk and COVID-19 risk. Similarly, the correlation was weak (0.28) between disaster risk and epidemic risk. In the case of vulnerability, the epidemic risk was moderately correlated (0.56) with COVID-19 risk, and the correlation between disaster risk and COVID-19 risk was found to be moderate (0.36) as well. Similarly, a moderate correlation (0.46) was observed between epidemic and disaster risks. For lack of coping capacity, a very strong correlation (0.95) was found between epidemic risk and COVID-19 risk, whereas a moderate correlation (0.45) was observed between disaster risk and COVID-19 risk. Similarly, a moderate correlation (0.47) was found between disaster risk and epidemic risk.
Hypothesis 3 was tested using ANOVA, which was performed in IBM SPSS Statistics 26. The ratings assigned to all variables were determined to be normally distributed using the Q-Q plot. Subsequently, the homogeneity of variances was examined by performing Levene's test at the 5% significance level. The homogeneity of variances was violated in all cases except the hazard and exposure related to epidemics. Therefore, the Welch-Satterthwaite correction (Dikmen et al. 2018) was used and the results were obtained (see Table 5). The impact of the change in the state of each variable was found to be statistically significant on the variation in the mean ratings for all three risk categories except in the case of COVID-19-related vulnerability and disaster risk, and disasterrelated hazard and exposure and COVID-19 risk.
Games-Howell post hoc method (Seidl et al. 2013) was used to identify the major differences in the risk ratings relative to different states of risk dimensions. The difference in the disaster risk ratings relative to the low and medium states of epidemics-related hazard and exposure was found to be non-significant. Similarly, the difference in the disaster risk ratings relative to the low and medium states of epidemics-related lack of coping capacity was found to be non-significant. The differences in the epidemic risk ratings related to all three states of disaster-related hazard and exposure were found to be statistically non-significant. Further, similar non-    significant results were found in the case of the association between disaster-related vulnerability and each of COVID-19 risk and epidemic risk. Hypothesis 3 was also tested using the BBN model developed (see Figure 1). The main difference between the BBN-based analysis and one-way ANOVA is the ability of BBNs to capture multiple interactions across all variables in the model developed (see Figure 1). The mean and 'high' state probability values for all variables are presented in Table 6. Based on the mean values, epidemics-related vulnerability and disasters-related vulnerability are identified as the most and least critical variables, respectively. However, while considering the 'high' state probability values, disastersrelated lack of coping capacity and COVID-related vulnerability appear to be the most and least critical variables, respectively.
Countries associated with the high and low exposure of each of epidemic risk, COVID-19 risk, and disaster risk were analyzed for their corresponding assessment of other variables. The mean and probability value of 'high' state were assessed for individual variables given the change in the extreme states of individual risks. Subsequently, the relative importance of individual variables was established based on the change in values relative to the two extreme states of individual risks (see Tables 7-9). Epidemics-related hazard and exposure was identified as the most critical variable influencing epidemic risk, whereas COVID-related vulnerability was considered as the least important variable. Similarly, disasters-related hazard and exposure was evaluated as the most critical variable influencing disaster risk, whereas COVID-related vulnerability was considered as the least important variable. On the other hand, COVID-related lack of coping capacity was identified as the most critical variable influencing COVID-19 risk, whereas COVID-related vulnerability was considered as the least important variable.
The relative importance of individual risk dimensions was established relative to their impact on the three risk categories given the realization of their extreme states (see Table  10). Epidemics-related vulnerability significantly influenced both epidemic risk and COVID-19 risk, whereas disaster risk was mainly influenced by its vulnerability dimension. It is interesting to note that while all dimensions influence their respective risk categories in the case of epidemic risk and disaster risk, the influence of COVID-19-related risk dimensions is relatively higher on epidemic risk compared to COVID-19 risk. The relative importance of individual variables was further analyzed using the Spearman's rank correlation test (see Table 11). The ranking schemes relative to the importance of variables with respect to COVID-19 risk and epidemic risk were found to be strongly correlated (0.98). However, the correlation between disaster-related and each of COVID-19 risk and epidemic-related risk ranking schemes was found to be statistically non-significant at a significance level of 0.05, implying that the relative importance of variables is quite different in these cases.

Discussion
Hypothesis 1 was tested using Pearson's correlation analysis. The correlation between epidemic risk and COVID-19 risk was found to be significantly strong, whereas disaster risk was moderately correlated with the other two risks. Although all three risks shared few common variables influencing the vulnerability and lack of coping  capacity dimensions (see Table 1), disaster risk was not mainly focused on epidemics regarding the scope of hazard and exposure. Hypothesis 2 was also tested using Pearson's correlation analysis. Surprisingly, among the three risk dimensions, only lack of coping capacity-based ratings were found to be strongly correlated between epidemic risk and COVID-19 risk. Therefore, although epidemic risk ratings can fairly represent the country-level COVID-19 risk exposure, these ratings fail to predict the risk exposure across multiple dimensions of COVID-19 risk except lack of coping capacity. Hence, early warning risk indicators may provide limited information to policy-makers for prioritizing factors influencing critical dimensions of COVID-19 risk as such indicators fail to capture events involving deep uncertainty.
Hypothesis 3 was tested using ANOVA and BBNs. While ignoring the interaction effects among variables, the impact of the change in the state of each variable (risk dimension) was generally found to be statistically significant on the variation in the mean ratings for all three risk categories. However, this change was not always significant across all three states of individual risk dimensions. For instance, sometimes, the change relative to the 'low and medium' and 'medium and high' states was found to be statistically non-significant. Therefore, this might pose a challenge in ascertaining the relative importance of individual variables while considering different levels of country-level risk exposure. Further, the synergistic effects of multiple interactions among variables were ignored in this analysis.
Contrary to the results based on ANOVA, BBNs provided useful insights while capturing the probabilistic interactions among variables. While focusing on epidemics-based low-and high-risk countries, hazard and exposure was identified as the most critical variable influencing epidemic risk. Similarly, disasters-related hazard and exposure was evaluated as the most critical variable influencing disaster risk. However, the lack of coping capacity was identified as the most critical risk dimension relative to the assessment of COVID-19-based country-level risk exposure.
While capturing the network-wide impact of individual risk dimensions relative to all three risk categories, epidemics-related vulnerability significantly influenced both epidemic risk and COVID-19 risk, whereas disaster risk was mainly influenced by its vulnerability dimension. Therefore, not all three dimensions of risk were found to be equally important in the case of individual risk categories. Further, the influence of COVID-19-related risk dimensions was found to be relatively higher on epidemic risk compared to COVID-19 risk. Therefore, the epidemics-based risk assessment framework may benefit from integrating unique factors from the COVID-19-based risk assessment framework. The ranking schemes relative to the importance of individual risk dimensions influencing all three risk categories provided useful insights. For instance, the ranking schemes relative to the importance of variables with respect to COVID-19 risk and epidemic risk were found to be strongly correlated. This implies that the same set of variables may be prioritized for both epidemic and COVID-19 risks. However, the correlation between disaster-related and each of COVID-19 risk and epidemic-related risk ranking schemes was found to be statistically non-significant implying that the relative importance of risk dimensions is quite different in these cases.
This study makes several contributions to the literature on risk management related to disasters and pandemics. First, three real data-sets encompassing multidimensional factors associated with the assessments of disaster, epidemic and COVID-19 risks are consolidated and analyzed for relations among multiple risk ratings and associated risk dimensions. Second, to the best of the authors' knowledge, a  comparison between the pre-COVID-19 risk ratings and the actual COVID-19 risk ratings has never been explored. Third, a probabilistic network-based model is introduced to capture complex interactions among multi-dimensional factors. This study provides the research community in risk analysis with a novel research theme. Researchers can explore the efficacy of other early warning systems in establishing the actual exposure of risk associated with devastating events such as COVID-19. Other risk ratings and data-sets can be explored to identify the association between underlying factors and establish the predictive ability of existing risk assessment frameworks. Available artificial intelligence tools and techniques may be investigated to enhance the limited capability of existing early warning systems. Further, various risk ratings associated with disasters and epidemics may be consolidated for predicting events involving deep uncertainty.
This study provides useful insights to policy-makers and practitioners in risk management. Policy-makers can realize the limitations of existing risk assessment frameworks and risk indicators in projecting the realistic exposure of rare disasters such as COVID-19. Although the existing risk ratings can reasonably predict the overall country-level exposure to COVDI-19, these fail to provide insights into the relative importance of specific risk dimensions and underlying factors. The BBN-based analysis indicates the relative importance of individual risk dimensions specific to the country-level risk exposure. Further, policy-makers can develop the model presented in this study to explore the interactions among factors influencing risk dimensions across all three risk categories. Subsequently, limited resources can be allocated to the three dimensions of COVID-19 risk in proportion to their relative importance.

Conclusions
The main purpose of this paper was to establish the efficacy of early warning indicators in predicting the actual risk exposure of individual countries to COVID-19. In particular, the association between the pre-COVID-19 disasters and epidemics-based risk ratings and the actual exposure of COVID-19 risk was explored while considering all three dimensions of risk. Utilizing real data by INFORM and a combination of Pearson's correlation analysis, one-way ANOVA, and BBNs, associations among the risk dimensions across all three risk categories were analyzed. This study provides useful insights to policy-makers regarding the efficacy of proactive risk management measures such as early warning indicators (risk ratings) in projecting the actual exposure of COVID-19. Although such early warning indicators provide useful information about the actual risk exposure of rare events such as COVID-19, these indicators fail to project the actual exposure associated with risk dimensions and underlying factors.
This study has certain limitations. Multi-dimensional factors associated with individual risk dimensions were not captured in the analyses. Risk rating data were used from a single source (rating agency). The dynamic behavior of risk was not analyzed. The sensitivity of results to the variation in the discretization scheme of variables was not performed. Future research may benefit from validating the model developed and comparing the results using data from other sources. A comprehensive model may be developed to integrate complex interactions across factors that influence multiple dimensions of risk. Dynamic BBN models may be developed to explore the dynamic nature of country-level COVID-19 risk. A sensitivity analysis may be performed to establish the sensitivity of results to the choice of the discretization scheme for individual risks and risk dimensions.

Funding
This publication is supported by the American University of Sharjah.

Disclosure statement
No potential conflict of interest was reported by the authors.

Data availability
The data that support the findings of this study are available from the corresponding author upon request.