The predictive value of the Oxford Acute Severity of Illness Score for clinical outcomes in patients with acute kidney injury

Abstract Objective To compare the performance of the Oxford Acute Severity of Illness Score (OASIS), the Acute Physiology and Chronic Health Evaluation II (APACHE II) score, the Simplified Acute Physiology Score II (SAPS II), and the Sequential Organ Failure Assessment (SOFA) score in predicting 28-day mortality in acute kidney injury (AKI) patients. Methods Data were extracted from the Beijing Acute Kidney Injury Trial (BAKIT). A total of 2954 patients with complete clinical data were included in this study. Receiver operating characteristic (ROC) curves were used to analyze and evaluate the predictive effects of the four scoring systems on the 28-day mortality risk of AKI patients and each subgroup. The best cutoff value was identified by the highest combined sensitivity and specificity using Youden’s index. Results Among the four scoring systems, the area under the curve (AUC) of OASIS was the highest. The comparison of AUC values of different scoring systems showed that there were no significant differences among OASIS, APACHE II, and SAPS II, which were better than SOFA. Moreover, logistic analysis revealed that OASIS was an independent risk factor for 28-day mortality in AKI patients. OASIS also had good predictive ability for the 28-day mortality of each subgroup of AKI patients. Conclusion OASIS, APACHE II, and SAPS II all presented good discrimination and calibration in predicting the 28-day mortality risk of AKI patients. OASIS, APACHE II, and SAPS II had better predictive accuracy than SOFA, but due to the complexity of APACHE II and SAPS II calculations, OASIS is a good substitute. Trial Registration This study was registered at www.chictr.org.cn (registration number Chi CTR-ONC-11001875). Registered on 14 December 2011.


Introduction
Acute kidney injury (AKI) is a common and serious complication in intensive care unit (ICU) patients, and it is an important risk factor for increased early and longterm morbidity and mortality during hospitalization [1][2][3][4]. Early identification and diagnosis, correct assessment of prognosis, and active treatment are the keys to reducing the mortality rate. Many severity scores have been developed to evaluate the prognosis of the critically ill patients, including AKI patients.
The Acute Physiology and Chronic Health Evaluation II (APACHE II) score is the most commonly used disease severity scoring system in ICUs around the world [5]; it includes 12 physiological and laboratory parameters and two disease-related variables [6]. The Simplified Acute Physiology Score II (SAPS II) was first described in 1984 as an alternative to the APACHE scoring system [7], and it is an effective tool for evaluating AKI patient outcomes [8,9]. However, all of the above models require considerable effort for data collection. Although the Sequential Organ Failure Assessment (SOFA) score [10] is simple to use and accurate in predicting the mortality outcome of AKI patients [11][12][13], it depends on laboratory results, and some important prognostic factors were not included.
In 2013, Johnson et al. performed a retrospective cohort study of 72 474 ICU patients in 68 ICUs at 49 U.S. hospitals from 2007 to 2011 and developed a new reduced severity of illness score using machine learning algorithms, the Oxford Acute Severity of Illness Score (OASIS), which contained 10 parameters without any laboratory tests and had discrimination and calibration equivalent to more complex existing models, the highest score is 75 [14].
The predictive value of OASIS was validated in mixed ICU patient populations, but its performance in AKI patients remains unknown. The aim of this multicenter study was to evaluate the performance of OASIS for the assessment of mortality in AKI patients in China, and compare with APACHE II, SAPS II, and SOFA.

Study setting and data collection
This study used a database from the Beijing Acute Kidney Injury Trial (BAKIT) [15], a prospective, multicenter, observational study that investigated the epidemiology of acute kidney injury (AKI) in critically ill patients in 30 ICUs at 28 tertiary hospitals in Beijing, China, conducted between 1 March and 31 August 2012 (for a complete list of these hospitals and the persons responsible for the data acquisition, see Additional file 1). The study subjects included all adult patients (age ! 18 years) admitted consecutively to the ICUs. Only the initial ICU admission was considered in this study. The following patients were excluded: patients with preexisting end-stage chronic kidney disease, patients already receiving renal replacement therapy (RRT) before admission to the ICU, and patients who had received kidney transplantation in the previous three months. Preexisting comorbidities were diagnosed based on the International Classification of Diseases (ICD-10) codes. The patients were followed up until death, until hospital discharge, or for 28 days.
Thorough follow-up of all patients included in the study was conducted in the first 10 days after ICU admission. The collected data included demographics, anthropometrics, admission diagnosis, comorbidities, daily vital signs and laboratory data (which were used to automatically calculate the APACHE II score, the SAPS II, and the SOFA score), days from hospital to ICU admission, ICU length of stay (LOS), hospital LOS, use of vasoactive drugs, the occurrence of AKI, and length of mechanical ventilation (MV). RRT data were also reported.
Mortality data up to 28 days after ICU discharge were collected from hospital records, including records from hospital admissions and visits to outpatient clinics.
AKI was defined and classified according to the Kidney Disease Improving Global Outcomes (KDIGO) guidelines [16]. Patients were categorized on the basis of serum creatinine (SCr) or urine output or both. Baseline creatinine was defined as the lowest known SCr value in the last three months [17]. For patients without baseline creatinine, we used the estimated baseline creatinine or the lowest SCr in the ICU course, whichever was lower. The baseline creatinine was estimated by the Modification of Diet in Renal Disease (MDRD) equation [18], assuming a glomerular filtration rate of 75 mL/min/1.73m 2 [19].
We calculated the OASIS within the first day of ICU admission. The parameters used to calculate the OASIS are shown in Table S1.

Outcomes
The primary outcome was 28-day mortality, and the secondary outcomes were ICU mortality and hospital mortality. The ICU LOS and hospital LOS were calculated only for statistical description. ICU mortality and ICU LOS were determined by the first ICU stay only.

Statistical analysis
Nonnormally distributed continuous variables are expressed as the medians with interquartile ranges (IQRs) and were compared using the Mann-Whitney U test or Kruskal-Wallis analysis-of-variance test with Bonferroni correction. Categorical variables are expressed as the number of cases and proportions and were compared using the Mantel-Haenszel Chi-square test.
Receiver operating characteristic (ROC) curves were drawn according to the sensitivity and specificity of the four scoring systems for predicting the 28-day mortality risk of patients. The ROC curve comparison function of Medcalc software was used for pairwise comparisons of the area under the curve (AUC), the larger the AUC, the higher the predictive value. AUCs of ! 0.9, 0.8 to 0.89, 0.7 to 0.79, 0.6 to 0.69 or < 0.6 were classified as excellent, very good, good, fair, and poor, respectively.
Cutoff values, sensitivities, specificities, positive predictive values, and negative predictive values were calculated by ROC analysis. The best cutoff values for the prediction of 28-day mortality, ICU mortality, and hospital mortality were determined by the maximum of the Youden index (i.e., sensitivity plus specificity minus one) calculated from the ROC analysis. The Hosmer-Lemeshow goodness-of-fit test was used to test the calibration of the scoring system.
We used a logistic regression model to evaluate the effect of OASIS on the 28-day mortality in AKI patients. Because OASIS was collinear with APACHE II and SAPS II scores, the variables considered for multivariable analysis included age, sex, OASIS, SOFA, use of vasoactive drugs, MV, RRT, and underlying diseases. OASIS was entered as a continuous variable and a categorical variable, respectively.
To verify the predictive effect of OASIS on the 28-day mortality of patients with different AKI grades and among different populations of AKI patients, subgroup analyses were performed by ROC analysis.
All statistical analyses were performed using SPSS software (IBM Corp., Statistics for Windows, version 22.0, Armonk, NY, USA), with a two-sided p values < .05 considered statistically significant.

Study population
During the study period, 9079 patients were admitted consecutively. Of them, patients were excluded because of the following reasons: 5725 patients had an ICU LOS of less than 24 h, 110 patients were younger than 18 years old, one patient received renal transplantation during the past three months, 95 patients had received RRT before admission to the ICU, and 11 had insufficient clinical recordings. Thus, 3107 patients were enrolled in the BAKIT study. Of these patients, 194 were excluded because of incomplete data for calculating OASIS, and finally, 2954 patients were included in our study ( Figure 1).
There were statistically significant differences in age, MV, sepsis, AKI, RRT, OASIS, APACHE II, SAPS II, SOFA, admission category, ICU LOS, and hospital LOS between survivors and non-survivors.
Comparison of characteristics between the survival and non-survival groups of AKI patients AKI patient characteristics according to 28-day mortality are shown in Table 2. Non-surviving AKI patients were older (p < .001), had higher illness severity scores, and were more likely to be diagnosed with sepsis. Positive  Figure 1. Study flow chart with 28-day mortality rate.  fluid balance in the first 24 h was more common among non-survivors.

The 28-day mortality of AKI patients according to OASIS
The distribution of OASIS in AKI patients is shown in Figure 2. OASIS ranged from 6 to 64, and the median value was 31 (IQR: 24, 39). The distributions of the OASIS with corresponding 28-day mortality are also presented in Figure 2. As each score increased, the 28day mortality of AKI patients increased accordingly, indicating more serious illness and worse prognosis.
Comparison of ROC curve and AUCs of the four scoring systems in evaluating the 28-day mortality of AKI patients In Figure 3, OASIS had the highest discriminatory power in predicting the prognosis of AKI patients. , respectively, which were higher than that of SOFA (0.686; p < .001). Table 3 shows the pairwise comparison of the ROC curves, and there were no statistically significant differences between the AUC values of OASIS, APACHE II, and SAPS II in predicting 28day mortality.
The predictive ability of OASIS, APACHE II, SAPS II, and SOFA score for poor outcomes The ROC curves for the prediction of 28-day mortality, ICU mortality, and in-hospital mortality by each severity scale are shown in Table 4. The sensitivities, specificities, positive predictive values, and negative predictive values of the optimal cutoff values (from the Youden  index) for each scale to predict the three outcomes are listed in Table 5. The cutoff value of OASIS for the prediction of 28-day mortality was 33 with a sensitivity of 87.75% and specificity of 46.26%, as calculated by the ROC curve analysis. OASIS ! 33 predicts poor shortterm prognosis in patients with AKI.

Logistic regression analyses of 28-day mortality in AKI patients
Logistic regression model was used to test the efficacy of OASIS in predicting 28-day mortality in patients with AKI (

Subgroup analyses
According to the KDIGO criteria, AKI patients were divided into stage 1, stage 2, and stage 3. Patient characteristics by AKI stage are shown in Table S2. The ROC curves of OASIS, APACHE II, SAPS II, and SOFA score for predicting of 28-day mortality in each subgroup are shown in Table 7. OASIS had a good predictive effect in each subgroup. Table 7 shows the calibration of the risk scores. OASIS had good calibration in each subgroup, except the stage 3 subgroup. To verify the predictive effect of OASIS on the 28-day mortality among different populations of AKI patients, we divided the AKI patients into elective surgery, non-elective surgery, MV, non-MV, Sepsis, non-sepsis, RRT, non-RRT, over 65 years, and up to 65 years groups, as shown in Table S3. OASIS had a good prediction effect in most subgroups.

Discussion
In this large, multicenter prospective study, we evaluated the ability of the OASIS, APACHE II, SAPS II, and SOFA score to predict the 28-day mortality in AKI patients, and we found that the performance of OASIS was the best, followed by APACHE II and SAPS II, but there were no significant differences among the three scoring systems. The predictive value of the SOFA score was the worst, and the difference was statistically significant compared with the other three scores. OASIS   was significantly associated with a higher risk of death in the logistic regression model, whether as a continuous variable or a categorical variable, which further indicated that OASIS had good value in judging the severity of AKI patients. OASIS has been studied in the mixed ICU [20], in the cardiac ICU [21][22][23], in patients with sepsis [24][25][26][27][28], and in patients admitted to the surgical intensive care unit (SICU) [29], but to date, there has been no study on OASIS in AKI patients. Deliberato et al. performed a secondary analysis of the electronic health records of patients included in the eICU Collaborative Research Database (eICU-CRD), 108 402 patients in 189 different ICUs across the USA were included in the analysis [20]. In this study, underweight patients had higher OASIS scores, the median value was 31 (IQR: 25,38), and OASIS demonstrated good discrimination (AUC ¼ 0.79 (0.78-0.80)) in predicting in-hospital mortality in all body mass index (BMI) subgroups. Our study showed similar results, the median value was 31 (IQR: 21, 39), and the performance of OASIS in predicting in-hospital mortality was good (AUC ¼ 0.783 (0.756-0.811)).
Recently, Hu et al. [26] collected the data of 2470 sepsis patients recorded in the Medical Information Mart for Intensive Care III (MIMIC-III) database [30] from 2001 to 2012 and retrieved the SOFA, SAPSII, OASIS and Logistic Organ Dysfunction System (LODS) scores [31] of the patients within the first day admission to the ICU and compared the predictive value of the four scoring systems for ICU mortality of the patients. The AUC values of SAPSII and OASIS were 0.768 (0.745-0.791) and 0.762 (0.738-0.785), respectively, which were significantly higher than those of the other two scoring systems. In our study, the AUC values of OASIS, APACHE II and SAPS II in predicting the 28-day mortality of AKI patients were 0.771 (0.742-0.799), 0.764 (0.735-0.792), and 0.767 (0.739-0.796), respectively, which were higher than that of the SOFA score (0.686; p < .001) (Figure 3). Although the subjects of the two studies were different, we both found that OASIS had good predictive value for mortality in ICU patients. In contrast, another study found that SAPS II (AUC ¼ 0.741 (0.703-0.778)) and SOFA score (AUC ¼ 0.687 (0.645-0.728)) showed significantly and slightly better discrimination than OASIS (AUC ¼ 0.684 (0.643-0.725)) [28]. More clinical studies are needed to investigate the validity of OASIS.
In our study, with the increase in OASIS, the mortality rate of patients increased (Figure 2). In AKI patients, OASIS of the non-survivors was higher than that of the survivors (39 vs 28, p < .001), which is consistent with the finding of another study (38 vs. 33, p < .001) [24], indicating that OASIS has a good predictive value for the 28-day mortality. Moreover, it has a good predictive value for both the ICU mortality and the in-hospital mortality, which is similar to the findings of the original study, hospital and ICU mortality increased exponentially as OASIS increased [14]. In contrast to that in the original study, the AUC value of OASIS in our study (AUC¼ 0.771) was significantly lower than that of the former (AUC ¼ 0.902), but significantly higher than that of Chen et al. (AUC ¼ 0.652) [24]. The reasons are as follows: first, the original research was performed in a mixed ICU, and the other study was conducted in septic patients, while our subjects were AKI patients. Second, the original research admitted 72 474 ICU patients in 68 ICUs at 49 U.S. hospitals, the other study was conducted using data from a public database, and a total of 10 305 septic patients were included. The large sample size of the studies reduced selection bias and made the results more convincing. Third, the above two studies were retrospective cohort studies, while ours was a prospective observational study. Retrospective studies are prone to confusion and bias.
ROC curve analysis showed that the cutoff value of OASIS for predicting 28-day mortality in AKI patients was 33, OASIS had the highest sensitivity (87.75%) for predicting 28-day mortality, but a lower specificity (46.26%). In another study, the best threshold of OASIS was 34.5, with a specificity of 55.80% and a sensitivity of 64.93% [24]. We divided AKI patients into three subgroups according to the KDIGO criteria, OASIS increased with the increase of the AKI classification (Table S2); Table 7 shows that OASIS had a good predictive value in each subgroup. We also divided AKI patients into groups based on their characteristics. Other studies also grouped subjects to determine the predictive value of OASIS, for example, septic patients were grouped according to age [25], cardiac intensive care unit (CICU) populations were grouped by admission diagnosis [21], and ICU patients were grouped by BMI [20]. Subgroup analysis may be more predictive of patient outcomes. In addition, OASIS includes elective surgery, which is not included in other scores. There were 1459 (49.4%) elective surgery patients in our study, so it may be more meaningful to use OASIS to evaluate their prognosis.
There were some limitations in the present study. First, we did not consider factors such as the etiology, duration, and whether RRT was used for AKI, which might affect the predictive power of OASIS. Second, although this was a prospective study, OASIS was not included in the study design, resulting in incomplete OASIS data for some patients. Third, all AUC values were less than 0.8, indicating that the four risk scores may be inaccurate for AKI patients, which prompt further prospective studies and the development of new scales in this population. OASIS is not widely used at present, probably because it is simpler than APACHE II and SAPS II, but more complex than the SOFA score. Moreover, the values used are all the worst ones selected from the daily minimum and maximum values. If the data records of patients are incomplete, the application of OASIS will be limited. At present, most studies on OASIS are retrospective studies [14,20,22,23,[25][26][27][28][29]; therefore, large-scale prospective studies are needed to further verify the predictive value of OASIS.

Conclusion
Because of the simplicity and effectiveness of OASIS, this study recommends the use of OASIS to evaluate 28-day mortality in patients with AKI admitted to the ICU. OASIS ! 33 should be considered an indicator of a negative short-term outcome.

Statement
Our manuscript was published as a preprint on 28 June 2021. DOI: https://doi.org/10.21203/rs.3.rs À640541/v1. The website is Research Square. We declare that this manuscript has not been formally published in part or in whole elsewhere, nor has it been considered by any other journal.