Nomogram for Prediction of fatal outcome in Patients with Severe COVID-19: A Multicenter Study

Background & Aims: To develop an effective model of predicting fatal Outcome in the severe coronavirus disease 2019 (COVID-19) patients. Methods: Between February 20, 2020 and April 4, 2020, consecutive COVID-19 patients from three designated hospitals were enrolled in this study. Independent high- risk factors associated with death were analyzed using Cox proportional hazard model. A prognostic nomogram was constructed to predict the survival of severe COVID-19 patients. Results: There were 124 severe patients in the training cohort, and there were 71 and 76 severe patients in the two independent validation cohorts, respectively. Multivariate Cox analysis indicated that age ≥ 70 years (HR 1.184, 95% CI 1.061-1.321), Panting(breathing rate ≥ 30/min) (HR 3.300, 95% CI 2.509-6.286), lymphocyte count < 1.0 × 10 9 /L (HR 2.283, 95% CI 1.779-3.267), and IL-6 >10pg/mL (HR 3.029, 95% CI 1.567-7.116) were independent high-risk factors associated with fatal outcome. We developed the nomogram for identifying survival of severe COVID-19 patients in the training cohort (AUC 0.900, [95% CI 0.841-0.960], sensitivity 95.5%, specicity 77.5%); in validation cohort 1 (AUC 0.811, [95% CI 0.763-0.961], sensitivity 77.3%, specicity 73.5); in validation cohort 2 (AUC 0.862, [95% CI 0.698-0.924], sensitivity 92.9%, specicity 64.5%). The calibration curve for probability of death indicated a good consistence between prediction by the nomogram and the actual observation. The prognosis of severe COVID-19 patients with high levels of interleukin-6 (IL-6) receiving tocilizumab was better than that of those patients without tocilizumab both in the training and validation cohorts, but without difference (p = 0.105 for training cohort, p = 0.133 for validation cohort 1, and p = 0.210 for validation cohort 2). Conclusions: This nomogram could help clinicians to identify severe patients who have high risk of death, and to develop more appropriate treatment strategies to reduce the mortality of severe patients. Tocilizumab may improve the prognosis of severe COVID-19 patients with high levels of IL-6.


Introduction
The outbreak of coronavirus disease 2019 (COVID-19) caused by severeacute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is a new pneumonia that has spread rapidly throughout the world [1]. Because of the rapid transmission spread of COVID-19, the number of new cases and death is increasing. COVID-19 has become a major public health crisis [1]. Previous studies have indicated that in all COVID-19 patients, the incidence of severe cases is approximately15% [2,3]. The mortality rate of severe COVID-19 patients varies from 8% to 61.5% and signi cantly increases among older patients [4][5][6][7][8][9].
Early medical intervention is very important to reduce the mortality of severe patients. Thus, it is of great importance to screen out severe patients with a high risk of death promptly and accurately at the initial admission [10]. However, this is particularly di cult because of limited medical resources and staff and the large number of patients. Therefore, elucidating the independent risk factors and establishing an accurate model for predicting severe COVID-19 patients at high risk of death is necessary. A previously established nomogram suggested ve prognostic factors for predicting the outcome: APACHE II, CK, CRP, IgA, and the interaction between CK and APACHE II [11]. However, risk factors associated with fatal outcomes in severe patients are unclear. This study aimed to provide a model to help clinicians identify patients with severe COVID-19 at high risk of death, which may be bene cial for decision making regarding treatment strategies.

Study population
Between February 20, 2020 and April 4, 2020, consecutive con rmed COVID-19 patients were assessed to enter into this study from three designated hospitals of COVID-19 in China: the Guanggu Branch of the Women and Children's Hospital of Hubei Province, Tongji TaiKang Hospital and Huoshen Mountain Hospital. The diagnosis of COVID-19 was based on the WHO interim guidance and guidelines for diagnosis and treatment of novel coronavirus pneumonia (5 th version) released by National Health Commission of China [12,13]. The presence of SARS-CoV-2 in respiratory specimens was con rmed by a positive result of quantitative real-time reverse transcriptase-polymerase-chain-reaction (qRT-PCR) assay from nasal or pharyngeal swab specimens. Swab samples were collected and tested for SARS-CoV-2 with the Chinese Center for Disease Control and Prevention (CDC) recommended Kit (BioGerm, Shanghai, China), following WHO guidelines for qRT-PCR [14][15][16]. All samples were tested for SARS-CoV-2 by use of qRT-PCR with the CDC recommended Kit. The test results were con rmed by nested RT-PCR with designed primers. The nested RT-PCR assay was performed according to previous report [17].
During the study period, a total of 2541 patients were enrolled into this study. Of 2541 cases included in this study, 1056 patients were from Guanggu Branch of the Women and Children's Hospital of Hubei Province, 726 were from Tongji TaiKang Hospital, and 759 were from Huoshen Mountain Hospital. According to the de nition of severe COVID-19 described above, there were 124, 71, and 76 severe cases from the three hospitals, respectively. Therefore, 124 patients with severe diseases from Guanggu Branch of the Women and Children's Hospital of Hubei Province were included in the training cohort, while 71 and 76 severe cases in other two hospitals formed validation cohort. The selection of the study population was shown in Supplemental Figure. 1.The study was approved by the Ethics Committee of all centers.
Written informed consent was waived by the Ethics Commission of each hospital for emerging infections.

Data Collection
All patients received chest computed tomography (CT) and serological examinations at admission. Laboratory tests included routine blood tests, liver function, renal function, coagulation pro le, C-reactive protein (CRP), procalcitonin (PCT), interleukin-6 (IL-6), and arterial blood gas. The oxygen saturation (SaO 2 ) was measured using pulse oxygen saturation in room air at resting status. Comorbidity was de ned as having at least one of the followings: hypertension, diabetes, cardiovascular disease, cerebrovascular disease, chronic lung disease, and malignant tumor for at least 6 months. All data were collected, re-checked for accuracy independently by at least two researchers.

Statistical analysis
Continuous variables were expressed as mean (SD). Categorical variables were expressed as frequency (percentage). Categorical variables were compared by the χ 2 test or Fisher's exact test. Continuous variables were compared by the Student's t test or Mann-Whitney U test. Survival curves were analyzed using the Kaplan-Meier method. Differences between curves were assessed using the log-rank test.
Univariate and multivariate COX proportional regression analysis was used for investigating the independent risk factors of death. The independent risk factors associated with the risk of mortality of patients with severe COVID-19 were used to build the nomogram in the training cohort. The performance and accuracy of the established nomogram was assessed by receiver operating characteristic (ROC) curve and calibration with 1000 bootstrap samples. The area under ROC (AUC) and optimal cut-off values were determined. Decision curve analysis (DCA) based on the net bene t was depicted by the package of rmdain R. The nomogram was validated in the validation cohorts 1 and 2, respectively. The nomogram was constructed and evaluated using the R software version 3.4.1 package with the rms and hmisc. All statistical analysis was performed using R version3.4.1, a p <0.05 in two-tailed was the signi cance threshold.

Patient clinical characteristics
Of 2541 patients included in this study, 271 patients had severe COVID-19 and 2270 were non-severe cases. Supplemental Table 1 indicated the basic characteristics of the study cohorts. Compared with nonsevere cases, patients were older (p < 0.001) and more males (p < 0.001) were found in the severe disease cohort. In addition, there were more comorbidities and smokers in severe cases (p < 0.001). The proportion of patients with panting was higher signi cantly in the severe cases (p < 0.001). Signi cantly higher levels of white blood cell (WBC) count, neutrophil count, CRP, PCT, IL-6, total bilirubin (TBIL), alanine aminotransferase (ALT), aspartate aminotransferase (AST), lactate dehydrogenase (LDH), γglutamyl transpeptidase (γ-GT), and creatinine (Cre) were identi ed in severe cases (p < 0.001), while the levels of lymphocyte count, platelet (PLT), and oxygen saturation (SaO2) of patients with severe diseases were signi cantly lower than those of non-severe ones (p < 0.001). There were 186 patients with no manifestations of chest CT in non-severe COVID-19 cohort, and no signi cant differences of the ndings of chest CT between severe and non-severe cases were found (p > 0.05). By the end of April 4, 2020, there were 58 and 39 patients died in the severe and non-severe disease group, respectively. (Supplemental Table 1).
Comparison of baseline characteristics between patients in training and validation cohorts can be seen in Table 1. There were signi cant differences of the age, proportion of smokers, incidence of panting, white blood cell count (WBC), neutrophil count, CRP and SaO 2 at admission between the three cohorts (p < 0.05). There was no signi cant difference in the other variables between the three cohorts (p > 0.05). By the end of April 4, 2020, 22 severe COVID-19 patients died in the training group, and 22 and 14 patients died in the validation group 1 and validation group 2, respectively ( Table 1).
The baseline characteristics of patients in the training cohort were shown in Table 2. There were no signi cant differences in gender, TB, ALT, AST, LDH, γ-GT, Cre, PLT, and the proportion of smoker between survivors and non-survivors (p > 0.05). Survivors were signi cantly younger than the non-survivors in the training cohort (p < 0.05), however, the proportion of patients with multiple comorbidity and panting (breathing rate ≥30/min) was signi cantly higher in non-survivors (p < 0.05). In addition, WBC and neutrophil count, CRP, D-dimer, PCT, and IL-6 was also signi cantly higher in non-survivors (p < 0.05). The lymphocyte count was signi cantly lower in non-survivors (p < 0.05).
Survival Analysis in the Patients with High Level of IL-6 Due to high level of IL-6 correlating with poor outcomes in severe COVID-19 patients, the therapeutic effect of tocilizumab in the patients with high IL-6 was further analyzed. In the training cohort, it was demonstrated that the prognosis of patients receiving tocilizumab was better than the that of patients not receiving tocilizumab, but without signi cance (p = 0.105) (Supplemental Figure.2A). Similar results were also observed in the validation cohort 1 and validation cohort 2, respectively (p = 0.133, p = 0.210) (Supplemental Figure.2B-C).

Construction and validation of the nomogram
Four independent risk factors found to be associated with the risk of mortality of patients in the multivariate analyses were incorporated into the nomogram (Figure.1). The ROC curve was employed to assess the predictive ability of the established nomogram, and the result demonstrated that the AUC was 0.900 (95% CI: 0.841-0.960) in the taring cohort, with a sensitivity of 95.5% and speci city of 77.5% ( Figure. 2A). Moreover, the calibration curves for nomogram predicted mortality indicated that a good consistency between observed actual outcomes and predicted ones in the training cohort ( Figure. Clinical application of the nomogram DCA based on the net bene t and threshold probabilities was performed to assess the clinical applicability of the risk prediction nomogram. The DCA showed that our risk prediction nomogram had a superior net bene t with a wide range of threshold probabilities in the training cohort and validation cohorts ( Figure. 4A-C).

Discussion
Previous studies have shown that the mortality rate of severe COVID-19 patients was signi cantly higher than that of mild patients [6,18]. Therefore, a reduction in the mortality of severe patients is pivotal in the case of treatment. Our study revealed the clinical characteristics and risk factors for fatal outcomes in con rmed severe COVID-19 patients based on multicenter cohorts.
Multivariate Cox analysis in this study indicated that age, lymphopenia, respiratory rate ≥ 30/min, and IL-6 were independent high-risk factors associated with poor prognosis. Older age has been proven to be a risk factor for survival in many previous studies [19][20][21][22]. Elderly patients with severe COVID-19 were more likely to develop fatal outcomes because of the rapid progression of the disease, which reminded us of providing early intervention for elderly severe patients. Similarly, lymphopenia was more common in the nonsurvivors and severe cases according to previous reports, suggesting dysregulation of the immune response in patients with COVID-19 [23][24][25]. Nevertheless, most of these were only descriptive studies. A study clari ed that lower lymphocyte counts were predictive of COVID-19 progression [26], whereas the impact of lymphocyte counts on the survival of severe COVID-19 was unclear. This study demonstrated that a lymphocyte count < 1.0 × 10 9 /L was independently associated with death in severe cases.
Recent studies have found that the cytokine storm is an important factor leading to rapid disease progression and poor prognosis [27,28]. IL-6 is one of the major cytokines involved in cytokine storms [29,30]. A previous univariate analysis showed that the IL-6 level was associated with worse survival without signi cance [20]. Our study showed that a high level of IL-6 was a predictor of death in severe COVID-19 patients. Additionally, the survival curve showed that the outcome was better in patients with tocilizumab than in patients without tocilizumab in the training cohort. Nonetheless, no signi cant difference was found. Similar results were also found in validation cohort 1 and validation cohort 2. The reason for this result may be attributable to the small sample size. The effect of tocilizumab on the survival of patients with severe COVID-19 needs further investigation in larger cohorts.
Increasing respiratory rate is an important clinical feature of acute respiratory distress syndrome (ARDS), which is a major cause of death in severe COVID-19 patients [9,19,31]. A previous study showed that a respiratory rate ≥24/min was a risk factor for death in the univariate analysis [20], whereas no signi cance was found after the multivariate regression analysis. Our multivariate regression analysis clari ed that a respiratory rate ≥30/min was a predictor of death. For patients with increasing respiratory rates, especially those with respiratory rate ≥30/min, it is necessary for physicians to be aware of the potential progression of ARDS.
Many previous studies have shown that comorbidity was signi cantly associated with high mortality rate and disease progression [26,32]. Nevertheless, in this study, the signi cance of comorbidity was only indicated in the univariate analysis, but not in the multivariate regression analysis, which may be ascribed to different patients enrolled in these studies. All the patients included in this study had severe COVID-19, and the proportion of comorbidities was approximately 70%, which was signi cantly higher than those in other studies.
Previous studies have shown several prediction models with different parameters [2,26,32,33]. Compared with other studies, the predictive effect of age, panting, and lymphopenia has been described in previous reports, while the main feature of this study is the analysis of the prognostic value of IL-6 in severe COVID-19 patients for the rst time. IL-6 plays an important role in the pathophysiological changes of severe cases. However, the prediction value of IL-6 was not shown in other prediction models. In addition, our study also indicated that tocilizumab, a monoclonal antibody against the IL-6 receptor, may improve the prognosis of severe COVID-19 patients with high levels of IL-6.
Our study has some limitations. First, this is a retrospective study, and there may be potential biases in the selection of patients. Second, the sample size of the study was relatively small, and the results need to be further validated in a larger cohort.

Conclusion
This study rstly developed a nomogram for predicting fatal outcomes in the severe COVID-19 patients. The four predictors included in the model are easy to obtain. The prediction risk of the model indicated a good consistence with the observed one. Hence, this nomogram may be conducive to more effective treatment to reduce the mortality of those severe cases at high risk of death. Abbreviations: WBC, white blood cell; PLT, platelet; CRP, C-reactive protein; PCT: procalcitonin; IL-6, interleukin-6; SaO2, Oxygen saturation; TBIL, total bilirubin; ALT, alanine aminotransferase; AST, aspartate aminotransferase; LDH, lactate dehydrogenase; γ-GT, γ-glutamyl transpeptadase; Cre, creatinine.    The receiver operating characteristic (ROC) curves of the nomogram in the training cohort (A), validation cohort 1 (B) and validation cohort 2 (C).

Figure 3
The calibration curves of the nomogram in the training cohort (A), validation cohort 1 (B) and validation cohort 2 (C).