American Journal of Cardiovascular and
Thoracic Surgery

Objective: To Validate the EuroSCORE II as a method for cardiac surgery risk stratification in Mexican adult population.

Methods: We included adult patients undergoing to cardiac surgery, in order to determine the predictive value of EuroSCORE II on morbidity and mortality risk. Continuous variables are presented as mean ± SD or median with its interquartile range as appropriate; categorical variables were described as n, % or rate. To validate the EuroSCORE II scale, the assessment was done with Hosmer- Lemeshow (HL) test. In terms of discrimination, we used the features of the receiver operation characteristic (ROC) curves.

Results: They were 704 patients, grouped into five categories: simple (one vessel) Coronary Artery Bypass Grafting (CABG) surgery, n= 299 (43%) cases. CABG revascularization (two or more vessels), n= 208 (30%). Double Procedure (CABG + valve replacement) 174 (25%) cases. Triple procedure (CABG + valve + aorta surgery) 23 (3.3%) patients. The mortality observed within 30 days of the surgery was 88 (12.5%). Meanwhile, the mean of the expected mortality predicted by EuroSCORE II was 3.63 ± 5.91 (95% CI: 3.19-4.06). The EuroSCORE II scale presented a good capacity for discrimination in the studied population reaching an area under the ROC curve of 0.821 (p < 0.000, 95% CI: 0.772-0.871). A calibration for the scale measured through logistic regression with goodness of adjustment of Hosmer-Lemeshow was determined (χ2 = 17.74, p = 0.64).

Conclusion: EuroSCORE II showed moderate discrimination ability in general. The scale can be useful to identify some problems in our hospital, however, the mortality rate might be underestimated.

Key words: Euroscore II; Adult Cardiac Surgery; Surgical Risk

The European System for Cardiac Operative Risk Evaluation (EuroSCORE) is a method for easy and affordable surgical risk stratification. Its utility was released in 1999 and was developed at 132 medical centers in eight European countries. Its validity studies performed in 19,030 patients and 97 variables were analyzed by logistic regression. 18 indicators were associated with mortality rate; it was possible to develop a predictive model with additive score. [1] In 2003, a logistic model from the beta coefficient of each variable was generated. [2] The tool was validated in eight European countries; While United States North American approved its use with data from the Thorax Society with more than 400,000 patients. [3] In addition, in Japan demonstrated its usefulness to predict mortality in operated patients of thoracic aorta. [4] As a consequence of these studies it was determined that EuroSCORE is a magnificent tool for clinical research. Thus, it was adopted as a measure of the operative risk and has been used to evaluate and to compare the surgical performance in cardiac surgery in adults in more than 1,300 studies. [5] However, the important changes in the cardiac surgery in adults have shown that preoperative risk prediction not necessarily can be done with a total assertiveness, since it is a dynamic target. [6] Therefore we performed this paper to validate the EuroSCORE II method at a unique 3rd level Cardiology Hospital of Mexico City. Secondarily, we explore the predictive value of EuroSCORE II to mortality rate at thirty days after cardiac surgery; the correlation degree between different level of predicted risk with Mortality and to determine the variables with more capacity for mortality prediction.

Patients admitted to a unique 3rd level Cardiology Hospital from the National Medical Center, XXI-century, IMSS-Mexico, Mexico City that were submitted to cardiac surgery from August 1 2015 to October 31, 2016 was included. All patients over 18 years old and from both sexes which had a complete medical record. An electronic data collection instrument was used for further analysis.

This is an open retrospective observational study.

Continuous variables were expressed through mean ± standard deviation or 95% CI if its distribution was normal if not median and interquartile range was used which were summarized by applying an analysis by percentiles by dividing the score EuroSCORE II in classes and showed it through graphs. Discrete variables were expressed through frequencies and percentages. Bilateral Pearson correlation test was used for comparison and contrast.

Regarding the calibration of the scale, the evaluation was carried out with test Hosmer-Leme show where two factors were compared: the mortality observed vs. mortality expected in risk deciles. [7] In addition, the calibration was considered poor if the test was significant. In terms of discrimination, the curve of receiver operating characteristics (ROC) was used to differentiate between a sample of individuals who suffer an event--in this case, the death - and those who did not. [8] This study was approved by committees of ethics and research.

Was collected a sample of 704 patients who were undergoing some type of intervention in cardiac surgery. 413 cases of them (58.7%) were men and 291 cases (41.3%) were women. The mean age was of 61.45 ± 12.54 years old, with a minimum of 18 and a maximum of 87. Table 1 shown descriptive data by age groups.

Of the total number of surgeries performed, 286 (40.6%) were elective, 378 (53.7%) were urgent, 38 (5.4%) and two emerging rescue (0.3%).

For the analysis, were considered 20 different types of procedures, which were grouped into four categories:

1. Simple procedures without myocardial revascularization = 299 cases (42.5%). Includes five surgical procedures: valve mitral; valvular aortic; Tricuspid valve; pulmonary valve replacement; and closure of a ventricle septal defect post-acute myocardial infarction.

2. Myocardial revascularization = 208 cases (29.5%).

3. double procedures = 174 cases (24.7%); Includes 10 procedures: aortic valve tube (Bentall); CAGB + aortic valve replacement; CAGB + mitral valve; valve mitral + aortic valve replacement; mitral valve + tricuspid valve replacement; myomectomy + mitral valve replacement; aortic valve + extension ring; aortic valve replacement + closure of ventricular septal defect; aortic valve + tricuspid valve; and aortic valve + myomectomy.

4. triple procedure = 23 cases (3.3%), here referred to five procedures: coronary artery bypass grafting + aortic valve and valve mitral replacement; aortic valve + valve mitral + tricuspid valve replacement; Tricuspid valve + mitral valve + myocardial revascularization; and VSD + valve mitral + myocardial revascularization.

Table 2 shows the breakdown of frequencies of the types of surgery and procedures.

Variables associated with the risk of death in patients of cardiac surgery, such as comorbidities and risk factors, which form part of the EuroSCORE II were also measured. Their frequencies are shown in table 3.

Mortality within 30 days after cardiac surgery occurred in 88 cases (12.5%). Meanwhile, the mean of the expected mortality predicted by EuroSCORE II was 3.63 ± 5.91 (95% CI: 3.19-4.06). Based on both data, calculable rate of mortality-adjusted by risk (MARI); which is equivalent to the ratio of the observed mortality between the expected, was 3.44. The same procedure was performed after stratification by each type of surgery. In general the obtained results showed a considerable underestimation of mortality by the EuroSCORE II over the entire study population. The simple and double procedures were those in which we found a greater gap. The results are shown in the table 4.

Derived from the differences between observed and expected mortality that were found, and to better understand how the sample is distributed in relation to mortality observed variables, score EuroSCORE II by type of surgery, were developed as histograms to starting from an analysis by percentiles. Therefore we divided to study population based in the EuroSCORE II into the 11 classes using the Sturges rule, with a class length of 1.031. From the grouping of scores, 0.5, 0.75, 0.80, 0.85, 0.90 0.95 percentiles were taken as reference and 0.975, who threw scores of 1.94, 3.68, 4.46, 5.54, 7.15, 11.34 and 18.92 on the assessed scale, respectively. Then, odds for each of the classes were obtained based on the expected mortality, both in general and for each type of surgery (therefore the extent that data exist for the calculation), weighing the total mortality by the number of items within each class. In this way, we found behaviors that are running, as the score II of a patient EuroSCORE rises, the likelihood of death so too does.

EuroSCORE II scale presented a good capacity for discrimination into the study population reaching an area under-curve (ROC) of 0.821 (P < 0.000, 95% CI: 0.772-0.871), which gives a good discriminating ability to the test. The same thing happened in the case of each surgery, where reached areas between 0.800 and 0.837. These results are shown in table 5 and figures 1 and 2. Here, EuroSCORE II shows a greater capacity for discrimination in cases of myocardial revascularization surgery, simple procedures and double, while into the triple procedure the significance was minor.

On the other hand, a calibration in the global consistency to the EuroSCORE II scale was measured by logistic regression using Hosmer-Lemeshow Goodness-of-fit test (X2 = 17.74, P = 0.64), that showed no significant values (P > 0.058). This means that the scale EuroSCORE II can be used in all study population to predict mortality. However, when the analysis was stratified by type of surgical procedure, this no significance of Goodness-offit test was lost. We found important differences into the CABG group (P= 0.000), which indicates that the model did not have availability to difference between expected and observed data, accordingly, would be improperly calibrated for the measurement of that variable. Also, in terms of the variables of the equation, the significance of the Beta coefficient was low for triple procedure. All of this can be seen in table 5.

Finally, Pearson correlations were performed to measure the affinity between mortality and some variables considered in the calculation of EuroSCORE II. This, with the purpose of finding some relationship of affinity between the two that could indicate how it is that these have an impact on the result. Thus, we found significant positive correlations between mortality and type of surgery, creatinine clearance, ejection fraction of the left ventricle and pulmonary arterial hypertension; as well as negative with weak mobility, previous surgery, critical state, classification of NYHA and previous surgery of Aorta. Furthermore, we found significant positive correlations of EuroSCORE II type of surgery, clearance of creatinine and ejection fraction of the left ventricle and negative with all other. Its significance and correlation values are presented in table 6.

According to the statistical Receiver Operating Characteristics (ROC analysis), the discriminatory power of EuroSCORE is being suitable with an area under the curve of 0.75-0.80. [8] In spite of this, its relative overestimation of hospital mortality and low achievement in high-risk patients produced the discussion on the need for an improved tool to evaluate the surgical performance. Therefore, the most recent version of the system, called EuroSCORE II, establishes a new set of risk factors that include original variables of EuroSCORE, which were modified and supplemented to improve its accuracy and renew the EuroSCORE in the interest of optimizing its usefulness in contemporary surgical practice. [9]

This new predictive model of postoperative mortality was built on the basis of surgical outcomes observed in more than 22,000 patients in hospitals around the world, mainly in European countries. In this study, a discriminating capacity similar to EuroSCORE is demonstrated. The results were presented in the following way: • regarding the Area under the curve (ROC), the result of EuroSCORE II was 0.81, as opposed to the value obtained for simple EuroSCORE, which resulted from 0.78. [9] • As regards obtaining the Chi-square in EuroSCORE II, was a good calibration, since it was obtained a figure of 15,48 (p = 0.0505). Just as in the original model, this updated tool has been subjected to various tests of validity in order to guarantee their applicability.

Mortality in the perioperative period is a leading cause of death in many countries; therefore, it is desirable to have predictive tools that can discriminate among patients of high and low risk. Although there is a wide range of predictive models, there is concern in other fields of research, which forecast that the predictive literature suffers from poor quality methodology and lack of external validation. [6] In contrast, there are positions that argue that used predictive models allow the stratification of risk to facilitate informed consent at the time of decisionmaking and surgical intervention. For example, in patients with aortic stenosis, you can take into account a higher surgical risk determined by the score of cardiac surgery of the STS risk models to justify a surgical trans-catheter valve replacement. [9]

The relationship between the utility of the Multivariable models used to predict mortality in cardiac surgery and the practices of reporting has not been systematically studied. Only, there are studies that evaluated the evidence from comparisons between the II EuroSCORE, STS score and age assessment, creatinine and ejection fraction (ACEF, in English). These models are used to predict the Perioperative mortality in the context of cardiac surgery and have been adopted by the recent guidelines [5, 10-14].

A study of meta-analysis systematically reviewed the reports related to the relative performance of STS, EuroSCORE II and ACEF performance of discrimination, mortality at 30 days and in consideration of the informed bias. Revised articles met the inclusion criteria. Information about the design of the study, the predictive performance of risk models and the potential for bias is extracted. Subsequently, a meta-analysis was conducted to calculate the difference between the AUC of the models. Among the 22 selected studies, containing 33 comparisons revealed that scoring EuroSCORE II and STS behaved similarly, in terms of his AUC, while it surpassed the score of ACEF. [15]

The identification of potential risk factors for Perioperative mortality is a key issue to ensure surgical performance of high quality, and properly allocate resources both clinical and economic. However, it is not the only measure of quality available and patient global assessment cannot be limited to hospital outcomes. Although it has been shown that high risk patients have the greatest benefit after cardiac surgery, their Comorbidities can lead to poor results in the short and long term, which rolls back the results. [16] Therefore, the stratification of risk in long term outcomes is essential, regardless of immediate results, to adapt the health care to the individual profile of the patients. Also, it helps the clinical decision-making.

Given its extensive use and well recognized standard definitions of risk factors, EuroSCORE II may represent the base to explore the relationship between preoperative risk factors and mortality in the long run. In addition, it has shown EuroSCORE is an independent predictor of these variables. On the other hand hospital readmissions for cardiovascular events and costs at the monitoring have been previously associated with EuroSCORE. Despite this early evidence, it has not analyzed any further assessment of the relationship between long term outcomes and preoperative Comorbidities defined the EuroSCORE, especially with the new classification proposed in the updated model. [16]

A study conducted in Italy evaluated the impact of the EuroSCORE II and the mortality in the long run, subsequent to the completion of cardiac surgery. The complete data of 10,033 patients resumed for a period of 7 years. Mortality in the follow-up was analyzed with the analysis of the time until the event. Kaplan- Meier Survival estimates were at 1 and 5 years, respectively, 95.0% ± 0.2% and 84.7% ± 0.4%. Both discrimination and calibration of EuroSCORE II decreased in the prediction of mortality at 1 and 5 years. However, it was confirmed that the system was an independent predictor of mortality in the long run with a nonlinear trend. Several EuroSCORE II variables were independent risk factors for mortality in the long run in a regression model, especially with a very low ejection fraction. [19, 20]

In the final model, isolated mitral valve surgery and the myocardial revascularization is associated with improved longterm survival. In this study, it was concluded that the EuroSCORE II is not a direct risk of death long-term, Estimator since their performance fades for mortality follow-up of more than 30 days.

He is nonetheless associated linearly with the indicator. Most of its variables are risk factors for mortality in the long run. Therefore, you can use a different algorithm to stratify the risk after surgery. [17]

In contrast, the new score achieved could not function equal in predicting mortality in low risk patients. Multicenter clinical data collected prospectively from the United Kingdom suggested that it is a tool of acceptable contemporary generic risk, although it is poorly calibrated for both isolated coronary artery bypass surgery in the patients most at risk as in those who are at the minimum level. In addition, some concern remains for its precision for aortic valve replacement, aortic surgery and other procedures. Based on these, some authors agree that additional prospective validation of EuroSCORE II in larger, especially in diverse geographical regions populations is required. [18]

ROC curve analysis showed a good overall discrimination, by presenting an area of 0.856 and in the subgroup of non-CABG was 0.857 (p = 0.0001). Discrimination in the subgroup of CABG was poorer, to generate an area of 0.794 (p = 0.014). The model showed a good calibration in predicting in-hospital mortality, both in general and for each subgroup. [21]

Finally, in Mexico, Rodriguez Chavez et al. published a study applied a series of 1,188 patients undergoing valvular surgery in a hospital in 2017 [22]. EuroSCORE additive and logistic models were applied to predict mortality in patients with valvular surgery in March 2004 to March 2008.

The goodness of fit of Hosmer-Lemeshow test was employed to evaluate the calibration and the area under the ROC curve to determine discrimination. Patients were ages of 51.3 ± 14.5 years and 52% of the sample was comprised of women. The results showed that the total mortality was 9.7%, with predicted of 5% and 5.6% by additive and logistic EuroSCORE. According to the first, 11.3% of the sample presented low risk, 53% intermediate and 36% reported a high level. For these groups, the mortality was 0.7%, 6.4% and 17.4%, against those predicted from 2%, 3.9% and 7.64%.

Hosmer-Lemeshow test had a p < 0.001 for both models, and the area under the ROC curve of 0.707 and 0.694 for additive and logistic EuroSCORE. In this study EuroSCORE underestimated the risk of mortality. The clinical practice guidelines of the American Heart Association and the American College of Cardiology defines as fundamental the use of models for the estimation of risk of mortality and hospital morbidity in surgery to control the quality surgical and institutional, so to estimate the risk of death. [22]

Likewise, in the correlation developed between postoperative mortality in patients with different levels of risk in surgery, shows that the valve surgery procedure with greater probability of mortality, where in the 53.4% of the cases there was death when the average of the EuroSCORE II score predicted 2.81. Meanwhile, the category of others just showed a 2.3% of cases of death, against an expected 8.2. This contrasts with the results of previous studies analyzed, except for the case of the Rodríguez Chávez et to the in 2017, where we have obtained similar results in terms of underestimation of mortality. [22]

Now, on the results of the ROC curve and Hosmer-Lemeshow logistic regression tests, scale shows a good capacity for both general discrimination as for every type of surgery, as well as a calibration in general adequate. However, in some of the layers of analysis by surgery, discrimination was not significant and calibration was insufficient, which may well explain the gap in outcomes between the observed and expected, in particular with the variable surgery Valve. Looking at the results of the logarithm of likelihood, values so high in general can be seen as for almost all categories, which is indicator that, in reality, there is a poor association between EuroSCORE II and observed mortality.

Finally, the Pearson correlations realize the significance of some confounding variables and related mortality, which may have an impact in one way or another in it. However, no correlation score is high enough to explain all the variability of the results.

The data displayed in EuroSCORE II was not sufficient enough to be able to accurately estimate the probability of mortality in our study population, therefore, its use is recommended partially and other variables should be taken in account. This, we found scale scores underestimated the mortality in contrast to what was observed. The correlation of postoperative mortality of patients based on the assessed scale shows no clear behaviors that indicate an association among variables and observed mortality, EuroSCORE II scale, and type of surgical procedure.

Now, despite the fact that in the statistical scale is not valid, it could be useful to identify problems within the hospital. In this sense, it is necessary to create new areas of opportunity to employ EuroSCORE II in decision making for surgery, as well as the communication to patients and their families about the risk that exists with certain procedures and conditions. It is possible that it may be complementary to other tools of prediction of mortality and surgical risk stratification.

Recommendations to assess deeper the EuroSCORE II scale and its potential usefulness in other geographical areas should be recommended. In addition, is suggested to ponder the results against the environment variables that may influence to determine if there are intrusive that they should be excluded or conditioning variables to take into consideration to weigh the results.

1. Nashef SA, Roques F, Gauducheau E, Lemeshow S and Salamon R. the EuroSCORE study group. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg. 1999;16(1):9-13.
2. Roques F, Nashef SA, Michel P, Gauducheau E, de Vicentits C and Bauder E. et al. Risk factors and outcome in European cardiac surgery: Analysis of the EuroSCORE multi-national database of 19,030 patients. Eur J Cardiothorac Surg. 1999;15(6):816-23.
3. Nashef SAM, Roques F, Hammill BG, Peterson ED, Michel P and Grover FL y col. Validation of European System for Cardiac Operative Risk Evaluation (EuroSCORE) in North American cardiac surgery. Eur J Cardiothorac Surg. 2002;22(1):101-105.
4. Nishida T, Masuda M, Tomita Y, Tokunaga S, Tanoue Y and Shiose A. et al. The logistic EuroSCORE predicts the hospital mortality of the thoracic aortic surgery in consecutive 327 japanese patients better than the additive EuroSCORE. Eur J Cardiothorac Surg. 2006;30(4):578-82.
5. Sergeant P, Meuris B and Pettinari M. EuroSCORE II, illum qui est gravitates magni observe. Eur J Cardiothorac Surg. 2012;41(4):729-31.
6. Takkenberg JJ, Kappetein AP, Steyerberg EW. Editorial Comment: the role of EuroSCORE II in 21st century cardiac surgery practice. Eur J Cardiothorac Surg. 2013;43(1):32-33.
7. Lemeshow S and Hosmer D. A review of goodness of fit statistic for use in the development of logistic regression models. Am J Epidemiol. 1982;115(1):92-106.
8. Hanley JA and McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982; 143(1):29-36.
9. Nashef SAM, Roques F, Sharples LD, Nilsson J, Smith C and Goldstone AR. et al. EuroSCORE II. Eur J Cardiothorac Surg. 2012;41(4): 744-745. Doi: 10.1093/ejcts/ezs043.
10. Siontis GC, Tzoulaki I, Castaldi PJ and Ioannidis JP. External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. J Clin Epidemiol. 2015;68(1):25-34. Doi: 10.1016/j.jclinepi.2014.09.007.
11. Leon MB, Smith CR, Mack M, Miller DC, Moses JW and Svensson LG. et al. PARTNER Trial Investigators. Transcatheter aortic-valve implantation for aortic stenosis in patients who cannot undergo surgery. N. Engl. J. Med. 2010;363(17):1597-1607.
12. Shahian DM, O’Brien SM, Filardo G, Ferraris VA, Haan CK and Rich JB. et al. Society of Thoracic Surgeons Quality Measurement Task Force. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1-coronary artery bypass grafting surgery. Ann Thorac Surg. 2009;88(1 Suppl):S2-22. Doi: 10.1016/j.athoracsur.2009.05.053.
13. O’Brien SM, Shahian DM, Filardo G, Ferraris VA, Haan CK and Rich JB. et al. Society of Thoracic Surgeons Quality Measurement Task Force. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 2-isolated valve surgery. Ann Thorac Surg. 2009;88 (1 Suppl):S23-42. Doi: 10.1016/j.athoracsur.2009.05.056.
14. Shahian DM, O’Brien SM, Filardo G, Ferraris VA, Haan CK and Rich JB. et al. Society of Thoracic Surgeons Quality Measurement Task Force. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 3-valve plus coronary artery bypass grafting surgery. Ann Thorac Surg. 2009;88(1 Suppl):S43-S62. Doi: 10.1016/j.athoracsur.2009.05.055.
15. Sullivan PG, Joshua DW, John PA. Ioannidis. Meta-Analysis Comparing Established Risk Prediction Models (EuroSCORE II, STS Score, and ACEF Score) for Perioperative Mortality During Cardiac Surgery. Am J Cardiol. 2016;116:1-8.
16. Takkenberg JJ. Editorial comment: dynamic trends in cardiac surgery require dynamic models. Eur J Cardiothorac Surg. 2013;43(6): 1153. Doi: 10.1093/ejcts/ezs650.
17. De Maria R, Mazzoni M, Parolini M, Gregori D, Bortone F and Arena V. et al. Predictive value of EuroSCORE on long-term outcome in cardiac surgery patients: a single institution study. Heart. 2005;91(6):779–784.
18. Barili F. The Impact of EuroSCORE II Risk Factors on Prediction of Long-Term Mortality. Ann Thorac Surg. 2016;101:1-8.
19. García-Valentín A, Bernabeub E, Pereda D, Josac M, Cortina JM and Mestres CA. et al. Validación de EuroSCORE II en España. Cir Cardiov. 2014;21(4):246-251.
20. García-Valentín A. Validaciones externas de EuroSCORE II ¿ Podemos llegar a alguna conclusión? Cir Cardiov.2017;24(2):51-52.
21. Borracci RA, Rubio M, Celano L, Ingino CA, Allende NG and Ahuad Guerrero RA. Prospective validation of EuroSCORE II in patients undergoing cardiac surgery in Argentinean centres. Interactive Cardio Vasc Thorac Surg. 2014;18(5):539-543.
22. Rodríguez-Chávez LL, Figueroa-Solano J, Muñoz-Consuegra CE, Avila-Vanzzini N and Kuri-Alfaro J. EuroSCORE subestima el riesgo de mortalidad en cirugía cardiaca valvular de población mexicana. Arch Cardiol Mex. 2017;87(1):18-25.