Eur J Cardiothorac Surg 2007;32:281-285. doi:10.1016/j.ejcts.2007.04.010
Copyright © 2007, European Association for Cardio-Thoracic Surgery. Published by Elsevier B.V. All rights reserved
Preoperative risk stratification in infective endocarditis. Does the EuroSCORE model work? Preliminary results
Carlos A. Mestres*,
Miguel A. Castro,
Eduardo Bernabeu,
Miguel Josa,
Ramón Carta
á,
José L. Pomar,
José M. Miró,
Jaime Mulet, the Hospital Clínico Endocarditis Study Group
Hospital Clinico, University of Barcelona, Barcelona, Spain
Received 15 September 2006;
received in revised form 2 April 2007;
accepted 6 April 2007.
* Corresponding author. Address: Cardiovascular Surgery, Hospital Clínico, University of Barcelona, Villarroel 170, 08036 Barcelona, Spain. Tel.: +34 93 2275749; fax: +34 93 4514898. (Email: cmestres{at}clinic.ub.es).
 |
Abstract
|
|---|
Objective: There is an important role for risk prediction in cardiac surgery. Prediction models are useful in decision making and quality assurance. Patients with infective endocarditis (IE) have a particularly high risk of mortality. The aim was to assess the performance of European System for Cardiac Operative Risk Evaluation (EuroSCORE) in IE. Methods: The additive and logistic EuroSCORE models were applied to all patients undergoing surgery for IE (Duke criteria) between January 1995 and April 2006 within our prospective institutional database. Observed and predicted mortalities were compared. Model calibration was assessed with the Hosmer–Lemeshow test. Model discrimination was tested by determining the area under the receiver operating characteristic (ROC) curve. Results: One hundred and eighty-one consecutive patients undergoing 191 operations were analyzed. Observed mortality was 28.8%. For the entire cohort the mean additive score was 10.4 (additive predicted mortality of 14.2%). The mean logistic predicted mortality was 27.1%. Discriminative power was good for the additive and the logistic models for the entire series. Area under ROC curve were 0.83 (additive) and 0.84 (logistic) for the entire cohort, 0.81 and 0.81 for the aortic position, 0.91 and 0.92 for the mitral position, 0.81 and 0.81 for the native valve, 0.82 and 0.83 for the prosthetic valves, and 0.81 and 0.51 for the gram-positive microorganisms, respectively. Conclusions: This initial sample may be small; however, additive and logistic EuroSCORE adequately stratify risk in IE. Logistic EuroSCORE has been calibrated in IE, a special group of very high-risk patients. Further studies with larger sample sizes are required to confirm these initial results.
Key Words: Risk stratification • EuroSCORE • Infective endocarditis
|
1. Introduction
|
|---|
Infective endocarditis (IE) is an uncommon disease associated with significant morbidity and mortality. Cardiac and extracardiac complications may develop [1]. As with any infection within the cardiovascular system, early diagnosis and aggressive management are indicated. IE is a medical and surgical disease which must be managed by a multidisciplinary team with shared interests. Surgery for IE, especially in the acute phase, is still challenging, and results are often suboptimal. Moreover, the role of surgery has not been appropriately defined and understood in some subgroups of patients [2]. Preoperative status and intraoperative difficulties resulting in extended aortic cross-clamping and cardiopulmonary bypass times play a fundamental role in outcomes.
There is much concern in the surgical community with regard to reporting of perioperative deaths. Scoring systems have been used and validated. They allow for comparing outcomes, appropriately making predictions and helping in the decision-making process – in summary, for benchmarking. Furthermore, they are also a very good defensive tool against the pressure of the administrators and policy makers. A key point is to consider the case mix for accounting [3]. The European System for Cardiac Operative Risk Evaluation (EuroSCORE), developed in Europe, was designed to predict 30-day mortality of patients undergoing cardiac surgery based on a significantly higher proportion of patients undergoing coronary artery bypass graft (CABG) surgery [4]. This model is widely used throughout Europe and has been validated in different populations across the world [5–7].
There are some doubts that both the additive [8] and the currently available logistic [9] models may perform well in the subsets of very high risk (additive score > 10). Some authors have already discussed in this issue, particularly in relation with the surgery of the thoracic aorta, another classical example of high-risk surgery [7,10,11]. As IE is a pathology that carries a significant risk of morbidity and mortality, especially during the acute phase, we sought to investigate the validity of the additive and logistic EuroSCORE approaches in the same dataset.
|
2. Material and methods
|
|---|
2.1 Design of the study
Retrospective study of all patients, including the general population, intravenous drug abusers, and those with positive serology for the human immunodeficiency virus (HIV) undergoing an operation under extracorporeal circulation (ECC) for IE at our institution between January 1995 and April 2006. Our institution is an 830-bed tertiary care hospital covering a population around 1,000,000. Demographics, preoperative data, diagnosis, intra-, and postoperative records have been analyzed, follow-up data have been collected from outpatient clinic visits and telephone interviews with patients, relatives, or doctors. All IE patients have been prospectively followed at our institution since 1979 through the activities of the Hospital Clinic Endocarditis Study Group (HCESG), a collaboration of members of the Departments of Infectious Diseases, Microbiology, Cardiology, Pathology, and Cardiovascular Surgery (Appendix A
). Patients suffering from HIV infection are also prospectively followed by the same group. Information about both diseases is stored in specific databases. Medical and surgical results have already been reported [12,13]. Although retrospective calculations can be made for every patient in our database, we choose to study patients operated on until early 1995, which is the year in which data started to be collected for development of the EuroSCORE scoring system [4].
2.2 Definitions
2.2.1 Reporting
Results were reported following the guidelines for reporting morbidity and mortality for cardiac valvular operations as published by Edmunds et al. [14].
2.2.2 Infective endocarditis
The diagnosis of infective endocarditis was made according to the 1994 Durack criteria [15].
2.2.3 Active infective endocarditis
Acute IE was considered to be that in which the patient is on specific antibiotic therapy for a given pathogen or empiric treatment for negative blood cultures at the time of surgery. Antibiotic therapy should extend up to 6 weeks depending upon the pathogen, valve position, and pathological findings.
2.2.4 HIV infection
HIV infection was accepted if patients had two positive ELISA test or a positive ELISA test plus a positive Western blot test. HIV infection was classified according to the 1993 Centers for Disease Control (CDC) recommendations.
2.2.5 EuroSCORE predictors
The EuroSCORE scoring system was used and calculations were performed in relation to the previously published factors [16] that are widely available (the official website of the EuroSCORE cardiac surgery scoring system: www.euroscore.org). EuroSCORE predictors were previously defined in literature [4,16]. Among these factors, we stress on the definition of early death after surgery, which was that occurring at any time during the hospital stay, regardless of its length or the first 30th postoperative days for outpatients. In addition, active endocarditis is that in which the patient is still under antibiotic treatment for endocarditis at the time of surgery as previously defined in this section. EuroSCORE allocates 3 points in the additive model to active IE. Both additive and logistic EuroSCORE were calculated for each patient as previously reported [17].
2.3 Statistical method
Data are presented as mean ± standard deviation or absolute and relative frequencies. The discriminating ability of the additive and logistic models was tested with receiver operating characteristic (ROC) curves [18]. Calibration was assessed by the Hosmer–Lemeshow goodness-of-fit test, comparing observed and predicted deaths for each model [19]. Chi square test – one sample, unpaired – was used for comparing the incidence of observed and predicted deaths and the Student t-test was used to compare means between groups. The dataset was imported and analyzed using the SPSS 11.0 (SPSS, Inc., Chicago, IL, USA) software.
|
3. Results
|
|---|
During the study period, 180 patients underwent 191 operations with extracorporeal circulation for IE involving all four valve positions, pacemakers, and automated-implantable cardioverter-defibrillators (AICD), alone or in combinations. These operations were those strictly related and primarily performed for IE and did not include any other type of operation like postoperative cardiac tamponade, bleeding, or mediastinitis. Table 1
shows the demographics and characteristics of the entire series divided by groups, including those who survived the operation and those who died. The prevalence of predictors in our sample is shown in Table 2
. It is to be noticed that in 178 operations (93%) IE was considered to be active according to previous definitions. This may be a matter of concern. After carefully reviewing the database, it was found that a few patients underwent surgery (13, 7%) immediately after the end of intravenous antibiotics during this initial admission. It could have been easy to consider them still under antibiotics but for some reasons like the clinical condition that did preclude discharge; although the strict definition of acute IE as stated above did not apply to them, they were not excluded from the analysis.
There were 55 cases of mortality for a 28.8% observed mortality. Logistic EuroSCORE predicted 27.1% mortality. There was no statistical significant difference between the observed and the expected mortality for the entire series (p
= 0.70). Discriminating ability was good for both additive and logistic models, with areas under the ROC curve of 0.83 for the additive model and 0.84 for the logistic model (Fig. 1
). These areas under the ROC curve of 0.83 (additive) and 0.84 (logistic) were such for the entire cohort. Other areas under the ROC curves were 0.81 and 0.81 for the aortic position, 0.91 and 0.92 for the mitral position, 0.81 and 0.81 for native valve, 0.82 and 0.83 for prosthetic valves, and 0.81 and 0.51 for the gram-positive microorganisms, respectively. Calibration was good for both the additive (0.96) and logistic (0.94) models.
When the results were analyzed by risk strata defined by EuroSCORE (low (0–2), medium (3–5), high (6 plus)), the observation was made that calibration and discrimination for the high-risk stratum was also very good (Hosmer–Lemeshow 0.47 and 0.82 (0.76–0.89)). However, for the medium- and low-risk strata, discrimination power was poor (0.56 (0.13–1.00)).
|
4. Discussion
|
|---|
Preoperative risk prediction with statistical models is an important and established part of our practice in cardiac surgery today. The EuroSCORE risk model, introduced at the end of the past century, is widely used across Europe. As it is well known, it comes in two versions, additive and logistic. The additive model is more simple and practical and the logistic gives the surgeon a probability of death. There has been some controversy with regard to the possible advantages of one model over the other. Shamnugam et al. concluded that there was no advantage of using the logistic over the additive model, as the latter was not supposed to be more accurate in the subset of high-risk patients [8]. However, Jin and Grunkemeier [9] confirmed that the logistic model had a relatively constant ratio between observed and expected mortalities across risk groups and, therefore, is more accurate for different patient groups.
The problem of high-risk patients has also been addressed by several authors who referred to EuroSCORE and the surgery for the thoracic aorta, which is a good example of high-risk surgery [7,10,11]. According to these authors, logistic EuroSCORE was able to appropriately predict the expected mortality. As discussed earlier, EuroSCORE seems to be a reliable tool in cardiac surgery with some already available validations in different populations [5–7]. There are some exceptions like the failure to validate both the additive and logistic models in an Australian population [20]. However, in this Australian analysis, discrimination remained excellent as the area under the ROC curve was 0.82–0.83 as noted by Nashef in his comments [21]. In any case, the overall impression is that logistic EuroSCORE seems to work appropriately even in the high-risk groups [22].
As patients suffering from IE are usually patients with a significant risk of perioperative death, we decided to make an initial approach to validate EuroSCORE in our population of patients undergoing an operation under ECC for IE. As in the case of surgery of the thoracic aorta, IE is a good model of a high risk patient. The initial results presented here seem to support, with some limitations, the fact that EuroSCORE may be useful at the time to make serious decisions in very complex patients. There has been no significant difference between the observed and the expected mortalities using the logistic model (28.8 vs 27.1%, p
= 0.70). The calibration and discrimination of both additive and logistic EuroSCORE were good in our limited series, and this is of particular importance if we consider that 87.4% of our series had an additive EuroSCORE > 6. This may be of help, provided that larger series of surgical patients with IE can be recruited for such an extended analysis.
There are some other issues in IE that may have to eventually be taken into account at the time of analyzing intra- and postoperative mortality that are more important than in other types of less risky operations. Then, the length of aortic cross-clamping and ECC times (intraoperative factor), and the presence of vegetations, abscesses, or fistulae of the type of pathogen (preoperative factors) may have an additional impact on outcomes.
|
5. Limitations
|
|---|
Some of the limitations of this study are well known. First, EuroSCORE was initially based on a predominant population of patients undergoing CABG with some exceptions like that of Spain [23], although this may be due to some epidemiological differences among countries. Secondly, risk models are not perfect and, as considered by us and discussed by others [24], variability in cardiac surgery outcomes and the intrinsic complexity of some diseases and the surgical type of repair required may not be explained in full by using some models. Thirdly, and this is strictly related to our series, the sample size may not be large enough, especially if we are to consider subgroups. Even though we found that EuroSCORE seems to grossly perform well as predictor of mortality in IE, a reduced sample size like in this preliminary study will be counteracted by more volume accumulated from future cooperative efforts that are underway. In addition, even though EuroSCORE has been a successful European project, the model itself may be a little outdated as it has been operational for more than a decade [24]. Patient profile has significantly changed since information was being incorporated into the EuroSCORE dataset between September and November 1995 [16].
|
6. Conclusions
|
|---|
The risk stratification EuroSCORE model seems to satisfactorily predict early mortality in our surgical series of IE. Observed mortality does not differ from the expected according to the logistic model (28.8 vs 27.1%, p
= 0.75). The risk model EuroSCORE, both in its additive and logistic versions, shows an adequate calibration and discrimination in our series. When the analysis is performed by risk strata, according to additive EuroSCORE, calibration and discrimination of the logistic model are maintained. The still limited size of the sample may be an influencing factor with this regard. This preliminary experience does not exclude discussion about the need of reconsidering recalibration of EuroSCORE after more than a decade in use, considering the degree of concern with regard to this very popular and reliable model in the surgical community.
|
Appendix A
|
|---|
Conference discussion
Dr M. Antunes (Coimbra, Portugal): This is a very important piece of work, and it raises some doubts in my mind, because I was under the impression that, in fact, the EuroSCORE is outdated for most of the pathology. It may very well be that it performs well for complex pathology like this and not so well for simple pathology where the surgeons have been able to improve on their mortality rates. In a paper that we are going to present at this meeting tomorrow, we have studied the performance of not only the EuroSCORE but also the Parsonnet and the Ontario Provincial Scores in our population of coronary patients. We studied more than 4500 patients to establish our own scoring system, because we found that the EuroSCORE prediction was completely out of our current experience. Can you tell me what your experience or your institutional experience is for other subsets of patients, not this very complex pathology? In these more complex pathologies the results have not really improved because they are not so much related to the technique or care that we use but with the pathology in itself, which of course we cannot modify over time.
Dr Mestres: I think you are right, and this is why we tried to collect some additional information on specific subsets of patients like those at high risk, because we were not convinced that we had a good tool to look after some decisions in very high-risk patients. It is interesting because we found that the model was validated in our hands, which doesn’t necessarily mean it is going to be working well in other hands. We still have a little bit of doubt, because we may eventually need a little bit more sample size coming from a cooperative study that we are actually proposing to some other institutions in Europe, perhaps to finally decide if it is a good model for this particular pathology or not. In any case, in most of the not so serious pathologies, I think it works well so far. I look forward to seeing data from your institution and to invite you to join this little study to shed more light on that.
|
Appendix B
|
|---|
Members of the Barcelona Hospital Clinic Endocarditis Study Group
- Del Río, A. Moreno, J.M. Gatell, J.M. Miró (Infectious Diseases).
- García de la María, M. Almela, F. Marco, M.T. Jiménez de Anta (Microbiology).
- J.C. Paré, M. Azqueta, M. Sitges (Cardiology).
- R. Cartaá, J.L. Pomar, S. Ninot, C.A. Mestres (Cardiovascular Surgery).
- J.M. Ramírez, N. Pérez, M.T. Ribalta (Pathology).
- E. de Lazzari (Biostatistics).
|
Footnotes
|
|---|
\#9734; Presented at the joint 20th Annual Meeting of the European Association for Cardio-thoracic Surgery and the 14th Annual Meeting of the European Society of Thoracic Surgeons, Stockholm, Sweden, September 10–13, 2006.
\#9734;\#9734; Dr J.M. Miró was a recipient of a Research Grant from the ‘Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS)’ and the ‘Conselleria de Salut de la Generalitat de Catalunya, Barcelona (Spain)’, and this study was supported by the ‘Ministerio de Sanidad y Consumo, Instituto de Salud Carlos III, Spanish Network for the Research in Infectious Diseases (REIPI RD06/0008). Madrid, Spain’.
|
References
|
|---|
- Anguera I, del Río A, Moreno A, Paré JC, Mestres CA, Miró JM. Complications of native and prosthetic valve infective endocarditis: update in 2006. Curr Infect Dis Rep 2006;8:280-288.[CrossRef][Medline]
- Wang A, Pappas P, Anstrom KJ, Abrutyn E, Fowler Jr. VG, Hoen B, Miro JM, Corey GR, Olaison L, Stafford G, Mestres CA, Cabell CH, the International Collaboration on Endocarditis Investigators The use and effect of surgical therapy for prosthetic valve infective endocarditis: a propensity analysis of a multicenter, international cohort. Am Heart J 2005;150:1086-1091.[CrossRef][Medline]
- Gallivan S, Utley M, Pagano D, Treasure T. MADCAP: a graphical method for assessing risk scoring systems. Eur J Cardiothorac Surg 2006;29:431-433.[Abstract/Free Full Text]
- Nashef SAM, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R, the EuroSCORE Study Group European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999;16:9-13.[Medline]
- Nashef SAM, Roques F, Hammill BG, Petersin ED, Michel P, Grover FL, Wyse RK, Ferguson TB. Validation of European System for Cardiac Operative Risk Evaluation (EuroSCORE) in North American cardiac surgery. Eur J Cardiothorac Surg 2002;22:101-105.[Abstract/Free Full Text]
- Geissler HJ, Holzl P, Marohl S, Kuhn-Regnier F, Mehlhorn U, Suckamp M, De Vivie ER. Risk stratification in heart surgery: a comparison of six scoring systems. Eur J Cardiothorac Surg 2000;17:400-406.[Abstract/Free Full Text]
- Kawachi Y, Nakashima A, Toshima Y, Arinaga Y, Kawano H. Risk stratification analysis of operative mortality in heart and thoracic aorta surgery: comparison between Parsonnet and EuroSCORE additive model. Eur J Cardiothorac Surg 2001;20:961-966.[Abstract/Free Full Text]
- Shamnugam G, West M, Berg G. Additive and logistic EuroSCORE performance in high-risk patients. Interact Cardiovasc Thorac Surg 2005;4:299-303.[Abstract/Free Full Text]
- Jin R, Grunkemeier GL. Does the logistic EuroSCORE offer an advantage over the additive model?. Interact Cardiovasc Thorac Surg 2006;5:15-17.[Abstract/Free Full Text]
- Barmettler H, Immer FF, Berdat PA, Eckstein FS, Kipfer B, Carrel TP. Risk-stratification in thoracic aortic surgery: should the EuroSCORE be modified?. Eur J Cardiothorac Surg 2004;25:691-694.[Abstract/Free Full Text]
- Nishida T, Masuda M, Tomita Y, Tokunaga S, Tanoue Y, Shiose A, Morita S, Tominaga R. The logistic EuroSCORE predicts the hospital mortality of the thoracic aortic surgery in consecutive 327 Japanese patients better than the additive EuroSCORE. Eur J Cardiothorac Surg 2006;30:578-582.[Abstract/Free Full Text]
- Martínez E, Mocroft A, García-Viejo MA, Pérez-Cuevas JB, Blanco JL, Mallolas J, Bianchi L, Conget I, Blanch J, Phillips A, Gatell JM. Risk of lipodystrophy in HIV-1-infected patients treated with protease inhibitors: a prospective cohort study. Lancet 2001;357:592-598.[CrossRef][Medline]
- Mestres CA, Chuquiure JE, Claramonte X, Muoz J, Benito N, Castro MA, Pomar JL, Miró JM, the Hospital Clinic Endocarditis Study Group Long-term results after cardiac surgery in patients infected with the human immunodeficiency virus type-1 (HIV-1). Eur J Cardiothorac Surg 2003;23:1007-1016.[Abstract/Free Full Text]
- Edmunds Jr. LH, Clark RE, Cohn LH, Grunkemeier GL, Miller DC, Weisel RD. Guidelines for reporting morbidity and mortality after cardiac valvular operations. Eur J Cardiothorac Surg 1996;10:812-816.[Abstract]
- Durack DT, Lukes AS, Bright DK. New criteria for diagnosis of infective endocarditis: utilization of specific echocardiographic findings. Duke Endocarditis Service. Am J Med 1994;96:200-209.[CrossRef][Medline]
- Roques F, Nashef SAM, Michel P, Gauducheau E, de Vincentiis C, Baudet E, Cortina J, David M, Fauchney A, Gabrielle F, Gams E, Harjula A, Jones MT, Pinna Pintor P, Salamon R, Thulin L. Risk factors and outcome in European cardiac surgery: analysis of the EuroSCORE multinational database of 19030 patients. Eur J Cardiothorac Surg 1999;15:816-823.[Abstract/Free Full Text]
- Zingone B, Pappalardo A, Dreas L. Logistic versus additive EuroSCORE. A comparative assessment of two models in an independent population sample. Eur J Cardiothorac Surg 2004;26:1134-1140.[Abstract/Free Full Text]
- Hanley JA, McNeil J. The meaning and use of area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36.[Abstract/Free Full Text]
- Lemeshow S, Hosmer DW. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1982;115:92-106.[Abstract/Free Full Text]
- Yap CH, Reid C, Yii M, Rowland MA, Mohajeri M, Skillington PD, Seevanayagam S, Smith JA. Validation of the EuroSCORE model in Australia. Eur J Cardiothorac Surg 2006;29:441-446.[Abstract/Free Full Text]
- Nashef SAM. Editorial comment. Eur J Cardiothorac Surg 2006;29:446
doi:10.1016/j.ejcts.2006.01.003
.[Free Full Text]
- Michel P, Roques F, Nashef SAM, The EuroSCORE Project Group Logistic or additive EuroSCORE for high-risk patients?. Eur J Cardiothorac Surg 2003:684-687.
- Roques F, Nashef SAM, Michel P, Pinna Pintor P, David M, Baudet E, the EuroSCORE Study Group Does EuroSCORE work in individual European countries?. Eur J Cardiothorac Surg 2000;18:27-30.[Abstract/Free Full Text]
- Shahian DM, Blackstone EH, Edwards FH, Grover FL, Grunkemeier GL, Naftel DC, Nashef SAM, Nugent WC, Peterson ED. Cardiac surgery risk models: a position article. Ann Thorac Surg 2004;78:1868-1877.[Abstract/Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
C. A. Mestres
Invited Commentary
Ann. Thorac. Surg.,
May 1, 2008;
85(5):
1569 - 1570.
[Full Text]
[PDF]
|
|
|
Top
Abstract
1. Introduction
