|
|
||||||||
Eur J Cardiothorac Surg 2007;31:607-613. doi:10.1016/j.ejcts.2006.12.035
Copyright © 2007, European Association for Cardio-Thoracic Surgery. Published by Elsevier B.V. All rights reserved
a Department of Cardiothoracic Surgery, The Cardiothoracic Centre-Liverpool, Thomas Drive, Liverpool L14 3PE, UK
b Department of Clinical Governance, The Cardiothoracic Centre-Liverpool, Thomas Drive, Liverpool L14 3PE, UK
c Department of Cardiothoracic Surgery, Blackpool Victoria Hospital, UK
d Department of Cardiothoracic Surgery, Manchester Royal Infirmary, UK
e Department of Cardiothoracic Surgery, South Manchester University Hospital, UK
Received 10 September 2006; received in revised form 18 December 2006; accepted 19 December 2006.
* Corresponding author. Tel.: +44 151 228 1616; fax: +44 151 288 2371. (Email: manojkud{at}hotmail.com).
| Abstract |
|---|
|
|
|---|
Key Words: Aortic valve replacement Mortality Risk factors Risk prediction
| 1. Introduction |
|---|
|
|
|---|
There are various established and well validated risk prediction tools which are used predominantly for predicting risk for coronary artery bypass grafting, including the Parsonnet Score, additive EuroSCORE, the logistic EuroSCORE and others [14]. Even though there are some reports of the EuroSCORE predicting outcome after combined valve and coronary bypass surgery [5], there are other studies which have shown that there are limitations to the ability of the EuroSCORE to accurately predict outcomes after valve or combined operations [68]. Internationally, the evidence is highly suggestive that additive EuroSCORE performance generally over-estimates mortality at lower EuroSCOREs (EuroSCORE
6) and under-estimates mortality at higher EuroSCOREs (EuroSCORE > 13) [9]. In our own patient population in the north west of England, it was found that the logistic EuroSCORE overestimated risk in the patients undergoing valve surgery with or without coronary artery bypass grafting. In addition the logistic EuroSCORE significantly over-estimates mortality in the high risk patients (additive EuroSCORE > 5) [8].
We therefore aimed to develop and validate a multivariate prediction model for in-hospital mortality following aortic valve surgery in a multi-centre UK setting to establish a contemporaneous tool for risk-adjustment.
| 2. Methods |
|---|
|
|
|---|
Data was collected on a total of 4550 consecutive patients undergoing aortic valve replacement between 1 April 2001 and 31 March 2004 in the north west of England. Aortic valve replacement with concomitant coronary artery bypass grafting (CABG) was included in the study. Patients undergoing aortic valve replacement that was combined with other valve repair or replacement, resection of a ventricular aneurysm, surgery on the thoracic aorta, or other surgical procedure were excluded.
Data collection methods and definitions are available from http://www.nwheartaudit.nhs.uk and have been previously published [10]. In brief, each operation had a dataset collected which included demographics, heart disease severity, acuity, co-morbidity, procedural details and outcome. Validation of data was conducted in each centre, which involved checking each record for completeness and flagging back to the relevant surgical team any erroneous data. All records entered onto the databases were also cross-checked against finance activity lists and theatre books to ensure capture of all cases. Data was collected in each hospital and returned to a central source for analysis on a 6-monthly basis. Data would be returned to the providing hospital if data completeness did not achieve a rate of 98% or above. Any missing risk factor data after acceptance into the central registry was treated as absent and this occurred in less than 2%.
Patient characteristics collected are listed in Table 1 and were defined in accord with the Society of Cardiothoracic Surgeons of Great Britain and Ireland minimum dataset [Fifth National Adult Cardiac Database ReportThe Society of Cardiothoracic Surgeons of Great Britain and Ireland: http://www.ctsnet.org/file/5thBlueBook2003.pdf]. Renal dysfunction was defined as serum creatinine > 200 µmol/l or history of dialysis support or functioning renal transplant. New York Heart Association class IV was defined as cardiac disease resulting in an inability to conduct any physical activity without discomfort. Symptoms of cardiac failure may be present even at rest. If any physical activity is undertaken discomfort is increased. Hypertension was defined as a history of blood pressure > 140/90 mmHg or lower if treated. Atrial fibrillation was defined as the occurrence of chronic or paroxysmal atrial fibrillation on admission or prior to surgery. Non-elective surgery was defined as patients who have not been scheduled for routine admission and require urgent or emergency surgery during the current admission for medical reasons. Such patients are unable to be sent home prior to receiving surgery and may require treatment within 24 h of the admission or even cardiopulmonary resuscitation on route to theatre. Cardiogenic shock was defined as hypoperfusion with a systolic blood pressure < 80 mmHg and central filling pressure > 20 mmHg without inotropes or a cardiac index < 1.8 l/min m2 or inotropes + IABP required to maintain CI > 1.8 l/min m2. The outcome for the study was in-hospital mortality, defined as death within the same hospital admission regardless of cause.
|
The statistical model was internally validated using the technique of bootstrap resampling [13]. This technique is efficient and provides nearly unbiased estimates of the predictive accuracy of the model [14]. We developed the multivariate model on the entire dataset, then, 100 samples of 70% were drawn at random with replacement. The ROC curve was calculated for each sample. This allowed the calculation of the standard deviation of the mean ROC statistic. External validation of the model was carried out on 816 consecutive aortic valve replacement cases from the north west of England covering the time period 1 April 2004 to 31 March 2005. Logistic EuroSCORE was also calculated for each patient to assess its ability to predict accurately the risk of death following aortic valve replacement surgery compared to our own contemporary local model.
A simplified clinical risk assessment tool was developed from the multivariate risk prediction model and was scored by rounding the adjusted odds ratio for each variable to the nearest 0.5. These weights were then summed. The relationship between this clinical risk score and the probability calculated from the risk prediction model was read from a graph. This clinical risk assessment tool therefore approximates the risk that would have been calculated from the risk prediction model.
In all cases a p-value < 0.05 was considered significant. All statistical analysis was performed with SAS for Windows Version 8.2.
| 3. Results |
|---|
|
|
|---|
The patient characteristics are reported in Table 1. The majority of patients were male (63.1%) and the median age of the cohort was 69 (25th and 75th percentiles: 6175) years. Concomitant CABG occurred in 41.5% of cases.
3.2 Univariate association with in-hospital mortality
Table 1 shows the univariate association with in-hospital mortality. Significant patient characteristics included increasing age, female gender, angina and New York Heart Association class, renal dysfunction, peripheral vascular disease, respiratory disease, hypertension, atrial fibrillation, poor ejection fraction, previous cardiac surgery, non-elective surgery, cardiogenic shock and concomitant CABG.
3.3 Independent risk factors for in-hospital mortality
The independent risk factors for in-hospital mortality, along with co-efficients, standard errors, odds ratios, confidence limits, and p-values, are shown in Table 2
. The area under the ROC curve for the multivariate prediction model was 0.78 (Fig. 1
). The predicted risks of individual patients were rank-ordered and divided into 10 groups. Within each group of estimated risk, the number of in-hospital deaths predicted was compared with the number of observed in-hospital deaths. The HosmerLemeshow goodness-of-fit statistic across groups of risk was not statistically significant (Fig. 2
, p
= 0.73), indicating little departure from a perfect fit.
|
|
|
|
Applying the logistic regression equation to data for 816 consecutive aortic valve replacement cases performed between 1 April 2004 and 31 March 2005 revealed a ROC curve of 0.78. This was similar to previous values, indicating a good discrimination power. The HosmerLemeshow test was not statistically significant with a p-value of 0.15. The expected mortality compared to observed mortality in this validation period was 4.7% versus 4.1%, respectively.
The logistic EuroSCORE had an acceptable ROC curve of 0.78 when applied to the 816 patients within the validation dataset. However, the expected mortality predicted by the logistic EuroSCORE was 8.1% which is almost double that predicted by the locally derived model. The Hosmer-Lemeshow test was also significantly different with a p-value of 0.005, as the logistic EuroSCORE over-predicted risk in the low- to medium-risk groups.
3.5 Simplified model
A simplified clinical risk assessment tool derived from the logistic regression equation, described at the bottom of Table 2, is shown in Fig. 3
. Using the logistic equation, a 77 years old, with renal dysfunction, ejection fraction < 30%, and concomitant CABG would have a 23.3% risk of in-hospital mortality. However, using the simplified model in Fig. 3, the same patient would have an in-hospital death score of 10.5, which approximates to around a 20% risk of in-hospital mortality.
|
| 4. Discussion |
|---|
|
|
|---|
The logistic EuroSCORE is an internationally recognized risk prediction model for mortality following cardiac surgery. However, growing evidence shows that the logistic EuroSCORE is becoming outdated [8]. Although relatively new to many cardiac surgeons with it being introduced in a publication by Roques and colleagues in 2003 [3], the logistic EuroSCORE was actually based on patients operated between September and November 1995 and was originally released as a simple additive score for easy use. Our results clearly show that although the ROC curve is good at 0.78, the model over-predicts mortality by almost two times that observed in our aortic valve replacement cohort. This could of course be because, as a region, the north west of England is exceptional at aortic valve replacement, however as our mortality rates are similar to the national average of 5% this is unlikely to be the case [Fifth National Adult Cardiac Database ReportThe Society of Cardiothoracic Surgeons of Great Britain and Ireland: http://www.ctsnet.org/file/5thBlueBook2003.pdf]. The more likely reason for the over-prediction in mortality is that the logistic EuroSCORE is calibrated to predict a mortality rate from the mid-1990s and not contemporary practice were the mortality results observed have been significantly reduced. The risk model contained within this article is contemporary and therefore predicts more accurately the risk of mortality for patients currently undergoing aortic valve surgery.
There are previously published studies which have developed preoperative risk prediction models for heart valve operations [1923]. The majority of them are based on different patient populations in the United States of America [1922]. We could find one risk prediction model for heart valve surgery in a UK population, based on the database of the Society of Cardiothoracic Surgeons of Great Britain and Ireland (SCTS) [23]. This study included a final study sample of 32,839 patients after excluding 9213 patients for reasons of missing data. Even though this study included a very large number of patients in the analysis, exclusion of a significant number of patients from the initial data set may bias the results if the excluded patients are not representative of the ones included in the data analysis. The best way of producing reliable conclusions from retrospective analysis of data is to include consecutive patients in the data analysis, and make the best attempt to have the data complete for all patients. In our study, the data were complete in greater than 98% of patients. No patients were excluded from the initial data set, and any missing risk factor data after acceptance into the central registry was treated as absent and this occurred in less than 2%. This approach has been used previously in other studies [24]. In addition the accuracy of our data was fully validated. These factors add to the robustness of the data, and hence its analysis and the accuracy of risk prediction.
The mortality risk prediction model for valve surgery developed by the Northern New England Cardiovascular Disease Study Group (NNECDSG) [21] developed a mortality risk prediction tool for aortic and mitral valve surgery. Their data set included 5793 consecutive patients undergoing aortic valve replacement between 1991 and 2001. Registry data was quite complete, however serum creatinine, which was one of 11 independent risk factors for mortality, had 20% missing values and the technique of non-informative missing value imputation was used to rectify this [14]. In comparison, our data is more complete and the time period when the operations were performed (19972004) is more contemporaneous. The independent predictors in their study included body surface area and previous stroke, and these were not found to be significant in our risk prediction model even though another measure of body habitus, body mass index, and cerebrovascular disease were included in the univariate analysis. Left ventricular ejection fraction < 30% and hypertension were independent predictors in our model, while this was not the case in the NNECDSG prediction tool. In addition the NNECDSG developed a separate model for mitral valve replacement.
Providence Health System Cardiovascular Study Group (PHS) [22] validated the NNECDSG model on their own data set of 3324 patients who underwent aortic valve replacement between 1997 and 2004. In addition they modified the NNECDSG model by using additional risk factors used in other heart valve risk models and developed their own single risk model for both aortic and mitral valve replacement. However, in this study, of the total of 6578 patients who underwent heart valve surgery, only 4920 cases met with the NNECDSG criteria, and the rest were excluded from the data set. This again could bias the results of the analysis since consecutive patients were not included in the dataset.
Even though the use of a simplified model converting additive points into approximate risk as shown is Fig. 3 is very convenient and user friendly, the use of the logistic equation is more reliable and accurate, and is the recommended method of using our risk prediction tool. With the use of hand held computers and modern day technology, the logistic equation can be keyed into any electronic spreadsheet, therefore making additive models redundant. However, we have included a simplified model if one prefers to use it as a rough guide during clinical consultation, giving a fairly close estimate of the predicted risk to the patient during the consenting process.
There are limitations with this model which need to be considered. The model includes only the preoperative variables available on our dataset, and there may other variables that could potentially affect outcomes, but which are not routinely collected. The model does not take into account intraoperative variables such as surgical technique, myocardial protection, and other individual surgeon practices. In addition, some of the data defining preoperative cardiac function is quite crude, and includes only left ventricular ejection fraction, which is very observer dependent. More sophisticated parameters such as left ventricular end systolic and end diastolic dimensions, left ventricular wall thickness and right heart pressures are not routinely measured in all patients, and these may have a significant impact on postoperative outcomes. In addition, although the data used to develop the model has been subjected to local validation and has the confidence of clinicians, the performance of the model in predicting in-hospital mortality in aortic valve replacement patients outside of the north west of England is still to be tested and a process of external validation is necessary to check the validity of the model across other geographical areas.
In conclusion, we have developed a contemporaneous multivariate prediction model for in-hospital mortality following aortic valve replacement. This tool can be used in day-to-day practice to calculate patient-specific risk by the logistic equation or a simple scoring system with an equivalent predicted risk. This tool may also be useful for risk adjusting hospital and operator mortality figures, allowing for appropriate comparisons of in-hospital mortality following AVR surgery.
| Appendix A |
|---|
|
|
|---|
Dr R. Frater (Bronx, New York): You dont mention whether there was any difference between the four hospitals that you studied.
Dr Kuduvalli: The mortality in the four hospitals ranged from 4.1% to 5.6%. There were small differences within the hospitals.
Dr Frater: And no statistical significance?
Dr Kuduvalli: No statistical significance.
Dr Frater: This kind of study is very interesting. I noticed it's the North West Quality Improvement Program in Cardiac Surgery. To get quality improvement, you are going to have to look at a whole set of other things, which is the way the operation is conducted. In the New York state system you could have two hospitals out of 32 that were better than average, beating the risk, so to speak, and two hospitals at the bottom that were worse than average, all the rest in between. But the way those two at the top would actually do better than expected was to take all of the patients, not exclude anybody, and then do it better than the others were doing it. So that is what you have to concentrate on in quality improvement. It is nice to be able to say you have a risk of 20% or 2% or whatever, but you need to look at all the other factors that go into a cardiac operation to be able to do quality improvement. Are you planning to do something like that?
Dr Kuduvalli: There are limitations to the way you can predict or develop a risk tool with the data available. Obviously there is an area of 0.22 above the ROC curve which is entirely unaccounted for, and that is usually the case in any study of this kind. We use the data which is readily available from the minimum data set which we collect, and obviously there will be other factors such as surgeon specificity, myocardial protection, different surgical techniques, et cetera, which will contribute to the mortality of a patient but which cannot be recorded or are not routinely recorded on the database, and that limitation exists with any risk score normally when you try and develop them.
Dr R. Schistek (Salzburg, Austria): Is this software available for calculating your risk score?
Dr Kuduvalli: The score is actually calculated using a logistic equation. It is quite complex so I havent put it up on the slide here, but you can actually feed the logistic equation into an Excel spreadsheet, and if you put the data in an Excel spreadsheet it will give you the estimated risk. It can be quite easily done, we have done it, but you just need an Excel spreadsheet, which is fairly commonly available and most people know how to use it.
Dr Schistek: I need the equation also. Can you provide the equation?
Dr Kuduvalli: I dont have it here on my slide at the moment. This manuscript has been submitted for publication and it will be there in the manuscript. It is not there in the slides at the moment, I am afraid.
Dr K. Khargi (The Hague, Netherlands): If you are using this model, I always correlate it to the EuroSCORE where you basically have the same principle. Your model, however, suggests to be more specific by including more criteria such as atrial fibrillation, New York Heart Classification and other presented parameters. Still I would like to encourage you to correlate it always to the EuroSCORE, because that will eventually be the model, which has to be refined more specifically for either aortic valves or CABGs or combined procedures.
Dr Kuduvalli: I appreciate your comment. The EuroSCORE did not actually concentrate only on valve replacement patients, it was a whole group of patients, and it concentrated quite significantly on coronary artery bypass grafting. This particular data set is concentrating on aortic valve replacement, so hopefully it will be more specific for aortic valve replacement patients.
The second thing is that the data in this particular data set is extremely finely validated; it is almost 98% complete. Anybody who sent the data to the central registry and was less than 98% complete was sent back, and there was less than 2% of data which was not complete in the whole data set. So we think that adds to the robustness of the data and it is more than 4000 patients, so we think it might be a good tool and very specific to the particular operation.
Dr Khargi: Yes, I agree with that.
Dr J. Bonatti (Innsbruck, Austria): Have you discussed including parameters that are more specific to aortic valve disease. I am thinking about factors like extent of left ventricular hypertrophy or diastolic dysfunction?
Dr Kuduvalli: That would be ideal, I totally agree with you, but one of the limitations of this data is that the left ventricular function is determined by very crude parameters like left ventricular ejection fraction, and I totally agree with that. It would be ideal to have more sophisticated parameters like left ventricular internal dimensions or left ventricular wall thickness. Unfortunately, we dont record this on our database as a matter of routine, and as Dr Frater suggested, there will be multiple variables like that which will contribute to mortality but will unfortunately not be reflected in this data model because of the limitations of collecting data. I agree with that.
| Acknowledgments |
|---|
We would also like to thank for their considerable efforts Stephen Bullough, Suzanne Chaisty, Janet Deane, Jenni Law and Catherine Malpas, who maintain the quality and ensure completeness of data collected in our Cardiac Surgery Registry. A special thank you to Dr Mark Jackson for statistical advice.
| Footnotes |
|---|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
M. van Gameren, N. Piazza, A. J J C Bogers, J. J M Takkenberg, and A P. Kappetein How to assess risks of valve surgery: quality, implementation and future of risk models Heart, December 1, 2009; 95(23): 1958 - 1963. [Full Text] [PDF] |
||||
![]() |
S. M. O'Brien, D. M. Shahian, G. Filardo, V. A. Ferraris, C. K. Haan, J. B. Rich, S.-L. T. Normand, E. R. DeLong, C. M. Shewan, R. S. Dokholyan, et al. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 2--isolated valve surgery. Ann. Thorac. Surg., July 1, 2009; 88(1 Suppl): S23 - S42. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Lehmann, T. Walther, S. Leontyev, J. Kempfert, J. Garbade, M. A. Borger, and F. W. Mohr The Toronto Root Bioprosthesis: Midterm Results in 186 Patients. Ann. Thorac. Surg., June 1, 2009; 87(6): 1751 - 1756. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Khaladj, M. Shrestha, S. Peterss, I. Kutschka, M. Strueber, L. Hoy, A. Haverich, and C. Hagl Isolated surgical aortic valve replacement after previous coronary artery bypass grafting with patent grafts: is this old-fashioned technique obsolete? Eur. J. Cardiothorac. Surg., February 1, 2009; 35(2): 260 - 264. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. M. Brown, S. M. O'Brien, C. Wu, J. A. H. Sikora, B. P. Griffith, and J. S. Gammie Isolated aortic valve replacement in North America comprising 108,687 patients in 10 years: changes in risks, valve types, and outcomes in the Society of Thoracic Surgeons National Database. J. Thorac. Cardiovasc. Surg., January 1, 2009; 137(1): 82 - 90. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Banach, A. Goch, M. Misztal, J. Rysz, R. Jaszewski, and J. H. Goch Predictors of paroxysmal atrial fibrillation in patients undergoing aortic valve replacement. J. Thorac. Cardiovasc. Surg., December 1, 2007; 134(6): 1569 - 1576. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |