HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Nashef, S.A.M. Articles by Charman, S. Search for Related Content PubMed PubMed Citation Articles by Nashef, S.A.M. Articles by Charman, S. Related Collections Professional affairs Cardiac - other

Eur J Cardiothorac Surg 2001;19:817-820
© 2001 Elsevier Science NL

The relationship between predicted and actual cardiac surgical mortality: impact of risk grouping and individual surgeons

Papworth Hospital, Cambridge CB3 8RE, UK

Received 11 October 2000; received in revised form 26 March 2001; accepted 3 April 2001.

Corresponding author. Tel.: +44-1480-364299; fax: +44-1480-364744
e-mail: sam.nashef{at}papworth-tr.anglox.nhs.uk

	Abstract

Top Abstract 1. Introduction 2. Patients and methods 3. Results 4. Discussion Appendix A. Conference... References

 
Objective: To study the relationship between predicted and actual mortality in a cardiac surgical practice and to determine whether there is a consistent relationship across risk groups and surgeons. Methods: Risk information (Parsonnet score) was prospectively collected for 6213 consecutive adult patients undergoing cardiac surgery at one institution. The relationship between predicted mortality and actual mortality was analysed by risk group for all patients and for individual surgeons’ practices. Results: Predicted mortality was 10.2%. Actual mortality was 4.2%, giving a mortality ratio of 41% of predicted. This ratio was not consistent across the five major risk groups, ranging from 32% in moderate risk to 67% in very low risk patients. When analysed by individual surgical practices, the results were even more disparate, with a mortality index range between 0% for one surgeon's low risk patients to 150% for another surgeon's very low risk patients. Conclusion: The relationship between predicted and actual mortality at one institution may vary across the risk spectrum and between surgeons. This should be taken into account in preoperative risk assessment and informed patient consent. Individual surgeons may have strengths and weaknesses which are related to preoperative risk stratification.

Key Words: Mortality • Cardiac surgery • Risk stratification

	1. Introduction

Top Abstract 1. Introduction 2. Patients and methods 3. Results 4. Discussion Appendix A. Conference... References

 
Preoperative estimation of the operative mortality is important for a number of reasons: it is useful in the determination of an indication to operate, particularly where operation is being performed at least partly on prognostic grounds. It is therefore an integral part of surgical decision making. In an age of increasing patient awareness and sophistication, operative risk estimation also becomes an integral part of informed consent. Finally, it is useful in the assessment of performance of surgeons and institutions. It is useful to review the many methods by which mortality risk can be quoted for a particular procedure on a particular patient:

1. The lowest mortality achieved in the best published series on that procedure.
   
2. The average mortality achieved in a meta-analysis of major series.
   
3. The average mortality achieved nationally (from publications or registry data).
   
4. The mortality for that procedure in the hospital in question.
   
5. The mortality for that procedure for the surgeon in question.
   
6. The risk-stratified mortality for the patient and procedure using a scoring system.
   
7. The risk-stratified mortality, adjusted for the performance of the hospital.
   
8. The risk-stratified mortality, adjusted for the performance of the surgeon.
   

It can be clearly seen that the best, and perhaps the most honest, approach is the last: assessing predicted mortality by risk stratification and adjusting it by the performance of an individual surgeon in relation to the same risk stratification system. The purpose of this paper is to explore the relationship between predicted and observed mortality, particularly the impact on this relationship of risk grouping and individual surgeons.

	2. Patients and methods

Top Abstract 1. Introduction 2. Patients and methods 3. Results 4. Discussion Appendix A. Conference... References

 
Between 1996 and 2000, 6213 consecutive patients undergoing open heart surgery at one institution were prospectively risk-stratified using the standard Parsonnet score [1] and all risk and outcome data entered onto a database. Mortality was defined as death within 30 days or within the same hospital admission as operation.

The patients were divided into five risk groups according to Parsonnet predicted mortality risk (scores 0–4, 5–9, 10–14, 15–19 and 20+). For each risk group the risk-adjusted mortality ratio (RAMR) was calculated as 100x the actual death rate divided by the mean predicted death rate. 95% confidence intervals for the RAMR were calculated assuming that the observed deaths follow a Poisson distribution. To assess whether there was any heterogeneity between observed and Parsonnet predicted mortality rates the chi-squared test used. First, the observed deaths were compared to the unadjusted predicted deaths (column 4 of Table 1). Once it was established that Parsonnet overpredicts mortality, expected death rate was divided by a constant amount (the hospital overprediction factor) in each group and the chi-squared test was repeated to assess whether the discrepancy between actual and predicted deaths was significantly different across risk groups. This second test used the adjusted predicted deaths, in column 6 of the Table 1.

	3. Results

Top Abstract 1. Introduction 2. Patients and methods 3. Results 4. Discussion Appendix A. Conference... References

 
Overall, predicted mortality was 10.2% and actual mortality was 4.2% (RAMR or observed mortality 41% of predicted). All surgeons had observed mortality rates well below predicted (Fig. 1).

View larger version (43K): [in this window] [in a new window]   Fig. 1. Actual and predicted mortality expressed as a percentage for seven individual surgeons A–G.

Observed and Parsonnet-predicted deaths are given in Table 1 and plotted in Fig. 2. It is clear from the table and the figure that the Parsonnet score overpredicts mortality (P<0.001), with hospital observed mortality 41% of Parsonnet predicted. Assuming that observed mortality is 41% of Parsonnet predicted in every risk group, expected death rates and adjusted RAMRs are given in columns 6 and 7 of the table. The difference between these RAMRs is no longer significant (P=0.768), despite the apparently high ratio in the low risk group.

View larger version (12K): [in this window] [in a new window]   Fig. 2. Actual to predicted mortality ratios (%) by five risk groups for the hospital with 95% confidence intervals.

View larger version (40K): [in this window] [in a new window]   Fig. 3. Actual to predicted mortality ratio (%) for individual surgeons A–G across five risk groups.

View larger version (16K): [in this window] [in a new window]   Fig. 4. ROC Curve for Parsonnet Score.

	4. Discussion

Top Abstract 1. Introduction 2. Patients and methods 3. Results 4. Discussion Appendix A. Conference... References

Measuring predicted mortality, by using Parsonnet, EuroSCORE or other models, provides a useful standard against which surgical performance can be assessed. Many hospitals in the UK have now set performance targets based on risk adjusted mortality ratios (RAMR). It is therefore important to study RAMRs in order to see how they are affected by individual surgeons and risk groups. This paper is a descriptive work aimed at exploring the relationship between actual and predicted mortality in one hospital using one scoring system. Because of the relatively small number of deaths, statistically significant differences were not found. Nevertheless, the tendency towards such differences is an incentive to extend this study to larger numbers of patients in more than one institution. Individual surgeons’ performances in relation to risk group have necessarily wide confidence intervals, yet they provide a useful guide to informed consent and surgical decision making.

Risk-adjusted mortality prediction is important in two separate fields. The first is quality monitoring: measuring actual versus predicted mortality in an institution allows assessment of clinical surgical performance while adjusting for the risk profile of the patients. Regardless of the weaknesses of the scoring systems, such monitoring is important so that major underperformance does not go undetected for long, and so that measures can be taken to correct underperformance before the risk to patients is significantly and adversely affected. The second field is that of informed consent and clinical decision-making. Risk-adjusted predicted mortality should form an important part of patient and surgeon decisions on whether or not to proceed with surgery. It is this particular aspect of risk stratification that our paper explores. We have shown that risk-adjusted mortality prediction may be usefully enhanced by reference to a hospital's and a particular surgeon's performance in relation to risk groups. Experience in the UK after the Bristol affair shows that it is no longer acceptable to quote predicted mortality figures to patients and their families if these figures are not consistent with local surgical outcomes. We believe that risk stratified predicted mortality, adjusted by the RAMR for the surgeon in question, provides the best available information for proper informed consent.

Finally, the Parsonnet model has reasonable discriminatory power but is known to overscore. Much of this overscoring is related to age and valve surgery and occurs mainly in the higher risk groups. It is therefore possible that some of our observed results may be affected by peculiarities of the scoring system itself. Further studies are needed to determine whether the pattern observed in this study will be repeated with a more robust and powerfully discriminatory system such as EuroSCORE [2].

	Footnotes

 
Presented at the 14th Annual Meeting of the European Association for Cardio-thoracic Surgery, Frankfurt, Germany, October 7–11, 2000.

	Appendix A. Conference discussion

Top Abstract 1. Introduction 2. Patients and methods 3. Results 4. Discussion Appendix A. Conference... References

 
Dr E. Wolner (Vienna, Austria): We performed a similar investigation with the Parsonnet score. We also experienced that the Parsonnet score overestimates the risk of a surgical procedure. Comparatively, the EuroSCORE underestimates, albeit minimal, the risk of a surgical procedure. As in your publication, we also observed a Parsonnet score of about 50% higher. Have you begun to evaluate your patients using the EuroSCORE?

Mr Nashef: Yes, we have. The problem is that even with a series of over 6000 patients it is difficult to achieve statistical significance.

Dr Wolner: I mean from the year 1999.

Mr Nashef: Yes, from the year 1999 we have been collecting EuroSCORE data, and I expect that in about 2 years time we will be able to carry the same analysis on EuroSCORE.

As you know, one of the difficulties with studies like this is that the skew may be either due to the individual surgeon's and hospital's performance, but it may be due to the peculiarity of the scoring system; and we all know that Parsonnet, for example, particularly overscores some groups of patients, and it may be that EuroSCORE will eliminate some of these difficulties, but we will be studying that in the future.

Dr A. Arbulu (Detroit, MI, USA): I suggest that this is an important paper. It is very important to stratify the patients by their co-morbidities and also correlate the results of operations to the number of operations each surgeon performs. I am the senior surgeon in my group at the medical center, and I have the best results. The reason is very simple: I select my cases, I operate one or two cases a week, and therefore I am the shining star. There are other surgeons that operate five times in a week what I operate upon. So I think it this extremely important, because otherwise, at least in the United States, in the famous ‘report cards’ that are being published in relation to the doctors, it is very important to be sure that this type of stratification is done in fairness to our extremely busy and very good surgeons.

Dr F. Grover (Denver, CO, USA): I think one comment is that the relatively high risk prediction for the Parsonnet score probably reflects the fact that that has been in place for a number of years, and I think, again, emphasizes the importance, at least what we try to do with the STS database and the United States VA database, to recalculate the risk coefficient on an annual basis, and that would probably correct for that.

I was curious what the overall individual volumes of your surgeons is. We used this in our own institution also to look at each of us individually, but you obviously have to be very careful when you start breaking it down by surgeon, because you tend to have a small number and you can get a lot of variation in your results and have to be careful in how you interpret that.

Mr Nashef: Yes, I fully agree with you, and what I showed you as to individual surgeons’ results was really purely descriptive. If you place confidence intervals on those, they would be very wide because of the relatively small numbers.

Dr J. Vaage (Stockholm, Sweden): I enjoyed your paper, however, some of your findings were quite unexpected with these intersurgeon differences. I was particularly surprised by the fact that there was a surgeon who had very good results in the high risk group and the highest risk in the low risk group. Have you discussed possible explanations for some of these intersurgeon differences?

Mr Nashef: Yes, we have discussed them, and in fact we have instituted some measures to deal with them. But I think it is important to point out that in spite of the fact that there are surgeons who perform better in one group than another, all of the surgeons’ collective performance is well within the predicted Parsonnet risk, indeed it is only a fraction of it, and overall these are results that we are proud of. The differences are in small subgroups with wide confidence intervals and they do need to be addressed, but we do not perceive them as a problem.

Dr A. Arifi (Hong Kong, China): I totally agree with you that surgical skills influence the surgical results, as you mentioned. I use the EuroSCORE and Parsonnet score in the Asian population and have noted that the Parsonnet score is quite overestimating and the EuroSCORE is slightly underestimating.

One thing I would like to ask, why do you think some surgeons have high mortality in the low risk while an expected mortality in the high risk?

Mr Nashef: We are now going into the realm of speculation, but one possible explanation for this is a bias in your type of surgery. As we know, Parsonnet tends to be quite accurate in coronary artery bypass grafts in the low risk groups and tends to widely overscore the elderly with valve problems. So your practice may be skewed in that way.

Dr Y. Balbaa (Cairo, Egypt): I have used the Parsonnet scoring system myself on a very large group of patients from Cairo University, and I would like to tell you that I have contacted Mr Parsonnet, and he also agrees that his score is overestimating mortality, and since 1999 he has totally changed the scoring system and he has been using a new system for the last year.

Mr Nashef: Yes, I agree with you, there are several versions of Parsonnet now. Unfortunately the other versions have not really been accepted or have not been used widely internationally, and it is difficult for an audience to relate to the multiple versions that have come out, and that is why we thought we would stick with the original plan and adjust for the overestimation.

	References

Top Abstract 1. Introduction 2. Patients and methods 3. Results 4. Discussion Appendix A. Conference... References

 

1. Parsonnet V., Dean D., Bernstein A.D. A method of uniform stratification of risk for evaluating the results of surgery in acquired adult heart disease. Circulation 1989;79(6 Pt 2):12-13.
2. Nashef S.A.M., Roques F., Michel P., Gauducheau E., Lemeshow S., Salamon R. European system of cardiac operative risk evaluation (EuroSCORE). Eur J Cardio-thorac Surg 1999;16:9-13.[Abstract/Free Full Text]

This article has been cited by other articles:

				D. Kalavrouziotis, K. J. Buth, and J.-F. Legare Patient selection for minimally invasive aortic valve implantation Eur. J. Cardiothorac. Surg., July 1, 2007; 32(1): 188 - 188. [Full Text] [PDF]

				J. Bergsland, E. Kabil, E. Mujanovic, I. Terzic, J. Roislien, J. L. Svennevig, and E. Fosse Training of Cardiac Surgeons for Bosnia and Herzegovina: Outcomes in Coronary Bypass Grafting Surgery Ann. Thorac. Surg., February 1, 2007; 83(2): 462 - 467. [Abstract] [Full Text] [PDF]

				A.A. Albert, J.A. Walter, B. Arnrich, W. Hassanein, U.P. Rosendahl, S. Bauer, and J. Ennker On-line variable live-adjusted displays with internal and external risk-adjusted mortalities. A valuable method for benchmarking and early detection of unfavourable trends in cardiac surgery Eur. J. Cardiothorac. Surg., March 1, 2004; 25(3): 312 - 319. [Abstract] [Full Text] [PDF]

				S. C. Stoica, L. D. Sharples, I. Ahmed, F. Roques, S. R. Large, and S. A.M. Nashef Preoperative risk prediction and intraoperative events in cardiac surgery Eur. J. Cardiothorac. Surg., January 1, 2002; 21(1): 41 - 46. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Nashef, S.A.M. Articles by Charman, S. Search for Related Content PubMed PubMed Citation Articles by Nashef, S.A.M. Articles by Charman, S. Related Collections Professional affairs Cardiac - other