HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Gallivan, S. Articles by Stark, J.F. Search for Related Content PubMed PubMed Citation Articles by Gallivan, S. Articles by Stark, J.F.

Eur J Cardiothorac Surg 2001;20:1214-1219
© 2001 Elsevier Science NL

Early identification of divergent performance in congenital cardiac surgery

^a Clinical Operational Research Unit, Department of Mathematics, University College London, 4 Taviton Street, London, WC1H 0BT, UK
^b Institute of Child Health, London, UK
^c Centre for Mathematics and Physics in the Life Sciences and Experimental Biology, University College, London, UK

Received 13 June 2001; received in revised form 6 September 2001; accepted 11 September 2001.

Corresponding author. Tel.: +44-20-7679-4508; fax: 44-20-7813-2814
e-mail: s.gallivan{at}ucl.ac.uk

	Abstract

Top Abstract 1. Introduction 2. A standardized VLAD... 3. A practical example... 4. The complexity profile... 5. Discussion References

 
Objectives: Heterogeneous caseload and poorly quantified risk stratification make it difficult to monitor outcomes in congenital cardiac surgery. Reliance on formal statistical hypothesis testing may lead to substantial delays before a pattern of poor outcome can be established. Here, we report alternative methods for alerting surgeons to potential problems at an earlier stage. Methods: Graphical methods developed for monitoring adult cardiac surgery have been adapted for use in congenital cardiac surgery. To illustrate their potential, we have retrospectively examined mortality data for a single surgeon involving 315 cases. Partial risk adjustment has been carried out according to patient's age and the open/closed categorization of the surgical procedure. Additional information has been derived by ranking procedures in order of their complexity and displaying the proportion of the surgeon's cases in each complexity stratum. Results: The display of a surgeon's mortality data adjusted for age and open/closed category provides an easily understood chart of performance and allows one to identify periods when performance appears divergent, relative to the surgeon's own overall standards. Cases carried out during such periods can then be scrutinized by alternative methods. One such method is to examine caseload complexity during the periods of apparent divergent performance compared with other periods. Conclusions: These methods, while in no way representing formal statistical tests, provide a means that can alert surgeons to potential problems and help to identify sequences of cases that might benefit from further scrutiny.

Key Words: Congenital cardiac surgery • Monitoring • Outcomes • Mortality • Risk • Complexity

	1. Introduction

Top Abstract 1. Introduction 2. A standardized VLAD... 3. A practical example... 4. The complexity profile... 5. Discussion References

 
There is growing pressure for more widespread introduction of methods for monitoring outcomes for clinicians. This is particularly so in the UK, in the wake of events at the Bristol Royal Infirmary where concerns about the standards of cardiac surgery led the government to mount a public inquiry. It is ironic that cardiac surgery should find itself at the centre of this debate since the field is very much at the better end of the spectrum as regards recording and scrutinizing outcomes, well ahead of most other clinical specialities.

Methods for monitoring outcomes, particularly the use of so-called Cusum charts [1], are now common in adult cardiac surgery. In its simplest form, a Cusum chart (an abbreviation for Cumulative Sum Chart) is a chart giving a running tally of the total number of peri-operative deaths that occur during the course of a series of operations. Such charts provide a useful analytical tool to assist the clinical audit process. Various modifications to the Cusum methods have been devised. Preoperative risk factors have been used to develop scoring systems, such as Parsonnet's score or Euroscore [2,3] that can forecast the risk of peri-operative death. Charts taking such preoperative risk factors into account have been developed for use in adult cardiac surgery to display overall performance and to highlight any disturbing trends [4]. These charts are called Variable Life Adjusted Displays, or VLAD charts, which is their commonly used acronym.

It would clearly be useful to have such methods for monitoring outcomes in congenital heart surgery (CHS), however, there are many difficulties. A paediatric cardiac surgeon performs many palliative and corrective procedures and often operates on patients with complex defects who may require more than one procedure, far more frequently than is the case in adult cardiac surgery. Thus, collecting and analyzing data in CHS has been considerably more difficult than in adult cardiac surgery, where surgeons perform relatively large numbers of fairly standard operations, such as coronary bypass or valve replacement and performance monitoring can focus on these operations.

The absence of a scoring system for preoperative risk factors in CHS presents another difficulty. Risk factors influencing outcome of operations have been known for some time. They include young age, prematurity, low weight at operation, use of a heart lung machine, preoperative ventilation and/or dialysis, presence of associated non-cardiac congenital defects and ‘syndromes’, multiplicity and complexity of congenital heart defects, etc. As we do not have quantitative assessment of most of these factors, we cannot incorporate them into the risk scoring models used for construction of VLAD charts.

Although our knowledge of the importance of the risk factors in CHS is imperfect, some quantitative evidence is available [5–7]. We have devised a modification of the VLAD plot, referred to as a standardized VLAD plot that uses such information to assist congenital heart surgeons in monitoring their performance. In this paper, we discuss the differences between this standardized VLAD plot and the ‘original’ VLAD plot [4]. We describe the method for constructing these plots. Using examples, we suggest how this method could help surgeons to identify periods where their performance diverges from their own standards.

	2. A standardized VLAD plot based on partial risk stratification

Top Abstract 1. Introduction 2. A standardized VLAD... 3. A practical example... 4. The complexity profile... 5. Discussion References

 
The standardized VLAD plot is based on a modification of the VLAD charting method used in adult cardiac surgery [4]. We shall first describe the original VLAD technique.

Preoperative risk factors can be used to derive a cumulative plot for the number of deaths expected from a series of cases based on individual risk forecasts.

Fig. 1a illustrates a typical cumulative plot of expected mortality superimposed on a cumulative plot of the deaths that did actually occur, the latter being the Cusum chart. The difference between these two gives an estimate for the ‘net life gain’. A VLAD chart is a plot of this ‘net life gain’ against the number of operations that a surgeon has performed. It is shown in Fig. 1b, using fictitious data for illustration purposes. Here, surgeon A is performing rather poorly since the downward trend of his VLAD plot indicates higher mortality than would have been predicted by the risk scoring model. On the other hand, the overall mortality for surgeon B broadly matches what would have been expected from the risk model, and his VLAD plot centres on the horizontal axis. Surgeon C is generally performing better than the risk model predicts.

View larger version (35K): [in this window] [in a new window]   Fig. 1. (a) Derivation of net lives saved comparing actual cumulative mortality and what would be predicted using estimates of preoperative risk for each patient (fictitious data). (b) Use of such values to plot VLAD charts comparing three (fictitious) surgeons. Charts that move upwards indicate performance better than expected, those that move down indicate performance worse than expected.

To do this, two forms of information are used. Firstly, there is the surgeon's own average mortality rate over the review period in question. Secondly, a categorization of each of the surgeon's cases according to whether the operation is open or closed, and according to the age band of the patient (neonate, 0–28 days; infant, 29 days to 1 year; child, over 1 year).

This information was available from a recent study [6] of results in congenital cardiac surgery from five UK departments over a 2 year period. The mortality rates disaggregating cases according to whether operations were open or closed and according to the age band are summarized in Table 1.

	3. A practical example of the method's use

Top Abstract 1. Introduction 2. A standardized VLAD... 3. A practical example... 4. The complexity profile... 5. Discussion References

 
In this section, we illustrate the method using data on operations performed by one congenital heart surgeon between 1 April 1997 and 31 March 1999.

All cardiac operations classified as either open or closed were included in the present analysis. Other operations, such as ECMO, thoracic procedures and procedures for complications, were excluded. Altogether, 315 operations satisfied these criteria. A standardized VLAD plot has been produced for the surgeon as described in the previous section. The plot in Fig. 2 illustrates that initially, the surgeon performed better compared with his ‘average’ performance until a period of relatively worse performance was noted. Thus, this plot could alert the surgeon and stimulate a detailed review of operations performed during such a period comparing them with operations in periods when the curve indicated better than average performance. To assist with this review, a second analytical technique was developed.

View larger version (18K): [in this window] [in a new window]   Fig. 2. A standardized VLAD plot for 315 congenital cardiac operations carried out by a single surgeon.

	4. The complexity profile of a surgeon's caseload

Top Abstract 1. Introduction 2. A standardized VLAD... 3. A practical example... 4. The complexity profile... 5. Discussion References

Accurate quantitative estimates of mortality rates are not yet available for individual operation types; it was therefore impossible to use the complexity of procedures in the construction of the standardized VLAD plots. Instead, a graphical method that helps to summarize the inherent complexity mix of a surgeon's case load has been devised.

As a starting point, open and closed operations have been categorized into six groups of increasing complexity as shown in Table 2. This categorization is a modification of the groupings originally described by Jenkins et al. [8] in 1994. The modification has been agreed by congenital heart surgeons participating in the recent study [5,6]. This grouping, as well as the one suggested by Jenkins [8], was based on subjective perception of operation complexity rather than mortality data. However, when we examined the average mortality rates in the six groups using data from one department, we found that there was a progressively increasing mortality from group 1 to group 6. This accords with, but does not establish, the plausible hypothesis that the complexity categorization reflects inherent risk.

In spite of the fact that the information summarized in Table 2 is qualitative and based on subjective opinion, it can still be analyzed using quantitative methods, as long as these only make use of ordinal information. One such method is based on examining the proportions of a surgeon's case load that fall in each of the six complexity groups. Although this can be done simply by plotting a frequency histogram (Fig. 3a) , it is more informative to plot a cumulative chart (Fig. 3b). Both Fig. 3a and Fig. 3b show fictitious data for illustration purposes.

View larger version (31K): [in this window] [in a new window]   Fig. 3. (a) Frequency histogram showing a complexity profile to illustrate case-mix differences between three surgeons (fictitious data). (b) Cumulative chart showing a complexity profile to illustrate case-mix differences between three surgeons (fictitious data).

Fig. 4 shows the complexity chart of a surgeon whose outcomes have been shown in the standardized VLAD plot in Fig. 2. Here, rather than comparing his case-mix with that of the other surgeons, we contrast the complexity profile for the period of apparently divergent performance with the complexity profile during the rest of the review period. It can be seen that the case-mix was not more complex during the period of divergent performance; indeed, the complexity profile chart suggests the reverse. In this case, the complexity profile provided little insight into the reasons for the divergent performance.

View larger version (12K): [in this window] [in a new window]   Fig. 4. Cumulative chart showing a complexity profile chart examining case-mix of a single surgeon during a period of divergent performance as illustrated in Fig. 2 compared with case-mix during remainder of review period.

	5. Discussion

Top Abstract 1. Introduction 2. A standardized VLAD... 3. A practical example... 4. The complexity profile... 5. Discussion References

For congenital cardiac surgery, the picture is very different. The complexity of congenital heart defects, the need for multiple procedures, either during the same operation or during the same admission, the lack of unified nomenclature, and small numbers in each subgroup of congenital heart defects all contribute to making formal analysis of outcomes difficult.

There are relatively few major data collection studies and there is little in the way of national or international standards. Relatively little has yet been done to examine risk factors in quantitative terms [7,9,10], and an accepted risk scoring system for congenital heart operations is not yet available.

In this paper, we have discussed two graphical methods that can assist the surgeon in monitoring his/her results. The standardized VLAD method helps to identify periods of divergent performance that would benefit from further investigation. Complexity profile charts provide one means of helping such investigation.

Users of the charting methods discussed in this paper should be mindful that they are designed purely to present a graphical summary of overall performance and to assist surgeons to identify periods of performance that might benefit from further scrutiny. In compiling such charts, there are many factors that could not have been taken into account in the analysis, and consequently, there is a danger of over-interpretation. This might lead either to undue concern or undue complacency. We should stress that the charting methods certainly do not represent formal statistical hypothesis testing.

Factors not included were: prematurity, weight at operation, whether patients were on dialysis or preoperative ventilation, emergency and elective status of operations, whether procedures were ‘redo’, the presence of associated non-cardiac congenital defects and ‘syndromes’.

The standardized VLAD charts described in this paper use hospital mortality as their sole outcome measure. However, with the improved surgical results, some, even complex procedures can be now performed with relatively low mortality [6,11,12]. Thus, mortality as the only outcome measure may not be adequate. Other outcome measures such as ‘near misses’ [1], rate of re-operations and rate of complications, length of postoperative ventilation, etc. may become helpful indicators of performance.

Outcome of operations is usually attributed to the performance of a surgeon. It is, however, important to realize that the skills of all members of the team are needed for the successful management of a patient with a congenital heart defect [13]. It is thus sensible that any review of the period of apparently divergent performance should include a review of the work of all members of the team and not just the surgeon.

The ethos underlying the new standardized VLAD plots is different in intent from that of the original VLAD charts which were designed to uncover sub optimal performance, as judged by comparisons with existing standards. Given the present state of knowledge about risk factors in CHS, and the absence of national or international standards, this would be a dubious prospect. Instead, our methods are intended for use in a more constructive fashion. The aim is to help clinicians to identify periods where their performance has apparently diverged relative to their own standards. Operations from such periods can then be scrutinized to determine if changes in clinical management are required which would to improve outcomes.

	Acknowledgments

 
The authors are grateful to the following surgeons whose data were used in this analysis: Mr M.J. Elliott, Mr J.R.L. Hamilton, Mr A. Hasan, Mr M.P. Haw, Mr M. Jamieson, Mr K. McArthur, Mr J.L. Monro, Mr J.C.S. Pollock, Miss K. Van Dorn, Mr K.G. Waterson and Mr V.T. Tsang. The authors are particularly grateful to Professor M.R. de Leval whose data were used to construct the plots in Figs. 2 and 4. Professor Gallivan and Mrs Davis are funded by a grant from the UK Department of Health.

	References

Top Abstract 1. Introduction 2. A standardized VLAD... 3. A practical example... 4. The complexity profile... 5. Discussion References

 

1. de Leval M.R., Francois K., Bull C., Brawn W., Spiegelhalter D. Analysis of a cluster of surgical failures. Application to a series of neonatal arterial switch operations. J Thorac Cardiovasc Surg 1994(107(3):914-924.
2. Parsonnet V., Dean D., Berstein A.D. A method of uniform stratification of risk for evaluating the results of surgery in acquired adult heart disease. Circulation 1989;779(6 Suppl I):I-3-I-12.
3. Nashef S.A., Roques F., Michel P., Gauducheau E., Lemeshow S., Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999(16(1):9-13.
4. Lovegrove J., Valencia O., Treasure T., Sherlaw-Johnson C., Gallivan S. Monitoring the result of cardiac surgery by variable life adjusted display (VLAD). Lancet 1997(350):1128-1130.
5. Stark J., Gallivan S., Lovegrove J., Hamilton J.R.L., Monro J.L., Pollock J.C.S., Watterson K.G. Mortality rates after surgery for congenital heart defects in children and surgeons’ performance. Lancet 2000;355:1004-1007.[Medline]
6. Stark J., Gallivan S., Davis K., Hamilton J.R.L., Monro J.L., Pollock J.C.S., Watterson K.G. Assessment of mortality rates for congenital heart defects and surgeons’ performance. Ann Thorac Surg 2001;72:169-175.[Abstract/Free Full Text]
7. Moller J. Paediatric Cardiac Care Consortium 1984–1995. Armonk, NY: Futura Publishing Company, Inc, 1998.
8. Jenkins KJ, Newburger JW, Kyn L, Lock JE, Mayer JE, Iezzoni LI. Procedure-specific mortality risk for congenital heart surgery. Presented at the 67th American Heart Association Scientific Sessions, Dallas, TX, November 14–17, 1994.
9. Hannah E.L., Racz M., Kavey R.E., Quaegebeur J.M., Williams R. Paediatric cardiac surgery: the effect of hospital and surgeon volume on in-hospital mortality. Pediatrics 1998;101:963-969.[Abstract/Free Full Text]
10. Jenkins K.J., Gauvreau K., Newburger J.W., Spray T., Moller J.H., Iezzoni L.I. Consensus-based method for risk adjustment. J Thorac Cardiovasc Surg 2001 in press.
11. Stark J. How to choose a cardiac surgeon. Circulation 1996;94(Suppl II):II-1-II-4.
12. Lundstrom N.R., Berggren H., Bjorkhem G., Sunnergardh J. Centralisation of paediatric heart surgery in Sweden. Paediatr Cardiol 2000;21:353-357.[Medline]
13. Hamilton JRL, Silove E. Report on the clinical case note review. Available at URL: http: www.bristol-inquiry.org.uk/ (Accessed July 2000).

This article has been cited by other articles:

				S. Gallivan, J. Stark, C. Pagel, G. Williams, and W. G. Williams Dead reckoning: can we trust estimates of mortality rates in clinical databases? Eur. J. Cardiothorac. Surg., March 1, 2008; 33(3): 334 - 340. [Abstract] [Full Text] [PDF]

				C. Sherlaw-Johnson, S. Gallivan, T. Treasure, and S.A.M. Nashef Computer tools to assist the monitoring of outcomes in surgery Eur. J. Cardiothorac. Surg., November 1, 2004; 26(5): 1032 - 1036. [Abstract] [Full Text] [PDF]

				N. Kang, T. Cole, V. Tsang, M. Elliott, and M. de Leval Risk stratification in paediatric open-heart surgery Eur. J. Cardiothorac. Surg., July 1, 2004; 26(1): 3 - 11. [Abstract] [Full Text] [PDF]

				J. P. Jacobs, C. Mavroudis, M. L. Jacobs, F. G. Lacour-Gayet, C. I. Tchervenkov, J. W. Gaynor, D. R. Clarke, T. L. Spray, B. Maruszewski, G. Stellin, et al. Lessons learned from the data analysis of the second harvest (1998-2001) of the Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database Eur. J. Cardiothorac. Surg., July 1, 2004; 26(1): 18 - 37. [Abstract] [Full Text] [PDF]

				R R Chaturvedi, D Macrae, K L Brown, M Schindler, E C Smith, K B Davis, G Cohen, V Tsang, M Elliott, M de Leval, et al. Cardiac ECMO for biventricular hearts after paediatric open heart surgery Heart, May 1, 2004; 90(5): 545 - 551. [Abstract] [Full Text] [PDF]

				J. L Gibbs, J. L Monro, D. Cunningham, and A. Rickards Survival after surgery or therapeutic catheterisation for congenital heart disease in children in the United Kingdom: analysis of the central cardiac audit database for 2000-1 BMJ, March 13, 2004; 328(7440): 611. [Abstract] [Full Text] [PDF]

				D. A. Bettex, D. Schmidlin, M.-A. Bernath, R. Pretre, M. Hurni, R. Jenni, P.-G. Chassot, and E. R. Schmid Intraoperative Transesophageal Echocardiography in Pediatric Congenital Cardiac Surgery: A Two-Center Observational Study Anesth. Analg., November 1, 2003; 97(5): 1275 - 1282. [Abstract] [Full Text] [PDF]

				W. Daenen, F. Lacour-Gayet, T. Aberg, J.V. Comas, S.H. Daebritz, R. Di Donato, J.R.L. Hamilton, H. Lindberg, B. Maruszewski, and J. Monro Optimal Structure of a Congenital Heart Surgery Department in Europe: by EACTS Congenital Heart Disease Committee1 Eur. J. Cardiothorac. Surg., September 1, 2003; 24(3): 343 - 351. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Gallivan, S. Articles by Stark, J.F. Search for Related Content PubMed PubMed Citation Articles by Gallivan, S. Articles by Stark, J.F.