EJCTS Click here for details of sales representative
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Personal Folders
Right arrow Download to citation manager
Right arrow Permission Requests
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Albert, A.A.
Right arrow Articles by Ennker, J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Albert, A.A.
Right arrow Articles by Ennker, J.
Related Collections
Right arrow Education
Right arrow Professional affairs
Right arrow Cardiac - other
Right arrow Minimally invasive surgery

Eur J Cardiothorac Surg 2004;25:312-319
© 2004 Elsevier Science NL

On-line variable live-adjusted displays with internal and external risk-adjusted mortalities. A valuable method for benchmarking and early detection of unfavourable trends in cardiac surgery

A.A. Alberta*, J.A. Walterb, B. Arnrichb, W. Hassaneina, U.P. Rosendahla, S. Bauera, J. Ennkera

a Clinic for Cardiothoracic Surgery, Heart Institute Lahr/Baden, Lahr 77933, Germany
b Institute of Neuroinformatics, University of Bielefeld, Bielefeld, Germany

Received 19 October 2003; received in revised form 10 December 2003; accepted 15 December 2003.

* Corresponding author. Tel.: +49-7821-925-367; fax: +49-7821-925-253
e-mail: alexander.albert{at}heart-lahr.com


   Abstract
Top
 Abstract
1. Introduction
 2. Method
 3. Results
 4. Discussion
 5. Summary
 Appendix A. Conference...
 References
 
Objective: Benchmarking and early detection of unfavourable trends. Methods: We implemented a dedicated project-orientated data warehouse, which continuously supplies data for on-line computing of the variable live-adjusted displays (VLADs). To calculate the expected cumulative mortality, we used the multi-variate logistic regression model of the EuroSCORE model. In addition to the external EuroSCORE standard, we calculated a centre-specific risk score for internal standards by analysing the data of 9135 patients, which enables both internal and external comparisons. The VLADs are embedded into the multi-purpose web-based information portal, so that the physicians can investigate several types of VLADs interactively: performance of different types of surgery and individual surgeons for different time intervals. We investigated clinically important events such as modification of operative techniques and personnel changes of the team by the VLADs. Results: We found transient declines in the performance curves during major changes in patient management, indicating that systemic—rather than accidental or patient related factors—were involved in the mortality risk. The internal standard line represents these clusters more clearly than the external line. We evaluated examples of how periods of increased risk could be monitored by the VLAD curves: (1) the introduction of OPCAB surgery; (2) training of surgeons; (3) staff changes and staff-related management.

CONCLUSIONS: On-line VLADs based on a day-to-day updated database, displaying both internal and external standards, are a helpful visualisation tool for earlier detection of unfavourable trends. They enable the surgeon teams and clinical management to take countermeasures at an early stage.

Key Words: Variable live-adjusted displays • Risk adjustment • EuroSCORE • Cumulative sum method • Data warehouse


   1. Introduction
Top
 Abstract
 1. Introduction
2. Method
 3. Results
 4. Discussion
 5. Summary
 Appendix A. Conference...
 References
 
Clinicians are encouraged to improve their methods of investigation and analysis of outcomes, which still tend to be underdeveloped in comparison to methods available in industry [1]. In cardiac surgery, outcome data are usually presented as 30 day or in-patient mortality. Nowadays, it is generally accepted procedure to adjust mortality rates for the estimated perioperative risk [2,3]. The ability of the risk adjustment models on an individual basis is limited. Predictive models cannot consider all patient characteristics that may have an influence on surgical mortality. Generally, they do not consider non-patient-related factors which are known to influence clinical practice and contribute to the outcome such as knowledge and experience of individual staff members, work environment, organisation and management factors [1]. In order to identify these contributory factors, it appears very helpful to consider the temporal occurrence of clinical incidents. A promising method of continuous monitoring of surgical results was introduced to cardiac surgeons in 1994 [4]. It was previously developed in industry for monitoring quality on a production line. For this cumulative sum method (CUSUM) cases were plotted sequentially on the horizontal axis and the plot moves up one unit for each death to represent the cumulative sum of deaths. A series of surgical failures related to neonatal arterial switch operations was detected by characteristic movements of the resulting curve. The method was refined by incorporating success and failure as well as previously estimated risk using the Parsonnet Score for each case [5,6] and further by adding prediction intervals to indicate increasing departures from expected mortalities [7]. There is one observational study that used variable live-adjusted display (VLAD) successfully to compare the performance of two individual hospitals in treating patients with myocardial infarction [8]. Beyond this pioneering work, there is only little experience with VLADs in medicine.


   2. Method
Top
 Abstract
 1. Introduction
 2. Method
3. Results
 4. Discussion
 5. Summary
 Appendix A. Conference...
 References
 
Building a web-based information portal required several steps: first, we implemented a dedicated project orientated data warehouse (data mart), followed by the calculation of VLAD charts according to published techniques [5,6] and subsequently developed a technique to provide multi-purpose VLAD charts on-line.

2.1. Data warehouse/data mart
All heart operations performed at the Heart Institute Lahr/Baden from January 1996 on were analysed continuously. Data for quality assessment were collected prospectively on all patients as part of a national quality assessment trial of cardiothoracic surgery (Quality Assurance Data Review Analysis, Quadra) and cardio-anaesthesia using standardised protocols of the German Society of Thoracic and Cardiovascular Surgery and Intensive Care Medicine [9,10]. A technical assistant for data collection and medical documentation controlled the data collection and tested the inter-rater reliability. Preoperatively, the data is fed into three independently operating commercial hospital information systems (Medwork, Datapec, Clinicom). From there, our data warehouse extracts, transforms, and consolidates the data in order to build up a large, comprehensive database, which appears to the investigator as a single data source. Mortality within 30 days was meticulously recorded with a completeness of more than 99% using a specific procedure based on questionnaires and phone calls to the patients after discharge.

2.2. Calculation of VLADs
The 18 EuroSCORE risk variables could be continuously determined from our data warehouse. In order to determine the estimated percentage of death, the items and additive weights were derived from the simple additive EuroSCORE model, which uses only dichotomous or ordinal data [11]. The second, centre-specific risk score was calibrated to the locally observed mortality by rescaling the data of 9135 patients from 1st January 1996 to 1st April 2002, such that the sum of the expected mortality (EM) of all individuals equals the total number of observed mortalities (OM). For all patients the EM was in the range of 0.002–0.1 [12]. VLADs were created by continuous display of operative results. In the case of a successful operation, the curve goes up by the value of EM for this patient; if the patient dies, the curve decreases by the value of 1-EM. If the patient was at a high risk of perioperative death, the surgeon's mortality figures are not unduly penalised but mortality figures are penalised when a low-risk patient dies. The results for every single patient are cumulated and altogether display the performance. The expected cumulative mortality minus the actual cumulative mortality is called ‘net live saved’.

2.3. Web-based information portal
In our institution, VLADs are part of a multi-purpose intranet information portal, made available to all physicians via intranet. It provides various statistics on operative results and allows the export of various anonymous patient data for all authorised physicians. VLADs can be selected for several operating types, time intervals and/or individual surgeons. The access to VLADs displaying individual surgeon's performance is limited to the surgeon himself and specially authorised persons. Our VLAD plot simultaneously displays curves, which are based on predictive mortalities derived from the EuroSCORE publications and curves after recalibration. VLADs as well as all statistics presented at the web-based information portal are derived directly from the data warehouse and are updated automatically.

We established weekly meetings of our physicians for a joint analysis of adverse outcomes. VLADs were correlated to clinical events and factors possibly influencing clinical practice and contributing to adverse events. They are interpreted regularly by the staff. From this study, we present examples that should be of general interest.


   3. Results
Top
 Abstract
 1. Introduction
 2. Method
 3. Results
4. Discussion
 5. Summary
 Appendix A. Conference...
 References
 
Overall, 30-day mortality for 14 487 cases was 2.2%. The area under the receiver-operating characteristic (ROC) curve before and after recalibration was 0.75 and 0.78, respectively. In nearly all VLADs, the external standard line elevated continuously, since OM were lower than the EM of EuroSCORE and, on average, the internal curve moves along the zero line. A transient decline in a curve, with an accumulation of adverse events, is called a cluster. These clusters were easier to detect in the internal line because, in cases of transient declines, the external line shows only a small directional change in the ascending slope. We found clusters during major changes in patient management. During the training period of the surgeons, clusters occurred at the onset of training and after an uneventful period of 300–600 operations a second cluster surprisingly occurs (Figs. 1 and 2) . The learning curve cluster after starting with cardiac surgery revealed OM between 2.9 and 2.2% (Surgeon A, 5/170 operations; Surgeon B, 5/224). The second learning curve cluster shows mortalities between 4.7 and 3.3% (Surgeon A, 8/170; Surgeon B, 4/120) and a decrease in net lives saved (internal score) by 4.1 (Surgeon A) and 1.1 (Surgeon B). The learning curve after introduction of OPCAB surgery is characterised by a cluster with an observed mortality of 3.9% (5/129) and a decrease in net lives saved of 2.86 (Fig. 3) . The overall performance of the cardiosurgical clinic showed the most impressive clusters during the year 2002 (Fig. 4) , independent of the performance of individual surgeons (Figs. 1, 5 and 6) . During 1416 operations, 54 mortalities occurred (3.8%) and net lives saved (internal score) decreased by 24.2. Net lives saved of the two surgeons A and B decreased from 1.2 to -8.2 and 3.4 to -3.2, OM was 6.7% (21/315, Surgeon C) and 4.6% (13/285, Surgeon D) during this time period. Also surgeon A showed a marked decrease of the performance during the year 2002. Net lives saved decreased from 9.2 to 5.3 and OM was 4.2% (8/190). Fig. 7 shows how the performance increased from January 2001, beginning with the decrease in risk-adjusted in-hospital mortality (increase of the VLAD curve). In-hospital mortality was zero and net lives saved increased by 6.3.



View larger version (18K):
[in this window]
[in a new window]
  		Fig. 1. During the training of a Surgeon A, two periods with an accumulation of adverse events occurred: the first after starting with training, the second cluster 4 years later (about 600 cases). Additionally, two clusters were observed during the year 2002. See also Figs. 4–7.

 


View larger version (18K):
[in this window]
[in a new window]
  		Fig. 4. The overall performance of the cardiosurgical clinic is displayed. Clusters were found in the year 2002. Improvement concerning the staff on the ICU, and adaptation to the changes in organisation and management resulted in an increase in performance.

 


View larger version (17K):
[in this window]
[in a new window]
  		Fig. 2. During the training of a Surgeon B, two periods with an accumulation of adverse events occurred: one during the first year and a second cluster after 4 years (about 600 cases).

 


View larger version (18K):
[in this window]
[in a new window]
  		Fig. 3. A learning curve in OPCAB is displayed: there is an accumulation of adverse events during the first 150 OPCAB operations.

 


View larger version (18K):
[in this window]
[in a new window]
  		Fig. 5. Experienced Surgeon C showed a marked decrease in performance during the year 2002, which is accompanied by the decline in the overall performance of the cardiosurgical clinic. See Figs. 1, 4, 6 and 7.

 


View larger version (17K):
[in this window]
[in a new window]
  		Fig. 6. Experienced Surgeon D showed a similar curve to that of Surgeon C (Fig. 5) with a marked decrease in performance during the year 2002, which is accompanied by the decline in the overall performance of the cardiosurgical clinic. See also Figs. 1, 4 and 7.

 


View larger version (20K):
[in this window]
[in a new window]
  		Fig. 7. Regaining of overall performance started in January 2003 with the decrease of in-hospital mortality (increase in net lives saved based on internal score and in-hospital mortality).

 

   4. Discussion
Top
 Abstract
 1. Introduction
 2. Method
 3. Results
 4. Discussion
5. Summary
 Appendix A. Conference...
 References
 
These VLAD charts revealed increased frequencies of fatalities, which correlate with specific circumstances and alterations in patient care: onset of surgeons' training, delegation of more responsibilities to trainees, learning of new surgical techniques and changes in staff and general and clinical management. These performance patterns are invisible when analysed by simple grouping in specific time periods.

For example, the annually performed risk-adjusted statistics could not demonstrate any difference in 30-day mortality between OPCAB versus on-pump. However, motivated by the results of the VLAD curves, we performed retrospective studies of the individual patients, which revealed an increased rate of perioperative myocardial infarction due to bypass graft thrombosis and incomplete revascularisation. This finding supports the impression given by the VLADs, which attributes the cluster of adverse events after starting with OPCAB not to the risk of the patients but to the learning ability of the surgeons. At that time, various surgeons performed OPCAB surgery primarily in high-risk patients (>80 years, severely decreased pulmonary function, preoperative stroke, etc.), so that complications were interpreted individually rather than systemically as an effect of a learning curve in OPCAB surgery. Later, when OPCAB surgery was limited to three surgeons who specialised in this, performance increased. Probably a continuous monitoring of performance by VLADs at that time would have detected the learning curve earlier. Another observation concerning a learning curve in OPCAB was made using non-risk-adjusted CUSUM analysis [13].

The training of cardiac surgeons showed similar patterns of performance. When considering annual statistics we could not identify any difference in outcome between trainees, registrars, and consultants. This is in line with studies analysing the effect of surgical training on outcome in cardiac surgery [14–16], which could not prove any detrimental effect on patient outcome in operations performed by trainees. However, the observed clusters of fatalities at the beginning of the training and later after 200–600 operations were identified in several surgeons. This corresponds with the original report studying VLADs in individual surgeons, which found clusters at the beginning of training in two surgeons, and a later cluster after 250–350 cases in one surgeon [5]. One report applying CUSUM retrospectively to display the 10-year performance of one surgeon also identified a cluster during the first year of training [17]. Technical error, inexperience and poor judgement are readily reflected in operative mortality in cardiac surgery, conditions which are probably involved in the emergence of these first clusters.

The decline in performance after 300–600 operations we observed in our trainees most likely reflects a drop in supervision, greater optimism and overestimation of the trainee's abilities as far as difficult cases were concerned. Our observation led to an increased supervision of trainees as well as of apparently more experienced colleagues.

The third example reveals a decrease in overall performance of the cardiosurgical department in the year 2002. VLADs suggest that the increase in mortality is not only a consequence of the higher risk of the patients but of other contributory factors. In the year 2002, our facility, especially the ICU-team, experienced significant and relevant changes in staff and staff-related management. Measures were taken to restructure the ICU staff, and to improve organisational and management changes, which resulted immediately in regaining good overall performance.

What is strongly recommended for all situations when information is drawn from clinical databases [18] is the participation of the physicians in the process of data acquisition and analysis. This applies in particular to the VLADs. In order to avoid a misuse of incomplete or inaccurate data we built a data warehouse, which allows the inspection and verification of the whole data integration process. Additionally, case-based verification tools were developed and plausibility checks are performed automatically. The power of the VLADs depends on the model of EM and therefore, on the choice of the underlying risk score. In the first instance, this concerns the type of risk model. Studies applying external risk scores to centre-specific populations reported a wide range of areas under the ROC curve (0.67 [19] and 0.831 [20]).

Secondly, this concerns the local adaptation or recalibration of risk scores. The human visual system can detect subtle absolute slope trends around the horizontal line much better than higher slope trends, e.g. -1°/+1° versus 20°/22° elevation angle. Therefore, we recalibrated the EuroSCORE expected mortalities to fit the observed centre-specific mortalities. This allows for easier visual recognition of the periods of performance changes.

Despite these refinements, we still observe inaccuracies of some VLADs, such as an overall decline in performance in valve operations in comparison to bypass operations. Thus, for an additional improvement in VLADs it would appear advantageous to fit the individual weightings of the EuroSCORE to the population specific for our facility or to alter the EuroSCORE parameter mix. For example, we could recently show how renal function can be better assessed by replacing the variable creatinine by the creatinine clearance, which significantly increased the area under the ROC curve [12].

Optimal adjustment for the clinic specific population requires the development of a risk model beyond the EuroSCORE and another time window of the risk score outcome than 30-day mortality. It was demonstrated that the hazard of dying after CABG induced by the preprocedural condition and the procedure itself continues to exist for 2 months beyond the date of the procedure [21]. It was confirmed that the risk incurred by the highest risk patients is likely to be underestimated by 30-day mortality [22]. The suggestion was made to postpone the endpoint of 30-day mortality to 1 year post-operative for a case mix [23]. In this context, it must be pointed out that precise comparisons between therapies and appropriate medical decision making requires follow-up intervals as long as possible as well as more sophisticated data exploration and analysis [24]. The more uniform the patient population is, the more feasible is the application of more complex, time-related analysis for continuous monitoring of surgical results. Furthermore, it has to be considered that higher the costs and the more special parameters included in the scoring model the less suitable it is for interclinic benchmarking [23]. VLADs regain the biggest benefit in a large case mix. The generation of VLADs with both external and internal standards enables us to apply the more complex techniques of risk adjustment to the internal standard and to maintain a simple score like the EuroSCORE as a ‘common language’.

In order to specify how much variation in the plot is expected under good surgical performance, and hence a deviation from the expected, should be a cause for concern, prediction intervals were incorporated into the VLADs [7,25]. The normal distribution of mortality rates within a sizeable sequence of operations was used to construct nested prediction intervals [7]. The prediction intervals were colour coded and superimposed onto the VLAD chart to indicate increasing departures from EM. Ref. [25] designed CUSUM graphs which ‘signals’ if sufficient evidence has accumulated to indicate that the surgical failure rate has substantially changed. The run length until the CUSUM signifies the deterioration in performance and the discriminative ability of the test depends on the choice of the control limits, which can be varied according to the medical context and the heterogeneity of the case mix. Other investigators do without prediction intervals [6,8]. They stress that the interpretation of VLADs requires the consideration of other qualitative trends and obvious changes in patient management is more important. Furthermore, a failure to act on the qualitative trend observed until a significant P value has been achieved could result in the avoidable loss of lives.


   5. Summary
Top
 Abstract
 1. Introduction
 2. Method
 3. Results
 4. Discussion
 5. Summary
Appendix A. Conference...
 References
 
VLADs enable us to monitor different aspects of surgical performance and to discover non-patient-related factors contributing to surgical risk. In order to monitor different aspects of patient management, VLADs should be accessible for all team members and provide the selection based on several aspects, e.g. operating procedures, individual surgeons, or time periods. To solve the choice problem of generalised score for interclinic benchmarking versus a specialised intraclinic score, we recommend supporting multiple VLAD chart types in a readily accessible intranet information system.

On-line VLADs based on a day-to-day updated database are a helpful visualisation tool for earlier detection of unfavourable trends, enabling the surgeon teams and clinical management to take countermeasures at an early stage.


   Footnotes
 
Presented at the Joint 17th Annual Meeting of the European Association for Cardio-thoracic Surgery and the 11th Annual Meeting of the European Society of Thoracic Surgeons, Vienna, Austria, October 12–15, 2003.


   Appendix A. Conference discussion
Top
 Abstract
 1. Introduction
 2. Method
 3. Results
 4. Discussion
 5. Summary
 Appendix A. Conference...
References
 
Dr G. Laufer (Innsbruck, Austria): What was the mean EuroSCORE in your institution, because it is well known that if the EuroSCORE is rather low that the risk of the patient is overestimated with a classic EuroSCORE, which you could avoid using the logistic EuroSCORE?

Dr Albert: We used the simple additive EuroSCORE without continuous data, which we considered sufficient as external standard. But we performed recalibration of the EuroSCORE for a more accurate internal standard by fitting the expected mortalities of the EuroSCORE to our observed mortalities. Therefore the internal standard demonstrates better our results. So I suggest to go even further steps beyond the EuroSCORE to develop a clinical score.

Dr Laufer: But what was your mean EuroSCORE, for example, for the year 2002?

Dr Albert: I don't have the data now.

Dr Laufer: Because that reflects a little bit the risk profile of your patients.

Dr Albert: Yes, but it is calculated together with all VLADs and included automatically within them. We did not present these numbers, from which expected mortalities of VLADs are calculated, because we preferred here to show graphics rather than tables.

Dr B. Buxton (Heidelberg, Vic., Australia): We also use the VLAD system for monitoring early changes. I wonder whether you have looked at adapting this to other outcome measures such as wound infection or stroke, these creep up on you sometimes more subtly than you think, and I wonder whether that would also be a good outcome to measure using this technique?

Dr Albert: Excuse me?

Dr Buxton: Have you adapted this system to the assessment of stroke or perhaps, more importantly, wound infections?

Dr Albert: Yes, we looked for some complications, trying to find a correlation with the rise of mortality. The results are not yet clear so I didn't like to present them here. Often they go parallel to the mortality data sometimes they arise earlier. We used as evidence of complications the length of stay in an intensive care unit more than 7 days, we looked for the stroke rate, bleeding and for postoperative CK-MB greater than 50.

Dr G. Rizzoli (Padova, Italy): To me this is just another quality control chart.

Dr Albert: Yes, it is.

Dr Rizzoli: I really don't understand how you can attribute to a single surgeon results that depend from the entire process of patient's care. We know that patients die not only because of the surgeon's performance in the operative room, but also because of the standard of care of the environment. So, did you do a multivariate analysis to isolate if the single surgeon was an incremental risk factor or not?

Dr Albert: As you know, the surgeon has a great impact on the outcome of the patients, and his responsibility goes further beyond the operating room. I showed in the last example that it is not always the surgeon but there were a lot of contributory factors, it is the team and the staff on the ICU which contributes to the outcome, and this becomes clear in our last examples presented here.

Dr T. Carrel (Berne, Switzerland): Can you tell us a little bit what kind of consequences you draw from your examination and how fast are you able to react when you see a cluster coming in, if you have an on-line analysis?

Dr Albert: It is not a method for reglementation or punishment. As I told you every surgeon can view his own VLADs, the overall performance of the clinic and the performance concerning different types of surgery. Mostly the interpretation of the VLADs is really easy and it is easy to get a consensus because they correlate with known changes in patient care. Every surgeon or member of the team has to think about what he can change. I showed you in the last example, we employed more qualified staff on the ICU and we discussed other problems.

Dr Laufer: Do you know if you are allowed or the chief of the institution in Germany is allowed to analyse the data for an individual surgeon in this manner or does he need the permission from the individual surgeon? It is the case in Austria according to the data safety law or ‘daten schutzgesetz’ as we call it in Austria.

Dr Albert: Yes, from the beginning we discussed this issue and all surgeons agreed that they could view their own VLADs, not the VLADs of other colleagues, but the chief can view all. But I told you, it is not a system for punishment.


   References
Top
 Abstract
 1. Introduction
 2. Method
 3. Results
 4. Discussion
 5. Summary
 Appendix A. Conference...
 References
 

  1. Vincent C. Understanding and responding to adverse events. N Engl J Med 2003;348(11):1051-1056.[Free Full Text]
  2. Nashef S.A., Carey F., Charman S. The relationship between predicted and actual cardiac surgical mortality: impact of risk grouping and individual surgeons. Eur J Cardiothorac Surg 2001;19:817-820.[Abstract/Free Full Text]
  3. Sergeant P., de Worm E., Meyns B. Single centre, single domain validation of the EuroSCORE on a consecutive sample of primary and repeat CABG. Eur J Cardiothorac Surg 2001;20:1176-1182.[Abstract/Free Full Text]
  4. De Leval M.R., Francois K., Bull C., Brawn W., Spiegelhalter D. Analysis of a cluster of surgical failures. Application to a series of neonatal arterial switch operations. J Thorac Cardiovasc Surg 1994;107:914-923.[Abstract/Free Full Text]
  5. Lovegrove J., Valencia O., Treasure T., Sherlaw-Johnson C., Gallivan S. Monitoring the results of cardiac surgery by variable life-adjusted display. Lancet 1997;18:1128-1130.
  6. Poloniecki J., Valencia O., Littlejohns P. Cumulative risk adjusted mortality chart for detecting changes in death rate: observational study of heart surgery. Br Med J 1998;316(6):1697-1700.[Abstract/Free Full Text]
  7. Sherlaw-Johnson C., Lovegrove J., Treasure T., Gallivan S. Likely variations in perioperative mortality associated with cardiac surgery: when does high mortality reflect bad practice?. Heart 2000;84:79-82.[Abstract/Free Full Text]
  8. Lawrance R.A., Dorsch M.F., Sapsford R.J., Mackintosh A.F., Greenwood D.C., Jackson B.M., Morrell C., Robinson M.B., Hall A.S. Use of cumulative mortality data in patients with acute myocardial infarction for early detection of variation in clinical practice: observational study. Br Med J 2001;323:324-327.[Abstract/Free Full Text]
  9. Struck E., De Vivie E.R., Hehrlein F., Hugel W., Kalmar P., Sebening F., Wilde E. Multicentric quality assurance in cardiac surgery. Quadra study of the German Society and Cardiovascular Surgery (QUADRA: quality retrospective analysis). Thoracic Cardiovasc Surg 1990;38:123-134.[Medline]
  10. Schirmer U., Dietrich W., Lüth J.U., Baulig W. Der erweiterte Datensatz Kardioanästhesie. Anästhesiol Intensivmed 2000;41:683-691.
  11. Roques F., Nashef S.A., Michel P., Gauducheau E., de Vincentiis C., Baudet E., Cortina J., David M., Faichney A., Gabrielle F., Gams E., Harjula A., Jones M.T., Pintor P.P., Salamon R., Thulin L. Risk factors and outcome in European cardiac surgery: analysis of the EuroSCORE multinational database of 19030 patients. Eur J Cardiothorac Surg 1999;15:816-822.
  12. Walter J., Mortasawi A., Arnrich B., Albert A., Frerichs I., Rosendahl U., Ennker J. Creatinine clearance versus serum creatinine as a risk factor in cardiac surgery. BMC Surg 2003;17:4.[CrossRef]
  13. Novick R.J., Fox S.A., Stitt L.W., Swinamer S.A., Lehnhardt K.R., Rayman R., Boyd W.D. Cumulative sum failure analysis of a policy change from on-pump to off-pump coronary artery bypass grafting. Ann Thorac Surg 2001;72:S1016-S1021.[Abstract/Free Full Text]
  14. Anderson J.R., Parker D.J., Unsworth-White M.J., Treasure T., Valencia O. Training surgeons and safeguarding patients. Ann R Coll Surg Engl 1996;78:116-118.[Medline]
  15. Jenkins D.P., Valencia O., Smith E.E. Risk stratification for training in cardiac surgery. Thorac Cardiovasc Surg 2001;49:75-77.[CrossRef][Medline]
  16. Goodwin A.T., Birdi I., Ramesh T.P., Taylor G.J., Nashef S.A., Dunning J.J., Large S.R. Effect of surgical training on outcome and hospital costs in coronary surgery. Heart 2001;85:454-457.[Abstract/Free Full Text]
  17. Novick R.J., Stitt L.W. The learning curve of an academic cardiac surgeon: use of the CUSUM method. J Card Surg 1999;14:312-320.[Medline]
  18. Kouchoukos N.T. Quality initiatives and the power of the database: where we stand. Ann Thorac Surg 1995;60:1526-1529.[Abstract/Free Full Text]
  19. Pons J.M., Espinas J.A., Borras J.M., Moreno V., Martin I., Granados A. Cardiac surgical mortality: comparison among different additive risk-scoring models in a multicenter sample. Arch Surg 1998;133:1053-1057.[Abstract/Free Full Text]
  20. Baretti R., Pannek N., Knecht J.P., Krabatsch T., Hubler S., Hetzer R. Risk stratification scores for predicting mortality in coronary artery bypass surgery. Thorac Cardiovasc Surg 2002;50:237-246.[CrossRef][Medline]
  21. Sergeant P., Blackstone E., Meyns B., K.U. Leuven Coronary Surgery Program Validation and interdependence with patient-variables of the influence of procedural variables on early and late survival after CABG. Eur J Cardiothorac Surg 1997;12:1-19.[Abstract]
  22. Osswald B.R., Blackstone E.H., Tochtermann U., Thomas G., Vahl C.F., Hagl S. The meaning of early mortality after CABG. Eur J Cardiothorac Surg 1999;15:401.
  23. Sergeant P., Meyns B. La critique est aisée mais l'art est difficile. Lancet 1997;350:1114-1115.[CrossRef][Medline]
  24. Sergeant P., Blackstone E.H. Closing the loop: optimizing physicians' operational and strategic behavior. Ann Thorac Surg 1999;68:362-366.[Abstract/Free Full Text]
  25. Steiner S.H., Cook R., Farewell V. Risk-adjusted monitoring of binary surgical outcomes. Med Decis Making 2001;21:163-169.[Abstract]



This article has been cited by other articles:


Home page
 Eur. J. Cardiothorac. Surg.Home page
J. M. Bernal, J. Lorca, D. Prieto-Salceda, I. Pulitani, A. Ponton, I. Garcia, and J. M. Revuelta
Performance at 10 years of the CarboMedics 'Top-Hat' valve. Postclamping time is a predictor of mortality
Eur. J. Cardiothorac. Surg., February 1, 2006; 29(2): 144 - 149.
[Abstract] [Full Text] [PDF]

Home page
 Ann. Thorac. Surg.Home page
G. J. Murphy, C. A. Rogers, M. Caputo, and G. D. Angelini
Acquiring Proficiency in Off-Pump Surgery: Traversing the Learning Curve, Reproducibility, and Quality Control
Ann. Thorac. Surg., November 1, 2005; 80(5): 1965 - 1970.
[Abstract] [Full Text] [PDF]

Home page
 BMJHome page
B. Keogh, D. Spiegelhalter, A. Bailey, J. Roxburgh, P. Magee, and C. Hilton
The legacy of Bristol: public disclosure of individual surgeons' results
BMJ, August 21, 2004; 329(7463): 450 - 454.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Personal Folders
Right arrow Download to citation manager
Right arrow Permission Requests
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Albert, A.A.
Right arrow Articles by Ennker, J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Albert, A.A.
Right arrow Articles by Ennker, J.
Related Collections
Right arrow Education
Right arrow Professional affairs
Right arrow Cardiac - other
Right arrow Minimally invasive surgery

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
ANN THORAC SURG ASIAN CARDIOVASC THORAC ANN EUR J CARDIOTHORAC SURG
J THORAC CARDIOVASC SURG ICVTS ALL CTSNet JOURNALS