| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Eur J Cardiothorac Surg 2005;28:240-243
© 2005 Elsevier Science NL
Original articles |
Providence Health System, Portland, OR, USA
Received 24 March 2005; received in revised form 5 April 2005; accepted 6 April 2005.
* Corresponding author. Address: Providence St Vincent Hospital and Medical Center, 9205 SW Barnes Road, LL33, Portland, OR 97225, USA. Tel.: +1 503 216 7276; fax: +1 503 216 7274. (Email: ruyun.jin{at}providence.org).
Abstract
Objective: Logistic regression is most often used to produce a cardiac operative risk model. But the logistic equation requires a computer to solve. Thus, simple additive models have been derived from logistic models by adding the odds ratios or modified coefficients. However, this simplification has no statistical justification, and the additive scores do not equal the original logistic probabilities. Methods: The EuroSCORE risk model is a very successful and widely used cardiac surgery risk model and it comes in both an additive and a full logistic version. We applied the EuroSCORE model to the 28,337 cardiac surgeries in the Providence Health System Cardiovascular Study Group database. The discrimination of the models was assessed by the c index. The comparison of the mortality predictions of the logistic and the additive model are mostly descriptive and graphical. Results: Theoretical considerations would predict that the additive model greatly underestimates the risk for the higher risk patients, and clinical data confirm this fact. For the 23,463 (83%) cases with complete data, the predicted mortality was 8.3% by the logistic model and 5.4% by the additive model. The discrimination (c index) of the additive (0.794) and logistic (0.791) models was equally good. A modified additive score is proposed (the mean of the logistic predicted mortality for each original additive score) which could be provided as a look-up table along with the scoring sheet. Conclusions: The additive EuroSCORE gives excellent discrimination, as good as the logistic risk model, but it greatly underestimates the risk of high-risk patients, compared to the logistic. The logistic equation should be used to predicate the mortality when possible. If this is not feasible, a modified additive score could be employed at the bedside. But the logistic should always be used for comparison of providers and for research publications.
Key Words: Additive model • Logistic model • Cardiac surgery
1. Introduction
Cardiothoracic surgery has lead the way in the reporting of operative results using risk models. Logistic regression is most often used to produce an operative risk model. It yields an equation that provides each patient with an individualized prediction of his or her operative risk. But the logistic equation requires a computer to solve. So, to create a simple, convenient ‘back of the envelope’ or bedside risk predictor, some investigators have converted the full logistic model to a simpler, additive model by approximating the odds ratios (OR) or modified coefficients from the logistic equation with integers, which can then be added together without a computing aid.
Deriving an additive model from a logistic equation by adding the odds ratios has no statistical justification since the OR's are intrinsically multiplicative, not additive. Surprisingly, there have been few published objections to this somewhat common policy, except for letters to editors [1–3]. A simple model may have merit, but it should not sacrifice accuracy; the output is too important. We will (1) summarize the theoretical objections to additive models, (2) use a large cardiac surgery data set to confirm these objections, and (3) suggest a possible solution.
2. Materials and methods
The EuroSCORE risk model, a successful multi-national system with website dissemination and deployment, is reported the most widely evaluated cardiac surgery risk model [4]. The EuroSCORE comes in both an additive and a full logistic version (Table 1 ). The additive model was published in 1999 [5], based on a logistic equation which was intended to be used directly [6]. The full logistic regression model was finally published in 2003 [7,8]. We examined the difference between logistic and additive models using the EuroSCORE as an example since it is so widely utilized, but it is the methodology not the model itself that is under scrutiny.
|
3. Results
3.1. Theoretical objections to additive scores
ORs multiply the odds of death, not add to its probability. Adding ORs can theoretically produce a probability that exceeds 100%. For additive model, they are rounded off, usually to an integer, which discards precision. There is no natural way to incorporate a protective factor (one with an OR of less than 1), unless by subtraction; but this could create a probability less than zero. Nor is there a way to use a continuous variable (e.g. age, ejection fraction), except by collapsing it into groups.
The Appendix contains an example that shows the difference between the true logistic probability and the sum of its odds ratios. In general, with a small number of risk factors, the additive score can be lower than the logistic probability, and as risk factors are added, the logistic probability will increasingly exceed the additive risk score (Table 2 ).
|
The discrimination of the additive and logistic models as assessed by receiver operating characteristic curve analysis was equally good for both models: the c index was 0.794 for additive and 0.791 for logistic, respectively. But the predicted mortality by the two models is quite different. Fig. 1 shows that, as expected from the theoretical considerations (Appendix, Table 2), the additive score slightly overestimates the logistic probability below about 6%, and increasingly underestimates it above 6%. The risk for a hypothetical very high-risk patient produced by the additive and logistic would be 28 and 93%, respectively [8]. Our patients in fact almost reached these two values (Fig. 1). And similar values were obtained for our virtual patient in the Appendix (Table 2).
|
|
|
Recently, two clinical papers have compared the additive and logistic EuroSORE models [11,12]. Both papers showed that the additive version underestimated risk, especially in higher risk patients. Another systematic review found that the additive EuroSCORE overestimated risk at lower scores and underestimated risk at higher scores [4]. The EuroSCORE developers admit this too, but argue that the simplicity of the additive version offsets this weakness of the multiplicative [8].
The first cardiac surgery risk model was an additive model [13]. The univariate OR of death for each risk factor was computed and tested, and the predicted mortality was the sum of the ORs for the 15 significant risk factors. This model was updated in 1996 as an additive model, which still considered the risk factors independently [14]. Finally, an easy-to-use model, based on a logistic regression using all 47 available predictors, was created for bedside use by adding the modified values of the coefficients to get a score. This score is then converted into a predicted probability using a nomogram [15].
Other recent examples of additive models include the PAMI model for myocardial infarction [16] and the Northern New England model for heart valve surgery [17]. These models, too, used a nomogram or a look-up table to transform an additive score, to recapture the original probability. This cannot work perfectly, because the same additive scores can give risk to different probabilities, and two different additive score can equal the same probability (Fig. 1). But it is an improvement over using the additive scores directly. We strongly recommend that logistic risk model should be used whenever possible; we reluctantly included the modified additive model for situations, where this is impossible.
The observation interval for both versions of the EuroSCORE and for all other cardiac surgery risk models is in-hospital or 30 days after operation; however, a longer time period (e.g. 60–180 days) has been recommended to capture all of the risk of early death [18].
5. Conclusions
The additive EuroSCORE gives excellent discrimination, as good as the logistic risk model, but it greatly underestimates the risk of high-risk patients, compared to the logistic. The logistic equation should be used to predicate the mortality when possible. It could even be performed at bedside, using a PDA (personal digital assistant). If this is not available, a modified additive score, such as suggested above (Table 3), could be employed at the bedside. But the full logistic version should always be used for comparison of providers, and in research publications.
Logistic regression models the logarithm of the odds of death as a linear combination (LC) of risk factors. If there are n risk factors X, then
where B
0 is the intercept.
To extract logistic probability itself from this linear combination:
First, exponentiate LC to obtain the odds of death:
Note that this turns the addition of the individual coefficients B into multiplication of their exponentials exp(B), which are the odds ratios (ORs) of their respective risk factors.
Finally, convert the odds into the required probability:
Example
To demonstrate the difference between sums of ORs and the logistic probability, take as a simple example a 50-year-old patient with no risk factors. His LC is just the intercept term (B
0=–4.8, Table 1). The additive score for this patient is 0 and his logistic probability is 0.8% (Table 2). Now, suppose we start adding risk factors to this patient. For simplicity, let us use only those with an additive score of 2 (there are 8 in the EuroSCORE, but we used 12). Let us assume no rounding was necessary and the OR was also exactly 2 for each of these factors, to remove one source of error. Table 2 shows the number of factors and the resulting additive EuroSCORE, along with the odds ratio and the resulting probability. The 13 diamonds in Fig. 1 represent a plot of the additive vs. logistic probabilities from Table 2. The additive score at first underestimates the logistic probability and then, as the number of risk factors increase, greatly underestimates it.
Acknowledgments
The following hospitals provided cardiac surgery patient data: Alaska-Providence Anchorage Medical Center; Washington-Providence Seattle Medical Center, Providence Everett Medical Center, Providence St Peter Hospital (Olympia), Providence Yakima Medical Center; Oregon-Providence Portland Medical Center, Providence St Vincent Medical Center (Portland); California-Providence St Joseph Medical Center (Burbank), Providence Holy Cross Medical Center (Mission Hills), Providence Little Company of Mary Hospital (Torrance).
References
This article has been cited by other articles:
|
|
 |
|
  F Bhatti, A D Grayson, G Grotte, B M Fabri, J Au, M Jones, B Bridgewater, and on behalf of the North West Quality Improvement Pr The logistic EuroSCORE in cardiac surgery: how well does it predict operative risk? Heart, December 1, 2006; 92(12): 1817 - 1820. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. Jin and G. L. Grunkemeier Does the logistic EuroSCORE offer an advantage over the additive model? Interactive CardioVascular and Thoracic Surgery, February 1, 2006; 5(1): 15 - 17. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
