Page 127 - Hall et al (2015) Principles of Critical Care-McGraw-Hill

CHAPTER 13: Assessment of Severity of Illness 93

78
N of cases Hospital death rate die without CPR (cardiopulmonary resuscitation), and many die after
400 100 withholding or withdrawal of care.
A major portion of ICU resources is spent on patients who have mini-
mal chances of survival. However, until a public consensus is reached
79
80 about dealing with these very difficult issues, broad ethical principles
300 of beneficence, nonmalfeasance, and autonomy are likely to be more
important components of end-of-life decisions than quantitative data
provided by scoring systems. Broader social and economic policy issues
60
should be separate concerns.
200
SOURCES OF ERROR AND BIAS IN SCORING SYSTEMS
40
Severity-of-illness scoring systems are not perfect, partially because of
error and bias. Error and bias limit the reproducibility of scoring systems
100
20 outside the original sample of patients, and thus limit the applicability
of scoring systems to different clinical situations. Specifically, bias of
scoring systems can be related to the selection of included variables,
to the collection of data, to the lead time before the onset of the acute
0 + 0
0 10 20 30 40 50 60 70 80 90 100 disease and admission of the patient to the ICU, to the imprecision in
10-Point APACHE III ranges (first day) choosing a principal admission diagnosis, to the inaccuracy of certain
scoring systems for specific disease categories, and finally to the use
N of cases Predicted Observed of scoring systems for purposes they were not meant to accomplish.
FIGURE 13-3. Relationship between first-day APACHE III score and risk of hospital mor- ■
tality for trauma admissions to APACHE III study. With distribution of the sample into specific BIAS RELATED TO THE SELECTION OF VARIABLES
disease categories, the number of high risk of mortality patients used in the validation set is AND TO THE COLLECTION OF DATA
fairly low. In the highest score subset of patients, the mortality for these groups remains much Variables can be included in a severity-of-illness score by a multivariate
lower than 99%. Also, severity-of-illness scoring systems are prone to underestimating the risk analysis that shows that each variable is a statistically independent pre-
of mortality in high-risk patients. (Data from Watts CM, Knaus WA. The case for using objective dictor of mortality. Alternatively, variables can be selected by consensus
scoring systems to predict intensive care unit outcome. Crit Care Clin. January 1994;10(1):73-89.) of experts. Consensus panel selection of variables is subjective, and
variables can be interrelated. The problem with interrelated variables is
15
that two such variables are not independent of each other as predictors
and Preferences for Outcomes and Risks of Treatments) is important of mortality. Noncontinuous variables increase error in the computation
because it was designed to determine whether providing physicians with of risk of mortality. Noncontinuous variables are classified as present or
accurate predictions of death would change physician behavior, patient absent, so a single misclassification results in a large error in outcome
satisfaction, and decisions regarding care. SUPPORT was designed to prediction. 15
estimate survival of seriously ill hospitalized patients who were not Detection bias is another cause of bias of the included variables.
74
necessarily in an ICU. The SUPPORT prognosis model includes nine Detection bias means that variables are only detected if measured.
diagnostic groups and the following 15 prognostic factors: disease However, because scoring systems use variables measured in clinical
group, 11 physiologic variables, age, history of malignancy, and the practice, not all variables will be measured on all patients on all days.
number of days the patient was hospitalized before study entry. In phase Therefore, in several scoring systems, unmeasured (undetected) vari-
I of the study, the investigators noted shortcomings in communication, ables are assigned a normal value. The assumption that unmeasured
variability in frequency of aggressive treatment, and variability in care at physiologic variables are normal can underestimate the risk of mortality.
the time of death (CPR, comfort care, pain management, etc). In phase II APACHE II, APACHE III, and SAPS II contain some variables that are
of the study, physicians in the intervention group received probability not used routinely in daily care, such as albumin and bilirubin levels.
75
estimates of 6-month survival, outcome of cardiopulmonary resuscita- Use of the worst value of a variable in 24 hours also causes errors. Most
tion, and incidence of functional disability at 2 months. Specifically scoring systems use the worst value of a variable in a 24-hour period.
trained nurses made multiple contacts with the patients, families, physi- However, selection of the worst value can be subjective. For example, the
cians, and hospital staff to elicit preferences, improve understanding of GCS contributes a large number of APACHE II points; however, many
outcomes, encourage attention to pain control, and facilitate advance intubated critically ill patients require sedation and narcotics to facilitate
care planning and patient-physician communication. Importantly, the intubation and ventilation. Thus, a patient could deteriorate from a GCS
phase II intervention did not improve care or patient outcomes. Patients of 13 prior to intubation to 3-5 after intubation. Therefore, clinicians
experienced no improvement in patient-physician communication. often record the “native” GCS as the GCS prior to sedation. The GCS is
Also, there was no change in the incidence or timing of written DNR thus more inaccurate during heavy sedation; some use GCS after partial
(do not resuscitate) orders, physicians’ knowledge of their patients’ pref- withdrawal of sedation (eg, daily awakening trials) to compute the daily
erences not to be resuscitated, number of days spent in the ICU before APACHE II. There are other errors associated with collection of data,
death, or use of hospital resources. Thus the SUPPORT study showed including temperature conversion from Fahrenheit to Celsius, creatinine
that providing physicians with objective outcome predictions did not conversion to the international system, use of the GCS on deeply sedated
change physicians’ attitudes and behavior. patients, transcription errors, and errors in analysis of data. Direct
54
Several observations suggest that there is a gap between scoring computer data entry may decrease transcription error.
system predicted outcome and decisions to withhold and withdraw
APACHE II predicted mortality on the day of ICU admission of only ■ BIAS RELATED TO POOR CALIBRATION
ICU care. Patients in whom care was withdrawn in a medical ICU had
77
61% ± 22%. Furthermore, patients with prolonged multiorgan system Statistical regression in scoring systems has a propensity for poor cali-
failure who continue to require life support generally do not have very bration. Regression techniques tend to underpredict the likelihood of
abnormal physiologic parameters and thus have relatively low APS death of more severely ill patients, and tend to overpredict the likelihood
54
scores. Finally, an increasing proportion of critically ill patients in ICUs of death of patients with less severe illness (Fig. 13-3). These errors can

Section01.indd 93 1/22/2015 9:37:28 AM