Page 41 - Hall et al (2015) Principles of Critical Care-McGraw-Hill

10 PART 1: An Overview of the Approach to and Organization of Critical Care

indicators. The classic medical example is screening a population for An SCC relies on serial measurements of the process or outcome of
10
elevated blood pressure and offering treatment to those with hyperten- interest in the population or a random subset of patients. In ICUs, these
sion. Regardless of the efficacy of this treatment, the next set of blood measurements may take any of the indicator forms: proportions, rates,
pressures will be lower. The same phenomenon occurs in quality. This continuous measures, or indicators. The type of data is important as
can clearly be a problem when selecting outcomes to improve or even it defines what type of distribution will be used to construct the SCC.
selecting hospitals with a quality problem. Since the labeled outliers may Different types of data require different types of control charts, which
not be real outliers, their ratings will improve in the next measurement use specific formulas for the graphs. The reader is referred elsewhere for
regardless of the presence of a quality issue or the efficacy of the quality a more in-depth discussion. 14,15
improvement project. Many before-after quality improvement projects After understanding what types of data are in use, each data point is
suffer from this potential error. One of the solutions to this problem is plotted in a graph, organized by time on the x-axis and the results on the
the use of serial measurements of quality indicators. Therefore, trends y-axis. Three lines are then constructed: a center line (CL), which usu-
over time demonstrating consistently poor quality prior to an interven- ally uses the arithmetic mean of the process, but can also use the median
tion and sustained improvement after are the best insurance against or an expected value. Then two lines are traced, the upper control line
regression to the mean. (UC) and lower control line (LC), using three standard deviations (SD)
■ CONFOUNDING above and below the CL.
14
When a measurement is observed outside the UC or LC lines, the
Just as epidemiologists are aware of confounding variables when draw- process has undergone a special-cause, or nonrandom, variation. Other,
ing conclusions about causality, quality scientists must be aware of these more complex, rules exist, such as drawing control lines at two SD and
identifying two out of three points outside the lines as special variation.
variables. Confounding measures are those that are associated with the
ICU and the quality measure but do not necessarily cause the problem. Trends are also important, and a sequence of seven points moving in
the same direction (either increasing or decreasing) also points toward
For example, if it is known that patients post-cardiovascular surgery
are less prone to develop a VAP compared to patients intubated for special-cause variation. To conclude that a process is under control,
stability of at least 25 data points is required.
shock, the comparison between units could be confounded if the patient
demographics in the ICUs are very different. Obviously, this is less of a
problem when following a single unit over time; however, major varia-
tions in the case mix of an ICU over time could cause this phenomenon. MEASURING TO IMPROVE
excludes certain subsets of patients where the quality measure is known ■ PUBLIC REPORTING OF QUALITY METRICS
There are standard approaches to address confounding. Restriction
to be more or less common. Adjustment mathematically balances con- There is a growing interest in using quality measurements to identify
founding factors across sites. The most common approach would be to high- and low-quality performers at a systems level, which would prompt
use a severity of illness measure to adjust the risk of death in analyzing actions to help low performers improve. Examples of such initiatives include
17
16
mortality differences between ICUs. the UK star system, Canada’s HSMR system, and the New York State
■ SECULAR TRENDS Department of Health reporting of adjusted mortality after coronary artery
bypass graft surgery. In fact, public reporting of hospital performance has
18
Quality indicators may improve over time for reasons apart from spe- been proposed as a means of improving quality of care while ensuring both
transparency and accountability. A recently published systematic review
19
cific efforts to change practice. These changes, usually called secular of 45 articles examined the evidence that public reporting actually improves
trends, are not necessarily problematic when the aim is to demonstrate quality. Eleven studies suggested that public reporting increased quality
that quality is improving over time, but may be misleading when the improvement activities in hospitals, with 20% to 50% of hospitals imple-
data are used to attribute the changes to a specific intervention. An menting changes in response to the reports. The relationship between public
excellent example of this problem can be seen from the original descrip- reporting and improved outcomes is less clear. New York State has imple-
tion of the central line bundle to decrease CLIs. The published report mented a public reporting system on cardiac surgery since 1991. Although
11
20
demonstrated a significant decrease in CLI rates, from 2.7 to 0 per 1000 several reports point toward decreased mortality after the introduction of
catheter-days. The reported rates are likely correct, but at the same the system, concurrent data from other states that did not introduce public
21
time CLIs were decreasing without the implementation of the bundle. reporting demonstrated that the decrease in mortality occurred at similar
12
Therefore, what can be concluded is that there is a real decrease in CLI rate, which questions the real effect of the statewide reporting system. 22
rates over time, but the use of the bundle may or may not be the cause, as Public reporting clearly creates the incentive to improve perfor-
rates may have been declining due to secular trends. To solve this prob- mance, but does not necessarily direct providers on how to improve.
lem when trying to infer causality, different models of analysis, beyond Expectations would be that improved metrics would be preceded by
the scope of this chapter, should be used, such as an interrupted time efforts to implement evidence-based practices. However, metrics can
series or controlled interrupted time series. 13 also be improved by avoiding high-risk patients or by manipulating the
■ STATISTICAL CONTROL CHARTS FOR PERFORMANCE MONITORING way the indicator is measured. In fact, many of the perceived improve-
23
ments in cardiac surgery outcomes from public reporting in New York
Some of the statistical problems discussed can be addressed with a State were due to these changes. Higher-risk patients in New York were
24
simple monitoring tool, the statistical control chart (SCC). Chance, also less likely to receive percutaneous coronary intervention (PCI) than
regression to the mean, and secular trends are addressed by SCCs. were those in Michigan, which did not have PCI public reporting. This
25
This approach has its origin in industry and was initially developed in migration of patients to other states not only biases the reports, but has
1924 by Walter Shewhart at Bell Laboratories, but is widely applicable the negative consequences of overwhelming neighboring health systems
in health care, under multiple formats, depending on the type of data and ignoring patient preferences for care.
available. Briefly, SCCs use statistical methods to distinguish random Other unintended consequences include the widespread adoption of
14
variability from special-cause variation from real changes introduced default therapies to patients who may not need them to enhance quality
into the system. For example, although the rates of self-extubations in measures. For example, observational studies suggest an absolute reduc-
ICUs are relatively constant, there may be variations in the exact num- tion of 1% in mortality when antibiotics are administered early (within
ber during any given month. SCCs are designed to distinguish random 4 hours of hospital arrival) for patients with community-acquired
variation, which is not interesting to clinicians from special-cause varia- pneumonia (CAP). Notwithstanding the small benefit of the proposed
26
tion due to changes in, for example, a sedation protocol. process of care, this association was the basis for the recommendation

Section01.indd 10 1/22/2015 9:36:43 AM