Page 77 - Hall et al (2015) Principles of Critical Care-McGraw-Hill

CHAPTER 7: Interpreting and Applying Evidence in Critical Care Medicine 45

treatment. In this case, a poor outcome may be erroneously associated
to one’s patients also necessitates assessing the number needed to with the treatment rather than the disease that actually caused it. 5
treat (NNT) to see a benefit to the population. Bias in observational studies, which results from systematic errors in
6
• Evaluating clinical research evidence also requires addressing the the design or conduct of a study, falls into two major categories: selec-
meaning of p values and confidence intervals. These statistical mea- tion bias and information bias. Selection bias results when individuals
sures aid the assessment of whether observed differences in outcomes have differing probabilities of being included in the study sample based
between groups reflect true differences or simply chance variation. on a factor that is relevant to the study design. Information bias results
in systematic misclassification of participants in a study based on a vari-
• To correctly interpret a variety of diagnostic tests, one must under- ety of sources of misinformation including recall bias, interviewer bias,
stand how well that test reflects the actual presence or absence observer bias, and respondent bias. Both confounding and the influence
6
of disease in any given patient. The sensitivity and specificity of of information bias introduced by loss to follow-up are discussed below
https://kat.cr/user/tahir99/
a given test reflect how closely the result of that test reflects the in our examination of randomized controlled trials.
“truth” about a patient’s disease process.
• Qualitative methods can serve a variety of purposes in critical care ■ RANDOMIZED CONTROLLED TRIALS
research and should be reviewed no less critically than quantitative
methods. The randomized controlled trial (RCT) is an important experimental
design used to assess the efficacy of a medical intervention. In RCTs,
subjects are randomly assigned to either the treatment or control group.
The process of randomization minimizes the risk of confounding
because it increases the likelihood that both known and unknown con-
founders will be equally distributed between the two groups.
INTRODUCTION
Assessing Study Validity: Several factors should be carefully considered
Without a rational approach to interpreting and applying research find- by the reader of any RCT before deciding whether the results of the
ings at the bedside, clinicians can be frustrated in their efforts to inte- trial are valid, including randomization, blinding, loss to follow-up,
grate the results of empirical studies into the care of their patients. Here and post-randomization confounding.
we review important elements of clinical research study design, outcome
measures, measures of association, and statistical testing relevant to Randomization Critical evaluation of an RCT should include a comparison
research in intensive care units (ICUs). We also discuss the nature and of the control and treatment groups at baseline to ensure that potential
role of qualitative research in intensive care medicine and summarize confounders have been adequately balanced between the two groups by
strategies to assess the rigor of a qualitative research study. the randomization process. This evaluation is especially important for
small studies in which randomization does not always result in equiva-
lency between groups at baseline.
STUDY DESIGN AND RELATED ISSUES Blinding Blinding (or masking) refers to the process by which study par-
■ OBSERVATIONAL STUDIES ticipants or investigators are prevented from knowing to which study

Clinical research studies generally fall into one of two categories: group subjects have been assigned. Blinding of both the investigator and
the research subject (double-blinding) protects against bias that may
observational studies or experimental studies. Observational stud- arise from either one being aware of the group to which the research
ies may include case series, case-control studies, prospective cohort participant was randomized. Blinding of the investigator assessing out-
studies, and cross-sectional studies. Each type of observational study comes is especially important if the outcome being measured is subjec-
has different strengths and weaknesses, but all involve observing the tive, as with a self-reported measure of post-ICU quality of life.
results of a subject’s exposure to a factor of interest that was intro-
duced independent of a research protocol. The goal of the observa- Loss to Follow-Up It is also necessary to carefully assess the adequacy
tion is to evaluate associations between exposures and one or more of follow-up when evaluating the validity of study findings. Loss to
outcomes of interest to investigators. Although observational studies follow-up can occur in either differential or nondifferential fashion.
can help identify associations between exposures and outcomes, they Non-differential loss to follow-up involves loss of subjects who are not
generally cannot be used to establish a causal link between the predic- different in important respects from those for whom follow-up data are
tor and outcome of interest. 1 obtained. Non-differential losses usually result in a loss of power since
There are numerous well-known examples in which the results of there will be fewer participants than planned at the final analysis. Such
an observational study suggested a causal link that did not withstand underpowered RCTs are problematic because they often produce falsely
the scrutiny of further scientific testing. One example is the effect negative findings, resulting in missed opportunities to identify benefi-
of hormone replacement therapy on coronary heart disease. Early cial therapies. Differential loss to follow-up presents a more challenging
2
observational studies suggested that hormone replacement therapy problem. In this case, those who are not followed through to the end of
was significantly protective against coronary heart disease, but the study are in some way systematically different from those who are
randomized trials later showed that hormone replacement therapy observed throughout entire the study period. Differential losses result in
either had no impact on coronary heart disease or increased the risk both loss of power and potential bias in the findings due to uncontrolled
of disease. A variety of reasons for these differences have been confounders. It has been argued that readers can do a rudimentary
3,4
suggested, all relating to potentially unidentified confounders in the assessment of the potential impact of loss to follow-up by assuming that
observational study. all losses from the treatment group had poor outcomes and all losses
When assessing an observational study, one must be aware that from the control group had positive outcomes. Recalculating the overall
such studies are subject to a variety of types of confounding and bias. outcome using this assumption provides an estimate of the impact of
7
Confounding, in which a factor is associated with both a predictor or those losses.
risk factor and the outcome being studied, can have the effect of appear- Post-Randomization Confounding Confounding may enter in after the
ing either to strengthen or weaken the association between the predictor randomization process. A recent study of extracorporeal membrane
and the outcome. One very common type of confounding in observa- oxygenation (ECMO) for management of acute respiratory failure by
tional studies is confounding by indication. This type of confounding Peek et al randomized subjects with acute respiratory failure to either
8
occurs because those who receive treatment in an observational study routine critical care management or referral to an ECMO center. That
are more likely to have worse disease than those who do not receive study documented better outcomes in the patients randomized to referral

Section01.indd 45 1/22/2015 9:36:57 AM