"Estimating Treatment Effects with Observational Data: A New Approach
to Using Hospital-level Variation in Treatment Intensity"

Mark McClellan and Douglas Staiger

ABSTRACT

Estimating the effect of medical treatment on patient outcomes is one
of the central problems in medical research.  Traditionally,
randomized controlled trials have been the only definitive method for
establishing such treatment effects. However, in response to a variety
of perceived problems with randomized controlled trials, ranging from
their expense to their external validity, there has recently been an
increased interest in estimating treatment effects using observational
data (e.g. patient chart or claims data). The central problem with
observational data, however, is that treatment is endogenous, i.e. is
not randomly assigned across patients and is likely to be related to
unmeasured patient characteristics which also influence outcomes.  

Two basic approaches have been taken to this endogeneity problem (see
McClellan and Noguchi, 1998). One approach has been to collect
detailed patient information (e.g. from charts) and control directly
for risk factors other than treatment which may influence patient
outcomes.  This approach is inevitably limited because patient
outcomes, such as mortality, may directly affect whether a patient
receives treatment independent of patient risk factors.  A second
approach to the endogeneity problem has been to use instrumental
variables (IV) estimation.  In IV estimation one uses factors
(instruments) that influence treatment decisions, but are unrelated to
patient risk factors, to identify how treatment variation that is
unrelated to risk factors is associated with patient outcomes.  This
approach is limited by the availability of suitable instruments,
particularly in cases in which many treatments must be considered.  

In this paper we propose a method of estimating treatment effects
which combines these two prior approaches.  We estimate patient-level
outcome equations, controlling for detailed patient risk factors,
using hospital-level variation in treatment intensity as an instrument
for the treatment variables.  After controlling for the detailed risk
factors, we argue that variation in treatment across hospitals is
unlikely to be related to unobserved risk factors affecting patient
outcomes (unlike the variation in treatment within hospitals).  In
contrast to previous IV approaches, our method provides suitable
instruments for any treatment of interest, so long as there is
variation across hospitals (for any reason) in treatment intensity.
Recent work on weak instruments suggest that standard IV estimation
methods will yield biased estimates of treatment effects in our
approach (Staiger and Stock, 1996).  Therefore, we develop an
alternative IV estimator, closely related to the methodology developed
in McClellan and Staiger (1998a,b), that does not suffer from this
bias.  

We apply our method to estimating the effects of hospital treatment on
survival following a heart attack.  We use detailed medical chart
review data and linked Medicare administrative records from the
Cooperative Cardiovascular Project on over 180,000 elderly heart
attack patients from 1994-1995.  Treatments considered include whether
the patient receives cardiac catheterization, angioplasty, bypass
surgery, as well as the use of drugs such as aspirin, thrombolytics,
or beta blockers.