How to Ask the Right Question: Choosing Estimands for Health Policy Research

Abstract

Evaluating the effects of health policies is hard. Practical constraints like heterogeneous policy implementation details, slow rollouts, and small sample sizes add to the complexity of estimating causal effects. Adding to these difficulties is a lack of clearly articulated considerations for choosing estimands in health policy evaluation research, which can lead to studies in which the choice of scientific question is guided by available analytic tools, rather than the other way around.

In this talk, I discuss common estimands in health policy evaluation, their strengths and limitations, and how they have become prominent in non-experimental research. I then highlight more advanced causal questions that better reflect the complex nature of health policy evaluation and require estimands beyond, say, the average treatment effect among the treated. This talk serves as a prelude to the rest of the session, in which complex health policy questions are addressed with novel estimands and estimators to better understand policies’ nuanced effects and to move toward a healthier, more equitable future.