Embedding Qualitative and Quantitative Methods for Policy Evaluation

Abstract

Policy evaluation is complex, requiring deep knowledge regarding the policies themselves, the contexts in which they are enacted, the degree to which they are implemented, as well as appropriate data and statistical methods for making causal inferences about policy effects on outcomes. This workshop will describe mixed methods approaches for policy evaluation, specifically the role of mixed methods in application of policy trial emulation framework. Target trial emulation is an approach to designing rigorous non-experimental studies by “emulating” key features of a clinical trial. Most commonly used outside policy contexts, this approach is also valuable for policy evaluation as policies typically are not randomly assigned. Using the policy trial emulation framework to conduct and report on research design and methods supports transparent assessment of threats to causal inference in non-experimental studies intended to assess the effect of a health policy on clinical or population health outcomes. Mixed methods play a crucial role in designing policy trial emulations. For example, legal mapping and characterization of policy implementation through qualitative and survey methods can support clear definition of the policy question of interests, the relevant units, the estimand, etc. Participants will learn about the 7 components of policy trial emulation, unique considerations for applying the framework in a mixed-methods policy evaluation context, and analytic strategies for causal inference. The workshop is designed to accommodate all levels of statistical expertise and aims to provide participants with new tools to design rigorous, high-quality policy evaluation studies, and to interpret their results with the appropriate scientific context and information.

Nicholas J. Seewald

Assistant Professor of Biostatistics

Assistant Professor of Biostatistics at the University of Pennsyvlania Perelman School of Medicine