Access the full text.
Sign up today, get DeepDyve free for 14 days.
S. Julious (2000)
Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design by L. Frison and S.J. Pocock, Statistics in Medicine 1992; 12: 1685-1704.Statistics in medicine, 19 22
G. Imbens, D. Rubin (2015)
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Brantly Callaway, Pedro Sant’Anna (2018)
Difference-in-Differences with Multiple Time PeriodsCJRN: Criminology Research Methodology (Topic)
Howard, S., Bloom (2006)
MINIMUM DETECTABLE EFFECTS A Simple Way to Report the Statistical Power of Experimental Designs
Peter Schochet (2013)
Estimators for Clustered Education RCTs Using the Neyman Model for Causal InferenceJournal of Educational and Behavioral Statistics, 38
R. Zimmer, G. Henry, Adam Kho (2017)
The Effects of School Turnaround in Tennessee’s Achievement School District and Innovation ZonesEducational Evaluation and Policy Analysis, 39
K. Borusyak, Xavier Jaravel (2017)
Revisiting Event Study DesignsKauffman: Entrepreneurship Scholars Initiatives (Topic)
Fang Zhang, A. Wagner, D. Ross-Degnan (2011)
Simulation-based power calculation for designing interrupted time series analyses of health policy interventions.Journal of clinical epidemiology, 64 11
J. Ferron, Gianna Rendina-Gobioff (2005)
Interrupted Time Series Design
J. Heckman, Hidehiko Ichimura, Petra Todd (1997)
Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training ProgrammeThe Review of Economic Studies, 64
M. Lipsey, Kelly Puzio, Cathy Yun, Michael Hebert, Katarzyna Steinka-Fry, M. Cole, M. Roberts, K. Anthony, Matthew Busick (2012)
Translating the Statistical Representation of the Effects of Education Interventions Into More Readily Interpretable Forms
Stephen Donald, K. Lang (2007)
Inference with Difference-in-Differences and Other Panel DataThe Review of Economics and Statistics, 89
L. Hedges, E. Hedberg (2007)
Intraclass Correlation Values for Planning Group-Randomized Trials in EducationEducational Evaluation and Policy Analysis, 29
K. Imai (1998)
Survey SamplingCurrent Sociology, 46
Peter Schochet (2015)
Statistical Theory for the RCT-YES Software: Design-Based Causal Inference for RCTsMathematica Policy Research Reports
S. Hawley, M. Ali, K. Berencsi, A. Judge, A. Judge, D. Prieto-Alhambra (2019)
Sample size and power considerations for ordinary least squares interrupted time series analysis: a simulation studyClinical Epidemiology, 11
(2021)
SCHOCHET, PhD, is a senior fellow at Mathematica with interests in causal inference methodology and conducting field experiments in the education and labor areas
(2012)
Econometric analysis, 7th Edition
Peter Schochet (2007)
Is regression adjustment supported by the Neyman model for causal inferenceJournal of Statistical Planning and Inference, 140
C. Chaisemartin, Xavier d'Haultfoeuille (2018)
Two-Way Fixed Effects Estimators with Heterogeneous Treatment EffectsAmerican Economic Review
Travis St.Clair, K. Hallberg, T. Cook (2016)
The Validity and Precision of the Comparative Interrupted Time-Series DesignJournal of Educational and Behavioral Statistics, 41
C. Cameron, Douglas Miller (2015)
A Practitioner’s Guide to Cluster-Robust InferenceThe Journal of Human Resources, 50
J. Angrist, Jörn-Steffen Pischke (2008)
Mostly Harmless Econometrics: An Empiricist's Companion
William Davis (2001)
Design and Analysis of Cluster Randomization Trials in Health ResearchJournal of the American Statistical Association, 96
H. Bloom (1999)
Estimating Program Impacts on Student Achievement Using “ Short ” Interrupted Time Series
C. Hill, H. Bloom, A. Black, M. Lipsey (2008)
Empirical Benchmarks for Interpreting Effect Sizes in ResearchChild Development Perspectives, 2
P. Holland (1985)
Statistics and Causal InferenceJournal of the American Statistical Association, 81
M. Lipsey, David Wilson (1993)
The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis.The American psychologist, 48 12
Alberto Abadie (2005)
Semiparametric Difference-in-Differences EstimatorsThe Review of Economic Studies, 72
Orley Ashenfelter, David Card (1984)
Using the Longitudinal Structure of Earnings to Estimate the Effect of Training ProgramsNBER Working Paper Series
L. Hedges (2007)
Effect Sizes in Cluster-Randomized DesignsJournal of Educational and Behavioral Statistics, 32
(2005)
Causal inference using potential outcomes: Design, modeling, decisions
Marie-Andrée Somers, Pei Zhu, Robin Jacob, H. Bloom (2013)
The Validity and Precision of the Comparative Interrupted Time Series Design and the Difference-in-Difference Design in Educational Evaluation
Ariel Linden (2015)
Conducting Interrupted Time-series Analysis for Single- and Multiple-group ComparisonsThe Stata Journal, 15
Jacob Cohen (1969)
Statistical Power Analysis for the Behavioral SciencesThe SAGE Encyclopedia of Research Design
David McKenzie (2011)
Beyond Baseline and Follow-Up: The Case for More T in ExperimentsPhilosophy & Methodology of Economics eJournal
J. Bernal, S. Cummins, A. Gasparrini (2016)
Interrupted time series regression for the evaluation of public health interventions: a tutorialInternational Journal of Epidemiology, 46
E. Kontopantelis (2018)
ITSPOWER: Stata module for simulation based power calculations for linear interrupted time series (ITS) designsStatistical Software Components
B. Baltagi, P. Wu (1999)
UNEQUALLY SPACED PANEL DATA REGRESSIONS WITH AR(1) DISTURBANCESEconometric Theory, 15
Fiona Burlig, L. Preonas, M. Woerman (2017)
Panel Data and Experimental DesignDevelopment Economics: Microeconomic Issues in Developing Economies eJournal
Peter Schochet (2005)
Statistical Power for Random Assignment Evaluations of Education ProgramsJournal of Educational and Behavioral Statistics, 33
D. Rubin (1974)
Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of Educational Psychology, 66
Marianne Bertrand, E. Duflo, S. Mullainathan (2001)
How Much Should We Trust Differences-in-Differences Estimates?Experimental & Empirical Studies eJournal
W. Shadish, Thomas Cook, Donald Campbell (2001)
Experimental and Quasi-Experimental Designs for Generalized Causal Inference
(1992)
Hierarchical linear models for social and behavioral research. Applications and data analysis Methods
S. Abraham, Liyang Sun (2018)
Estimating Dynamic Treatment Effects in Event Studies With Heterogeneous Treatment EffectsPolitical Economy: Government Expenditures & Related Policies eJournal
Christopher Redding, Tuan Nguyen (2020)
The Relationship Between School Turnaround and Student Outcomes: A Meta-AnalysisEducational Evaluation and Policy Analysis, 42
H. Bloom (2003)
Using “Short” Interrupted Time-Series Analysis To Measure The Impacts Of Whole-School ReformsEvaluation Review, 27
J. Daw, Laura Hatfield (2017)
Matching and Regression to the Mean in Difference‐in‐Differences AnalysisHealth Services Research, 53
(2015)
Regression based quasi - experimental approach when randomization is not an option : Interrupted time series analysis
L. Frison, S. Pocock (1992)
Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design.Statistics in medicine, 11 13
Katherine Baicker, Theodore Svoronos (2019)
Testing the Validity of the Single Interrupted Time Series DesignERN: Hypothesis Testing (Topic)
P. Shrout (1980)
Quasi-experimentation: Design and analysis issues for field settings: by Thomas D. Cook and Donald T. Campbell. Chicago: Rand McNally, 1979Evaluation and Program Planning, 3
S. Athey, G. Imbens (2018)
Design-Based Analysis in Difference-in-Differences Settings with Staggered AdoptionNBER Working Paper Series
K. Liang, S. Zeger (1986)
Longitudinal data analysis using generalized linear modelsBiometrika, 73
Alberto Abadie, S. Athey, G. Imbens, J. Wooldridge (2017)
When Should You Adjust Standard Errors for Clustering?PSN: Econometrics
O. Ashenfelter (1976)
Estimating the Effect of Training Programs on EarningsThe Review of Economics and Statistics, 60
Wei Liu, Shangyuan Ye, B. Barton, Melissa Fischer, Colleen Lawrence, E. Rahn, M. Danila, K. Saag, P. Harris, S. Lemon, J. Allison, Bo Zhang (2019)
Simulation-based power and sample size calculation for designing interrupted time series analyses of count outcomes in evaluation of health policy interventionsContemporary Clinical Trials Communications, 17
S. Raudenbush (1997)
Statistical analysis and optimal design for cluster randomized trialsPsychological Methods, 2
D. Murray (1998)
Design and Analysis of Group- Randomized Trials
Peter Schochet, Nicole Pashley, Luke Miratrix, Tim Kautz (2020)
Design-Based Ratio Estimators and Central Limit Theorems for Clustered, Blocked RCTsJournal of the American Statistical Association, 117
D. Freedman (2008)
On regression adjustments to experimental dataAdv. Appl. Math., 40
Peter Schochet (2020)
Analyzing Grouped Administrative Data for RCTs Using Design-Based MethodsJournal of Educational and Behavioral Statistics, 45
Peter Schochet (2009)
Statistical Power for Regression Discontinuity Designs in Education EvaluationsJournal of Educational and Behavioral Statistics, 34
Andrew Goodman-Bacon (2018)
Difference-in-Differences with Variation in Treatment TimingEconometrics: Multiple Equation Models eJournal
This article develops new closed-form variance expressions for power analyses for commonly used difference-in-differences (DID) and comparative interrupted time series (CITS) panel data estimators. The main contribution is to incorporate variation in treatment timing into the analysis. The power formulas also account for other key design features that arise in practice: autocorrelated errors, unequal measurement intervals, and clustering due to the unit of treatment assignment. We consider power formulas for both cross-sectional and longitudinal models and allow for covariates. An illustrative power analysis provides guidance on appropriate sample sizes. The key finding is that accounting for treatment timing increases required sample sizes. Further, DID estimators have considerably more power than standard CITS and ITS estimators. An available Shiny R dashboard performs the sample size calculations for the considered estimators.
Journal of Educational and Behavioral Statistics – SAGE
Published: Aug 1, 2022
Keywords: difference-in-differences designs; interrupted time series designs; statistical power; treatment effect estimation; variation in treatment timing
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.