SREE-IES Webinar Series

Applying Standards for Excellence in Education Research

 The Institute for Education Sciences (IES) has launched the Standards for Excellence in Education Research (SEER) to make education research more transparent, actionable, and focused on consequential outcomes. To support this effort, IES is producing a series of practical guides for researchers on how to implement SEER to improve the quality and relevance of their studies. 

SREE is excited to partner with IES to sponsor two upcoming webinars that each cover the recommendations from a specific guide aligned with the SEER. These webinars are free, open to the public, and relevant to all researchers who seek to ensure their studies are useful to policymakers and educators. This webinar series is also co-sponsored by APPAM. 

Webinar 1: Enhancing the Generalizability of Impact Studies in Education
May 16, 2022

Webinar Recording

Generalizability in education research indicates how well the results of a study apply to broader populations of interest to educators and policymakers. However, in studies that evaluate the impacts of educational interventions, the generalizability of study findings is often unclear because each study is conducted with only a specific sample. Educators and policymakers may question whether such findings can help them make decisions about how to best support the populations they serve.

This webinar lays out recommendations that researchers can take to enhance generalizability when planning impact studies in education. It walks researchers through the key steps involved in identifying a target population of schools, developing a list of schools in this population, and selecting a representative sample. It also provides an overview of steps to recruit schools into the study, assess and adjust for differences between the sample and population, and report generalizability appropriately. The webinar is based on IES’s recently released guide on this topic.

Dr. Elizabeth Tipton is an Associate Professor of Statistics, a Faculty Fellow in the Institute for Policy Research, and the co-Director of the Statistics for Evidence Based Policy and Practice (STEPP) Center at Northwestern University. Her research focuses on the development of methods for improving generalizations from experiments, both through the design and analysis of field experiments and through the use of meta-analysis.
Dr. Rob Olsen is a Senior Fellow at the George Washington Institute of Public Policy at George Washington University. His research focuses on the generalizability of randomized trials in education. Olsen is the Principal Investigator of a study testing different sampling methods for selecting sites for impact studies, and he is the co-Principal Investigator for a study testing different regression methods for predicting the impacts of educational interventions in individual schools using data from multi-site randomized trials. Finally, he consults for national research organizations on how to design and implement impact studies for greater generalizability.

 

Webinar 2: The BASIE (BAyeSian Interpretation of Estimates) Framework for Interpreting Findings from Impact Evaluations
June 13, 2022

Webinar Recording

In studies that evaluate the impacts of educational interventions, educators and policymakers typically want to know whether an intervention improved outcomes. However, researchers cannot provide a simple yes/no answer because all impact estimates have statistical uncertainty. Researchers have often used statistical significance and p-values to assess this uncertainty, but these statistics are often misinterpreted and cannot help educators and policymakers figure out how likely it was that the intervention had an effect.

This webinar lays out a Bayesian framework for interpreting impact estimates without the pitfalls of relying only on p-values. This framework allows researchers to calculate the probability an intervention had a meaningful effect, given the impact estimate and prior evidence on similar interventions. The webinar will explain key concepts and introduce a convenient, easy-to-use Excel tool for applying this framework. With these concepts and tools, researchers can extract more accurate and interpretable lessons from impact findings to support informed decisions about educational interventions. The webinar is based on IES’s recently released guide on this topic.

Dr. John Deke is an economist and senior fellow at Mathematica. His work has focused primarily on the statistical methodologies used in impact evaluations and systematic reviews, especially in K-12 education.

Dr. Mariel Finucane is a principal statistician at Mathematica. Her work uses Bayesian hierarchical modeling and tree-based methods to study social policies.