LFI(2019)-manuscript+supp.pdf (1.16 MB)
Rigorous large-scale educational RCTs are often uninformative: Should we be concerned?
journal contribution
posted on 2019-02-05, 16:42 authored by Hugues Lortie-Forgues, Matthew InglisMatthew InglisThere are a growing number of large-scale educational Randomized Controlled Trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the EEF (UK) and the NCEE (US) which evaluated interventions aimed at improving academic achievement in K-12 (141 RCTs; 1,222,024 students). The mean effect size was 0.06 standard deviations (SDs). These sat within relatively large confidence intervals (mean width 0.30 SDs) which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational RCTs often find small and uninformative effects.
History
School
- Science
Department
- Mathematics Education Centre
Published in
Educational ResearcherVolume
48Issue
3Pages
158 - 166Citation
LORTIE-FORGUES, H. and INGLIS, M., 2019. Rigorous large-scale educational RCTs are often uninformative: Should we be concerned?. Educational Researcher, 48 (3), pp.158-166.Publisher
SAGE PublicationsVersion
- AM (Accepted Manuscript)
Rights holder
© AERAPublisher statement
This paper was published in the journal Educational Researcher and the definitive published version is available at https://doi.org/10.3102/0013189X19832850.Acceptance date
2019-01-22Publication date
2019-03-11Copyright date
2019ISSN
0013-189XeISSN
1935-102XPublisher version
Language
- en