Fairness in Quantitative Measures used in Educational Effectiveness Research Studies

Abstract

Quantitative educational effectiveness research is critical for informing schools, teachers, parents, and students about effective practices for promoting learning. We hypothesize that the educational effectiveness literature base is lacking evidence for fair measurement, which calls into question the validity of our educational effectiveness knowledge for different groups of students. The purpose of this study is to explore the extent to which researchers provide evidence for fairness in the measurement scores used to evaluate effectiveness in educational research publications. We found that the majority (86.94%) of educational effectiveness studies in the What Works Clearinghouse provide no evidence for fair measurement in outcome variables. This means that many groups of students may work with educators in their schools who have selected interventions they believe are supported by evidence of effectiveness, but those interventions may not truly have such evidence for all student groups included in the study. It is the responsibility of researchers to ensure fair measurement in their study outcomes before the public is given information about effectiveness that may be inaccurate for some of the student groups they serve.