A Cautionary Note on Shrinkage Estimates of School and Teacher Effects

Abstract

Increasing use of “shrinkage” estimates of school and teacher effects on student achievement in educational accountability programs has been associated with arguments that multilevel models are more appropriate for the hierarchical structure of the school situation. Such estimates are usually presented as statistically optimal in that they minimize the mean square error of the estimates, a desirable property achieved by intentional introduction of a bias into the effect estimate. There is little evidence that those designing accountability programs are aware of the possible problematic nature of the differential bias associated with shrinkage estimates. In particular, intuitive rankings of school or teacher effects that are based on observed achievement means can under some circumstances be dramatically changed when shrinkage estimates are used. It is argued that all stakeholders in educational accountability programs should be aware of and agree to this feature of a system based on shrinkage estimates.