The Classification Power: A Metric for Evaluating and Comparing Postsecondary Mathematics Placement Policies

Abstract

We introduce a new metric—the classification power—to examine the effectiveness of postsecondary mathematics placement policies. This metric addresses the methodological challenges of contextualizing the effectiveness of a single placement policy and comparing the effectiveness of multiple placement policies across different student populations. We leverage an information-theoretic approach to construct the classification power as a measure of the improvement of the implemented policy over a hypothetical policy that places students by a Bernoulli coin-flip process. We expect an effective policy to perform significantly better than this hypothetical weighted coin-flip placement. We illustrate the utility of this metric by applying this methodology to institutional data from our four-year institution. We find the classification power provides more information about the quality of placement compared to the previously defined severe error rate metric.