Holistic Admissions in Higher Education: Challenges and Promises

Definition

Holistic review has gained increasing popularity in the last few decades. Research shows that 95% of admissions officers claim that they adopt holistic review in their admissions decisions (Bastedo, 2021). However, there is no standard definition of what holistic admissions entail and the practice varies widely across institutions. In a nutshell, holistic review refers to the evaluation of higher education applications, at the undergraduate or graduate level, that relies on a spectrum of criteria that reflect applicants’ attitude, knowledge, skills, and competencies. The essential element is the consideration of contextual factors that impact an applicant’s qualifications, including the opportunities available to them, their family, their community, and the schools they attended. Typically, holistic reviews shun the use of any cut scores in any of the quantitative metrics such as GPA and test scores; instead they consider the strengths and limitations of applicants across a range of criteria and the contextual factors (Bastedo, 2021). However, practices on the ground vary widely across schools and programs which claim to implement holistic review.

Holistic admissions practices prevalent today typically consist of a combination of quantitative and qualitative criteria used in conjunction to determine an applicant’s fit for an institution or program. In undergraduate admissions, the most commonly used evaluation criteria include high school GPA, standardized test scores, personal statements, reference letters, and resumes, CVs, or similar documents detailing additional achievements (e.g., awards) and activities (e.g., volunteer work, leadership roles). The Common App has sections specifically dedicated to activities and awards which apply to any school using the Common App and has largely replaced resumes and CVs. Specific fields, such as fine arts, may also require direct demonstrations (e.g., portfolios, auditions) of relevant skills. At the graduate level, especially for doctoral program admissions, an additional emphasis is often placed on research experience. For international students who speak English as a second language, English language proficiency is another important criterion. Most institutions require a minimum language proficiency as indicated by test performance on standardized language assessments such as TOEFL and IELTS. Many institutions also offer language preparation programs after enrollment for international students in need.

Within the realm of holistic admissions, there are a number of distinct practices at the undergraduate and graduate levels. Admissions are typically conducted in a centralized manner for undergraduate admissions across schools and colleges in an institution, and the admissions policy tends to adopt the same criteria across domains except for specialized fields such as fine arts degrees; however, admissions are much more decentralized at the graduate level, where programs, departments, schools, and colleges often independently decide on their own admissions requirements. Required qualifications can vary across departments and programs and by degree levels (e.g., master’s, doctorate). For example, as of March 8, 2022, the programs in the Yale Graduate School of Arts and Sciences have varying standardized test requirements. For the Applied Mathematics Program, GRE results are not required, but a GRE Subject Test is encouraged (Yale University, 2022a). For the Applied Physics Program, GRE scores are required, but not at the time of application submission—only when students matriculate (Yale University, 2022b). For the East Asian Studies M.A. Program, GRE scores are required (Yale University, 2022c). Another distinction is on the involvement of faculty members. Individual faculty members are typically not involved in the admissions decision at the undergraduate level but can be heavily engaged in graduate admissions.

Although 95% of selective colleges in the United States claim that they practice holistic admissions, how exactly each institution implements holistic admissions is nebulous (Bastedo et al., 2018). Very little information can be found on institutional websites detailing how admissions are conducted, other than descriptions of the criteria that are considered. Such ambiguity provides institutions with maximum flexibility in interpreting and defending their admissions practices and also serves to protect them from litigation.

Standardized Tests

Of all the commonly used admissions criteria, standardized tests have received the most scrutiny recently. Standardized tests have long been criticized for correlating with family income. Students from families with income over $100,000 scored an average of 1,091 on the SAT, while students from low-income families (i.e., less than $25,000) scored an average of 968 (Kantrowitz, 2021). Some of the contributing factors that are associated with family income include the quality of schools students attend, teacher qualifications, family support, and peer influence. Wealthy parents also spend considerable money on test preparation. For example, one-on-one tutoring sessions from a leading test preparation organization could cost parents as much as $2,600 for 10 hours of private instruction (Wellemeyer, 2019), and some of the highest hourly rates can be $750 an hour (Kaplan, 2005).

Research has produced mixed findings on the effect of coaching on standardized test performance. Domingue and Briggs (2009) reported that coaching has significantly improved students’ scores on the SAT mathematics section, and that coaching is more effective for students who have taken challenging academic coursework and are from wealthy backgrounds. Dulin (2017) investigated the effect of a SAT preparation course at a low income urban high school and found that 11^th graders in the intervention group had significantly larger score gains than students in the control group. Briggs (2001) conducted analyses based on over 4,000 students and concluded that after controlling for group differences, the average coaching effect on the SAT mathematics section was fairly small, from 14 to 15 points on a 200–800 scale. Although the benefits of coaching or attending coaching schools may not be as large as some of the commercial coaching companies claim, this type of extra instruction is widely perceived as an unfair advantage provided to students from resourceful families. A meta-analysis based on studies of 107 samples and over 130,000 students also revealed a moderate effect of test preparation (Hausknecht et al., 2007).

Standardized tests are also criticized for showing persistent racial and ethnic performance differences. According to 2019 data from the National Center for Education Statistics (NCES), Asian and White SAT test takers had significantly higher scores, 1,223 and 1,114 respectively, than Black and Hispanic test takers, 933 and 978 respectively (U.S. Department of Education, 2021). It is important to note, though, that racial and ethnic performance differences do not just occur on standardized admissions tests. In fact, performance gaps are reflected in K-12 academic assessments among even much younger students. For example, the nation’s report card, the National Assessment of Educational Progress (NAEP), showed a consistent performance difference for fourth-grade math, with Black and Hispanic students performing significantly lower than White and Asian students. Similar patterns exist on other subjects such as reading and sciences and for other grades (U.S. Department of Education, 2017b). Difference in performance can even be found in the kindergarten age range; on average Hispanic and Black students start kindergarten with math scores lower than those of White students by .77 standard deviations, based on nationally representative data sponsored by NCES (Reardon & Galindo, 2009). The trends evident in these data suggest that standardized college admissions assessments do not create performance differences along racial lines—such differences may more accurately be seen as a cumulative reflection of the unequal educational resources accessible to students of varying backgrounds, along with other contributing factors such as motivation, language status, and parental expectation (U.S. Department of Education, 2017b; Wright, 2018). As Harden (2022) argues, “The SAT doesn’t create inequalities in these academic skills. It reveals them.”

Because of the performance differences along racial and ethnic lines discussed above, standardized tests have garnered negative publicity in the United States. As institutions contemplate the role of standardized testing in their holistic admissions policies, many have moved to test-optional or even test-blind policies. A test-optional policy allows applicants to decide if they want to submit a test score or not. If they do submit, the admissions committee will consider the test scores. A test-blind policy excludes the use of standardized test scores from its evaluation. As of September 2019, just prior to the COVID-19 pandemic, there were about 1,050 colleges that were test optional (FairTest, 2019).

The global COVID-19 pandemic upended higher education institutions and accelerated the test-optional movement. Due to health and safety concerns, many testing centers were closed during the pandemic, and there was very limited access for students to take assessments. Many institutions made temporary or even permanent decisions to stop the use of standardized tests, in order to accommodate students under these difficult conditions. Among the institutions that made decisions to abolish the use of standardized tests, one notable example is the University of California (UC) system. In May 2021, the UC system made the decision to no longer consider standardized test scores in admissions and plans on completely phasing out the SAT and ACT by the spring of 2025 (Chavez, 2021). Given the global status of the UC system, the impact of their admissions decision is far-reaching. As of March 2022, over 1,700 four-year institutions have adopted a test-optional admissions policy.

As institutions reconsider the role of standardized tests in college admissions, they will inevitably rely more heavily on other types of criteria, such as GPA, essays, and reference letters, among others. It is important we approach all criteria with a critical view, carefully examine their strengths and weaknesses, and evaluate the intended and unintended consequences of using such criteria for admissions decisions.

GPA

If standardized tests face the criticism of showing racial performance difference, GPA shows the same consistent pattern. From 1990 to 2019, White and Asian students had significantly higher high school GPA than Black and Hispanic students (U.S. Department of Education, 2019). At the college level, racial disparity in GPA is also prevalent. For example, the average college GPA over six years was 2.7, 2.9, 2.5, and 2.0 for White, Asian, Hispanic, and Black students respectively (Chen et al., 2020). While 75% of White bachelor’s degree recipients had an average GPA of 3.0 or higher, only 55% of Black graduates had an average GPA of 3.0 or above. At the lower end of the GPA scale, Black graduates were three times as likely as White graduates to have an average GPA of less than 2.5 (Woo et al., 2012).

In addition to racial disparities in GPA, another factor that limits the usefulness of GPA in college admissions is grade inflation. Studies report significant grade inflation in high schools as a national trend (Gershenson, 2018; Hurwitz & Lee, 2018). For example, from 1990 to 2019, the average GPA increased by .51 and .54 for Asian and White high school students but increased by only .34 and .41 for Hispanic and Black students (U.S. Department of Education, 2019). Research shows that the gap in GPA between more and less affluent schools started as .31 in 2005 and increased to .41 in 2016 (Gershenson, 2018)—and many minority students come from less affluent schools. In other words, grade inflation is more serious in affluent schools than in lower-income schools. As high school GPA continues to be inflated, the differential inflation rates across the racial groups will further exacerbate the racial inequities in college admissions. College GPA also shows evidence of inflation. For example, the national average GPA at four-year colleges and universities in 1983 was 2.8, and increased to 3.2 in 2013 (Rojstaczer, 2016).

During the pandemic, due to campus closures and many disruptions related to technological issues associated with online learning and online assignment, many college instructors turned to using pass or fail grading rather than letter grades (Tamez-Robledo, 2021). Some K-12 districts are also following practices in higher education to switch to pass and fail grading (Sawchuk, 2020). The shifts in grading policies are intended to ease pressure on students, but they will also have implications for admissions, as GPA is an important metric commonly considered in both college and graduate admissions.

In addition to GPA, admissions requirements around coursework may also have unintended impacts on students from underserved groups. For example, completion of “A-G courses” (prerequisite subject requirements) are required by the UC system as minimum requirements for college admissions. However, a Standardized Testing Task Force formed by the UC Academic Council concluded that “disparities in access to and completion of A-G courses account for a disproportionate lack of UC eligibility for students who are members of underrepresented groups (University of California Academic Senate, 2020, p. 105).”

Letters of Recommendation

Reference letters are another commonly used metric in college admissions. Many institutions require reference letters to gain knowledge about applicants’ skills, accomplishments, and character that may not be captured by GPA and test scores. It is believed that reference letters help reveal aspects of the applicant that discrete evaluation metrics cannot. In most cases, the requirement of reference letters is fairly open-ended in that the evaluators have the flexibility to comment on any aspect that they prefer the admissions committee to know about the applicant. For graduate admissions, some programs may require references from former professors or employers when applicable.

A long-term concern with reference letters has been that applicants from wealthy backgrounds are more likely to find referees who are well-known and influential than applicants from less privileged backgrounds. There are also issues with racial and gender bias and cultural differentiation (Dalal et al., 2022; Highhouse, 2008). Dalal and coauthors reported that there are systematic race and gender differences in the content and evaluation of letters of recommendation, based on over 37,000 letters submitted for over 10,000 graduate school applicants. For example, they found that letters for female applicants tend to focus on personality, ability, and motivation, while letters for males tend to emphasize critical thinking. The content differences in racial and gender groups can favor or disfavor certain groups (Madera et al., 2009). In addition to access and equity issues, evaluation of reference letters is also challenged by low reliability, or consistency. Raters are typically admissions officers in charge of admissions evaluation at the undergraduate level or faculty members in charge of graduate admissions. It is unclear what kind of training is provided to ensure fairness and consistency in their ratings. Research shows that interrater reliability across three raters often fails to exceed .40 (Baxter et al., 1981). Such low reliability limits the predictive validity of reference letters (Klieger et al., 2022). Even quality letters of recommendation provide only modest prediction of graduate school outcomes such as GPA and faculty ratings (Kuncel et al., 2014). Furthermore, admissions officers who evaluate the letters typically lack training, and therefore consensus, on the interpretation of the letters and often rely on idiosyncratic approaches to make inferences about applicants’ qualifications (Dalal et al., 2022).

Research shows that the quality of a recommendation letter can profoundly impact selection outcomes (Liu et al., 2009). Letters that are clearly written with great focus and specificity are more likely to benefit applicants than vague and poorly written letters. Longer letters also tend to be judged more favorably than short and concise letters (Aamont et al., 1998). This may particularly affect international applicants whose evaluators may not be familiar with the norms of letters in the United States and may fail to write a letter that is culturally appropriate and compelling. Because of the unequal access to quality letters across applicants, a small portion of institutions exclude letters of recommendation in their admissions. For example, the University of California, Los Angeles (UCLA) is among the institutions that ban the use of reference letters in their undergraduate admissions. University administrators cite as their reason for excluding letters of recommendation that most public school counselors are heavily loaded with student requests, and that low-income students in particular are likely to have teachers who are overworked and thus may not have the bandwidth to write a letter that adequately represents the student (Jaschik, 2022). There are other institutions that are letter optional. For example, University of California, Berkeley allows reference letters but does not require them.

Given the open-endedness in reference letters and idiosyncrasies in interpretation of letters, efforts have been made to standardize reference letters for better clarity and improved equity in admissions (Klieger et al., 2022; Liu et al., 2009). For example, Klieger and coauthors reported the use of a standardized tool for letters of recommendation, called the Personal Potential Index (PPI), in graduate school admissions. Referees can evaluate applicants on a 5-point Likert scale on six dimensions: Knowledge and Creativity, Communication Skills, Teamwork, Resilience, Planning and Organization, and Ethics and Integrity. Klieger’s validity study found that the PPI was able to predict both graduate school GPA and outcomes such as a student being placed on academic probation. The tool also has potential to promote racial, ethnic, and gender diversity, as there were minimal rating differences shown across racial and gender subgroups. Yet even with standardized tool for letters of recommendation, one challenge that remains is the comparability of ratings due to differences in response styles (e.g., acquiescent response style—tendency to always agree; extreme response style—tendency to use the extreme values on a scale; midpoint response style—tendency to use the midpoint on a scale). Lack of level-setting before evaluators fill out the ratings could lead to inconsistency across ratings.

Essays or Personal Statements

Essays are another commonly used evaluation criterion in both college and graduate school admissions. Sometimes they are also called personal statements or admissions essays. At the undergraduate level, there is wide variation in terms of the number of essays required by institutions and the selection of topics. Some of the most popular topics ask students to share their personal stories, describe how they overcame obstacles or challenged a common belief, share a learning and growth experience, or describe someone or something that inspires them. Many students use Common App, a nonprofit college-application platform, which helps streamline applications to multiple institutions that have shared requirements, to submit essays that can meet multiple application needs. Applicants may also send essays directly to institutions when those institutions have unique essay requirements. At the graduate level, especially for doctoral program applicants, essay requirements tend to focus on applicants’ research interests and prior research experience. Given the disciplinary focus of graduate studies, the essay prompt often places a stronger emphasis on demonstration of relevant domain knowledge and skills than on generic aspirations.

A variety of commercial services are available providing coaching and editing to help applicants write compelling admissions essays. Costs for editing a single essay could run up to hundreds or even thousands of dollars (Jaschik, 2017). This type of service provides advantages to students from wealthy backgrounds, which adds an equity concern to the admissions process. Similarly, students from families with college-educated parents often get support from their parents, and it is almost impossible to determine the degree to which their essays are polished by their parents.

As mentioned earlier, standardized tests are criticized for correlating with family income. Recent research shows that essays are not immune from the influence of family income either. Researchers at Stanford University (Alvero et al., 2021) found that essay content is strongly correlated with students’ self-reported family income, and the correlation is even stronger than the association between SAT scores and household income. Essay content refers to the topics, diction, grammar, and punctuation in an essay. In particular, there seems to be significant differences in the topics students choose to write about between high- and low-income students. Based on an analysis of 240,000 admissions essays from 60,000 applications to the University of California, the authors found that applicants from high-income families tend to write about human nature and seeking answers while applicants from low-income families tend to write about time management and family relationship.

The above discussion of commonly used admissions criteria suggests that although standardized tests have received heightened attention and criticism for equity and fairness concerns, other criteria clearly have their own limitations that should not be neglected. Each individual criterion, although imperfect in and of itself, offers complementary information about an applicant’s strengths and weaknesses, and should be considered in conjunction with other admissions criteria to paint a more holistic picture of an applicant.