Holistic Admissions in Higher Education: Challenges and Promises
Ou Lydia Liu*
Educational Testing Service
Abstract
As U.S. higher education institutions adapt their admissions policies to advance diversity and inclusion, holistic admissions has taken a center stage in many institutions’ admissions practices. This article provides an overview of the definition of holistic admissions, the differences between holistic admissions and other types of admissions, challenges and promises in implementing holistic review, particularly around the test-optional component, evidence of desired outcomes of holistic admissions, and future research needs.
Keywords: higher education, holistic admissions, fairness and diversity, college admissions, graduate admissions
* Contact: lliu@ets.org
© 2022 Liu. This open access article is distributed under a Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/)
Holistic Admissions in Higher Education: Challenges and Promises
Background of Holistic Review in Higher Education
College education was limited to very few people at the beginning of the 20th century. Around 1910, only 3% of the population obtained a college degree and about 35% of them were female (Caplow et al., 2001). Most of the college students came from wealthy and White families.
College preparatory schools played a significant role in preparing students, exclusively boys at that time, for the challenges of college life. The high tuition and boarding costs of such preparatory schools made it impossible for students from low- and middle-class families to attend. In college admissions, a strong emphasis was put on character and leadership, in addition to academic criteria such as Latin, Greek, and mathematics (Karabel, 2005). With the intention to promote meritocracy and expand access to college to a larger population, the College Board was created in 1900 to help streamline the college admission process and increase access to higher education (College Board, 2022). The first SAT was administered in 1926, marking the introduction of an objective and standardized measure into college admissions. Standardized tests were perceived as a fairer tool for college admissions, compared with subjective evaluation criteria such as the ones focusing on character and leadership. Since that time, standardized tests have served as a key component in the admissions process. Despite the popularity of standardized tests, there has been consistent concern with standardized tests in that they correlate with family income (e.g., Rampell, 2009) and demonstrate consistent racial performance difference (Smith & Reeves, 2020). Because of the increasing criticism of standardized tests, in recent years the role of standardized tests has been deemphasized in admissions, in that test-optional has been gaining popularity in admissions among institutions.
Definition
Holistic review has gained increasing popularity in the last few decades. Research shows that 95% of admissions officers claim that they adopt holistic review in their admissions decisions (Bastedo, 2021). However, there is no standard definition of what holistic admissions entail and the practice varies widely across institutions. In a nutshell, holistic review refers to the evaluation of higher education applications, at the undergraduate or graduate level, that relies on a spectrum of criteria that reflect applicants’ attitude, knowledge, skills, and competencies. The essential element is the consideration of contextual factors that impact an applicant’s qualifications, including the opportunities available to them, their family, their community, and the schools they attended. Typically, holistic reviews shun the use of any cut scores in any of the quantitative metrics such as GPA and test scores; instead they consider the strengths and limitations of applicants across a range of criteria and the contextual factors (Bastedo, 2021). However, practices on the ground vary widely across schools and programs which claim to implement holistic review.
Holistic admissions practices prevalent today typically consist of a combination of quantitative and qualitative criteria used in conjunction to determine an applicant’s fit for an institution or program. In undergraduate admissions, the most commonly used evaluation criteria include high school GPA, standardized test scores, personal statements, reference letters, and resumes, CVs, or similar documents detailing additional achievements (e.g., awards) and activities (e.g., volunteer work, leadership roles). The Common App has sections specifically dedicated to activities and awards which apply to any school using the Common App and has largely replaced resumes and CVs. Specific fields, such as fine arts, may also require direct demonstrations (e.g., portfolios, auditions) of relevant skills. At the graduate level, especially for doctoral program admissions, an additional emphasis is often placed on research experience. For international students who speak English as a second language, English language proficiency is another important criterion. Most institutions require a minimum language proficiency as indicated by test performance on standardized language assessments such as TOEFL and IELTS. Many institutions also offer language preparation programs after enrollment for international students in need.
Within the realm of holistic admissions, there are a number of distinct practices at the undergraduate and graduate levels. Admissions are typically conducted in a centralized manner for undergraduate admissions across schools and colleges in an institution, and the admissions policy tends to adopt the same criteria across domains except for specialized fields such as fine arts degrees; however, admissions are much more decentralized at the graduate level, where programs, departments, schools, and colleges often independently decide on their own admissions requirements. Required qualifications can vary across departments and programs and by degree levels (e.g., master’s, doctorate). For example, as of March 8, 2022, the programs in the Yale Graduate School of Arts and Sciences have varying standardized test requirements. For the Applied Mathematics Program, GRE results are not required, but a GRE Subject Test is encouraged (Yale University, 2022a). For the Applied Physics Program, GRE scores are required, but not at the time of application submission—only when students matriculate (Yale University, 2022b). For the East Asian Studies M.A. Program, GRE scores are required (Yale University, 2022c). Another distinction is on the involvement of faculty members. Individual faculty members are typically not involved in the admissions decision at the undergraduate level but can be heavily engaged in graduate admissions.
Although 95% of selective colleges in the United States claim that they practice holistic admissions, how exactly each institution implements holistic admissions is nebulous (Bastedo et al., 2018). Very little information can be found on institutional websites detailing how admissions are conducted, other than descriptions of the criteria that are considered. Such ambiguity provides institutions with maximum flexibility in interpreting and defending their admissions practices and also serves to protect them from litigation.
Commonly Used Criteria in Holistic Admissions
In this section, I discuss the many metrics commonly used in holistic admissions. They each offer unique information about an applicant’s strengths and weaknesses, but at the same time, none of the criteria are immune to equity and fairness concerns. There has been considerable concern around standardized tests in college and graduate admissions, yet there is growing evidence that many other metrics (e.g., letters of recommendation, essays), perceived as more equitable demonstrations of applicants’ qualifications, are equally, if not more, prone to influence of family resources and income and can differentially impact applicants of different racial and gender groups (e.g., Alvero et al., 2021; Dalal et al., 2022).
Standardized Tests
Of all the commonly used admissions criteria, standardized tests have received the most scrutiny recently. Standardized tests have long been criticized for correlating with family income. Students from families with income over $100,000 scored an average of 1,091 on the SAT, while students from low-income families (i.e., less than $25,000) scored an average of 968 (Kantrowitz, 2021). Some of the contributing factors that are associated with family income include the quality of schools students attend, teacher qualifications, family support, and peer influence. Wealthy parents also spend considerable money on test preparation. For example, one-on-one tutoring sessions from a leading test preparation organization could cost parents as much as $2,600 for 10 hours of private instruction (Wellemeyer, 2019), and some of the highest hourly rates can be $750 an hour (Kaplan, 2005).
Research has produced mixed findings on the effect of coaching on standardized test performance. Domingue and Briggs (2009) reported that coaching has significantly improved students’ scores on the SAT mathematics section, and that coaching is more effective for students who have taken challenging academic coursework and are from wealthy backgrounds. Dulin (2017) investigated the effect of a SAT preparation course at a low income urban high school and found that 11th graders in the intervention group had significantly larger score gains than students in the control group. Briggs (2001) conducted analyses based on over 4,000 students and concluded that after controlling for group differences, the average coaching effect on the SAT mathematics section was fairly small, from 14 to 15 points on a 200–800 scale. Although the benefits of coaching or attending coaching schools may not be as large as some of the commercial coaching companies claim, this type of extra instruction is widely perceived as an unfair advantage provided to students from resourceful families. A meta-analysis based on studies of 107 samples and over 130,000 students also revealed a moderate effect of test preparation (Hausknecht et al., 2007).
Standardized tests are also criticized for showing persistent racial and ethnic performance differences. According to 2019 data from the National Center for Education Statistics (NCES), Asian and White SAT test takers had significantly higher scores, 1,223 and 1,114 respectively, than Black and Hispanic test takers, 933 and 978 respectively (U.S. Department of Education, 2021). It is important to note, though, that racial and ethnic performance differences do not just occur on standardized admissions tests. In fact, performance gaps are reflected in K-12 academic assessments among even much younger students. For example, the nation’s report card, the National Assessment of Educational Progress (NAEP), showed a consistent performance difference for fourth-grade math, with Black and Hispanic students performing significantly lower than White and Asian students. Similar patterns exist on other subjects such as reading and sciences and for other grades (U.S. Department of Education, 2017b). Difference in performance can even be found in the kindergarten age range; on average Hispanic and Black students start kindergarten with math scores lower than those of White students by .77 standard deviations, based on nationally representative data sponsored by NCES (Reardon & Galindo, 2009). The trends evident in these data suggest that standardized college admissions assessments do not create performance differences along racial lines—such differences may more accurately be seen as a cumulative reflection of the unequal educational resources accessible to students of varying backgrounds, along with other contributing factors such as motivation, language status, and parental expectation (U.S. Department of Education, 2017b; Wright, 2018). As Harden (2022) argues, “The SAT doesn’t create inequalities in these academic skills. It reveals them.”
Because of the performance differences along racial and ethnic lines discussed above, standardized tests have garnered negative publicity in the United States. As institutions contemplate the role of standardized testing in their holistic admissions policies, many have moved to test-optional or even test-blind policies. A test-optional policy allows applicants to decide if they want to submit a test score or not. If they do submit, the admissions committee will consider the test scores. A test-blind policy excludes the use of standardized test scores from its evaluation. As of September 2019, just prior to the COVID-19 pandemic, there were about 1,050 colleges that were test optional (FairTest, 2019).
The global COVID-19 pandemic upended higher education institutions and accelerated the test-optional movement. Due to health and safety concerns, many testing centers were closed during the pandemic, and there was very limited access for students to take assessments. Many institutions made temporary or even permanent decisions to stop the use of standardized tests, in order to accommodate students under these difficult conditions. Among the institutions that made decisions to abolish the use of standardized tests, one notable example is the University of California (UC) system. In May 2021, the UC system made the decision to no longer consider standardized test scores in admissions and plans on completely phasing out the SAT and ACT by the spring of 2025 (Chavez, 2021). Given the global status of the UC system, the impact of their admissions decision is far-reaching. As of March 2022, over 1,700 four-year institutions have adopted a test-optional admissions policy.
As institutions reconsider the role of standardized tests in college admissions, they will inevitably rely more heavily on other types of criteria, such as GPA, essays, and reference letters, among others. It is important we approach all criteria with a critical view, carefully examine their strengths and weaknesses, and evaluate the intended and unintended consequences of using such criteria for admissions decisions.
GPA
If standardized tests face the criticism of showing racial performance difference, GPA shows the same consistent pattern. From 1990 to 2019, White and Asian students had significantly higher high school GPA than Black and Hispanic students (U.S. Department of Education, 2019). At the college level, racial disparity in GPA is also prevalent. For example, the average college GPA over six years was 2.7, 2.9, 2.5, and 2.0 for White, Asian, Hispanic, and Black students respectively (Chen et al., 2020). While 75% of White bachelor’s degree recipients had an average GPA of 3.0 or higher, only 55% of Black graduates had an average GPA of 3.0 or above. At the lower end of the GPA scale, Black graduates were three times as likely as White graduates to have an average GPA of less than 2.5 (Woo et al., 2012).
In addition to racial disparities in GPA, another factor that limits the usefulness of GPA in college admissions is grade inflation. Studies report significant grade inflation in high schools as a national trend (Gershenson, 2018; Hurwitz & Lee, 2018). For example, from 1990 to 2019, the average GPA increased by .51 and .54 for Asian and White high school students but increased by only .34 and .41 for Hispanic and Black students (U.S. Department of Education, 2019). Research shows that the gap in GPA between more and less affluent schools started as .31 in 2005 and increased to .41 in 2016 (Gershenson, 2018)—and many minority students come from less affluent schools. In other words, grade inflation is more serious in affluent schools than in lower-income schools. As high school GPA continues to be inflated, the differential inflation rates across the racial groups will further exacerbate the racial inequities in college admissions. College GPA also shows evidence of inflation. For example, the national average GPA at four-year colleges and universities in 1983 was 2.8, and increased to 3.2 in 2013 (Rojstaczer, 2016).
During the pandemic, due to campus closures and many disruptions related to technological issues associated with online learning and online assignment, many college instructors turned to using pass or fail grading rather than letter grades (Tamez-Robledo, 2021). Some K-12 districts are also following practices in higher education to switch to pass and fail grading (Sawchuk, 2020). The shifts in grading policies are intended to ease pressure on students, but they will also have implications for admissions, as GPA is an important metric commonly considered in both college and graduate admissions.
In addition to GPA, admissions requirements around coursework may also have unintended impacts on students from underserved groups. For example, completion of “A-G courses” (prerequisite subject requirements) are required by the UC system as minimum requirements for college admissions. However, a Standardized Testing Task Force formed by the UC Academic Council concluded that “disparities in access to and completion of A-G courses account for a disproportionate lack of UC eligibility for students who are members of underrepresented groups (University of California Academic Senate, 2020, p. 105).”
Letters of Recommendation
Reference letters are another commonly used metric in college admissions. Many institutions require reference letters to gain knowledge about applicants’ skills, accomplishments, and character that may not be captured by GPA and test scores. It is believed that reference letters help reveal aspects of the applicant that discrete evaluation metrics cannot. In most cases, the requirement of reference letters is fairly open-ended in that the evaluators have the flexibility to comment on any aspect that they prefer the admissions committee to know about the applicant. For graduate admissions, some programs may require references from former professors or employers when applicable.
A long-term concern with reference letters has been that applicants from wealthy backgrounds are more likely to find referees who are well-known and influential than applicants from less privileged backgrounds. There are also issues with racial and gender bias and cultural differentiation (Dalal et al., 2022; Highhouse, 2008). Dalal and coauthors reported that there are systematic race and gender differences in the content and evaluation of letters of recommendation, based on over 37,000 letters submitted for over 10,000 graduate school applicants. For example, they found that letters for female applicants tend to focus on personality, ability, and motivation, while letters for males tend to emphasize critical thinking. The content differences in racial and gender groups can favor or disfavor certain groups (Madera et al., 2009). In addition to access and equity issues, evaluation of reference letters is also challenged by low reliability, or consistency. Raters are typically admissions officers in charge of admissions evaluation at the undergraduate level or faculty members in charge of graduate admissions. It is unclear what kind of training is provided to ensure fairness and consistency in their ratings. Research shows that interrater reliability across three raters often fails to exceed .40 (Baxter et al., 1981). Such low reliability limits the predictive validity of reference letters (Klieger et al., 2022). Even quality letters of recommendation provide only modest prediction of graduate school outcomes such as GPA and faculty ratings (Kuncel et al., 2014). Furthermore, admissions officers who evaluate the letters typically lack training, and therefore consensus, on the interpretation of the letters and often rely on idiosyncratic approaches to make inferences about applicants’ qualifications (Dalal et al., 2022).
Research shows that the quality of a recommendation letter can profoundly impact selection outcomes (Liu et al., 2009). Letters that are clearly written with great focus and specificity are more likely to benefit applicants than vague and poorly written letters. Longer letters also tend to be judged more favorably than short and concise letters (Aamont et al., 1998). This may particularly affect international applicants whose evaluators may not be familiar with the norms of letters in the United States and may fail to write a letter that is culturally appropriate and compelling. Because of the unequal access to quality letters across applicants, a small portion of institutions exclude letters of recommendation in their admissions. For example, the University of California, Los Angeles (UCLA) is among the institutions that ban the use of reference letters in their undergraduate admissions. University administrators cite as their reason for excluding letters of recommendation that most public school counselors are heavily loaded with student requests, and that low-income students in particular are likely to have teachers who are overworked and thus may not have the bandwidth to write a letter that adequately represents the student (Jaschik, 2022). There are other institutions that are letter optional. For example, University of California, Berkeley allows reference letters but does not require them.
Given the open-endedness in reference letters and idiosyncrasies in interpretation of letters, efforts have been made to standardize reference letters for better clarity and improved equity in admissions (Klieger et al., 2022; Liu et al., 2009). For example, Klieger and coauthors reported the use of a standardized tool for letters of recommendation, called the Personal Potential Index (PPI), in graduate school admissions. Referees can evaluate applicants on a 5-point Likert scale on six dimensions: Knowledge and Creativity, Communication Skills, Teamwork, Resilience, Planning and Organization, and Ethics and Integrity. Klieger’s validity study found that the PPI was able to predict both graduate school GPA and outcomes such as a student being placed on academic probation. The tool also has potential to promote racial, ethnic, and gender diversity, as there were minimal rating differences shown across racial and gender subgroups. Yet even with standardized tool for letters of recommendation, one challenge that remains is the comparability of ratings due to differences in response styles (e.g., acquiescent response style—tendency to always agree; extreme response style—tendency to use the extreme values on a scale; midpoint response style—tendency to use the midpoint on a scale). Lack of level-setting before evaluators fill out the ratings could lead to inconsistency across ratings.
Essays or Personal Statements
Essays are another commonly used evaluation criterion in both college and graduate school admissions. Sometimes they are also called personal statements or admissions essays. At the undergraduate level, there is wide variation in terms of the number of essays required by institutions and the selection of topics. Some of the most popular topics ask students to share their personal stories, describe how they overcame obstacles or challenged a common belief, share a learning and growth experience, or describe someone or something that inspires them. Many students use Common App, a nonprofit college-application platform, which helps streamline applications to multiple institutions that have shared requirements, to submit essays that can meet multiple application needs. Applicants may also send essays directly to institutions when those institutions have unique essay requirements. At the graduate level, especially for doctoral program applicants, essay requirements tend to focus on applicants’ research interests and prior research experience. Given the disciplinary focus of graduate studies, the essay prompt often places a stronger emphasis on demonstration of relevant domain knowledge and skills than on generic aspirations.
A variety of commercial services are available providing coaching and editing to help applicants write compelling admissions essays. Costs for editing a single essay could run up to hundreds or even thousands of dollars (Jaschik, 2017). This type of service provides advantages to students from wealthy backgrounds, which adds an equity concern to the admissions process. Similarly, students from families with college-educated parents often get support from their parents, and it is almost impossible to determine the degree to which their essays are polished by their parents.
As mentioned earlier, standardized tests are criticized for correlating with family income. Recent research shows that essays are not immune from the influence of family income either. Researchers at Stanford University (Alvero et al., 2021) found that essay content is strongly correlated with students’ self-reported family income, and the correlation is even stronger than the association between SAT scores and household income. Essay content refers to the topics, diction, grammar, and punctuation in an essay. In particular, there seems to be significant differences in the topics students choose to write about between high- and low-income students. Based on an analysis of 240,000 admissions essays from 60,000 applications to the University of California, the authors found that applicants from high-income families tend to write about human nature and seeking answers while applicants from low-income families tend to write about time management and family relationship.
The above discussion of commonly used admissions criteria suggests that although standardized tests have received heightened attention and criticism for equity and fairness concerns, other criteria clearly have their own limitations that should not be neglected. Each individual criterion, although imperfect in and of itself, offers complementary information about an applicant’s strengths and weaknesses, and should be considered in conjunction with other admissions criteria to paint a more holistic picture of an applicant.
Caveats around the Implementation of Holistic Review: Corruption and Bias
Setting aside the fact that each admissions criterion has its own merits and shortcomings, it is the implementation of these criteria that determines whether the intended goals of holistic admissions, such as improving diversity, can be achieved. Despite good intent, there are significant issues with the transparency and fairness of holistic admissions in its implementation (Bastedo, 2021). Given that multiple measures are considered in holistic admissions and subjective judgement is heavily involved in evaluating non-standardized criteria, transparency becomes a serious concern. For example, corruption scandals have arisen as university staff worked with wealthy or influential parents to provide unethical and at times illegal advantages to their children in college admissions. In 2019, a scandal arose over a criminal scheme in which William Rick Singer, the main responsible party of the scheme, used fraudulent practices and bribery, on behalf of applicants’ parents, to influence admissions outcomes at several top U.S. institutions. The FBI investigation into this criminal phenomenon was coded Operation Varsity Blues. Thirty-three parents, including some celebrities, were accused of paying millions of dollars to bribe their offspring’s way into elite institutions. Dozens of parents and others have pleaded guilty (Kasakove, 2021).
There are other examples when given the nebulous practice of holistic reviews, subjective human judgement falls to bias and prejudice. For instance, in the lawsuit filed by Students for Fair Admissions against Harvard College, court documents revealed that Asian American applicants consistently received lower ratings on traits such as likability, courage, kindness, and being “widely respected” based on data from over 160,000 student records (Hartocollis, 2018). It is difficult to quantify how many Asian students were deprived of opportunities to attend Harvard because of such lower ratings, but the impact is detrimental and long-lasting. In January 2022, the Supreme Court agreed to hear the case and a similar one related to admissions practices at the University of North Carolina (Liptak & Hartocollis, 2022).
There are also other factors that are considered in many institutions’ holistic admissions practices that are controversial. For instance, although the focus of the above-referenced Harvard lawsuit was on race-conscious admissions, the exposed legacy admissions drew wide public attention. While Harvard College has an extremely low acceptance rate at 6%, the acceptance rate is 33% for applicants with legacy status (Puig, 2021). Advocates for legacy admissions argue that this practice allows institutions to attract more donor funding which can then be used to expand scholarships and assistantships to help students from underserved backgrounds. However, research-based data from 100 elite universities shows that there is no correlation between legacy admissions preference and increased fundraising for financial aid (Shadowen et al., 2009). In fact, due to the apparent equity issues with legacy admissions, institutions have joined forces to abolish the practice. For example, Amherst College ended its legacy admissions in 2021, joining California Institute of Technology and Massachusetts Institute of Technology (Puig, 2021). In February 2022, lawmakers proposed a bill to ban legacy admission preference at colleges and universities (Merkley, 2022).
Holistic admissions practices are growing in prevalence, yet if they are not implemented in a principled manner, the kind of sorting and ranking of applicants that will occur through holistic review could result in inequities that defeat its purpose. That is why it is critical for institutions to clearly articulate their admissions goals and to focus on evidence to evaluate whether their admissions outcomes have accomplished their admissions objectives, which to a large extent include increasing fairness for applicants from underserved backgrounds.
Resources on Practicing Holistic Admissions and Evidence of Outcomes
In the following section I discuss the resources available to help institutions practice holistic admissions and the research that documents the outcomes of holistic review. Membership organizations, such as the American Association of Collegiate Registrars and Admissions Officers (AACRAO) and the Association of Public Land-Grant Universities (APLU), offer information such as definitions of holistic admissions, criteria typically used in holistic admissions, and events devoted to advancing holistic review. Assessment organizations, whose products and services are included in admissions, also offer insights on holistic review. For example, Educational Testing Service (ETS) has a website devoted to holistic admissions (https://www.holisticadmissions.org/) which discusses the benefits of practicing holistic admissions, how GRE scores can be used equitably in graduate admissions review, and how holistic admissions can be used to increase student diversity.
The Association of American Medical Colleges (AAMC), the organization that administers the MCAT program, discusses core principles of holistic review and lists tools and resources available to schools to practice holistic admissions (https://www.aamc.org/services/member-capacity-building/holistic-review). Given the prevalence of holistic review practice and the need for additional guidance, the Association of Test Publishers (ATP) established a Committee on Holistic Review in 2021. The committee consisted of representatives from major testing organizations including the AAMC, ACT, College Board, Enrollment Management Association (EMA), ETS, Graduate Management Admissions Council (GMAC), and Law School Admissions Council (LSAC). The committee was charged to develop best practice guidelines for implementing holistic review for both undergraduate, graduate, and professional school admissions, including test-optional practices.
Some institutions that practice holistic review also tend to offer brief information on their admissions webpages on what it means to adopt a holistic admissions policy and what general qualifications are considered. However, very little information is provided on how holistic review is executed and how various criteria are considered in deriving a decision on acceptance or rejection.
Despite the widespread adoption of holistic admissions, little research exists documenting its implementation and outcomes in terms of achieving desired admissions goals. Among the limited research that exists, a preponderance of evidence is gathered from health profession programs. For example, a survey of 163 public universities with two or more health profession schools examined to what degree schools incorporated elements of holistic review (e.g., whether the institution includes diversity in its mission statement for admissions; whether the institution considers non-academic criteria as well as academic qualifications in its initial screening of applicants; to what degree the institution considers students’ background such as their first-generation status, race, language proficiency, and community of origin). Results revealed that there is significant variation in the number of elements adopted by schools. Among the institutions that self-identified as practicing holistic review, 38% used many elements considered central to holistic review, 48% adopted some of the elements, and 14% used very few or no elements typically associated with holistic admissions. In terms of the outcomes of holistic review, 72% of the institutions reported increased student diversity, 26% reported unchanged status in diversity, and 2% reported decreased diversity. The impact of holistic review on graduation rate is fairly limited—80% of the institutions reported no change in graduation rate. Around 40% of institutions reported increased average GPA and standardized test scores of the incoming cohorts and about 50% of the institutions reported no change in these two academic metrics (Urban Universities for Health, 2014).
Grabowski (2018) compared admissions outcomes at the Oakland University William Beaumont School of Medicine between holistic review, in which both academic metrics and demographic and experiential factors were considered, and admissions in which solely academic criteria were considered. The researcher found that holistic review yielded acceptance of a higher than expected percentage of female, underrepresented, and first-generation applicants.
Researchers have also created simulated scenarios to see when contextual information is intentionally provided, whether admissions committees are able to admit more students from disadvantaged backgrounds (Bastedo & Bowman, 2017; Bastedo et al., 2021). For example, Bastedo and coauthors (2021) recruited admissions staff from eight selective institutions who were then assigned to applicant cases with some of them randomly including an Environmental Context Dashboard. The dashboard provided contextual information about the applicant’s high school and neighborhood, and a summary index of the high school and neighborhood challenge measures. All applications had been previously reviewed in the high-stakes review process. When comparing the admissions outcomes in the simulated review condition and in the original review condition, the authors found that admissions officers were more likely to grant admission to students from underserved backgrounds when they had access to the applicants’ high school and neighborhood information. The researchers concluded that greater diversity in admissions can be achieved through intentional use of contextual information about the applicants.
Given the unknowns in content coverage and rigor among international higher education systems, and even among domestic homeschool curricula, U.S. institutions and programs often adopt a different set of criteria for applicants from these backgrounds. In some cases, certain requirements are made more stringent in the evaluation of international applications. For example, for international students who apply to graduate studies at Arizona State University, they must have maintained a GPA of 3.00 or above on a 4-point scale in the last 60 semester hours or 90 quarter hours of undergraduate work. For domestic applicants, although the admissions website states that competitive applicants typically have a GPA at or above 3.00, such a GPA is not a requirement for application. Mount St. Mary’s University is another example. They adopt a test-optional admissions policy for domestic applicants but require test scores from international applicants. They also require test scores, or a transcript from an accredited college showing at least 12 completed credits, for homeschooled students. Similarly, Worcester State University has a test-optional admissions policy but requires tests for both international and homeschooled students.
In addition, international applicants whose native language is not English are typically required to submit proof of English language proficiency. Institutions normally offer a range of approved English language assessments that international applicants can use to prove their language skills. Although many institutions using holistic review have done away with testing cut scores in other areas, when it comes to the evaluation of language proficiency, cut score requirements have commonly been maintained. For example, TOEFL is one of the language assessments approved by the test-optional California State Universities—they require a minimum score of 500 on the paper-based exam or a score of 61 on the internet for the TOEFL iBT test for international undergraduate applicants, or scores of 550 and 80 on those tests for international graduate applicants.
Conclusion and Discussion
In this article, I reviewed how holistic review is defined and generally practiced and shared a reminder of why it must be implemented in a principled manner. I also reviewed some of the limited evidence that has been collected of its effects on admissions outcomes. Holistic review represents a movement in higher education to expand the definition of merit by looking beyond a small number of traditionally spotlighted metrics to a broader range of qualifications and experiences. It also stresses the context in which applicants have achieved their accomplishments.
Among all the commonly used evaluation metrics, standardized tests have received the most scrutiny on equity issues such as showing consistent racial performance differences and correlating with household income. Going test-optional has become a trend among U.S. higher education institutions. Although many institutions initially adopted a test-optional policy as a temporary measure in response to the COVID-19 pandemic, these policies may be here to stay. Thus far, other commonly used admissions evaluation metrics such as GPA, letters of recommendation, and essays, have been examined and criticized less closely compared to standardized tests. Yet as the field has begun to examine these measures, evidence has emerged that they are not necessarily immune to the alleged shortcomings of standardized exams (e.g., consistent racial performance differences in high school GPA, correlation of essays with household income, influence of high school resources on letters of recommendation).
Given the shortcomings of each individual admissions criterion, recognizing the strengths and limitations of each metric so that it might be used in an equitable, fair, and transparent way is a new imperative. Completely eliminating the use of a certain criterion may actually hurt the applicants for whom admissions officers intend to expand access. For example, test-optional policies offer flexibility for applicants who believe their chances of acceptance can be increased by submitting their test scores. For institutions that go with a test-blind admissions policy, that opportunity is taken away from applicants. For example, for international students who come from less known colleges and who cannot secure U.S. faculty members to provide letters of reference, standardized test scores may be the only criterion that can put them on a common ground for evaluation with domestic applicants. This is especially so given the variation in how grades are assigned and how GPA is calculated in different countries.
While the essence of holistic review is to move from reliance on a single criterion to draw on a range of qualifications and experiences, there is much variation in how it is implemented across institutions and programs. Additional research is required which, like the study by Urban Universities for Health (2014), examines what specific elements of holistic review institutions are using, and whether such practices are helping to achieve the admissions goals set by institutions. The fact that 14% of the schools that claim that they practice holistic admissions use very few or none of the elements of holistic review is disconcerting. More research should be conducted to shed light on the practice of holistic admissions for improved fidelity and transparency.
Another issue to pay close attention to is what resources are needed to support student development after enrollment. Students identified through holistic review who demonstrate great potential but who have not had an excellent academic record due to extra barriers in life especially need additional resources provided to them to succeed in college. The ultimate value of achieving diversity in an admitted class through holistic review is to ensure that institutions also graduate a diverse student body who can then become effective members of a diverse workforce in the American economy. Data show that the 6-year graduation rate at 4-year institutions of Black (40%) and Hispanic (54%) students is significantly lower than that of White (64%) and Asian (74%) students (U.S. Department of Education, 2017a). Students who are admitted, receive inadequate support, incur substantial debts, and eventually fail to graduate from college are in no better place than students who are not admitted to college, especially considering the employment income they will have foregone while pursuing their education.
I recognize that admissions alone does not compensate for the long-standing inequities in educational achievement and many factors contribute to the racial performance difference by the time students apply for college or graduate school. Many policies and programs are designed to promote equity in the higher education pipeline such as dual enrollment programs at high schools, supports for the free application to federal student aid, and programs offered to prevent summer learning loss for low-income students (Castleman & Page, 2014). It takes an ecosystem approach to address equity issues in higher education. That said, admissions is an important piece of the puzzle and if done right, could help thousands of students benefit from a college education and become effective contributors to the economy and informed citizens.
References
Aamont, M. G., Nagy, M. S., & Thompson, N. (1998, June). Employment references: Who are we talking about? [Paper presentation]. International Personnel Management Association Assessment Council, Chicago, IL.
Alvero, A. J., Giebel, S., Gebre-Medhin, B., Antonio, A. L., Stevens, M. L., & Domingue, B. W. (2021). Essay content is strongly related to household income and SAT scores: Evidence from 60,000 undergraduate applications. Stanford Center for Education Policy Analysis. http://cepa.stanford.edu/wp21-03
Bastedo, M. (2021). Holistic admissions as a global phenomenon. In H. Eggins, A. Smolentseva & H. de Wit (Eds.), Higher education in the next decade (pp. 91–114). Brill.
Bastedo, M., Bell, M., Howell, J., Hsu, J., Hurwitz, M., Perfetto, G., & Welch, M. (2021). Admitting students in context: Field experiments on information dashboards in college admissions. The Journal of Higher Education, 93(3), 1–48. https://doi.org/10.1080/00221546.2021.1971488
Bastedo, M. N., & Bowman, N. A. (2017). Improving admission of low-SES students at selective colleges: Results from an experimental simulation. Educational Researcher, 46(2), 67–77. https://doi.org/10.3102/0013189x17699373
Bastedo, M. N., Bowman, N. A., Glasener, K. M., & Kelly, J. L. (2018). What are we talking about when we talk about holistic review? Selective college admissions and its effects on low-SES students. The Journal of Higher Education, 89(5), 782–805. https://doi.org/10.1080/00221546.2018.1442633
Baxter, J. C., Brock, B., Hill, P. C., & Rozelle, R. M. (1981). Letters of recommendation: A question of value. Journal of Applied Psychology, 66(3), 296–301. https://doi.org/10.1037/0021-9010.66.3.296
Briggs, D. C. (2001). The effect of admissions test preparation: Evidence from NELS-88. Chance, 14(1), 10–18. https://doi.org/10.1080/09332480.2001.10542245
Caplow, T., Hicks, L., & Wattenberg, B. (2001). The first measured century: An illustrated guide to trends in America, 1900–2000. American Enterprise Institute Press.
Castleman, B. L., & Page, L. C. (2014). Summer melt: Supporting low-income students through the transition to college. Harvard Education Publishing Group.
Chavez, J. (2021, May 23). UC agrees to no longer consider ACT/SAT scores in admissions. The University of California, San Diego Guardian. https://ucsdguardian.org/2021/05/23/uc-agrees-to-no-longer-consider-act-sat-scores-in-admissions/
Chen, X., Caves, L. R., Pretlow, J., Caperton, S. A., Bryan, M., & Cooney, D. (2020). Courses taken, credits earned, and time to degree: A first look at the postsecondary transcripts of 2011–12 beginning postsecondary students (NCES 2020–501). U.S. Department of Education. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2020501
College Board. (2022, March 9). About us. https://about.collegeboard.org/
Dalal, D. K., Randall, J. G., Cheung, H. K., Gorman, B. C., Roch, S. G., & Williams, K. J. (2022). Is there bias in alternatives to standardized tests? An investigation into content differences in letters of recommendation by race, gender, and degree program. International Journal of Testing, 22(1), 21–42. https://doi.org/10.1080/15305058.2021.2019751
Domingue, B., & Briggs, D. C. (2009). Using linear regression and propensity score matching to estimate the effect of coaching on the SAT. Multiple Linear Regression Viewpoints, 35(1), 12–29.
Dulin, J. (2017). The effect of an SAT prep course on SAT scores at a low income urban school [Master’s thesis, Goucher College]. Goucher Special Collections & Archives.
FairTest. (2019, September 17). 2019 the best year ever for test-optional higher ed. admissions. https://www.fairtest.org/2019-best-year-ever-testoptional-higher-ed-admissi
Gershenson, S. (2018). Grade inflation in high schools (2005–2016). Thomas Fordham Institute. https://fordhaminstitute.org/national/research/grade-inflation-high-schools-2005-2016
Grabowski, C. J. (2018). Impact of holistic review on student interview pool diversity. Advances in Health Sciences Education, 23(3), 487–498. https://doi.org/10.1007/s10459-017-9807-9
Harden, K. P. (2022, April 2). The SAT isn’t what’s unfair: MIT brings back a test that, despite its reputation, helps low-income students in an inequitable society. The Atlantic. https://www.theatlantic.com/ideas/archive/2022/04/mit-admissions-reinstates-sat-act-tests/629455/
Hartocollis, A. (2018, June 15). Harvard rated Asian-American applicants lower on personality traits, suit says. The New York Times. https://www.nytimes.com/2018/06/15/us/harvard-asian-enrollment-applicants.html
Hausknecht, J. P., Halpert, J. A., Di Paolo, N. T., & Moriarty Gerrard, M. O. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92(2), 373–385. https://doi.org/10.1037/0021-9010.92.2.373
Highhouse, S. (2008). Stubborn reliance on intuition and subjectivity in employee selection. Industrial and Organizational Psychology, 1(3), 333–342. https://doi.org/10.1111/j.1754-9434.2008.00058.x
Hurwitz, M., & Lee, J. (2018). Grade inflation and the role of standardized testing. In J. Buckley, L. Letukas, & B. Wildavsky (Eds.), Measuring success: Testing, grades, and the future of college admissions (pp. 64–93). Johns Hopkins University Press.
Jaschik, S. (2017, October 16). When application essay ‘help’ crosses a line. Inside Higher Ed. https://www.insidehighered.com/admissions/article/2017/10/16/worries-grow-about-application-essay-help-may-go-too-far
Jaschik, S. (2022, February 22). Are recommendation letters a form of discrimination? Inside Higher Ed. https://www.insidehighered.com/admissions/article/2022/02/22/colleges-consider-whether-recommendations-are-fair?utm_source=Inside+Higher+Ed&utm_campaign=ac6ea4d212-DNU_2021_COPY_02&utm_medium=email&utm_term=0_1fcbc04421-ac6ea4d212-236844202&mc_cid=ac6ea4d212&mc_eid=61c0f6156d
Kantrowitz, M. (2021, May 21). How admissions tests discriminate against low-income and minority student admissions at selective colleges. Forbes. https://www.forbes.com/sites/markkantrowitz/2021/05/21/how-admissions-tests-discriminate-against-low-income-and-minority-student-admissions-at-selective-colleges/?sh=43a9d1eb3cc1
Kaplan, J. (2005). The effectiveness of SAT coaching on math SAT scores. Chance, 18(2), 25–34. https://doi.org/10.1080/09332480.2005.10722718
Karabel, J. (2005). The chosen: The hidden history of admission and exclusion at Harvard, Yale, and Princeton. Houghton Mifflin.
Kasakove, S. (2021, October 9). The college admissions scandal: Where some of the defendants are now. The New York Times. https://www.nytimes.com/2021/10/09/us/varsity-blues-scandal-verdict.html
Klieger, D., Bochenek, J., Ezzo, C., Holtzman, S., Cline, F., & Olivera-Aguilar, M. (2022). Using third-party evaluations to assess socioemotional skills in graduate and professional school admissions. International Journal of Testing, 22(1), 72–99. https://doi.org/10.1080/15305058.2021.2019748
Kuncel, N. R., Kochevar, R. J., & Ones, D. S. (2014). A meta-analysis of letters of recommendation in college and graduate admissions: Reasons for hope. International Journal of Selection and Assessment, 22(1), 101–107. https://doi.org/10.1111/ijsa.12060
Liptak, A., & Hartocollis, A. (2022, January 24). Supreme court will hear challenge to affirmative action at Harvard and U.N.C. The New York Times. https://www.nytimes.com/2022/01/24/us/politics/supreme-court-affirmative-action-harvard-unc.html
Liu, O. L., Minsky, J., Ling, G., & Kyllonen, P. (2009). Using the standardized letters of recommendation in selection: Results from a multidimensional Rasch model. Educational and Psychological Measurement, 69(3), 475–492. https://doi.org/10.1177/0013164408322031
Madera, J. M., Hebl, M. R., & Martin, R. C. (2009). Gender and letters of recommendation for academia: Agentic and communal differences. Journal of Applied Psychology, 94(6), 1591–1599. https://doi.org/10.1037/a0016539
Merkley, J. (2022, February 2). Merkley, Bowman introduce bill to end legacy admission preferences at colleges and universities [Press release]. https://www.merkley.senate.gov/news/press-releases/merkley-bowman-introduce-bill-to-end-legacy-admission-preferences-at-colleges-and-universities-22
Puig, S. (2021, October 28). High time to end legacy admissions. The Harvard Crimson. https://www.thecrimson.com/article/2021/10/28/high-time-to-end-legacy-admissions/
Rampell, C. (2009, August 27). SAT scores and family income. The New York Times. https://economix.blogs.nytimes.com/2009/08/27/sat-scores-and-family-income/
Reardon, S., & Galindo, C. (2009). The Hispanic-white achievement gap in math and reading in the elementary grades. American Educational Research Journal, 43(3), 853–891. https://doi.org/10.3102/0002831209333184
Rojstaczer, S. (2016, March 29). Grade inflation at American colleges and universities. https://www.gradeinflation.com/
Sawchuk, S. (2020, April 1). Grading students during the coronavirus crisis: What’s the right call? EducationWeek. https://www.edweek.org/teaching-learning/grading-students-during-the-coronavirus-crisis-whats-the-right-call/2020/04
Shadowen, S. D., Tulante, S. P., & Alpern, S. L. (2009). No distinctions except those which merit originates: The unlawfulness of legacy preferences in public and private universities. Santa Clara Law Review, 49, 51–136. https://ssrn.com/abstract=1299224
Smith, E. & Reeves, R. (2020, December 1). SAT math scores mirror and maintain racial inequity. Brookings. https://www.brookings.edu/blog/up-front/2020/12/01/sat-math-scores-mirror-and-maintain-racial-inequity/
Tamez-Robledo, N. (2021, May 26). Pass/fail grading was an act of pandemic compassion. Is it here to stay? Edsurge. https://www.edsurge.com/news/2021-05-26-pass-fail-grading-was-an-act-of-pandemic-compassion-is-it-here-to-stay
University of California Academic Senate. (2020). Report of the UC Academic Council Standardized Testing Task Force (STTF). https://senate.universityofcalifornia.edu/committees/sttf/reports.html
Urban Universities for Health. (2014). Holistic admissions in the health professions: Findings from a national survey.
U.S. Department of Education, National Center for Education Statistics. (2017a). Integrated Postsecondary Education Data System (IPEDS), Winter 2016–17, graduation rates component. Table 326.10 https://nces.ed.gov/programs/digest/
U.S. Department of Education, National Center for Education Statistics. (2017b). National Assessment of Educational Progress (NAEP), selected years, 1990–2017, Table 222.10. https://nces.ed.gov/programs/raceindicators/indicator_rcb.asp
U.S. Department of Education, National Center for Education Statistics. (2019). High school transcript study, various years, 1990–2019. https://nces.ed.gov/nationsreportcard/hsts/
U.S. Department of Education, National Center for Education Statistics. (2021). Digest of Education Statistics, 2019 (NCES 2021–009), Table 226.10. https://nces.ed.gov/programs/digest/d19/tables/dt19_226.10.asp
Wellemeyer, J. (2019, July 7). Wealthy parents spend up to $10,000 on SAT prep for their kids. MarketWatch. https://www.marketwatch.com/story/some-wealthy-parents-are-dropping-up-to-10000-on-sat-test-prep-for-their-kids-2019-06-21
Woo, J., Green, C., & Matthews, M. (2012). Web tables: Profile of 2007–2008 first-time bachelor’s degree recipients in 2009. U.S. Department of Education. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2013150
Wright, B. (2018, April 25). Some racial and ethnic groups do much better on NAEP than others. Here are some factors that might explain why. Fordham Institute. https://fordhaminstitute.org/national/commentary/some-racial-and-ethnic-groups-do-much-better-naep-others-here-are-some-factors
Yale University. (2022a). Applied Mathematics Program Admissions. https://applied.math.yale.edu/graduate-program/admissions
Yale University. (2022b). Department of Applied Physics Admissions. https://appliedphysics.yale.edu/academics/graduate-studies/admissions
Yale University. (2022c). Yale Macmillan Center Council on East Asian Studies Admissions. https://ceas.yale.edu/admissions