Early Assessment of Cognitive Skills, Self-Regulated Learning Skills, and Attitudes toward Education Predict University Success at Graduation

Cognitive Skills and Academic Success

Historical arguments supporting college entrance exams were placed on notions of “aptitude”—with claims that performance on those measures were strong and valid indicators of likely academic success in the first year of college (e.g., Hezlett et al., 2001). However, research has repeatedly demonstrated that standardized admissions assessments in isolation are relatively ineffectual in determining first year performance (Hannon, 2014; Heller, 2015), and biased against traditionally underserved populations—illustrating that the scores on these aptitude measures are largely a function of the high schools the students attended or more strongly correlated with socioeconomic status than ability (Allensworth & Clark, 2020; Zwick & Green, 2007). In line with this, there is a growing body of research demonstrating that measures of learning strategies and attitudes are more predictive of performance in university settings than “ability” indicators in the form of entrance exams (Crede & Kuncel, 2008; Kitsantas et al., 2008).

While traditional standardized measures of college readiness have contributed to biased admissions processes and perpetuated an implicitly racist and sexist system of access to education (Allensworth & Clark, 2020; Buchmann et al., 2010), superior cognitive processing skills and abilities are beneficial to learners engaged in academic tasks (Hannon, 2014). The work that focused on “grit” as more instrumental in determining student outcomes (e.g., Duckworth & Seligman, 2005) maintained that cognitive skills or intelligence were instrumental as well. Recent replications of that work have re-opened the debate over which is more important (self-control/grit vs ability; see Vazsonyi et al., 2022), but this debate maintains the important result that both cognitive abilities and SRL skills (discussed later) are instrumental in determining academic performance in college.

The academic advantage afforded by strong cognitive processing skills is rooted in evidence connecting academic success to superior executive function (Baars et al., 2015), general intelligence (Coyle & Pillow, 2008), or neuropsychological features tied to working memory operation (D’Esposito & Postle, 2015). A comprehensive review of cognitive skills related to academic performance is beyond the scope of this investigation, but we offer that a simple representation of processing efficiency or cognitive load captures the essence of this argument effectively. Consistent with dominant cognitive models of learning and memory (e.g., Baddeley & Hitch, 2019; Mayer, 2019), a cognitive load or processing efficiency perspective asserts that the ability to process content with greater fluidity minimizes the “cognitive effort” required to complete a given task (Sweller, 2015; van Merriënboer & Sweller, 2005). At the most basic level, cognitive load theory (CLT) explains that learning events (e.g., reading, listening to lectures, working on problems) tap our limited processing capacity in working memory through three types of load (Moreno & Park, 2010). Intrinsic load is the necessary processing effort required by the task, influenced by the complexity of the content—which is unique to each learner based on experience and ability. Extraneous load is any irrelevant or distracting aspect of the learning task that draws resources away from the actual task—either poor instructional design or intruding distractions (DeLeeuw & Mayer, 2008). Finally, germane load is the proportion of working memory capacity devoted to generative learning or “deep processing” that promotes long term memory storage strength or conceptual change (Fiorella & Mayer, 2016; Wittrock, 2010).

In agreement with the CLT framework, research on achievement emotions (e.g., Eysenck et al., 2007) has implicated processing efficiency as a key aspect that differentiates outcomes for learners who are negatively impacted by negative affective stimuli (Eysenck & Calvo, 1992). Put simply, these models hold that learners with greater cognitive skills and background knowledge in domains are able to more efficiently process the intrinsic load required by tasks and thereby have a greater reserve of available working memory capacity that can promote generative processing (i.e., germane load) or tolerate extraneous load imposed by distractions, poor instructional design, or perseveration on negative affective stimuli (Eysenck et al., 2007; Moreno, 2009; Sweller, 2015). Naturally, limiting being “overloaded” improves the ability to select, organize, and integrate learning content effectively, supporting academic success (Fiorella & Mayer, 2016).

Self-Regulated Learning and Student Success Outcomes

As students proceed through academic levels, external support and control of study behaviors and operations is decreased. In essence, the educational model generally anticipates progressively higher expectations of independent learning skills to be demonstrated by students as they advance from elementary programs through postsecondary education. However, in the first year of university or college, many students lose a significant portion of their academic support network that was built over their elementary and secondary education careers (Heller & Cassady, 2017). During that transitional phase in education, student achievement has been reliably and repeatedly tied to their SRL strategies (Crede & Kuncel, 2008; Weinstein & Palmer, 2002; Weinstein et al., 1987). Similarly, intervention studies focused on promoting SRL strategies for learners have translated effectively into enhancing both student skills and student outcomes (Cleary & Platten, 2013).

Self-regulated learners effectively monitor and control cognitive, motivational, and emotional processes to learning to support their academic success (Lens & Vansteenkiste, 2012). Instead of viewing themselves as passive receivers of information, self-regulated learners perceive themselves as agentic (Bandura, 2008) and in control of their learning outcomes (Zimmerman et al., 2017). Given the centrality of self-monitoring and adjustment to meet their short- and long-term academic goals, learners with high SRL skills effectively recognize performance-inhibiting and maladaptive strategies (e.g., procrastination, distractibility), address environmental demands (e.g., time and resource management), and monitor strategy effectiveness (Kitsantas & Cleary, 2016; Zeidner & Matthews, 2005; Zimmerman et al., 2017). As a core conceptual framework within educational psychology, SRL skills consistently predict significant variations in student performance across disciplines (Kitsantas & Cleary, 2016; Zimmerman & Kitsantas, 2014), and is often attributed as a principal feature differentiating high and low achieving students with otherwise similar backgrounds (Zimmerman et al., 2017).

Student Affect and Emotionality Influences on Performance

In addition to the impacts outlined above that student affect has on cognitive efficiency and SRL skills, the literature has identified that students’ affective (sometimes termed emotional or attitudinal) orientations to academic content may be a direct predictor of success and retention (Crede & Kuncel, 2008; Pekrun, 2006; Pekrun et al., 2007). Student affect can be represented as either state (fluid) or trait (dispositional) dimensions with positively (interest, enthusiasm) or negatively (distress, irritability) valanced emotional responses (Boekarts & Pekrun, 2015; Eschleman et al., 2012). Contemporary literature frequently highlights the influence affect maintains over long-term performance within academic settings; positive affect is generally associated with academic success whereas negative affect is linked to reductions in working memory capacity, processing efficiency, or adopting behaviors (e.g., avoidance, procrastination, withdrawal) that promote academic self-handicapping (Eysenck et al., 2007; Zeidner & Matthews, 2005).

The influence attitudes and emotions have on academic success and performance has been explained through the Emotional Information Processing model (EIP; Cassady & Boseck, 2008; Cassady & Thomas, 2020). Building upon constructivist principles of individualized representations of contextual cues, emotion regulation strategies, and SRL models with iterative feedback loops, the EIP seeks to explain how students navigate emotionally challenging events in a recursive cycle. Put simply, the EIP postulates that individuals identify and interpret situational challenges and establish goals that are either growth-oriented (generally positive) or ego-preserving (generally maladaptive) before envisioning and enacting solutions or strategies in contexts where they are emotionally charged (Cassady & Thomas, 2020).

As such, when learners’ conceptions of academic contexts center on perceived threats, goals and strategies tend to be more maladaptive or protective—focused on reducing the perceived threat of the situation rather than activating coping mechanisms to “solve the problem” at hand (Lazarus, 2006). Conversely, learners with a generally positive affective orientation toward academic contexts are characterized by positive attitudes, interest in learning, and curiosity for the topics (Alexander & Grossnickle, 2016). In those settings where the learner maintains the more positive affective orientation, the range of responses tend to be focused on adaptive coping strategies and engagement with the content (Cassady & Thomas, 2020).

Generational Status and Higher Education Success

One key learner characteristic that is commonly discussed in relation to retention and success in higher education is generational status. Applied research and institutional analyses of higher education success have centered on the unique challenges navigated by many FG college students as compared to their NFG peers. Studies have attributed a general trend of lower outcome achievement for FG students to issues including financial challenges (Cho et al., 2008), familial support (Saenz & Barrera, 2007), prior academic preparation (Atherton, 2014), mismatch in social or cultural norms (Stephens et al., 2012), and a range of emotional responses that orient toward a perception of the university setting as more threatening than NFG learners (Harackiewicz, et al., 2014; Woosley & Shepler, 2011).

While we agree that attention to FG students is critical to support long term success for a growing population of higher education learners, it is important to recognize Woosley and Shepler’s (2011) assertion that there is wide diversity within the broad population of FG students, and overgeneralization of findings in the literature is a perilous error. To that effect, the purpose of this direct focus on FG and NFG populations is to explore if there are aspects of higher education entry factors that appear to be more influential in predicting performance trends for students in these two broad groups. The benefit of this focus is the potential to identify domains of influence in postsecondary success that are unique for FG learners, leading to potential intervention developments to meet needs for FG learners (see Goldman et al., 2020; Harackiewicz et al., 2014).

Current Study

Using data captured from an institutional perspective, this study employed a predictive model determining long-term student performance outcomes in a traditional four-year university located in the Midwest United States. Using institutional data collected from all incoming first year students over a seven-year period, this study explores a model of long-range student success examining key characteristics at the point of university matriculation. Specifically, using data from the Learning and Study Strategies Inventory—High School Edition (LASSI-HS), this study explores the relative and combined influences of four primary dimensions on student success over the course of their college careers (as measured by graduating GPA). The four contributing features of interest were basic cognitive skills, SRL strategies, affective responses to educational settings, and anxiety toward academic contexts. In addition, the study focused on generational status to identify if there were differential patterns of influence for these key variables on graduating GPAs for learners who were FG and NFG learners.

Measures

Data Analysis

In order to address the research questions of interest, four separate structural equation models (SEMs) were fit to the data. The models we explored were founded on the theoretical notion that cognitive skills and SRL influenced long-term performance (i.e., graduating GPA), but that the relationship was potentially mediated by attitudes and beliefs about education (i.e., “affect”). Furthermore, our focus on anxiety as a moderating variable required removing that component of affect. The four models reflected a fully mediated model (Figure 1, Panel A), a partially mediated model (Figure 1, Panel B), a moderated fully mediated model (Figure 1, Panel C), and a moderated partially mediated model (Figure 1, Panel D). The optimal model was determined based on the AIC, BIC, and sample size adjusted BIC (aBIC), with smaller values indicating overall better models.

Once the optimal model was identified, multiple group SEMs were used to ascertain whether there were differences in structural coefficients between FG and NFG students. Two such models were fit, one in which the structural coefficients were allowed to vary between the groups and the other in which the coefficients were constrained to be equal across groups. The fit of the two models were compared by using a Chi-square difference test for which the null hypothesis was that the models allowed structural coefficients to vary across groups and that constrained them to be equal by group yield equivalent model fit. Therefore, a statistically significant result for the Chi-square statistic indicates that the structural coefficients differed between groups. Finally, prior to fitting the aforementioned SEMs, measurement models for each of the factors were fit to ensure that the hypothesized factor structure fit the data appropriately. Fit of these models was ascertained using commonly used fit statistics, including CFI, TLI, and RMSEA. Values of CFI and TLI ≥ 0.95 and RMSEA values ≤ 0.05 were taken to indicate acceptable fit (Kline, 2015). All model parameter estimation was carried out using maximum likelihood with Mplus, version 8.6 (Muthèn & Muthèn, 2020).

Descriptive Statistics

The disaggregated means, standard deviations, 95% confidence intervals, and Cohen’s d values for specific means comparisons appear in Table 1. The focus of those explorations was centered on comparisons among FG and NFG students as well as those who ultimately graduated within six years and those who did not. Based on Cohen’s (1992) guidelines, the mean differences between FG and NFG students on Cognitive Skills (COG), SRL, Affect, and Anxiety were all negligible. The difference in mean graduating GPA between the groups fell into the small range (Cohen, 1992), with NFG students having a higher mean GPA than FG students. With respect to graduation status, mean differences were negligible with the exception of Affect, for which the difference was small in magnitude. Students who ultimately graduated had a higher mean Affect score than did those who did not graduate within six years of matriculation.

Measurement Models

Measurement models for each of the factors were fit to the data, and yielded evidence that the hypothesized latent structure did in fact fit the data, with each being represented by a single factor. The model fit statistics for the individual factor models were: SRL (RMSEA=0.031, CFI=0.97, TLI=0.96), COG (RMSEA=0.038, CFI=0.96, TLI=0.95), Affect (RMSEA=0.029, CFI=0.97, TLI=0.97), and Anxiety (RMSEA=0.034, CFI=0.95, TLI=0.95). Given these positive results for model fit, it was possible to continue with the SEMs.

Table 1. First-Generation Status and Graduation Status Group Comparisons on Primary Variables

Note. SD = standard deviation; CI = confidence interval.

Affect Mediates Self-Regulated Learning and Cognitive Skills

As expected, both SRL and cognitive skills had a measurable, direct influence on graduating GPA. However, those relationships were also primarily transmitted through the educational attitudes (cf. “Affect”) they maintained at the point of university entry (see Table 2). Interpretation of these effects for the total sample are somewhat premature given the unique interaction identified based on first-generation status (discussed below). However, the generalized pattern demonstrates that simple analyses exploring the influence of cognitive abilities and/or SRL skills on student performance outcomes without including measures of learner affect (i.e., achievement emotions, attitudes toward education) oversimplify the relationships. This finding reinforces work illustrating the importance of examining academic achievement within a complete context with attention to the interactions among learners’ cognitive and affective responses in academic settings (Cassady & Thomas, 2020; Pekrun, 2006). It is worth noting that a simple representation of these data of merit is that attitudes and beliefs learners hold toward education at the university level were the most powerful predictor of graduation and graduating GPA in our sample.

Academic Anxiety Moderates Self-Regulated Learning and Cognitive Skills

After accounting for the influence of general Affect in the model, the data also demonstrate that the influence of Anxiety (as measured by the LASSI-HS) moderates the influence of cognitive skills and SRL on GPA. This effect indicates that for individuals with high Anxiety, the influence of cognitive skills on graduating GPA is significantly greater than for learners with moderate and low Anxiety. This relationship essentially demonstrates that the range of graduating GPA is greatest for the high Anxiety learners, and high Anxiety learners with the lowest levels of cognitive skills demonstrate the lowest levels of GPA over time. Conversely, learners reporting high levels of Anxiety with high cognitive skills had the highest overall GPAs at graduation (see Figure 2). The same moderation effect was noted for the influence of SRL skills on graduating GPA.

The results demonstrate that high levels of Anxiety reported on the LASSI-HS scale do not directly translate to lower performance over time—the true impact of anxiety is only realized when examining how anxiety interacts with a student’s cognitive skills and SRL skills (Zeidner & Matthews, 2005). These patterns align with models identifying the potential facilitative elements of academic anxieties, where it serves as an “activating” stimulus within a transactional processing model (Lazarus, 2006). As illustrated in Figures 2 and 3, the presence of anxiety functions positively for those learners with more confidence in their cognitive and SRL skills. For those learners with low cognitive abilities and SRL skills, the pattern was reversed. From a basic transactional processing framework, these data would be explained as demonstrating that when faced with an academic stressor (perceived threat appraisal), learners with high levels of cognitive and SRL skills are likely to identify and employ more effective—or adaptive—coping strategies leading to success (see also the EIP; Cassady & Thomas, 2020). Conversely, anxious learners with lower skills faced with perceived threats in academic settings are more likely to adopt failure predictions, avoidance behaviors, and employ ineffective coping strategies (leading to eventual performance declines; Thomas et al., 2017).

The moderating effects highlight the importance of explicitly acknowledging the bidirectional and interactive nature in the relationships among learners’ affective and cognitive constructs (Pekrun, 2006). The results of the moderation analyses may also be explained through a buffering effect, consistent with the Processing Efficiency Theory (PET; Eysenck & Derekshan, 2011). The PET (and related Attention Control Theory [ACT]) predicts that learners with higher cognitive and SRL skills dealing with anxiety may not perform at optimal efficiency, but they may not be disrupted enough to show performance declines (Wong et al., 2013). Examining academic anxiety as extraneous cognitive load (Plass & Kalyuga, 2019), the PET (and ACT) would predict that those with high levels of skill will have a higher degree of tolerance to the distracting anxiety stimuli (Pekrun, 2006; Pekrun et al., 2007).

First-Generation Status and Predicted Graduation Outcomes

An additional extension of this analysis was to explore the differential effects demonstrated in the data when examining those students identified as FG learners as compared to NFG. The results demonstrated that there was a significant difference in the relationships among the study variables for the two groups, calling attention to the importance of examining student success models in higher education institutions with attention to academic generational status.

More specifically, the results indicate that the Anxiety effects discussed in the “total sample” model were only present for the NFG students. That is, when we disaggregated the data set and examined the influence of Anxiety as a moderating variable for FG and NFG learners separately, the moderation effect did not hold for the FG students. As shown in Panel B of Figures 2 and 3, FG students with high, moderate, and low Anxiety all reported the same degree of relationship between cognitive skills and GPA (as mediated through Affect) and between SRL and GPA. However, the NFG learners demonstrated that the moderating influence of Anxiety was present.

Similarly, the results demonstrate a significant difference in the predictive power of Affect on graduating GPA when comparing FG and NFG learners. The influence of Affect on graduating GPA was significantly stronger for NFG students than FG students. The attitudes, interests, and motivation toward education held by NFG learners was the strongest predictor of graduating GPA (see Table 2)—and the difference in the magnitude of the coefficients reflecting those relationships was approximately 25%.

This pattern of results raises questions regarding why FG students and NFG students had differential experiences with the importance of anxiety on eventual graduating GPA. Examination of the overall group values demonstrate similarity in both mean and standard deviations in the Anxiety measure for both groups, discounting a simple measurement explanation. One explanation for this may rest in the differential focus of academic anxiety for the NFG and FG students. While FG students demonstrated similar levels of anxiety reported at the point of college entry when compared to their NFG peers, that anxiety was not instrumentally related to their confidence in the COG and SRL domains. As such, it is possible that the source of academic anxiety was attached to a broader uncertainty of what their college experience may entail or require. Given the lower degree of familial experiences with university life for FG learners as compared to the NFG population, the anxiety over success and thriving in school may have been focused more on external threats (finances, housing, transition) than on the features examined by the LASSI (Cho et al., 2008; Saenz & Barrera, 2007; Stephens et al., 2012). Conversely, NFG students’ anxiety appears to be directly connected to their perceived abilities in COG and SRL, reflecting a more internal source of their uncertainty about outcomes (and subsequently reflected in overall performance levels).

Limitations and Extensions

As with any institutional dataset, limitations to the study are to be noted. This study was limited in the ability to explore the degree to which learners over the seven years sought and received support for initially-identified limitations in cognitive, self-regulatory, or affective orientations to education. Programs of support exist in most postsecondary education settings that provide training on study skills, counseling support, and engagement in their fields of study (see Kitsantas et al., 2008). These factors are likely to promote optimal performance, and were inaccessible on the individual level for these analyses. However, given the large sample, we propose that these effects were likely distributed across our sample and did not pose a significant threat to validity in our findings. Furthermore, specific demographic information was not available for these analyses. As such, the data do not provide the opportunity to explore variations among learners from diverse populations (beyond the simple FG vs NFG variable). The university in this study serves a primarily White population drawn largely from the Midwestern United States, leaving the question of whether these patterns are fully representative of a more diverse population. Finally, we were unable to control for program majors in these analyses. While all students were in a four-year program with a common core set of liberal arts content for all majors, there are possible variations across programs (e.g., STEM vs the Arts) that would provide insights into support services for students in diverging areas of study.

Conclusions

The conclusions from this large-scale study of student success and retention illustrate that early identification systems (i.e., universal assessment) can provide critical information relevant to the success of students in higher education. While there is considerable evidence that cognitive skills, SRL skills, and affective dispositions toward education are all meaningful predictors to success, this study provides a clear test of the relative and interactive influences of these features in determining factors instrumental in predicting long-range success (as measured by graduating GPA) at the university level. Overall, the results demonstrated that while confidence in cognitive skills and strong SRL habits at the start of postsecondary education have a measurable influence on graduating GPAs (4–6 years later), these factors are heavily influenced by affective variables (both positive affect and anxiety). This finding highlights the importance of early identification of learners with maladaptive attitudes toward education (e.g., academic anxiety) to provide institutional support to mitigate negative influences on performance and support optimal outcomes. While this study was focused only on the entry point of college, we believe that ongoing and iterative assessments of student attitudes, anxiety, and skills are a simple solution to provide universities with systemic data addressing the changing needs of their student populations. This universal assessment strategy provides the opportunity to more directly target supports for all learners, isolating those individuals who may differentially benefit from support programs focused on affective interventions (e.g., positive self-care, mindfulness training, emotional reframing), proactive coping strategies focused on SRL skills (e.g., study strategies, time management, planning), or both, depending on the profile of needs demonstrated.