Abstract Background: A Network Approach to URM STEM Student Success and Organizational Change Literature Review Context Leadership Supporting URM STEM Students Instituting Instructional Innovation Creating Mentoring and Research Opportunities Theoretical Framework Research Design Findings Discussion Implications Conclusion Disclosure Statement Conflict of Interest Statement References

It’s in Our DNA: Leadership Perspectives on Institutionalizing STEM Success in an Alliance

Jonathan J. Okstad*, Victoria E. Callais, Norma López, Funmilayo Ojikutu, Demetri L. Morgan, & Alaa Abdelghaffar

Loyola University Chicago

Abstract

This study investigated how institutional leaders within an alliance navigate and use their agency to cultivate organizational change to support the success of underrepresented racial minority (URM) science, technology, engineering, and math (STEM) students. As part of this study, we partnered with the Illinois Louis Stokes Alliance for Minority Participation (ILSAMP), a signature program of the National Science Foundation (NSF), to explore our research question. The phenomenon of interest is the institutional leaders’ perceptions of their agency and their organization’s efforts to engage in the Alliance and support URM STEM student success through the various initiatives. The research team conducted 20 semi-structured interviews with institutional leaders and faculty at 11 public and private institutions as part of the STEM alliance. We utilized Braun and Clarke’s (2006) six-phased thematic analysis to identify patterns of meaning within the data that respond to the research question. Findings revealed a leadership perspective that frames approaches to STEM initiatives becoming a part of an institution’s fabric. The contribution of this study relates to the illumination of the tension between institutional leaders’ agency to make change sustainable versus structural and leadership networks inhibiting STEM success efforts.

Keywords: organizational change, leadership, STEM, underrepresented racial minorities

* Contact: jokstad@luc.edu

© 2023 Okstad et al. This open access article is distributed under a Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/)

It’s in Our DNA: Leadership Perspectives on Institutionalizing STEM Success in an Alliance

With evolving socio-political tensions, higher education institutions are required to be more innovative, financially prudent, aware of workforce realities, and responsive to student, staff, and faculty demands. In order to assure the successes and triumphs of student-centered initiatives amid these tensions, they must ensure sustainability of organizational change efforts by scaling-up proven programs. Kezar and Sam (2013) point to “leadership as one of the key levers in institutionalizing organizational change and innovation into a culture” (p. 61). Too often, the assumption is that innovations can be applied to any context without altering the core structures or activities. Yet, those same core structures can act as incentives that prohibit change in scale-up or institutionalization efforts (Kezar, 2011).

To scale-up means to “achieve a broad reach of programs and interventions that work to improve access and help students to transfer and persist in their college education” [emphasis added] (Kezar, 2011, p. 235). Kezar (2011) highlights that many scaling-up efforts in educational organizations, especially tied to NSF (National Science Foundation)-funded work, have not reached maximum replication, or broad reach, because the efforts “often involve a static innovation that is considered to work in different contexts, even as circumstances change over time” (p. 240). The circumstances that change over time refer to leadership, funding, and community interest in the intervention or activity (Kezar, 2011).

The shifting demands on institutions amid public health crises and the aforementioned tensions prompt a deeper examination of organizational change and scale-up efforts. To study this issue in particular, our guiding proposition is that scale-up efforts require a focus on leadership dynamics in response to organizational change challenges and how they vary across all levels of a network that is focused on the success of underrepresented racial minority (URM) students majoring in science, technology, engineering, and math (STEM). Accordingly, for this study, we partnered with a STEM alliance which aims to increase the number of STEM baccalaureate and graduate degrees awarded to populations historically underrepresented in these disciplines: African Americans, Hispanic Americans, American Indians, Alaska Natives, Native Hawaiians, and Native Pacific Islanders, as defined by the NSF.

Background: A Network Approach to URM STEM Student Success and Organizational Change

The NSF Louis Stokes Alliance for Minority Participation (LSAMP) program was established in 1993 and is a signature program of the NSF which is geared towards providing resources for institutions to design, implement, and synergize institutional supports that have proven to contribute to the success of URM students majoring in STEM (Baber & Jackson, 2018; Clewell et al., 2006). The Urban Institute’s comprehensive report on the LSAMP program concluded that since the general LSAMP framework of initiatives is successful (e.g., promoting mentoring, internships, undergraduate research, peer support), all alliances should seek to replicate and expand the most promising LSAMP characteristics (Clewell et al., 2006). However, the report provided little guidance on the best strategies to promote the scaling-up of the identified successful components from one institution to another within an alliance.

As a result of these shifting realities, while examining initiatives within LSAMP for what works (National Center for Education Evaluation and Regional Assistance, 2018), equity-centered STEM research runs the risk of overlooking the organizational components that impede or propel the scaling efforts. Consequently, there is a need to identify the ever-evolving challenges educators and leaders face (or create) in broadening access and success for URM students in STEM.

The Illinois Louis Stokes Alliance for Minority Participation (ILSAMP, or Alliance) represents one of several consortia of colleges and universities throughout the United States and Puerto Rico and consists of three 2-year institutions, nine 4-year institutions, and two industry partners in the state of Illinois (see Table 1). In terms of innovative initiatives within the ILSAMP, beyond sharing exemplar practices on cultivating undergraduate research or mentoring URM students in STEM, at the outset of a new funding cycle, the ILSAMP leadership agreed to implement a learning assistants’ program (see Sabella et al., 2016) at each member institution as its designated innovative intervention. Therefore, at the kickoff of the new round of funding in 2019, the Alliance and its institutions were primed to undertake an organizational change process to scale-up.

Hence, this study presents findings geared towards helping alliances and interested URM STEM organizations conceptualize and respond to the circumstances that organizations face in adopting and integrating empirically backed initiatives. The guiding research question was: How do higher education leaders’ agency affect the launch and development of initiatives to support the success of URM STEM students?

Table 1. Study Participant Institutional Type

Partner Type

Alliance Membership

Individual Study Participants

2-year Public Institution

3

4

4-year Public Institution

6

10

4-year Private Institution

3

3

Industry Partner

2

3

Total

14

20

Literature Review

In researching an equity-centered STEM network, like LSAMP, it is essential to underscore that the Alliance is undertaking several innovations at once. LSAMP’s scale-up efforts are multi-layered and complex due to the varied changes they hope to make. The LSAMP initiative is designed to provide access and support to URM STEM students, which includes creating mentoring and research opportunities, as well as implementing pedagogical innovations in STEM classrooms. Essentially, this requires an analysis of several types of literature, as there is varied research in each of these areas, and different types of researchers (e.g., higher education vs. STEM researchers) will have different perspectives on how to approach questions about STEM reform (Henderson et al., 2011). The literature on inclusive practices in STEM (i.e., sensitivity training for faculty), mentoring and research, and instructional innovation can differ in their findings and recommendations; however, Kezar (2011) points to a handful of challenges that are present in many efforts to scale-up initiatives.

Context

Kezar (2011) argued that “scale assumes that a successful innovation is independent of the implementation setting” (p. 237). There is little consideration for changing the implementation context prior to the innovation. Furthermore, innovation is considered essential and beneficial for progress and will therefore be routinely adopted without first preparing the context through structural or cultural changes that will prepare the context for the innovation. Structural changes can include policies and funding; cultural changes involve a change in beliefs and norms. Adjusting some (e.g., policies) but not all (e.g., core beliefs and funding) of these contextual issues will inevitably lead to failed scale-up efforts (Hill, 2020). Contexts also change over time and must therefore be monitored carefully to be adjusted as needed (Kezar, 2011). Additionally, innovations should be altered to fit the context. For example, an innovation implemented by an office of one staff member versus an office of 20 will be different and have different support needs. Further, contextual changes rely on people making adjustments.

Leadership

Another challenge to innovation is engagement or ownership of the reform by leaders at all levels (Jones, 2016; Kezar, 2011). The central critique involving leadership is that innovation cannot be a top-down mandate but is most often successful as a bottom-up, or grassroots, effort that involves “deep engagement of people over time” (Kezar, 2011, p. 238). Yet, positional leaders must be interested and committed to adjusting their context and monitoring the institutionalization (Jones, 2016). Leaders with institutional authority can also act as institutional agents: they can “directly transmit, or negotiate the transmission of, highly valued resources” to students (Stanton-Salazar, 2011, p. 1067). Institutional agents are especially important for students who have minoritized identities that have limited ability to develop these social networks through their families. Moreover, Gomez and colleagues (2021) argued that institutional agents were conceptualized as figures that would alter students’ behavior, capital, and resources, rather than changing the institution itself. Indeed, Stanton-Salazar (2011) discussed modeling and teaching students appropriate networking behavior to empower them to navigate oppressive systems. Instead, Gomez and colleagues (2021) offered transformation leaders, who “have authority, and unlike institutional agents, use their authority to change the institution to better adapt to students’ needs” (p. 17).

Scale-up efforts are also more likely to be successful if they can engage with other networks of leaders who are similarly participating in reform, like the LSAMP Alliance (Jones, 2016). This network provides support and a sounding board if effectively organized and utilized. With an emphasis on the challenges of context and recommendations for leadership within scale-up endeavors, the remainder of this literature review will focus on the three types of innovations the LSAMP is implementing: supporting URM STEM students, instituting instructional innovation, and creating mentoring and research opportunities, as well as how leaders can leverage their position to transform their institutions.

Supporting URM STEM Students

Underrepresented racial and ethnic minorities (e.g., African American, Latinx, and Native American) face multiple racialized conditions in the STEM fields. These range from inhospitable environments created by faculty and students (Espinosa, 2011) to faculty mindset beliefs (Canning et al., 2019). Many of these circumstances are related to a STEM culture and ecosystem (Lord et al., 2019) that is White and masculine (Carlone & Johnson 2007; Cheryan & Markus, 2020). One way that STEM culture is entrenched in masculine norms is the competitive nature of introductory courses with the intention of weeding out weak students (Carlone & Johnson, 2007), often aimed at deterring students from staying in STEM. For example, Espinosa (2011) found that high institutional selectivity was a negative predictor of persistence among women of color in STEM. Espinosa (2011) attributed this to an “institutional culture that values research over teaching and actively discourages students from STEM through competitive grading practices” (p. 234). Additionally, women of color in STEM often face microaggressions in the classroom but also find that courses are misaligned and inaccessible to their professional “goals of contributing to society” (Espinosa, 2011, p. 234). Regrettably, these are the universities that are well-appointed in training and guiding students in STEM fields. To create learning environments that promote a welcoming atmosphere for women of color, Espinosa (2011) recommends academic peer relationships, research exposure, curricular reform, and improved pedagogical practices. This is precisely what LSAMP is undertaking; however, there are additional STEM practices that invalidate and exclude URM students.

One such harmful characteristic of STEM faculty is a fixed mindset. Canning et al. (2019) found that STEM “faculty mindset beliefs predicted student achievement and motivation above and beyond any other faculty characteristic, including their gender, race/ethnicity, age, teaching experience, or tenure status” (p. 1). Faculty who believed in fixed intelligence had greater racial achievement gaps among students in their classes. A fixed mindset aligns with the “weed out” culture that not everyone is cut out for STEM. Moreover, students expressed lower motivation to do their best work when taking a course with a faculty member with a fixed mindset and suggested that these faculty were less likely to design pedagogical conditions that focused on developmental learning (Canning et al., 2019). Faculty beliefs about intelligence matter most for student performance in a course. For this reason, Canning et al. (2019) recommend providing resources for faculty training on growth mindset rather than policies that place the responsibility of change on students. Excluding faculty equity trainings such as these places the onus on students to navigate an inequitable and unethical STEM culture.

Jones (2016) advocated that to overcome the obstacles faced by URM students in STEM, leaders must first have a thorough understanding of their universities’ environment. This goes beyond having a command of policies and structures to knowing how various people relate to each other and to their students. Our study highlights how leaders interested in STEM diversity and equity should also know how to be aware of how different people can access resources and influence their departments (Jones, 2016), as well as use their authority to transform their institution. This knowledge would in turn allow leaders to be strategic in navigating diversity initiatives at their institution, and arguably knowing how to adjust contexts to make them more receptive to innovations. Jones (2016) further contended that this is an issue of understanding the political terrain, which means knowing who is likely to challenge or support scale-up efforts, as well as knowing where the tensions in relationships occur. Leaders cannot avoid the discomfort of these political circumstances as it comes at the cost of URM student success. Networks can leverage various strategies, including addressing faculty beliefs (Canning et al., 2019) to support URM students, but the focus needs to be on classroom practices as well and cannot be narrowed to one department. Broadening leadership training across the university could be the difference between changing the organization versus changing the student behavior to adjust to the university and STEM culture (Gomez et al., 2021).

Instituting Instructional Innovation

The literature on supporting URM STEM students distinctly calls for instructional and pedagogical change for the benefit of underrepresented students (Colbeck, 2002). Generally, “recent decades have seen increasing calls for fundamental change in the teaching of science, technology, engineering, and mathematics (STEM)” (Henderson et al., 2011, p. 953). These reform appeals highlight the need to move from faculty-centered teaching to student-centered learning. Despite the call for change, courses, particularly introductory STEM courses that act as gatekeepers, continue to eschew active and engaged learning (Matz et al., 2018). The study led by Matz and colleagues (2018) highlighted that departments with a centralized and supportive structure were able to incorporate change to their gateway science courses, which included assessment changes (i.e., from multiple choice to constructed response) by adding support in the form of teaching and learning assistants, as well as hands-on activities. Changing pedagogical practices to focus on student-centered learning requires concentrating on policy and teacher reflection to change long-held beliefs about teaching and learning (Henderson et al., 2011). Yet again, these changes require leadership agency to address politically loaded collaborations (Jones, 2016).

Colbeck (2002) proposed utilizing a model of institutionalization that focuses on curriculum content and approach, as well as creating a more welcoming environment for URM students. Normative institutionalization in pedagogical reform requires demonstrated support from colleagues, department chairs, and deans. Yet, support is not enough, and regulative processes incorporate structure to that support. By rewarding—not penalizing—faculty for the effort and labor of engaging in instructional reform, professors are more likely to employ instructional innovation. These rewards should be present in what is traditionally considered the most uncompromising process: tenure and promotion (Gomez et al., 2021). Finally, cognitive institutionalization processes require that faculty fully embrace the principles of instructional reform (Colbeck, 2002). This model addresses structures, individual reflection and training, and policy changes, but all rely on leader agency to institute long-lasting changes.

Creating Mentoring and Research Opportunities

Another frequent exemplary practice for URM students is providing mentoring and research opportunities that connect students of color with faculty and peers in STEM activity. However, like instructional reform, there has also been an appeal to provide these opportunities for all STEM students. This broader request is largely due to the need to train students as researchers. Flowers (2020) described the research process—for example, exploring the scientific method and research ethics—as an art. However, providing mentoring and research opportunities for URM students has additional implications for the development of a science identity (Carlone & Johnson, 2007). Formal and informal science interactions and activity contribute to how one perceives their science identity and how others perceive them as scientists. Recognition as a scientist is “a key component of science identity development for women of color” (Carlone & Johnson, 2007, p. 1197). Each learning activity, like research, advances the development of a science identity, and Rodriguez and Lehman (2017) argued the importance to not only women of color in general but specifically at community colleges where they also create mentoring opportunities. However, the opportunities must come with reflection and training, not unlike teaching, to develop an awareness and value of intersectional identities (Rodriguez & Lehman 2017). This further underpins the connection between all the interventions in order to be successful.

According to the literature on supporting URM students, the ILSAMP and networks like it may be well-positioned to provide the best possible support to URM students in STEM. This can be accomplished through programmatic initiatives already in place: instructional innovations that include peer to peer academic relationships (i.e., learning assistants), research opportunities that can provide both exposure to research methods and opportunities for STEM mentoring, and a network of other leaders who are also engaging in scale-up initiatives (i.e., the Alliance). Yet, the looming consideration remains around whether the network (i.e., LSAMP) leadership, across all levels of involvement in the entity, has the agency to adjust the context (e.g., programs, policies, organizational culture) so these innovations can be successfully implemented and/or sustained. Specifically, our review accentuated and focused on the potential of whether the leadership of a network is prepared to be transformational leaders (Gomez et al., 2021). They can do this by making changes to entrenched cultural norms within each of their unique contexts by drawing from insights gleaned through their participation in a STEM education network.

Theoretical Framework

Through the lens of Curry’s (1992) organizational change, Kezar and Sam’s (2013) institutionalization theory, and Alexander’s (2005) institutional transformation and planning, our three-pronged framework provided the foundation to understand and theorize the steps needed for change within an organization. It also allowed us to understand how to make innovative ideas long-lasting through institutionalization while acknowledging the specific context of each organization and the positional structure embedded within the organization. Our study reiterated that context is important and there is no one-size-fits-all approach when it comes to innovation, even if multiple institutions or organizations are interested in implementing a similar program. Specifically, this framework further highlighted the important role institutional leaders play in institutionalization within their respective organizations to allow for change to occur.

We utilized Curry’s (1992) notion of organizational change and innovation to help contextualize the potential of a complex change process. Curry (1992) noted that organizational change often occurs in three steps: (a) mobilization, (b) implementation, and (c) institutionalization. Curry (1992) further argued that “innovations cannot become lasting without a rather significant role from leaders” (p. 5) and emphasized that leadership was not restricted to the highest-ranking officers of an institution. Rather, all members involved in the change process are integral to institutionalization and essential to understanding organizational change. This was important for our study because there are leaders within different positions and hierarchy (i.e., presidents, provosts, deans, coordinators, faculty) who each are a part of the Alliance, but their role varies significantly based on their respective campuses and job titles. Understanding how these leaders enact or support change at their institution or organization can be further understood through the change process. The first step, mobilization, involves increasing awareness and disseminating information to people around a certain cause or issue. Implementation, the second step, essentially begins to create structures to support the cause, issue, or reform, and can increase the number of those involved. Institutionalization, step three, is when the cause or issue becomes a normative feature of the organization (Curry, 1992). Ironically, when an innovation becomes institutionalized, it is no longer seen as innovative (Kezar & Sam, 2013). While Curry (1992) and Kezar and Sam (2013) articulated the significant role leaders have in this process, how they contribute is dependent on their own understanding of their capacity and agency. With our study’s focus on institutional leader agency, the steps of organizational change are important to acknowledge as they shape and influence perceived agency.

Curry’s (1992) steps of organizational change are essential to understand in relation to institutionalization theory. Kezar and Sam (2013) highlight institutionalization as one form of change and defined institutionalization as “a particular type of change that becomes sustainable and embedded into the fabric of the institution” (p. 59). This theory describes how practices, policies, or procedures are considered institutionalized once they are rooted in the culture of the organization. Institutionalization is a key factor for innovations continuing to exist and persist in organizations, and as mentioned, leaders have a significant role in this process. Institutionalization theory notes that leaders need to be aware of the stage that is occurring in order to know what strategy is best to use (Alexander, 2005; Curry, 1992; Kezar & Eckel, 2002). Our study utilized this theory to better understand how leaders within institutions in the Alliance were able or unable to institutionalize programming supported by NSF at their respective schools. In addition, we explored the leaders’ understanding of their roles and strategies available to them throughout the change process. As institutionalization is taking place to better support URM students, leaders’ awareness of this form of change impacts their agency and role in the change process.

Further, there is a need to understand how institutional transformation occurs and the inner workings of institutional design. Alexander (2005) defined institutional design as “the devising and realization of rules, procedures, and organizational structures that will enable and constrain behavior and action so as to accord with held values, achieve desired objectives, or execute given tasks” (p. 213). Alexander (2005) described three levels of institutional design as being applied to “whole societies or addresses significant macro-societal processes and institutions” (p. 214); meso-level as “planning and implementation structures and processes” (p. 214), such as physical planning; and the lowest level involving “intra-organizational design” (p. 215). Understanding at which level a leader is working is necessary in order to recognize how sustainable change is possible. Knowledge of institutional design is limited, though governance, coordination, and agency are three identified areas of knowledge that can help leaders utilize institutional design in their planning (Alexander, 2005). Although there is no prescriptive way of using institutional design, leaders need to be aware of this at their institutions to support a transformation that leads to institutionalization. Our study sought to further explore leaders’ perceptions of these dynamics at their institutions and how they used their agency to navigate transformation. Overall, this three-pronged theoretical framework informed how we conducted Braun and Clarke’s (2006) six-phased thematic analysis by using terms and concepts as starting points to review the data and create codes and themes, ultimately interpreting the findings and implications moving forward.

Research Design

Study Context and Design

This study was part of a larger ongoing constructivist (Wortham & Jackson, 2008), qualitative multi-site, multi-case study (Yin, 2014) conducted by the ILSAMP research team. Constructivism is the idea that knowledge construction incorporates building upon previous understandings and experiences in combination with new information being presented (Wortham & Jackson, 2008). This was important for our study because we were looking to better understand how institutional leaders understand their role and ability to incorporate change based on their evolving knowledge base. A case study was selected to reflect that each institution in the Alliance makes up a site and each leader makes up a particular case. The variability of the participant institutions and leaders added a comparative lens to our study’s inquiry. The boundaries of the case study included institutional leaders who worked within the Alliance during fall of 2020. The phenomenon of interest was the institutional leaders’ perceptions of their agency and their organization’s efforts to engage in the Alliance and support URM STEM student success through the various initiatives. The research question created from this phenomenon of interest was: How do higher education leaders’ agency affect the launch and development of initiatives to support the success of URM STEM students?

Sample

Our sampling strategy, informed by our theoretical framework, focused on what we describe as “institutional leaders.” Institutional leaders range from presidents to site coordinators from each of the organizations within the Alliance. To be considered for our study, institutional leaders needed to be employed at their respective institution or organization within the Alliance and be involved in some capacity with supporting ILSAMP programming. Twenty institutional leaders participated in the study. The research team recruited participants via virtual newsletters in the fall of 2020, with each newsletter targeting a specific level of institutional leaders (i.e., coordinator, provost, president). A total of 43 people were initially contacted to participate, including site coordinators, provosts, presidents, and additional leaders who were referred to the research team. See Table 2 for more information on the participants. Additionally, Table 1 breaks down participants by institutional type and institutional role to show the range of institutional type and leader type that was included in the study.

Data Collection

After receiving institutional review board approval, members of the ILSAMP research team conducted semi-structured interviews via Zoom with participants in the fall of 2020. These interviews lasted, on average, 50 minutes and were conducted by various members of the ILSAMP research team. Each participant selected a pseudonym at the beginning of their interview in an effort to decrease participant identification. The theoretical framework focused on institutionalizing organizational change and institutional transformation. This influenced the interview protocol by focusing on the institutional leaders’ experiences to better understand their position in enacting change. Specifically, the interview protocol included questions addressing their role on campus, what student support looks like on their campus, what faculty/staff support looks like on their campus, provost/president/industry partner-specific questions, program development, and NSF funding. While interviews’ length varied by participant, each interview consisted of issues related to institutional leadership and working with the Alliance and institutional support of URM STEM students. After the interviews were completed, they were transcribed by a professional transcription service, then cross-checked by Author 1 for accuracy. After transcription and review, the interviews were uploaded to Dedoose, a qualitative analysis software.

Analysis

This study utilized Braun and Clarke’s (2006) six-phased thematic analysis to identify patterns of meaning within the data that respond to the research question. Braun and Clarke identify these six steps as (a) familiarizing yourself with your data, (b) generating initial codes, (c) searching for themes, (d) reviewing themes, (e) defining and naming themes, and (f) producing the report. Step one, familiarizing yourself with your data, included assigning multiple transcripts to each author for initial review. Transcripts were divided evenly among authors, with each transcript having two authors assigned to review. Each author read over the assigned transcript before coding to understand the content as well as any initial connections between the transcripts. Step two, generating initial codes, took place in three parts. The first included each individual author reading over the theoretical framework and generating a list of potential codes from these concepts. The second part of step two took place at a research team meeting where each author discussed the list of potential codes they created, and the group discussed which codes presented made sense in the context of the research question. The final part of step two took place when Author 1 reviewed meeting notes and finalized the first set of initial codes. Examples of initial codes included sustainability, culture and values, and initiating change. The third step, searching for themes, took place at a research team meeting after coding was completed by each author individually. The fourth step, reviewing themes, took place when each author created a list of themes they identified and shared with the research team to review on an online group communication platform (i.e., Twist). After individual review, the authors met to discuss overlap and ultimately create and define final themes that addressed the research question—step five, defining and naming themes. The final step, producing the report, was organized by Author 1. Each author was assigned individual sections of the report to write and cross-edit other sections to best represent the story being told by the data in relation to the research question.

Table 2. Study Participant Institutional Role

Institutional Role

Participants

Percent of Sample (%)

Administrator*

9

45

Faculty

7

35

Staff

4

20

Total

20

100

Note. *Institutional roles such as Dean, Provost, Vice President, President, or other significant leadership positions not already included in the staff participant count.

In summary, steps one through three focused on reading transcripts, creating a codebook, and the first level of coding. We followed a hybrid inductive/deductive approach initiated by codes pulled from our theoretical framework and then added new codes during first cycle coding. Some of the initially developed codes were “culture and values,” “organizational structures,” and “institutional boundaries.” Each transcript was coded at least twice in the first cycle. At the end of the first cycle of coding, the research team met to condense the generated codes to move forward to the second level of coding, which entailed finding persistent patterns among the codes to be used as preliminary themes. After the research team met and defined the themes that were present, we created a storyline to showcase the connections and tensions among the themes in connection to our research question.

Positionality

It is important to acknowledge the positionality of the research team because positionality influences every stage of a research project, including project design and analysis (Holmes, 2020; Tracy, 2010). The research team was made up of six scholars, with four identifying as cisgender women (Authors 2, 3, 4, and 6) and two identifying as cisgender men (Authors 1 and 5). Authors 4 and 5 identify as Black, Authors 1 and 2 identify as White, Author 3 identifies as Latina, and Author 6 identifies as Middle Eastern/North African. Authors 2, 3, 4, 5, and 6 identify as heterosexual and Author 1 identifies as gay/queer. The team has a range of experience of working with institutional leaders, from serving as faculty members, teaching instructors, administrators, and student affairs professionals. These experiences influenced how we understood and made meaning of the reflections the participants shared with the research team. In terms of the team’s connection to ILSAMP, it is important to note that the entire research team contributes to research concerning the Alliance, with a range of roles from volunteers to co-principal investigator. This informed our research design decisions in various ways, including a pre-existing level of trust with the participants to share their experiences, frustrations, successes, etc. The combination of our identities and shared experiences and the context we were situated in, allowed us to cross-examine our understanding and reflections on the institutional leaders’ narratives in relation to the theoretical framework. This contributed to our overall understanding and interpretation of the findings.

Trustworthiness and Limitations

Trustworthiness was addressed through collective engagement with data, such as reading, revising, and coding data twice. Each source of data was reviewed by more than one author in hopes to capture different viewpoints and not limit our analysis to one author’s perception. Braun and Clark (2006) provide a checklist on “good thematic analysis” (p. 79). This checklist was used to maintain alignment between our practice and research design. Periodic team meetings were held to confirm a mutual, cohesive analysis of data (Tracy, 2010). Team meetings also served as an opportunity to present disagreements about how data was interpreted and captured any tension in analysis through notes collected by Authors 1 and 5.

Limitations of our study include a smaller participant sample and the context of data collection (i.e., COVID-19 pandemic, racial injustice perpetuated by state sanctioned actors). The COVID-19 pandemic impacted our study by how data were collected and the experiences of the institutional leaders that were being interviewed. The virtual setting rendered necessary by the pandemic changed or impacted how institutional leaders have previously engaged ILSAMP, therefore impacting their agency to enact change and sustain change. It is also important to acknowledge the racial injustice that was taking place at the time of our study which impacted institutional leaders personally and professionally. Many of the institutional leaders communicated the need they felt to address what was happening in real time to the students they worked with and how that was or was not prioritized within their institution. A larger sample that incorporated every institutional leader within the Alliance could provide a more expansive understanding of their perceived agency within the organization, as well as produce a better comparison of how agency to enact change differs based on position type and even institutional comparison.

Findings

Study findings suggest that leadership commitments are insufficient to materialize the scale-up of new STEM initiatives, despite institutions being primed for organizational change as a result of their connection to the ILSAMP Alliance. However, even as leaders shared their own dedication to change, they also underscore how their institutions are limited in their capacity to scale STEM activities and how their role within the institution can impact scalability. Adding depth and complexity to this issue, the following three identified themes exhibit the structural shortfalls that negatively impact institutional support efforts for URM STEM students within the Alliance: (a) limited communication prevents opportunity, (b) organizational structures constrain agency, and (c) resources and leadership impact innovation and development. In this section we discuss each finding in relation to Curry’s (1992) notion of organizational change.

Limited Communication Prevents Opportunity

Communication prevents mobilization; in other words, there is a lack of organized information dissemination and connection with the purpose of building a strong Alliance program. Collaborative, participatory administration, both within the academic institution and among the Alliance institutions, was a need institutional leaders thought necessary to the success of institutional efforts for change. A constant thread heard throughout the interviews was the desire for there to be more unity amongst the Alliance members. Leaders wanted to hear what did and did not work when trying to institutionalize programs. Mary, a senior administrator at a four-year public institution, commented, “We’re just starting. I need to know . . . what have other institutions done to become sustainable?” and followed with the suggestion of the need for a central place where information on what everyone is doing could be housed. Peter, a senior administrator at a four-year public institution, took the desire for more unity a step further by suggesting the Alliance take the lead on building support to help URM students:

I do think the Alliance is a good program. I think it’s way too small for the need. That would be one comment. That’s what we’re doing with our different scholarships, is we’re saying, all these federal programs and foundation programs just aren’t enough. We need to coalesce some support if we’re really going to have the kind of impact on the underrepresented student population that we want to have. That’s in our DNA. That’s part of our mission.

Peter’s comment indicates the Alliance lacks the capacity and capital to institutionalize support for URM students. Furthermore, the data suggested responsibility for fostering a sense of community fell mostly on faculty and staff likely due to their proximity to students compounding the challenges with capacity and capital.

More support could be provided amongst the institutions in the Alliance. A member for over 20 years, Mister, a senior administrator at a four-year public institution, expressed the need for more support from partner institutions after noticing a recent decline in activities and less awareness of historical memory of past programming as staff, faculty, and administrators turn over for various reasons. Mary pushed for further transparent communication among the Alliance to start opening discussions on “how to best sustain” the programs that support URM STEM students across the Alliance. Taylor, a faculty member at a two-year public institution, warned against the lack of coordination and faculty’s agency and involvement in decision making, noting:

I’m already concerned that I’m not going to be able to spend the money because I actually found out I was the PI [principal investigator] of this grant after it had been received. I wasn’t involved in the budgeting. I wasn’t involved in anything . . .

This statement shows how the lack of transparent communication between institutional leaders created a certain level of hesitancy and uncertainty, if not professional anxiety, about their institutional duties. This dissonance, according to the institutional leaders, harms not only institutional effectiveness but also cracks its leadership and hinders sustainable change. In the current structure of the Alliance, limited communication is normalized. This ultimately contributes to constraining opportunity when it comes to cross-campus collaborations, sharing of best practices, and ability to pivot programming based on world events (e.g., the COVID-19 pandemic).

Organizational Structures Constrain Agency

Implementation cannot occur unless structures are put in place to support the issue, cause, and program of interest (Curry, 1992). Within the Alliance, organizational structures and boundaries have the potential to support individual agency. The ability to navigate existing institutional structures greatly influences the potential for implementing change or innovations. Within the Alliance, the interviews highlighted how the agency leaders felt in their role depended largely on where they were situated within the institution and their role (e.g., faculty, staff, or administrator). As financial decision-makers, senior-level leaders experienced more autonomy to allocate resources in annual funding provided to STEM programs. Conversely, faculty and staff often found institutional bureaucracy a boundary when seeking support for new programming, especially when the programming required funding, as explained by Laila, a faculty member at a two-year public institution:

If they do have to pay for it, then I think those kinds of things probably have to be vetted all the way up through our president . . . I wrote a grant for NSF and that was a nightmare in terms of just getting them to sign off on it.

There were instances where leaders on all levels conveyed interest in implementing new programs only to be hindered by external forces. “We are not allowed to add any new courses or any new programs without approval from them [state accrediting board],” noted Taylor, a faculty member at a two-year public institution. In this instance, the institution planned to move ahead with adding a degree program where the courses were already planned.

Such structures highlight the inefficiencies the leaders experienced in their institutions’ efforts of support to URM students, which frequently made those institutional attempts unsustainable. Peter, a senior-level administrator at a four-year private institution, acknowledged that historically “our programs for underrepresented students in research, and in academic advancement, and in leadership have suffered a little bit from neglect because we didn’t make it a priority.” For Rose, a faculty member at a four-year private institution, an absence of institutional support was “one main reason” for ending efforts or initiatives that might have served URM STEM students. In the end, Rose was doubtful that their institution prioritized creating opportunities to support URM STEM students. When asked whether financial resources would be available for launching a new program for serving students from underrepresented, minoritized backgrounds, Rose satirically answered that “it depends on the buy-in from the university.”

Additionally, some organizational structures prevented efforts from being “institutionalized.” Mostly efforts were “individualized” as they relied on faculty and staff to create community. According to Claire, a senior administrator at a four-year public institution, this negatively impacted the sustainability of the institutional efforts to support its own URM students. These “individualized approaches” were found to be unsustainable because they were created by an individual for an individual. These examples are emblematic of the institutional hurdles faculty and staff face compared to those serving in senior administrative roles. While leaders are navigating structures and boundaries at their institutions, they are also navigating structures and boundaries within the Alliance.

Resources and Leadership Impact Innovation and Development

Institutionalization is impacted due to access to resources or decision-making concerning resources. Lack of financial resources constrains the current STEM program’s ability to expand or change trajectory to better support URM STEM students’ needs. Mister, a senior administrator at a four-year public institution, for example, elaborated that their institution is on an endless hunt for “other pots of money” so that STEM programs and other initiatives that serve URM undergraduate students can be kept in place. Nate, a senior administrator at a four-year public institution, explained how their institution’s “slow reaction to the changing demographics of the effects of technology” limits their university’s ability to keep up with world changes. Both examples illustrate how resources (e.g., financial, time) impact leaders’ ability to expand programming within their roles.

Moreover, leadership plays a significant role in program innovation, sustainability, and development. While many institutional leaders felt positive about their individual ability to enact change, some expressed discontent at the lack of support experienced institutionally. For example, one senior administrator focused on promoting innovative practices and increasing student retention for URM students. Another leader, Mary, a senior administrator at a four-year public institution, noted the lack of key leaders advocating for STEM. Laila discussed leaders’ misunderstanding of what it takes to teach science education. Senior institutional leadership involvement and understanding of the URM STEM student population influences programming and support.

Sally, a senior administrator at a four-year public institution, underscored that appointing someone “who doesn’t really know much about the program, doesn’t know the access, what it can help provide, and may not even know who the minorities or the African American minority students are on their own campus” has had a negative impact on the sustainability of the changes and/or initiative programs that the top administration introduces in support of the URM STEM students. Sally posed a broader, yet basic, question about the reasons for making these appointments that often result in inconsistency of practice and/or necessitate prolonged trainings on equitable approaches to support URM student success.

Peter, a senior administrator at a four-year public institution, felt that URM programs have not been a priority for his institution, as mentioned in the previous theme. Peter further elaborated:

The biggest challenge that our institution faces is culture change. We have a very diverse faculty, but what I mean by that is we have faculty from 30 different countries. We have faculty from all walks of life in America. and I think the problem with a really pluralistic faculty culture is not all of them understand the importance of racism in America. And as a result, many of them are not aware that there’s a lot of systematic racism in America that’s really held down the ability of people of color to advance. So, I think we have a culture change challenge. We have to educate our faculty in a way that they will receive the information and embrace it, and then begin to change the culture themselves, so that this becomes a priority for all of our faculty.

How resources and leadership are positioned at institutions ultimately can constrain or expand support for URM STEM students. While individual institutions’ support, or lack thereof, seems specific to each campus, this theme showcases how leaders who seek institutionalization of innovative and sustainable ideas are navigating additional obstacles in their role. These obstacles (e.g., resources) may be out of their control or access and can limit what they can and cannot do to better support URM STEM students.

Summary of Findings

Organizational structures, limited resources, and lack of communication restrict opportunities to institutionalize STEM best practices. The ILSAMP is contending with issues that are embedded either in structural ways or through leadership assumptions about prioritizing URM students. However, there is also a real commitment to URM students that is often described as engrained in cultural norms, or in their mission or DNA. Despite the challenges to the change process (i.e., implementation, institutionalization, mobilization) many administrators, faculty, and staff are working hard to find resources, communicate better, and scale up best practices, but encounter obstacles at varying levels of leadership.

Discussion

Recognizing organizational change and innovation is a complex process (Curry, 1992), and our study’s findings indicate that institutional leaders perceive their leadership as one of the critical levers to institutionalize organizational change and innovation. However, organizational structures serve as inhibitors to institutionalization and sustainability (Kezar, 2011), which results in additional barriers to support student success. Moreover, findings suggest limited change has occurred to the core structures and culture of the institutions to implement innovative interventions.

The contribution of this study relates to the illumination of the tension between institutional leaders’ agency to make change sustainable versus structural and leadership networks inhibiting STEM success efforts. Leaders’ efforts were focused on implementing the innovation itself—which led to individualization rather than institutionalization—rather than priming the environment for change. Hence, we discuss and theorize the need for a revised role of institutional leaders utilizing their agency to cultivate organizational changes to support the success of URM STEM students (Alexander, 2005; Curry, 1992; Kezar & Sam, 2013).

The study is in alignment with Jones (2016), advocating that leaders should have a thorough understanding of their university’s environment to overcome the obstacles faced by URM students in STEM. We were able to see that leaders in the Alliance had a clear understanding of the structural and political structure of their institutions. Where they could, participant leaders enact change within the power allowed by their respective roles. There was a clear delineation between senior administrators who could support overarching goals with large amounts of funding and the faculty and staff who instituted low-cost initiatives within their programs. Findings also showed leaders interested in STEM diversity and equity knew how to access resources and influence their departments (Jones, 2016), which sometimes came into conflict when trying to engage leaders outside of the program. Additionally, the leaders’ understanding of the university environment could play a larger role within the Alliance and relates to Jones’ (2016) contention that this is an issue of understanding the political terrain. There is an underdeveloped opportunity for senior administrators to use their experience to help faculty and staff navigate the upper echelons of their respective institutions for more support and funding.

Generally, institutions across the Alliance shared understanding and commitment for the success of URM STEM students. Efforts, both individual and institutional, to support the students have been extensively implemented. Evidence can be found in the programs developed due to the COVID-19 pandemic, such as student accommodations and partnerships with local government agencies. In these instances, the leaders received support from colleagues, department chairs, and deans of institutionalized pedagogical reform (Colbeck, 2002). It remains to be seen what, if any, of those reforms will remain and if the leaders will continue to receive the same level of support.

Yet, looking closely at the underlying tensions within the organizational structures of the participant Alliance institutions, we could see persistent organizational issues that restrain and sometimes nullify institutional efforts to scaffold URM students to success. As elaborated in the Findings section, lack of financial resources, lack of communication within institutions and among the Alliance, and inconsistencies in sustaining efforts, are sources of some of the tensions experienced and reported by the institutional leaders. Further, it was argued that the lack of diversity training among faculty could contribute to the unsustainability of change. Canning et al. (2019) explain that this could be due to students not seeing themselves represented in the faculty as well as the latter not fully understanding the social and economic challenges faced by URM students.

For Alexander (2005), institutional leaders’ awareness and practice of active agency and coordination scaffold their institutional planning, thereby institutionalizing change and making it sustainable. However, ambiguous and inconsistent engagement of faculty and administrators in the change processes was another prominent structural flaw the leaders reported as a source of institutional inefficiency that often leads to ineffective outcomes in supporting URM students. Alexander (2005) further posited that for institutional design to be effective and efficient, engaging leaders of varying levels is crucial. At the highest of levels, the strategic plan takes place, the meso-level “involves the institutional design of planning and implementation structures and processes” (Alexander, 2005, p. 214), and the lowest levels encompass “processes and interactions” (p. 215). While institutional leaders in this study demonstrated a strong commitment to the Alliance’s success, leadership at other levels struggled to change cultural norms and structures that prevent institutionalization.

Implications

Implications for Practice

Based on the study’s findings, we offer some implications for improved practice which the Alliance partnered institutions can adopt and implement to better support URM STEM students. First, we suggest that there be a shared network of effective leadership practices among the Alliance’s institutions that can guide universities and new leaders through unpredictable activity (e.g., taking on a new grant as described by Taylor or difficulty finding support for a grant as described Laila). This shared space should offer leadership practices and decision-making mechanisms that leaders can consult with and tailor to their own institutions when needed. In realizing the need for this shared access to institutional practices that has been proven effective, Mary, for example, wished that “there will be kind of, a central place where all this information will be located so I can go and see it and learn from what other people are doing.”

Consistency and unity of effort are vital for the sustainability of institutional support for URM STEM students. Some participants openly commented that individual efforts, if not with structure or backed with institutional support, cannot be sustained. Therefore, they would be short-lived with unreliable outcomes. Also, echoing the recommendation made by Canning et al. (2019) about faculty training for “growth mindset” (p. 1), we believe that such trainings are vitally needed to enrich faculties’ understanding of the racial, historical, and socioeconomic intricacies involved in and crucial to supporting URM students. Yet, important to address is the financial demands those trainings would impose on the Alliance institutions: who would oversee funding those trainings, supervising them, and ensuring their effectiveness and sustainability in enriching the faculty and staff’s knowledge and practices. One way to address the need for trainings and coordination across Alliance institutions is to assign these important tasks to a communications coordinator at each institution. The current Alliance leaders are focused on the delivery of the innovation and not the preparation of the context, which has implications for the scale-up effort itself. Communication between institutions and within the institution surfaced as a critical issue in preparing the context for innovation in terms of both structural and cultural changes needed to implement the innovation successfully.

Implications for Future Research

Our study confirmed our guiding proposition that scale-up efforts require a focus on leadership across all levels. Further examination is needed to investigate the roles power and politics play in sustaining institutional change, especially from the differing perspectives of administrators, faculty, and staff. Curry (1992) emphasized that leadership was not restricted to the highest-ranking officers and that innovations cannot become “lasting without a rather significant role from leaders” (p. 6). This could provide additional insight into what institutionalization would resemble across functions, departments, and programs in postsecondary institutions. Continuing this study with participatory action research would contribute to evaluating the level of institutionalization achieved by the Alliance.

Additionally, we believe it is worth exploring the nuances of how institution type impacts institutionalization. As seen in the Alliance, implementing change was approached differently depending on if the institution was public, private, two-year, or four-year. Student population also provided additional nuances on which and how programs were implemented.

Conclusion

Institutionalizing STEM success for URM students is a complex process that requires much more than temporary funding and sporadic leadership support. Individualized efforts are bound to be erratic, benefiting small numbers of students and providing inconsistent experiences until there is alignment across all levels of leadership. However, the reality for many of the institutions in this Alliance is that they are highly dependent on grant, federal, and state funding to maintain their support of URM students. Nevertheless, institutionalization requires institutional commitment, which may be elusive given “there is no real pressure (from peers) to achieve genuine progress” (López et al., 2022, p. 11). When other institutions do not achieve this success, what is the incentive to make changes to contexts that lead to meaningful and lasting change?

Disclosure Statement

This work was supported by the National Science Foundation (NSF) award number HRD-1911341.

Conflict of Interest Statement

On behalf of all authors, the corresponding author states that there is no conflict of interest.

References

Alexander, E. R. (2005). Institutional transformation and planning: From institutionalization theory to institutional design. Planning Theory 4(3), 209–223. https://doi.org/10.1177/1473095205058494

Baber, L. D., & Jackson, J. (2018). From the edge of success to the center: Development of LSAMP alliances, 1987–2017. IINSPIRE LSAMP Research Brief 2.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Canning, E. A., Muenks, K., Green, D. J., & Murphy, M. C. (2019). STEM faculty who believe ability is fixed have larger racial achievement gaps and inspire less student motivation in their classes. Science Advances, 5(2). https://doi.org/10.1126/sciadv.aau4734

Carlone, H. B., & Johnson, A. (2007). Understanding the science experiences of successful women of color: Science identity as an analytic lens. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 44(8), 1187–1218. https://doi.org/10.1002/tea.20237

Cheryan, S., & Markus, H. R. (2020). Masculine defaults: Identifying and mitigating hidden cultural biases. Psychological Review, 127(6), 1022–1052. https://doi.org/10.1037/rev0000209

Clewell, B. C., de Cohen, C. C., Tsui, L., Forcier, L., Gao, E., Young, N., Deterding, N., & West, C. (2006). Final report on the evaluation of the National Science Foundation Louis Stokes alliances for minority participation program: Full technical report and appendices. Program for Evaluation and Equity Research (PEER), The Urban Institute.

Colbeck, C. L. (2002). Assessing institutionalization of curricular and pedagogical reforms. Research in Higher Education, 43(4), 397–421. https://doi.org/10.1023/A:1015594432215

Curry, B. K. (1992). Instituting enduring innovations: Achieving continuity of change in higher education. ASHE-ERIC Higher Education Report 7. The George Washington University.

Espinosa, L. (2011). Pipelines and pathways: Women of color in undergraduate STEM majors and the college experiences that contribute to persistence. Harvard Educational Review, 81(2), 209–241. https://doi.org/10.17763/haer.81.2.92315ww157656k3u

Flowers, L. O. (2020). CURES and increasing participation in STEM. International Education and Research Journal, 6(4), 29–31.

Gomez, A. K., Cobian, K. P., & Hurtado, S. (2021). The role of STEM program directors in broadening the impact of STEM interventions. Education Sciences, 11(11), 742. https://doi.org/10.3390/educsci11110742

Henderson, C., Beach, A., & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48(8), 952–984. https://doi.org/10.1002/tea.20439

Hill, L. B. (2020). Understanding the impact of a multi-institutional STEM reform network through key boundary-spanning individuals. The Journal of Higher Education, 91(3), 455–482. https://doi.org/10.1080/00221546.2019.1650581

Holmes, A. G. D. (2020). Researcher positionality—A consideration of its influence and place in qualitative research—A new researcher guide. Shanlax International Journal of Education, 8(4), 1–10. https://doi.org/10.34293/education.v8i4.3232

Jones, S. (2016). More than an intervention: Strategies for increasing diversity and inclusion in STEM. Journal for Multicultural Education, 10(2), 234–246. https://doi.org/10.1108/JME-12-2015-0046

Kezar, A. (2011). What is the best way to achieve broader reach of improved practices in higher education? Innovative Higher Education, 36(4), 235–247. https://doi.org/10.1007/s10755-011-9174-z

Kezar, A., & Eckel, P. D. (2002). The effect of institutional culture on change strategies in higher education: Universal principles or culturally responsive concepts? The Journal of Higher Education, 73(4), 435–460. https://doi.org/10.1080/00221546.2002.11777159

Kezar, A., & Sam., C. (2013). Institutionalizing equitable policies and practices for contingent faculty. The Journal of Higher Education, 84(1), 56–87. https://doi.org/10.1353/jhe.2013.0002

Lord, S. M., Ohland, M. W., Layton, R. A., & Camacho, M. M. (2019). Beyond pipeline and pathways: Ecosystem metrics. Journal of Engineering Education, 108(1), 32–56. https://doi.org/10.1002/jee.20250

López, N., Morgan, D. L., Hutchings, Q. R., & Davis, K. (2022). Revisiting critical STEM interventions: A literature review of STEM organizational learning. International Journal of STEM Education, 9(1), 1–4. https://doi.org/10.1186/s40594-022-00357-9

Matz, R. L., Fata-Hartley, C. L., Posey, L. A., Laverty, J. T., Underwood, S. M., Carmel, J. H., Herrington, D. G., Stowe, R. L., Caballero, M. D., Ebert-May, D., & Cooper, M. M. (2018). Evaluating the extent of a large-scale transformation in gateway science courses. Science Advances, 4(10). https://doi.org/10.1126/sciadv.aau0554

National Center for Education Evaluation and Regional Assistance. (2018). What works in STEM education: Resources for national STEM day, 2018. https://ies.ed.gov/blogs/ncee/post/what-works-in-stem-education-resources-for-national-stem-day-2018

Rodriguez, S. L., & Lehman, K. (2017). Developing the next generation of diverse computer scientists: The need for enhanced, intersectional computing identity theory. Computer Science Education, 27(3–4), 229–247. https://doi.org/10.1080/08993408.2018.1457899

Sabella, M. S., Van Duzor, A. G., & Davenport, F. (2016). Leveraging the expertise of the urban STEM student in developing an effective LA program: LA and instructor partnerships. 2016 Physics Education Research Conference Proceedings (pp. 288–291). National Science Foundation.

Stanton-Salazar, R. D. (2011). A social capital framework for the study of institutional agents and their role in the empowerment of low-status students and youth. Youth & Society, 43(3), 1066–1109. https://doi.org/10.1177/0044118X10382877

Tracy, S. J. (2010). Qualitative quality: Eight ‘big-tent’ criteria for excellent qualitative research. Qualitative Inquiry, 16(10), 837–851. https://doi.org/10.1177/1077800410383121

Wortham, S., & Jackson, K. (2008). Educational constructionism. In J. A. Holstein & J. F. Gubrium (Eds.), Handbook of constructionist research (pp. 107–128). The Guilford Press.

Yin, R. K. (2014). Case study research design and methods (5th ed). Sage Publications.