The Student Inclusiveness Scale: An Initial Examination of Factor Structure, Reliability, and Group Differences
Abstract
The purpose of the current study was to pilot a decentralized inclusiveness and belonging scale with university students in a large psychology department. We assessed the scale’s factor structure and reliability using results from 650 bachelors, masters, and doctoral students. Exploratory factor analysis revealed four cohesive factors within the scale, each of which contributed meaningfully to the variance of the scale as a whole. The factors were: general feelings of inclusiveness in the department, feelings of inclusiveness from peers, feelings of inclusiveness from faculty, and campus engagement encouragement. Internal consistency reliability was good for each factor. There were differences in the favorability ratings of the subscales (e.g., undergraduate versus graduate students, White versus BIPOC students) suggesting that the scale was sensitive enough to detect important group differences. The results provided preliminary evidence that The Student Inclusiveness Scale is a quality and useful scale. It provided valuable feedback to the department about students’ feelings of inclusiveness and belonging in general, in relation to faculty, and in relation to peers.
Keywords: diversity inclusion; higher education; student perspectives of inclusiveness; student belonging
Introduction
Human diversity takes many forms, including but not limited to race, ethnicity, socioeconomic status, religion/spirituality, sex, sexual orientation, education, gender identity, language, nationality, life-cycle stages, and ability (American Psychological Association, 2002; Smith, 2020). In this sense, multiculturalism can be considered fluid, constantly evolving, a key component of personal identity development, and a way in which we make sense of our world (American Psychiatric Association, 2013). These traits are significant for individuals, groups, institutions, and society, especially given the rapidly growing cultural diversity in the United States (Krogstad et al., 2021; Smith, 2020).
In higher education, the saliency of multicultural consideration in education, advising, and training is crucial as educational settings have similarly become more diversified in the past decades (National Center for Education Statistics, 2019). Despite this growing diversification, members of marginalized groups continue to be underrepresented and under-supported in academic institutions. These individuals can be the target of both subtle and overt acts of discrimination, face negative stereotypes about their abilities or language barriers, and experience a lack of inclusion by their instructors and peers (Cheryan et al., 2009; Ifedi, 2017; Rogers et al., 2019; Spencer & Castano, 2007; Wiggan, 2007). With multiculturalism on the rise, society places an onus on institutions to initiate inclusion, articulate and defend the myriad benefits of diversity, and create a safe campus climate for students of all backgrounds. Diversity is an imperative that must be embraced for colleges and universities to remain interconnected with the world and fulfill the mission of institutional excellence.
Marginalized Experiences in Higher Education
The mismatch between university norms and culture-specific norms frequently can lead to increased stress and negative emotions for minority group students (Stephens et al., 2012). Not surprisingly, students from underrepresented and international groups reported a lesser sense of belonging compared to students of majority group status, as well as mixed and unsuccessful attempts at social integration (Booker & Campbell-Whatley, 2019; Griffin et al., 2016; Summers & Volet, 2008). Such students may be even excluded from student study groups or bonding events outside of class, potentially leading to psychological disengagement, poorer educational outcomes, feelings of “invisibility”, and increased vulnerability for school drop-out (Sanchez et al., 2018; Slavin, 1990; Spencer & Castano, 2007). In addition to these risks, students from underrepresented backgrounds can experience significant concern of stereotype threat, or the anxiety of “confirming a negative stereotype associated with their ingroup” if failing to perform strongly in a course (Spencer & Castano, 2007, p. 418). College students of ethnic minorities in the United States also tend to report diminished beliefs in their abilities to succeed independently, notably higher emotional distress, and increased experiences of targeted microaggressions or discriminatory comments (Clark et al., 2012; Hurtado et al., 2008; Ifedi, 2017).
Overall, research on higher education suggests that immersing people in culturally diverse settings enhances cultural competency, understanding of other groups, perspective taking, and relatedly, creativity; however, simply putting diverse people together does not automatically lead to positive outcomes for all, especially cultural minorities (Bai et al., 2020; Hoever et al., 2012; Smith, 2020). These widely varied experiences of student success, psychological wellbeing, and inclusion across cultural groups demand action and support from higher educational institutions. Greater support and inclusion of underrepresented and minority groups helps all students, faculty, and administration thrive. To accomplish this, “we must approach the diversity implementation process with a focus on real change, results, and impact” (Williams, 2008, p. 29).
Marginalization during COVID-19
The COVID-19 worldwide pandemic, which started in early 2020, brought many changes to the higher education community for faculty and students alike. Faculty had to quickly, often with minimal training, pivot to online instruction and navigate online learning platforms that were unfamiliar to many. Students too experienced a shift to online instruction, but they also were frequently affected by financial and/or residential changes that affected many aspects of their lives.
Studies conducted during the pandemic showed that members of racial/ethnic minority groups, compared to their non-Hispanic white counterparts, experienced increased discrimination and experiences with adverse socioeconomic outcomes (Strassle et al., 2022; Zhang & Kolady, 2022). Oh and colleagues (2021) also demonstrated that experiencing increased amounts of discrimination during the pandemic contributed to higher rates of moderate to severe depression in college students. Further, college students experienced increased anxiety when instruction was shifted online, and many reported increased stress, isolation, and solitary time during the months following the online shift (Browning et al., 2021). Those at highest risk for emotional distress were female, those with eight or more daily hours of screentime, and those of fair/poor general health (Browning et al., 2021). Jehi et al. (2023) also found that females were at greater risk for anxiety, in addition to those who experienced financial hardship, worked full-time, quarantined in isolation, and had concerns about the future. Here is one poignant quote from a student being interviewed for a qualitative study by Hagedorn and colleagues (2022), “‘I had to purchase a computer when classes went online and libraries closed. I am a single mom with very little income so this was a HUGE blow to my budget and minuscule savings'” (p. 1077).
In addition to females, another societal group that showed increased stress and anxiety in the early days of the pandemic were those who identify with transgender, gender diverse, and sexual minority groups. Their self-reports of stress and anxiety were higher than their cisgender, heterosexual peers (Hoyt et al., 2021). The stress and anxiety in this group remained higher than cisgender, heterosexual peers, and actually increased, in the months after the onset of the pandemic (Hoyt et al., 2021). Interestingly, Hoyt et al.’s second data collection, in summer 2020, showed increased levels of stress and anxiety, compared to the first data collection, in Black and multiracial students. If members of these groups were more greatly influenced socioeconomically than other groups, then it is not surprising that these groups showed more stress and anxiety in the months following the pandemic. For reasons such as these mentioned here, the need for institutions to understand and address the needs of different groups is heightened.
Diversity Support Responses and Shortcomings
In general society, there appear to be dramatically different perspectives on methods of reducing cultural inequalities across settings (Horowitz et al., 2020). Such methods include mandatory diversity training for employees, specific minority hiring recruitment programs, and diversity inclusion statements; however, these demonstrate varied success in such disparity reduction (Moshiri & Cardon, 2019). Research suggests that this lack of consensus is mirrored in academic settings, with notable evidence that diversity support agendas and mandatory cultural trainings are not necessarily sufficient interventions to facilitate healthy, supportive campus and classroom environments (Groggins & Ryan, 2013; Moshiri & Cardon, 2019; Williams et al., 2005). Shortcomings may particularly occur when diversity support efforts are defensive responses to a crisis or are proposed via a loosely structured centralized diversity support initiative.
Centralized efforts are campus-wide diversity plans that originate from a high university office position to be implemented across all university subunits, such as individual colleges, offices, or departments. These institutional responses can emerge organically or as a form of crisis management following an overtly discriminatory campus incident. Example incidents include a disparaging comment about a minority group by a professor or an on-campus racial incident that draws national attention via press or social media (Hurtado et al., 2008; Williams, 2008). Such centralized action may also result from student-led protests or complaints, which often come to a crux after a critical need is either unprovided for over a long period of time or is only temporarily met and removed without warning (Comeaux et al., 2021; Matos, 2016).
Rose and Paisley (2012) proposed that the process of social change begins with self-awareness, accepting and believing experiences of others, and exploring when those with privilege impose oppressive values on underrepresented groups. However, when administration rapidly rolls out crisis-management efforts, the speed and generality of implementation can quickly become unfocused and disorganized. This most notably diminishes the opportunity for the development of awareness. Similarly, while centralized agendas commendably communicate diversity as a core focus of their institution, tasks to accomplish this may become delegated to a small group of employees who lack the resources to enact the proposed change (Wilson, n.d.). These diversity support initiatives can easily become stymied in their most inchoate stages, remaining immaterial and symbolic; of consequence, a true understanding of the cause for the initial diversity crisis remains unrealized (Hurtado et al., 1998; Williams, 2008). Such efforts may ultimately help to control the fire, but fail to extinguish what sparked the blaze.
Williams (2008) estimates that college campuses tend to regress to their original organizational culture and practices within a year-and-a-half of the initial crisis. One contributing factor to this may be that faculty and administration struggle with a sense of role ambiguity in how to discuss or integrate multicultural perspectives or promote inclusion efforts in class due to a lack of clear guidance and objective goals outlined in centralized diversity statements (Hurtado et al., 1998). While many faculty deeply value diversity inclusion, cohesion, and clarity in how to make their classrooms and departmental climate safe for all students, they may be at a loss for how to best do this, especially given the unique demands of their respective fields (Rogers et al., 2019; Williams et al., 2005).
Alternative Solutions and the Present Study
Alternative approaches to centralized, crisis-management responses in academia include decentralized diversity initiatives, departmental-level interventions, and proactive interventions designed to prevent or rectify existing injustices (Barrile et al., 2023; Williams, 2008). Decentralized initiatives occur when departments or colleges within a university spearhead the implementation process of a centralized diversity plan. Such a method can allow for more intentional student support inside and outside the classroom. These approaches can help departments attain a deeper understanding of the shortcomings of existing diversity support in respective areas, create attainable and measurable outcomes, and quickly implement new interventions based on student feedback. This can include the provision or coordination of multiple academic support systems for an individual student that remains tailored to their specific needs and training requirements (Dufault, 2017; Williams, 2008; Wilson, n.d.). Measurable outcomes can be attained by departmental climate assessments completed by students, either to inform or assess the effectiveness of cultural competency initiatives (Hurtado et al., 2008; Sanchez et al., 2018; Smith et al., 2017).
Assessing student perceptions of a supportive climate may be most effectively spearheaded by a departmental planning team elected by a department head or college dean who are able to take action immediately following student feedback. This can include collaboratively increasing program activities and campus accessibility for student minority groups, identifying feasible and measurable target outcomes, and repeating climate measures to assess the success of such outcomes (Hurtado et al., 2008; Rose & Paisley, 2012). In this manner, the department can “[establish] a culture of evidence” for the success of its diversity support initiatives (Williams, 2008, p. 34).
Although there are existing campus climate surveys for higher education purposes (e.g., Sanchez et al., 2018; Smith et al., 2017), they tend to have different objectives or have one or more of the following restrictions: 1) they use centralized approaches to assess climate across campus rather than within a single department; 2) they assess climate for specific populations, such as of LGBTQ+ students (e.g., Szymanski & Bissonette, 2020); 3) they are long, which contributes to reduced participation; and/or 4) they cannot assess specific actions from both faculty and fellow students. This study piloted a new decentralized student survey that inquired about various aspects of a psychology department’s diversity inclusiveness and support. The ultimate aim of the study was to establish a measure that would (a) assess the sense of belonging and psychological safety within the entire student body, including students from diverse cultural backgrounds, both in and outside of departmental class settings, (b) provide guidance for departmental inclusion efforts, (c) gain a working scale with clear factor structure and reliability for replication, and (d) determine if the scale demonstrated sufficient sensitivity to detect any existing differences between groups based on educational levels, racial/ethnic identification, and sexual orientation.
Method
Participants
Participants were 650 students taking courses in a psychology department during a fall semester in the COVID-19 pandemic. The department is part of a mid-size four-year public university set in a town of 65,000 in the southeastern U.S. Participants’ ethnicity, gender, age, and sexual orientation are shown in Table 1. It should be noted that some participants exited the scale before completing some or all of the demographics questions. Additionally, we inadvertently left off the gender question from the scale and noticed this toward the end of data collection. When it was noticed, we added the question for the remainder of data collection. As such, the numbers for this demographic variable only represent a subsample of the larger sample. However, the percentages for the various demographics are in line with known percentages for the larger department population, including those for the gender.
As for program of study, the frequencies and percentages were as follows: 514 (79.077%) undergraduates, 56 (8.615%) in the Masters program, 23 (3.538%) in the doctoral program, and 57 (8.769%) did not answer this question before exiting the survey. For major/minor status, 304 (46.769%) were majors, 34 (5.231%) were minors, 189 (29.077%) were not majors or minors, 66 (10.154%) indicated that this was not applicable, likely because they considered themselves graduate students and this would not apply, and 57 (8.769%) did not answer this question before exiting the survey. As for the number of courses taken in the department, 238 (36.615%) reported that they had taken 1-3 courses, 211 (32.462%) reported that they had taken 4-10 courses, and 144 (22.154%) reported 11 or more classes taken. Fifty-seven (8.769%) students did not answer this question.
Item Generation
Our team – composed of one tenured faculty member, two full-time non-tenured faculty, and two graduate students – collaboratively generated the items related to general feelings of inclusiveness in the department, feelings about faculty, feelings about peers, and feelings about engagement on campus. We subsequently met with a group of students to receive feedback on our scale before administering it within the department. We grouped similar items together based on area being assessed (ex. faculty and peers), rather than alternating between the areas. This decision was based on research suggesting that, when respondents must switch back and forth between contexts in a survey or scale, it may place cognitive load on the respondent and compromise the psychometric properties of the scale (Ickes et al., 2018). Additionally, items were actionable, in that results were tied to specific recommended actions; for instance, one item read, “The majority of my instructors present or discuss the scholarly work of individuals from diverse backgrounds.” Low scores on this item could lead to very specific future actions for instructors.
Our original scale consisted of 39 items regarding perceptions of inclusiveness and 7 demographic items. With the exception of the demographic items, all items were rated by students on a 1 (totally disagree) to 7 (totally agree) scale. Rated items began with “Most of the time, I feel…” or “The majority of my instructors (or peers)…” so that students would not focus solely on their current class or a recent specific experience. However, we also did not want to minimize single negative incidents that students may have experienced. As such, we allowed students to share any particular good, neutral, or bad experience at the end of each section in a comment box. A sample item for general feelings about the department was, “Most of the time, I feel the department is an inclusive environment.” A sample item for feelings toward faculty was, “The majority of my instructors include me as much as they include other students.” Finally, a sample item for feelings toward peers was, “The majority of my peers treat me as an equal.” The final Student Inclusiveness Scale items, along with the factors in which each item was ultimately included, can be found in Appendix A.
Procedure
Following approval by the university’s Institutional Review Board, recruitment occurred in departmental classes over a two-week period. Faculty were provided with a short script introducing the study and asked to share a QR code that linked to our survey with students. Some instructors gave their students time in class to complete the scale, whereas others simply shared the link with their students to complete the scale on their own time. For fully online classes, professors shared this information as an announcement on their learning management system. Following the script, the survey presented students with a consent form that explained the purpose of the study was “to assess students’ perceptions of inclusiveness in the department” in a confidential manner. Once participants consented, they completed four sections: general departmental impressions, feelings toward faculty, feelings towards peers, and demographics. Items to encourage campus engagement were interleaved in the general departmental impressions and faculty sections.
Results
Factor Structure
To examine the factor structure of our scale, we conducted an exploratory factor analysis. Our item level statistics indicated that the scores skewed toward the high end of the scale. We therefore used principal axis factoring, as this factor analytic method is recommended for skewed data (Fabrigar et al., 1999). In addition, we used an oblimin rotation since any emerging factors were expected to correlate with one another. Initial factor analyses suggested dropping 13 items due to one or more of the following issues: low communalities, high an illogical cross-loadings, factors accounted for negligible variance, and reverse-scored items which, regardless of content, all loaded onto one factor. The last issue is not uncommon and many caution against using reverse-scored items at all (Conrad et al., 2004; Magazine et al., 1996; Suárez Álvarez et al., 2018). As such, we dropped the 13 reverse-scored items and reran the factor analysis without those items.
The Kaiser-Meyer-Olkin statistic (.948) indicated that we had a sufficiently large sample for conducting factor analysis, as values above .90 indicate a sufficient sample (Field, 2013). In addition, Bartlett's Test of Sphericity indicated that the items were sufficiently correlated for factor analysis, χ2 = 9389.467, p < .001. Four factors emerged with good eigenvalues above the cut-off of 1. Additionally, all factor loadings were above Garson’s (2010) recommended cut-off value of .40. Items (n = 6) loading onto the first factor all related to general feelings of inclusiveness in the department. The items (n = 7) that loaded onto the second factor all related to feelings of inclusiveness from peers. The third factor consisted of items (n = 9) related to feelings of inclusiveness from faculty. The fourth factor consisted of items (n = 3) assessing campus engagement encouragement. Together, all four factors accounted for 62.59% of the variance, with the first factor accounting for 43.19%, the second accounting for 7.77%, the third accounting for 6.04%, and the fourth accounting for 5.60%. Items and their loadings onto their respective factors are shown in Table 2.
Internal Reliability
Once we identified factors, we examined internal reliability for each subscale. Subscale reliabilities for the entire sample were quite good: general feelings of inclusiveness in the department (α = .925), feelings of inclusiveness from peers (α = .886), feelings of inclusiveness from faculty (α = .879), and campus engagement encouragement (α = .761). We also assessed subscale reliability for specific groups in our sample. These groups were: undergraduate students, graduate students, major/minor students, non-major/minor students, White students, BIPOC (Black, Indigenous, Persons of Color) students, heterosexual students, and LGB/Other (lesbian, gay, bisexual, other non-heterosexual) students. With one exception, all the subscale reliabilities were above .70, indicating good internal reliabilities for each group. The internal reliability for campus engagement encouragement, as rated by graduate students was .686.
Differences in Subscale Scores
Next, we examined subscale means for our entire sample to get a sense of how students rated the four aspects of department climate. We calculated subscale scores for each participant by averaging the items in that subscale. These means indicated mostly positive ratings. The subscale means, from highest to lowest, were general feelings of inclusiveness in the department (M = 5.831; SD = 1.087), feelings of inclusiveness from faculty (M = 5.786; SD = 0.948), feelings of inclusiveness from peers (M = 5.519; SD = 1.069), and campus engagement encouragement (M = 5.02; SD = 1.404). For all scales the lowest possible score was 1 and the highest possible score was 7, and these scores did indeed occur in the data.
Differences in Subscale Scores Between Groups
A series of mixed model ANOVAs allowed us to determine if various groups in the department differed in their subscale ratings (white and BIPOC, heterosexual and LGB/Other, undergraduate and graduate, and majors/minors and non-majors/minors). This allowed us to see if certain groups had more or less favorable ratings of the department for each subscale and if the order of strengths with regard to the subscales were viewed differently across the groups. In some instances, we had to combine several groups into one, such as BIPOC students, as there were not enough students in specific ethnic groups (e.g., Native American, Asian American) to run analyzes with each specific group. The means for these comparisons across the various dependent measures are shown in Table 3.
Differences in Subscale Scores Between White and BIPOC Groups
A 4 (subscale) x 2 (ethnicity) mixed model ANOVA showed a main effect for the type of subscale assessed, Wilk’s Lambda = .723, F(3, 591) = 73.335, p < .001, ηp2 = .277. Overall, general feelings of inclusiveness in the department had the highest mean, followed by feelings of inclusiveness from faculty, feelings of inclusiveness from peers, and then campus engagement encouragement. Bonferroni post-hoc tests indicated that there were significant differences between all these subscales (all p’s < .001), except between general feelings of inclusiveness in the department and feelings of inclusiveness from faculty (p = 1.00).
There was also a main effect of ethnicity, F(1, 593) = 38.644, p < .001, ηp2 = .061. When looking at overall ratings, collapsed across the subscales, White students had significantly higher ratings than BIPOC students. There was no significant interaction effect, Wilk’s Lambda = .993, F(3, 591) = 1.455, p = .226, ηp2 = .007. This indicated that the relationship between ethnicity and ratings did not depend on the subscale., nor did the relationship between subscale and ratings depend on ethnicity.
Differences in Subscale Scores Between Sexual Orientation Groups
A 4 (subscale) x 2 (sexual orientation) mixed model ANOVA showed a significant main effect for type of subscale, Wilk’s Lambda = .709, F(3, 563) = 77.210, p < .001, ηp2 = .291, such that general feelings of inclusiveness in the department had the highest mean, followed by feelings of inclusiveness from faculty, feelings of inclusiveness from peers, and then campus engagement encouragement. Bonferroni post-hoc tests indicated that there were significant differences between all these subscales (all p’s < .001), except between general feelings of inclusiveness in the department and feelings of inclusiveness from faculty (p = 1.00).
There was a significant main effect of sexual orientation, F(1, 565) = 4.269, p = .039, ηp2 = .007. However, it should be noted that this difference was very small. Overall, heterosexual students had significantly higher overall ratings than those who were in the LGB/Other group. Interestingly, there was a significant interaction effect, such that group differences depended on the particular subscale that was examined. The two groups had similar ratings for each subscale (all p’s > .05) with one exception. Heterosexual students had significantly higher ratings for feelings of inclusiveness from peers than those in the LGB/Other group (p < .001).
Differences in Subscale Scores Between Undergraduate and Graduate Groups
A 4 (subscale) x 2 (program level) mixed model ANOVA showed that, again, there was a main effect for type of subscale, Wilk’s Lambda = .762, F(3, 563) = 61.222, p < .001, ηp2 = .238, such that general feelings of inclusiveness in the department had the highest mean, followed by feelings of inclusiveness from faculty, feelings of inclusiveness from peers, and then campus engagement encouragement. Bonferroni post-hoc tests indicated that there were significant differences between all these subscales (all p’s < .001), except between general feelings of inclusiveness in the department and feelings of inclusiveness from faculty (p = 1.00).
There was no significant main effect of program level (undergraduate versus graduate), F(1, 591) = .127, p = .722, ηp2 = .000. This indicated that, collapsed across subscale, these groups had similar ratings. However, there was a significant interaction, Wilk’s Lambda = .975, F(3, 589) = 5.086, p = .002, ηp2 = .025, which showed that there was a group difference for a specific subscale. Graduate students had significantly higher ratings on feelings of inclusiveness from faculty than undergraduate students (p = .034).
Differences in Subscale Scores Between Major/minor and Non-major/minor Groups
A 4 (subscale) x 2 (major/minor status) mixed model ANOVA indicated that there was a main effect of subscale, Wilk’s Lambda = .676, F(3, 589) = 105.377, p < .001, ηp2 = .324. Again, the favorability ratings went in the following highest to lowest order: general feelings of inclusiveness in the department, feelings of inclusiveness from faculty, feelings of inclusiveness from peers, and campus engagement encouragement. There was no main effect of major/minor versus non-major/non-minor status, F(1, 591) = .135, p = .713, ηp2 = .000. However, there was a significant interaction effect, Wilk’s Lambda = .971, F(3, 589) = 5.940, ηp2 = .029, such that majors/minors had significantly higher ratings for campus engagement encouragement than non-majors/non-minors (p = .006).
Summary of Findings for Group Differences
Regardless of group, students had generally favorable ratings on the four subscales. Additionally, they tended to agree about the order of those strengths. General feelings of inclusiveness in the department consistently had the highest average rating, followed by feelings of inclusiveness from faculty, feelings of inclusiveness from peers, and campus engagement encouragement, respectively. However, certain minority groups (e.g., BIPOC, LGB/Other, and graduate students) tended to have lower ratings than their comparison majority group, for some of the subscales. These results show that our scale was sufficiently sensitive to detect differences between groups of different educational levels, race/ethnicity, and sexual orientation.
Discussion
The current study sought to meaningfully quantify student perceptions of inclusiveness in a large department within a mid-size university using a newly created scale of four specific aspects of inclusiveness. These four aspects include general feelings of inclusiveness in the department, feelings of inclusiveness from faculty, feelings of inclusiveness from peers, and campus engagement encouragement. With responses from almost half of our department’s student body, we determined that our scale assessed four specific aspects of departmental functioning, as determined by factor analysis, with high reliability. Results were skewed positive overall, and interestingly, there was a consistent pattern in the order in which subscales were rated from highest to lowest. The overall sample and all subgroups of participants rated the subscales from highest to lowest in the following order: general feelings of inclusiveness in the department, feelings of inclusiveness from faculty, feelings of inclusiveness from peers, and campus engagement encouragement.
Further, subgroup comparisons showed that both white participants had higher average ratings than BIPOC participants and heterosexual participants had higher average ratings than the LGB/Other participants. This provides initial evidence of the validity of the scale, as we would expect these differences to emerge to some degree. These results support previous research in which non-majority college students differ in their perceptions of and experiences in higher education settings (Barker, 2011; Booker & Campbell-Whatley, 2019; Griffin et al., 2016; Winberg et al., 2019). Further, these results may also reflect the increasing number of incidents of discrimination faced by BIPOC participants during the pandemic and the feelings of isolation and depression faced by members of this group (Oh et al., 2021; Strassle et al., 2022). The stress and anxiety levels in members of the LGBT group observed by Hoyt et al. (2021) may be factors in the lower average ratings from our participants who self-identified as LGB/Other.
Strengths and Implications
The assessment of student perceptions is especially meaningful because perceptions can affect actions, which in turn, can affect success in higher education. As research suggests that students from minority racial/ethnic groups, those with dis/abilities, and those from sexual minority groups experience oppression, discrimination, and alienation on the basis of their cultural identities, it is crucial to identify pathways for improvement in academia and means to bolster protective factors in students (Tovar-Murray et al., 2012). In our case, the relatively lower ratings in feelings of inclusiveness from peers have implications for our faculty. For example, our faculty can provide more guidance on inclusive behaviors both in and out of the classroom to peers as they work and interact with one another.
Pathways for improvement are difficult when a scale is used at the institutional or college level, due to the limited item specificity and applicability toward particular changes at the department or classroom level; however, it is at these very levels that many diversity-related surveys occur (Person et al., 2015; Sanchez et al., 2018; Williams, 2008). The current decentralized scale was created in a manner that allows clear, meaningful, and actionable direction for improving inclusion and belonging in a department. Items on the two subscales related to faculty and peers specifically can be translated into specific behaviors that can be taught, supported, and encouraged at the department or classroom level. Our scale, similar to one developed by Smith, Wessel, and Polacek (2017), has items that readily connect to specific behavioral change and can be used for ongoing assessments of the departmental climate. Further, the current scale possesses generalizability beyond the particular aspects of functioning at our institution. Given its focus on shared classroom and student experiences, it is readily usable at other institutions. Such generalizability has been a criticism of previous scales (e.g., Hurtado et al., 2008), and a limitation that our scale can remedy.
Reflections on an Evolving Climate. Since this study was completed, there has been a stark increase in discouragement of or prohibitions on college Diversity Equity and Inclusion (DEI) efforts in the United States. Multiple U.S. political figures argue that a focus on cultural awareness, certain aspects of history, and/or unique minority experiences in education elicits anti-patriotic sentiments, societal division, and self-hatred or guilt in majority groups (Exec. Order No. 13950, 2020; Kirsanow, 2022). As a result, numerous politicians have increasingly called for a “culture-blind” approach to education in attempt to promote a nationalist identity and, in recent years, have passed formal laws that discourage or prohibit explicit DEI statements, efforts, and collection of information related to DEI as part of institutional accreditation (H.B. 1512, 2025; S. 71, 2024; U.S. News & World Report, 2023). This has led to further concern, anxiety, and uncertainty amongst departments and faculty as to if, how, and when it is appropriate to consider or address diversity and inclusion issues in class spaces (Briscoe, 2024). Burgeoning research that explores the organizational and psychological consequences of these acts suggests that removal of campus DEI considerations reduces feelings of safety and inclusion in minority students, disrupts organizational functioning, and affects student enrollment decisions (Dhanani et al., 2024; Green, 2024).
It is important to discuss the utility of the Student Inclusiveness Scale within this current, often polarized, sociopolitical climate regarding higher education. First, this study represents a faculty-led research effort to evaluate departmental climate. The scale was utilized strictly as a tool for internal improvement and student success modeling, rather than for formal accreditation or administrative compliance. Second, the authors assert that the core constructs measured by this scale are belonging and psychological safety. These factors have well-documented relationships with academic success (Van Kessel et al., 2025). Our scale was designed to assess the student’s perception of the learning environment rather than any department or instructor’s adherence to specific ideologies. The scale does not contain any language that targets specific protected groups to the exclusion of others, nor does it contain prescriptive items (e.g., requiring students or instructors to declare or ask about pronouns). Instead, items focus on the presence of general psychological safety. For instance, our item about gender pronouns assesses whether it is safe to share one’s identity if one chooses to do so. Similarly, the item about whether diverse scholarly voices are represented does not assess which specific diverse voices are included. Rather, it allows the student to assess inclusivity based on their own perception. By focusing on these distinct psychological experiences rather than political benchmarks, the Student Inclusiveness Scale offers individual college departments a rigorous, data-driven metric to monitor student support and retention that remains viable and relevant across varied political contexts.
Limitations and Future Directions
The current study is not without limitations. The results presented here are limited to one large department at one mid-size institution, therefore, additional uses in other departments or at other institutions would provide additional data on whether it can reliably assess the four identified subscales. Similar to samples in other studies, our sample was limited in racial/ethnic and other types of diversity. As a result, we were able to make only a few comparisons of perspectives between groups. Lastly, the current study did not include an outcome measure to which feelings of inclusiveness were compared. Ultimately, one of the primary goals of inclusive efforts in higher education is to improve educational persistence and degree attainment for students who belong to marginalized groups.
As the student population on college and university campuses continues to diversify, institutions have an ethical, and possibly moral, responsibility to understand and meet the needs of different student groups. This may be especially true in times of crisis or societal unrest. Such responsibility can take the form of centralized campus-wide initiatives that all are mandated to follow or specific smaller unit activities designed to closely assess the particular students within a unit. The Student Inclusiveness Scale described here provides a factor-analyzed and reliable method of understanding student perceptions and providing guidance for unit-level improvements.
References
American Psychiatric Association (2013). Diagnostic and statistical manual of mental
disorders (5th Ed.).
American Psychological Association (2002). Guidelines on multicultural education, training,
research, practice, and organizational change for psychologists. Retrieved from:
https://www.apa.org/about/policy/multicultural-guidelines-archived.pdf
Arkansas House of Representatives, H.B. 1512, 95th Gen. Assemb., Reg. Sess. (Ark. 2025).
https://arkleg.state.ar.us/Home/FTPDocument?path=%2FACTS%
2F2025R%2FPublic%2FACT341.pdf
Bai, X., Ramos, M. R., & Fiske, S. T. (2020). As diversity increases, people paradoxically
perceive social groups as more similar. Psychological and Cognitive Sciences, 117(23),
12741-12749. DOI: 10.1073/pnas.2000333117
Barker, M. J. (2011). Racial context, currency, and connections: Black doctoral students and
white advisor perspectives on cross-race advising. Innovations in Education and
Teaching International, 48(4), 387-400. DOI: 10.1080/14703297.2011.617092
Barrile, G.M., Bernard, R.F., Wilcox, R.C., Becker, J.A., Dillon, M.E., Thomas-Kuzilik, R.R.,
Bombaci, S.P., Merkle, B.G. (2023). Equity, community, and accountability: Leveraging
a department-level climate survey as a tool for action. PLoS One, 18(8), e0290065.
doi: 10.1371/journal.pone.0290065.
Briscoe, K. L. (2024). Dismantling DEI in higher education: An analysis of how diversity
professionals view political bans (Fellowship project, 2023–2024 class). Free Speech
Center, University of California.
Booker, K. C., & Campbell-Whatley, G. (2019). Student perceptions of inclusion at a
Historically Black University. The Journal of Negro Education, 88(2), 146-158.
DOI: 10.7709/jnegroeducation.88.2.0146
Browning, M.H., Larson, L.R., Sharaievska, I., Rigolon, A., McAnirlin, O., Mullenbach, L., et
al. (2021) Psychological impacts from COVID19 among university students: Risk factors across seven states in the United States. PLoS ONE,16(1): e0273938. https://doi.org/10.1371/journal.pone.0273938
Cheryan, S., Plaut, V. C., Davies, P. G., & Steele, C. M. (2009). Ambient belonging: How
stereotypical cues impact gender participation in computer science. Journal of
Personality and Social Psychology, 97(6), 1045–1060. DOI: 10.1037/a0016239
Clark, C. R., Mercer, S. H., Zeigler-Hill, V., & Dufrene, B. A. (2012). Barriers to the success of
ethnic minority students in school psychology graduate programs. School Psychology
Review, 41(2), 176-192. DOI: 10.1080/02796015.2012.12087519
Comeaux, E., Mireles, D., & Acha, A. (2021). Dis/abled student campusmaking: Sites of new
possibility. Education Sciences, 11(11), 745. DOI: 10.3390/educsci11110745
Conrad, K. J., Wright, B. D., McKnight, P., McFall, M., Fontana, A., & Rosenheck, R. (2004).
Comparing traditional and Rasch analyses of the Mississippi PTSD Scale: revealing limitations of reverse-scored items. Journal of
Applied Measurement, 5(1), 15-30.
Dhanani, L. Y., Arena, D. F., Jr., & Bogart, S. M. (2024). The unequal burden of DEI bans.
Industrial and Organizational Psychology, 17(4), 503–506.
https://doi.org/10.1017/iop.2024.44
Dismantle DEI Act of 2024, S. 71, 118th Cong. (2024). https://www.vance.senate.gov/wp-
content/uploads/2024/06/Dismantle-DEI-Act-Vance-6.12.2024.pdf
Dufault, K. H. (2017). Rethinking partnerships on a decentralized campus. The Learning
Assistance Review, 22(1), 9-17.
Executive Order No. 13950, 85, 3 C.F.R. 60683 (2020). https://www.federalregister.gov
/documents/2020/09/28/2020-21534/combating-race-and-sex-stereotyping
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use
of exploratory factor analysis in psychological research. Psychological methods, 4(3), 272.
Field, A. (2013). Discovering statistics using IBM SPSS statistics. Sage.
Garson, G. D. (2010). Factor analysis. Retrieved from:
http://faculty.chass.ncsu.edu/garson/PA765/factor.htm
Green, N. M. (2024, February 9). Causes and consequences of ending diversity, equity, and
inclusion on Texas college campuses. Scholars Strategy Network.
https://scholars.org/contribution/causes-and-consequences-ending-diversity
Griffin, K. A., Cunningham, E. L., & Mwangi, C. A. G. (2016). Defining diversity: Ethnic
differences in Blacks students’ perceptions of racial climate. Journal of Diversity in
Higher Education, 9(1), 34-49. DOI: 10.1037/a0039255
Groggins, A. & Ryan, A.M. (2013). Embracing uniqueness: The underpinnings of a positive
climate for diversity. Journal of Occupational Organizational Psychology, 86, 264-
282. DOI: 10.1111/joop.12008
Hagedorn, R. L., Wattick, R. A., & Olfert, M. D. (2022). ‘My entire world stopped’: College
students’ psychosocial and academic frustrations during the COVID-19 pandemic. Applied Research in Quality of Life, 17, 1069-1090.
Hoever, I. J., van Knippenberg, D., van Ginkel, W. P., & Barkema, H.G. (2012). Fostering team
creativity: Perspective taking as key to unlocking diversity's potential. Journal of
Applied Psychology, 97(5), 982-996. DOI: 10.1037/a0029159
Horowitz, J. M., Parker, K., Brown, A., & Cox, K. (October 6, 2020). “Amid National
Reckoning, Americans Divided on Whether Increased Focus on Race Will Lead to Major Policy Change”. Pew Research Center.
on-race-will-lead-to-major-policy-change/
Hoyt, L. T., Cohen, A. K., Dull, B., Castro, E. M., & Yazdani, N. (2021). ‘Constant stress has
become the new normal’: Stress and anxiety inequalities among U.S. college students in
the time of COVID-19. Journal of Adolescent Health, 68, 270-276.
Hurtado, S., Griffin, K. A., Arellano, L., & Cuellar, M. (2008). Assessing the value of climate
assessments: Progress and future directions. Journal of Diversity in Higher Education,
1(4), 204-221. DOI: 10.1037/a0014009
Hurtado, S. Milem, J. F., Clayton-Pedersen, A. R., & Allen, W. R. (1998). Enhancing campus
climates for racial/ethnic diversity: Educational policy and practice. The Review of
Higher Education, 21(3), 279-302. DOI: 10.1353/rhe.1998.0003
Ickes, W., Babcock, M., Hamby, T., Park, A., Robinson, R., & Taylor, W. (2018). Side streets
and u-turns: Effects of context switching, direction switching, and factor switching
on interitem correlations and misresponse rates. Journal of Personality Assessment, 101(3), 326-339.
Ifedi, R. (2017). African-born female academics in the U.S.: Experiences of inclusion,
exclusion, and access - building careers on marginalized identities. International Journal
of Bias, Identity and Diversities in Education, 2(1), 1-12.
DOI:10.4018/IJBIDE.201701010
Jehi, T., Khan, R., Dos Santos, H., & Majzoub, N. (2023). Effect of COVID-19 outbreak on
anxiety among students of higher education: A review of literature. Current Psychology, 42, 17475-17489.
Kirsanow, P. (2022). ‘Anti-racism’ and common sense. National Review. https://www.nationalreview.com/corner/anti-racism-and-
common-sense/
Krogstad, J. M., Dunn, A., & Passel, J. S. (2021, August 23). Most Americans say the declining
share of white people in the U.S. is neither good nor bad for society. Pew Research
Center. https://www.pewresearch.org/fact-tank/2021/08/23/most-americans-say-the-
declining-share-of-white-people-in-the-u-s-is-neither-good-nor-bad-for-society/
Magazine, S. L., Williams, L. J., & Williams, M. L. (1996). A confirmatory factor analysis
examination of reverse coding effects in Meyer and Allen's Affective and
Continuance Commitment Scales. Educational and Psychological Measurement, 56(2), 241-250.
Matos, A. (2016). ‘Racism at AU is bananas’: Hundreds protest incidents on American U.
Campus. Retrieved from: https://www.washingtonpost.com/local/education/racism-at-au-
is-bananas-hundreds-protest-incidents-on-american-u-campus/2016/09/19/4526a4ba-
7e97-11e6-9070-5c4905bf40dc_story.html
Moshiri, F., & Cardon, P. W. (2019). Best practices to increase racial diversity in business
schools: What actually works according to a nationwide survey of business schools.
Journal of Education of Business, 94(2), 113-124.
DOI: 10.1080/08832323.2018.1503583
National Center for Education Statistics (2019). Status and trends in the education of racial and
ethnic groups. Institute of Education Sciences. https://nces.ed.gov/pubs2019/2019038.pdf
Oh, H., Marinovich, C., Rajkumar, R., Besecker, M., Zhou, S., Jacob, L., Koyanagi, A., &
Smith, L. (2021). COVID-19 dimensions are related to depression and anxiety among US
college students: Findings from the Healthy Minds Survey 2020. Journal of Affective
Disorders, 292, 270-275.
Person, S. D., Jordan, C. G., Allison, J. J., Fink Ogawa, L. M., Castillo-Page, J., Conrad, S.,
Nivet, M. A., & Plummer, D. L. (2015). Measuring diversity and inclusion in academic medicine: The Diversity Engagement Survey. Academic Medicine, 90, 1675-1683.
DOI: 10.1097/ACM.0000000000000921
Rogers, E. B., Taylor, J., & Rose, J. (2019). Perceptions and experiences of diversity and
Inclusion of outdoor educators in higher education. Journal of Outdoor Recreation,
Education, and Leadership, 11(2), 134-150. DOI: 10.18666/JOREL-2019-V11-I2-8997
Rose, J., & Paisley, K. (2012). White privilege in experiential education: A critical reflection.
Leisure Sciences, 34, 136-154. DOI:10.1080/01490400.2012.652505
Sanchez, J. E., DeFlorio, L., Wiest, L.R., & Oikonomidoy, E. (2018). Student perceptions of
inclusiveness in a college of education with respect to diversity. College Student Journal,
52(3), 397-409.
Slavin, R. E. (1990). Research on cooperative learning: Consensus and controversy.
Educational Leadership, 47(4), 52–54.
Smith, D. G. (2020). Diversity's promise for higher education: Making it work (3rd Ed.). Johns
Hopkins University Press.
Smith, T. M. E., Wessel, M. T., & Polacek, G. N. L. J., (2017). Perceptions of cultural
competency and awareness among college students: implications for awareness in higher
education. The ABNF Journal, 28(2), 25-33.
Spencer, B., & Castano, E. (2007). Social class is dead. Long live social class! Stereotype
threat among low socioeconomic status individuals. Social Justice Research, 20(4), 418–
432. DOI: 10.1007/s11211-007-0047-7
Stephens, N. M., Fryberg, S. A., Markus, H. R., Johnson, C. S., & Covarrubias, R. (2012).
Unseen disadvantage: How American universities’ focus on independence undermines
the academic performance of first-generation college students. Interpersonal Relations and Group Processes, 102(6), 1178-1197. DOI: 10.1037/a0027143
Strassle, P.D., Stewart, A.L., Quintero, S.M., et al. (2022). COVID-19–related discrimination
among racial/ ethnic minorities and other marginalized communities in the United States. American Journal of Public Health, 112(3), 453–466.
Suárez Álvarez, J., Pedrosa, I., Lozano, L. M., García Cueto, E., Cuesta Izquierdo, M.,
& Muñiz Fernández, J. (2018). Using reversed items in Likert scales: A
questionable practice. Psicothema, 30(2), 149-158.
Summers, M., & Volet, S. (2008). Students’ attitudes towards culturally mixed groups on
international campuses: Impact of participation in diverse and non-diverse groups. Studies in Higher Education, 33(4), 357-370. DOI:
Szymanski, D. M., & Bissonette, D. (2020). Perceptions of the LGBTQ college campus climate
scale: Development and psychometric evaluation. Journal of Homosexuality, 67(10),
1412-1428. DOI: 10.1080/00918369.2019.1591788
Tovar-Murray, D., Jenifer, E.S., Andrusyk, J., D’Angelo, R., & King, T. (2012). Racism-related
stress and ethnic identity as determinants of African American college students’ career
aspirations. The Career Development Quarterly, 60, 254-262.
DOI: 10.1002/j.2161-0045.2012.00021.x
U.S. News & World Report (2023, April 11). Tracking the attack on critical race theory in
education. https://www.usnews.com/opinion/articles/2023-04-11/tracking-the-
attack-on-critical-race-theory-in-education
Van Kessel, G., Ryan, C., Paras, L., Johnson, N., Zariff, R. Z., & Stallman, H. M. (2025).
Relationship between university belonging and student outcomes: A systematic review
and meta-analysis. The Australian Educational Researcher, 52, 2511-2534.DOI: 10.1007/s13384-025-00822-8
Wiggan, G. (2007). Race, school achievement, and educational inequality: Toward a student-
based inquiry perspective. Review of Educational Research,77(3), 310–333.
DOI: 10.3102/003465430303947
Williams, D. A., Berger, J. B., & McClendon, S. A. (2005). Toward a Model of Inclusive
Excellence and Change in Postsecondary Institutions. Association of American Colleges
and Universities.
Williams, D. A. (2008). Beyond the diversity crises model: Decentralized diversity planning and
implementation. Planning for Higher Education, 36(2),27-41.
Wilson, J. L. (n.d.). Diversity, equity, and inclusion planning. The Society for College and
University Planning.
https://www.scup.org/planning-type/diversity-equity-and-inclusion-planning/
Winberg, C., Coleman, T., Woodford, M. R., McKie, R. M., Travers, R. & Renn, K. A. (2019).
Hearing “that’s so gay” and “no homo” on campus and substance use among sexual
minority college students. Journal of Homosexuality, 66(10), 1472–1494. DOI: 10.1080/00918369.2018.1542208
Zhang, W., & Kolady, D. (2021). The COVID-19 pandemic’s unequal socioeconomic impacts
on minority groups in the U.S. Demographic Research,47, 1019-1031.
