Development and Validation of the Assessment of Racial Microaggressions in Academic Settings (ARMAS) Scale

Racial Microaggressions (RM) in academic settings can have pervasive effects on students of color, specifically in graduate programs. A national sample (N = 289) was collected from programs approved by APA in order to validate a newly developed Scale called Assessment of Racial Microaggressions in Academic Settings (ARMAS). An exploratory factor analyses was conducted which yielded eight factors: (1) Ascription of Intelligence, (2) Assumptions of being a foreigner, (3) Multicultural issues seen as not a priority and being treated differently (4) Invisibility/Felt ignored, (5) Assumptions about me and my work with clients/representing entire race, (6) Colorblindness, (7) Assumptions of professional advantage because of race/ethnicity, and (8) Stereotypical assumptions about my race/ethnicity. Reliability along with discriminant and convergent validity was analyzed. Results of the study suggest that the ARMAS is a valid and reliable measure of RMs in academic settings. Additional results indicate that half of the sample considered dropping-out more frequently during the first three years of their programs. Higher scores in the ARMAS were in factors that are unique of this study of RMs in academic settings (Assumptions about me and my work with clients, and Assumptions of professional advantage because of race/ethnicity). Participants’ main reasons for dropping out included: lack of support from faculty, lack of confidence, overwhelmed about academic demands and RMs. Microinvalidations was one of the top three reasons for dropping out for Black, American Indian/Alaska Native, and Multiracial graduate students. Practical implications to support graduate students of color and future directions for research are discussed.

RMs include communications that consciously or unconsciously convey a derogatory message to a person of color and can adversely affect the mental health (Sue et al, 2008), self-esteem (Nadal, Griffin, Wong, Hamit, & Rasmus, 2014), and self-efficacy (Blume et al., 2012) of students of color. RMs also negatively affect the academic engagement of students of color (Clark et al. 2012). Due to their adverse impacts, RMs could depress retention and graduation rates of people of color in graduate programs in psychology, which in turn could contribute to the shortage of ethnic minority professionals in fields such as Psychology (Ortiz-Frontera, 2013).

Racial Microaggresions
Racism is a delicate topic in many social contexts. Since the Civil Rights movement, society has tried to have a more egalitarian view of races (Dovidio & Gaertner, 2000;Sue et al., 2007). As a consequence, racism in its blatant or overt form is prohibited by law, to the extent that nowadays many can argue that racism does not exist and that it is not a problem in the US. However, contemporary researchers contest that idea by presenting studies that suggest the existence of forms of modern racism that are covert and subtle (Dovidio, Gaertner, Kawakami & Hodson, 2002).
Pearson, Dovidio and Gaertner (2009) developed the theory of aversive racism, which is defined as "a form of prejudice characterizing the thoughts, feelings, and behaviors of the majority of well-intentioned and ostensibly non-prejudiced White Americans (p.315)." Among these forms of racism, racial microagressions deserve special consideration. The term racial microaggressions (RMs) was first introduced by Chester Pierce in 1978 and was defined as "subtle, stunning, often automatic, and nonverbal exchanges which are 'put downs'" (Pierce, Carew, Pierce-Gonzalez & Willis, 1978, p. 66; as cited in Sue et al, 2007, p. 273).
Given their subtle nature, the occurrence and prevalence of RMs should be of special importance to mental health professions because of interactions with people of color who are clients and/or service providers. It is particularly important to investigate how racial microaggressions affect the daily life of Psychologists of color who serve a diverse population in the US and those who are currently in graduate school training about to join the field.
There are several studies that have evaluated RMs on university campuses, in the counseling process, and among faculty in university environments. For example, a study conducted by Sue and colleagues (2007)  Microassaults are conscious and intentional discriminatory actions characterized by a verbal or non-verbal attack with the intention of hurting the victim.
Examples of microassaults include using racial epithets, displaying White supremacist symbols (e.g. swastikas) or preventing one's son or daughter from dating outside of their race. Microinsults are verbal, nonverbal, and environmental communications that slightly convey rudeness and insensitivity aimed at demeaning a person's racial heritage or identity. An example of a microinsult is when an employee asks a coworker of color how she got her job, implying she may have landed it through an affirmative action or quota system. Microinvalidations are communications that subtly exclude, negate, or nullify the thoughts, feelings or experiential reality of a person of color. For example, a White person asking a Latino/a where they were born, conveying the message that they are perpetual foreigners in their own land.
The collection of personal narratives helped the researchers to code the information from the narratives and classify it into nine different categories of microaggressions with distinct themes; these are: (1) alien in own land, (2) ascription of intelligence, (3) color blindness, (4) assumptions of criminality or criminal status, Several studies have explored how racial microaggressions are experienced by different minority groups, including African Americans (Sue et al., 2008), Latina/os (Rivera, Forquer & Rangel, 2010), Asian Americans (Sue, Bucceri, Lin, Nadal, & Torino, 2010), indigenous people (Hill, Kim, & Williams, 2010, Clark et al., 2011, and students of color (Sue, Lin, Torino, Capodilupo, & Rivera, 2009). These researchers found that people of color experiencing different RMs in their everyday lives are subject to pervasive and negative impacts on their mental health (Nadal, 2011, Sue et al., 2008. People of color who experience microaggressions in their everyday lives are subject to pervasive and negative impacts on their mental health (Nadal, 2011;Sue et al., 2008). Previous studies confirm that perceived discrimination by African Americans is related to poor psychological outcomes (Clark, Anderson, Clark, & Williams, 1999). Specifically, a study by Kessler, Mickelson and Williams (1999) reported that 25% of African American participants in their study sample reported frequent day-to-day discrimination experiences. This finding suggests that African Americans dealing with discrimination experiences (such as RMs) on a daily basis are susceptible to negative influences on their psychological well-being (Torres, Driscoll, & Burrow, 2010). More recently, another study confirmed that RMs affect negatively the mental health of people of color (Nadal, Griffin, Wong, Hamit, & Rasmus, 2014a).
In the 2014 study Nadal and colleagues found that higher frequencies of RM events negatively predicted the mental health of participants. They also found a significant correlation between RMs and depressive symptoms and negative affect (Nadal et al., 2014a).
The literature suggests that the ambiguous and unconscious nature of RMs (sometimes for both the victim and the perpetrator) produces more pervasive effects on the psychological well-being of people of color than overt forms of discrimination (Solórzano, Ceja, & Yosso, 2000). Sue and colleagues (2007) (Solórzano, Ceja & Yosso, 2000;The Graduate Assembly, 2014, Yosso, Smith, Ceja & Solórzano, 2009). Therefore, the frequent exposure to RMs could, in turn, affect their retention and the completion of their graduate degrees.
Self-esteem is another area in which RMs are reported to have a negative impact. A study found that RMs negatively predicts lower self-esteem in people of color (Nadal, Wong, Griffin, Davidoff, & Sriken, 2014b). In other words, the more RM experiences the participants had, the lower they reported their self-esteem. The study also found that the RMs occurring in educational and workplace environments were particularly harmful to the participant's self-esteem.

Clash of Racial Realities
The racial realities of students of color are different from what their White counterparts experience. Studies have found that, for example, African Americans believe that racism is something they constantly have to deal with, while most White Americans tend to minimize and say that racism is a thing of the past (Sue, 2010). For example, when African Americans are asked how much discrimination still exists against them today most say "a lot", while only 10% of Whites said "a lot". Another study found that over 50% of Whites believe that people of color have gained equality and think that they are doing better than they really are, which is contradictory to the perceptions of people of color in areas such as employment, education, and housing opportunities (Harris Poll, 1994;as cited in Sue, 2010). Studies suggest that the gap between Black and White perceptions are astounding (Sue, 2010). Across African Americans, Asian Americans, and Latino/Hispanic Americans, there is agreement that White Americans believe they are superior, entitled to control others and insensate to race issues (Harris Poll, 1994;as cited in Sue, 2010, p. 45). In regards to racial discrimination and bias, there is a big gap in the perception of its existence between  (Sue, 2010). Additionally, he criticizes RMs by stating that everyone experiences verbal, behavioral or environmental indignities regardless of their race.
Even though all groups experience insult and slights in their lives, it is important to note that equating the experience of a political conservative with the experiences of racism is wrong (Sue, 2010). Thomas is imposing the race reality of White Americans, who historically have had more power, on those who have less power and have been marginalized. These realities are completely different with respect to choice. Whereas everyone can choose their political affiliation and decide or not to expose themselves to being offended, people of color cannot escape their realities; they are born with them and cannot change the color of their skin to avoid these experiences. In general, the perception by others of RMs as doing minimal harm is something people of color face frequently when deciding to discuss it (Sue, 2010).
Even though RMs vary in severity, and some may seem innocent, each one nevertheless contributes to the accumulation of racial indignities that can cause harm to people of color (Sue, 2010).
Another area where there is gap in perception in an academic interaction is between the faculty advisor and the graduate student of color. A qualitative study found that there was a difference in perceptions of professional advantage because of the race or ethnicity of the student. The study analyzed race as currency, which "referred to the social value placed on one's race," whether it was a benefit or a disadvantage (Barker, 2011, p. 393). In this case, the faculty advisors, who identified as White Europeans, viewed their students' race mostly as an advantage. However, some advisors expressed concerns for their students that in their future jobs they might not be taken seriously because of assumptions that they got the job because of their race. Conversely, graduate students of color (African American) perceived their race as only a liability and not a benefit for their future academic careers (Barker, 2011). In other words, students of color felt that they constantly have to prove themselves in their academic careers more than does a student from the majority race. The perceptions of the majority race faculty and peers on this issue are dramatically different and sometimes invalidating.

Racial Microaggressions in Academic Settings
A study of RMs in academic settings found that African American participants Frequent RMs in academic settings contribute to the perception of an unwelcoming and hostile campus climate. Many students of color have reported feeling invisible, due to their experiences as African Americans being omitted, distorted and stereotyped in their classes (Solórzano, Ceja, & Yosso, & 2000, p. 65).
The African American participants in the study also felt that faculty maintained low expectations of them, and that regular negative interactions have made them doubt their own abilities and intelligence. Participants also felt isolated, especially when others did not consider them to be part of study groups. The effects of dealing with all these RM experiences left them feeling drained and mentally exhausted because they have had to constantly prove themselves in the academic setting. The study also found that in social spaces within and around the campus, participants experienced more overt racism, rather than more covert and subtle forms of racism in academic settings (Solórzano, Ceja, & Yosso, & 2000). Thus, frequent experiences with RMs of students of color could produce negative perceptions of the campus racial climate. The effects of RMs in students of color are deleterious, affecting their mental health, self, esteem, and interfering with their academic performance.
RMs also affect the academic engagement and sense of belonging of students of color (Clark, Mercer, Zeigler-Hill, & Dufrene, 2012). In one study, Clark and colleagues (2012) evaluated the factors that could be barriers to the success of ethnic minority graduate students in the field of School Psychology. Specifically, these researchers assessed academic achievement and social and emotional experiences (belongingness and emotional distress). They found that ethnic minority students experienced a higher level of emotional distress, a lower level of belongingness and more negative race-related experiences with lower perception of belongingness (Clark et al., 2012). Therefore, if students feel that professors and peers do not socially support them, RMs could negatively affect their psychological adjustment.
Furthermore, these negative consequences could hinder the necessary efforts aimed at promoting academic achievement and, consequently, retention within the graduate program/university (Clark et al., 2012;Solórzano et al., 2007).

Retention and Attrition of Students of Color
The US is becoming more diverse and there is a need for more psychologists and other mental health professionals of color to represent this growing diversity. Meanwhile, doctoral ethnic minority students comprised 20.1% of students in APA programs. This contrast between APA student and regular memberships could suggest that ethnic minority graduate students are either leaving graduate programs or that some of them might not necessarily choose to join the field following graduation.
Also, there is a notable decline in the participation of ethnic minorities in postdoctoral fellowships that might signal a decline in the number of future ethnic minority psychology faculty and researchers (APA, 2008). Therefore, there are fewer ethnic minorities in doctoral programs, postdoctoral programs, and faculty/research positions.
Consequently, this difference could suggest that there is a problem with retention of graduate students of ethnic minority backgrounds in psychology. The higher percentages of ethnic minority students in graduate programs may also predict a higher percentage of regular APA members in the future (APA, 2008). Although, a greater recruitment does not necessarily mean a substantive increase of ethnic minorities completing their degrees (Griffin, Muniz, & Smith, 2016). On the other hand, even 25 or 30% people of color representation in psychology may quickly fall short of the corresponding percentage in the general population.
While it is important to continue the efforts of increasing recruitment of ethnic minorities in graduate programs, there is a great need to also promote equity in their education outcomes and the quality of their experiences, as well as their retention (Griffin, Muniz, & Smith, 2016). For this reason, it is critical to increase and retain graduate students of color in psychology and other mental health fields who could better serve the evolving population of children in the schools and clients generally.
Generally, graduate student retention is problematic, especially in doctoral programs with only 57% of students completing their degree across disciplines (Council of Graduate Schools, 2008). The report by the Council of Graduate School (2008) showed that the rate of completion of doctoral programs in social sciences is 56%. More specifically, the same report found that in psychology doctoral programs only 65% of students complete their degree. Thus, 35% of students who enter a doctoral program did not attain the degree. The main reasons for dropping included student-program mismatch, program difficulty, absence of financial support, and lack of community support within departments and campuses (Wojcik, 2012).
Unfortunately, there was no specific data from APA available to me that depicts the number of students of color who might have dropped down to a master's degree or decide to leave their graduate programs altogether. However, The Council of Graduate Schools (2008) found that within the area of social sciences, the lowest rate of doctoral degree completion was among Asian students (44%), followed by African American students (47%), Hispanic students (55%) and White students (57%). At a glance, the numbers suggest that all racial groups complete their degree at comparable rates, however, it is important to reiterate that the rate of recruitment is much less for students of color. Another study by Maton, Kohout, Wicherski, Leary, and Vinokurov (2006) found no growth in the percentage of PhD degrees received by students of color since 1999, and that the growth of African American and Hispanic/Latino(a) students showed little to no growth since 1997. Similarly, they found that faculty of color in psychology is low and this trend has not changed considerably (Maton et al., 2006). Thus, the recruitment of graduate students of color has not changed much in almost 20 years, and these students also are more likely to leave their programs than their counterparts (Rogers & Molina, 2006).

Campus Climate and Attrition
Racial and ethnic minority students are at greater risk for attrition due to higher negative experiences with departmental integration and socialization, access to financial resources, interactions with faculty, and racial climate, among other variables (Griffin, Muniz, & Smith, 2016 Johnson et al., 2014). In turn, these negative race-related experiences, including RMs, affect psychological processes and persistence in their degree programs (Johnson et al., 2014). In other words, the decision to stay in a program is negatively affected by the hostile campus climate. Additionally, a study by Wei, Ku, & Liao (2011) and Johnson et al., (2014) reported that for students of color at PWIs a unique form of stress that they experience with more frequency was race-related stress in their academic environment, which had negative effects on their degree persistence decisions. Thus, if a campus climate is supportive and positive towards students of color it can lead to better student outcomes and persistence. In contrast, a negative campus climate towards students of color may be associated with poor academic performance and high dropout rates, particularly among African Americans students (Solórzano, Ceja, & Yosso, 2000). For this reason, it is important to pay attention to the racial campus climate and take steps to monitor the frequency of RMs and create interventions.

Measuring Racial Microaggressions
The majority of the research examining the occurrence and prevalence of RMs is qualitative, given that RMs are a relatively recent topic of study. Recently, there has been a move for more quantitative studies, mostly in the field of scale development, to measure RMs. Currently, there are three scales published in peer-reviewed journals.
The first published scale was the Inventory of Microaggressions against Black Individuals (IMABI) (Mercer, Zeigler-Hill, Wallace, & Hayes, 2011). The IMABI was developed and validated using a university sample, where Black or African American undergraduate students answered a 14-item scale capturing both microinsults and microinvalidations, but highly focused on the latter (Mercer et al., 2011). The measure was associated with general distress and perceived stress and had good reliability (r = .79).and validity. Another measure that was developed shortly after by Nadal (2011)  The scales described above appear to have good reliability and validity to measure RMs for both general community sample and college samples. However, to the best of my knowledge, there is no RM scale that measures the occurrence or prevalence of microaggressions in academic settings and addressing academic-related activities for students of color, particularly in psychology. The literature has found that the most frequent setting where people of color experienced more RM was in the school setting or the workplace (Nadal et al., 2014a). Thus, a special focus should be provided and for that reason a specific measure of RM in the school setting and workplace is needed. The lack of such a scale is a major limitation to academic achievement efforts since RMs could pose a serious menace to retention and academic success. For that reason, the purpose of this study is to develop and validate a scale measuring RM experienced by graduate students of color in psychology and other related fields that have a required practicum or field component. The present study will extend prior work by the author on RMs by developing and testing a quantitative measure to assess the themes found in previous research (Ortiz-Frontera, 2013, see Appendix 1). Most importantly, the goal of developing this scale is to assist university programs or departments in psychology and related fields in assessing the types of RMs and settings where RMs may occur, and consequently begin prevention and intervention efforts. In order to obtain a clear description of the problem, there is a need for a measurement tool that assesses the specific and unique RM experiences among graduate students of color in academic settings. The results of this study provides valuable information that could assist in the creation of interventions tailored by race to support graduate students through the completion of their graduate degrees.
The new scale, ARMAS, can be utilized by departments to monitor RMs across time and evaluate progress or problems in the racial climate.
For the current study, the following research questions have been developed: (1) What are the psychometric properties of the newly developed measure ARMAS?
And, (2) are RM experiences a factor in graduate student consideration of leaving their graduate programs?

Scale Development
The purpose of this stage of the study was to develop items that would create the Assessment of Racial Microaggressions in Academic Settings (ARMAS) scale.
The following discussion provides a description of the item creation process, including a discussion of the previous study the author conducted where written responses of students of color in psychology were utilized to assist in creation of items for the ARMAS scale (Ortiz-Frontera, 2015). In addition, the process of an expert review of the items, and a small pilot study are described.

Item Development
The Next, the openended questions also asked participants to describe the ways they coped with their RM experiences in the specified settings. Then, participants' responses were qualitatively coded into themes based on Sue's RM theory (Ortiz-Frontera, 2015). The specific responses and the themes that arose from this data assisted in the creation of 47 items for the ARMAS. The initial items were then reviewed by an expert panel to assess item quality, face validity and content validity.

Expert Panel Review
The expert panel consisted of faculty who are the committee members of the author. The expert panel included three faculty members. One was a female professor and researcher, an expert on human development and multicultural issues on college campuses. She has studied RM and is very knowledgeable of the theory proposed by Sue and colleagues . Another was a male professor with research expertise on peace and nonviolence and social psychology. The third member was a male professor expert on nonviolence training and school psychology. The panel reviewed all items of the ARMAS to assess item quality, face validity and content validity.
After receiving panel feedback related to the need for more items, 25 more items were added. Also, the existing items were clarified and modified. This process happened again to review the new items. Based on the final feedback, a total of 72 items were created.

Pilot Study
A small pilot study (N = 17) was conducted in order to have a better idea of how the newly developed measure would fare with a sample from local state graduate programs in Psychology and Social Work. The purpose of this pilot study was to assess the items' wording and clarity, decide upon scale length, and delete weak items.
Also, the pilot study provided feedback on the format of the Likert categories. Even though six items were deleted at first, with changes in phrasing the researcher decided to keep three of those items. Then, two more items were added based on the feedback provided by participants and consistent with themes based on theory. This resulted in 71 items on the ARMAS for the validation phase. More details about the characteristics of the sample are provided below.
The majority of the participants spoke English as their first language (n = 14, 82.4%). The three remaining participants spoke another language as their primary tongue; these included Icelandic (n = 1, 5.9%), Turkish (n = 1, 5.9%) and Spanish (n = 1, 5.9%). The highest degree completed from participants was a Master's degree (n = 10, 58.8%), followed by Bachelor's degree (n = 7, 41.2%). Most of the pilot study participants were enrolled in a Doctoral program (n = 12, 70.6%), followed by a Master's program (n = 5, 29.4%). Participants were mostly in the first (n = 6, 35.3%) and fourth year (n = 5, 29.4%) of their programs; the remaining participants were in second and fifth year, each representing 17.6% of the sample.
The number of faculty of color in the participants' programs ranged from zero to "four or more." The largest group (47.1%) reported that in their program there were no faculty of color; 23.5% reported two faculty of color; and 17% reported one. Only one participant reported having four or more faculty of color in their graduate program. Similarly, students reported the number of graduate students enrolled in their programs. The majority of participants (64.7%) reported having between four and six graduate students of color in their programs, followed by 17.6% each reporting having from seven to nine or 10 or more graduate students of color in their programs.

Measures
The The questions were adapted from Nadal (2011), and are the following: " (1) Please describe what you believe these questions were trying to measure, (2) Please write three keywords or key phrases that can be used to label the various experiences that are described above., and (3)
Few participants decided not to disclose their race (n = 5, 1.7%). In the validation sample, the majority of participants spoke only English (n = 147, 62%).
Only 43.1% reported speaking a language other than English as their primary language.
The sample also had participants of color who graduated from their programs less than a year previously (n = 14, 5%). In the sample, there were also first generation graduate students (n = 111, 39%). The highest degrees attained in the sample were a these include: zero (n = 3, 1%), one to three students (n = 56, 19.4%), four to six (n = 83, 29%), seven to nine (n = 46, 16%), 10 or more students of color (n = 80, 28%), and 25 participants or about 9% of the sample were not sure the number of graduate students of color enrolled in their programs. The majority of participants reported that their program was located in an urban area (n = 210, 80%), followed by a rural area (n = 49, 17%), and suburban (n = 29, 10%).

Measures
Similarly to the pilot study, the measures used in this study consisted of a packet of questionnaires including a consent form, the newly developed ARMAS

Racial and Ethnic Microaggressions Scale (REMS). The scale developed by
Nadal (2011)  The Aggression Questionnaire. The scale was developed by Buss and Perry (1992). The Aggression Questionnaire has 29-items and was administered to participants in order to evaluate discriminant validity with the new scale ARMAS.

Procedure
Similar to the pilot study, an invitation to participate in the validation study was sent to the psychology and social work graduate programs around the US.

Exploratory Factor Analysis
The internal structure of a new scale is usually examined by conducting an exploratory factor analysis (EFA). This EFA helps the investigator to determine the number of latent variables on a set of items, explain the variability between the items that will later create factors, and assist in defining the meaning of the factors that are accounting for the variation in the new instrument (DeVellis, 2012).
First, to evaluate whether the data was suitable for a factor analysis the sample size had to be considered. It is noted that there is a lack of agreement in the literature regarding the right sample size (Williams, Brown & Onsman, 2010). However, studies have found that in the majority of cases, using a sample size of 150 participants should be adequate to assess EFA accurately (Guadagnoli & Velicer, 1988;as cited in Hinkin, 1995). The ARMAS scale had N = 289 observations, so the sample size is deemed suitable for factor analysis.
Before running the EFA, two tests were done to evaluate the suitability of the data for factor analysis (Williams, Brown & Onsman, 2010). These tests were the Kaiser-Meyer-Olkin (KMO), a measure of sampling adequacy, and the Bartlett's Test of Sphericity (Williams, Brown & Onsman, 2010). The KMO index of 0.50 or larger is considered suitable for factor analysis (Williams, Brown & Onsman, 2010). For this study, the KMO had an index of 0.90, suggesting that the sample in this study is adequate for factor analysis. Moreover, it indicates that the extracted variables will account for a substantial amount of the variance. Then, the Barlett's Test of Sphericity was significant (p < .0001), indicating suitability for factor analysis.
Next, the EFA was run specifying the extraction methods as Principal axis factoring (PAF). In addition, since the factors are correlated it was determined that Oblique rotation was an appropriate method, specifically Promax rotation (Furr & Bacharach, 2008). Choosing a rotation assists by providing a way of presenting the results in a manner that is easier to interpret (Williams, Brown & Onsman, 2010). This extraction method was chosen because Principal components analysis (PCA) is usually recommended when no priori theory exists, and this validation study is based on the RM theory proposed by Sue and colleagues (2007).
Furthermore, the criteria utilized for factor extraction were based on Thompson and Daniel (1988), where multiple decisions or criteria were reviewed to reach a decision on the number of factors to extract. The first criterion that was analyzed was the eigenvalues that were greater than one. Based on this criteria, the ARMAS had 16 factors, (See Appendix J). Secondly, the Scree Test plot was analyzed (Tabachnick & Fidell, 1989). The Scree Test consists of identifying the point the slope starts to become flat (Furr & Bacharach, 2008). This test is subjective and it requires the researcher's judgment. In this case, it was determined that the slope became flatter between factors eight and nine (see Figure 1). However, another extraction technique was used called Parallel Analysis (PA) to confirm the number of factors. This technique is described as more thorough and one of the best methods for deciding amount of factors for extraction, although underused because of its limited availability on popular statistical programs (Williams, Brown, & Onsman, 2010 confidence that the eigenvalues extracted will be not due to chance; see Table 1 below.
Based on this PA procedure, nine factors would qualify. However, the factor nine difference was very small and a decision to keep eight factors was confirmed. The final ARMAS-47 factor structure is shown in Table 2.
The PA determined with confidence that for the ARMAS eight factors will be extracted explaining 65% of the variance. In addition, to survive item deletion the items needed to have a loading of .40 or higher on one factor (Furr & Bacharach, 2008). Based on this criterion, 24 items were deleted. The process of item deletion was done deleting one item at a time to see carefully how deleting one affected the other items. After this process of item deletion, the final ARMAS consisted of 47 items (see Appendix I).

Factor naming
It was found that the ARMAS had a multidimensional structure with eight factors (see Appendix I. Higher scores on any of these factors suggest that the participant is having higher frequency of RMs related to a specific area. The first factor is called Ascription of intelligence and consisted of nine items accounting for 32% of the total variance.
The items described experiences where perpetrators were surprised by participants' capabilities and/or their intelligence was questioned. Factor two, or Assumption of being a foreigner, consisted of nine items as well, accounting for 9% of the variance.
The items were related to the assumption that an ethnic minority must be a foreigner, or an alien in own land issue. The third factor was called Multicultural issues seen as not a priority and being treated differently, which had eight items accounting for 5.4% of the variance. The items in factor three described instances where the multicultural issues were seen as a waste of time and other items related to being treated differently by faculty because of race/ethnicity.

Reliability
The Cronbach's alpha reliability coefficients or internal consistency were

Convergent Validity
The Racial and Ethnic Microaggressions Scale (REMS) (Nadal, 2011)  Only one association was not significant, the correlation between ARMAS factor seven, Assumptions of professional advantage because of race/ethnicity and the REMS_B Second-class citizen and assumption of criminality subscale (r = .032, p > .05). This non-significant correlation could be due to the characteristics of the sample being graduate students in academic settings. Perhaps they experience less assumptions of criminality due the nature of the academic setting, which could be different in this respect from a community setting.

Discriminant Validity
The

RACIAL MICROAGGRESSIONS ASSESSMENT USING ARMAS
The purpose of this section is to evaluate how the ARMAS-47 performs as a measure of RM in academic settings and to compare it with a demographic variable, in this case race, to answer the following question: Is there any significant difference between the ARMAS-47 scores across racial groups? In addition, are these RM experiences impacting graduate student consideration of leaving their graduate programs?
The results on this section were analyzed by looking at average scores of the ARMAS-47 factors by race groups. This data is illustrated in Figure 2. These results suggest that for factor one (Ascription of Intelligence), American Indian and Alaska Natives scored the highest, followed by Black participants and those who declined to identify their race. Those participants who declined to identify their race, most were international students and identified their ethnicity (n = 5), African, Moroccan, Arab and two Arab Americans. For factor two (Assumptions of being foreigner), participants who declined to identify their race had the highest scores, followed by Asian and Hispanic or Latino/a participants. On factor 3 (Multicultural issues seen as not a priority and being treated differently) have higher scores for American Indian and Alaska Native, then Asian and Black participants. Next, factor 4 (Invisibility/Felt ignored) showed higher scores of participants who chose to not disclose their race, followed by American Indian and Alaska Natives and Asians. For Factor 5 (Assumptions about me and my work with clients/representing entire race), the ARMAS-47 scores were higher for American Indian and Alaska Natives, and then Hispanic or Latino/a and Asian participants. Colorblindness is the theme for factor six, and the racial groups that rated it higher were American Indian and Alaska Natives, Asian and Black participants. Factor seven (Assumptions of professional advantage because of race/ethnicity), were endorsed more often by Hispanic or Latino/a participants, followed by Asians and American Indian and Alaska Natives. Finally, for factor eight (Stereotypical assumptions about my race/ethnicity), scored the high among American Indian and Alaska Native, followed by participants who declined to identify race and Asians in the sample.
Furthermore, these findings suggest that factor five (Assumptions about me and my work with clients/representing entire race), was the factor with the higher scores across the race groups with the exception of those participants who declined to identify their race and Hispanic or Latino/a participants. Even though, participants endorsed more this factor five, there was not a significant difference as seen below in the ANOVA results, due to probably the large variability of responses.
Moreover, Hispanic or Latino participants scored the highest on Factor 7 (Assumptions of professional advantage because of race/ethnicity). These high scores are both in factors that are unique of this study, and specific to RMs in academic settings in psychology. White participants' scores were consistently lower than those of participants of color. The data is in agreement with previous findings that White individuals experience with RMs is different than those of people of color (Sue, 2010). Multicultural issues seen as not a priority and being treated differently (4) Invisibility/Felt ignored, (5) Assumptions about me and my work with clients/representing entire race, (6) Colorblindness, (7) Assumptions of professional advantage because of race/ethnicity, and (8) Stereotypical assumptions about my race/ethnicity.

One-Way Analysis of Variance (ANOVA)
A one-way Analysis of Variance (ANOVA) was calculated on participants' ARMAS subscale scores to answer the question whether there are differences on microaggressions by race as measured by the ARMAS-47. Overall, the analyses were To determine which race(s) and factors were different, Post hoc Tukey HSD tests were conducted. Results indicated several significant differences at p < .05. Only the significant mean differences were included in Table 4. Note. Data represents a Post hoc Tukey HSD summary of multiple comparisons of significant groups with p < .05.
The results on this section were analyzed by looking at average scores of the ARMAS-47 factors by race groups in this subsample (n = 145) of participants who indicated they have had drop out considerations. This data is illustrated in Figure 3.
Taking this subsample, it was found that for factor one, (Ascription of Intelligence), the higher scores was endorsed by American Indian and Alaska Natives, followed by Hispanic or Latino/a, and Black participants. For factor two, (Assumptions of being a foreigner), higher scores were reported by Asian, Hispanic or Latino and then American Indian or Alaska Native participants. Factor three (Multicultural issues seen as not a priority and being treated differently), scores were higher for American Indian and Alaska Natives, followed by Asian and Hispanic or Latino/a participants. Next, factor four (Invisibility/Felt ignored) participants who had higher scores were Asians, American India and Alaska Native, followed by Blacks. Factor five (Assumptions about me and my work with clients/representing entire race) was endorsed more by American India and Alaska Natives, which was the highest score in this subsample, followed by Hispanic or Latino/a and Asians. The following factor six (Colorblindness) had higher scores among American India and Alaska Natives, Asian and Black participants. Then, factor seven (Assumptions of professional advantage because of race/ethnicity) scores were higher among Hispanic or Latino/a participants, followed by Asians and Blacks. Finally, for factor eight (Stereotypical assumptions about my race/ethnicity) participants who identified as American India and Alaska Natives had higher scores, followed by Asian and Hispanic or Latino/a.
Comparably to the whole sample results, factor five (Assumptions about me and my work with clients/representing entire race) appears to be the factors that across race groups was rated the highest, except for Hispanic or Latino/a participants who endorsed more frequently items in factor seven (Assumptions of professional advantage because of race/ethnicity). This data suggests that the graduate students in this sample experience RM events more frequently related to those RM that are unique to this study of and their specific experiences in academic settings. Post hoc Tukey tests suggests that there are significant differences, by races on those factors that were significant. Results indicated several significant differences at p < .05. For the purpose of this paper, the significant mean differences were included in Table 5. Additionally, we asked participants to choose what reasons they had when they were considering dropping gout of their programs. To see the difference among race groups and their reasons to dropout see Figure 4 below. Results in Figure 4 indicate that White participants' tope three reasons to dropout were similar to Asian participants, both reported being overwhelmed about academic demands, lack of support from faculty, and lack of confidence in their own abilities. Black participants reported that top three main reasons to dropout was lack of support from faculty, not having enough professors of their race, and microinvalidations. Hispanic or Latino/a graduate students reported the main reasons to be lack of support from faculty, lack of confidence in their own abilities, and overwhelmed about academic demands. Participants who were American Indian or Alaska Natives reported their reasons to consider dropping out of their programs included lack of support of faculty, lack of confidence in their own abilities and microinvalidations. Lastly, Multiracial participants reported their reasons to dropout as being overwhelmed about academic demands, lack of confidence in their own abilities and microinvalidations.
In terms of RMs, these results are in agreement with previous findings where people of color tended to experience more frequent events related to microinvalidations and then microinsults, especially in the school or workplace settings. Microinvalidations were part of the top three reasons to dropout for Blacks, American Indian and Alaska Natives, and Multiracial graduate students.

GENERAL DISUSSION
The term Racial Microaggressions is defined as "brief and commonplace daily verbal, behavioral, and environmental indignities, whether intentional or unintentional, that communicate hostile, derogatory, or negative racial slights and insults to the target person or group" (Sue et al, 2007, p.273). Microaggressions exists in both social and academic collegiate environments (Solórzano, Ceja and Yosso, 2000). For that reason, it is important to explore RM in academic settings and ways to support graduate students of color while completing their academic degrees in order to continue diversifying all levels of education. Reliability along with discriminant and convergent validity was analyzed.

Summary
Results of the study suggest that the ARMAS is a valid and reliable measure of RMs in academic settings.
Further results suggest that half of the sample experienced RMs and considered dropping-out of their programs more frequently during the first three years of their graduate programs. Furthermore, the findings suggest that factor five (Assumptions about me and my work with clients/representing entire race), was the factor with the higher scores across the race groups with the exception of those participants who declined to identify their race and Hispanic or Latino/a participants. Additionally, Hispanic or Latino participants scored the highest on Factor 7 (Assumptions of professional advantage because of race/ethnicity). These high scores are both in factors that are unique of this study, and specific to RMs in academic settings in psychology. White participants' scores were consistently lower than those of participants of color. The data is in agreement with previous findings that White individuals experience with RMs is different than those of people of color (Sue, 2010).
Looking at the reasons students had when they considered dropping out of their programs suggest that the top main reasons for White, Asian and Hispanic or Latino/a students included: being overwhelmed about academic demands, lack of support from faculty, and lack of confidence in their own abilities. However, Black participants reported lack of support from faculty, not enough professors of my race and microinvalidations. One of the top three reasons for dropping out of their graduate programs for Black, American Indian/Alaska Native, and Multiracial graduate students was microinvalidations.

Limitations
There were several limitations. The data collection was online and participants were not supervised, there was no opportunity to make sure they were in fact graduate students of color in psychology. Also, the study required self-report of their experiences with RMs and these are difficult to assess because of different factors such as being aware of RM and the capacity of recall of RM events (Ortiz-Frontera, 2013). Additionally, the ARMAS-47 needs further evaluation of its psychometric properties and check replicability and consistency. There was also variability in the responses leading large standard deviations. A more interdisciplinary sample is needed so the ARMAS could be tested for validity of RMs in different fields and academic settings.

Implications
The ARMAS measure could serve as an initial assessment that will help recruiters, faculty, and program directors who work in higher education to assess RM experiences of their students. This will foster awareness of RMs and consequently will aid in the creation of prevention and intervention strategies to minimize harmful RM experiences. The information from the measure will aid in tailoring of coping strategies by race. Research suggests that all students of color will benefit from emotional support, but in addition to that Black students successfully coped with RM using religion and spirituality strategies (Ortiz-Frontera, 2013). In addition, American Indian and Alaska Native appeared to experience frequent RMs and more research on RMs with this racial group is warranted to understand their specific experiences (Hill, Kim, & Williams, 2010).
Faculty can employ the findings to help create mentoring programs or support groups for students of color to foster retention and graduation. Moreover, this RM measure is important because it will provide graduate students of color with validation of their experiences and may help to create ways that they can positively cope with RM. Another important implication of this RM scale tailored for academic settings in psychology and related fields, is that it will help the program to assess and reinforce its multicultural training at the program level and possibly throughout the campus and across time to track progress (APA, 2008). While doing this, departments of psychology will promote acceptance of and a welcoming environment for graduate students of color, enhancing their confidence, self-efficacy and academic engagement in order to succeed and complete their graduate degree and eventually join their field.
In order to foster a positive and welcoming environment to students of color in academic settings, students color would benefit from creating counter spaces where they can be involved with other graduate students of color going through similar experiences to obtain emotional support and peer mentoring (Ortiz-Frontera, 2013;Solórzano, Ceja, & Yosso, 2000). For example, creating a graduate students of color organization or student network managed by students and possibly overseen by the graduate school diversity officer could be a safe counter space for students coping with RMs (Good, Halpin, & Halpin, 2000;Grier-Reed, 2010). Building community within the department and campus is important for students to feel welcome and valued (Yosso, Smith, Ceja, & Solórzano, 2009).
It is important to provide and encourage access to mental health counselors on campus so graduate students of color have a safe place to vent and cope with possible depressive symptoms (The Graduate Assembly, 2014). A systematic training of faculty on effective mentorship practices is needed to better the graduate experience and reduce the risk of attrition (Griffin, Muniz, & Smith, 2016).

Future directions
Future directions are to continue to explore more carefully the validity of the People avoid discussions about race/ethnicity in the classrooms 3. People avoid discussions about race/ethnicity in social or informal events 4. People avoid discussions about race/ethnicity in my practicum placement 5. There is a lack of diversity in my program 6. People say that focusing on multicultural and diversity issues is a waste of time 7. People say that there is too much emphasis on racial/ethnic minority issues 8. The majority of the classroom discussions are from a white perspective 9. I have heard people say that people of color are overly sensitive 10. People assume that I got a scholarship or a graduate assistantship only because of my race/ethnicity 11. People say that I will find a job quickly because of my race/ethnicity 12. People say that I'm lucky to be bilingual 13. People say that I will be paid more on my job because I'm bilingual o Alien in own land a. Asked to represent entire race 1. I am often asked my perspective on things because of my race 2. People assume that I'm not from the US 3. People assume that I don't speak English 4. People ask me from what country I'm from 5. In classroom discussions people ask me what I think about a topic related to race/ethnicity b. Exoticization and Assumption of Similarity 1. I get confused with other students who are from a similar race/ethnicity background 2. People often comment that I don't look like others from my race/ethnicity 3. People ask me to dance or do things assumed to be associated with my race/

2011)
(1) Describe what you believe the last questions were trying to measure.
(2) Write three keywords or key phrases that can be used to label or explain the various experiences that are described above.
(3) Do you remember any questions or experiences that were not written in a clear or concise manner? If so, please list them. 4. Describe the reasons you had when you contemplated the possibility of leaving the graduate program. 5. Describe the reasons why you decided to stay in your graduate program. 6. Describe how you coped with having thoughts of leaving the graduate program.

Buss-Perry Scale and its subscales (Buss & Perry,1992)
Please rate each of the following items in terms of how characteristic they are of you. Use the following scale for answering these items.