HISTORICALLY MARGINALIZED GROUPS IN PSYCHOLOGICAL SCIENCE

Over the past thirty years there has been a concerted effort to evaluate the inclusion of historically marginalized groups (HMG) – women, racial, ethnic, and sexual minorities, and low-income individuals – in research. This has been done through content analyses of research literature published in top-tier psychological journals. The purpose of this study was to examine the research literature to assess the degree to which the current literature includes ethnic, racial, and sexual minorities, women, and those of varying socioeconomic status and whether research questions focusing on historically marginalized groups are being examined within mainstream journals. Six issues from 2012 from five top-tier APA journals were content analyzed for inclusion of HMG and focus on HMG. Additionally, 148 authors who published articles in one of those journals were anonymously surveyed about the importance and relevance of HMG to their research, and the factors that influence their actual practices in conceptualizing, designing, and conducting research on HMG. A cumulative 10.65% of articles had a focus on HMG, while reporting of demographic characteristics differed greatly by journal and characteristic. Journal authors indicated gender was the most important (of race and ethnicity, sexual orientation, gender, or socioeconomic status) to answering their research questions, and were most likely to specifically target men or women when enrolling participants. Authors indicated many barriers to enrolling HMG in research. While considerable work is still to be done, the author survey indicated that many early career researchers are doing research focused on historically marginalized groups at least some of the time. Barriers ranging from funding, to publication biases, to difficulty in recruiting participants, were some of the various barriers that need to be addressed. iii" " ACKNOWLEDGEMENTS I wish to thank my Major Professor, Dr. Kathleen Gorman, as well as my committee members; Drs. Jasmine Mena, Kyle Kusz, Margie Rogers, and committee chair Dr. Susan Hannel. Additionally, I would like to thank Dr. Michael Stein for his encouragement and mentorship over the years. This achievement would not have been possible without the support of my family, especially my parents, and in-laws, and friends. I am especially grateful to Dr. Patty Castle for her help understanding the Census data. And finally, I thank my wife, Dr. Alexis Lamb, who is my inspiration and support in all endeavors.

For the purposes of the current research the following definitions will be used when referring to historically marginalized groups: 1) Racial and ethnic minorities: Black or African American, Hispanic or Latino/a, Asian, Pacific Islander or Native Hawaiian, Native American or Alaska Native, Bi-racial, or Non-white; 2)Women: Anyone who identifies herself as a woman or female ; 3) Sexual Minorities: Lesbian, gay, bisexual, transgender, queer, or non-heterosexual identity; 4) Low-SES: Low income, low education, low employment or unskilled laborers, low social class individuals.
Any discussion of historically marginalized groups in psychological research should begin with an acknowledgement that there is no singular definition of race, ethnicity, socioeconomic status (SES; an attempt to understand ones' social and economic position, generally measured using some combination of education, income, and employment), sexual orientation, or gender, the variables of interest here, as they are socially derived constructs and have meant different things across time and place.
These variables were chosen because of the complex ways these variables interact with psychological phenomena and with each other. Additionally, prior content analyses were likely to identify and examine at least one of these variables, although not all four (Cundiff, 2012;Graham, 1992;Hunt, Jackson, Powell, & Steelman, 2000;Imada & Schiavo, 2005;Raad, Bellinger, McCormick, Roberts, & Steele, 2008;Ram, Starek, & Johnson, 2004).
Additionally, "top-tier" APA journals are especially important to consider in the context of this research. Based on impact factor and prestige both within and 5" " outside the field, some journals are held up as the best research psychology has to offer and perhaps most likely to be read or cited most widely outside of the field.
Historically, publishing research on historically marginalized groups in top-tier journals has been challenging and thus it is those journals that are of interest in this inquiry. In a study of cross-cultural and ethnic minority psychology between 1993 and 1999, most articles focused on historically marginalized groups were published in specialty journals, not prestigious, mainstream psychology journals (G. C. Hall & Maramba, 2001). In Graham's review (1992) 17-37.5% of articles focused on African Americans were published as brief reports, not afforded the space of a full research article.

Background, Definitions, Transitions.
Race and ethnicity, gender, social class, and sexual orientation are not new areas of research interest in psychological science. In fact, dating back to the late 19 th century researchers were conducting studies on differences between what, at the time were perceived to be biological differences between races. Much of this early work, on eugenics and differences between "races", especially as it pertained to intelligence testing, a focus on deficits in some groups compared to others, and physical feature measurement, laid the foundation for the racist science we still combat today (Guthrie, 2004;Richards, 2004) "Psychology is at a critical junction in its lifecycle; it can adapt to the changing demographics of the United States or risk obsolescence" (C. C. I. Hall, 1997, pg 650).
Although this was written in 1997, it remains just as true today. The United States is experiencing rapid changes in the racial and ethnic composition of its population, the 6" " number of individuals living in poverty, and the acceptance of gay, lesbian, and transgendered individuals. The reality of rapidly changing demographics highlights the need for psychology, as a field, to conduct more inclusive science, leading to a more inclusive, diverse, and generalizable research literature base.
Advances in ethnic minority and multicultural psychology, which were the beginnings of the current movement toward more representation of historically marginalized groups in psychology, came initially due to the extraordinary efforts of a group of pioneering psychologists (Franklin, 2009;S. Sue, 2009). Mainstream contemporary psychology had long ignored the influences of culture, race, ethnicity, gender, social class, religion, or other aspects of identity on psychological phenomena (Miranda, Nakamura, & Bernal, 2003;Reid, 1993), instead assuming that the experiences of white, middle-class males generalize to all individuals. Without efforts to establish ethnic psychological associations, advocate for an end to racist practices in research, better inclusion of racial and ethnic minorities in graduate schools and American Psychological Association (APA) governance, and practical training that is culturally competent, the idea of Multicultural Psychology as a subdiscipline, and diverse research practices more generally, may not exist today (Franklin, 2009;S. Sue, 2009). However, as we move forward as a field, the task of laying the groundwork for better scientific research practice for future generations should not rest solely on the shoulders of those researchers who are racial or ethnic minorities, women, gay, lesbian or transgender, or who choose research related to historically marginalized groups.
Scientific best practice is the responsibility of all (Betancourt & López, 1993;Carnes, 7" " Morrissey, & Geller, 2008; C. C. I. Hall, 1997;Hyde, 1994;Reid, 1993;2002;Uhl, Parekh, & Kweder, 2007). Article Reporting Standards, 2008;Federal Register, 1994). In 1994 the National Institutes of Health (NIH), a major source of research funding instituted a policy requiring the inclusion of women and minorities in all clinical trials involving human subjects (Federal Register, 1994). The policy, which has been updated over the years, also provides that in all Phase III clinical trials (which include behavioral intervention trials) sufficient numbers of women and minorities must be enrolled to conduct subgroup analyses. Geographic location and cost are not appropriate reasons for failing to fulfill the requirements. In a study of NIH Scientific Review Group members, overwhelming majorities felt the guidelines were partly responsible for study sections' attention to inclusion of women and minorities (Taylor, 2008). Over half of those surveyed felt inclusion had increased as a result of the guidelines.
The APA Guidelines on Multicultural Education, Training, Research, Practice, and Organizational Change for Psychologists (American Psychological Association, 2003), offers the following guidance for researchers: "Culturally sensitive psychological researchers are encouraged to recognize the importance of conducting culture-centered and ethical psychological research among persons from ethnic,

8"
" linguistic, and racial minority backgrounds" (pg. 388). The guidelines go on to describe the implications of this advice in all phases of research, from generation of the research question, assessment, to analysis and interpretation of data. Further, the 6 th edition of the APA publication manual gives the following guidance related to sample description: "Describe the sample adequately. Detail the sample's major demographic characteristics, such as age; sex; ethnic and/or racial group; level of education; socioeconomic, generational, or immigrant status; disability status; sexual orientation; gender identity; and language preference as well as important topic-specific characteristics (e.g., achievement level in studies of educational interventions). As a rule, describe the groups as specifically as possible, with particular emphasis on characteristics that may have bearing on the interpretation of results. Often, participant characteristics can be important for understanding the nature of the sample and the degree to which results can be generalized….Even when a characteristic is not used in analysis of the data, reporting it may give readers a more complete understanding of the same and the generalizability of results and may prove useful in meta-analytic studies that incorporate the article's results." (American Psychological Association, 2010, pg 29-30).

9"
" Over the past three decades psychologists from a range of disciplines have been exploring the state of the psychological literature as it relates to inclusion of diverse participants, reporting of sample characteristics, and analysis of results based on a priori hypotheses by subgroups. In general, this research falls into two distinct styles of content analyses. The first style, clustered especially in the Eighties and early Nineties examined journal content for articles with a specific focus, often racial or ethnic minorities or women (Carter, Akinsulure-Smith, Smailes, & Clauss, 1998;Graham, 1992;Hunt et al., 2000;Imada & Schiavo, 2005;Iwamasa & Smith, 1996;Loo, Fong, & Iwamasa, 1988;Ponterotto, 1988), for example an article on depression in women, or autism rates in Hispanic children. The second style of content analysis, seemingly favored in the mid-Nineties to present day, were focused much more on analyzing sample reporting practices, and sample representativeness (Bernal & Enchautegui-de-Jesús, 1994;Case & Smith, 2000;Cundiff, 2012;Dan & Beekman, 1972;Delgado-Romero, Galván, Maschino, & Rowland, 2005;Duda & Allison, 1990;Mak, Law, Alvidrez, & Pérez-Stable, 2007;Munley et al., 2002;Park, Adams, & Lynch, 1998;Raad et al., 2008;Ram et al., 2004;Saris & Johnston-Robledo, 2000;Shelton, Delgado-Romero, & Wells, 2011;Sifers, Puddy, Warren, & Roberts, 2002), for example, how many women were being enrolled in research studies and how many researchers were reporting their demographics in their manuscripts. Although a clear explanation or reason is not presented for the shift, perhaps there was a hope that increased sample representativeness would lead to additional analyses by subgroup, thereby leading to more research applicable to a wider array of individuals.
10" " Alternatively, perhaps researchers were hoping that as more historically marginalized individuals were enrolled in research, a host of new research questions would arise during data analysis, which in turn would spark a new line of research focused specifically on historically marginalized groups.
Independent of the style of analyses, a summary of the results suggests that while reporting practices seem to have improved over time, relatively little publication space is being devoted to studies with a focus on research questions relevant to historically marginalized groups (Graham, 1992;Imada & Schiavo, 2005). Each of the identified historically marginalized groups is further expanded upon in the following sections.
Race and Ethnicity. Historically, race and ethnicity has been understudied and underreported in psychological science. As mentioned previously, two styles of content analyses have dominated the extant literature. The first style, largely seen in the Eighties and Nineties, saw authors examining journal content for articles with a racial or ethnic focus (Carter et al., 1998;Graham, 1992;Hunt et al., 2000;Imada & Schiavo, 2005;Iwamasa & Smith, 1996;Loo et al., 1988;Ponterotto, 1988 Across psychological disciplines (social, behavioral, counseling, community), and from the mid-Eighties to mid-Nineties, the findings from this style of content analysis stayed largely the same. Depending on coding criteria and the discipline analyzed, representative content ranged from 1.31% of articles focused on ethnic minorities to 15% of community psychology articles (Buboltz, Deemer, & Hoffmann, 2010;Carter et al., 1998;Hunt et al., 2000;Iwamasa & Smith, 1996;Loo et al., 1988).
The most recent content analysis of this style, was done in Social Psychology Quarterly, on all articles published from 2000-2012 (Hunt, Jackson, Kye, Powell, & Steelman, 2013). It was done as an update to previous work, from the same authors and in the same journal (Hunt et al., 2000). The updated analysis found that nearly a quarter of articles "seriously considered" race or ethnicity, which was marked improvement from the previous analysis. They cautioned however that experimental and theoretical articles were still lacking consideration of topics related to race and ethnicity at high levels (Hunt et al., 2013).

13"
" Delgado-Romero et al., (2005) examined three counseling psychology journals from 1990-1999 and collected not only whether specific variables were reported but also total number of participants by demographic characteristics.
As one might expect when analyzing literature over thirty years, there was a great deal of variability in the reporting of racial and ethnic sample characteristics, often dependent on the years the studies were conducted and also the research area from which they were taken. Reporting of race and ethnicity ranged from 3.8% to 91.7% (Bernal & Enchautegui-de-Jesús, 1994;Blancher, Buboltz, & Soper, 2011;Buboltz et al., 2010;Carter et al., 1998;Case & Smith, 2000;Cundiff, 2012;Delgado-Romero et al., 2005;Duda & Allison, 1990;Mak et al., 2007;Munley et al., 2002;Park et al., 1998;Raad et al., 2008;Ram et al., 2004;Shelton et al., 2011;Sifers et al., 2002). Those finding the highest reporting rates, of participant race and ethnicity, examined multicultural journals (Shelton et al., 2011), pediatric psychology journals (Raad et al., 2008), and National Institute of Mental Health journals (Mak et al., 2007) where authors may receive a large amount of funding from NIH. The lowest reporting rates were found in content analyzed in sports psychology journals (Duda & Allison, 1990;Ram et al., 2004), and those done many years ago (Bernal & Enchautegui-de-Jesús, 1994;Carter et al., 1998;Case & Smith, 2000;Park et al., 1998), perhaps indicating that reporting rates are steadily increasing. However, in the latest analysis, done on two issues from 2007 in eight prominent psychological journals spanning disciplines, only 52.2% reported race and ethnicity of their samples (Cundiff, 2012), which is higher than previous findings, but still far from an ideal percentage of authors reporting their samples' race and ethnicity. 14" " Additionally, in order to test the representativeness of the literature base as a whole, some studies have compared reported samples, taken in aggregate, across journals, to census data (Case & Smith, 2000;Cundiff, 2012;Delgado-Romero et al., 2005;Mak et al., 2007;Shelton et al., 2011) and found racial and ethnic minorities continue to be underrepresented in psychological science. Although the goal of a research body does not need to exactly match census demographics, at a study, or field-wide level, this analysis can give a general sense of whether underrepresented groups are being enrolled in psychological research studies.

Gender and Sexuality.
As with race and ethnicity, some authors analyzed journal content by focus on gender or women specifically (Blancher et al., 2011;Carter et al., 1998;Hunt et al., 2000;. However, these studies were also ones in which race and ethnicity were analyzed, and in three cases, that was the prime focus. Hunt et al., (2000Hunt et al., ( , 2013  The majority of content analyses that addressed gender tabulated reporting practices and sample sizes. Gender was reported across disciplines and time with more consistency than race and ethnicity. Reporting rates ranged from 80.4%-98.1% (Blancher et al., 2011;Cundiff, 2012;Dan & Beekman, 1972;Delgado-Romero et al., 15" " 2005;Mak et al., 2007;Munley et al., 2002;Park et al., 1998;Raad et al., 2008;Shelton et al., 2011;Sifers et al., 2002). Somewhat interestingly, one of the lower rates (84.1%) was found in an analysis of four multicultural journals over an eighteen year span (Shelton et al., 2011).
Sexual orientation was only examined in three studies (Blancher et al., 2011;Hunt et al., 2013;Ram et al., 2004) and only 1.00%-2.4% had a focus on sexual orientation. In the case of Blancher et al., (2010) this actually represented a precipitous decrease in focus on gays and lesbians since the previous content analysis, when the percentage had been 2.5% from 1988-1996 (Blancher et al., 2011).

16"
" Although the methodology, timeframes for analysis, and selected journals differ greatly, in general, findings reveal a lack of meaningful improvement in the research literature on historically marginalized groups over time. That is an increase in number of articles, but also improvement in quality, and emphasis on this line of inquiry The lack of progress warrants further study. These findings show more needs to be done to both increase the diversity of research participants and improve the output of literature that will aid our field going forward, in the implementation of service delivery and the training of the next generation.
Although reporting sample demographics is an important and worthwhile fieldwide goal, it is also important to note that there has been serious and thoughtful debate about the utility and appropriateness of using demographic variables in research, especially those pertaining to race and ethnicity and their social construction (Helms & Talleyrand, 1997;Helms, Jernigan, & Mascher, 2005;Kaplan & Bennett, 2003;A. Smedley & Smedley, 2005;Winker, 2006;Yee, Fairchild, Weizmann, & Wyatt, 1993) and SES (Braveman et al., 2005). Specifically, much debate has surrounded whether race should be recorded and used as a variable in scientific reports at all. Many have argued that using a socially constructed category as an independent variable in research, infuses it with biological and trait characteristics that do not exist, and in so doing racializes psychological science and risks further widening the racial stereotypes that exist in our society (Helms et al., 2005;Helms & Talleyrand, 1997;A. Smedley & Smedley, 2005).
Although SES for example is difficult to accurately measure (Braveman et al., 2005;Shavers, 2007) recording and attending to the demographics of study samples is 17" " an important part of responsible research practice. Including demographic variables such as race, ethnicity, SES, gender, sexual orientation, and age in published papers is not an endpoint for improving diversity in psychological research, but it is a place to start (Blauwet, 2011;Carnes et al., 2008;Flores et al., 2001;Uhl et al., 2007). Not including these data allows for the assumption of universality, which is unfair to participant, researcher, and consumer. Making these data available in published papers, even without further analysis or discussion, allows for interpretation of the applicability across populations (Glasgow, 2008;Green & Glasgow, 2006), lends legitimacy to the research as a whole (C. C. I. Hall, 1997), allows for the advancement of theoretical thinking, through hypothesis generation and discussion (Corbie-Smith, Miller, & Ransohoff, 2004), and makes data available for meta-analysis or public policy reports (Miranda et al., 2003).
We should also seek to move beyond simply reporting our participant characteristics. We also need to explore the complex ways in which aspects of individual identity interact with each other (Reid, 2002), and with psychological phenomena, and do so in a responsible way. However, this is currently not found consistently in the literature. For example, Saris and Johnson-Robledo (2000) found that of the small number of abstracts (searched through PsycLit) pertaining to poor women, a disproportionate number related to health concerns, specifically, AIDS, sexually transmitted infections, and motherhood as it related to missed prenatal appointments and substance abuse. Additionally, some analyses found small sample sizes, insufficient for subgroup analysis (Mak et al., 2007;Ponterotto, 1988) or race comparative models (Graham, 1992;Loo et al., 1988) which gives the impression that 18" " whiteness, or maleness is the norm (C. C. I. Hall, 1997) (Mitchell, 2012). Additionally, cost, geographic location, research interests, and an acceptability of this line of research are all barriers to conducting and/or publishing research on historically marginalized groups (Glasgow, 2008;Green & Glasgow, 2006;Henry, 2008;Hunt et al., 2000;Iwamasa & Smith, 1996;Miranda et al., 2003;Mitchell, 2012;Reid, 2002;Sears, 2008). 19" " analyzed and recorded each authors' reporting of a set of demographic characteristics including, total sample size, race/ethnicity, gender, age, and SES. I also noted whether the study was conducted in the United States or internationally. My final tally left me with pilot data on over 1,000 individual samples totaling greater than 2,650,000 participants, including reporting practices for major demographic categories, and representation of historically marginalized groups across disciplines.

Pilot Work
Overall, gender was the most consistently reported participant characteristic (91% of the samples reported), and over half of study participants were women (53.1% vs. 41.1%). However, less encouraging was reporting of race and ethnicity, where 59% of the studies did not report race or ethnicity of their participants (representing 38% of the total number of participants identified). Less than 25% of studies reported any measure of SES and those that did report SES did so in a wide variety of ways. Types of SES reporting were recorded and grouped into one of five categories, "income", "employment", "education", "social class", and "other".

Area for Further Study
One of the most recent content analysis was conducted on two issues in each of eight psychology journals in 2007 (Cundiff, 2012). A total of 255 articles were examined for assumptions of white male normativity as well as reporting of race and ethnicity and gender. Additionally, demographic information (race/ethnicity and gender) of the journal editors and first authors of each paper was collected where possible. Since then significant growth and progress has occurred in the field, 20" " specifically related to research on historically marginalized groups (National Multicultural Conference and Summit, 2013).
Although as noted above, previous work has been done on this topic, the results show reporting practices are still inconsistent and research on historically marginalized groups is not widely published in top-tier journals. As such, it is important to continue to track the progress of psychological science and assess the change and improvement over time. Additionally, unlike some previous studies the current study analyzed whether research studies focused on historically marginalized groups are examining variables across multiple aspects of identity, something which has not always been done in the past (Reid, 2002). Finally, the current study advances  In keeping with past research, small improvement were expected. Consistent with the literature, I expected gender to be reported more frequently than race and ethnicity, age, sexual orientation, and SES. As was true in the reviewed literature, I did not expect sexual orientation to be reported unless the article is using that variable in the analyses, and therefore, I expected the reporting to be very low.
Additionally, I hypothesized that reporting rates would differ by discipline, with clinical and developmental psychology reporting all variables most frequently and social psychology reporting least frequently. It was hypothesized that when compared to current census information, the data, taken in aggregate, would show that racial and ethnic minorities were underrepresented in current psychological research (Case & Smith, 2000;Cundiff, 2012;Delgado-Romero et al., 2005;Mak et al., 2007;Shelton et al., 2011). I expected women to be about equally represented in study samples, although I expected this would vary by discipline. For example, in disciplines where college students were used frequently, more women may be enrolled as research participants. This may be 22" " true given the reliance on undergraduate psychology students as participants and the higher number of women majoring in psychology as compared to men. Additionally, I expected university students to be significantly overrepresented in the literature. Based on the extant literature, I hypothesized that only between 10-15% of articles will have a focus on a historically marginalized group. This would represent an improvement in representation in the literature over the past thirty years as many of the previous content analyses of this type were done in the Eighties and early Nineties and found between 1.3% and 13% to have this focus (Carter et al., 1998;Graham, 1992;Hunt et al., 2000;Imada & Schiavo, 2005;Iwamasa & Smith, 1996;Loo et al., 1988;Ponterotto, 1988).  Psychology. All journals included six issues, published bi-monthly, except, Journal of Personality and Social Psychology, which published an issue monthly. To avoid oversampling from social psychology, six issues were randomly chosen, and coded.

24"
" Each issue was examined and coded in its entirety by one of four trained coders, the author, or an undergraduate research assistant (more information provided below). Although past content analyses have utilized database searches to procure articles, I accessed each article, in order, directly through the journal's website.
Articles that did not include human subjects, or did not use human subjects as the unit of analysis were excluded. These included theoretical articles, reviews, letters to the editor, meta-analyses, and other ancillary content. and Pacific Islander/Native American were combined to form one category due to the frequency with which these two categories were reported together in the articles.
Additionally, racial and ethnic codes account for the fact that some authors do not provide a full breakdown of their sample and instead choose to report "white" and 25" " "nonwhite". Due to the large number of international samples found in the pilot study and the variable ways race and ethnicity is conceptualized internationally, as well as a specific focus on historically marginalized group focus in psychological science in the United States, all international samples were coded as having an "international" ethnicity regardless of country of origin. This remained true regardless of whether study authors provided additional racial and ethnic breakdown of their sample beyond stating its international origin.
In instances where percentages were reported instead of raw numbers in any of the above categories, the percentages were converted to numerical values by multiplying by the total sample size. Mean age and age range of the sample was also recorded and articles were coded positively if they utilize one hundred percent undergraduate students as study participants in keeping with previous, conservative coding of undergraduate samples (Henry, 2008).
Although there is no consensus measure of SES, it is measured in a variety of ways, most often measurement includes at least one of the following variables; income, education, and/or employment information. For the purposes of coding the SES of participants in the articles under study, a broad definition of SES was used.
Based on findings from the pilot study, which provided extensive examples of the variables researchers use to capture SES data, five categories were created for the purposes of coding in this study, "income", "education", "employment", "social class" and "other". Indicators of each were taken directly from the pilot data and sorted into these categories. For example, "receipt of government aid", "mean income", and "eligible for free or reduced lunch" are all classified under "income". Extensive During coding, anytime an author used one or more of the identified variables, or a new variable, it was noted, and total participants (for example, total number completing high school), or means (for example mean income level of a sample) was also recorded. Given that SES is often measured as a composite, the coding allowed for data to be captured across multiple categories. For example if an author had stated that single-mother participants were recruited from a low income medical clinic, with a mean of 11.2 years of education (range 7-15 years), and 82% of those mothers were employed, those data would be coded as three separate variables; recruitment from a low income medical clinic, education, and employment. Each would be under a different SES category noted above. For "income" and "employment", total n would also be recorded, 100% and 82% respectively in this example. For "education", mean and range would be recorded, 11.2 and 7-15 respectively.
Additionally, low SES participants were coded post hoc, using the following definitions: income below $10,000, author definition of low income, receipt of government aid, education or parental education less than 12 years, unemployment, blue collar, or unskilled labor, low social class, or other definition of low SES provided by the author (for example, homelessness, or participant defined low SES).

Article Focused on A Historically Marginalized Group.
In order to determine whether research articles focused on historically marginalized groups, the title and abstract of each of the articles were examined. If either the title or the abstract of an article mentioned race or ethnicity (or a specific racial or ethnic group), sexual 27" " minorities, women, or low-income individuals, this was coded yes. An analysis of title and abstract content has been used previously to determine multicultural focus of articles (Hunt et al., 2000;Iwamasa & Smith, 1996;Loo et al., 1988;Saris & Johnston-Robledo, 2000), assuming that those who have made a commitment to a focus on historically marginalized groups will reflect that in their title and abstract.
Each of the four potential groups of focus were coded separately, meaning that individual articles could be coded "yes" in all four categories. Additionally, for articles focusing on racial and ethnic and sexual minorities, additional subcoding took place. Articles with a focus on race or ethnicity were coded by racial or ethnic group of focus (White, Black, Hispanic, Asian, American Indian/Alaska Native, Pacific Islander/Native Hawaiian). Articles focusing on sexual minority status were coded by category (Gay, Lesbian, Bisexual, Transgender). Articles with a focus on international populations were not coded as having a racial or ethnic group focus.
Validity Check. Trained undergraduate research assistants completed a portion of the article coding. The undergraduate coders included one woman and two men. All three were psychology majors who were entering their junior year or higher.
They were extensively trained on the codebook, reading and understanding research articles, interpreting differing types of research designs, and each had an understanding of psychology and historically marginalized groups. The coding book, complete with definitions and examples, was provided to the undergraduate research assistants prior to their beginning training (See Appendix B).
During coding 38% of articles were crosschecked for reliability, spread out throughout the year. All articles were coded in a Google Drive Spreadsheet, which 28" " was accessible to all four coders (myself and the three research assistants). When questions arose among any of the coders, comments were left for another coder, seeking clarification. All comments and questions were reviewed and discrepancies were resolved as a group, so the same questions did not continue to arise, or mistakes were not repeated throughout the data. Of the 173 articles double coded for reliability, 62 articles contained at least one coding disagreement (35.84%), which was discussed and resolved. Each article contained fifty-five unique codes, so this rate of disagreement between raters per code is remarkably low (99.40% agreement across all coding). constitute 10% or more of sample. I also added two codes to Mak et al.'s, (2007) dichotomous criteria. Many articles had multiple subgroups, some of which were adequately powered, and some of which were not. Therefore I included a "partially powered" code, which indicates the possibility for limited subgroup analyses. Finally, I coded when only a single racial or ethnic group was included in the study and subgroup analyses were not possible, but not because of lack of power. Authors were invited to participate through an initial email introducing the study briefly and informing them a SurveyMonkey survey invitation would soon follow. I followed up with Google searches for new email addresses for individuals whose email addresses bounced back to me after this initial email. All quantitative study material was collected anonymously online through SurveyMonkey. The full author survey can be found in Appendix C. Although all answers were collected anonymously, I was able to track those who had not completed the survey by email, thus allowing follow-up with non-completers.

Instrument/Measures.
Sample Demographics. Basic demographic information including race, ethnicity, gender, sexual orientation, highest degree completed, and psychological area of focus was collected. Race, ethnicity, gender, and sexual orientation were openended, self-report categories.

Research Focus on Historically Marginalized Groups. A variety of questions
sought to determine the importance and relevance, to the research authors, of historically marginalized groups in the formation of research questions, recruitment of study participants, and analysis of data, in their research as a whole, not specifically related to the article they published in one of the five journals of interest here.
Additionally, authors were asked about the relevance of the study of historically marginalized groups to their field of research interest.

Actual Practice and Barriers or Limitations.
Authors were also asked about their actual research practices as they relate to the study of historically marginalized groups and whether they consider themselves researchers with a focus on historically 31" " marginalized groups. Questions were asked related to barriers and limitations to conducting research on historically marginalized groups, and what may prevent authors from conducting and publishing this type of research.
Statistical Analysis. Quantitative data analysis is presented descriptively.
Many survey items are on one to ten scales, allowing for a wide range of author responses. Taken as a whole, this survey provides preliminary descriptive data on how research on historically marginalized groups is viewed and operationalized in the field and what barriers are encountered by researchers who are actively seeking to conduct research on historically marginalized groups but are limited in a variety of ways.

Preliminary Analyses
Prior to conducting any analyses, all data were important into Microsoft Excel, coded journal article data from Google Drive spreadsheets, and author surveys from SurveyMonkey, and cleaned. Post hoc procedures were undertaken, including creating new variables to aid later analyses (e.g. whether a sample recruited only international participants, or focused on a single, or multiple historically marginalized groups).
Journal characteristics, including total articles coded, number of samples comprised within those articles, and average participants per sample can be found in A review of the data indicated that one sample, from Journal of Consulting and Clinical Psychology was extremely large (n = 5,772,282) and skewed all further interpretation of data from that journal as the remaining samples combined (n = 40936) totaled less than one percent of that single sample. This article (article was a single sample article) then was removed from subsequent analyses.
Article level coding was used to describe a focus on historically marginalized groups in the title or abstract. Sample level coding was used to describe author reporting of demographic information and post hoc power analysis. Total ns were utilized to describe the representation of historically marginalized groups in studies that reported demographic information, as well as to compare coded data to data collected during the 2010 Census. The following is a detailed presentation of the results.

Research Question 1. Rates of Demographic Reporting
Reporting rates of race and ethnicity, gender, sexual orientation, SES, and age are reported in Table 2. Age was frequently reported, but of the historically marginalized characteristics, gender was reported with the greatest percentage, followed by SES, race and ethnicity, and finally sexual orientation. Reporting rates The gender of enrolled participants was the most frequently reported demographic characteristic associated with historically marginalized groups, across all five journals, with rates ranging from 89%-97%. Of those samples reporting gender, across journals, women were overrepresented in all journals, with the exception of Neuropsychology (Table 3).
When determining samples that calculated race and ethnicity, international samples were excluded from consideration. Forty to seventy nine percent of participants across the five journals were international participants and thus considered to have an "international" ethnicity, regardless of whether authors provided additional categorization. International participants are not included in discussions of race and ethnicity. Composition of participants by race and ethnicity varied greatly by journal. Psychology), reported SES for less than 9% of samples. Socioeconomic status was coded first dichotomously (reported or not reported), and for those reported, then subcoded into category of reporting type (for example reported income, education, or employment etc). The number of low SES participants was also coded. Table 5 provides details on the numbers of samples reporting SES, what measure of SES was employed, and whether more than one measure was used, by journal. The majority of authors reported SES in either one (46%-86%) or two (14%-33%) ways, with a minority reporting it three or more ways (Table 5).
Low SES participants (as defined by income below $10,000, author definition of low income, receipt of government aid, education or parental education less than 12 years, unemployment, blue collar, or unskilled labor, low social class, or other definition of low SES), were enrolled in 2%-50% of samples across journals. Total number of samples with low SES participants by SES category, average number of participants per sample, and percentages across journals are presented in Table 6. The remaining participants were either not able to be categorized (for example, authors reported a mean income, with no information about household size, or only number of participants employed were included, but no information was given on numbers unemployed), or participants were outside of the above definition of low SES.

35" "
Age of sample participants varied greatly by discipline. For example, Developmental Psychology enrolled more children and adolescents than other disciplines, while Journal of Personality and Social Psychology had 75% college students as participants across samples. A full reporting of age of participants is provided in Table 7.

Research Question 2. Journal Demographics Compared to 2010 US Census
Total number of participants enrolled across samples was used to determine if those enrolled in psychological science are reflective of the make up of our society, especially with respect to race and ethnicity. Total participant data across all journals, and using each journal separately, were compared to U.S. Census race and ethnicity data from 2010.
In order to compare coded journal demographic race and ethnicity information to census data, it was necessary to collapse the coded data into three categories; White, Hispanic, and Non-White. This was done because census data is captured by race and ethnicity separately, while the study data were not. It was not possible therefore to compare White coded participants (as reported by journal article authors), to the US Census category "white", as the census category also included White Hispanics. In the current study, "white" and "Hispanic" were distinct categories, as journal authors rarely, if ever reported race and ethnicity separately. The only way to ensure comparison across the same categories was to use the Census ethnicity data categories, which include all Hispanic categories, Non-Hispanic White, and Non-Hispanic, Non-White.

36" "
The results of the full demographic journal comparison are presented in Table   8. The five psychology journals enrolled fewer white participants, fewer Hispanic participants, and more non-white participants than the 2010 U.S. Census (χ2 (2) = 9527.41, p<.001).
The ethnicity characteristics of the participants from each journal were also significantly different than the Census data when individual χ2 were run. These data are presented in Table 9. All journals enrolled higher percentages of non-white participants than the percentage reported on the Census. However, with the exception of Developmental Psychology, all journals enrolled lower percentages of Hispanic participants than those represented on the U.S. Census.
Women comprised 48%-59% of participants across the five journals, making women slightly overrepresented in aggregate. Additionally, college students were used as participants to varying degrees by journal. Seventy six percent of samples in Journal of Personality and Social Psychology enrolled entirely college samples, while no other journal reached nine percent.

Research Question 3. Articles Focused on Historically Marginalized Groups
Articles focused on historically marginalized groups, that is, articles that mentioned a historically marginalized group in the title or abstract were represented in a relatively small percentage of articles in all journals. These data are presented in detail in Table 10. Articles with a focus on women were the most prevalent with 16%-37" " 39% having this focus. A focus on racial or ethnic minorities and SES were roughly similar across journals, between 3% and 19% were focused on each of those topics.
Less than 3% of any journal had articles focused on sexual minorities. The majority of articles had a focus on only a single historically marginalized group; however, some included multiple foci. Across all journals, in articles that had a focus on historically marginalized groups, 71.51% had only a single focus, while nearly a quarter focused on two historically marginalized groups. Just over four percent focused on three historically marginalized groups, most commonly race and ethnicity, gender, and SES.
No article focused on all four historically marginalized groups.
The gender and racial and ethnic makeup of samples with a focus on a women or racial or ethnic minorities are presented in Tables 11 and 12 (Table 11).
There were also higher percentages of racial and ethnic minority participants included in articles with a racial or ethnic focus than the percentage of racial and ethnic minorities across all samples. The enrollment, however, was not consistent across journals. For example, 12% of participants from articles with a racial or ethnic 38" " focus in Developmental Psychology were identified as American Indian or Alaska Native, while no other journal had more than 0.16% of their samples identified as American Indian or Alaska Native (Table 12). Similarly, Health Psychology articles had nearly 8% participants who identified as Asian or Pacific Islander; all other journals fell well below 2% (Table 12). All journals had lower percentages of white participants enrolled in articles with a focus on racial or ethnic minorities, than the percentage of white participants in the total sample for each journal, although for Health Psychology that difference was very small. Overall, articles focused on historically marginalized groups enrolled greater numbers of the population of focus, although there was variation by journal.

Ancillary Analyses.
It is common for researchers to enroll either targeted samples, or samples of convenience, and from those samples test differences by gender or race and ethnicity.
However, recruiting in this way may not yield adequate sample sizes for subgroups analyses. In order to explore this further, articles in the current study were evaluated, post hoc, to determine if there was sufficient power for subgroup analyses by race or ethnicity. This post hoc coding only tested whether subgroup analyses would have been adequately powered had they taken place, there was no evaluation of whether analyses were undertaken, how data were analyzed, or subgroups used.
A substantial percentage of samples (30%-50%) were inadequately powered for any analysis by subgroup (i.e. could not run any racial or ethnic group comparison analyses), while less than twenty seven percent of samples from any journal (with the 39" " exception of samples from Neuropsychology, which had very small ns) were powered for full subgroup analyses (i.e. could run analyses analyzing their data across all racial and ethnic groups, as enrolled, for example, White, African American, Hispanic, Asian, American Indian, etc). Full data are presented in Table 13 Additionally, forty to 75% of samples were powered for at least partial subgroup analyses, for example, collapsing racial and ethnic categories into "white" vs. "non-white", or comparing, white, and Hispanic, or African-American, to "all other racial groups".

Research Question 4. Author Survey Results
Sample Demographics. Valid email addresses were identified for five hundred and sixty nine corresponding authors from the five journals used to code provided the high and low ranges.

Difficulty Publishing Research Focused on Historically Marginalized
Groups. In order to examine the perceived difficulty of publishing historically ~Female, Hispanic, PhD: Answered "1" "Diversity is very popular now and journals love this stuff." ~Male, White, PhD: Answered "1" "I think that this focus may actually make it easier to publish the research versus much of the research which is conducted on white, young, college students. My guess is that having a larger historically marginalized sample improves the [publishability] (with perhaps the exception of large numbers of women which can be overrepresented in some areas of research)."

42"
"~M ale, White, PhD: Answered "1" "There is increased interest in marginalized groups (e.g., sexual orientation)" Female, Asian, Graduate Student: Answered "2" "Much research in social psychology is attempting to understand behavior on average, not the behavior of particular subsets of the population. Having said that, most of our samples are non-representative, particularly with regard to the groups you've asked about." ~Male, White, PhD: Answered "5" "I'm not sure there are particular barriers to conducting studies with these groups, as long as the questions are interesting" ~Female, White, Graduate Student: Answered "5" "I actually think it is EASIER to publish articles on historically marginalized groups; people are very interested in this. But it's hard to get some of these articles through the review process due to factors like higher degrees of sample attrition" Female, White, PhD: Answered "5" "Research is easier to publish when the hypotheses resonant with the intuitions of the powerful people in the discipline." Male, Black, PhD: Answered"6" "They seem to get returned without review or rejected more often that studies that focus on either population samples or student samples" ~Male, White, PhD: Answered "7" 43" " "I see very few articles published that are specifically relevant to marginalized groups; of course this could be because there are so few investigators doing this type of research (or doing it well) rather than a bias against this type of research in APA journals." Female, White, PhD: Answered "8" "Unfortunately, often the methods utilized in research focused on marginalized groups are more descriptive/phenomenological and this does not fit with the RCT approach favored in top notch journals, like JCCP. I'm not sure WHAT to do about this, but it is my impression that sometimes we lower the criterion for excellence when we study such topics to allow for researchers with fewer resources to be able to contribute. I see value in both realms and also as a minority I do feel that sometimes these descriptive approaches are the only ones that can capture the issues. Having said that, unless we learn to speak "in their language," minority issues may continue to be relegated to second tiered journals (as gauged by impact factor, for example)." ~Female, Latina, PhD: Answered "9" "My reading of the missions of truly top journals is that they focus on presenting the most broad and impactful research. By definition, some studies of marginalized groups would fail to meet this criteria. Also, for a broad swath of psychologists, the effects of gender, SES, and race/ethnicity are variables to be reported and accounted for, and not the topic of study. There are narrow, more targeted journals that would provide a better home for these studies. Finally, the category of "marginalized groups" is decidedly ad hoc. The characteristics of age differences are, to me, far removed from gender, SES, and race/ethnicity, and sexual orientation is even farther removed from all of these other factors. These variables are not the same kinds of things in various ways." Male, White, PhD: Answered "9" "Because they are seen as "specialty" topics, and the field as a whole is not really progressive with respect to these issues, relative to other related fields such as Sociology or Gender Studies" ~Female, White, PhD: Answered "10" 44" "

Importance of Historically Marginalized Groups to Authors' Research.
Study participants were asked about the importance of historically marginalized groups to answering their research questions. They were also asked how often they enrolled members of specific historically marginalized groups in their research studies.
Answers were provided on a 1 to 10 scale. Gender (mean=6.23, SD 2.84) was rated as the most important to answering research questions, followed by SES (mean=5.77, SD 2.90), race and ethnicity (mean=4.98, SD 3.16), and sexual orientation (mean=2.78, SD 2.51). There was considerable variation by discipline, which can be seen in Figure   2.
Survey participants as a whole did not endorse recruiting specific historically marginalized groups into their research studies frequently, something which is necessary to answering research questions. On a scale from 1 to 10 with 1 being never, and 10 being all of the time participants reported being most likely to recruit men or women specifically (mean=5.26, SD 3.44), followed by a specific racial or ethnic group (mean=3.95, SD 3.37), specific SES group (mean=3.30, SD 2.77), and finally specific sexual minority (mean=1.63, SD 1.67). Across disciplines, specific gender groups remained the most frequently targeted, with the exception of the "other" discipline. Full data are presented in Figure 3.
Survey participants were asked to indicate how often their research included a focus on race or ethnicity, gender, sexual orientation, or SES. Their answer choices ranged from 0 (Never) to 4 (Always). The total sample mean was 1.89 (SD 1.28). On a follow-up question, asking, regardless of their answer to the previous question,

45"
" whether they considered themselves a researcher who does research on historically marginalized groups, 34.46% (n=51) answered "yes". However when these data were examined by the self-reported race and ethnicity, sexual orientation, and gender of the authors, racial and ethnic and sexual and minorities and women were more likely to consider themselves researchers with a focus on historically marginalized groups (although sexual and racial and ethnic minority ns were small). Additionally, women, and racial or ethnic minority participants' mean scores were greater for endorsing research including a focus on historically marginalized groups, than both the total sample, and then white or straight authors, or men (Table 17).

Barriers to Conducting Research Focused on Historically Marginalized
Groups. Authors indicated a number of barriers to conducting research focused on historically marginalized groups. The most frequently cited barrier was related to participant recruitment and finding and retaining participants from historically marginalized groups.
Examples of reported barriers include: "Difficulty with attendance, low education and difficulty conducting assessments, poor health impacting study participation, mistrust in answering research assessments" "Participant recruitment is a huge barrier. I previously worked in a very rural community and even with extra funding to recruit participants from a broader range of SES it was very difficult to bridge the inherent mistrust between the community and the "ivory tower" institution. Similarly, I'm now in a larger city and my research study actively participates in outreach to Black and Latino communities, but there is historically-rooted mistrust that is slow to break down. (That said, we're making progress, so it's not all pessimistic!)" ~Female, White, PhD: Not Focused on Historically Marginalized Groups "It has not been a focus of mine to study historically marginalized groups. However, I think if I were to switch my "niche" to historically marginalized groups, I'd encounter the following barriers: 1) as a minority person myself, other researchers may perceive me as being "on a war path" and think that my work is more personal than professional; 2) the journals I'd publish my findings would likely be less prestigious and have a lower impact factors; 3) I'd often feel caught between being a scientist and an advocate for social justice-sometimes this is hard to reconcile--which one does one put first?; 4) Less respected by colleagues. I've seen this among colleagues who are in Women's Studies. The moment someone mentions being in women's studies in committees, etc, other colleagues seem to value them less (stop making as much eye contact, do not initiate conversations). Not sure if this is my bias, but I'd love someone to conduct this type of study! 5) This already happens to me: given my ethnicity, I often get students from disadvantaged backgrounds who require considerable mentoring not to give up and also who have significant skills deficits by virtue of hailing from disadvantaged backgrounds. Mentoring such students require more effort and skill than mentoring someone who is Caucasian and comes from a high income bracket. Yet, this is rarely acknowledged in terms of doling out accolades to researchers--it's assumed that graduating one PhD (say someone who hails from an affluent background with strong training in critical thinking) is the same as graduating another PhD student (someone who comes from a poor family, racially diverse, etc). It is not and I wish minority faculty got the credit they deserve in working hard to train the new generation of minority researchers." A surprisingly low reporting rate (36.53% total) was found across journals for racial and ethnic characteristics of study participants. The extant literature has shown great variation in the rates of reporting, with ranges from 4-92% (Bernal & 49" " Enchautegui- de-Jesús, 1994;Blancher et al., 2011;Buboltz et al., 2010;Carter et al., 1998;Case & Smith, 2000;Cundiff, 2012;Delgado-Romero et al., 2005;Duda & Allison, 1990;Mak et al., 2007;Munley et al., 2002;Park et al., 1998;Raad et al., 2008;Ram et al., 2004;Shelton et al., 2011;Sifers et al., 2002), however, the most recent content analyses have found rates to generally be increasing, ranging from 52%-89% (Blancher et al., 2011;Buboltz et al., 2010;Cundiff, 2012;Mak et al., 2007;Raad et al., 2008;Shelton et al., 2011).

Given the publishing standards set forth by APA (American Psychological
Association, 2010), specifically as it relates to adequate sample description. It is unclear why these top-tier journals are not better adhering to such standards. It is especially concerning that Neuroscience, representing a cutting edge and rapidly developing field, had only 13% reporting race and ethnicity of the samples. Now is certainly not the time to return to the scientific practices of decades past. Transparent sample description is important for the field in order to make data available for interpretation by peers, policy makers, provide data for meta-analyses, it lends legitimacy to the field as a whole, and can spur advances in theoretical and creative thinking (Corbie-Smith et al., 2004;Glasgow, 2008;Green & Glasgow, 2006; C. C. I. Hall, 1997;Miranda et al., 2003). Although researchers certainly bear responsibility for ensuring their data are collected appropriately and carefully and reported accurately, there is also an onus on reviewers and editors to ensure that the standards put forth by APA as important to the field, are being followed throughout the literature representing the field.

50"
" One area where improvement was noted was in the case of SES reporting.
Previous content analyses noted reporting rates between 15% and 57%, with most close to 33% (Graham, 1992;Liu et al., 2004;Munley et al., 2002;Park et al., 1998;Ponterotto, 1988;Raad et al., 2008;Saris & Johnston-Robledo, 2000;Sifers et al., 2002). With the exception of Journal of Personality and Social Psychology, all percentages in the current study were between 65% and 76%. This is a marked improvement. In the current study, SES was measured in a variety of ways, which may have increased the incidence of reporting. Munley et al., (2002) found 79% reporting of education, but less than 20% each of income, employment, and social class. More diversity of reporting types, and combination of types of SES indicators was found in the current study.
Although SES is being more frequently measured and reported, much of the data reported was difficult to interpret. For example, without a context in which the data were collected, mean income, is little more than a number. Additionally, in certain types of articles, or in the majority of articles in some journals, similar patterns of SES reporting were seen (i.e. almost all Neuropsychology articles that reported SES included education). SES is a complex, difficult construct to capture, and is context specific (Braveman et al., 2005;Shavers, 2007). It is therefore incumbent upon future scientists to not only collect and report SES information, but also describe their measures in a way that allows full understanding, and also provide theoretical and contextual justification for choosing their measures. Similar arguments have been made for including and using racial and ethnicity information from participants, notably that race and ethnicity is a fluid construct, definitions should be clearly 51" " established and study specific, and utilization should be thoughtful and transparent (Helms et al., 2005;Kaplan & Bennett, 2003;Miranda et al., 2003;A. Smedley & Smedley, 2005;Winker, 2006).
Although few studies reported information on sexual minorities, and survey participants were equivocal on the importance of including them in research, there is an argument to be made for asking about and reporting on the sexual orientation of research participants. Including sexual orientation as standard practice in research would likely be a controversial and divisive issue. While it could be argued that it is unnecessarily intrusive, by including this information as normative demographic background information, it may help remove some of the stigma associated with sexual minority groups. Currently, when studies focus on sexual minorities in psychological science, they are unusual, and could have the tendency to pathologize an already stigmatized group. Additionally, when sexual orientation is not measured and reported, the assumption is that all study participants were heterosexual. Even if no subgroup analyses are run, it continues to be important for the field to better understand the context in which hypotheses are tested and results are garnered. Asking about sexual orientation may not be appropriate for every research study, for example with adolescent samples, where parental disclosure is a risk, but the fact that so few are asking now does not seem appropriate either. What demographic information is collected, and reported is a potentially sensitive issue and should seemingly be theoretically driven. However, given APA's guidance on adequate sample description, regardless of whether data are used in analyses, sexual orientation can be added as part of a battery of demographic questions. There is of course the risk of burdening both 52" " participant and researcher with an overlong demographic questionnaire, however if we return to APA's definition of multiculturalism provided at the beginning of this report, sexual orientation is an important aspect of individual identity. It can be argued that this definition also includes religious affiliation and disability status, which are likely not queried routinely. It would be worthwhile to gain perspective from representative groups from sexual minority communities both within and outside APA regarding the pros and cons of this issue as we as a field seek best research practice when it comes to providing adequate sample description and respect and privacy of research participants.
Although reporting demographic information is a necessary part of responsible research, as mentioned previously, research with a focus on historically marginalized groups is what will provide the lasting impact by providing data for treatment recommendations, public policy decisions, measure development and validation, and a host of other future scientific, clinical, and behavioral pursuits. Without these data, not only are there significant gaps in the knowledge base, but it is difficult to make adequate treatment guidelines, or policy recommendations, as data only exists for white, middle class, male populations (Miranda et al., 2003).
It was therefore disappointing that so few articles across all journals had a specific focus on a historically marginalized group. There were small improvements seen from previous content analyses, as hypothesized. For example, previous content analyses looking at race and ethnicity focused articles in APA journals found rates less than 6% (Graham, 1992;Imada & Schiavo, 2005). The cumulative 10.65% found in 53" " the current study is similar to percentages found in non-APA journals, and counseling journals in the extant literature (Buboltz et al., 2010;Carter et al., 1998;Imada & Schiavo, 2005;Iwamasa & Smith, 1996;Iwamasa, Sorocco, & Koonce, 2002).
However, the latest content analysis, done Social Psychology Quarterly, found nearly a quarter of articles "seriously considered" race or ethnicity (Hunt et al., 2013), a percentage much higher than found in the current study. Given the very broad definition of "focus" utilized in the current study, it is disappointing to not see more gains in percentage of articles with a focus on historically marginalized groups from previous content analyses, especially as many of them were done ten or more years ago. Although caution needs to be taken when simply counting numbers in order to reach an unspecified threshold, without accounting for the quality and content of the measured studies, an important first step is the measurement of publication space allocated for research on historically marginalized groups.
SES and gender were examples of areas where relatively high rates of reporting sample demographics did not necessarily translate to focused research questions. SES was reported in the current content analysis at much higher rates than those found in previous works, however, relatively few articles focused on SES.

Journal of Consulting and Clinical Psychology especially attended to SES in a very
small number of articles. This is especially alarming given the disparities that continue to be seen in access effective to mental health treatments among low SES individuals (Wang et al., 2005), something clinical psychology researchers are positioned to study.
The same can be said for Developmental Psychology, where researchers enrolled nearly 69% children and adolescents, who are disproportionately affected by poverty 54" " in the United States (United States Census Bureau, 2013), effecting all aspects of their lives and development (Evans, 2004). However, given the positive steps seen in reporting SES information in the current study, as compared to previous work, there is reason for optimism that SES will continue to gain traction as an important focus in psychological science.
Although women focused articles were the most frequently coded (27%) of all of the historically marginalized groups under consideration in the current study, improvements in women focused research have been inconsistent across the extant literature, with some evidence suggesting that forward momentum gained over the past decades has stalled, or even reversed course (Carnes et al., 2008), something that is worthy of further investigation, especially in light of follow-up content analyses that have found fewer articles with female focus than found previously (Hunt et al., 2000;. Interestingly, gender was rated, by the survey participants, as the most important (of race and ethnicity, gender, SES, and sexual orientation), to answering their research questions and they were most likely to enroll men or women into their studies than other historically marginalized groups.  Miranda et al., 2003). When specific groups are consistently underrepresented it can lead to knowledge gaps and health, and mental health disparities (Blauwet, 2011;Iwamasa et al., 2002). In the current study for example, Hispanic participants were enrolled at much lower rates than are found in the general population, especially when assessed by journal. The Hispanic ethnicity represents a 56" " tremendous number of individual, unique ethnicities, so an argument for more, not less research in this diverse, underrepresented population can be made. Given the growth in the Hispanic population in the United States, this is finding is particularly alarming.
Some survey participants mentioned how studies focused on historically marginalized groups, which enroll only one racial or ethnic group, can face barriers, in grant funding, in acceptance by peers, publication, and critique of research methodology. However, it was also noted that these types of studies are valuable at answering complex research questions pertaining to specific populations. A welldesigned research study focused on a single population group, even if it is small, qualitative, or utilizes mixed methods, adds more to our understanding than post hoc subgroup analyses conducted from larger clinical trials utilizing convenience samples.
As noted by Miranda et al., (2003) "Learning to treat ethnic minorities appropriately will mean learning to engage minority communities in the research process." (pg. 479). This can be said for any historically marginalized group.
Improving the representation of research focused on historically marginalized groups in psychological science is the responsibility of the entire research community (Uhl et al., 2007), not only a select few. Survey participants who identified themselves as belonging to a historically marginalized group also reported engaging in research with a focus on historically marginalized groups more than other groups, and were much more likely to consider themselves researchers with a focus on historically marginalized groups. The numbers of racial and ethnic and sexual minorities in the current sample were small, but the trend is similar to those reported previously (G. C.

57"
" Hall & Maramba, 2001). The onus to improve the representation of a diverse set of research agendas is on the entire research community, not just a small subset of researchers. When a members of historically marginalized groups are the only ones seen conducting research on these important topics, it is easier for majority groups, uninterested in "making room at the table" to write off the research as a special interest topic, something that was noted by a survey participant as a barrier.
Another possibility for the low rates of focus on historically marginalized groups in the content analysis in the current study is articles with this focus are not being published in the journals chosen for the current analysis. There are multicultural journals, published by APA and non-APA entities, and those outlets may be where studies with specific historically marginalized group focus end up. However, when queried, survey participants, who were authors in the journals of focus, felt it was important that research with a focus on historically marginalized groups find a home in top-tier APA journals.
Survey participants differed greatly when asked to first, identify how difficult they believe publishing articles with a focus on historically marginalized groups is in top-tier APA journals, and second, explain why. Qualitatively, there were participants that argued that focusing on historically marginalized groups made publishing easier, while others said reviewers and journals found the focus too narrow, specialized, or niche and publishing was much more difficult. The lack of qualified reviewers, or editor or reviewers understanding the significance of the contributions of historically 58" " marginalized group focused research was also noted. Lack of qualified editors and reviewers has been found in previous work (G. C. Hall & Maramba, 2001).
The quality of the work was cited many times, with variations of the theme: If the research is high quality, the topic doesn't matter. However, this brings up interesting questions of what constitutes "quality", who holds the power to decide, and if it really is true, that all else being equal, that topic doesn't matter, something a few survey participants touched upon. I would argue that there are still significant barrier to high quality research focused on historically marginalized groups being published in top-tier APA journals.

Limitations.
This study had limitations that should be considered. First, the journals considered were chosen based on a combination of inclusion in previous content analyses and general standing in the field. It is possible that there are higher rated journals (based on impact factor), in each of the disciplines coded, however these five APA-published journals have been well respected over a number of years. Second, there are many disciplines within psychology not examined within the context of this study, for example School Psychology, or Community Psychology and therefore these results may not generalize to all disciplines of psychology. Third, only empirical studies, where human participants were the unit of measure, were included for coding purposes. It is possible theoretical articles or commentaries would have a higher 59" " percentage focused on historically marginalized groups. Fourth, article coding, between the undergraduate research assistants and myself was a collaborative process.
We utilized Google Drive spreadsheets, which allowed for continuous and immediate feedback and consultation, which encouraged engagement on both ends. While I believe this lead to better, more accurate data collection, and more comfort with the process for the research assistants, it may have introduced my coding biases more completely throughout the processes. Fifth, historically marginalized group focus relied on mention in the title or abstract of the article, and while this methodology has been used previously (Hunt et al., 2000;Iwamasa & Smith, 1996;Loo et al., 1988;Saris & Johnston-Robledo, 2000), it is possible articles with a historically marginalized group focus were missed. Sixth, low-SES participants were coded utilizing a priori defined coding, however, many data points were uncodable, or uninterpretable. Low-income numbers are likely an underestimate. Seventh, each journal had a different number of total articles published, and therefore a varying number of authors available for invitation to the author survey. As a result, sample sizes by discipline are uneven in the author survey. Finally, survey participants were given the option to skip any item they chose. As a result, there is missing data throughout the survey.

Strengths.
This study has important strengths. First, this study coded both a focus on historically marginalized groups, as well as their inclusion in current literature, across samples. There is room for many kinds of research, but study designs that delve into 61" " the intersections of culture and identity have the potential to move the field forward in exciting ways.
However, history has shown that quantity does not necessarily indicate quality.
Blindly calling for an increase in research focused on historically marginalized groups could inadvertently usher in an era of biased, sexist, racist, classist, homophobic, heteronormative research. Even without that dire consequence, it is important that studies focused on historically marginalized groups are not only being published when trying to solve a problem (ex: substance use rates higher in one population), or population specific research (ex: HIV interventions and gay men), which can inadvertently reinforce negative stereotypes. Instead, studies focused on historically marginalized groups should be published on a range of topics, as studies with other foci are. This will ensure that treatment guidelines, public policy, and the next generation of theoretically founded research agendas are based on culturally sound science.

Conclusions.
Findings from this study highlight the progress that has been made, and also the work that still needs to be done to ensure historically marginalized groups are represented in top-tier APA-published journals. Although historically marginalized research participants are being more accurately reported and more frequently included in research studies, disappointing increases were seen with regard to studies focused 62" " on historically marginalized groups as compared to findings from past content analyses, and content analyses done recently. Although progress is hard to measure, seeing an increased number of well-designed studies consistently focused on historically marginalized groups across disciplines, as well as studies focused on multiple aspects of identities, would be good for the literature base.
While considerable work is still to be done, the author survey indicated that many early career researchers are doing research focused on historically marginalized groups at least some of the time. There were a number of identified barriers, which as a field could be better addressed to help researchers continue their work.
Slow, at time disappointing progress was seen from previous content analyses and yet enough differences were seen between the coding results and author responses to provide some hope that the field is continuing to move in the right direction. French philosopher Michel Foucault critiqued the prison design utilizing a "panopticon" or all-seeing eye. The simplicity of the original design was one guard tower, placed at the center of a circle of cells could provide the perception of constant surveillance, even while it was impossible for one guard to simultaneously observe all inmate cells.
Foucault took it further and posited the idea of constant surveillance had pervaded all aspects of society. Perhaps it is now time for psychologists to turn their collective gaze inward and utilize the power of the "panopticon" for a more noble purpose. As with the prison, it is not possible to oversee all research at any time, however, knowing that higher standards, and better research practices are expected from anyone, may raise the bar of research reporting and quality from everyone.  "This research was support by the National Institutes of Health Grants R03HD060696, R01ES017876, AG033590, and UL1RR024999"

Race or ethnicity focused in Title
If the title of an article makes any mention of any racial or ethnic group, or race, or ethnicity, this should be coded as "yes". Please note, international studies are coded separately.
Possible terms used in the title could include, but are not limited to: African-American, Latino/a, Native American, Asian, Race, Ethnicity, Racial or Ethnic Minority. No = 0 Yes = 1 International = 2 1. "Understanding narrative effects: The impact of breast cancer survivor stories on message processing, attitudes, and beliefs among African American women." 2. Title "Social disadvantage and the self-regulatory function of justice beliefs." 3."An initial evaluation of the role of emotion and impulsivity in explaining racial/ethnic differences in the use of corporal punishment."

Specific Racial/Ethnic Focus in Title
If studies are coded "yes" above, they should be further coded by specific racial or ethnic group (i.e. African American, Asian, Native American/Alaska Native, Hispanic, Native Hawaiian/Pacific Islander) focus of the article.
Possible terms used in the title could include, but are not limited to: African-American, Latino/a, Native American, Black = 1 Hispanic = 2 Asian = 3 American Indian/Alaska Native = 4 Native Hawaiian/Pacific Islander = 5 None = 6 Other = 7 (Describe) 1. "Randomized controlled trial of a preventive intervention for perinatal depression in high-risk Latinas." 2. "Childhood socioeconomic status is associated with psychosocial resources in African Americans: The Pittsburgh Healthy Heart Project" Asian, Race, Ethnicity, Racial or Ethnic Minority.

Race or ethnicity focused in Abstract
If the abstract of an article makes any mention of any racial or ethnic group, or race, or ethnicity, this should be coded as "yes". Please note, international studies are coded separately.
Possible terms used in the abstract could include, but are not limited to: African-American, Latino/a, Native American, Asian, Race, Ethnicity, Racial or Ethnic Minority.
1. "Five studies support the hypothesis that beliefs in societal fairness offer a self-regulatory benefit for members of socially disadvantaged groups. Specifically, members of disadvantaged groups are more likely than members of advantaged groups to calibrate their pursuit of long-term goals to their beliefs about societal fairness. In Study 1, low socioeconomic status (SES) undergraduate students who believed more strongly in societal fairness showed greater intentions to persist in the face of poor performance on a midterm examination. In Study 2, low SES participants who believed more strongly in fairness reported more willingness to invest time and effort to achieve desirable career outcomes. In Study 3, ethnic minority participants exposed to a manipulation suggesting that fairness conditions in their country were improving reported more willingness to invest resources in pursuit of long-term goals, relative to ethnic minority participants in a control condition. Study 4 replicated Study 3 using an implicit priming procedure, demonstrating that perceptions of the personal relevance of societal fairness mediate these effects. Across these 4 studies, no link between fairness beliefs and self-regulation emerged for members of advantaged (high SES, ethnic majority) groups. Study 5 contributed evidence from the World Values Survey and a representative sample (Inglehart, Basañez, Diez-Medrano, Halman, & Luijkx, 2004). Respondents reported more motivation to work hard to the extent that they believed that rewards were distributed fairly; this effect emerged more strongly for members of lower SES groups than for members of higher SES groups, as indicated by both self-identified social class and ethnicity."

Specific Racial/Ethnic Focus in Abstract
If studies are coded "yes" above, they should be further coded by specific racial or ethnic group (i.e. African American, Asian, Native American/Alaska Native, Hispanic, Native Hawaiian/Pacific Islander) focus of the article.
Possible terms used in the abstract could include, but are not limited to: African-American, Latino/a, Native American, Asian, Race, Ethnicity, Racial or Ethnic Minority. Black = 1 Hispanic = 2 Asian = 3 American Indian/Alaska Native = 4 Native Hawaiian/Pacific Islander = 5 None = 6 Other = 7 (Describe) 1. "Objective: To determine whether lower childhood socioeconomic status (SES) was associated with fewer psychosocial resources independent of adult SES, and whether these associations differed by race/ethnicity. Method: Cross-sectional study of 342 middle-aged (M = 60.5 ± 4.7) African American (n = 49) and Caucasian (n = 293) adults. Childhood SES and adult SES were assessed via highest parental education and participant education, respectively. Participants completed: (a) 6 days of ecological momentary assessment via electronic diaries to assess social support and the number of social interactions and (b) self-report measures of social support, social network diversity, and coping-specifically, active, planning, and emotion focused coping. Results: The interaction term for childhood SES and race/ethnicity significantly predicted several psychosocial resources. Lower childhood SES was associated with less perceived social support in daily life, a less diverse social network, and more limited use of proactive coping strategies in adulthood among African Americans, regardless of adult SES. Comparable associations were not observed among Caucasians. Conclusions: Childhood SES is associated with psychosocial resources in adulthood among African Americans, independent of SES in adulthood. Given emerging associations between childhood SES and health in adulthood, future studies to disentangle the role of psychosocial resources as a mediating pathway and to further examine racial/ethnic variations across these associations are warranted"

Women Focused in Title
If the title of an article uses No = 0 any terms referring to women this should be coded as "yes".
Possible terms used in the title could include, but are not limited to, women, female, mothers, and girls.
* Please note, terms such as Latina indicate ethnicity as well as gender.
Yes = 1 impacts cardiovascular stress responses among depressed and nondepressed women" 2. "Sexual risk among African American girls: Psychopathology and mother-daughter relationships" 3. "Randomized controlled trial of a preventive intervention for perinatal depression in high-risk Latinas"

Women Focused in Abstract
If the abstract of an article uses any terms referring to women this should be coded as "yes".
Possible terms used in the abstract could include, but are not limited to, women, female, mothers, and girls.
* Please note, terms such as Latina indicate ethnicity as well as gender.
No = 0 Yes = 1 1. "To examine the associations among mental health problems, maternal monitoring and permissiveness, mother-daughter communication and attachment, and sexual behaviors among African American girls receiving outpatient psychiatric care. Youths with mental health problems report higher rates of HIV-risk behavior than do their peers, and African American girls have higher rates of sexually transmitted infections than do girls of all other racial groups. Method: A sample of 12-to 16-year-old African American girls (N = 266, mean age = 14.46 years) and their female caregivers (73% biological mothers) completed computerized assessments of girls' mental health symptoms, maternal monitoring and permissiveness, and mother-daughter communication and attachment. Girls indicated their sexual risk behaviors (vaginal/anal sex, consistent condom use, number of partners). Results: African American girls who reported clinically significant externalizing problems, more permissive parenting, less open mother-daughter sexual communication, and more frequent mother-daughter communication were more likely to report having had vaginal and/or anal sex. Sexually active girls with greater maternal attachment were less likely to report inconsistent condom use. Conclusions: Findings revealed important risk and protective factors for African American girls in psychiatric care. HIV-prevention programs may be strengthened by improving mother-daughter relationships and communication and by reducing girls' mental health problems."

Sexual Minority Focused Y/N in Title
If the title of an article uses any term referring to sexual orientation or gender identity this should be coded as "yes".
Possible terms used in the title could include, but are not limited to: Gay, lesbian, bisexual, transgender, transsexual, questioning, sexual minority, queer, women who have sex with women (WSW), men who have sex with men (MSM).
No = 0 Yes = 1 1. "Retrospective recall of sexual orientation identity development among gay, lesbian, and bisexual adults." 2. "The influence of sexual orientation and masculinity on young men's tobacco smoking." 3. "The impact of minority stress on mental health and substance use among sexual minority women."

Specific Sexual Minority Focus in Title
Additionally, if studies are coded "yes", they should be further coded by specific orientation (i.e. gay, lesbian, bisexual, transgender, other) focus of the article.
Possible terms used in the title could include, but are not limited to: Gay, lesbian,

Sexual Minority Focused Y/N in Abstract
If the abstract of an article uses any term referring to sexual orientation or gender identity this should be coded as "yes".
Possible terms used in the abstract could include, but are not limited to: Gay, lesbian, bisexual, transgender, transsexual, questioning, sexual minority, queer, women who have sex with women (WSW), men who have sex with men (MSM).
No = 0 Yes = 1 1. "The prevalence of smoking among gay men is considerably higher than in the general population. To investigate possible causes of this health risk disparity, this study used multilevel modeling of daily diary data to examine the temporal relationship between smoking and both sexual orientation concealment and masculine gender role variables. Gay (n = 136) and heterosexual (n = 56) university students (mean age = 20.56, SD = 2.13) completed measures of boyhood and current gender nonconformity, as well as daily measures of smoking, negative affect, and masculinity self-consciousness across 9 days. Gay participants additionally indicated the extent to which they concealed their sexual orientation each day. The same percentage of gay (17.7%; n = 24) and heterosexual (17.9% n = 10) participants smoked over the course of the study. Gay men who smoked, however, smoked on more days across the study, t = 2.20, p < .05. Boyhood gender nonconformity and current masculinity significantly predicted the average odds of smoking for all participants. Daily masculinity self-consciousness also predicted the odds of smoking for all participants, although it predicted those odds more strongly for heterosexual men (b = 1.00, p < .001) than for gay men (b = .31, p = .06). Gay participants' attempts to conceal their sexual orientation on a given day positively predicted their likelihood of smoking that day.
Results suggest the need to consider the role of gender nonconformity, masculinity self-consciousness, and sexual orientation stress in future investigations of smoking among young men"

Specific Sexual Minority Focus Abstract
Additionally, if studies are coded "yes", they should be further coded by specific orientation ( Nearly all patterns of development were identity centered, with average age of self-identification as GLB preceding average age of first same-sex sexual activity. Overall, younger participants and the majority of older participants were classified to the early profile, suggesting that early development is common regardless of age cohort. The additional gender differences observed in the onset and pace of sexual orientation identity development warrant future research."

Socioeconomic Focus Y/N Title
If the title of an article uses any term referring to socioeconomic status this should be coded as "yes".
Possible terms used in the title could include, but are not limited to: Socioeconomic status, socially disadvantaged, income, low-income, food insecure, social class, poverty, education, occupation.
No = 0 Yes = 1 1. "Social disadvantage and the self-regulatory function of justice beliefs." 2. "Does money really matter? Estimating impacts of family income on young children's achievement with data from random-assignment experiments" 3. "Socioeconomic inequalities in colorectal cancer screening uptake: Does time perspective play a role?" 4. "Childhood socioeconomic status is associated with psychosocial resources in African Americans: The Pittsburgh Healthy Heart Project"

Socioeconomic Focus Y/N Abstract
If the abstract of an article uses any term referring to socioeconomic status this should be coded as "yes".
Possible terms used in the abstract could include, but are not limited to: Socioeconomic status, socially disadvantaged, income, low-income, food insecure, social class, poverty, education, occupation.
No = 0 Yes = 1 1. "Five studies support the hypothesis that beliefs in societal fairness offer a self-regulatory benefit for members of socially disadvantaged groups. Specifically, members of disadvantaged groups are more likely than members of advantaged groups to calibrate their pursuit of long-term goals to their beliefs about societal fairness. In Study 1, low socioeconomic status (SES) undergraduate students who believed more strongly in societal fairness showed greater intentions to persist in the face of poor performance on a midterm examination. In Study 2, low SES participants who believed more strongly in fairness reported more willingness to invest time and effort to achieve desirable career outcomes. In Study 3, ethnic minority participants exposed to a manipulation suggesting that fairness conditions in their country were improving reported more willingness to invest resources in pursuit of long-term goals, relative to ethnic minority participants in a control condition. Study 4 replicated Study 3 using an implicit priming procedure, demonstrating that perceptions of the personal relevance of societal fairness mediate these effects. Across these 4 studies, no link between fairness beliefs and self-regulation emerged for members of advantaged (high SES, ethnic majority) groups. Study 5 contributed evidence from the World Values Survey and a representative sample (Inglehart, Basañez, Diez-Medrano, Halman, & Luijkx, 2004). Respondents reported more motivation to work hard to the extent that they believed that rewards were distributed fairly; this effect emerged more strongly for members of lower SES groups than for members of higher SES groups, as indicated by both self-identified social class and ethnicity." 2. "Social scientists do not agree on the size and nature of the causal impacts of parental income on children's achievement. We revisit this issue using a set of welfare and antipoverty experiments conducted in the 1990s. We utilize an instrumental variables strategy to leverage the variation in income and achievement that arises from random assignment to the treatment group to estimate the causal effect of income on child achievement. Our estimates suggest that a $1,000 increase in annual income increases young children's achievement by 5%-6% of a standard deviation. As such, our results suggest that family income has a policy-relevant, positive impact on the eventual school achievement of preschool children"

Total Sample N
Record total number of participants enrolled in the study.
** For articles where multiple studies are conducted, participants should be pooled, 1. "Subjects were eight 5-year-olds (M 5.56, SD 0.23; six female), eight 7-year-olds (M7.38, SD 0.29; six female), and 14 undergraduate students" 2. "Participating adolescents (n = 218) completed home interviews during the summers following their to give a total n for the entire article.
fifth-, sixth-, and seventh grade school years." 3. "The study initially enrolled 1,364 one-month-old infants and their families located in or near 10 sites across the United States. Because 115 participants were missing data on all study measures examined in this article, we utilized data from a subsample of 1,249 participants."

Gender Reported
Record whether study authors provide information on gender breakdown of their sample **If no, skip to racial/ethnic reporting section.

Total Men and Women
Record total number of participants identified as male and female in each study **If data are presented as a percentage, you must convert the percentage of total n into number of participants (i.e. if total n=150 and 26% are men, total men = 39; total women = 111) 1. "Over half of the women (62%) were Caucasian, 34% were African American, and 4% identified themselves as belonging to other racial groups." 2. "Participants in this study were 1,189 non-Hispanic telephone screen candidates who identified their race as either Black/African American (n = 819, 62%) or White/Caucasian (n = 370, 38%)."

International
If participants are recruited from outside the United States, regardless of country of origin, their race/ethnicity should be recorded as "international" 1. "Participants were recruited from seven secondary schools located in urban and suburban areas of the Netherlands." 2. "The sample of 135 preschoolers included 58 Farsispeaking Iranians in Shiraz, Iran, plus 77 Englishspeaking Australians"

Sexual Orientation Reported
Record whether study authors provide information on sexual orientation breakdown of their sample **If no, skip to low-income reporting section.

Total Sexual Orientation Breakdown
Record total number of participants identified as Gay, Lesbian, Bisexual, Transgender, or Other in each study 1. "Only male participants who reported that they had sex with males were included in the current analyses (N=122)." 2. "Forty-six percent identified as lesbian, 4% as gay, 29% as bisexual, 16% as queer, 2% as two-spirit, and 3% as other." **If data are presented as a percentage, you must convert the percentage of total n into number of participants.

Socioeconomic Status Reported
Record whether study authors provide information on the socioeconomic status of their sample

Mean, Standard Deviation & Range
If for the above income type means, standard deviation, and ranges are provided, they should be recorded here 1. "and socioeconomically diverse: participant mean annual income $31,070 (range $0-$200,000);" 2. "The average family income-to-needs ratio (combined across data collected at 6 and 15 months of age) was 3.7 (SD = 3.03)."

N of low-income
If, for the above income type, low-income Ns are provided, they should be recorded here 1. "57% received some form of government assistance" 2. "Older participants (hereafter referred to as patients) were recruited from a community-based medical clinic serving low income elderly" 3. "…and socioeconomic background (55% received free-or reduced-price lunch)."

Education Type
For studies reporting SES by describing the education level, or a similar measure, please code the type of measure used to describe the sample Parental education (years) = 1 Maternal education (years) = 2 % parental college attendance = 3 Education = 4 Education path to career type = 5 Paternal education (years) = 6 % high school completion = 7 1. "Women had a mean education level of 16 years (i.e., college educated; women's education ranged from 12 to 26 years" 2. "Participants were mostly of lower socioeconomic status (less than 8% of the participants had at least one parent with a 4-year college degree, and 31.6% did not Other = 8 (Describe) have a parent with a high-school diploma)" 3. "In terms of education, 9% of mothers had not completed high school, 11% were high school graduates, 20% completed specialized training or partial college, 30% completed a standard 4-year college degree, and 30% started or completed a graduate or professional degree."

Mean, Standard Deviation, and Range
If for the above education type means, standard deviations, and ranges are provided, they should be recorded here 3. "Data on socioeconomic status indicated mean parental levels of education falling between "some college, university or apprenticeship program" and "completed a college/apprenticeship/ technical diploma."

SES Other
If low-socioeconomic status is reported in a way that does not fit into any of the above categories, please record it here.

Other (describe)
Mean, Standard Deviation, and Range

Age
Mean age, when reported (mean, standard deviation, and range) will be recorded 1. "Youths' ages ranged from 7.15 to 13.97 years, with a mean of 10.62 years (SD=1.81)" 2. "Ages ranged from 18 to 33 with a mean of 21.55 (SD = 2.71)."

Age Category
Each sample will be categorized by general age range. 83# # Appendix C. Author Questionnaire The following questions are about your research interests, practices, and areas of expertise. I am interested in understanding how historically marginalized groupswomen, racial and ethnic, and sexual minorities, and low-income individuals -are being included in psychological science. Please answer as honestly and with as much detail as possible. 37. On a scale from 1 to 10, with 1 being never, and 10 being all of the time, how often do you recruit research participants from the following sources: -colleges -clinical settings -online (facebook, craiglist, etc) -community -other 38. On a scale from 1 to 10, with 1 being never, and 10 being all of the time, how often do you recruit specific racial or ethnic groups in your research studies? 39. On a scale from 1 to 10, with 1 being never, and 10 being all of the time, how often do you recruit specific age groups in your research studies? 40. On a scale from 1 to 10, with 1 being never, and 10 being all of the time, how often do you recruit specific SES groups in your research studies? 41. On a scale from 1 to 10, with 1 being never, and 10 being all of the time, how often do you recruit specific sexual minority groups in your research studies? 42. On a scale from 1 to 10, with 1 being never, and 10 being all of the time, how often do you specifically recruit men or women in your research studies? 43. On a scale from 1 to 10, with 1 being not at all important, and 10 being very important, how important a consideration is race and ethnicity to the generation of your research questions? 44. On a scale from 1 to 10, with 1 being not at all important, and 10 being very important, how important a consideration is sexual orientation to the generation of your research questions? 45. On a scale from 1 to 10, with 1 being not at all important, and 10 being very important, how important a consideration is gender to the generation of your research questions?

88#
# 46. On a scale from 1 to 10, with 1 being not at all important, and 10 being very important, how important a consideration is SES to the generation of your research questions? 47. Have issues related to race or ethnicity been raised during a manuscript review process? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 48. Have issues related to race or ethnicity ever influenced the likelihood your manuscript would be published? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 49. Have issues related to race or ethnicity ever influenced where you sent an article for consideration? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 50. Have issues related to gender been raised during a manuscript review process? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 51. Have issues related to gender ever influenced the likelihood your manuscript would be published? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 52. Have issues related to gender ever influenced where you sent an article for consideration? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 53. Have issues related to SES been raised during a manuscript review process? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 54. Have issues related to SES ever influenced the likelihood your manuscript would be published? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 55. Have issues related to SES ever influenced where you sent an article for consideration? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always

89#
# 56. Have issues related to age been raised during a manuscript review process? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 57. Have issues related to age ever influenced the likelihood your manuscript would be published? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 58. Have issues related to age ever influenced where you sent an article for consideration? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 59. Have issues related to sexual orientation been raised during a manuscript review process? 0) Never 1) Rarely 2) Sometimes 3) Often 4) Always 60. Have issues related to sexual orientation ever influenced the likelihood your manuscript would be published? 67. Thinking about your discipline and your specific area of research interest, on a scale from 1 to 10, with 1 being not at all important and 10 being very important, how important are issues of race and ethnicity to answering your research questions? 68. Thinking about your discipline and your specific area of research interest, on a scale from 1 to 10, with 1 being not at all important and 10 being very important, how important are issues of gender to answering your research questions? 69. Thinking about your discipline and your specific area of research interest, on a scale from 1 to 10, with 1 being not at all important and 10 being very important, how important are issues of SES to answering your research questions? 70. Thinking about your discipline and your specific area of research interest, on a scale from 1 to 10, with 1 being not at all important and 10 being very important, how important are issues of sexual orientation to answering your research questions? 71. Thinking about your discipline and your specific area of research interest, on a scale from 1 to 10, with 1 being not at all important and 10 being very important, how important are issues age to answering your research questions? 72. On a scale from 1 to 10, with 1 being not at all and 10 being extremely, how likely are you to design a future study with a focus on historically marginalized groups?

Initial Contact Email:
Dear ____ My name is Celeste Caviness. I am a doctoral degree candidate in the psychology department at the University of Rhode Island. My dissertation project is focused on historically marginalized groups -women, racial, ethnic, and sexual minorities, and lowincome individuals -and their representation in psychological science. As part of this project, I am interested in the opinions of researchers such as yourself.
In one week, you will receive an email invitation to participate in a one-time, anonymous, online survey about historically marginalized groups in psychological science. The survey will take approximately 15-20 minutes to complete. You are being invited to participate because you published a research article in either, Developmental Psychology, Neuropsychology, Journal of Consulting and Clinical Psychology, Health Psychology, or Journal of Personality and Social Psychology in 2012 and were identified as the corresponding author.
Your research does not have to have a specific focus on historically marginalized groups to participate. a JCCP = Journal of Consulting and Clinical Psychology; DP = Developmental Psychology; HP = Health Psychology; NP = Neuropsychology; JPSP = Journal Personality and Social Psychology b In single racial or ethnic group studies, based on how the data were presented, (for example, no further delineation by Hispanic or Asian ethnic group), no subgroup analyses by race or ethnicity would be run. Therefore, an exploration of power for subgroup analyses is unnecessary.  2008-2013 2003-2007 1998-2002 1993-1997 Before 1992 (51) 48.45% (47) 38.14% (37) 27.27% (3) 63.64% (7) 36.36% (8)