Extreme group comparisons: Nature, prevalence, and impact on psychological research
Normative comparisons for psychological and neuropsychological measures, as well as predictions made from studies of specific populations, suffer when such groups are improperly constructed. It was posited such inappropriate group formation is common in psychological science, partly from a between-group failure to identify the clinical distinction of interest, and in part from a within-group failure to mirror the characteristics of the overall population by placing undue emphasis on group purity (that is, exclusion of participants with comorbid conditions). Furthermore, participants are often selected for group membership due to the extremity of the condition, eliminating diagnostic ambiguity but sacrificing generalizability to less severe cases. It was hypothesized this improper group formation can be readily detected through inflated effect sizes and, subsequently, spuriously magnified meta-analyses. This study sough to establish the nature of such a relationship through several smaller investigations. Foremost, an exploration of the literature revealed inflated effect sizes to be prevalent in the investigated domains. Second, reviewed a volume of a prominent experimental journal to reveal high levels of exclusionary criteria were common. Next, an examination of the Flynn effect strongly supported a relationship between number of exclusionary criteria and increasing magnitude of effect. Evidence was likewise found for a relationship between exclusionary criteria and very large effect sizes throughout four meta-analyses of malingering detection with the MMPI-2. Finally, it was demonstrated some preliminary corrective measures may be feasible for properly interpreting literature prone to such inflation of effects. ^
Psychology, Clinical|Psychology, Psychometrics
David C Ahern,
"Extreme group comparisons: Nature, prevalence, and impact on psychological research"
Dissertations and Master's Theses (Campus Access).