Errors in Scoring Objective Personality Tests

Author

Gregory Allard, University of Rhode Island

Date of Award

1997

Degree Type

Thesis

Degree Name

Master of Arts in Psychology

Department

Psychology

First Advisor

David Faust

Abstract

Scoring objective personality tests is considered clerical, and presumably, straightforward in nature. This may be the reason that few studies, if any, have investigated the impact of scoring error on widely used tests, such as the MMPI or 801. Errors, even if infrequent (e.g. as few as 1% of tests), may adversely affect many hundreds or thousands of tests administered annually, however. In a study of three popular tests taken from three independent settings, this study found that the interpretation of popular tests are vulnerable even to small errors (i.e., 1 or 2 misscored items per test). This study explored the influence of two factors, scoring procedure complexity and commitment to scoring accuracy, hypothesized to be related to the occurrence of scoring error with fewer errors occurring when higher commitment to accuracy and lower scoring procedure complexity are present. The scoring procedure complexity effect was predicted to be subordinate to the commitment to accuracy effect. Three popular tests were sampled from three different settings and rescored to check for accuracy. Twenty-one percent of tests scored with low commitment to accuracy were erroneous, while tests scored with full commitment to accuracy had 1% errors. Scoring procedure complexity, categorized as high and low, yielded 29% and 14% erroneous tests, respectively, in the less than full commitment to accuracy sample, and 0 and 4% in the full commitment to accuracy sample. The results provide strong support for the factors as major predictors of scoring error, as well as the interaction effect anticipated. Other risk factors, such as commercial computer scoring errors and lack of agreement on test scoring standards, were also found to distort scores. The frequency and severity of erroneous findings in this study, the author argues, are unlikely to be specific to this study, but instead more general. The author shows how awareness of the two factors, as well as other sources of error, can be used to reduce the risk of scoring error and offers practical recommendations to improve scoring accuracy.

Recommended Citation

Allard, Gregory, "Errors in Scoring Objective Personality Tests" (1997). Open Access Master's Theses. Paper 1105.
https://digitalcommons.uri.edu/theses/1105