Impact of non-informative alteration in assessment: Identification and corrective procedures
The accuracy of psychological and neuropsychological evaluation may be degraded when non-informative alteration (NIA) in data, or alteration in the manner data are represented but not in core meaning, impacts interpretation. If NIA exerts an impact, it may also lead to underutilization of truly useful information. Certain interpretive practices that are based on configural relationships and are already problematic (e.g., scatter analysis) may be particularly vulnerable to NIA and thereby further compromised. This study examined: (1) judgments regarding inter-test scatter across a neuropsychological battery, (2) whether NIAs impact judgments regarding scatter, and (3) whether truncating the visual presentation of scatter alters misjudgments about the frequency or pathological significance of scatter. Participants (N = 193) were neuropsychologists and graduate students who have received training in neuropsychological assessment. When judging neuropsychological profiles, participants markedly overperceived normal levels of scatter as rare or aberrant. The influence of NIA was mixed. Changing the visual plotting of percentiles from equal-to unequal-sized units did not alter judgments. In contrast, simply changing the designated metric from percentiles to T-scores, while holding visual plotting constant, reduced overperception of scatter, although only partially or insufficiently. An intervention that truncated visual scatter further improved judgmental accuracy (i.e., truncated visual scatter compared to larger visual scatter with mathematically identical information attenuated misjudgments about the normality of scatter). This study provides preliminary evidence for a previously underidentified source of error in the interpretation of psychological test data. Future research should determine whether the current findings can be replicated, advance the design of interventions as needed, and assist in developing evidence-based standards for representing graphical displays that diminish the influence of NIAs. ^
Aaron M Baker,
"Impact of non-informative alteration in assessment: Identification and corrective procedures"
Dissertations and Master's Theses (Campus Access).