Limitations of Performance Validity Tests in Dementia Evaluations: The Role of Base Rates

Document Type

Article

Date of Original Version

9-22-2022

Abstract

Performance validity tests (PVTs) are frequently used to detect invalid performance on cognitive testing. The inclusion of PVTs in cognitive test batteries is commonplace irrespective of the condition of interest. However, base rates of invalid performance vary across clinical populations. Research accounting for base rates of invalid performance in varying clinical populations and PVT classification accuracy rates are not commonly synthesized. To address this gap, the present study examined the clinical utility of select PVTs used with older adults presenting for dementia evaluations. We computed posterior probabilities of invalid performance for the select PVTs using an estimated 5% base rate of invalid performance based on prior published studies. Posterior probabilities of invalid performance based on a PVT failure (i.e., invalid performance identified as invalid) ranged from 7.3% to 60.3% across PVTs; posterior probabilities of a false positive (i.e., valid performance identified as invalid) ranged from 39.7% to 92.7%. Conversely, posterior probabilities of a true negative (i.e., valid performance identified as valid) ranged from 95.7% to 99.3%; posterior probabilities of a false negative (i.e., invalid performance identified as valid) ranged from 0.7% to 4.3%. Results call into question the utility of PVTs in dementia evaluations. Consequently, the use of PVTs in dementia evaluations is likely to erroneously identify valid test data as invalid (i.e., false-positive error) at a frequency that exceeds the estimated 5% base rate of invalid performance. Further research examining correlates of invalid performance among older adults will clarify base rate estimates and potentially enhance the utility of PVTs.

Publication Title, e.g., Journal

Psychological Assessment

Volume

34

Issue

11

Link to Full Text

Share

COinS