Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Quantifying the Distorting Effect of Rapid Guessing on Estimates of Coefficient Αlpha

Quantifying the Distorting Effect of Rapid Guessing on Estimates of Coefficient Αlpha An underlying threat to the validity of reliability measures is the introduction of systematic variance in examinee scores from unintended constructs that differ from those assessed. One construct-irrelevant behavior that has gained increased attention in the literature is rapid guessing (RG), which occurs when examinees answer quickly with intentional disregard for item content. To examine the degree of distortion in coefficient alpha due to RG, this study compared alpha estimates between conditions in which simulees engaged in full solution (i.e., do not engage in RG) versus partial RG behavior. This was done by conducting a simulation study in which the percentage and ability characteristics of rapid responders as well as the percentage and pattern of RG were manipulated. After controlling for test length and difficulty, the average degree of distortion in estimates of coefficient alpha due to RG ranged from −.04 to .02 across 144 conditions. Although slight differences were noted between conditions differing in RG pattern and RG responder ability, the findings from this study suggest that estimates of coefficient alpha are largely robust to the presence of RG due to cognitive fatigue and a low perceived probability of success. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Applied Psychological Measurement SAGE

Quantifying the Distorting Effect of Rapid Guessing on Estimates of Coefficient Αlpha

Rios, Joseph A.; Deng, Jiayi
Applied Psychological Measurement , Volume 46 (1): 13 – Jan 1, 2022
Download PDF
13 pages

Loading next page...
 
/lp/sage/quantifying-the-distorting-effect-of-rapid-guessing-on-estimates-of-CSZ2gGZLh7

References (37)

Publisher
SAGE
Copyright
© The Author(s) 2021
ISSN
0146-6216
eISSN
1552-3497
DOI
10.1177/01466216211051719
Publisher site
See Article on Publisher Site

Abstract

An underlying threat to the validity of reliability measures is the introduction of systematic variance in examinee scores from unintended constructs that differ from those assessed. One construct-irrelevant behavior that has gained increased attention in the literature is rapid guessing (RG), which occurs when examinees answer quickly with intentional disregard for item content. To examine the degree of distortion in coefficient alpha due to RG, this study compared alpha estimates between conditions in which simulees engaged in full solution (i.e., do not engage in RG) versus partial RG behavior. This was done by conducting a simulation study in which the percentage and ability characteristics of rapid responders as well as the percentage and pattern of RG were manipulated. After controlling for test length and difficulty, the average degree of distortion in estimates of coefficient alpha due to RG ranged from −.04 to .02 across 144 conditions. Although slight differences were noted between conditions differing in RG pattern and RG responder ability, the findings from this study suggest that estimates of coefficient alpha are largely robust to the presence of RG due to cognitive fatigue and a low perceived probability of success.

Journal

Applied Psychological Measurement – SAGE

Published: Jan 1, 2022

There are no references for this article.