Access the full text.
Sign up today, get DeepDyve free for 14 days.
JM Blake, GR Norman, DR Keane, CB Mueller, J Cunnington, N Didyk (1996)
Introducing progress testing in McMaster University’s problem-based medical curriculum: Psychometric properties and effect on learningAcademic Medicine, 71
AM Muijtjens, HV Mameren, RJ Hoogenboom, JL Evers, VD Vleuten (1999)
The effect of a ‘don’t know’ option on test scores: Number-right and formula scoring comparedMedical Education, 33
C Ravesloot, M Schaaf, C Haaring, C Kruitwagen, E Beek, CO Ten (2012)
Construct validation of progress testing to measure knowledge and visual skills in radiologyMedical Teacher, 34
S Messick (1989)
Educational measurement
(Kampmeyer, D., Matthes, J., & Herzig, S. (2014). Lucky guess or knowledge: A cross-sectional study using the Bland and Altman analysis to compare confidence-based testing of pharmacological knowledge in 3rd and 5th year medical students. Advances in Health Sciences Education Theory Practice.)
Kampmeyer, D., Matthes, J., & Herzig, S. (2014). Lucky guess or knowledge: A cross-sectional study using the Bland and Altman analysis to compare confidence-based testing of pharmacological knowledge in 3rd and 5th year medical students. Advances in Health Sciences Education Theory Practice.Kampmeyer, D., Matthes, J., & Herzig, S. (2014). Lucky guess or knowledge: A cross-sectional study using the Bland and Altman analysis to compare confidence-based testing of pharmacological knowledge in 3rd and 5th year medical students. Advances in Health Sciences Education Theory Practice., Kampmeyer, D., Matthes, J., & Herzig, S. (2014). Lucky guess or knowledge: A cross-sectional study using the Bland and Altman analysis to compare confidence-based testing of pharmacological knowledge in 3rd and 5th year medical students. Advances in Health Sciences Education Theory Practice.
LB Bliss (1980)
A test of lord’s assumption regarding examinee guessing behavior on multiple-choice tests using elementary school studentsJournal of Educational Measurement, 17
D Mattson (1965)
The effects of guessing on the standard error of measurement and the reliability of test scoresEducational and Psychological Measurement, 25
AM Muijtjens, HV Mameren, RJ Hoogenboom, JPM Evers, KML Geraerdts, CPM Leunissen (1997)
De vraagtekenoptie bij juist/onjuist-items in toetsen: goed of fout?Bulletin Medisch Onderwijs, 16
TB Kurz (1999)
A review of scoring algorithms for multiple-choice tests
RF Burton (2004)
Multiple choice and true/false tests: Reliability measures and some implications of negative markingAssessment and Evaluation in Higher Education, 29
JP Byrnes, DC Miller, WD Schafer (1999)
Gender differences in risk taking: A meta-analysisPsychological Bulletin, 125
AR Hakstian, TE Whalen (1976)
A K-sample significance test for independent alpha coefficientsPsychometrika, 41
S Messick (1995)
Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaningAmerican Psychologist, 50
FM Lord (1963)
Formula scoring and validityEducational and Psychological Measurement, 23
MA Albanese (1988)
The projected impact of the correction for guessing on individual scoresJournal of Educational Measurement, 25
J Diamond, W Evans (1973)
The correction for guessingReview of Educational Research, 43
TZ Keith (2006)
Multiple regression and beyond
S Kelly, R Dennick (2009)
Evidence of gender bias in true–false–abstain medical examinationsBMC Medical Education, 9
DB Swanson, KZ Holtzman, A Butler, MM Langer, MV Nelson, JW Chow (2010)
Collaboration across the pond: The multi-school progress testing projectMedical Teacher, 32
FM Lord (1975)
Formula scoring and number-right scoringJournal of Educational Measurement, 12
J McHarg, P Bradley, S Chamberlain, C Ricketts, J Searle, JC McLachlan (2005)
Assessment of progress testsMedical Education, 39
JW Cohen (1988)
Statistical power analysis for the behavioural sciences
(1989)
ValidityEducational measurement
Formula scoring (FS) is the use of a don’t know option (DKO) with subtraction of points for wrong answers. Its effect on construct validity and reliability of progress test scores, is subject of discussion. Choosing a DKO may not only be affected by knowledge level, but also by risk taking tendency, and may thus introduce construct-irrelevant variance into the knowledge measurement. On the other hand, FS may result in more reliable test scores. To evaluate the impact of FS on construct validity and reliability of progress test scores, a progress test for radiology residents was divided into two tests of 100 parallel items (A and B). Each test had a FS and a number-right (NR) version, A-FS, B-FS, A-NR, and B-NR. Participants (337) were randomly divided into two groups. One group took test A-FS followed by B-NR, and the second group test B-FS followed by A-NR. Evidence for impaired construct validity was sought in a hierarchical regression analysis by investigating how much of the participants’ FS-score variance was explained by the DKO-score, compared to the contribution of the knowledge level (NR-score), while controlling for Group, Gender, and Training length. Cronbach’s alpha was used to estimate NR and FS-score reliability per year group. NR score was found to explain 27 % of the variance of FS [F(1,332) = 219.2, p < 0.0005], DKO-score, and the interaction of DKO and Gender were found to explain 8 % [F(2,330) = 41.5, p < 0.0005], and the interaction of DKO and NR 1.6 % [F(1,329) = 16.6, p < 0.0005], supporting our hypothesis that FS introduces construct-irrelevant variance into the knowledge measurement. However, NR-scores showed considerably lower reliabilities than FS-scores (mean year-test group Cronbach’s alphas were 0.62 and 0.74, respectively). Decisions about FS with progress tests should be a careful trade-off between systematic and random measurement error.
Advances in Health Sciences Education – Springer Journals
Published: Apr 26, 2015
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.