Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Two models of raters in a structured oral examination: does it make a difference?

Two models of raters in a structured oral examination: does it make a difference? Oral examinations have become more standardized over recent years. Traditionally a small number of raters were used for this type of examination. Past studies suggested that more raters should improve reliability. We compared the results of a multi-station structured oral examination using two different rater models, those based in a station, (station-specific raters), and those who follow a candidate throughout the entire examination, (candidate-specific raters).Two station-specific and two candidate-specific raters simultaneously evaluated internal medicine residents’ performance at each station. No significant differences were found in examination scores. Reliability was higher for the candidate-specific raters. Inter-rater reliability, internal consistency and a study of station inter-correlations suggested that a halo effect may be present for candidates examined by candidate-specific raters. This study suggests that although the model of candidate-specific raters was more reliable than the model of station-specific raters for the overall examination, the presence of a halo effect may influence individual examination outcomes. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Advances in Health Sciences Education Springer Journals

Two models of raters in a structured oral examination: does it make a difference?

Download PDF
12 pages

Loading next page...
 
/lp/springer-journals/two-models-of-raters-in-a-structured-oral-examination-does-it-make-a-Vy601tETOC

References (24)

Publisher
Springer Journals
Copyright
Copyright © 2009 by Springer Science+Business Media B.V.
Subject
Education; Medical Education
ISSN
1382-4996
eISSN
1573-1677
DOI
10.1007/s10459-009-9175-1
pmid
19657717
Publisher site
See Article on Publisher Site

Abstract

Oral examinations have become more standardized over recent years. Traditionally a small number of raters were used for this type of examination. Past studies suggested that more raters should improve reliability. We compared the results of a multi-station structured oral examination using two different rater models, those based in a station, (station-specific raters), and those who follow a candidate throughout the entire examination, (candidate-specific raters).Two station-specific and two candidate-specific raters simultaneously evaluated internal medicine residents’ performance at each station. No significant differences were found in examination scores. Reliability was higher for the candidate-specific raters. Inter-rater reliability, internal consistency and a study of station inter-correlations suggested that a halo effect may be present for candidates examined by candidate-specific raters. This study suggests that although the model of candidate-specific raters was more reliable than the model of station-specific raters for the overall examination, the presence of a halo effect may influence individual examination outcomes.

Journal

Advances in Health Sciences Education – Springer Journals

Published: Mar 1, 2010

There are no references for this article.