Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Comparatively salient: examining the influence of preceding performances on assessors’ focus and interpretations in written assessment comments

Comparatively salient: examining the influence of preceding performances on assessors’ focus and... Recent literature places more emphasis on assessment comments rather than relying solely on scores. Both are variable, however, emanating from assessment judgements. One established source of variability is “contrast effects”: scores are shifted away from the depicted level of competence in a preceding encounter. The shift could arise from an effect on the range-frequency of assessors’ internal scales or the salience of performance aspects within assessment judgments. As these suggest different potential interventions, we investigated assessors’ cognition by using the insight provided by “clusters of consensus” to determine whether any change in the salience of performance aspects was induced by contrast effects. A dataset from a previous experiment contained scores and comments for 3 encounters: 2 with significant contrast effects and 1 without. Clusters of consensus were identified using F-sort and latent partition analysis both when contrast effects were significant and non-significant. The proportion of assessors making similar comments only significantly differed when contrast effects were significant with assessors more frequently commenting on aspects that were dissimilar with the standard of competence demonstrated in the preceding performance. Rather than simply influencing range-frequency of assessors’ scales, preceding performances may affect salience of performance aspects through comparative distinctiveness: when juxtaposed with the context some aspects are more distinct and selectively draw attention. Research is needed to determine whether changes in salience indicate biased or improved assessment information. The potential should be explored to augment existing benchmarking procedures in assessor training by cueing assessors’ attention through observation of reference performances immediately prior to assessment. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Advances in Health Sciences Education Springer Journals

Comparatively salient: examining the influence of preceding performances on assessors’ focus and interpretations in written assessment comments

Download PDF
23 pages

Loading next page...
 
/lp/springer-journals/comparatively-salient-examining-the-influence-of-preceding-AmHHzoKYT6

References (39)

Publisher
Springer Journals
Copyright
Copyright © 2018 by Springer Nature B.V.
Subject
Education; Medical Education
ISSN
1382-4996
eISSN
1573-1677
DOI
10.1007/s10459-018-9841-2
Publisher site
See Article on Publisher Site

Abstract

Recent literature places more emphasis on assessment comments rather than relying solely on scores. Both are variable, however, emanating from assessment judgements. One established source of variability is “contrast effects”: scores are shifted away from the depicted level of competence in a preceding encounter. The shift could arise from an effect on the range-frequency of assessors’ internal scales or the salience of performance aspects within assessment judgments. As these suggest different potential interventions, we investigated assessors’ cognition by using the insight provided by “clusters of consensus” to determine whether any change in the salience of performance aspects was induced by contrast effects. A dataset from a previous experiment contained scores and comments for 3 encounters: 2 with significant contrast effects and 1 without. Clusters of consensus were identified using F-sort and latent partition analysis both when contrast effects were significant and non-significant. The proportion of assessors making similar comments only significantly differed when contrast effects were significant with assessors more frequently commenting on aspects that were dissimilar with the standard of competence demonstrated in the preceding performance. Rather than simply influencing range-frequency of assessors’ scales, preceding performances may affect salience of performance aspects through comparative distinctiveness: when juxtaposed with the context some aspects are more distinct and selectively draw attention. Research is needed to determine whether changes in salience indicate biased or improved assessment information. The potential should be explored to augment existing benchmarking procedures in assessor training by cueing assessors’ attention through observation of reference performances immediately prior to assessment.

Journal

Advances in Health Sciences Education – Springer Journals

Published: Jul 6, 2018

There are no references for this article.