Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

How Can I Know What I Don't Know? Poor Self Assessment in a Well-Defined Domain

How Can I Know What I Don't Know? Poor Self Assessment in a Well-Defined Domain As the rapidity with which medical knowledge isgenerated and disseminated becomes amplified,an increasing emphasis has been placed on theneed for physicians to develop the skillsnecessary for life-long learning. One suchskill is the ability to evaluate one's owndeficiencies. A ubiquitous finding in thestudy of self-assessment, however, is thatself-ratings are poorly correlated with otherperformance measures. Still, many educatorsview the ability to recognize and communicateone's deficiencies as an important component ofadult learning. As a result, two studies havebeen performed in an attempt to improve uponthis status quo.First, we tried to re-define the limits withinwhich self-assessments should be used, usingRosenblit and Keil's argument that calibrationbetween perceived and actual performance willbe better within taxonomies that are regularlytested (e.g., factual knowledge) compared tothose that are not (e.g., conceptualknowledge). Second, we tried to norm referenceindividuals based on both the performance oftheir colleagues and their own historicalperformance on McMaster's Personal ProgressInventory (a multiple choice question test ofmedical knowledge).While it appears that students are able to (a)make macro-level self-assessments (i.e., torecognize that third year students typicallyoutperform first year students), and (b) judgetheir performance relatively accurately afterthe fact, students were unable to predict thepercentage of questions they would answercorrectly with a testing procedure in whichthey have had a substantial amount of feedback. Previous test score was a much better predictorof current test performance than wereindividuals' expectations. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Advances in Health Sciences Education Springer Journals

How Can I Know What I Don't Know? Poor Self Assessment in a Well-Defined Domain

Download PDF
14 pages

Loading next page...
 
/lp/springer-journals/how-can-i-know-what-i-don-t-know-poor-self-assessment-in-a-well-m2BIVQO6UU
Publisher
Springer Journals
Copyright
Copyright © 2004 by Kluwer Academic Publishers
Subject
Education; Medical Education
ISSN
1382-4996
eISSN
1573-1677
DOI
10.1023/B:AHSE.0000038209.65714.d4
pmid
15316272
Publisher site
See Article on Publisher Site

Abstract

As the rapidity with which medical knowledge isgenerated and disseminated becomes amplified,an increasing emphasis has been placed on theneed for physicians to develop the skillsnecessary for life-long learning. One suchskill is the ability to evaluate one's owndeficiencies. A ubiquitous finding in thestudy of self-assessment, however, is thatself-ratings are poorly correlated with otherperformance measures. Still, many educatorsview the ability to recognize and communicateone's deficiencies as an important component ofadult learning. As a result, two studies havebeen performed in an attempt to improve uponthis status quo.First, we tried to re-define the limits withinwhich self-assessments should be used, usingRosenblit and Keil's argument that calibrationbetween perceived and actual performance willbe better within taxonomies that are regularlytested (e.g., factual knowledge) compared tothose that are not (e.g., conceptualknowledge). Second, we tried to norm referenceindividuals based on both the performance oftheir colleagues and their own historicalperformance on McMaster's Personal ProgressInventory (a multiple choice question test ofmedical knowledge).While it appears that students are able to (a)make macro-level self-assessments (i.e., torecognize that third year students typicallyoutperform first year students), and (b) judgetheir performance relatively accurately afterthe fact, students were unable to predict thepercentage of questions they would answercorrectly with a testing procedure in whichthey have had a substantial amount of feedback. Previous test score was a much better predictorof current test performance than wereindividuals' expectations.

Journal

Advances in Health Sciences Education – Springer Journals

Published: Sep 21, 2004

References

  • Self-evaluation in undergraduate medical education: A longitudinal perspective
    Arnold, L.; Willoughby, T.L.; Calkins, E.V.
  • Report card from McMaster: student evaluation at a problem-based medical school.
    Blake, J.M.; Norman, G.R.; Smith, E.K.M.
  • Introducing progress testing inMcMaster University's problem-based medical curriculum: psychometric properties and effect on learning
    Blake, J.M.; Norman, G.R.; Keane, D.R.; Mueller, C.B.; Cunnington, J.; Didyk, N.
  • Applying Learning Taxonomies to Test Items: is a fact an artifact
    Cunnington, J.P.W.; Norman, G.R.; Blake, J.M.; Dauphinee, W.D.; Blackmore, D.E.
  • The effect of changes in medical knowledge on examination performance at the time of recertification
    Day, S.C.; Norcini, J.J.; Webster, G.D.; Viner, E.D.; Chirico, A.M.
  • A review of the validity and accuracy of self-assessments in health professions training
    Gordon, M.J.
  • Self-assessment programs and their implications for health professions training
    Gordon, M.J.
  • Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments.
    Kruger, J.; Dunning, D.
  • Self-assessment in problem-based groups
    Rezler, A.
  • The misunderstood limits of folk science: an illusion of explanatory depth
    Rozenblit, L.R.; Keil, F.C.
  • The need for evidence-based medicine
    Sackett, D.L.; Rosenberg, W.M.
  • Effect of problem-based, self-directed undergraduate education on life-long learning
    Shin, J.H.; Haynes, R.B.; Johnston, M.E.
  • A randomized trial of continuing medical education
    Sibley, J.C.; Sackett, D.L.; Neufeld, V.; Gerrard, B.; Rudnick, K.V.; Fraser, W.
  • Essentials of problem-based learning
    Walton, H.J.; Matthews, M.B.
  • Measuring self-assessment: current state of the art
    Ward, M.; Gruppen, L.; Regehr, G.