Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Vitalizing the evaluation of curricular implementation: a framework for attending to the “how and whys” of curriculum evolution

Vitalizing the evaluation of curricular implementation: a framework for attending to the “how and... As curricular reforms are implemented, there is often urgency among scholars to swiftly evaluate curricular outcomes and establish whether desired impacts have been realized. Consequently, many evaluative studies focus on summative program outcomes without accompanying evaluations of implementation. This runs the risk of Type III errors, whereby outcome evaluations rest on unverified assumptions about the appropriate implementation of prescribed curricular activities. Such errors challenge the usefulness of the evaluative studies, casting doubt on accumulated knowledge about curricular innovations, and posing problems for educational systems working to mobilize scarce resources. Unfortunately, however, there is long-standing inattention to the evaluation of implementation in health professions education (HPE). To address this, we propose an accessible framework that provides substantive guidance for evaluative research on implementation of curricular innovations. The Prescribed-Intended-Enacted-Sustainable (PIES) framework that is articulated in this paper, introduces new concepts to HPE—with a view to facilitating more nuanced examination of the evolution of curricula as they are implemented. Critically, the framework is theoretically grounded, integrating evaluation and implementation science as well as education theory. It outlines when, how, and why evaluators need to direct attention to curricular implementation, providing guidance on how programs can map out meaningful evaluative research agendas. Ultimately, this work is intended to support evaluators and educators, seeking to design evaluation studies that provide more faithful, useful representations of the intricacies of curricular change implementation. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Advances in Health Sciences Education Springer Journals

Vitalizing the evaluation of curricular implementation: a framework for attending to the “how and whys” of curriculum evolution

Download PDF
20 pages

Loading next page...
 
/lp/springer-journals/vitalizing-the-evaluation-of-curricular-implementation-a-framework-for-Il2Es0nber
Publisher
Springer Journals
Copyright
Copyright © The Author(s), under exclusive licence to Springer Nature B.V. 2021
ISSN
1382-4996
eISSN
1573-1677
DOI
10.1007/s10459-021-10083-6
Publisher site
See Article on Publisher Site

Abstract

As curricular reforms are implemented, there is often urgency among scholars to swiftly evaluate curricular outcomes and establish whether desired impacts have been realized. Consequently, many evaluative studies focus on summative program outcomes without accompanying evaluations of implementation. This runs the risk of Type III errors, whereby outcome evaluations rest on unverified assumptions about the appropriate implementation of prescribed curricular activities. Such errors challenge the usefulness of the evaluative studies, casting doubt on accumulated knowledge about curricular innovations, and posing problems for educational systems working to mobilize scarce resources. Unfortunately, however, there is long-standing inattention to the evaluation of implementation in health professions education (HPE). To address this, we propose an accessible framework that provides substantive guidance for evaluative research on implementation of curricular innovations. The Prescribed-Intended-Enacted-Sustainable (PIES) framework that is articulated in this paper, introduces new concepts to HPE—with a view to facilitating more nuanced examination of the evolution of curricula as they are implemented. Critically, the framework is theoretically grounded, integrating evaluation and implementation science as well as education theory. It outlines when, how, and why evaluators need to direct attention to curricular implementation, providing guidance on how programs can map out meaningful evaluative research agendas. Ultimately, this work is intended to support evaluators and educators, seeking to design evaluation studies that provide more faithful, useful representations of the intricacies of curricular change implementation.

Journal

Advances in Health Sciences Education – Springer Journals

Published: May 1, 2022

Keywords: Program evaluation; Curriculum implementation; Curricular innovation

References

  • Completeness of the reporting of evidence-based practice educational interventions: A review
    Albarqouni, L; Glasziou, P; Hoffmann, T
  • Definitions of evaluation use and misuse, evaluation influence, and factors affecting use
    Alkin, MC; King, JA
  • Aligning and applying the paradigms and practices of education
    Baker, L; Shing, LK; Wright, S; Mylopoulos, M; Kulasegaram, K; Ng, S
  • Avoiding type III errors in health education program evaluations: A case study
    Basch, CE; Sliepcevich, EM; Gold, RS; Duncan, DF; Kolbe, LJ
  • The role of metalearning in study processes
    Biggs, J
  • Connecting curriculum materials and teachers: Elementary science teachers’ enactment of a reform-based curricular unit
    Bismack, AS; Arias, AM; Davis, EA; Palincsar, AS
  • Competency-based medical education: The discourse of infallibility
    Boyd, VA; Whitehead, CR; Thille, P; Ginsburg, S; Brydges, R; Kuper, A
  • Accounting for discrepancies in teachers’ attitudes towards evidence use and actual instances of evidence use in schools
    Brown, C; Zhang, D
  • Resident learning trajectories in the workplace: A self-regulated learning analysis
    Brydges, R; Tran, J; Goffi, A; Lee, C; Miller, DMM
  • Exercising a bounded autonomy: Novice and experienced teachers’ adaptations to curriculum materials in an age of accountability
    Burkhauser, MA; Lesaux, NK
  • Fidelity of implementation: Bringing written curriculum materials into the equation
    Castro Superfine, A; Marshall, AM; Kelso, C
  • Overarching challenges to the implementation of competency-based medical education
    Caverzagie, KJ; Nousiainen, MT; Ferguson, PC
  • Learning theory and educational intervention: Producing meaningful evidence of impact through layered analysis
    Cianciolo, AT; Regehr, G
  • Creating an “education shark tank” to encourage and support educational scholarship and innovation
    Cofrancesco, J; Wright, SM; Vohr, E; Ziegelstein, RC
  • Implementing communication and decision-making interventions directed at goals of care: a theory-led scoping review
    Cummings, A; Lund, S; Campling, N; May, C; Richardson, A; Myall, M
  • Dynamic tensions following new pedagogy in undergraduate medical education
    Daneman, D; Benatar, S
  • Avoiding type III error in program evaluation. Results from a field experiment
    Dobson, D; Cook, TJ
  • Practicing change: Curriculum adaptation and teacher narrative in the context of mathematics education reform
    Drake, C; Sherin, MG
  • In search of the school curriculum
    Gehrke, NJ; Knapp, MS; Sirotnik, KA
  • RE-AIM planning and evaluation framework: Adapting to new science and practice with a 20-year review
    Glasgow, RE; Harden, SM; Gaglio, B
  • Errors in administrative education data: A cautionary tale
    Goldhaber, D; Holden, KL; Grout, C
  • Beyond curriculum reform: Confronting medicine’s hidden curriculum
    Hafferty, FW
  • Balancing fidelity with flexibility and fit: What do we really know about fidelity of implementation in schools?
    Harn, B; Parisi, D; Stoolmiller, M
  • Complex interventions: How “out of control” can a randomised controlled trial be?
    Hawe, P; Shiell, A; Riley, T
  • Getting off the carousel: Exploring the wicked problem of curriculum reform
    Hawick, L; Cleland, J; Kitto, S
  • Curriculum reform: The more things change, the more they stay the same?
    Hawick, L; Kitto, S; Cleland, J
  • Implementation of competency-based medical education: Are we addressing the concerns and challenges?
    Hawkins, RE; Welcher, CM; Holmboe, ES
  • Better reporting of interventions: Template for intervention description and replication (TIDieR) checklist and guide
    Hoffmann, TC; Glasziou, PP; Boutron, I
  • A call to action: The controversy of and rationale for competency-based medical education
    Holmboe, ES; Sherbino, J; Englander, R; Snell, L; Frank, JR
  • When are two interventions the same ? Implications for reporting guidelines in education
    Horsley, T; Regehr, G
  • When are two interventions the same? Implications for reporting guidelines in education
    Horsley, T; Regehr, G
  • Using a framework to implement large-scale innovation in medical education with the intent of achieving sustainability
    Hudson, JN; Farmer, EA; Weston, KM; Bushnell, JA
  • Do no harm Risk aversion versus risk management in the context of pedagogic frailty
    Hulme, JA; Winstone, NE
  • Educational innovations in academic medicine and environmental trends
    Irby, DM; Wilkerson, LA
  • Developing the concept of metalearning
    Jackson, N
  • The clock is ticking: Time constraint issues in mathematics teaching reform
    Keiser, JM; Lambdin, DV
  • Using developmental evaluation as a design thinking tool for curriculum innovation in professional higher education curriculum innovation in professional higher education
    Leonard, SN; Fitzgerald, RN; Riordan, G
  • Time pressure and instructional choices when teaching mathematics
    Leong, YH; Chick, HL
  • Supporting student awareness: Understanding student preconceptions of their subject matter within introductory courses
    Lucas, U; Meyer, JHF
  • Type I, type II, and occasionally type III
    Maltenfort, M
  • Science teachers’ beliefs and practices: Issues, implications and research agenda
    Mansour, N
  • Assigning medical students learning goals: Do they do it, and what happens when they don’t?
    Manzone, J; Regehr, G; Garbedian, S; Brydges, R
  • Exploring the interstitial space between the ideal and the practised: Humanism and the hidden curriculum of system reform
    Martimianakis, MA; Hafferty, FW
  • Theory-based change and change-based theory: Going deeper going broader
    McLaughlin, M; Mitra, D
  • Intervention descriptions in medical education: What can be improved? Academic medicine a systematic review and checklist
    Meinema, JG; Buwalda, N; van Etten-Jamaludin, F; Visser, MR; van Nynke, D
  • Does your implementation fit your theory of change?
    Montague, S
  • Evidence-informed practice in education: Meanings and applications
    Nelson, J; Campbell, C
  • Implementing competency-based medical education: What changes in curricular structure and processes are needed?
    Nousiainen, MT; Caverzagie, KJ; Ferguson, PC; Frank, JR
  • What questions guide investing in our faculty?
    O’Sullivan, PS
  • Useful to whom? Evaluation utilization theory and boundaries for program evaluation scope
    Onyura, B
  • Evidence for curricular and instructional design approaches in undergraduate medical education: An umbrella review
    Onyura, B; Baker, L; Cameron, B; Friesen, F; Leslie, K
  • Selecting for creativity and innovation potential: Implications for practice in healthcare education
    Patterson, F; Zibarras, LD
  • Embracing innovation in medical education
    Petty, E; Golden, RN
  • Development and validation of the guideline for reporting evidence-based practice educational interventions and teaching (GREET)
    Phillips, AC; Lewis, LK; McEvoy, MP
  • Introducing curriculum innovations in science: Identifying teachers’ transformations and the design of related teacher education
    Pintó, R
  • Betterment, undermining, support and distortion : A heuristic model for the analysis of pressure on evaluators
    Pleger, L; Sager, F
  • Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda
    Proctor, E; Silmere, H; Raghavan, R
  • Examining key concepts in research on teachers’ use of mathematics curricula
    Remillard, JT
  • An agenda for research on the sustainability of public health programs
    Scheirer, MA; Dearing, JW
  • Prose and cons about goal-free evalution
    Scriven, M
  • Faculty drivers and barriers: laying the groundwork for undergraduate STEM education reform in academic departments
    Shadle, SE; Marker, A; Earl, B
  • Planning far the sustainability of community-based health programs: Conceptual frameworks and future directions for research, practice and policy
    Shediac-Rizkallah, MC; Bone, LR
  • Curriculum strategy framework: Investigating patterns in teachers’ use of a reform-based elementary mathematics curriculum
    Sherin, MG; Drake, C
  • Acquiescence, accommodation, and resistance in learning to teach within a prescribed curriculum - Pr
    Smagorinsky, P; Lakly, A; Johnson, T
  • Till time (and poor planning) do us part: Programs as dynamic systems—Incorporating planning of sustainability into theories of change
    Sridharan, S; Nakaima, A
  • Repeal and replace? A note of caution for medical school curriculum reformers
    Stevens, CD
  • Rebuttal: will the Triple C curriculum produce better family physicians? Yes
    Tannenbaum, D
  • A core components framework for evaluating implementation of competency-based medical education programs
    Van Melle, E; Frank, JR; Holmboe, ES; Dagnone, D; Stockley, D; Sherbino, J
  • Is transferring an educational innovation actually a process of transformation
    Varpio, L; Bell, R; Hollingworth, G
  • An investigation of experienced secondary science teachers’ beliefs about inquiry: An examination of competing belief sets
    Wallace, CS; Kang, NH
  • Will the triple C curriculum produce better family physicians? No
    Whitehead, CR
  • Captive on a carousel: Discourses of “new” in medical education 1910–2010
    Whitehead, CR; Hodges, BD; Austin, Z
  • Self-regulating intellectual processes and outcomes: A social cognitive perspective
    Zimmerman, B; Schunk, DH; Dai, DY; Steinberg, RJ