Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Assessment of central venous catheterization in a simulated model using a motion-tracking device: an experimental validation study

Assessment of central venous catheterization in a simulated model using a motion-tracking device:... Background: Central venous catheterization (CVC) is a basic requirement for many medical specialties. Simulated training in CVC may allow the acquisition of this competency but few reports have established a valid methodology for learning and acquiring procedural skills for CVC. This study aims to validate the use of a tracking motion device, the imperial college surgical assessment device (ICSAD), by comparing it with validated global rating scales (GRS) to measure CVC performance in a simulated torso. Methods: Senior year medical students, first and last year residents (PGY1, LYR), and expert anesthesiologists per - formed a jugular CVC assessment in a simulated model (Laerdal IV Torso). A validated GRS for objective assessment of technical skills and motion analysis by ICSAD was used. Statistical analysis was performed through Mann–Whitney and Kruskal–Wallis tests for construct validity and Spearman correlation coefficients between the ICSAD and GRS scores for concurrent validity between both. Results: 32 subjects were recruited (10 medical students, 8 PGY1, 8 LYR and 8 experts). Total path length measured with ICSAD and GRS scores were significantly different between all groups, except for LYR compared to experts (p = 0.664 for GRS and p = 0.72 for ICSAD). Regarding jugular CVC procedural time, LYR and experts were faster than PGY1 and MS (p < 0.05). Spearman correlation coefficient was −0.684 (p < 0.001) between ICSAD and GRS scores. Conclusions: ICSAD is a valid tool for assessment of jugular CVC since it differentiates between expert and novice subjects, and correlates with a validated GRS for jugular CVC in a simulated torso. Keywords: Medical simulation, Central venous catheterization, Tracking motion device, Objective skills assessment, Medical training These adverse events are inversely related to practition - Background er’s clinical experience [5]. Central venous catheterization (CVC) is an essential Nowadays, simulated training for acquiring techni- competency required for many medical specialties [1, cal skills is becoming widespread for many medical spe- 2]. Annually, 5  million CVC are performed only in the United States [3], with serious and life-threatening com cialties [2, 6, 7], from simple procedures like a venous puncture to more complex surgical procedures like a plications occurring in up to 5–26  % [4] of the cases. laparoscopic jejuno-jejunostomy [8–10], shortening the learning curves of residents while doing so in a safe and *Correspondence: rmontana@med.puc.cl controlled environment [11]. Few studies have evalu- Anesthesiology Division, Clinic Hospital, School of Medicine, Pontificia ated the acquisition of CVC proficiency through simu - Universidad Católica de Chile, Marcoleta 367, Santiago, Chile lated models and their educational effectiveness [ 11–15]. Full list of author information is available at the end of the article © 2016 Varas et al. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/ publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Varas et al. Ann Surg Innov Res (2016) 10:2 Page 2 of 5 These studies have shown an increase in the rate of suc - cessful CVC [16] and a decrease of associated complica- tions after the simulated training programs [17, 18]. The Accreditation Council for Graduate Medical Edu - cation (ACGME) recommends the use of simulation and checklists as the “most desirable” evaluation methods for the assessment of competency in procedural skills [13], and they have been commonly used for the evaluation of CVC [4, 16, 19, 20]. However structured global rating scales as the objective structured assessment of technical skill (OSATS) [21, 22] have demonstrated better assess- Fig. 1 Simulated jugular central venous catheterization assessment ment and discrimination of different levels of skills than in an Adult IV simulated model, using a tracking motion device on checklists previously used [13]. their hands: the ICSAD The imperial college surgical assessment device (ICSAD) is a device that tracks hand-motion of the oper- ator during a procedure, using sensors placed on the back Table 1 Number of  prior CVC insertion by  each group of the trainee’s hands. Total path length of both hands is evaluated registered, providing an effective index of technical skill Students PGY1 LYR (n = 8) Experts during a procedure [7, 23]. The ICSAD has demonstrated (n = 10) (n = 8) (n = 8) construct validity in many surgical procedures [23, 24] and it has been used for objective assessment of profi - Median 0 (0–1) 4 (0–20) 50 (15–80) >100 (range) (200–400) ciency in anesthetic procedures such as labor epidural placement [25] and ultrasound-guided peripheral nerve CVC Central venous catheterization, PGY1 first year post graduate residents, LYR last year residents blockade [26]. To our knowledge, there are no previous reports using motion-tracking devices to assess profi - ciency in CVC. Prior to assessment The validation of a motion-tracking device may com - Before assessment, all groups were gathered at a 2 h mas- plement the use of global rating scales (OSATS) in assess- ter class where they were explained how to perform a ing better the differences between expert and novices jugular CVC in the simulated model. In addition, a DVD procedural skills. video was provided to each student with a step-by-step Therefore, the aim of this study is to establish the con - instructional guide emphasizing the key issues related to struct and concurrent validity of the tracking motion the procedure and most common mistakes. device (ICSAD) in assessing CVC in a simulated model. Jugular CVC assessment Methods After the introductory class, all groups were assessed per- The Institutional Review Board approved the study, forming one jugular CVC in an Adult Laerdal IV bench and written informed consent was obtained from all model (Laerdal IV Torso; Laerdal Medical Corp, Wap- participants. pingers Falls, NY) [13, 27]. Each task was video-recorded Four different groups from Pontificia Universidad and then blindly assessed by three independent expert Católica de Chile Medical School were studied between anesthesiologists using a validated OSATS global rating November–December 2013. All students and postgradu- scale (Modified from Ma et  al. [13]). Economy of move - ate residents that were available for the period of assess- ments was assessed using the ICSAD from the beginning ment were recruited. Based on previous studies by our of the technical procedure (total path length was meas- group a sample study of eight participants per group was ured in meters). Procedure time (in seconds) was also calculated. All groups underwent a simulated jugular recorded. Inter-rater reliability between OSATS evalua- central venous catheterization assessment in an Adult IV tors was calculated using Kappa coefficient (0–1) [28, 29]. simulated model, using tracking motion sensors attached on their hands (ICSAD) as shown in Fig. 1 [23, 24]. Statistical analysis Residents keep an active registration of the number of Data was analyzed with the Statistical Package for procedures they complete during the 3  year residency. the Social Sciences version 15.0 (SPSS, Chicago, IL, Prior jugular CVC experience of the different groups is USA) using non-parametric tests. Mann–Whitney and shown in Table 1. Varas et al. Ann Surg Innov Res (2016) 10:2 Page 3 of 5 Kruskal–Wallis tests were used for each variable and the one” approach [31]. This apprenticeship model requires results were exposed in median (range). that inexperienced residents perform the procedure on Spearman correlation coefficients between the ICSAD patients, in a clinical setting with few or none standard- scores and the validated OSATS global rating scale scores ized methodology to teach or evaluate the procedure were calculated to establish the concurrent validity of [31]. Therefore, this learning model does not ensure pro - ICSAD [13]. Following the Cohen guidelines, a positive ficiency in practical skills and jeopardizes patients’ safety or negative value between 0.5 and 1.0 indicates a large [32]. effect, 0.3–0.5 indicates a medium effect, and 0.3–0.1 Surgical specialties have vast experience in objective indicates a small effect [30]. P value was considered sta - assessment of technical skills for a procedure [7]. Global tistically significant when <0.05. rating scales, specific checklists and motion analysis like ICSAD have been used to evaluate many surgical pro- Results cedures, establishing a good correlation between scores A total of 32 subjects were recruited, divided in 10 medi- obtained and surgeon’s competency level [7, 23, 24]. In cal students (MS), 8 PGY1, 8 LYR and 8 expert anesthesi- the case of anesthesia, only few reports have used objec- ologist. Inter-rater reliability was established between the tive assessment of technical skills, demonstrating a good three OSATS evaluators, obtaining a Kappa coefficient of correlation between scores obtained with the assessment 0.76 (CI 0.58–0.92). tools and the expertise level [19, 20]. Tracking motion Results of total path length measured with ICSAD, devices like ICSAD have demonstrated construct validity OSATS global rating scores and procedural time are in discriminating the different levels of expertise in anes - shown in Table 2. Regarding ICSAD total path length, all thetic procedures, such as an epidural catheter insertion groups had significant differences between them, except [25] or an ultrasound-guided peripheral nerve blockade for LYR compared to experts (p = 0.664; Fig. 2a). [26]. In OSATS median scores, there were significant differ - This is the first validation study reporting the use ences between all groups, except for LYR compared to of ICSAD as an assessment tool for jugular CVC. The experts (p = 0.72; Fig. 2b). exercise of this motion device in the evaluation of jugu- Finally, concerning procedural time, there were no dif- lar CVC allows the obtainment of quantitative data that ferences between MS compared to PGY1 (p = 0.172) and complements global rating scales for differentiating LYR compared to experts (p  =  0.694). However, the last between novice and expert, thus, adding construct valid- two had significant less procedural time than PGY1 and ity to the simulated model. Both of these tools were more MS (p = 0.015). useful in discriminating the level of expertise when com- Spearman correlation coefficient between the total pared to procedural time as an assessment measurement path length measured with ICSAD and the validated (Table 2). OSATS global rating scale score demonstrated a strong In the case of concurrent validity of ICSAD for jugular correlation, with a Spearman correlation coefficient of CVC, a good correlation was achieved with the previous −0.684 (p < 0.001). validated OSATS global rating scale. The ICSAD is an objective numeric tool, non-dependent of the evaluating Discussion teacher which reduces assessment bias. Central venous catheterization is commonly learned No significant differences were found between expe - during residences trough the “see one, do one, teach rienced residents (LYR) and expert anesthesiologists in Table 2 JCVC assessment (bench model) A B C D AB BC BD CD Students (n = 10) PGY1 (n = 8) LYR (n = 8) Experts (n = 8) p value p value p value p value GRS (8–32) 11.5 (8–28) 18 (13–27) 27 (17–32) 29 (24–32) 0.029 0.014 0.004 0.664 TPL (m) 48.5 (44–89) 43 (33–56) 35 (28–42) 34 (28–44) 0.028 0.015 0.04 0.72 Procedural time (s) 344 (218–609) 243 (121–571) 133 (111–339) 122 (116–201) 0.172 0.053 0.015 0.694 Comparison between final year medical students, PGY1 and LYR anesthesiology residents, and expert Anesthesiologists GRS Global rating scores, TPL total path length, CVC central venous catheterization, PGY1 first year postgraduate residents, LYR last year residents AB A B p values obtained when comparing columns  and  with Mann–Whitney test BC B C p values obtained when comparing columns  and  with Mann–Whitney test BD B D p values obtained when comparing columns  and  with Mann–Whitney test CD C D p values obtained when comparing columns  and  with Mann–Whitney test Varas et al. Ann Surg Innov Res (2016) 10:2 Page 4 of 5 Fig. 2 Comparison between final year medical students, PGY1, LYR and expert in total path length ( TPL) (a) and global rating scale (GRS) scores (b) terms of total path length measured by ICSAD, global skills such a jugular CVC may improve the objectification rating scale or time of procedure. A possible explanation of competency acquisition in real patients. for this result is that most LYR residents have already flat - tened their learning curves and achieved the minimum Abbreviations proficiency needed to perform this procedure. They are CVC: central venous catheterization; GRS: global rating scales; ICSAD: impe- considered experienced non-experts and perform well rial college surgical assessment device; LYR: last year residents; MS: medical students; OSATS: objective structured assessment of technical skill; PGY1: on routine problems by unreflectively and automati - post-graduate year 1. cally applying the standard technique [33]. Hayter et  al. [25] had similar findings when they assessed the epidural Authors’ contributions JV, FL, RC, MC, RM helped design the study, conduct the study, analyze the catheter insertion in residents, and proposed to add vari- data, and write the manuscript. PA helped design the study, analyze the ables such as non-standard patient scenarios in order to data, and write the manuscript. NDLF helped design the study and analyze discriminate in a more subtle way the expertise level [25]. the data. RA helped analyze the data and technical support. LC, MPB helped conduct the study and analyze the data. MC helped design the study, conduct Our main study limitation is small sample groups, the study, analyze the data, and write the manuscript. JV, PA, FL, RC, MC has mainly due to local difficulties in gathering residents for seen the original study data, reviewed the analysis of the data, and approved an experimental protocol in our institution. However, the final manuscript. NDLF, LC, MPB has seen the original study data, reviewed and approved the final manuscript. RA has seen the original study data, there were statistically significant differences between revised critically the article and approved the final manuscript. RM has seen the assessed groups when comparing the tracking the original study data, reviewed the analysis of the data, approved the final motion measures; concluding that ICSAD may help dif- manuscript, and is the author responsible for archiving the study files. All authors read and approved the final manuscript. ferentiate between different skills level in CVC simulated assessment. Author details Finally, this is the first report establishing construct Experimental Surgery and Simulation Center, Department of Digestive Surgery, Clinic Hospital, School of Medicine, Pontificia Universidad Católica validity of the Laerdal IV Torso model. This training de Chile, Santiago, Chile. Anesthesiology Division, Clinic Hospital, School bench model, with ICSAD and GRS used as assessing of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 367, Santiago, tools, allows to discriminate between different levels of Chile. Department of Surgery, Faculty of Medicine, McGill University, Montreal, Canada. Arnold and Blema Steinberg Medical Simulation Centre, expertise. The technical skills gap observed in this simu - Faculty of Medicine, McGill University, Montreal, Canada. lated model between novices and experts provides learn- ing opportunities for trainees, setting the cutoff scores Competing interests This study was funded by the Anesthesiology Division, Faculty of Medicine, to be achieved. Further predictive validity studies are Pontificia Universidad Católica de Chile. RA is a consultant for Applied Medical. needed in order to determine whether the skills acquired The rest of the authors report no competing interests. through the simulated training may or not transfer to real Received: 23 October 2015 Accepted: 1 February 2016 life scenario with patients. In conclusion, ICSAD was correctly validated for assessing jugular CVC in a simulated model, as it dis- criminates between expert and novices and correlates with validated OSATS global rating scale. To have as References 1. Duffy FD, Holmboe ES. What procedures should internists do? Ann Intern many as possible instruments for evaluating procedural Med. 2007;146:392–3. Varas et al. Ann Surg Innov Res (2016) 10:2 Page 5 of 5 2. Ma IW, Sharma N, Brindle ME, Caird J, McLaughlin K. Measuring compe- 18. Barsuk JH, McGaghie WC, Cohen ER, O’Leary KJ, Wayne DB. Simulation- tence in central venous catheterization: a systematic-review. Springer- based mastery learning reduces complications during central venous plus. 2014;17(3):33. catheter insertion in a medical intensive care unit. Crit Care Med. 3. McGee DC, Gould MK. Preventing complications of central venous cath- 2009;37:2697–701. eterization. N Engl J Med. 2003;348:1123–33. 19. Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB. Mastery learn- 4. Dong Y, Suri HS, Cook DA, Kashani KB, Mullon JJ, Enders FT, et al. Simula- ing of temporary hemodialysis catheter insertion by nephrology fellows tion-based objective assessment discerns clinical proficiency in central using simulation technology and deliberate practice. Am J Kidney Dis. line placement: a construct validation. Chest. 2010;137:1050–6. 2009;1:70–6. 5. Sznajder JI, Zveibil FR, Bitterman H, Weiner P, Bursztein S. Central vein 20. Friedman Z, Katznelson R, Devito I, Siddiqui M, Chan V. Objective assess- catheterization. Failure and complication rates by three percutaneous ment of manual skills and proficiency in performing epidural anesthesia- approaches. Arch Intern Med. 1986;146:259–61. video-assisted validation. Reg Anesth Pain Med. 2006;31:304–10. 6. Aggarwal R, Mytton O, Derbrew M, Hananel D, Heydenburg M, Issenberg 21. Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, B, et al. Training and simulation for patient safety. Qual Saf Health Care. et al. Objective structured assessment of technical skill (OSATS) for surgi- 2010;19:34–43. cal residents. Br J Surg. 1997;84:273–8. 7. Reznick R, MacRae H. Teaching surgical skills—changes in the wind. N 22. Wilkinson J, Crossley J, Wragg A, Mills P, Cowan G, Wade W. Implementing Engl J Med. 2006;355:2664–9. workplace-based assessment across the medical specialties in the United 8. Varas J, Mejia R, Riquelme A, Maluenda F, Buckel E, Salinas J, et al. Signifi- Kingdom. Med Educ. 2008;42:364–73. cant transfer of surgical skills obtained with an advanced laparoscopic 23. Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic training program to a laparoscopic jejunojejunostomy in a live porcine motion tracking analysis to objectively measure open surgical skill in the model: feasibility of learning advanced laparoscopy in a general surgery laboratory-based model. J Am Coll Surg. 2001;193:479–85. residency. Surg Endosc. 2012;26:3486–94. 24. Aggarwal R, Dosis A, Bello F, Darzi A. Motion tracking systems for assess- 9. Boza C, Varas J, Buckel E, Achurra P, Devaud N, Lewis T, et al. A cadaveric ment of surgical skill. Surg Endosc. 2007;21:339. porcine model for assessment in laparoscopic bariatric surgery: a valida- 25. Hayter MA, Friedman Z, Bould MD, Hanlon JG, Katznelson R, Borges tion study. Obes Surg. 2013;23:589–93. B, et al. Validation of the Imperial College Surgical Assessment Device 10. Aggarwal R, Boza C, Hance J, Leong J, Lacy A, Darzi A. Skills acquisition (ICSAD) for labour epidural placement. Can J Anaesth. 2009;56:419–26. for laparoscopic gastric bypass in the training laboratory: an innovative 26. Chin KJ, Tse C, Chan V, Tan JS, Lupu CM, Hayter M. Hand motion analysis approach. Obes Surg. 2007;17:19–27. using the imperial college surgical assessment device: validation of a 11. Okuda Y, Bryson EO, DeMaria S Jr, Jacobson L, Quinones J, Shen B, et al. novel and objective performance measure in ultrasound-guided periph- The utility of simulation in medical education: what is the evidence? Mt eral nerve blockade. Reg Anesth Pain Med. 2011;36:213–9. Sinai J Med. 2009;76:330–43. 27. Millington SJ, Wong RY, Kassen BO, Roberts JM, Ma IW. Improving internal 12. Britt RC, Reed SF, Britt LD. Central line simulation: a new training algo- medicine residents’ performance, knowledge, and confidence in central rithm. Am Surg. 2007;73:680–3. venous catheterization using simulators. J Hosp Med. 2009;4:410–6. 13. Ma IW, Zalunardo N, Pachev G, Beran T, Brown M, Hatala R, et al. Compar- 28. Van Nortwick SS, Lendvay TS, Jensen AR, Wright AS, Horvath KD, Kim S. ing the use of global rating scale with checklists for the assessment of Methodologies for establishing validity in surgical simulation studies. central venous catheterization skills using simulation. Adv Health Sci Surgery. 2010;147:622–30. Educ Theory Pract. 2012;17:457–70. 29. Carlson J, Tomkowiak J, Knott P. Simulation-based examinations in physi- 14. Li T, Okuda Y. Impact of simulation training on central venous catheter cian assistant education: a comparison of two standard-setting methods. insertion: a review. JEDM. 2012;1:1–9. J Physician Assist Educ. 2010;21:7–14. 15. Marmol MT, Braga FT, Garbin LM, Moreli L, dos Santos CB, de Carvalho EC. 30. Cohen J. Statistical power analysis. Curr Dir Psychol Sci. 1992;1:98–101. Central catheter dressing in a simulator: the effects of tutor’s assistance or 31. Smith CC, Huang GC, Newman LR, Clardy PF, Feller-Kopman D, Cho M, self-learning tutorial. Rev Lat Am Enfermagem. 2012;20:1134–41. et al. Simulation training and its effect on long-term resident perfor - 16. Evans LV, Dodge KL, Shah T, Kaplan LJ, Siegel MD, Moore CL, et al. Simula- mance in central venous catheterization. Simul Healthc. 2010;5:146–51. tion training in central venous catheter insertion: improved performance 32. Volpp KG, Grande D. Residents’ suggestions for reducing errors in teach- in clinical practice. Acad Med. 2010;85:1462–9. ing hospitals. N Engl J Med. 2003;348:851–5. 17. Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of 33. Moulton CA, Regehr G, Mylopoulos M, MacRae HM. Slowing down simulation-based education to reduce catheter-related bloodstream when you should: a new model of expert judgment. Acad Med. infections. Arch Intern Med. 2009;169:1420–3. 2007;82:S109–16. Submit your next manuscript to BioMed Central and we will help you at every step: • We accept pre-submission inquiries • Our selector tool helps you to find the most relevant journal • We provide round the clock customer support • Convenient online submission • Thorough peer review • Inclusion in PubMed and all major indexing services • Maximum visibility for your research Submit your manuscript at www.biomedcentral.com/submit http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Annals of Surgical Innovation and Research Springer Journals

Assessment of central venous catheterization in a simulated model using a motion-tracking device: an experimental validation study

Loading next page...
 
/lp/springer-journals/assessment-of-central-venous-catheterization-in-a-simulated-model-7IrSUV3gya

References (36)

Publisher
Springer Journals
Copyright
Copyright © 2016 by Varas et al.
Subject
Medicine & Public Health; Surgery
ISSN
1750-1164
eISSN
1750-1164
DOI
10.1186/s13022-016-0025-6
pmid
26877764
Publisher site
See Article on Publisher Site

Abstract

Background: Central venous catheterization (CVC) is a basic requirement for many medical specialties. Simulated training in CVC may allow the acquisition of this competency but few reports have established a valid methodology for learning and acquiring procedural skills for CVC. This study aims to validate the use of a tracking motion device, the imperial college surgical assessment device (ICSAD), by comparing it with validated global rating scales (GRS) to measure CVC performance in a simulated torso. Methods: Senior year medical students, first and last year residents (PGY1, LYR), and expert anesthesiologists per - formed a jugular CVC assessment in a simulated model (Laerdal IV Torso). A validated GRS for objective assessment of technical skills and motion analysis by ICSAD was used. Statistical analysis was performed through Mann–Whitney and Kruskal–Wallis tests for construct validity and Spearman correlation coefficients between the ICSAD and GRS scores for concurrent validity between both. Results: 32 subjects were recruited (10 medical students, 8 PGY1, 8 LYR and 8 experts). Total path length measured with ICSAD and GRS scores were significantly different between all groups, except for LYR compared to experts (p = 0.664 for GRS and p = 0.72 for ICSAD). Regarding jugular CVC procedural time, LYR and experts were faster than PGY1 and MS (p < 0.05). Spearman correlation coefficient was −0.684 (p < 0.001) between ICSAD and GRS scores. Conclusions: ICSAD is a valid tool for assessment of jugular CVC since it differentiates between expert and novice subjects, and correlates with a validated GRS for jugular CVC in a simulated torso. Keywords: Medical simulation, Central venous catheterization, Tracking motion device, Objective skills assessment, Medical training These adverse events are inversely related to practition - Background er’s clinical experience [5]. Central venous catheterization (CVC) is an essential Nowadays, simulated training for acquiring techni- competency required for many medical specialties [1, cal skills is becoming widespread for many medical spe- 2]. Annually, 5  million CVC are performed only in the United States [3], with serious and life-threatening com cialties [2, 6, 7], from simple procedures like a venous puncture to more complex surgical procedures like a plications occurring in up to 5–26  % [4] of the cases. laparoscopic jejuno-jejunostomy [8–10], shortening the learning curves of residents while doing so in a safe and *Correspondence: rmontana@med.puc.cl controlled environment [11]. Few studies have evalu- Anesthesiology Division, Clinic Hospital, School of Medicine, Pontificia ated the acquisition of CVC proficiency through simu - Universidad Católica de Chile, Marcoleta 367, Santiago, Chile lated models and their educational effectiveness [ 11–15]. Full list of author information is available at the end of the article © 2016 Varas et al. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/ publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Varas et al. Ann Surg Innov Res (2016) 10:2 Page 2 of 5 These studies have shown an increase in the rate of suc - cessful CVC [16] and a decrease of associated complica- tions after the simulated training programs [17, 18]. The Accreditation Council for Graduate Medical Edu - cation (ACGME) recommends the use of simulation and checklists as the “most desirable” evaluation methods for the assessment of competency in procedural skills [13], and they have been commonly used for the evaluation of CVC [4, 16, 19, 20]. However structured global rating scales as the objective structured assessment of technical skill (OSATS) [21, 22] have demonstrated better assess- Fig. 1 Simulated jugular central venous catheterization assessment ment and discrimination of different levels of skills than in an Adult IV simulated model, using a tracking motion device on checklists previously used [13]. their hands: the ICSAD The imperial college surgical assessment device (ICSAD) is a device that tracks hand-motion of the oper- ator during a procedure, using sensors placed on the back Table 1 Number of  prior CVC insertion by  each group of the trainee’s hands. Total path length of both hands is evaluated registered, providing an effective index of technical skill Students PGY1 LYR (n = 8) Experts during a procedure [7, 23]. The ICSAD has demonstrated (n = 10) (n = 8) (n = 8) construct validity in many surgical procedures [23, 24] and it has been used for objective assessment of profi - Median 0 (0–1) 4 (0–20) 50 (15–80) >100 (range) (200–400) ciency in anesthetic procedures such as labor epidural placement [25] and ultrasound-guided peripheral nerve CVC Central venous catheterization, PGY1 first year post graduate residents, LYR last year residents blockade [26]. To our knowledge, there are no previous reports using motion-tracking devices to assess profi - ciency in CVC. Prior to assessment The validation of a motion-tracking device may com - Before assessment, all groups were gathered at a 2 h mas- plement the use of global rating scales (OSATS) in assess- ter class where they were explained how to perform a ing better the differences between expert and novices jugular CVC in the simulated model. In addition, a DVD procedural skills. video was provided to each student with a step-by-step Therefore, the aim of this study is to establish the con - instructional guide emphasizing the key issues related to struct and concurrent validity of the tracking motion the procedure and most common mistakes. device (ICSAD) in assessing CVC in a simulated model. Jugular CVC assessment Methods After the introductory class, all groups were assessed per- The Institutional Review Board approved the study, forming one jugular CVC in an Adult Laerdal IV bench and written informed consent was obtained from all model (Laerdal IV Torso; Laerdal Medical Corp, Wap- participants. pingers Falls, NY) [13, 27]. Each task was video-recorded Four different groups from Pontificia Universidad and then blindly assessed by three independent expert Católica de Chile Medical School were studied between anesthesiologists using a validated OSATS global rating November–December 2013. All students and postgradu- scale (Modified from Ma et  al. [13]). Economy of move - ate residents that were available for the period of assess- ments was assessed using the ICSAD from the beginning ment were recruited. Based on previous studies by our of the technical procedure (total path length was meas- group a sample study of eight participants per group was ured in meters). Procedure time (in seconds) was also calculated. All groups underwent a simulated jugular recorded. Inter-rater reliability between OSATS evalua- central venous catheterization assessment in an Adult IV tors was calculated using Kappa coefficient (0–1) [28, 29]. simulated model, using tracking motion sensors attached on their hands (ICSAD) as shown in Fig. 1 [23, 24]. Statistical analysis Residents keep an active registration of the number of Data was analyzed with the Statistical Package for procedures they complete during the 3  year residency. the Social Sciences version 15.0 (SPSS, Chicago, IL, Prior jugular CVC experience of the different groups is USA) using non-parametric tests. Mann–Whitney and shown in Table 1. Varas et al. Ann Surg Innov Res (2016) 10:2 Page 3 of 5 Kruskal–Wallis tests were used for each variable and the one” approach [31]. This apprenticeship model requires results were exposed in median (range). that inexperienced residents perform the procedure on Spearman correlation coefficients between the ICSAD patients, in a clinical setting with few or none standard- scores and the validated OSATS global rating scale scores ized methodology to teach or evaluate the procedure were calculated to establish the concurrent validity of [31]. Therefore, this learning model does not ensure pro - ICSAD [13]. Following the Cohen guidelines, a positive ficiency in practical skills and jeopardizes patients’ safety or negative value between 0.5 and 1.0 indicates a large [32]. effect, 0.3–0.5 indicates a medium effect, and 0.3–0.1 Surgical specialties have vast experience in objective indicates a small effect [30]. P value was considered sta - assessment of technical skills for a procedure [7]. Global tistically significant when <0.05. rating scales, specific checklists and motion analysis like ICSAD have been used to evaluate many surgical pro- Results cedures, establishing a good correlation between scores A total of 32 subjects were recruited, divided in 10 medi- obtained and surgeon’s competency level [7, 23, 24]. In cal students (MS), 8 PGY1, 8 LYR and 8 expert anesthesi- the case of anesthesia, only few reports have used objec- ologist. Inter-rater reliability was established between the tive assessment of technical skills, demonstrating a good three OSATS evaluators, obtaining a Kappa coefficient of correlation between scores obtained with the assessment 0.76 (CI 0.58–0.92). tools and the expertise level [19, 20]. Tracking motion Results of total path length measured with ICSAD, devices like ICSAD have demonstrated construct validity OSATS global rating scores and procedural time are in discriminating the different levels of expertise in anes - shown in Table 2. Regarding ICSAD total path length, all thetic procedures, such as an epidural catheter insertion groups had significant differences between them, except [25] or an ultrasound-guided peripheral nerve blockade for LYR compared to experts (p = 0.664; Fig. 2a). [26]. In OSATS median scores, there were significant differ - This is the first validation study reporting the use ences between all groups, except for LYR compared to of ICSAD as an assessment tool for jugular CVC. The experts (p = 0.72; Fig. 2b). exercise of this motion device in the evaluation of jugu- Finally, concerning procedural time, there were no dif- lar CVC allows the obtainment of quantitative data that ferences between MS compared to PGY1 (p = 0.172) and complements global rating scales for differentiating LYR compared to experts (p  =  0.694). However, the last between novice and expert, thus, adding construct valid- two had significant less procedural time than PGY1 and ity to the simulated model. Both of these tools were more MS (p = 0.015). useful in discriminating the level of expertise when com- Spearman correlation coefficient between the total pared to procedural time as an assessment measurement path length measured with ICSAD and the validated (Table 2). OSATS global rating scale score demonstrated a strong In the case of concurrent validity of ICSAD for jugular correlation, with a Spearman correlation coefficient of CVC, a good correlation was achieved with the previous −0.684 (p < 0.001). validated OSATS global rating scale. The ICSAD is an objective numeric tool, non-dependent of the evaluating Discussion teacher which reduces assessment bias. Central venous catheterization is commonly learned No significant differences were found between expe - during residences trough the “see one, do one, teach rienced residents (LYR) and expert anesthesiologists in Table 2 JCVC assessment (bench model) A B C D AB BC BD CD Students (n = 10) PGY1 (n = 8) LYR (n = 8) Experts (n = 8) p value p value p value p value GRS (8–32) 11.5 (8–28) 18 (13–27) 27 (17–32) 29 (24–32) 0.029 0.014 0.004 0.664 TPL (m) 48.5 (44–89) 43 (33–56) 35 (28–42) 34 (28–44) 0.028 0.015 0.04 0.72 Procedural time (s) 344 (218–609) 243 (121–571) 133 (111–339) 122 (116–201) 0.172 0.053 0.015 0.694 Comparison between final year medical students, PGY1 and LYR anesthesiology residents, and expert Anesthesiologists GRS Global rating scores, TPL total path length, CVC central venous catheterization, PGY1 first year postgraduate residents, LYR last year residents AB A B p values obtained when comparing columns  and  with Mann–Whitney test BC B C p values obtained when comparing columns  and  with Mann–Whitney test BD B D p values obtained when comparing columns  and  with Mann–Whitney test CD C D p values obtained when comparing columns  and  with Mann–Whitney test Varas et al. Ann Surg Innov Res (2016) 10:2 Page 4 of 5 Fig. 2 Comparison between final year medical students, PGY1, LYR and expert in total path length ( TPL) (a) and global rating scale (GRS) scores (b) terms of total path length measured by ICSAD, global skills such a jugular CVC may improve the objectification rating scale or time of procedure. A possible explanation of competency acquisition in real patients. for this result is that most LYR residents have already flat - tened their learning curves and achieved the minimum Abbreviations proficiency needed to perform this procedure. They are CVC: central venous catheterization; GRS: global rating scales; ICSAD: impe- considered experienced non-experts and perform well rial college surgical assessment device; LYR: last year residents; MS: medical students; OSATS: objective structured assessment of technical skill; PGY1: on routine problems by unreflectively and automati - post-graduate year 1. cally applying the standard technique [33]. Hayter et  al. [25] had similar findings when they assessed the epidural Authors’ contributions JV, FL, RC, MC, RM helped design the study, conduct the study, analyze the catheter insertion in residents, and proposed to add vari- data, and write the manuscript. PA helped design the study, analyze the ables such as non-standard patient scenarios in order to data, and write the manuscript. NDLF helped design the study and analyze discriminate in a more subtle way the expertise level [25]. the data. RA helped analyze the data and technical support. LC, MPB helped conduct the study and analyze the data. MC helped design the study, conduct Our main study limitation is small sample groups, the study, analyze the data, and write the manuscript. JV, PA, FL, RC, MC has mainly due to local difficulties in gathering residents for seen the original study data, reviewed the analysis of the data, and approved an experimental protocol in our institution. However, the final manuscript. NDLF, LC, MPB has seen the original study data, reviewed and approved the final manuscript. RA has seen the original study data, there were statistically significant differences between revised critically the article and approved the final manuscript. RM has seen the assessed groups when comparing the tracking the original study data, reviewed the analysis of the data, approved the final motion measures; concluding that ICSAD may help dif- manuscript, and is the author responsible for archiving the study files. All authors read and approved the final manuscript. ferentiate between different skills level in CVC simulated assessment. Author details Finally, this is the first report establishing construct Experimental Surgery and Simulation Center, Department of Digestive Surgery, Clinic Hospital, School of Medicine, Pontificia Universidad Católica validity of the Laerdal IV Torso model. This training de Chile, Santiago, Chile. Anesthesiology Division, Clinic Hospital, School bench model, with ICSAD and GRS used as assessing of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 367, Santiago, tools, allows to discriminate between different levels of Chile. Department of Surgery, Faculty of Medicine, McGill University, Montreal, Canada. Arnold and Blema Steinberg Medical Simulation Centre, expertise. The technical skills gap observed in this simu - Faculty of Medicine, McGill University, Montreal, Canada. lated model between novices and experts provides learn- ing opportunities for trainees, setting the cutoff scores Competing interests This study was funded by the Anesthesiology Division, Faculty of Medicine, to be achieved. Further predictive validity studies are Pontificia Universidad Católica de Chile. RA is a consultant for Applied Medical. needed in order to determine whether the skills acquired The rest of the authors report no competing interests. through the simulated training may or not transfer to real Received: 23 October 2015 Accepted: 1 February 2016 life scenario with patients. In conclusion, ICSAD was correctly validated for assessing jugular CVC in a simulated model, as it dis- criminates between expert and novices and correlates with validated OSATS global rating scale. To have as References 1. Duffy FD, Holmboe ES. What procedures should internists do? Ann Intern many as possible instruments for evaluating procedural Med. 2007;146:392–3. Varas et al. Ann Surg Innov Res (2016) 10:2 Page 5 of 5 2. Ma IW, Sharma N, Brindle ME, Caird J, McLaughlin K. Measuring compe- 18. Barsuk JH, McGaghie WC, Cohen ER, O’Leary KJ, Wayne DB. Simulation- tence in central venous catheterization: a systematic-review. Springer- based mastery learning reduces complications during central venous plus. 2014;17(3):33. catheter insertion in a medical intensive care unit. Crit Care Med. 3. McGee DC, Gould MK. Preventing complications of central venous cath- 2009;37:2697–701. eterization. N Engl J Med. 2003;348:1123–33. 19. Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB. Mastery learn- 4. Dong Y, Suri HS, Cook DA, Kashani KB, Mullon JJ, Enders FT, et al. Simula- ing of temporary hemodialysis catheter insertion by nephrology fellows tion-based objective assessment discerns clinical proficiency in central using simulation technology and deliberate practice. Am J Kidney Dis. line placement: a construct validation. Chest. 2010;137:1050–6. 2009;1:70–6. 5. Sznajder JI, Zveibil FR, Bitterman H, Weiner P, Bursztein S. Central vein 20. Friedman Z, Katznelson R, Devito I, Siddiqui M, Chan V. Objective assess- catheterization. Failure and complication rates by three percutaneous ment of manual skills and proficiency in performing epidural anesthesia- approaches. Arch Intern Med. 1986;146:259–61. video-assisted validation. Reg Anesth Pain Med. 2006;31:304–10. 6. Aggarwal R, Mytton O, Derbrew M, Hananel D, Heydenburg M, Issenberg 21. Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, B, et al. Training and simulation for patient safety. Qual Saf Health Care. et al. Objective structured assessment of technical skill (OSATS) for surgi- 2010;19:34–43. cal residents. Br J Surg. 1997;84:273–8. 7. Reznick R, MacRae H. Teaching surgical skills—changes in the wind. N 22. Wilkinson J, Crossley J, Wragg A, Mills P, Cowan G, Wade W. Implementing Engl J Med. 2006;355:2664–9. workplace-based assessment across the medical specialties in the United 8. Varas J, Mejia R, Riquelme A, Maluenda F, Buckel E, Salinas J, et al. Signifi- Kingdom. Med Educ. 2008;42:364–73. cant transfer of surgical skills obtained with an advanced laparoscopic 23. Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic training program to a laparoscopic jejunojejunostomy in a live porcine motion tracking analysis to objectively measure open surgical skill in the model: feasibility of learning advanced laparoscopy in a general surgery laboratory-based model. J Am Coll Surg. 2001;193:479–85. residency. Surg Endosc. 2012;26:3486–94. 24. Aggarwal R, Dosis A, Bello F, Darzi A. Motion tracking systems for assess- 9. Boza C, Varas J, Buckel E, Achurra P, Devaud N, Lewis T, et al. A cadaveric ment of surgical skill. Surg Endosc. 2007;21:339. porcine model for assessment in laparoscopic bariatric surgery: a valida- 25. Hayter MA, Friedman Z, Bould MD, Hanlon JG, Katznelson R, Borges tion study. Obes Surg. 2013;23:589–93. B, et al. Validation of the Imperial College Surgical Assessment Device 10. Aggarwal R, Boza C, Hance J, Leong J, Lacy A, Darzi A. Skills acquisition (ICSAD) for labour epidural placement. Can J Anaesth. 2009;56:419–26. for laparoscopic gastric bypass in the training laboratory: an innovative 26. Chin KJ, Tse C, Chan V, Tan JS, Lupu CM, Hayter M. Hand motion analysis approach. Obes Surg. 2007;17:19–27. using the imperial college surgical assessment device: validation of a 11. Okuda Y, Bryson EO, DeMaria S Jr, Jacobson L, Quinones J, Shen B, et al. novel and objective performance measure in ultrasound-guided periph- The utility of simulation in medical education: what is the evidence? Mt eral nerve blockade. Reg Anesth Pain Med. 2011;36:213–9. Sinai J Med. 2009;76:330–43. 27. Millington SJ, Wong RY, Kassen BO, Roberts JM, Ma IW. Improving internal 12. Britt RC, Reed SF, Britt LD. Central line simulation: a new training algo- medicine residents’ performance, knowledge, and confidence in central rithm. Am Surg. 2007;73:680–3. venous catheterization using simulators. J Hosp Med. 2009;4:410–6. 13. Ma IW, Zalunardo N, Pachev G, Beran T, Brown M, Hatala R, et al. Compar- 28. Van Nortwick SS, Lendvay TS, Jensen AR, Wright AS, Horvath KD, Kim S. ing the use of global rating scale with checklists for the assessment of Methodologies for establishing validity in surgical simulation studies. central venous catheterization skills using simulation. Adv Health Sci Surgery. 2010;147:622–30. Educ Theory Pract. 2012;17:457–70. 29. Carlson J, Tomkowiak J, Knott P. Simulation-based examinations in physi- 14. Li T, Okuda Y. Impact of simulation training on central venous catheter cian assistant education: a comparison of two standard-setting methods. insertion: a review. JEDM. 2012;1:1–9. J Physician Assist Educ. 2010;21:7–14. 15. Marmol MT, Braga FT, Garbin LM, Moreli L, dos Santos CB, de Carvalho EC. 30. Cohen J. Statistical power analysis. Curr Dir Psychol Sci. 1992;1:98–101. Central catheter dressing in a simulator: the effects of tutor’s assistance or 31. Smith CC, Huang GC, Newman LR, Clardy PF, Feller-Kopman D, Cho M, self-learning tutorial. Rev Lat Am Enfermagem. 2012;20:1134–41. et al. Simulation training and its effect on long-term resident perfor - 16. Evans LV, Dodge KL, Shah T, Kaplan LJ, Siegel MD, Moore CL, et al. Simula- mance in central venous catheterization. Simul Healthc. 2010;5:146–51. tion training in central venous catheter insertion: improved performance 32. Volpp KG, Grande D. Residents’ suggestions for reducing errors in teach- in clinical practice. Acad Med. 2010;85:1462–9. ing hospitals. N Engl J Med. 2003;348:851–5. 17. Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of 33. Moulton CA, Regehr G, Mylopoulos M, MacRae HM. Slowing down simulation-based education to reduce catheter-related bloodstream when you should: a new model of expert judgment. Acad Med. infections. Arch Intern Med. 2009;169:1420–3. 2007;82:S109–16. Submit your next manuscript to BioMed Central and we will help you at every step: • We accept pre-submission inquiries • Our selector tool helps you to find the most relevant journal • We provide round the clock customer support • Convenient online submission • Thorough peer review • Inclusion in PubMed and all major indexing services • Maximum visibility for your research Submit your manuscript at www.biomedcentral.com/submit

Journal

Annals of Surgical Innovation and ResearchSpringer Journals

Published: Feb 12, 2016

There are no references for this article.