Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Evaluation of Medication Administration Process: Tools and Techniques

Evaluation of Medication Administration Process: Tools and Techniques ABSTRACT Health care safety improvement focuses on redesign of high-risk health care processes such as the medication administration process (MAP) to enhance safety and reduce errors. A key limitation of MAP redesign efforts is the lack of validated tools by which to accurately record process observations. This paper discusses a pilot study conducted to develop and evaluate a handheld device application for recording observations of oral MAP functions and tasks. The aim of the study was to leverage existing technology to develop an observation recording device comparable to handwritten recordings commonly used to document workflow observations in health care settings. Device recordings were statistically compared to handwritten recordings of the same MAP functions and tasks. Findings provided preliminary evidence that handheld device recordings were consistent with handwritten recordings for documenting task duration but not task sequence. Interruptions were not examined due to insufficient data and additional research is needed. Keywords: medication administration process, evaluation, safety, process redesign, observational measurement 1. INTRODUCTION Medication errors in the United States constitute a serious health care safety risk that annually result in approximately 7,000 deaths, harm to approximately 1.5 million people, and billions of dollars in hospital treatment costs [1-4]. The majority of *Corresponding Author: Rita Snyder, PhD, RN, College of Nursing, University of South Carolina, 1601 Greene Street, Columbia, SC 29208. Phone: (803) 777-2868. E-mail: ritsny520@aol.com. Other authors: huynhn@cec.sc.edu; bocai@mailbox.sc.edu; atavakoli@mailbox.sc.edu medication errors are precipitated by fragmented and highly complex medication management processes [5]. This is especially true for the medication administration process (MAP). It is estimated that the increasingly complex MAP consumes approximately 40% of nursing time, making it a key process redesign target for medication safety improvement [6, 7]. Factors that contribute to MAP complexity include escalating patient acuity levels, numerous generic and trade medication names, expanded medication delivery routes, increased use of new and diverse medication safety technology, and an increased number of medication orders [6, 7]. MAP redesign efforts have increased in recent years in an effort to reduce risk and improve efficiency. However, these efforts are often limited due to MAP knowledge gaps, lack of standardization, and measurement limitations [8-10]. While there are diverse tools and approaches being used for MAP redesign, they typically have not been evaluated for their measurement quality. This limitation may increase the potential for unintended and unanticipated MAP redesign consequences. To address these challenges, the Institute of Medicine has encouraged development of reliable and valid health care process redesign approaches [11]. A pilot study was undertaken to address the question "Are handheld device recordings consistent with handwritten recordings by nurse subject matter experts (SMEs) for observations of the oral MAP?" The two-fold purpose of the study was to: 1) develop a handheld device application to record observations of oral MAP functions and tasks; and 2) statistically compare device application recordings with nurse SME handwritten recordings of a simulated oral MAP to determine the degree of consistency in documentation of MAP task sequence, duration and simultaneous interruptions. Efforts to decrease MAP redesign risk have most commonly targeted process interruptions. Findings from studies addressing the association between interruptions and errors have been mixed. This has been due, in part, to the use of inconsistent approaches to process observation. In an early study in an ambulatory care pharmacy, Flynn, et al. [12] found a significant relationship between interruptions, distractions and dispensing errors, the majority of which involved incorrect medication label information. In this study, interruptions were defined as the cessation of productive activity before the current prescription-filling task was completed for any externally imposed, observable, or audible reason. However, in a recent review of evidence underlying health care interruptions, Grundgeiger and Sanderson [13] concluded that there was limited evidence supporting a significant association between interruptions and medication errors. In a recent study by Westbrook, et al. [14] that examined the association of interruptions with increased risk and severity of medication administration errors, findings indicated that the frequency of interruptions was significantly associated with MAP failure. Specifically, interruptions occurred in 53% of all administrations with a 12% increase in process failures, and a 12.7% increase in medication administration errors [14]. MAP redesign efforts to improve efficiency have typically focused on time. In a study using an MAP redesign intervention, Franklin, et al. [15] applied a pre-post design to assess the impact of a closed loop automated medication management system on prescribing and administration errors and staff time. The study, which was conducted on a 28-bed general surgery ward of a London teaching hospital, found that the system reduced medication administration errors but increased time spent in medication-related tasks. Only one study was found that linked interruption and time variables. In a multi-site medication administration time study, Elganzouri, Standish, and Androwich [16] tested an observation method for assessing nursing effort and workflow in the MAP. A total of 151 nurses were observed during 980 unique medication observations in medical-surgical units at a rural hospital. Findings indicated that nurses averaged 15 minutes on each medication pass and were at risk of an interruption with every medication pass. Zheng, et al. [17] questioned the frequent discrepancy in time and motion studies between quantitative and qualitative findings related to the impact of health information technology on clinical workflow. They called for a new methodological perspective that incorporates analytical methods consisting of workflow fragmentation assessments, pattern recognition, and data visualization to uncover hidden regularities within the workflow [17]. Numerous methodological studies have examined the strengths and limitations of the observation method [18-19] and its application in health care process studies [16, 20-21]. The observation method requires events that are visible, predictable, and of limited duration [20]. The key advantages of observation are that it does not require subject knowledge, willingness to report, recall, communication ability, and selective subject perception [20]. Validity threats from observer inference can be controlled by consistent training and inter-rater reliability evaluations of experienced observers [20]. While video recording is the most accurate approach for observation data collection, it is typically not feasible in health care settings due to patient privacy and confidentiality restrictions. Consequently, handwritten recording of observations has been the traditional data collection approach in health care process studies [12, 14-15, 17, 22-24]. In recent years, the use of handheld devices as a replacement for handwritten recording of observations has become increasingly common. Recent efforts have improved handheld device efficiency for recording observations but there continues to be inadequate evidence supporting device recording consistency with traditional handwritten observation recording [14, 19, 23]. In an early effort to determine if handheld computers were the best choice for observation data collection, Weber and Roberts [25] concluded that they decreased data handling and associated errors. McBride et al. [26] found that observation recording with a handheld computer in an orthopedic outpatient clinic yielded comparable results to handwritten recording of observations. Guadagno et al. [27] found that Personal Digital Assistants (PDAs) were effective observation recording devices in a multi-site emergency department study on elder neglect. The primary caveats with use of handheld computers and PDAs for observation recording include user training and technical challenges such as data transfer between the handheld device and computer. However, the rapid growth in handheld technology has greatly decreased these challenges. In summary, there is substantial evidence to support observation as the gold standard for MAP evaluation. However, the evidence is less clear about the most effective approach for observation recording. Technological advances in handheld device design supports their use as an observation recording tool. However, there is limited evidence about the consistency between handheld device and handwritten observation recording of MAP task sequence, duration and interruptions. 2. METHODS The current pilot study was conducted in two phases. The first phase addressed the development of a handheld device application to record observations of oral MAP functions and tasks. In the second phase, device application recordings were statistically compared with handwritten recordings to assess the consistency of the two approaches for documenting oral MAP task sequence, duration and interruptions. 2.1. MAP Application Development The MAP application was designed as a web-based application using the HTML 5 specification. Desirable features of this specification included its support of ongoing application retrieval and revision by developers and users and its Offline Support for local device data storage. The Offline Support feature was considered especially important as it permits use of the handheld device MAP application in clinical locations that do not have Wi-Fi Internet connectivity. The jQTouch platform was used to develop the MAP application. This platform is an open source Javascript plug-in for the jQuery library that provides a native look and feel for mobile browsers such as the iPhone and iPod Touch [28]. A combination of client-storage and a SQLite database provided by the HTML 5 specification was used for the MAP application's core functionality to support recording of start and stop times for MAP tasks and interruptions in the current study. The efficiency of the MAP application was enhanced through automatic and instantaneous recording of task and interruption start and stop times that are accurate to the second. Application efficiency was also enhanced with e-mail capability to support data transmission to any e-mail account as a comma-separated values (CSV) file that is automatically created and delivered as an attachment. The CSV file can be opened with either a text application such as Notepad or a spreadsheet application such as Microsoft Excel. A rapid prototyping process was used to design the MAP application's graphical user interface (GUI). The initial GUI design was based on information from an expert registered nurse (RN) about oral MAP functions, tasks, and interruptions. The GUI was then modified and enhanced through trial use of the MAP application during oral MAP modeling episodes in a clinical simulation laboratory. In the trial stage, a number of modifications were made to correct usability issues as well as a number of enhancements to overcome shortcomings of the original design. For example, modifications included expanding the tasks list to include all possible choices, increasing the spacing between task names to avoid accidental selection of wrong task, and grouping of tasks in a way that allow for quick selection. The key enhancements involved reducing the number of screens the user needs to navigate to perform a task. For example, the selection of interruption type was modified such that the user is not forced to navigate to another screen, select the interruption type, click save, and navigate back. The final MAP application GUI is depicted in Figure 1. Figure 1. MAP Application GUI: (a) location selection; (b) task selection; (c) interruption type selection; (d) data management option selection. The procedure for using the MAP application is shown in the sequential screen shots in Figure 1. As indicated in Figure 1a, the user initially records the nurse's location. If there are multiple patient rooms, the user is provided with a dropdown menu from which to choose a room. When a room selection is made, the application records and stores it in the device database. Figure 1b shows the MAP application task screen. When the user selects a task, a start time is automatically recorded. The task stop time is automatically recorded when the task is selected a second time. The task changes color with each selection to indicate its status. Figure 1c displays the MAP application interruption screen. When the user selects the interruption key, a drop down menu displays choices for interruption type. The selection of the interruption type automatically initiates recording of the start time. The user selects the stop key to discontinue interruption recording. The MAP application stores all task and interruption start and stop times in the device database. In the reported study, interruptions were spontaneously introduced into the MAP simulation by the models and recorded successively. Recording more than one concurrent interruption at a time was not required. However, for future device use, simple programming modifications will be made to allow the user to concurrently select more than one interruption from a list and store them in the database. Figure 1d displays the MAP application data management options screen. Data can be exported via email or by copying raw data and pasting it into another application. The "Reset Database" selection is used to clear previously recorded data prior to each new observation. The final MAP application design supported the efficient capture of MAP task and interruption observations. Key features included a two-step activity recording process, i.e., select or scroll, capture of multiple concurrent tasks, and the use of standardized number sequencing for MAP functions and tasks. As shown in Table 1, key MAP functions and their associated tasks were sequentially numbered to better support data management activities. For example, "Prepare Medication" was function 1.0 with 12 corresponding tasks, e.g., "Greet Patient" task 1.1. This numbering convention was adopted to enhance observation recording efficiency and to ensure that all functions and their associated tasks adequately represented the nursing MAP. Table 1. Function and task naming convention. 2.2. MAP Application Evaluation Following development of the MAP device application, device recordings were statistically compared with handwritten recordings to assess their consistency when used to record oral MAP task sequence, duration and spontaneous interruptions. Oral medications were defined as those medications that can be swallowed by mouth, such as capsules, tablets, and liquids. 1) Setting Observational data were collected in a clinical simulation lab (CSL) hospital room over a period of 4 weeks. The CSL room contained a hospital bed with typical headboard monitoring and care equipment, bedside table, medication cart, bedside computer on wheels with a simulated electronic medication administration record, and patient-like manikin with armband and intravenous catheter. 2) Sample Three registered nurses (RNs) were recruited from a local medical center to model the basic oral MAP in the CSL. Inclusion criteria were experience with oral medication administration in the hospital setting and a minimum of one year of nursing experience. All RNs read and signed a study consent form approved by the University Institutional Review Board. Each RN was asked to perform the oral MAP in the same way that they would normally carry it out in their regular hospital clinical practice. They were also asked to spontaneously introduce interruptions as they would normally occur in their practice. Each RN modeled the oral MAP 3 times for a total of 9 combined observations per RN across the 3 data collection sessions, and for a grand total of 27 MAP observations for all RNs for all sessions. 3) Data Collection Procedures Two approaches were used to record MAP observations. First, the study engineer who developed the handheld device application used the device to record CSL MAP observations. All CSL data collection sessions were video recorded. Since the engineer had developed the handheld device application, no observer training sessions were required. During CSL recording sessions, the engineer observer/recorder stood in an inconspicuous location and did not interact with the RN model. The video recordings were made by a CSL technical support staff member and were captured by two ceilingmounted digital cameras at different angles. The high quality digital videotaping recordings were automatically downloaded and stored on a secure CSL server where they could be accessed via a password-protected website. Device MAP application and video observations were time-matched for analysis. De-identified and coded MAP application observation data for each oral MAP modeling episode were exported as Excel files and stored in the study analytic data repository. The corresponding video recording for each oral MAP modeling episode was stored on the CSL server in protected directories. Consistent with University Institutional Review Board requirements, all information gathered during the study was considered confidential and accessible only by the research team. For the second approach, handwritten recordings of video observations were made by three nurse SMEs of the same CSL MAP sessions observed by the study engineer observer/recorder. The engineer did not participate in this process. The handwritten recordings were made on a Video Observation Recording Form. Of the three nurse SMEs, only one had also been a participant in the CSL oral MAP modeling episodes. The recording form is comprised of three pages with directions and a recording example (page 1), a recording page (Page 2), and an "Other" page (Page 3). Page 2 contains the same listing of pre-defined MAP tasks and interruptions as those in the handheld device MAP application. Nurse SMEs were asked to sequentially number the tasks and interruptions observed on videos in the form's "Activity Sequence" column and record the corresponding start/stop times for each observed task and interruption in the start/stop time columns. Page 3 was used to identify observed tasks and interruptions not included in the page 2 pre-defined list. The page 3 format and recording procedure were identical to page 2 with the exception that observers were required to write-in new tasks and interruptions. Table 2 provides selected examples of the content for each page of the form. Table 2. Selected examples of MAP video observation recording form. Prior to observing videos, nurse SMEs participated in a group training session. The training session consisted of an introduction to the recording form and procedure, a group practice/discussion session with a sample video and an independent practice session with a second sample video. Baseline inter-rater reliability (IRR) for nurse SMEs was .85. Immediately following completion of the group training session, nurse SMEs were given additional recording forms and instructions for remote access to the password protected video recordings via the Internet. Independent nurse SME recording was completed in a convenient time and location. An additional IRR assessment was done after completion of 10 recording forms with .80 or higher agreement [29]. 3. RESULTS The consistency between nurse SME (Observer 1, Observer 2, and Observer 3) handwritten recordings and the engineer observer (Observer 4) device recordings was assessed through comparison of observer-selected tasks identified from a list of 70 tasks and task duration time measured in seconds. The degree of agreement between two observers for the selected tasks was examined via the Kappa statistic [30-31]. Table 3 shows the Kappa statistic for each pair of observers, and the averaged Kappa statistics for observers 1-3 and observers 1-4. Results indicated that Observer 1 and Observer 2 had a relatively low agreement with Observer 3. The overall Kappa among Observers 1, 2, and 3, which was determined by taking the average of the three Kappa values, indicated that nurse SME observers were comparable in their task assessments. In addition, the Kappa statistics between Observer 4 and the nurse SME observers indicated that Observers 1 and 2 had good agreement with Observer 4 while Observer 3 had fair agreement with Observer 4. Pearson correlation performed to assess the task duration IRR was greater than 0.87, indicating a strong agreement among the observers regarding task duration. Table 3. Kappa statistics and 95% confidence intervals (CI) for selected tasks. Table 4 shows the overall intra-class correlation (ICC) for each pair of observers and the averaged ICCs for observers 1-3 and observers 1-4 for task duration. Pairwise ICCs indicated good agreement among observers. The average of the ICC for observers 1-4 (0.88 with 95% CI 0.85-0.90) also indicated good agreement between Observer 4 and observers 1, 2, and 3. One-way analysis of variance (ANOVA) was performed to examine differences in task duration means between Observer 4 and the three nurse SME observers. The results (F=1.31, p-value=0.27) indicated no significant differences. Table 4. Intra-class correlation (ICC) and 95% confidence intervals for task durations. 4. DISCUSSION The poor agreement among the four observers on task sequence indicated that they varied on interpretation of events. For example, one observer recorded `knocking on the door' as the first task performed, whereas another observer choose to ignore this event and record `chat with patient' as the first task performed. Findings also indicated different observer interpretations about whether the nurse first `chatted with the patient' and then `identified the patient', or vice-versa. While there was little observer agreement about task sequence, there was strong agreement on the tasks performed, i.e., that the nurse did `chat with the patient' and `identify the patient.' Findings from this pilot study were encouraging and suggested that further development of MAP application of the handheld device for use in recording clinical process observations is warranted. However, due to the pilot nature of the study, there are some limitations and caveats that must be considered. First, the study had a very small sample of three RNs and 27 observations. Additional testing of the MAP device application with more RNs and observations will provide better insight into the measurement quality of the MAP application. Second, the simplistic CSL oral MAP modeling episodes were not characterized by the typical interruptions that occur in a busy hospital patient care unit. While the CSL setting provided the control required to psychometrically evaluate the measurement consistency between the two observation recording approaches, additional research is needed to determine the adequacy of the application to support observation recordings of MAP functions and tasks in a complex nursing care environment. In particular, additional device testing is needed in fastpaced, dynamic clinical environments where nurses often perform multiple and varied non-linear tasks with multiple interruptions. Third, the CSL oral MAP process was not representative of typical MAP complexity. Additional research is required to assess MAP application suitability with expanded types of medication administration such as injections and intravenous medications. Finally, there was limited observer agreement on MAP task sequence. This was an interesting finding that warrants additional research. The assumption is often made that medication administration tasks need to occur in a specific order to support safe medication administration. However, if evidence indicates that sequence standardization is atypical, this may provide insight into a key risk factor that needs to be addressed to improve MAP safety. 5. CONCLUSION Study findings provided preliminary support for the use of the handheld device to document MAP observations in health care settings. Evidence also supported the use of the device application by an observer/recorder (engineer) who was not a subject matter expert thus potentially expanding the pool of health care process observers as well as the quality of their observations. While task observations and their durations were found to be consistent between those recorded with the device MAP application and the handwritten approach, it is important to note that the study only considered selected tasks and their duration as there were insufficient interruptions to determine observation recording consistency. Additional research is required in this area. ACKNOWLEDGMENTS The authors thank the registered nurses who provided their expertise as CSL MAP models and subject matter experts. Their dedication to the project is appreciated. Special thanks to Mr. Robert Burris for his device MAP application programming and Ms. Gene Meding for her project direction. REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] Aspeden, P., et al. Preventing, Medication Errors: Quality Chasm Series. Institute of Medicine: Washington D.C. 2007. Burke, K. Executive summary: the state of the science on safe medication administration symposium. American Nursing, 2005, 105(3): Supplement, 4-9. Hughes R. and E. Ortiz. Medication errors: Why they happen, and how they can be prevented. American Nursing, 2005, 105(3): 14-24. Sullivan, C., et al. Medication reconciliation in the acute care setting: opportunity and challenge for nursing. Nursing Care Quality, 2005, 20(2): 95-98. Carayon, P., et al. Work system design for patient safety: The SEIPS model. Quality and Safety in Health Care, 2006, 15(Suppl1): i50-i58. McBride-Henry, K. and M. Foureur. Medication administration errors: Understanding the issues. Australian Advanced Nursing, 2006, 23(3): 33-42. Tang, F.I., et al. Nurses relate the contributing factors involved in medication errors. Clinical Nursing, 2007, 16: 447-457. Institute of Medicine. To err is human: Building a safer health system. National Academy Press: Washington D.C. 1999. [9] [10] [11] [12] [13] [14] [15] Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. National Academy Press: Washington D.C. 2001. Institute of Medicine. Keeping patients safe: Transforming the work environment of nurses. National Academies Press: Washington D.C. 2004. Reid, P.P., et al. Building a better delivery system: A new engineering/health care partnership. The National Academies Press: Washington D.C. 2005. Flynn, E.A., et al. Impact of interruptions and distractions on dispensing errors in an ambulatory care pharmacy. American Health-System Pharmacy, 1999, 56: 1319-1325. Grundgeiger, T. and P. Sanderson. Interruptions in healthcare: Theoretical views. Medical Informatics, 2009, 78: 293-307. Westbrook, J.I., et al. Association of interruptions with an increased risk and severity of medication administration errors. Archives of Internal Medicine, 2010, 170(8): 683-690. Franklin, B.D., et al. The impact of a closed-loop prescribing and administration system on prescribing errors, administration errors and staff time: A before-and-after study. Quality and Safety in Health Care, 2007, 16: 79-284. Elganzouri, E.S., C.A. Standish, and I. Androwich. Medication administration time study (MATS). Nursing Administration, 2009, 39(5): 204-210. Zheng, K., et al. Quantifying the impact of health IT implementations on clinical workflow: A new methodological perspective. the American Medical Informatics Association, 2010, 17: 454-461. Ampt, A., et al. A comparison of self-reported and observational work sampling techniques for measuring time in nursing tasks. Health Services Research & Policy, 2007, 12(1): 18-24. Westbrook, J.I. and A. Ampt. Design, application and testing of the Work Observation Method by Activity Timing (WOMBAT) to measure clinicians' patterns of work and communication. Medical Informatics, 2009, 78S: S25-S33. Barker, K.N., E. Flynn, and G.A. Pepper. Observation method of detecting medication errors. American Health-System Pharmacy, 2002, 59: 2314-2316. Dean, B. and N. Barber. Validity and reliability of observational methods for studying medication administration errors. American Health-System Pharmacy, 2001, 58: 54-59. Haw, C., J. Stubbs, and G. Dickens. An observational study of medication administration errors in old age psychiatric inpatients. International Journal for Quality in Health Care, 2007, 19(4): 210-216. Pizziferri, L., et al. Primary care physician time utilization before and after implementation of an electronic health record: A time-motion study. Biomedical Informatics, 2005, 38: 176-188. Whitten, J.L. and L.D. Bentley. Systems analysis and design methods, 7th edition. McGraw-Hill/Irwin: New York, NY. 2007. Weber B.A. and B.L. Roberts. Data collection using handheld computers. Nursing Research, 2000, 49(3): 173-175. Mcbride J., R. T. Anderson, and J. L. Bahnson. Using a hand-held computer to collect data in an orthopedic outpatient clinic: A randomized trial of two survey methods. Medical Care, 1999, 37(7): 647-651. Guadagno L., et al. Using PDAs for data collection. Applied Nursing Research, 2004, 17(4): 283-291. jQTouch. JQTouch jQuery plug-in for mobile web development. 2011 [cited 2011 August 4]; Available from: http://www.jqtouch.com. Schultz, K.S. and D.J. Whitney. Measurement theory in action. SAGE Publications: Thousand Oaks, CA. 2005. Altman, D. Practical statistics for medical research. Chapman and Hall/CRC: London. 1991. Landis, J. and G. Koch. The measurement of observer agreement for categorical data. Biometrics, 1977, 33: 159-174. [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] Rotating Machinery Engineering The Scientific World Journal Sensors Distributed Sensor Networks Control Science and Engineering Civil Engineering Advances in Submit your manuscripts at http://www.hindawi.com Robotics Electrical and Computer Engineering VLSI Design Advances in OptoElectronics Navigation and Observation Modelling & Simulation in Engineering Aerospace Engineering Chemical Engineering Antennas and Propagation Active and Passive Electronic Components Shock and Vibration Acoustics and Vibration Advances in http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Healthcare Engineering Hindawi Publishing Corporation

Evaluation of Medication Administration Process: Tools and Techniques

Download PDF
13 pages

Loading next page...
 
/lp/hindawi-publishing-corporation/evaluation-of-medication-administration-process-tools-and-techniques-fkB0re5X7o
Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2011 Hindawi Publishing Corporation.
ISSN
2040-2295
Publisher site
See Article on Publisher Site

Abstract

ABSTRACT Health care safety improvement focuses on redesign of high-risk health care processes such as the medication administration process (MAP) to enhance safety and reduce errors. A key limitation of MAP redesign efforts is the lack of validated tools by which to accurately record process observations. This paper discusses a pilot study conducted to develop and evaluate a handheld device application for recording observations of oral MAP functions and tasks. The aim of the study was to leverage existing technology to develop an observation recording device comparable to handwritten recordings commonly used to document workflow observations in health care settings. Device recordings were statistically compared to handwritten recordings of the same MAP functions and tasks. Findings provided preliminary evidence that handheld device recordings were consistent with handwritten recordings for documenting task duration but not task sequence. Interruptions were not examined due to insufficient data and additional research is needed. Keywords: medication administration process, evaluation, safety, process redesign, observational measurement 1. INTRODUCTION Medication errors in the United States constitute a serious health care safety risk that annually result in approximately 7,000 deaths, harm to approximately 1.5 million people, and billions of dollars in hospital treatment costs [1-4]. The majority of *Corresponding Author: Rita Snyder, PhD, RN, College of Nursing, University of South Carolina, 1601 Greene Street, Columbia, SC 29208. Phone: (803) 777-2868. E-mail: ritsny520@aol.com. Other authors: huynhn@cec.sc.edu; bocai@mailbox.sc.edu; atavakoli@mailbox.sc.edu medication errors are precipitated by fragmented and highly complex medication management processes [5]. This is especially true for the medication administration process (MAP). It is estimated that the increasingly complex MAP consumes approximately 40% of nursing time, making it a key process redesign target for medication safety improvement [6, 7]. Factors that contribute to MAP complexity include escalating patient acuity levels, numerous generic and trade medication names, expanded medication delivery routes, increased use of new and diverse medication safety technology, and an increased number of medication orders [6, 7]. MAP redesign efforts have increased in recent years in an effort to reduce risk and improve efficiency. However, these efforts are often limited due to MAP knowledge gaps, lack of standardization, and measurement limitations [8-10]. While there are diverse tools and approaches being used for MAP redesign, they typically have not been evaluated for their measurement quality. This limitation may increase the potential for unintended and unanticipated MAP redesign consequences. To address these challenges, the Institute of Medicine has encouraged development of reliable and valid health care process redesign approaches [11]. A pilot study was undertaken to address the question "Are handheld device recordings consistent with handwritten recordings by nurse subject matter experts (SMEs) for observations of the oral MAP?" The two-fold purpose of the study was to: 1) develop a handheld device application to record observations of oral MAP functions and tasks; and 2) statistically compare device application recordings with nurse SME handwritten recordings of a simulated oral MAP to determine the degree of consistency in documentation of MAP task sequence, duration and simultaneous interruptions. Efforts to decrease MAP redesign risk have most commonly targeted process interruptions. Findings from studies addressing the association between interruptions and errors have been mixed. This has been due, in part, to the use of inconsistent approaches to process observation. In an early study in an ambulatory care pharmacy, Flynn, et al. [12] found a significant relationship between interruptions, distractions and dispensing errors, the majority of which involved incorrect medication label information. In this study, interruptions were defined as the cessation of productive activity before the current prescription-filling task was completed for any externally imposed, observable, or audible reason. However, in a recent review of evidence underlying health care interruptions, Grundgeiger and Sanderson [13] concluded that there was limited evidence supporting a significant association between interruptions and medication errors. In a recent study by Westbrook, et al. [14] that examined the association of interruptions with increased risk and severity of medication administration errors, findings indicated that the frequency of interruptions was significantly associated with MAP failure. Specifically, interruptions occurred in 53% of all administrations with a 12% increase in process failures, and a 12.7% increase in medication administration errors [14]. MAP redesign efforts to improve efficiency have typically focused on time. In a study using an MAP redesign intervention, Franklin, et al. [15] applied a pre-post design to assess the impact of a closed loop automated medication management system on prescribing and administration errors and staff time. The study, which was conducted on a 28-bed general surgery ward of a London teaching hospital, found that the system reduced medication administration errors but increased time spent in medication-related tasks. Only one study was found that linked interruption and time variables. In a multi-site medication administration time study, Elganzouri, Standish, and Androwich [16] tested an observation method for assessing nursing effort and workflow in the MAP. A total of 151 nurses were observed during 980 unique medication observations in medical-surgical units at a rural hospital. Findings indicated that nurses averaged 15 minutes on each medication pass and were at risk of an interruption with every medication pass. Zheng, et al. [17] questioned the frequent discrepancy in time and motion studies between quantitative and qualitative findings related to the impact of health information technology on clinical workflow. They called for a new methodological perspective that incorporates analytical methods consisting of workflow fragmentation assessments, pattern recognition, and data visualization to uncover hidden regularities within the workflow [17]. Numerous methodological studies have examined the strengths and limitations of the observation method [18-19] and its application in health care process studies [16, 20-21]. The observation method requires events that are visible, predictable, and of limited duration [20]. The key advantages of observation are that it does not require subject knowledge, willingness to report, recall, communication ability, and selective subject perception [20]. Validity threats from observer inference can be controlled by consistent training and inter-rater reliability evaluations of experienced observers [20]. While video recording is the most accurate approach for observation data collection, it is typically not feasible in health care settings due to patient privacy and confidentiality restrictions. Consequently, handwritten recording of observations has been the traditional data collection approach in health care process studies [12, 14-15, 17, 22-24]. In recent years, the use of handheld devices as a replacement for handwritten recording of observations has become increasingly common. Recent efforts have improved handheld device efficiency for recording observations but there continues to be inadequate evidence supporting device recording consistency with traditional handwritten observation recording [14, 19, 23]. In an early effort to determine if handheld computers were the best choice for observation data collection, Weber and Roberts [25] concluded that they decreased data handling and associated errors. McBride et al. [26] found that observation recording with a handheld computer in an orthopedic outpatient clinic yielded comparable results to handwritten recording of observations. Guadagno et al. [27] found that Personal Digital Assistants (PDAs) were effective observation recording devices in a multi-site emergency department study on elder neglect. The primary caveats with use of handheld computers and PDAs for observation recording include user training and technical challenges such as data transfer between the handheld device and computer. However, the rapid growth in handheld technology has greatly decreased these challenges. In summary, there is substantial evidence to support observation as the gold standard for MAP evaluation. However, the evidence is less clear about the most effective approach for observation recording. Technological advances in handheld device design supports their use as an observation recording tool. However, there is limited evidence about the consistency between handheld device and handwritten observation recording of MAP task sequence, duration and interruptions. 2. METHODS The current pilot study was conducted in two phases. The first phase addressed the development of a handheld device application to record observations of oral MAP functions and tasks. In the second phase, device application recordings were statistically compared with handwritten recordings to assess the consistency of the two approaches for documenting oral MAP task sequence, duration and interruptions. 2.1. MAP Application Development The MAP application was designed as a web-based application using the HTML 5 specification. Desirable features of this specification included its support of ongoing application retrieval and revision by developers and users and its Offline Support for local device data storage. The Offline Support feature was considered especially important as it permits use of the handheld device MAP application in clinical locations that do not have Wi-Fi Internet connectivity. The jQTouch platform was used to develop the MAP application. This platform is an open source Javascript plug-in for the jQuery library that provides a native look and feel for mobile browsers such as the iPhone and iPod Touch [28]. A combination of client-storage and a SQLite database provided by the HTML 5 specification was used for the MAP application's core functionality to support recording of start and stop times for MAP tasks and interruptions in the current study. The efficiency of the MAP application was enhanced through automatic and instantaneous recording of task and interruption start and stop times that are accurate to the second. Application efficiency was also enhanced with e-mail capability to support data transmission to any e-mail account as a comma-separated values (CSV) file that is automatically created and delivered as an attachment. The CSV file can be opened with either a text application such as Notepad or a spreadsheet application such as Microsoft Excel. A rapid prototyping process was used to design the MAP application's graphical user interface (GUI). The initial GUI design was based on information from an expert registered nurse (RN) about oral MAP functions, tasks, and interruptions. The GUI was then modified and enhanced through trial use of the MAP application during oral MAP modeling episodes in a clinical simulation laboratory. In the trial stage, a number of modifications were made to correct usability issues as well as a number of enhancements to overcome shortcomings of the original design. For example, modifications included expanding the tasks list to include all possible choices, increasing the spacing between task names to avoid accidental selection of wrong task, and grouping of tasks in a way that allow for quick selection. The key enhancements involved reducing the number of screens the user needs to navigate to perform a task. For example, the selection of interruption type was modified such that the user is not forced to navigate to another screen, select the interruption type, click save, and navigate back. The final MAP application GUI is depicted in Figure 1. Figure 1. MAP Application GUI: (a) location selection; (b) task selection; (c) interruption type selection; (d) data management option selection. The procedure for using the MAP application is shown in the sequential screen shots in Figure 1. As indicated in Figure 1a, the user initially records the nurse's location. If there are multiple patient rooms, the user is provided with a dropdown menu from which to choose a room. When a room selection is made, the application records and stores it in the device database. Figure 1b shows the MAP application task screen. When the user selects a task, a start time is automatically recorded. The task stop time is automatically recorded when the task is selected a second time. The task changes color with each selection to indicate its status. Figure 1c displays the MAP application interruption screen. When the user selects the interruption key, a drop down menu displays choices for interruption type. The selection of the interruption type automatically initiates recording of the start time. The user selects the stop key to discontinue interruption recording. The MAP application stores all task and interruption start and stop times in the device database. In the reported study, interruptions were spontaneously introduced into the MAP simulation by the models and recorded successively. Recording more than one concurrent interruption at a time was not required. However, for future device use, simple programming modifications will be made to allow the user to concurrently select more than one interruption from a list and store them in the database. Figure 1d displays the MAP application data management options screen. Data can be exported via email or by copying raw data and pasting it into another application. The "Reset Database" selection is used to clear previously recorded data prior to each new observation. The final MAP application design supported the efficient capture of MAP task and interruption observations. Key features included a two-step activity recording process, i.e., select or scroll, capture of multiple concurrent tasks, and the use of standardized number sequencing for MAP functions and tasks. As shown in Table 1, key MAP functions and their associated tasks were sequentially numbered to better support data management activities. For example, "Prepare Medication" was function 1.0 with 12 corresponding tasks, e.g., "Greet Patient" task 1.1. This numbering convention was adopted to enhance observation recording efficiency and to ensure that all functions and their associated tasks adequately represented the nursing MAP. Table 1. Function and task naming convention. 2.2. MAP Application Evaluation Following development of the MAP device application, device recordings were statistically compared with handwritten recordings to assess their consistency when used to record oral MAP task sequence, duration and spontaneous interruptions. Oral medications were defined as those medications that can be swallowed by mouth, such as capsules, tablets, and liquids. 1) Setting Observational data were collected in a clinical simulation lab (CSL) hospital room over a period of 4 weeks. The CSL room contained a hospital bed with typical headboard monitoring and care equipment, bedside table, medication cart, bedside computer on wheels with a simulated electronic medication administration record, and patient-like manikin with armband and intravenous catheter. 2) Sample Three registered nurses (RNs) were recruited from a local medical center to model the basic oral MAP in the CSL. Inclusion criteria were experience with oral medication administration in the hospital setting and a minimum of one year of nursing experience. All RNs read and signed a study consent form approved by the University Institutional Review Board. Each RN was asked to perform the oral MAP in the same way that they would normally carry it out in their regular hospital clinical practice. They were also asked to spontaneously introduce interruptions as they would normally occur in their practice. Each RN modeled the oral MAP 3 times for a total of 9 combined observations per RN across the 3 data collection sessions, and for a grand total of 27 MAP observations for all RNs for all sessions. 3) Data Collection Procedures Two approaches were used to record MAP observations. First, the study engineer who developed the handheld device application used the device to record CSL MAP observations. All CSL data collection sessions were video recorded. Since the engineer had developed the handheld device application, no observer training sessions were required. During CSL recording sessions, the engineer observer/recorder stood in an inconspicuous location and did not interact with the RN model. The video recordings were made by a CSL technical support staff member and were captured by two ceilingmounted digital cameras at different angles. The high quality digital videotaping recordings were automatically downloaded and stored on a secure CSL server where they could be accessed via a password-protected website. Device MAP application and video observations were time-matched for analysis. De-identified and coded MAP application observation data for each oral MAP modeling episode were exported as Excel files and stored in the study analytic data repository. The corresponding video recording for each oral MAP modeling episode was stored on the CSL server in protected directories. Consistent with University Institutional Review Board requirements, all information gathered during the study was considered confidential and accessible only by the research team. For the second approach, handwritten recordings of video observations were made by three nurse SMEs of the same CSL MAP sessions observed by the study engineer observer/recorder. The engineer did not participate in this process. The handwritten recordings were made on a Video Observation Recording Form. Of the three nurse SMEs, only one had also been a participant in the CSL oral MAP modeling episodes. The recording form is comprised of three pages with directions and a recording example (page 1), a recording page (Page 2), and an "Other" page (Page 3). Page 2 contains the same listing of pre-defined MAP tasks and interruptions as those in the handheld device MAP application. Nurse SMEs were asked to sequentially number the tasks and interruptions observed on videos in the form's "Activity Sequence" column and record the corresponding start/stop times for each observed task and interruption in the start/stop time columns. Page 3 was used to identify observed tasks and interruptions not included in the page 2 pre-defined list. The page 3 format and recording procedure were identical to page 2 with the exception that observers were required to write-in new tasks and interruptions. Table 2 provides selected examples of the content for each page of the form. Table 2. Selected examples of MAP video observation recording form. Prior to observing videos, nurse SMEs participated in a group training session. The training session consisted of an introduction to the recording form and procedure, a group practice/discussion session with a sample video and an independent practice session with a second sample video. Baseline inter-rater reliability (IRR) for nurse SMEs was .85. Immediately following completion of the group training session, nurse SMEs were given additional recording forms and instructions for remote access to the password protected video recordings via the Internet. Independent nurse SME recording was completed in a convenient time and location. An additional IRR assessment was done after completion of 10 recording forms with .80 or higher agreement [29]. 3. RESULTS The consistency between nurse SME (Observer 1, Observer 2, and Observer 3) handwritten recordings and the engineer observer (Observer 4) device recordings was assessed through comparison of observer-selected tasks identified from a list of 70 tasks and task duration time measured in seconds. The degree of agreement between two observers for the selected tasks was examined via the Kappa statistic [30-31]. Table 3 shows the Kappa statistic for each pair of observers, and the averaged Kappa statistics for observers 1-3 and observers 1-4. Results indicated that Observer 1 and Observer 2 had a relatively low agreement with Observer 3. The overall Kappa among Observers 1, 2, and 3, which was determined by taking the average of the three Kappa values, indicated that nurse SME observers were comparable in their task assessments. In addition, the Kappa statistics between Observer 4 and the nurse SME observers indicated that Observers 1 and 2 had good agreement with Observer 4 while Observer 3 had fair agreement with Observer 4. Pearson correlation performed to assess the task duration IRR was greater than 0.87, indicating a strong agreement among the observers regarding task duration. Table 3. Kappa statistics and 95% confidence intervals (CI) for selected tasks. Table 4 shows the overall intra-class correlation (ICC) for each pair of observers and the averaged ICCs for observers 1-3 and observers 1-4 for task duration. Pairwise ICCs indicated good agreement among observers. The average of the ICC for observers 1-4 (0.88 with 95% CI 0.85-0.90) also indicated good agreement between Observer 4 and observers 1, 2, and 3. One-way analysis of variance (ANOVA) was performed to examine differences in task duration means between Observer 4 and the three nurse SME observers. The results (F=1.31, p-value=0.27) indicated no significant differences. Table 4. Intra-class correlation (ICC) and 95% confidence intervals for task durations. 4. DISCUSSION The poor agreement among the four observers on task sequence indicated that they varied on interpretation of events. For example, one observer recorded `knocking on the door' as the first task performed, whereas another observer choose to ignore this event and record `chat with patient' as the first task performed. Findings also indicated different observer interpretations about whether the nurse first `chatted with the patient' and then `identified the patient', or vice-versa. While there was little observer agreement about task sequence, there was strong agreement on the tasks performed, i.e., that the nurse did `chat with the patient' and `identify the patient.' Findings from this pilot study were encouraging and suggested that further development of MAP application of the handheld device for use in recording clinical process observations is warranted. However, due to the pilot nature of the study, there are some limitations and caveats that must be considered. First, the study had a very small sample of three RNs and 27 observations. Additional testing of the MAP device application with more RNs and observations will provide better insight into the measurement quality of the MAP application. Second, the simplistic CSL oral MAP modeling episodes were not characterized by the typical interruptions that occur in a busy hospital patient care unit. While the CSL setting provided the control required to psychometrically evaluate the measurement consistency between the two observation recording approaches, additional research is needed to determine the adequacy of the application to support observation recordings of MAP functions and tasks in a complex nursing care environment. In particular, additional device testing is needed in fastpaced, dynamic clinical environments where nurses often perform multiple and varied non-linear tasks with multiple interruptions. Third, the CSL oral MAP process was not representative of typical MAP complexity. Additional research is required to assess MAP application suitability with expanded types of medication administration such as injections and intravenous medications. Finally, there was limited observer agreement on MAP task sequence. This was an interesting finding that warrants additional research. The assumption is often made that medication administration tasks need to occur in a specific order to support safe medication administration. However, if evidence indicates that sequence standardization is atypical, this may provide insight into a key risk factor that needs to be addressed to improve MAP safety. 5. CONCLUSION Study findings provided preliminary support for the use of the handheld device to document MAP observations in health care settings. Evidence also supported the use of the device application by an observer/recorder (engineer) who was not a subject matter expert thus potentially expanding the pool of health care process observers as well as the quality of their observations. While task observations and their durations were found to be consistent between those recorded with the device MAP application and the handwritten approach, it is important to note that the study only considered selected tasks and their duration as there were insufficient interruptions to determine observation recording consistency. Additional research is required in this area. ACKNOWLEDGMENTS The authors thank the registered nurses who provided their expertise as CSL MAP models and subject matter experts. Their dedication to the project is appreciated. Special thanks to Mr. Robert Burris for his device MAP application programming and Ms. Gene Meding for her project direction. REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] Aspeden, P., et al. Preventing, Medication Errors: Quality Chasm Series. Institute of Medicine: Washington D.C. 2007. Burke, K. Executive summary: the state of the science on safe medication administration symposium. American Nursing, 2005, 105(3): Supplement, 4-9. Hughes R. and E. Ortiz. Medication errors: Why they happen, and how they can be prevented. American Nursing, 2005, 105(3): 14-24. Sullivan, C., et al. Medication reconciliation in the acute care setting: opportunity and challenge for nursing. Nursing Care Quality, 2005, 20(2): 95-98. Carayon, P., et al. Work system design for patient safety: The SEIPS model. Quality and Safety in Health Care, 2006, 15(Suppl1): i50-i58. McBride-Henry, K. and M. Foureur. Medication administration errors: Understanding the issues. Australian Advanced Nursing, 2006, 23(3): 33-42. Tang, F.I., et al. Nurses relate the contributing factors involved in medication errors. Clinical Nursing, 2007, 16: 447-457. Institute of Medicine. To err is human: Building a safer health system. National Academy Press: Washington D.C. 1999. [9] [10] [11] [12] [13] [14] [15] Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. National Academy Press: Washington D.C. 2001. Institute of Medicine. Keeping patients safe: Transforming the work environment of nurses. National Academies Press: Washington D.C. 2004. Reid, P.P., et al. Building a better delivery system: A new engineering/health care partnership. The National Academies Press: Washington D.C. 2005. Flynn, E.A., et al. Impact of interruptions and distractions on dispensing errors in an ambulatory care pharmacy. American Health-System Pharmacy, 1999, 56: 1319-1325. Grundgeiger, T. and P. Sanderson. Interruptions in healthcare: Theoretical views. Medical Informatics, 2009, 78: 293-307. Westbrook, J.I., et al. Association of interruptions with an increased risk and severity of medication administration errors. Archives of Internal Medicine, 2010, 170(8): 683-690. Franklin, B.D., et al. The impact of a closed-loop prescribing and administration system on prescribing errors, administration errors and staff time: A before-and-after study. Quality and Safety in Health Care, 2007, 16: 79-284. Elganzouri, E.S., C.A. Standish, and I. Androwich. Medication administration time study (MATS). Nursing Administration, 2009, 39(5): 204-210. Zheng, K., et al. Quantifying the impact of health IT implementations on clinical workflow: A new methodological perspective. the American Medical Informatics Association, 2010, 17: 454-461. Ampt, A., et al. A comparison of self-reported and observational work sampling techniques for measuring time in nursing tasks. Health Services Research & Policy, 2007, 12(1): 18-24. Westbrook, J.I. and A. Ampt. Design, application and testing of the Work Observation Method by Activity Timing (WOMBAT) to measure clinicians' patterns of work and communication. Medical Informatics, 2009, 78S: S25-S33. Barker, K.N., E. Flynn, and G.A. Pepper. Observation method of detecting medication errors. American Health-System Pharmacy, 2002, 59: 2314-2316. Dean, B. and N. Barber. Validity and reliability of observational methods for studying medication administration errors. American Health-System Pharmacy, 2001, 58: 54-59. Haw, C., J. Stubbs, and G. Dickens. An observational study of medication administration errors in old age psychiatric inpatients. International Journal for Quality in Health Care, 2007, 19(4): 210-216. Pizziferri, L., et al. Primary care physician time utilization before and after implementation of an electronic health record: A time-motion study. Biomedical Informatics, 2005, 38: 176-188. Whitten, J.L. and L.D. Bentley. Systems analysis and design methods, 7th edition. McGraw-Hill/Irwin: New York, NY. 2007. Weber B.A. and B.L. Roberts. Data collection using handheld computers. Nursing Research, 2000, 49(3): 173-175. Mcbride J., R. T. Anderson, and J. L. Bahnson. Using a hand-held computer to collect data in an orthopedic outpatient clinic: A randomized trial of two survey methods. Medical Care, 1999, 37(7): 647-651. Guadagno L., et al. Using PDAs for data collection. Applied Nursing Research, 2004, 17(4): 283-291. jQTouch. JQTouch jQuery plug-in for mobile web development. 2011 [cited 2011 August 4]; Available from: http://www.jqtouch.com. Schultz, K.S. and D.J. Whitney. Measurement theory in action. SAGE Publications: Thousand Oaks, CA. 2005. Altman, D. Practical statistics for medical research. Chapman and Hall/CRC: London. 1991. Landis, J. and G. Koch. The measurement of observer agreement for categorical data. Biometrics, 1977, 33: 159-174. [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] Rotating Machinery Engineering The Scientific World Journal Sensors Distributed Sensor Networks Control Science and Engineering Civil Engineering Advances in Submit your manuscripts at http://www.hindawi.com Robotics Electrical and Computer Engineering VLSI Design Advances in OptoElectronics Navigation and Observation Modelling & Simulation in Engineering Aerospace Engineering Chemical Engineering Antennas and Propagation Active and Passive Electronic Components Shock and Vibration Acoustics and Vibration Advances in

Journal

Journal of Healthcare Engineering – Hindawi Publishing Corporation

Published: Mar 19, 2016

There are no references for this article.