Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Evaluating incident learning systems and safety culture in two radiation oncology departments

Evaluating incident learning systems and safety culture in two radiation oncology departments IntroductionApproximately 50% of Australian cancer patients receive radiation therapy.1 The pathway from decision‐to‐treat to completion of a treatment course is highly complex, increasingly so due to continuously developing advanced technologies and techniques. There are many inter‐linking clinical and technical process steps for creating individualised treatment plans and delivering treatment over multiple fractions. Radiation oncology involves a multidisciplinary team (MDT) of radiation oncologists (ROs), radiation oncology medical physicists (ROMPs) and radiation therapists (RTs), including support from oncology nurses and allied health professionals. The MDT work together for high‐quality treatment and patient care, while also minimising risk. However, each activity/step, or interface between steps or information transfer point, has potential for error. Therefore, radiation oncology is subject to detailed quality management and control. Nevertheless, errors may occur.Errors and near misses are termed ‘incidents’, that is unintentional events or unwanted changes from the normal intended process, which potentially can cause an adverse event.2 Further categorisation designates actual incidents or near‐miss incidents.3 Actual incidents in radiation oncology are primarily errors where dose delivered to a patient deviates from the prescribed dose or plan, with or without a clinically measurable effect. A near‐miss incident is caught before any incorrect dose is delivered. Near misses can be identified during the pre‐treatment preparation/planning phases or at quality assurance (QA) checkpoints and rectified before a patient treatment begins. They can also be identified while the patient is on the treatment couch, immediately before each treatment delivery, by final QA checks or image guidance procedures. The proportion of treatments where an actual incident has occurred is considered relatively low within radiation oncology. For incidents where deviations are large enough to trigger mandatory reporting into national reporting systems, rates have been estimated at 0.2% per course and for those having clinically significant consequences, estimates are one or two orders of magnitude lower.4–6 Nevertheless, given the potential consequences of actual incidents, radiation oncology facilities deploy comprehensive QA programs and Incident Learning Systems (ILSs), alongside promoting a positive safety culture (SC).The Australian and New Zealand bi‐national radiation oncology practice standards (ROPS) recommend robust QA and incident management as requirements to mitigate risk.7 Internationally, radiation oncology departments have reported various local ILSs to support this.5,8–12 For robust incident evaluation, an ILS should include incident reporting, investigations (e.g. root cause analysis), data tracking, visualisation and practical feedback.13 It should also guide quality improvement (QI) and QA practices to guard against similar errors recurring. ILSs that analyse reported data to identify QI areas provide a more proactive and effective response to incident management than simple reactive changes to individual isolated reported incidents.8 Departmental ILSs that meet the reporting categories and needs of radiation oncology can impact on reducing error rates and provide appropriate data analysis and practical QI guidance.6 ILSs specifically designed for radiation oncology, coupled with continuous assessment and improvement, have reduced occurrence of significant incidents, provided QI insight and facilitated proactive approaches to quality and safety management.9,10,14ILS success relies on the department’s SC. Many studies have shown negativity and frustration from frontline staff regarding incident reporting and learning, more than from management staff.6,13,15,16 Negative responses have focussed more on unsatisfactory approaches to investigations, corrective actions and learning from incident reports rather than completing report forms.16 Staff show further frustration when ILSs are challenging or time‐consuming to use.13 Departmental SC can affect attitudes to an ILS and its utilisation and feedback to staff.As part of ongoing quality assessment and improvement, staff in the closely linked radiation oncology departments of two local health districts (LHDs) were surveyed, to determine current perceptions and understanding of the ILS in place and of departmental SC.At the survey time, the departments had multiple reporting systems, at three levels:Inhouse/departmentalILS including: paper‐based reporting forms for actual and near‐miss incidents and also departmental non‐radiation incidents; RO morbidity and mortality QA review meetings; RT senior staff meetings discussing all incident reports; and ROMP error discussion meetings.OrganisationalAn LHD‐level electronic platform, the Incident Information Management System (IIMS), for all actual incidents and near‐miss reports of any type.Mandatory bodiesHigher mandatory reporting at specified dose deviation levels; to the hospital/department Radiation Safety Officer (RSO) for dose deviations over 5%, and/or the NSW Environment Protection Authority (EPA), for greater than 10%, as per ROPS recommendations, state wide incident management policies and EPA legislation.7,17Incident classification followed the ROPS recommendations.7 The departments had similar processes and protocols around incident reporting, monitoring, learning and QI. Each professional discipline held monthly quality/safety meetings separately, with cross‐MDT discussion and collaboration for learning being mainly at inter‐group senior (management) levels.The aims of the survey were to identify staff understanding and use of the ILSs, any barriers to reporting and any needs for process change or departmental learning, as well as perceptions of SC. The findings prompted a QI project in one of the LHDs to evaluate and improve the ILS.MethodSurvey of ILS and SCCurrent understanding of ILSs and attitudes towards SC were evaluated via an anonymous online survey, distributed to Radiation Oncology professionals (all ROs, RTs and ROMPs) within radiation oncology departments in the two LHDs, Western Sydney (WSLHD) and Nepean Blue Mountains (NBMLHD). The survey took less than 15 min to complete. The project received ethics approval from each LHD's human research and ethics committees.Survey ‘gatekeepers’ were RTs who distributed the survey invitation to all RTs and ROMPs at WSLHD in October 2018 and all NBMLHD staff in December 2019. Distribution to WSLHD ROs was via RO administrative staff in February 2019. The survey was open for 2 weeks; gatekeepers sent reminder emails with current response rates on days 7 and 12. Participants were informed that completing the survey indicated consent to participate in the study.The survey content was based on radiation oncology SC surveys from the literature.6,11,13 Eleven staff across the MDT initially piloted the survey, then it was sent out on a larger scale. The survey captured occupation, years worked and role level; the last two were largely removed from analysis to ensure anonymity.The REDCap electronic data, capture tools hosted by The University of Sydney,18,19 were used to collect anonymous responses and manage the study data. Responses were exported to IBM SPSS20 for quantitative analysis, with results compared and summarised using descriptive statistics. Open‐ended questions were evaluated to derive any key themes.ResultsCharacteristics of respondentsInvitations were sent to 150 Radiation Oncology professional staff across WSLHD and NBMLHD; 95 (63%) responded, with similar overall response rates for each LHD (65% and 57% respectively). For the professional cohorts, response rates were 71%, 67% and 34% for RTs, ROMPs and ROs respectively.One respondent was excluded due to partial survey completion. The distribution of the 94 participants was 73% RTs, 15% ROMPs and 12% ROs, while staffing distribution at that time was 65%, 14%, and 21% respectively.Survey results between the two LHDs were not significantly different, utilising a Z test for two populations (P < 0.05), so were combined for analysis to further protect anonymity.Knowledge of incident reporting systemsRespondents showed various levels of ILS understanding and utilisation across the MDT. Table 1 shows the differences and similarities between the professional groups. Overall, 97% of respondents were aware of incident reporting. Those not familiar were all in their first employment year as RT Interns or RO Registrars.1TableUnderstanding of incident reporting and learning system.Survey questionsResponseRTn = 69 (71%)ROMPn = 14 (67%)ROn = 11 (34%)Totaln = 94 (63%)n%n%n%n%How many reporting systems in use are you aware of in your department?1913%536%218%1617%23754%536%436%4649%31826%214%19%2122%423%17%00%33%Don’t Know34%17%436%89%Please state the name of the reporting system/s you know of in use in the departmentNo response34%00%19%44%Don’t know23%214%218%66%In‐house/Departmental‐level only (e.g.: in‐house systems/meetings)710%00%00%77%Organisational level only (IIMS)2029%536%436%2931%Mandatory bodies only (RSO/EPA)00%00%00%00%Departmental and organisational level2841%643%19%3537%Organisational level and mandatory reporting23%00%218%44%Departmental, organisational and mandatory reporting dose deviation level (e.g.: RSO/EPA)710%17%19%910%Do you feel you can correctly Identify Actual Incidents vs Near Miss Incidents classification?Yes3957%1071%545%5457%No34%00%327%66%Some of the time2739%429%327%3436%Do you feel you can correctly identify radiation incident sub classification type in relation to the nine recommended categories within the radiation oncology practice standards?Yes4768%1071%545%6266%No46%214%545%1112%Some of the time1826%214%19%2122%Are you aware of a formal investigation system of actual or near miss radiation incidents in your department such as root cause analysis?Yes5377%964%218%6468%No1623%536%982%3032%EPA, environment protection authority; IIMS, incident information management system; RO, radiation oncologist; ROMP, radiation oncology medical physicist; RSO, radiation safety officer; RT, radiation therapist.The LHDs utilised three levels of reporting, as above: inhouse, organisational and mandatory. The largest respondent group (49%) acknowledged two systems in use. However, the professional cohorts showed differences in identifying systems. RTs (41%) and ROMPs (43%) largest responses identified dual systems at departmental and organisation levels. The RO cohort primarily stated organisational only (36%) or organisation and mandatory reporting (18%). Only 10% of respondents acknowledged all three.All cohorts indicated high confidence levels in categorising errors once identified (Table 1), with ROMPs having the strongest confidence (71%) for both category and sub‐categorisation.Utilisation and barrier perception of the current systemOverall, 51% of respondents had reported an incident to one or more systems in the six months before the survey (Table 2). Of the 46 staff who had not reported, 74% stated they had not observed or identified an error. The rest had identified an error but not reported, mostly because they had escalated to more senior staff who investigated and reported, or another team member had completed the incident report. Two respondents were aware of some near‐miss incidents that were not reported, and only one respondent indicated they chose not to report.2TableUtilisation and barriers to reporting.Survey questionsResponseRTn = 69ROMPn = 14ROn = 11Totaln = 94 (63%)n%n%n%n%Have you submitted an actual or near miss incident report in the last 6 months?Yes4464%321%19%4851%No2536%1179%1091%4649%What is the reason you did not submit a report in the last 6 months?Asked only ifprevious response was noChoose not to report00%17%00%12%Did not notice, observe or discover any incident or near‐miss event in the past 6 months2232%643%655%3474%Informed team leader who investigated and submitted the report23%17%218%511%Other open answer provided11%321%218%613%Number of barriers reported11928%536%436%2830%21319%214%218%1718%3812%17%436%1314%411%00%00%11%No perceived barriers2841%643%19%3537%What do you find is the biggest obstacle to you reporting actual or near miss radiation incidents in your department?(Multiple response allowed)Takes too long2333%17%545%2931%System is hard to access1319%17%545%1920%Don’t know how to use/or understand the system1217%214%764%2122%Don’t see the benefit of reporting46%17%00%55%Fear of negative action towards self or others1420%321%00%1718%I do not think there are any obstacles to reporting in my department2841%643%19%3537%Other open answer provided69%429%218%1213%RO, radiation oncologist; ROMP, radiation oncology medical physicist; RT, Radiation therapist.Overall, 37% of respondents perceived no barriers within the current system. However, 59 staff did perceive one or more barriers (Table 2). The most significant stated barrier was the time it takes (31%), then lack of knowledge/understanding of the system and its use (22%) and that it was hard to access (20%). Potential improvements to safety culture were indicated by the 18% of respondents that stated a barrier was related to fear of adverse action, and the 5% that did not see a benefit to reporting. These results show the most significant barriers were related to the system rather than departmental SC.Preferences for feedback and learningRespondents ranked their preference for learning and feedback (Table 3). The preferred method overall was an all‐staff MDT meeting with either mandatory or open attendance. Next was for selected staff to attend incident meetings, that is representatives from different work areas/groups who report back to others; 54% acknowledged this as the primary method currently utilised across the departments. RT and RO cohorts had similar ranking preferences for learning options. ROMPs showed some differences, such as a higher preference for newsletters to disseminate relevant information and lower for in‐service training.3TablePreference for feedback and learning.Preference rankingsRTROMPROTotalAll staff attendance at MDT incident reporting meeting. Either mandatory meeting or open attendance1st1st1st1stSelected staff attendance at incident reporting meeting (e.g.: Team leaders, or safety/quality team)2nd3rd2nd2ndAttending in‐service training3rd6th4th3rdNewsletter or email notification4th2nd3rd4thWord of mouth5th4th5th5thNone at all6th5th6th6thRO, radiation oncologist; ROMP, radiation oncology medical physicist; RT, Radiation therapist.Safety culture and learning capacityThe majority of staff (66%) feel encouraged to report, with 60% feeling comfortable reporting (Table 4). Most respondents (69%), but least strong for the ROMP group, thought their department practised a no‐blame culture (Table 4). Sixteen respondents stated their department did not practice a no‐blame culture, eight having either personally received, or witnessed others receive, adverse action, with two others declining to answer. Regarding assigning cause and blame between staff and processes, 37% of respondents gave this as 50%/50% staff/process, followed by 29% at 25%/75%. The majority of respondents (71%) gave positive responses to departmental learning capacity from reported incidents, with the RO cohort having the strongest perception.4TableSafety culture and learning.Survey questionsResponseRTn = 69ROMPn = 14ROn = 11Totaln = 94 (63%)n%n%n%n%Please rate how encouraged you feel to report actual or near miss radiation incidents within your radiation oncology department?Very discouraged to report11%00%00%11%Discouraged to report57%214%00%77%Neutral1420%750%327%2426%Encouraged to report2638%321%655%3537%Very encouraged to report2333%214%218%2729%Do you feel comfortable reporting actual or near miss radiation incidents within your radiation oncology department?Very uncomfortable reporting710%00%00%77%Uncomfortable reporting69%214%19%910%Neutral1319%643%327%2223%Comfortable reporting1928%321%655%2830%Very comfortable reporting2435%321%19%2830%Do you feel that your radiation oncology department practices a culture of no‐blame when errors are reported?Yes5275%643%764%6569%No913%643%19%1617%Other open answer provided812%214%327%1314%Have you ever personally received or witnessed other staff members receiving negative action towards them due to a reported radiation incident or near miss?Yes1420%429%19%1920%No5072%964%1091%6973%Do not wish to answer46%00%00%44%Other open answer provided00%17%00%11%After an actual or near miss radiation incident report is submitted, in your opinion, the cause/blame is mostly assigned to:0% on staff member, 100% on current process69%17%00%77%25% on staff member, 75% on current process1826%214%764%2729%50% on staff member, 50% on current process2841%536%218%3537%75% on staff member, 25% on current process1319%214%00%1516%100% on staff member, 0% on current process11%00%00%11%Other open answer provided34%429%218%910%How well do you believe your radiation oncology department is willing and able to learn from previous radiation actual and near miss incidents? Such as making positive process changes and or implementing appropriate training and education when necessaryUnable to learn from previous incidents00%17%00%11%Minimal ability to learn46%429%00%89%Neutral1623%214%00%1819%Some demonstrated ability to learn2942%429%545%3840%Demonstrated ability to learn from previous incidents2029%321%655%2931%RO, radiation oncologist; ROMP, radiation oncology medical physicist; RT, Radiation therapist.Qualitative resultsFour areas of free‐text answers were possible for questions related to (1) barriers to reporting, (2) no‐blame culture, (3) blame association to staff versus system/process and (4) open comment around the survey.The thematic analysis highlighted four key themes: (1) SC issues, (2) blame is situational dependant, (3) QI and learning weaknesses and (4) current department‐level and hospital‐level reporting system deficits.Respondents mentioned SC fourteen times. Overall, these supported a departmental no‐blame culture, but some responses, mainly from RTs at frontline and management levels, indicated some staff blame others and gossip. Blame assignment to staff versus process was identified as situational influenced, for example actual incident versus near miss, or perceived laziness of staff member/s involved.Nine open responses indicated inadequacies in the current reporting system, with 11 perceiving weakness in QI and learning. The main concerns were that the more general organisational reporting systems do not fit radiation oncology needs, and reports disappear into the system with no feedback or learning. Others indicated the analysis and learning do not focus on the actual error‐causing problems and perceptions that education is often not prioritised, with minimal preventive measures to mitigate future risk.Quality improvement projectThe survey findings were used to inform a QI project actioned in one LHD to evaluate and improve their current departmental (level i) ILS. An MDT QI project team was established to design, create and develop an electronic reporting system to suit departmental needs. This was guided by recommendations from literature, ROPS, barriers and other factors identified in the survey.Application of survey findings to improve local ILSElectronic report developmentA new customised electronic departmental‐level reporting system was locally developed on the Varian AriaTM oncology information system (Varian Medical Systems, Palo Alto, CA, USA) to improve use and support enhanced analysis and department learning and feedback. Integration into AriaTM was designed to reduce the following barriers: the time it takes to report, lack of understanding of system and use, and easier system access. It also enabled increased communication to appropriate staff and report extraction into Microsoft ExcelTM and Microsoft Power BITM (business intelligence/data analytics software) to increase data tracking and visualisation.Report analysis changesA dedicated MDT incident triage team was established and trained to support management. This provided a coordinated, centralised, structured and rapid approach to analysis and recommendations from reports.Feedback and learningRelevant meeting structures and attendance were changed, with MDT representatives in all meetings to ensure shared learning and discussion across professions. This increased communication with and between staff. Reports were easily available to all, since they were in a readily accessible electronic system. Quick feedback loops were introduced to staff involved in observed errors or barrier detection point fails and for urgent process changes. During COVID‐19 restrictions, a newsletter was used to provide regular feedback when meetings were not feasible.Focussed education on how and what to reportA three‐month pilot phase tested the new system within WSLHD's smaller campus to ensure that access and use were straightforward. The system was implemented across the whole LHD. Mandatory training for all staff included how to use the new system and what to report on. A protocol non‐compliance category was introduced to report near‐miss errors which had evaded one or more barriers before being found within QA pathways before the first treatment fraction. This was to strengthen knowledge of any systematic QA weaknesses.DiscussionThis work identified perceptions of SC and ILSs and barriers to reporting incidents in two Australian LHDs’ radiation oncology departments. The departments surveyed participated in joint tumour stream QA meetings and had similar in‐house ILSs. Most responding ROs worked across both LHD's, whereas only a few ROMPs and no RT staff did. The results show varied knowledge and understanding of the complete incident reporting systems, structure and associated learning.By profession, the survey response rate was highest for RTs, followed by ROMPs, in both LHDs. Although errors and incidents can occur at any point in the patient pathway, detection is highest at treatment delivery and QA checkpoints, performed most frequently by RTs, followed by ROMPs.21 Therefore, their greater use of reporting may influence higher response rates. RO staff primarily received reports either classified as actual incidents, with incorrect radiation delivery, or when high‐risk near‐miss incidents have been reported and discussed. Overall, RT and ROMP response rates were high compared to other literature‐reported surveys of medical professionals, for example Cook et al.20 presented median response rates of 59% to postal surveys of healthcare professionals.22 Cunningham et al.21 found a lower response rate (35%) for physicians.In the current work, professional cohorts showed differences in using and understanding the various ILSs (Table 1). RTs and ROMPs predominantly identified departmental and organisational systems, with ROs primarily identifying organisational and mandatory reporting. This reflected different groups main uses of systems, with RTs and ROMPs being predominantly involved in in‐house and organisational reporting and in‐house QA meetings for feedback and learning. ROs primarily used the organisational reporting systems and, when necessary, provided information for reports to mandatory bodies. In‐house system reporting capacity extends more widely and is more specific to radiation oncology needs than the organisational system. However, each serves different but overlapping purposes with other capabilities. Hence, the departments used both in parallel.Differences in each cohort's use of reporting systems and feedback and learning loops may have influenced their different responses. ROs and ROMPs groups more frequently acknowledged mandatory reporting. These groups are heavily involved in investigations and decisions, alongside the RT management teams. When dose deviations requiring such reporting occur, ROMPs quantify dose deviations and prescribing ROs determine whether there are any clinical consequences to the patient and they also further provide decision‐making around any changes to the patient plan or symptom management and to facilitate open disclosure to the patient. Fewer of the RT cohort mentioned mandatory reporting, which occurs after the initial report and involves investigations primarily completed by ROMPs, RT management and RO collaboration.All professional groups had limited awareness of the overall ILS processes, systems and intended purposes, indicating potential learning opportunities. Each cohort only showed a strong understanding of the system/s most utilised by their group. The differences indicated potential for improved learning across departments to give a more interdisciplinary collaborative approach to the overall ILS.Some findings indicated potential areas for departmental education and learning around description and categorisation of errors. Increased reporting accuracy should increase data reliability and more robust trend analysis from reported incidents.23The question ‘state the name of the reporting system/s you know of in use in the department?’ had few respondents (14%) stating any mandatory bodies. However, mandatory bodies were then mentioned in open answers by more staff, indicating staff have potentially interpreted the initial question as ‘within the department’, that is internal systems and not external reporting. This might have been better worded as ‘ … that are used by the department’.Reporting to any ILS was predominantly by the RT staff followed by ROMPs, supporting the literature that frontline staff are more likely to detect incidents or errors at QA checkpoints.21,23,24 It is positive that staff are willing to report when they are aware of an error, with only one respondent stating they chose not to and two indicating that sometimes near misses were not reported.Overall, 63% of respondents perceived one or more barriers to reporting within the current systems. The leading stated reason was the time involved, then lack of knowledge/understanding of the system or its use, or access difficulty. Obstacles related to negative SC, such as fear of negative actions or not seeing the benefit of reporting, had lower responses. It was promising that barriers relate to the structure, format and use of the current ILSs rather than SC awareness in staff or the perception of how the department treats safety issues. Ford et al.3 noted that electronic ILSs, customised to radiation oncology, reduce reporting barriers. ILS success is related to appropriate resources and utilisation, partnered with staff understanding and confidence that the SC is just and equitable.25Respondents' perceptions of SC and learning were stronger towards positive SC than negative, for example two‐thirds felt encouraged to report and comfortable reporting. A no‐blame culture was perceived by most respondents, with 73% not having witnessed or received adverse action. Two‐thirds of respondents perceive cause and blame after an error to be attributed 25–50% to staff and 50–75% to processes. This further supports a no‐blame culture. Our findings are similar to those of Bolderston et al.6 Overall, the responses support positive SC and the ability to learn. However, as some respondents perceived blame culture or negative SC, there is still room for improvement.The thematic review of the free‐text answers provided insight into some of the issues indicated in the quantitative results. There was a strong focus on developing changes to the ILS regarding feedback and learning pathways. Changes included developing an in‐house electronic reporting system that fit the scope of reporting needed within radiation oncology and provided fast consistent data analysis. This was developed and implemented in the ensuing QI project.The newly developed electronic reporting system is integrated onto the AriaTM platform, with rapid collaborative MDT triaging to manage reports in real time. This enables more effective education and reminders to staff when needed, ensuring continuous feedback between monthly meetings. Meetings are now MDT‐based rather than professionally separated and are open for anyone to attend. Hence feedback and learning pathways have a more collaborative approach with increased knowledge of all systems in each group. The meeting purpose is to review themes, discuss learning opportunities from reports and potential areas for improvement if process areas were regularly failing or of high risk and provide recommendations for working parties to consider. The overall goal was to improve feedback loops to the whole MDT, to help promote continued and increasing SC and learning and assist in QI decision‐making.Furthermore, now that all data capture is electronic, it is more comprehensive and consistent, and visualisation is improved. This helps to provide information for department educational needs to increase learning or identify when QI projects or QA checkpoint modifications are needed.In developing the new in‐house system, the need to reduce the three most significant perceived barriers to ILSs use was considered, the lack of knowledge/understanding of available systems, that access was difficult, and the system reporting time required. A clear structure was created to identify which system to report to, in‐house only or in‐house and hospital‐wide. The in‐house system was designed to be easy to access, quick to fill in and to readily provide all relevant patient data to reduce the number of data fields a reporter needs to complete compared to the previous paper‐based reporting forms. The incident triage team review in‐house system reports within 24 h of creation. When a report (near‐miss or actual incident) requires reporting to hospital systems, they coordinate this with all relevant information, including ROMP dosimetry reports, to assist the initial reporter. This has reduced the time involved and increased staff understanding of how to use and report into the hospital‐wide system. The in‐house system has streamlined the ROMP portion of dose difference evaluation and reporting and provided clearer and faster pathways if an incident is RSO or EPA reportable.Thus, the survey helped to understand the departmental SC and how staff perceive and use the current ILSs, which helped guide decisions around design and changes to the ILS and the change implementation process. It has also provided support for changes to learning and educational needs required across the MDT to ensure a comprehensive, collaborative and open approach to ILSs, reduce barriers, increase reporting and increase the positive SC.The new system has been in use across the whole WSLHD for seven months and ten months at the pilot campus. There are plans to survey again later to evaluate accumulated experience and the need for any further QI.ConclusionA survey of perceptions of SC and understanding of ILSs established a baseline understanding within two LHDs. In one LHD, the results led to the development of a QI project that significantly improved the ILS. Major changes were implemented to aspects of reporting and to the feedback and learning portions of the ILS, as the survey had highlighted barriers to reporting and areas to improve feedback and learning across the department. The study findings provide a reference for future evaluations of ILS and SC that may identify continued improvements as the impact of the changes continues to be assessed, including further regular surveys, review of data accuracy in reports and trend analysis of incidents.AcknowledgementsThe authors acknowledge Shamira Cross, Alexander Grant and Nicole Burns for assistance provided; Jillian Harris, Kevin Van Tilburg and A/Professor Verity Ahern for support of the project; and the staff within the Radiation Oncology Departments at WSLHD and NBMLHD for participating in this study.Conflict of InterestThere are no conflicts of interest.Funding InformationNo funding supported this project.Ethics StatementEthics approval WSLHD from QA/QI Committee on 15 February 2019. Approval: 1812‐02. Ethics approval NBMLHD from Appolo Committee on 15 November 2019. Approval: 19‐42(A).ReferencesRANZCR. Radiation Oncology Targeting Cancer: What is Radiation Therapy: The Faculty of Radiation Oncology. The Royal Australian and New Zealand College of Radiologists, 2017. Available from: https://www.targetingcancer.com.au/what‐is‐radiation‐therapy/D'Souza N, Holden L, Robson S, et al. Modern palliative radiation treatment: Do complexity and workload contribute to medical errors? Int J Radiat Oncol Biol Phys 2012; 84: e43–e8.Ford EC, Evans SB. Incident learning in radiation oncology: A review. Med Phys 2018; 45: e100–e19.Barton M, Shafiq J. Radiotherapy Risk Profile: Technical Manual. World Health Organization (WHO), Geneva, 2008.Ford E, de Los F, Santos L, Pawlicki T, Sutlief S, Dunscombe P. Consensus recommendations for incident learning database structures in radiation oncology. Med Phys 2012; 39: 7272–90.Bolderston A, Di Prospero L, French J, Church J, Adams R. A culture of safety? An international comparison of radiation therapists' error reporting. J Med Imaging Radiat Sci 2015; 46: 16–22.RANZCR. Radiation Oncology Practice Standards Sydney: The Faculty of Radiation Oncology (FRO), The Royal Australian and New Zealand College of Radiologist (RANZCR), Australian Society of Medical Imaging and Radiation Therapy (ASMIRT), The Australasian College of Physical Scientists and Engineers in Medicine (ASPSEM); 2018. Available from: https://www.ranzcr.com/whats‐on/news‐media/240‐new‐revised‐radiation‐oncology‐practice‐standardsClark BG, Brown RJ, Ploquin JL, Kind AL, Grimard L. The management of radiation treatment error through incident learning. Radiother Oncol 2010; 95: 344–9.Nyflot MJ, Zeng J, Kusano AS, et al. Metrics of success: Measuring impact of a departmental near‐miss incident learning system. Pract Rad Oncol 2015; 5: e409–e16.Gabriel PE, Volz E, Bergendahl HW, et al. Incident learning in pursuit of high reliability: Implementing a comprehensive, low‐threshold reporting program in a large, multisite radiation oncology department. Jt Comm J Qual Patient Saf 2015; 41(4): AP1.Milosevic M, Angers C, Liszewski B, et al. The Canadian national system for incident reporting in radiation treatment (NSIR‐RT) taxonomy. Pract Rad Oncol 2016; 6: 334–41.Arnold A, Delaney GP, Cassapi L, Barton M. The use of categorized time‐trend reporting of radiation oncology incidents: A proactive analytical approach to improving quality and safety over time. Int J Radiat Oncol Biol Phys 2010; 78: 1548–54.Montgomery L, Fava P, Freeman CR, et al. Development and implementation of a radiation therapy incident learning system compatible with local workflow and a national taxonomy. J Appl Clin Med Phys 2018; 19: 259–70.Romanofski D. The evolution of incident learning. Radiol Technol 2016; 88: 90–4.Shojania KG. The frustrating case of incident‐reporting systems. Qual Saf Health Care 2008; 17: 400–2.Cooke DL, Dunscombe PB, Lee RC. Using a survey of incident reporting and learning practices to improve organisational learning at a cancer care centre. Qual Saf Health Care 2007; 16: 342–8.NSW. Incident Management: Policy Directive. NSW Government, Sydney, 2020.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. A metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42: 377–81.Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019; 95: 103208.Corp I. IBM SPSS Statistics for Windows, Version 27.0. IBM Corp, Armonk, NY, 2019.Cunningham J, Coffey M, Knöös T, Holmberg O. Radiation oncology safety information system (ROSIS)–Profiles of participants and the first 1074 incident reports. Radiother Oncol 2010; 97: 601–7.Cook JV, Dickinson HO, Eccles MP. Response rates in postal surveys of healthcare professionals between 1996 and 2005: An observational study. BMC Health Serv Res 2009; 9: 1–8.Greenham S, Manley S, Turnbull K, et al. Application of an incident taxonomy for radiation therapy: Analysis of five years of data from three integrated cancer centres. Rep Pract Oncol Radiother 2018; 23: 220–7.Smith S, Wallis A, King O, et al. Quality management in radiation therapy: A 15 year review of incident reporting in two integrated cancer centres. Tech Innov Patient Support Radiat Oncol 2020; 14: 15–20.Liszewski B, Angers C, Kildea J. Mitigating the barriers to a culture of quality and safety in radiation oncology. Clin Oncol 2017; 29: 676–9. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Medical Radiation Sciences Wiley

Evaluating incident learning systems and safety culture in two radiation oncology departments

Loading next page...
 
/lp/wiley/evaluating-incident-learning-systems-and-safety-culture-in-two-O0Hf5LAZob

References (50)

Publisher
Wiley
Copyright
Copyright © 2022 Journal of Medical Radiation Sciences published by John Wiley & Sons Australia, Ltd on behalf of Australian Society of Medical Imaging and Radiation Therapy and New Zealand Institute of Medical Radiation Technology
ISSN
2051-3895
eISSN
2051-3909
DOI
10.1002/jmrs.563
Publisher site
See Article on Publisher Site

Abstract

IntroductionApproximately 50% of Australian cancer patients receive radiation therapy.1 The pathway from decision‐to‐treat to completion of a treatment course is highly complex, increasingly so due to continuously developing advanced technologies and techniques. There are many inter‐linking clinical and technical process steps for creating individualised treatment plans and delivering treatment over multiple fractions. Radiation oncology involves a multidisciplinary team (MDT) of radiation oncologists (ROs), radiation oncology medical physicists (ROMPs) and radiation therapists (RTs), including support from oncology nurses and allied health professionals. The MDT work together for high‐quality treatment and patient care, while also minimising risk. However, each activity/step, or interface between steps or information transfer point, has potential for error. Therefore, radiation oncology is subject to detailed quality management and control. Nevertheless, errors may occur.Errors and near misses are termed ‘incidents’, that is unintentional events or unwanted changes from the normal intended process, which potentially can cause an adverse event.2 Further categorisation designates actual incidents or near‐miss incidents.3 Actual incidents in radiation oncology are primarily errors where dose delivered to a patient deviates from the prescribed dose or plan, with or without a clinically measurable effect. A near‐miss incident is caught before any incorrect dose is delivered. Near misses can be identified during the pre‐treatment preparation/planning phases or at quality assurance (QA) checkpoints and rectified before a patient treatment begins. They can also be identified while the patient is on the treatment couch, immediately before each treatment delivery, by final QA checks or image guidance procedures. The proportion of treatments where an actual incident has occurred is considered relatively low within radiation oncology. For incidents where deviations are large enough to trigger mandatory reporting into national reporting systems, rates have been estimated at 0.2% per course and for those having clinically significant consequences, estimates are one or two orders of magnitude lower.4–6 Nevertheless, given the potential consequences of actual incidents, radiation oncology facilities deploy comprehensive QA programs and Incident Learning Systems (ILSs), alongside promoting a positive safety culture (SC).The Australian and New Zealand bi‐national radiation oncology practice standards (ROPS) recommend robust QA and incident management as requirements to mitigate risk.7 Internationally, radiation oncology departments have reported various local ILSs to support this.5,8–12 For robust incident evaluation, an ILS should include incident reporting, investigations (e.g. root cause analysis), data tracking, visualisation and practical feedback.13 It should also guide quality improvement (QI) and QA practices to guard against similar errors recurring. ILSs that analyse reported data to identify QI areas provide a more proactive and effective response to incident management than simple reactive changes to individual isolated reported incidents.8 Departmental ILSs that meet the reporting categories and needs of radiation oncology can impact on reducing error rates and provide appropriate data analysis and practical QI guidance.6 ILSs specifically designed for radiation oncology, coupled with continuous assessment and improvement, have reduced occurrence of significant incidents, provided QI insight and facilitated proactive approaches to quality and safety management.9,10,14ILS success relies on the department’s SC. Many studies have shown negativity and frustration from frontline staff regarding incident reporting and learning, more than from management staff.6,13,15,16 Negative responses have focussed more on unsatisfactory approaches to investigations, corrective actions and learning from incident reports rather than completing report forms.16 Staff show further frustration when ILSs are challenging or time‐consuming to use.13 Departmental SC can affect attitudes to an ILS and its utilisation and feedback to staff.As part of ongoing quality assessment and improvement, staff in the closely linked radiation oncology departments of two local health districts (LHDs) were surveyed, to determine current perceptions and understanding of the ILS in place and of departmental SC.At the survey time, the departments had multiple reporting systems, at three levels:Inhouse/departmentalILS including: paper‐based reporting forms for actual and near‐miss incidents and also departmental non‐radiation incidents; RO morbidity and mortality QA review meetings; RT senior staff meetings discussing all incident reports; and ROMP error discussion meetings.OrganisationalAn LHD‐level electronic platform, the Incident Information Management System (IIMS), for all actual incidents and near‐miss reports of any type.Mandatory bodiesHigher mandatory reporting at specified dose deviation levels; to the hospital/department Radiation Safety Officer (RSO) for dose deviations over 5%, and/or the NSW Environment Protection Authority (EPA), for greater than 10%, as per ROPS recommendations, state wide incident management policies and EPA legislation.7,17Incident classification followed the ROPS recommendations.7 The departments had similar processes and protocols around incident reporting, monitoring, learning and QI. Each professional discipline held monthly quality/safety meetings separately, with cross‐MDT discussion and collaboration for learning being mainly at inter‐group senior (management) levels.The aims of the survey were to identify staff understanding and use of the ILSs, any barriers to reporting and any needs for process change or departmental learning, as well as perceptions of SC. The findings prompted a QI project in one of the LHDs to evaluate and improve the ILS.MethodSurvey of ILS and SCCurrent understanding of ILSs and attitudes towards SC were evaluated via an anonymous online survey, distributed to Radiation Oncology professionals (all ROs, RTs and ROMPs) within radiation oncology departments in the two LHDs, Western Sydney (WSLHD) and Nepean Blue Mountains (NBMLHD). The survey took less than 15 min to complete. The project received ethics approval from each LHD's human research and ethics committees.Survey ‘gatekeepers’ were RTs who distributed the survey invitation to all RTs and ROMPs at WSLHD in October 2018 and all NBMLHD staff in December 2019. Distribution to WSLHD ROs was via RO administrative staff in February 2019. The survey was open for 2 weeks; gatekeepers sent reminder emails with current response rates on days 7 and 12. Participants were informed that completing the survey indicated consent to participate in the study.The survey content was based on radiation oncology SC surveys from the literature.6,11,13 Eleven staff across the MDT initially piloted the survey, then it was sent out on a larger scale. The survey captured occupation, years worked and role level; the last two were largely removed from analysis to ensure anonymity.The REDCap electronic data, capture tools hosted by The University of Sydney,18,19 were used to collect anonymous responses and manage the study data. Responses were exported to IBM SPSS20 for quantitative analysis, with results compared and summarised using descriptive statistics. Open‐ended questions were evaluated to derive any key themes.ResultsCharacteristics of respondentsInvitations were sent to 150 Radiation Oncology professional staff across WSLHD and NBMLHD; 95 (63%) responded, with similar overall response rates for each LHD (65% and 57% respectively). For the professional cohorts, response rates were 71%, 67% and 34% for RTs, ROMPs and ROs respectively.One respondent was excluded due to partial survey completion. The distribution of the 94 participants was 73% RTs, 15% ROMPs and 12% ROs, while staffing distribution at that time was 65%, 14%, and 21% respectively.Survey results between the two LHDs were not significantly different, utilising a Z test for two populations (P < 0.05), so were combined for analysis to further protect anonymity.Knowledge of incident reporting systemsRespondents showed various levels of ILS understanding and utilisation across the MDT. Table 1 shows the differences and similarities between the professional groups. Overall, 97% of respondents were aware of incident reporting. Those not familiar were all in their first employment year as RT Interns or RO Registrars.1TableUnderstanding of incident reporting and learning system.Survey questionsResponseRTn = 69 (71%)ROMPn = 14 (67%)ROn = 11 (34%)Totaln = 94 (63%)n%n%n%n%How many reporting systems in use are you aware of in your department?1913%536%218%1617%23754%536%436%4649%31826%214%19%2122%423%17%00%33%Don’t Know34%17%436%89%Please state the name of the reporting system/s you know of in use in the departmentNo response34%00%19%44%Don’t know23%214%218%66%In‐house/Departmental‐level only (e.g.: in‐house systems/meetings)710%00%00%77%Organisational level only (IIMS)2029%536%436%2931%Mandatory bodies only (RSO/EPA)00%00%00%00%Departmental and organisational level2841%643%19%3537%Organisational level and mandatory reporting23%00%218%44%Departmental, organisational and mandatory reporting dose deviation level (e.g.: RSO/EPA)710%17%19%910%Do you feel you can correctly Identify Actual Incidents vs Near Miss Incidents classification?Yes3957%1071%545%5457%No34%00%327%66%Some of the time2739%429%327%3436%Do you feel you can correctly identify radiation incident sub classification type in relation to the nine recommended categories within the radiation oncology practice standards?Yes4768%1071%545%6266%No46%214%545%1112%Some of the time1826%214%19%2122%Are you aware of a formal investigation system of actual or near miss radiation incidents in your department such as root cause analysis?Yes5377%964%218%6468%No1623%536%982%3032%EPA, environment protection authority; IIMS, incident information management system; RO, radiation oncologist; ROMP, radiation oncology medical physicist; RSO, radiation safety officer; RT, radiation therapist.The LHDs utilised three levels of reporting, as above: inhouse, organisational and mandatory. The largest respondent group (49%) acknowledged two systems in use. However, the professional cohorts showed differences in identifying systems. RTs (41%) and ROMPs (43%) largest responses identified dual systems at departmental and organisation levels. The RO cohort primarily stated organisational only (36%) or organisation and mandatory reporting (18%). Only 10% of respondents acknowledged all three.All cohorts indicated high confidence levels in categorising errors once identified (Table 1), with ROMPs having the strongest confidence (71%) for both category and sub‐categorisation.Utilisation and barrier perception of the current systemOverall, 51% of respondents had reported an incident to one or more systems in the six months before the survey (Table 2). Of the 46 staff who had not reported, 74% stated they had not observed or identified an error. The rest had identified an error but not reported, mostly because they had escalated to more senior staff who investigated and reported, or another team member had completed the incident report. Two respondents were aware of some near‐miss incidents that were not reported, and only one respondent indicated they chose not to report.2TableUtilisation and barriers to reporting.Survey questionsResponseRTn = 69ROMPn = 14ROn = 11Totaln = 94 (63%)n%n%n%n%Have you submitted an actual or near miss incident report in the last 6 months?Yes4464%321%19%4851%No2536%1179%1091%4649%What is the reason you did not submit a report in the last 6 months?Asked only ifprevious response was noChoose not to report00%17%00%12%Did not notice, observe or discover any incident or near‐miss event in the past 6 months2232%643%655%3474%Informed team leader who investigated and submitted the report23%17%218%511%Other open answer provided11%321%218%613%Number of barriers reported11928%536%436%2830%21319%214%218%1718%3812%17%436%1314%411%00%00%11%No perceived barriers2841%643%19%3537%What do you find is the biggest obstacle to you reporting actual or near miss radiation incidents in your department?(Multiple response allowed)Takes too long2333%17%545%2931%System is hard to access1319%17%545%1920%Don’t know how to use/or understand the system1217%214%764%2122%Don’t see the benefit of reporting46%17%00%55%Fear of negative action towards self or others1420%321%00%1718%I do not think there are any obstacles to reporting in my department2841%643%19%3537%Other open answer provided69%429%218%1213%RO, radiation oncologist; ROMP, radiation oncology medical physicist; RT, Radiation therapist.Overall, 37% of respondents perceived no barriers within the current system. However, 59 staff did perceive one or more barriers (Table 2). The most significant stated barrier was the time it takes (31%), then lack of knowledge/understanding of the system and its use (22%) and that it was hard to access (20%). Potential improvements to safety culture were indicated by the 18% of respondents that stated a barrier was related to fear of adverse action, and the 5% that did not see a benefit to reporting. These results show the most significant barriers were related to the system rather than departmental SC.Preferences for feedback and learningRespondents ranked their preference for learning and feedback (Table 3). The preferred method overall was an all‐staff MDT meeting with either mandatory or open attendance. Next was for selected staff to attend incident meetings, that is representatives from different work areas/groups who report back to others; 54% acknowledged this as the primary method currently utilised across the departments. RT and RO cohorts had similar ranking preferences for learning options. ROMPs showed some differences, such as a higher preference for newsletters to disseminate relevant information and lower for in‐service training.3TablePreference for feedback and learning.Preference rankingsRTROMPROTotalAll staff attendance at MDT incident reporting meeting. Either mandatory meeting or open attendance1st1st1st1stSelected staff attendance at incident reporting meeting (e.g.: Team leaders, or safety/quality team)2nd3rd2nd2ndAttending in‐service training3rd6th4th3rdNewsletter or email notification4th2nd3rd4thWord of mouth5th4th5th5thNone at all6th5th6th6thRO, radiation oncologist; ROMP, radiation oncology medical physicist; RT, Radiation therapist.Safety culture and learning capacityThe majority of staff (66%) feel encouraged to report, with 60% feeling comfortable reporting (Table 4). Most respondents (69%), but least strong for the ROMP group, thought their department practised a no‐blame culture (Table 4). Sixteen respondents stated their department did not practice a no‐blame culture, eight having either personally received, or witnessed others receive, adverse action, with two others declining to answer. Regarding assigning cause and blame between staff and processes, 37% of respondents gave this as 50%/50% staff/process, followed by 29% at 25%/75%. The majority of respondents (71%) gave positive responses to departmental learning capacity from reported incidents, with the RO cohort having the strongest perception.4TableSafety culture and learning.Survey questionsResponseRTn = 69ROMPn = 14ROn = 11Totaln = 94 (63%)n%n%n%n%Please rate how encouraged you feel to report actual or near miss radiation incidents within your radiation oncology department?Very discouraged to report11%00%00%11%Discouraged to report57%214%00%77%Neutral1420%750%327%2426%Encouraged to report2638%321%655%3537%Very encouraged to report2333%214%218%2729%Do you feel comfortable reporting actual or near miss radiation incidents within your radiation oncology department?Very uncomfortable reporting710%00%00%77%Uncomfortable reporting69%214%19%910%Neutral1319%643%327%2223%Comfortable reporting1928%321%655%2830%Very comfortable reporting2435%321%19%2830%Do you feel that your radiation oncology department practices a culture of no‐blame when errors are reported?Yes5275%643%764%6569%No913%643%19%1617%Other open answer provided812%214%327%1314%Have you ever personally received or witnessed other staff members receiving negative action towards them due to a reported radiation incident or near miss?Yes1420%429%19%1920%No5072%964%1091%6973%Do not wish to answer46%00%00%44%Other open answer provided00%17%00%11%After an actual or near miss radiation incident report is submitted, in your opinion, the cause/blame is mostly assigned to:0% on staff member, 100% on current process69%17%00%77%25% on staff member, 75% on current process1826%214%764%2729%50% on staff member, 50% on current process2841%536%218%3537%75% on staff member, 25% on current process1319%214%00%1516%100% on staff member, 0% on current process11%00%00%11%Other open answer provided34%429%218%910%How well do you believe your radiation oncology department is willing and able to learn from previous radiation actual and near miss incidents? Such as making positive process changes and or implementing appropriate training and education when necessaryUnable to learn from previous incidents00%17%00%11%Minimal ability to learn46%429%00%89%Neutral1623%214%00%1819%Some demonstrated ability to learn2942%429%545%3840%Demonstrated ability to learn from previous incidents2029%321%655%2931%RO, radiation oncologist; ROMP, radiation oncology medical physicist; RT, Radiation therapist.Qualitative resultsFour areas of free‐text answers were possible for questions related to (1) barriers to reporting, (2) no‐blame culture, (3) blame association to staff versus system/process and (4) open comment around the survey.The thematic analysis highlighted four key themes: (1) SC issues, (2) blame is situational dependant, (3) QI and learning weaknesses and (4) current department‐level and hospital‐level reporting system deficits.Respondents mentioned SC fourteen times. Overall, these supported a departmental no‐blame culture, but some responses, mainly from RTs at frontline and management levels, indicated some staff blame others and gossip. Blame assignment to staff versus process was identified as situational influenced, for example actual incident versus near miss, or perceived laziness of staff member/s involved.Nine open responses indicated inadequacies in the current reporting system, with 11 perceiving weakness in QI and learning. The main concerns were that the more general organisational reporting systems do not fit radiation oncology needs, and reports disappear into the system with no feedback or learning. Others indicated the analysis and learning do not focus on the actual error‐causing problems and perceptions that education is often not prioritised, with minimal preventive measures to mitigate future risk.Quality improvement projectThe survey findings were used to inform a QI project actioned in one LHD to evaluate and improve their current departmental (level i) ILS. An MDT QI project team was established to design, create and develop an electronic reporting system to suit departmental needs. This was guided by recommendations from literature, ROPS, barriers and other factors identified in the survey.Application of survey findings to improve local ILSElectronic report developmentA new customised electronic departmental‐level reporting system was locally developed on the Varian AriaTM oncology information system (Varian Medical Systems, Palo Alto, CA, USA) to improve use and support enhanced analysis and department learning and feedback. Integration into AriaTM was designed to reduce the following barriers: the time it takes to report, lack of understanding of system and use, and easier system access. It also enabled increased communication to appropriate staff and report extraction into Microsoft ExcelTM and Microsoft Power BITM (business intelligence/data analytics software) to increase data tracking and visualisation.Report analysis changesA dedicated MDT incident triage team was established and trained to support management. This provided a coordinated, centralised, structured and rapid approach to analysis and recommendations from reports.Feedback and learningRelevant meeting structures and attendance were changed, with MDT representatives in all meetings to ensure shared learning and discussion across professions. This increased communication with and between staff. Reports were easily available to all, since they were in a readily accessible electronic system. Quick feedback loops were introduced to staff involved in observed errors or barrier detection point fails and for urgent process changes. During COVID‐19 restrictions, a newsletter was used to provide regular feedback when meetings were not feasible.Focussed education on how and what to reportA three‐month pilot phase tested the new system within WSLHD's smaller campus to ensure that access and use were straightforward. The system was implemented across the whole LHD. Mandatory training for all staff included how to use the new system and what to report on. A protocol non‐compliance category was introduced to report near‐miss errors which had evaded one or more barriers before being found within QA pathways before the first treatment fraction. This was to strengthen knowledge of any systematic QA weaknesses.DiscussionThis work identified perceptions of SC and ILSs and barriers to reporting incidents in two Australian LHDs’ radiation oncology departments. The departments surveyed participated in joint tumour stream QA meetings and had similar in‐house ILSs. Most responding ROs worked across both LHD's, whereas only a few ROMPs and no RT staff did. The results show varied knowledge and understanding of the complete incident reporting systems, structure and associated learning.By profession, the survey response rate was highest for RTs, followed by ROMPs, in both LHDs. Although errors and incidents can occur at any point in the patient pathway, detection is highest at treatment delivery and QA checkpoints, performed most frequently by RTs, followed by ROMPs.21 Therefore, their greater use of reporting may influence higher response rates. RO staff primarily received reports either classified as actual incidents, with incorrect radiation delivery, or when high‐risk near‐miss incidents have been reported and discussed. Overall, RT and ROMP response rates were high compared to other literature‐reported surveys of medical professionals, for example Cook et al.20 presented median response rates of 59% to postal surveys of healthcare professionals.22 Cunningham et al.21 found a lower response rate (35%) for physicians.In the current work, professional cohorts showed differences in using and understanding the various ILSs (Table 1). RTs and ROMPs predominantly identified departmental and organisational systems, with ROs primarily identifying organisational and mandatory reporting. This reflected different groups main uses of systems, with RTs and ROMPs being predominantly involved in in‐house and organisational reporting and in‐house QA meetings for feedback and learning. ROs primarily used the organisational reporting systems and, when necessary, provided information for reports to mandatory bodies. In‐house system reporting capacity extends more widely and is more specific to radiation oncology needs than the organisational system. However, each serves different but overlapping purposes with other capabilities. Hence, the departments used both in parallel.Differences in each cohort's use of reporting systems and feedback and learning loops may have influenced their different responses. ROs and ROMPs groups more frequently acknowledged mandatory reporting. These groups are heavily involved in investigations and decisions, alongside the RT management teams. When dose deviations requiring such reporting occur, ROMPs quantify dose deviations and prescribing ROs determine whether there are any clinical consequences to the patient and they also further provide decision‐making around any changes to the patient plan or symptom management and to facilitate open disclosure to the patient. Fewer of the RT cohort mentioned mandatory reporting, which occurs after the initial report and involves investigations primarily completed by ROMPs, RT management and RO collaboration.All professional groups had limited awareness of the overall ILS processes, systems and intended purposes, indicating potential learning opportunities. Each cohort only showed a strong understanding of the system/s most utilised by their group. The differences indicated potential for improved learning across departments to give a more interdisciplinary collaborative approach to the overall ILS.Some findings indicated potential areas for departmental education and learning around description and categorisation of errors. Increased reporting accuracy should increase data reliability and more robust trend analysis from reported incidents.23The question ‘state the name of the reporting system/s you know of in use in the department?’ had few respondents (14%) stating any mandatory bodies. However, mandatory bodies were then mentioned in open answers by more staff, indicating staff have potentially interpreted the initial question as ‘within the department’, that is internal systems and not external reporting. This might have been better worded as ‘ … that are used by the department’.Reporting to any ILS was predominantly by the RT staff followed by ROMPs, supporting the literature that frontline staff are more likely to detect incidents or errors at QA checkpoints.21,23,24 It is positive that staff are willing to report when they are aware of an error, with only one respondent stating they chose not to and two indicating that sometimes near misses were not reported.Overall, 63% of respondents perceived one or more barriers to reporting within the current systems. The leading stated reason was the time involved, then lack of knowledge/understanding of the system or its use, or access difficulty. Obstacles related to negative SC, such as fear of negative actions or not seeing the benefit of reporting, had lower responses. It was promising that barriers relate to the structure, format and use of the current ILSs rather than SC awareness in staff or the perception of how the department treats safety issues. Ford et al.3 noted that electronic ILSs, customised to radiation oncology, reduce reporting barriers. ILS success is related to appropriate resources and utilisation, partnered with staff understanding and confidence that the SC is just and equitable.25Respondents' perceptions of SC and learning were stronger towards positive SC than negative, for example two‐thirds felt encouraged to report and comfortable reporting. A no‐blame culture was perceived by most respondents, with 73% not having witnessed or received adverse action. Two‐thirds of respondents perceive cause and blame after an error to be attributed 25–50% to staff and 50–75% to processes. This further supports a no‐blame culture. Our findings are similar to those of Bolderston et al.6 Overall, the responses support positive SC and the ability to learn. However, as some respondents perceived blame culture or negative SC, there is still room for improvement.The thematic review of the free‐text answers provided insight into some of the issues indicated in the quantitative results. There was a strong focus on developing changes to the ILS regarding feedback and learning pathways. Changes included developing an in‐house electronic reporting system that fit the scope of reporting needed within radiation oncology and provided fast consistent data analysis. This was developed and implemented in the ensuing QI project.The newly developed electronic reporting system is integrated onto the AriaTM platform, with rapid collaborative MDT triaging to manage reports in real time. This enables more effective education and reminders to staff when needed, ensuring continuous feedback between monthly meetings. Meetings are now MDT‐based rather than professionally separated and are open for anyone to attend. Hence feedback and learning pathways have a more collaborative approach with increased knowledge of all systems in each group. The meeting purpose is to review themes, discuss learning opportunities from reports and potential areas for improvement if process areas were regularly failing or of high risk and provide recommendations for working parties to consider. The overall goal was to improve feedback loops to the whole MDT, to help promote continued and increasing SC and learning and assist in QI decision‐making.Furthermore, now that all data capture is electronic, it is more comprehensive and consistent, and visualisation is improved. This helps to provide information for department educational needs to increase learning or identify when QI projects or QA checkpoint modifications are needed.In developing the new in‐house system, the need to reduce the three most significant perceived barriers to ILSs use was considered, the lack of knowledge/understanding of available systems, that access was difficult, and the system reporting time required. A clear structure was created to identify which system to report to, in‐house only or in‐house and hospital‐wide. The in‐house system was designed to be easy to access, quick to fill in and to readily provide all relevant patient data to reduce the number of data fields a reporter needs to complete compared to the previous paper‐based reporting forms. The incident triage team review in‐house system reports within 24 h of creation. When a report (near‐miss or actual incident) requires reporting to hospital systems, they coordinate this with all relevant information, including ROMP dosimetry reports, to assist the initial reporter. This has reduced the time involved and increased staff understanding of how to use and report into the hospital‐wide system. The in‐house system has streamlined the ROMP portion of dose difference evaluation and reporting and provided clearer and faster pathways if an incident is RSO or EPA reportable.Thus, the survey helped to understand the departmental SC and how staff perceive and use the current ILSs, which helped guide decisions around design and changes to the ILS and the change implementation process. It has also provided support for changes to learning and educational needs required across the MDT to ensure a comprehensive, collaborative and open approach to ILSs, reduce barriers, increase reporting and increase the positive SC.The new system has been in use across the whole WSLHD for seven months and ten months at the pilot campus. There are plans to survey again later to evaluate accumulated experience and the need for any further QI.ConclusionA survey of perceptions of SC and understanding of ILSs established a baseline understanding within two LHDs. In one LHD, the results led to the development of a QI project that significantly improved the ILS. Major changes were implemented to aspects of reporting and to the feedback and learning portions of the ILS, as the survey had highlighted barriers to reporting and areas to improve feedback and learning across the department. The study findings provide a reference for future evaluations of ILS and SC that may identify continued improvements as the impact of the changes continues to be assessed, including further regular surveys, review of data accuracy in reports and trend analysis of incidents.AcknowledgementsThe authors acknowledge Shamira Cross, Alexander Grant and Nicole Burns for assistance provided; Jillian Harris, Kevin Van Tilburg and A/Professor Verity Ahern for support of the project; and the staff within the Radiation Oncology Departments at WSLHD and NBMLHD for participating in this study.Conflict of InterestThere are no conflicts of interest.Funding InformationNo funding supported this project.Ethics StatementEthics approval WSLHD from QA/QI Committee on 15 February 2019. Approval: 1812‐02. Ethics approval NBMLHD from Appolo Committee on 15 November 2019. Approval: 19‐42(A).ReferencesRANZCR. Radiation Oncology Targeting Cancer: What is Radiation Therapy: The Faculty of Radiation Oncology. The Royal Australian and New Zealand College of Radiologists, 2017. Available from: https://www.targetingcancer.com.au/what‐is‐radiation‐therapy/D'Souza N, Holden L, Robson S, et al. Modern palliative radiation treatment: Do complexity and workload contribute to medical errors? Int J Radiat Oncol Biol Phys 2012; 84: e43–e8.Ford EC, Evans SB. Incident learning in radiation oncology: A review. Med Phys 2018; 45: e100–e19.Barton M, Shafiq J. Radiotherapy Risk Profile: Technical Manual. World Health Organization (WHO), Geneva, 2008.Ford E, de Los F, Santos L, Pawlicki T, Sutlief S, Dunscombe P. Consensus recommendations for incident learning database structures in radiation oncology. Med Phys 2012; 39: 7272–90.Bolderston A, Di Prospero L, French J, Church J, Adams R. A culture of safety? An international comparison of radiation therapists' error reporting. J Med Imaging Radiat Sci 2015; 46: 16–22.RANZCR. Radiation Oncology Practice Standards Sydney: The Faculty of Radiation Oncology (FRO), The Royal Australian and New Zealand College of Radiologist (RANZCR), Australian Society of Medical Imaging and Radiation Therapy (ASMIRT), The Australasian College of Physical Scientists and Engineers in Medicine (ASPSEM); 2018. Available from: https://www.ranzcr.com/whats‐on/news‐media/240‐new‐revised‐radiation‐oncology‐practice‐standardsClark BG, Brown RJ, Ploquin JL, Kind AL, Grimard L. The management of radiation treatment error through incident learning. Radiother Oncol 2010; 95: 344–9.Nyflot MJ, Zeng J, Kusano AS, et al. Metrics of success: Measuring impact of a departmental near‐miss incident learning system. Pract Rad Oncol 2015; 5: e409–e16.Gabriel PE, Volz E, Bergendahl HW, et al. Incident learning in pursuit of high reliability: Implementing a comprehensive, low‐threshold reporting program in a large, multisite radiation oncology department. Jt Comm J Qual Patient Saf 2015; 41(4): AP1.Milosevic M, Angers C, Liszewski B, et al. The Canadian national system for incident reporting in radiation treatment (NSIR‐RT) taxonomy. Pract Rad Oncol 2016; 6: 334–41.Arnold A, Delaney GP, Cassapi L, Barton M. The use of categorized time‐trend reporting of radiation oncology incidents: A proactive analytical approach to improving quality and safety over time. Int J Radiat Oncol Biol Phys 2010; 78: 1548–54.Montgomery L, Fava P, Freeman CR, et al. Development and implementation of a radiation therapy incident learning system compatible with local workflow and a national taxonomy. J Appl Clin Med Phys 2018; 19: 259–70.Romanofski D. The evolution of incident learning. Radiol Technol 2016; 88: 90–4.Shojania KG. The frustrating case of incident‐reporting systems. Qual Saf Health Care 2008; 17: 400–2.Cooke DL, Dunscombe PB, Lee RC. Using a survey of incident reporting and learning practices to improve organisational learning at a cancer care centre. Qual Saf Health Care 2007; 16: 342–8.NSW. Incident Management: Policy Directive. NSW Government, Sydney, 2020.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. A metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42: 377–81.Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019; 95: 103208.Corp I. IBM SPSS Statistics for Windows, Version 27.0. IBM Corp, Armonk, NY, 2019.Cunningham J, Coffey M, Knöös T, Holmberg O. Radiation oncology safety information system (ROSIS)–Profiles of participants and the first 1074 incident reports. Radiother Oncol 2010; 97: 601–7.Cook JV, Dickinson HO, Eccles MP. Response rates in postal surveys of healthcare professionals between 1996 and 2005: An observational study. BMC Health Serv Res 2009; 9: 1–8.Greenham S, Manley S, Turnbull K, et al. Application of an incident taxonomy for radiation therapy: Analysis of five years of data from three integrated cancer centres. Rep Pract Oncol Radiother 2018; 23: 220–7.Smith S, Wallis A, King O, et al. Quality management in radiation therapy: A 15 year review of incident reporting in two integrated cancer centres. Tech Innov Patient Support Radiat Oncol 2020; 14: 15–20.Liszewski B, Angers C, Kildea J. Mitigating the barriers to a culture of quality and safety in radiation oncology. Clin Oncol 2017; 29: 676–9.

Journal

Journal of Medical Radiation SciencesWiley

Published: Jun 1, 2022

Keywords: Incident learning; incident reporting; quality and safety; radiation oncology; radiation therapy; safety culture

There are no references for this article.