Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Learning from Incidents: A Qualitative Study in the Continuing Airworthiness Sector

Learning from Incidents: A Qualitative Study in the Continuing Airworthiness Sector aerospace Article Learning from Incidents: A Qualitative Study in the Continuing Airworthiness Sector 1 1 , 2 , James Clare and Kyriakos I. Kourousis * School of Engineering, University of Limerick, V94 T9PX Limerick, Ireland; james.clare@ul.ie School of Engineering, RMIT University, Melbourne, VIC 3001, Australia * Correspondence: kyriakos.kourousis@ul.ie Abstract: Learning from incidents (LFI) is a useful approach when examining past events and developing measures to prevent ensuing recurrence. Although the reporting of incidents in the aircraft maintenance and continuing airworthiness domain is well appointed, it is often unclear how the maximum effect of safety data can be efficaciously applied in support of LFI in the area. From semi-structured interviews, with thirty-four participants, the gathered data were thematically analyzed with the support of NVivo software. This study establishes a relationship between an incident in its lifecycle and the learning process. The main aim of this work is to elucidate factors that enable LFI. The analysis of the data revealed, for example, the benefits of a just culture and the use of formal continuation training programs in this respect. Moreover, it identified limitations inherent in current processes such as poor event causation and poorly designed learning syllabi. Additionally, aspects such as a lack of regulatory requirements for competence in the areas of learning for managers and accountable persons currently exist. This thematic analysis could be used in support of organizations examining their own processes for learning from incidents. Additionally, it can support the development of terms of reference for a continuing airworthiness regulatory working group to examine, strengthen and better apply LFI in the aviation industry. Citation: Clare, J.; Kourousis, K.I Keywords: learning form incidents; airworthiness; aircraft maintenance; safety management Learning from Incidents: A Qualitative Study in the Continuing Airworthiness Sector. Aerospace 2021, 8, 27. https://doi.org/10.3390/ 1. Introduction aerospace8020027 If it were possible for all organizations to learn effective lessons from the past, the effects of future unwelcome events might be limited [1]. Aviation safety depends to a large Academic Editor: Rosario Pecora extent on the efficacious efforts of all involved in the system [2]. Research has acknowl- Received: 16 December 2020 edged the importance of event information when it comes to learning and preventing Accepted: 18 January 2021 recurrence [3]. Thankfully, major events such as accidents are becoming less frequent and Published: 22 January 2021 generate less points for learning [4]. In contrast, there are numerous incidents with less severe consequences and if appropriately considered, these could offer an earlier insight Publisher’s Note: MDPI stays neutral into the circumstances that enable unwelcome events. Predefined and relevant information with regard to jurisdictional claims in harvested from incident reporting systems is a major element of learning and preserving published maps and institutional affil- acceptable levels of safety. Hobbs and Williamson [5] highlight the importance of aircraft iations. maintenance staff being aware of the cumulative effect of “seemingly insignificant” inci- dents as this amplifies the need to be proactive when it comes to learning from incidents. This research undertook a qualitative examination of staff involved in aircraft maintenance and continuing airworthiness operations in order to identify factors that could augment Copyright: © 2021 by the authors. learning from incidents within this industry sector. Licensee MDPI, Basel, Switzerland. In the areas of continuing airworthiness and aircraft maintenance, safety manage- This article is an open access article ment systems include incident and occurrence reporting [6] as an obligation. It is common distributed under the terms and for incidents to be discovered within organizations and reported with the assistance of conditions of the Creative Commons such “systems of systems” [7]. On an operational level, initial training on human factors Attribution (CC BY) license (https:// and company procedures is intended to specify and re-affirm the category and type of creativecommons.org/licenses/by/ occurrence and incident that should be reported. Recent developments in European Union 4.0/). Aerospace 2021, 8, 27. https://doi.org/10.3390/aerospace8020027 https://www.mdpi.com/journal/aerospace Aerospace 2021, 8, 27 2 of 20 (EU) regulations [8] empower voluntary and confidential reporting and are independent of all other individual obligations. Detecting and identifying hazards highlighted through incident reporting systems is also recommended by the International Civil Aviation Organi- zation (ICAO) standards and recommended practices as an effective means of augmenting levels of safety. However, Gerede [9] strongly suggests that a failure to foster a just cul- ture is considered to have a negative impact upon effective data collection (reporting), organizational learning and the subsequent ability to learn from incidents. Drupsteen and Wybo [10] reaffirm organizations use experience gained from past events in order to improve safety. Effective learning can be considered as a successful translation of safety information into knowledge. Utilizing information from events with learning potential can actively improve the operating environment and help prevent recurrence. Learning in this context can often be experienced as modifying or implementing new knowledge where cultural, technical or procedural elements are integrated. Therefore, when learning is transformed into measures to prevent re-occurrence, an organization often has a reasonable means of mitigating future similar events. Argyris and Schön [11] highlight the importance of learning to detect and address effective responses to errors. Their “theory in action” concept is the focal point for this determination. The first of its two components, “theory in use” is one that guides a person’s behavior. It is often “tacit” and is how people behave routinely. Very often these observed “habits” are unknown to the specimen. The second element is known as “espoused theory”, namely what people say or think they do. Drupsteen and Guldenmund [12] mention that espoused theory comprises of “the words we use to convey what we do, or what we like others to think we do”. However, it is important to re-affirm the linkages that exist between individuals and organizational learning. The introduction of safety management systems (SMS) has initiated a shift in how organizational errors are viewed. Firstly, equipment has become increasingly more reliable, but the human form has not displayed the same response. In the second instance, the impact of complexities associated with an increasing cognitive load for staff is just beginning to be realized. The existence of a potential for blaming an individual is now being aligned with organizational responsibilities. Prior to this, event causation was often misrepresented or even over quantified the human input as organizational factors were not always considered. They offer an insight into the connection between individual actions and organizational initiatives designed to secure the best safety outcomes. Fogarty et al. [13] also recognize the role that both individual factors have on human error and the inputs both can have on preventing recurrence. ICAO Doc 9859 ICAO [14] defines a template for aviation operators and regulators to support the application of a variety of proactive, predictive and reactive oversight methodologies. In addition to routine monitoring schemes, voluntary and mandatory reporting, post incident follow-up, there are also regular safety oversight audits. These audits and inspections often set out to establish if there is a difference between espoused theory and the theory in use (e.g., is the task being correctly performed in accordance with the documented procedure/work instruction or is there a deviation from approved data and practice?). However, Drupsteen and Guldenmund [12] caution auditors not to “focus too much on the documentation of procedures” alone. In such cases, the oversight audit may be ineffective because of its sole focus on espoused theories of the organization only and not the theory-in-use. These authors translate this idea of poor focus on theory in action, into a valid learning component arising from incidents. They also highlight the “espoused” aspect where those attempting to learn from incidents often fail to experience the desired learning because outcomes are not fully aligned with the practical objectives of a learning from incidents (LFI) initiative. For learning to be most effective, espoused theory and theory in use should be reasonably well aligned. Ward et al. [15] propose it is necessary to further develop an operational model that can account for “what is meant to happen and what actually happens”. Continuing airworthiness and aircraft maintenance and activities performed in EU member states are subjected to rules that mandate reporting of defined issues. Repositories Aerospace 2021, 8, 27 3 of 20 of reported data tend to be populated by sources that are predominantly the subject of mandatory reporting requirements. Conventional safety oversight models also only verify the presence of reporting media and repositories in this segment of the industry. Jacobsson et al. [16] avow the degree of interest invested in learning from incidents but question its efficiency in some organizations. Although unwelcome events are less prevalent, less severe events still provide learning opportunities. There is often only a primary focus for organizations upon reporting in line with each state’s own reporting obligations. Unfor- tunately, a narrow focus on this single element of an incident in its lifecycle can negate the potential benefits of learning from incidents at an organizational level. The absence of clearly defined competency requirements [6] that support a pedagogy for learning from incidents for continuing airworthiness staff could also be considered an impediment to effective learning in the domain. The featured industry sector is regulated by the application and upkeep of numerous requirements in the jurisdictions of operation. In general, a costly regulatory overhead tends to be carried by regulating states and operators to support safe and viable activity. However, a growing tendency to increase regulatory requirements in pursuance of safer activity across the segments may not always offer the same returns as previously realized by states. Brunel [17] (p.45) suggests, “ . . . it is impossible to make men perfect: the men will always remain the same as they are now and no legislation will make him have more presence of mind . . . ”. Furniss et al. [18] reviewed the Hollnagel [19] Functional Resonance Analysis Method (FRAM) which explores how functional variability resonates within systems, i.e., how well comprising elements function in a system. They also consider how FRAM can be modified to support complex socio technical system improvements. Perhaps as the paradigm supporting the linearity of regulatory oversight shifts, proactive regulatory inputs will also influence more effective safety outputs as intricacy increases. 1.1. Systematic Literature Review The primary reason for conducting a systematic review was to examine how learning from incidents occurs in aircraft maintenance and continuing airworthiness management. Other sectors and the issues impacting learning in these areas were also considered. The literature review sets out to establish factors that contribute to or potentially constrain learning from incidents in the subject domain. Applying a qualitative research approach is advantageous as it can provide a deeper contextual understanding of the literature and can assist with better research integration. The application of rigor and comprehensiveness can assist with advancing knowledge and identifying research gaps and aspects for further research in this area. Okoli and Schabram [20] suggest “a dedicated methodological approach is necessary in any kind of literature review”. A preliminary search of literature highlighted a scarcity of best-practice guidelines for conducting systematic literature reviews in this area. Qualitative research involves handling considerable volumes of data and a degree of discipline is required so that search results, decisions regarding subject inclusions and exclusions are recorded and references are well managed. Endnote was used in support of the literature review during this research. An electronic database is useful for supporting a search strategy, arranging publications and storing references [21]. The qualitative data analysis software NVivo (NVivo 12, QSR International, Melbourne, Australia) was used to augment the data management, storage and analysis associated with the literature review. NVivo possesses many functions, such as facilitating the synthesis of a review [22]. A sys- tematic search of in excess of 1000 publications was performed in the following databases: Web of Science, Scopus, IEEE Xplore, ProQuest and EBSCO. The following predefined search terms were applied: “learning from incidents”, “learning from experience”, “aircraft maintenance”, “aircraft management” and “safety management systems”. A practical screening of title and abstract was applied to each manuscript using predefined terms (e.g., subject, setting, publication, year). This part of the process had to be broad enough to create a sufficient number of applicable publications but also had to be practically man- Aerospace 2021, 8, 27 4 of 20 ageable. The following criteria were implemented for the practical screening of the source bibliographic details, title and abstract: Subject—Related to learning from incidents and past experiences. Setting—Any high reliability industry or sector where learning from incidents is critical. Publication—Journal or peer-reviewed conference proceedings. Date range—published post 1992. The year 1992 was the starting point for the screen- ing process, since at the time of planning the research project, 25 years was considered to be a reasonable timespan to include material pertaining to learning from incidents. The output of the practical screen step produced a list of publications denoted as the screened set of publications. An Endnote library was then created to store and manage the full text of the retrieved publications. The next step involved filtering the publications into primary and secondary publication subsets using only primary research manuscripts in the next phase. Applying a set of criteria helps to reduce any researcher bias in the screening system. A set of inclusion and exclusion criteria [23] was developed in accordance with the guidelines included in [24,25], listed in Table 1. Two researchers were involved in the screening process. Table 1. Inclusion and exclusion criteria used for the filtering of the subset of primary publications. Included Excluded Research studies Literature reviews Qualitative and mixed methods Quantitative methods Perceptions and experiences Focused on decision-making and legislative requirements Reference to just culture Not about “no blame” or a punitive approach High reliability settings Non high reliability settings Published post 1992 Peer-reviewed publications Industry based settings Original studies The final set of 18 papers was imported into NVivo and the following analysis approach, as defined by Bandara et al. [22], was used for the selection of the codifica- tion themes: Deductive—themes reported on are predetermined to some extent. In this case, these predetermined themes were the output of a focus group process. Inductive—themes reported are derived from analysis of the literature. NVivo is limited in terms of providing thematic classifications based on the occurrence of key words but can assist with identifying relationships between words and phrases amongst publications. It also provided thematic classifications of data based on the oc- currence of key words and phrases. The coding process consisted of selecting relevant passages of text that were captured in one or several of the framework nodes. Maykut and Morehouse [26] defines a propositional statement as “a statement of fact the researcher tentatively proposes, based on the data”. Memos were used to draft these summary state- ments which formed part of the literature review. Central to the idea of learning is how an incident is generally moderated during its useful existence. Section 1.2 documents this approach. 1.2. The Notion of a Generic Incident Lifecycle Figure 1 illustrates how an incident tends to be managed through its quiddity. This view is one possible way of representing the elements comprising a lifecycle view. Cooke and Rohleder [27] suggest it should also be evident that an incident system will operate most effectively when a safety management system has already been put in place and avoid- able risks are addressed. They propose an effective system that addresses: identification Aerospace 2021, 8, 27 5 of 20 1.2. The Notion of a Generic Incident Lifecycle Figure 1 illustrates how an incident tends to be managed through its quiddity. This view is one possible way of representing the elements comprising a lifecycle view. Cooke Aerospace 2021, 8, 27 5 of 20 and Rohleder [27] suggest it should also be evident that an incident system will operate most effectively when a safety management system has already been put in place and avoidable risks are addressed. They propose an effective system that addresses: identifi- and response, reporting, investigation, identifying causal structure, making recommenda- cation and response, reporting, investigation, identifying causal structure, making recom- mendationstions, , communicat communicating ing and reca andll rin ecalling g incident incident learning, learning, and im and plement implementing ing correct corr ive ective actions. Drupsteen et al. [28] also consider an incident from a learning perspective in its cycle. actions. Drupsteen et al. [28] also consider an incident from a learning perspective in its Their main constituents are investigating and analyzing incidents, planning interventions, cycle. Their main constituents are investigating and analyzing incidents, planning inter- intervening and evaluating (each of these four stages are further sub-divided into eleven ventions, intervening and evaluating (each of these four stages are further sub-divided sub-components). Continuing airworthiness-related incidents are notified by way of a into eleven sub-components). Continuing airworthiness-related incidents are notified by formal mechanism of reporting. During the data gathering phase of this research, the steps way of a formal mechanism of reporting. During the data gathering phase of this research, outlined in Figure 1 were found to be dictated by regulatory requirements [6,8]. Once the the steps outlined in Figure 1 were found to be dictated by regulatory requirements [6,8]. incident enters its lifecycle, it ideally transverses a process that transforms the information Once the incident enters its lifecycle, it ideally transverses a process that transforms the gathered into knowledge. Figure 1 and the contiguous paragraph offer an overview of how information gathered into knowledge. Figure 1 and the contiguous paragraph offer an the capture and processing of the incident information occurs in practice. overview of how the capture and processing of the incident information occurs in practice. Figure 1. An example of an aviation incident lifecycle within the continuing airworthiness and Figure 1. An example of an aviation incident lifecycle within the continuing airworthiness and aircraft maintenance sector. aircraft maintenance sector. Continuing airworthiness-related serious incidents are rare but often due to environ- Continuing airworthiness-related serious incidents are rare but often due to environ- mental, cognitive and mechanical demands, reportable and unreportable events do occur. mental, cognitive and mechanical demands, reportable and unreportable events do occur. All organizations in the industry segment subscribe to a reporting system and reports can All organizations in be made the electr ind onically ustry segmen or in paper t subsform cribe to in smaller a reportor ing ganizations. system and The repmain orts can underpinning be made elect regulation ronically or i in Eur n paper form ope, EU regulation in smalle2018/1139 r organizat[ ions. T 29], refers he main to a unde management rpinning system and regulation in Europe, EU regulation 2018/1139 [29], refers to a management system and mandates an organization to implement and maintain such a system to ensure compliance mandates an with orga these nizati essential on to imr pl equir ement a ements. nd m In aint practice, ain such althoug a systh em t a reporter o ensure co can mplianc report events e directly with these essential to an aviation requir authority ements. In , all pr oractice ganizations , although are requir a reporter ed to can have report an internal events di- reporting system also. A focal point/gate keeper will process these reports either internally and/or inform rectly to an aviation authority, all organizations are required to have an internal reporting third party stakeholders such as aviation authority or aircraft manufacturer as required system also. A focal point/gate keeper will process these reports either internally and/or by procedure. Depending on the event, technical management may determine there are inform third party stakeholders such as aviation authority or aircraft manufacturer as re- immediate actions required to recover a situation or restore serviceability. While a small quired by procedure. Depending on the event, technical management may determine number of scenarios will require an event to be investigated fully before an aircraft returns there are immediate actions required to recover a situation or restore serviceability. While to service, many incidents are investigated post event. As soon as causation is established, a small number of scenarios will require an event to be investigated fully before an aircraft if accepted by the relevant technical function, the report is closed. This management returns to service, many incidents are investigated post event. As soon as causation is system is strongly influenced by regulatory requirements and procedural form and is a established, if accepted by the relevant technical function, the report is closed. This man- pre-eminent influence on how an incident and its actors behave from the time a report is agement system is strongly influenced by regulatory requirements and procedural form made to the time its impact has been terminated. One of the limitations inherent in this and is a pre-eminent influence on how an incident and its actors behave from the time a cycle is that lessons tend to be delivered at a later point in time mostly through the medium report is made to the time its impact has been terminated. One of the limitations inherent of recurrent training programs such as continuation and human factors training. Therefore, in this cycle is that lessons tend to be delivered at a later point in time mostly through the there is often a hiatus in the feedback cycle. However, the effectiveness of the process and medium of recurrent training programs such as continuation and human factors training. the perceived contribution to learning are not fully reflected in this view. Therefore, there is often a hiatus in the feedback cycle. However, the effectiveness of the process and the perceived contribution to learning are not fully reflected in this view. 1.3. A Potential Learning Cycle Emerges According to Lindberg et al. [30], in order to prevent accidents, it is essential to learn from previous accidents and incidents. Lukic et al. [31] suggest that in order to increase the effectiveness of learning from incidents, it is necessary to understand who should be included in the learning process. In Figure 2, the incident lifecycle is aligned with the learning process in order to highlight where potential improvements might be made. As the incident is managed and causation is established, there are potential avenues Aerospace 2021, 8, 27 6 of 20 1.3. A Potential Learning Cycle Emerges According to Lindberg et al. [30], in order to prevent accidents, it is essential to learn from previous accidents and incidents. Lukic et al. [31] suggest that in order to increase the effectiveness of learning from incidents, it is necessary to understand who should be Aerospace 2021, 8, 27 6 of 20 included in the learning process. In Figure 2, the incident lifecycle is aligned with the learning process in order to highlight where potential improvements might be made. As the incident is managed and causation is established, there are potential avenues open for learning. The ultimate desired outcome is that adequate measures are put in place to pre- open for learning. The ultimate desired outcome is that adequate measures are put in vent a recurrence of the event. However, the lessons available in a potential learning prod- place to prevent a recurrence of the event. However, the lessons available in a potential uct are not always used to best effect when considering the Figure 1 process. Drupsteen learning product are not always used to best effect when considering the Figure 1 process. et al. [28] state that “many incidents occur because organizations fail to learn from past Drupsteen et al. [28] state that “many incidents occur because organizations fail to learn lessons”, because the traditional approach often stops short of preventing future incidents. from past lessons”, because the traditional approach often stops short of preventing future Their research examines: investigating and analyzing incidents, planning and prevention, incidents. Their research examines: investigating and analyzing incidents, planning and intervening and evaluating steps in a learning process. Ward et al. [15] found that the prevention, intervening and evaluating steps in a learning process. Ward et al. [15] found resulting relationship between the individuals and the systems have a direct impact upon that the resulting relationship between the individuals and the systems have a direct impact the system and prevailing environment. Silva et al. [3] examined how organizations use upon the system and prevailing environment. Silva et al. [3] examined how organizations accident information to reduce the occurrence of unwelcome events. Drupsteen and Wybo use accident information to reduce the occurrence of unwelcome events. Drupsteen and [10] found that hindsight can determine if an organization did learn from an event but Wybo [10] found that hindsight can determine if an organization did learn from an event there are no models to assist with gauging the “propensity” of an organization to learn. but there are no models to assist with gauging the “propensity” of an organization to Drupsteen and Hasle [1] suggest that learning can be improved if limiting factors are ad- learn. Drupsteen and Hasle [1] suggest that learning can be improved if limiting factors ar dresse e addr d. essed. Figure 2. Incident learning product and process (broken line denotes iterative learning feedback). Figure 2. Incident learning product and process (broken line denotes iterative learning feedback). The proposed enhancement (shown in Figure 2) to the generic lifecycle in the “tradi- The proposed enhancement (shown in Figure 2) to the generic lifecycle in the “tradi- tional” tional” appr appr oach oach rep reprr esents esentsa a nove novell vi view ewand and b brings rings the the learn learning ing p prr oduct oductinto intofocus. focus. Th This is figure highlights the benefits of ensuring the feedback loop of an incident is centered on the figure highlights the benefits of ensuring the feedback loop of an incident is centered on learning product. Treating its development as an iterative process ensures all steps in the the learning product. Treating its development as an iterative process ensures all steps in cycle are included and where deficiencies are noted, they can be identified and communi- the cycle are included and where deficiencies are noted, they can be identified and com- cated during the iterations. This can assist with delivery of timely and sustainable learning municated during the iterations. This can assist with delivery of timely and sustainable and help prevent an inability to think, talk and see what actions are proper in specific learning and help prevent an inability to think, talk and see what actions are proper in situations [32]. According to Drupsteen et al. [28], it is necessary to gain an insight into the specific situations [32]. According to Drupsteen et al. [28], it is necessary to gain an insight steps of the process to identify factors that hinder learning in order to make improvements. into the steps of the process to identify factors that hinder learning in order to make im- The research suggests an emphasis on accessible, timely and appropriate learning content provements. The research suggests an emphasis on accessible, timely and appropriate could provide all stakeholders in the process with better value for their efforts. Perhaps learning content could provide all stakeholders in the process with better value for their one reason that the customary incident lifecycle and its limitations prevail is related to efforts. Perhaps one reason that the customary incident lifecycle and its limitations prevail management theory. While innovators like Taylor [33] are responsible for advances in is related to management theory. While innovators like Taylor [33] are responsible for management, such theories have not always fully considered safety and learning. The advances in management, such theories have not always fully considered safety and early 1900s witnessed a time when it was necessary to inaugurate efficiencies in produc- learning. The early 1900s witnessed a time when it was necessary to inaugurate efficien- tion by initially decomposing tasks in order to introduce liner efficiencies. The limitations cies in production by initially decomposing tasks in order to introduce liner efficiencies. experienced in incident learning processes today may relate to this circumscribed tradition. The limitations experienced in incident learning processes today may relate to this cir- cumscribed tradition. 2. Materials and Methods 2.1. Philosophical Underpinnings The fields of science and philosophy consider ontology and epistemology in terms of What is the nature of reality? and How is human knowledge constructed? The ascendant ideologies of positivism and interpretivism can be applied in support of these philosophical differences [34–37]. Hirschheim [38] puts forward the aim of positivism to, “seek to explain and predict what happens in the social world by searching for irregularities and causal relationships between its constituent elements”. In contrast to this stance, Schwandt [39] Aerospace 2021, 8, 27 7 of 20 suggests the aim of interpretivism is to gain understanding. Interpretive research seeks to develop a richer understanding of the complex world of lived experience from the point of view of those who live in it. “This goal is variously spoken of as an abiding concern for the life world, for the emic point of view, for understanding meaning, for grasping the actor’s definition of a situation, for verstehen” [39] (p.118). The intent of this qualitative study was to understand how various situations impact on learning from incidents in the aircraft maintenance and continuing airworthiness man- agement domain by interacting with the participants on a social plane. Thus, in order to gain an empathetic understanding of the participants and their actions, the pursuit of “ver- stehen” considers adopting an interpretive paragon as an approach. This approach is not initiated with the aid of a hypothesis intended for testing but rather using a lodestar that guides the researcher to a point of discovery supported with an inductive modus operandi. The study is unwavering in its support for the view that (individual and combined) qual- itative and quantitative approaches possess equal value in terms of their investigative potential in this area of focus. In summary, the project employs a qualitative research methodology in an effort to generate “rich” findings in support of gaining a good un- derstanding of the learning environment in the featured domain. According to Maykut and Morehouse [26], the purpose of qualitative research is to discover the inner world of perceptions and meaning-making in order to gain an understanding to describe and explain certain social phenomenon from participants’ perspectives. In order to accomplish this, focus group activity was managed concurrently with the literature review. These activities cumulatively generated five themes which were used as the basis for a semi-structured interview template. The project employed a qualitative research methodology in an effort to generate “rich” findings in support of gaining a good understanding of the learning environment in the featured domain. The outcome of a qualitative research initiative was contextual findings as opposed to broad generalizations. 2.2. Focus Group According to Kitzinger [40], “focus groups are group discussions organized to explore a specific set of issues such as people’s views and experiences . . . ”. The idea of conducting group interviews is not a new one. Bogardus [41] is an early example of a reference to utilizing the group interview. Frey and Fontana [42] suggest that group interviews can be formally structured for a specific purpose or can be performed in a more informal setting where a researcher can “stimulate a group discussion”. Specific examples in the literature of focus groups being developed systematically within the area of aircraft maintenance and management are scarce. Frey and Fontana [42] state that although group interviews have implicitly informed research, often they are not formally acknowledged as part of the process. Powell and Single [43] remind us that when recruiting focus group participants, one must be mindful of systemic biases. Averting this was ensured by being careful to enlist the participants from different organizations and different positions of responsibility. Three sessions comprising of three industry professionals within each group were successfully moderated by the researcher. During the three phases of working with the focus group, statements and terms were recorded as dialogue amongst the members and observed. The second meeting of the focus group developed four codes (safety, regulatory compliance, root cause, reporting) that had emerged from the group’s earlier outputs. These four codes were further distilled during the focus group activity and were consolidated into two themes (reporting, root cause) that were to eventually form part of the piloted semi- structured interview instrument. Reporting and root-cause themes were the result of the draft consolidation of the comments and emerging codes. In concert with the focus group activities, a literature review was performed by the researchers and this generated three further themes as reflected in Table 2. Aerospace 2021, 8, 27 8 of 20 Table 2. Codification themes used in the NVivo analysis of the final set of publications. Codification Theme Description Origin Root Cause Reason to establish causation Focus Group Reporting Value of reporting to learning from incidents Focus Group Learning from Incidents Outcomes of learning from incidents Literature Analysis Just culture Impact of just culture on learning from incidents Literature Analysis Precursors Contribution of precursors to learning from incidents Literature Analysis The resulting draft semi-structured template (provided in Appendix A) containing the five themes was scrutinized by the focus group. The constituent questions relating to each theme and the running order of the document was subject to many minor changes during the individual piloting of the instrument with the three group members. 2.3. Data Collection Data were gathered from seven organizations using a semi-structured interview tem- plate. The participating organizations were involved in aircraft maintenance and continuing airworthiness activities. Building trust and commitment, as proffered by Chatzi [44] and Chatz et al., [45] was deemed to be a necessary tenet of a successful data collection exercise. Managing the interview process with the support of senior staff complimented visible top-down support for the research and ensured there would be no confusion regarding access to what some organizations often classify as sensitive commercial data. The aim was to explore how learning from incidents occurs and what can constrain learning in the area of focus. The pilot phase ensured the desired outcome of the main data collection phase would be congruent with the aims of the study. The interviews were recorded, transcribed and participants could not be identified from the recordings or transcripts. Full ethical approval for the data gathering was granted by the University. 2.3.1. Instrument Data were collected using semi-structured interviews, lasting on average sixty minutes. The “aide memoir” was arranged so that participants could offer a flexible response and any emerging themes could be identified. The semi-structured approach facilitated emphasis being placed upon any points that warranted further focus or examination by the researchers. An example of the interview template is included in Appendix A. Interviewees were asked to give an example of a recent incident they were familiar with. The structure of the template, (a) probed process around reporting and (b) elicited the participants perception of learning from incidents within their organizations. Following on from the initial contact on reporting, the participants discussed just culture, learning, root-cause and incident precursors during their individual engagements with the researchers. 2.3.2. Participants The “key issue in selecting and making decisions about the appropriate unit of analysis is to decide what it is you want to be able to say something about at the end of the study” [46] (p.168). The objective of this study was to investigate individuals’ perceptions of how learning from incidents takes place and the obstacles present in the maintenance and continuing airworthiness management domain of the aviation industry segment. There were thirty-four (34) participants in total, as presented in Table 3. Each of the organizations maintained between 6 and 300 aircraft at the time of the study. While traditional reporting and learning themes were evident outputs from the focus group meetings, it was decided that the data would be collected through one-to-one semi-structured interviews. Semi-structured interviews permitted the researchers to get a deeper understanding of complex organizational and social interactions and at the same time follow a construct. The participating organizations were selected based upon them being accredited to perform aircraft maintenance and continuing airworthiness activities since the inception of EU regulation 1321/2014 [6]. Within this domain, there are categories Aerospace 2021, 8, 27 9 of 20 of staff that are required to be aware of incident reporting and make a report as necessary (e.g., technical managers, certifying staff, quality assurance staff, stores personnel, technical services). Each organization is required under legislation to employ a satisfactory level of staff regardless of their aviation activities. As a minimum, at least one of each of these roles was represented in the study. It was ensured that at least one staff member from each discipline was included in the study and had made a report in the previous twelve months. As certifying staff, technical managers and quality assurance staff are by virtue of their position active reporters (due to their exposure to active operations), staff in these disciplines were well represented in the study’s cohort. Participation in the study was on a voluntary basis and all who participated were acquainted with the project prior to performing the interviews. All participants signed consent forms. Table 3. Participants in the study (n = 34). Participant Roles Number Category B1 Engineer 4 Supervisor 3 Category A Mechanic 3 Quality Assurance Engineer 3 Category B2 Engineer 2 Shift Controller 2 Contract Composite Inspector 1 Inspector 1 Aeronautical Engineer 1 Category B1/B2 Engineer 1 Maintenance Manager 1 Technical Safety Manager 1 Technical Services Manager 1 Line Maintenance Manager 1 Deputy Quality Manager 1 Maintenance Control Manager 1 Maintenance Planner 1 Maintenance Safety Officer 1 Apprentice Technician 1 2.4. Data Analysis Thematic analysis was the method chosen to support the analysis of the study’s data. The Braun and Clarke [47] six-step proposition, which consists of eight discreet cycles, in conjunction with the QDA Training [48] material, formed the basis of the analysis technique. A practical iterative approach was adopted throughout the analysis where the data were formally arranged into discrete phases. The eight individual stages of analysis distributed over the six phases were designed to support a robust and rigorous analysis of the data. Table 4 below illustrates the stages and processes outlined and performed in NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six-step approach that supports the application of thematic analysis is shown in column one and the corresponding application in NVivo is shown in column two. The third column features the strategic elements of coding as the researcher moved from the initial participant-led descriptive coding, to the secondary coding which was more interpretative in nature indicating this phase of coding was both researcher- and participant-led. The final abstraction to themes was researcher informed only. This phase was designed to allow the researchers to engage the participant in direct dialogue with a wider arena such as literature and policy or strategy for example. The fourth and final column illustrates the more iterative nature of the coding, analysis and reporting of proceedings that terminate in a conclusion. Aerospace 2021, 8, 27 10 of 20 the data. Table 4 below illustrates the stages and processes outlined and performed in NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- step approach that supports the application of thematic analysis is shown in column one Aerospace 2021, 8, 27 10 of 20 and the corresponding application in NVivo is shown in column two. The third column Aerospace 2021, 8, 27 10 of 20 features the strategic elements of coding as the researcher moved from the initial partici- pant-led descriptive coding, to the secondary coding which was more interpretative in nature indicating this phase of coding was both researcher- and participant-led. The final the data. Table 4 below illustrates the stages and processes outlined and performed in the data. Table 4 below illustrates the stages and processes outlined and performed in abstraction to themes was researcher informed only. This phase was designed to allow the NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- researchers to engage the participant in direct dialogue with a wider arena such as litera- step approach that supports the application of thematic analysis is shown in column one step approach that supports the application of thematic analysis is shown in column one ture and policy or strategy for example. The fourth and final column illustrates the more and the corresponding application in NVivo is shown in column two. The third column and the corresponding application in NVivo is shown in column two. The third column iterative nature of the coding, analysis and reporting of proceedings that terminate in a features the strategic elements of coding as the researcher moved from the initial partici- Aerospace 2021, 8, 27 10 of 20 features the strategic elements of coding as the researcher moved from the initial partici- Aerospace 2021, 8, 27 10 of 20 conclusion. pant-led descriptive coding, to the secondary coding which was more interpretative in pant-led descriptive coding, to the secondary coding which was more interpretative in nature indicating this phase of coding was both researcher- and participant-led. The final nature indicating this phase of coding was both researcher- and participant-led. The final Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING abstraction to themes was researcher informed only. This phase was designed to allow the abstraction to themes was researcher informed only. This phase was designed to allow the Training [48] material. the data. Table 4 below illustrates the stages and processes outlined and performed in researchers to engage the participant in direct dialogue with a wider arena such as litera- the data. Table 4 below illustrates the stages and processes outlined and performed in researchers to engage the participant in direct dialogue with a wider arena such as litera- NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- ture and policy or strategy for example. The fourth and final column illustrates the more NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- Analytical Process ture and policy or strategy for example. The fourth and final column illustrates the more step approac Practical ah p t ph lic atat suppor ion of Br ts taun he appl andica tion of t Stra h tegic Ob ematic an jec- alysis is shown in Iterative Proce column one ss Aerospace 2021, 8, 27 10 of 20 iterative nature of the coding, analysis and reporting of proceedings that terminate in a step approach that supports the application of thematic analysis is shown in column one (Braun and Clarke, 2006) iterative nature of the coding, analysis and reporting of proceedings that terminate in a and the corresponding application in NVivo is shown in column two. The third column Clarke in conjunction with NVivo tive Throughout Analysis conclusion. Aerospace 2021, 8, 27 and the corresponding application in NVivo is shown in column two. The third column 10 of 20 [47] conclusion. features the strategic elements of coding as the researcher moved from the initial partici- features the strategic elements of coding as the researcher moved from the initial partici- Transcribing data (if necessary), reading pant-led descriptive coding, to the secondary coding which was more interpretative in Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING pant-led descriptive coding, to the secondary coding which was more interpretative in Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING the data. Table 4 below illustrates the stages and processes outlined and performed in 1.Familiarizing yourself and re-reading the data, noting down Assigning data to re- nature indicating this phase of coding was both researcher- and participant-led. The final Training [48] material. Training [48] material. nature indicating this phase of coding was both researcher- and participant-led. The final NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- fined concepts to por- with the data initial ideas. Import data into the NVivo Data Management Table 4. Stages and Process abstraction Involved to in themes Qualitative was r Analysis. esearcher Adapted informed on from Braun ly. T and his phas Clarke e w [47 a]sand desQDA igned TRAINING to allow the abstraction to themes was researcher informed only. This phase was designed to allow the step approach that supports the application of thematic analysis is shown in column one Analytical Process data management tool tray meaning Analytical Process (Open and hierar- Training [48] material. researchers to engage the participant in direct dialogue with a wider arena such as litera- Practical application of Braun and Strategic Objec- Iterative Process researchers t Practic o engag al app elic that e p ion art of Br icipant aun in and direct dialo Str gue wit ategic Ob h a w jec- ider arenIt a s eu rat ch ive Proce as litera- ss and the corresponding application in NVivo is shown in column two. The third column (Braun and Clarke, 2006) Phase 2.Open Coding:Coding interest- (Braun and Clarke, 2006) ture and policy or strategy for example. The chal co fourtdin h an g thr d fiona ugh l column illustrates the more Clarke in conjunction with NVivo tive Throughout Analysis ture and policy or strategy for example. The fourth and final column illustrates the more features the strategic e Clarke in conjunction with NVi lements of cod vo ing as the rese tiv archer moved e Through from the init out Analys ial partic is i- Practical application of [47] iterative nature of the coding, analysis and reporting of proceedings that terminate in a 2. Generati[4 ng 7] init ial ing features of the data in a systematic NVIVO) Analytical Process (Braun Iterative Process iterative natuBraun re of tand he cod Clarke ing, in analysis and report Strategic ing Objective of proceedings that terminate in a pant-led descriptive coding, to the secondary coding which was more interpretative in Transcribing data (if necessary), reading and Clarke, 2006) [47] conclusion. Throughout Analysis codes fashion across the entire data set, col- Transcribing data (if necessary), reading Conjunction with NVivo conclusion. nature indicating this phase of coding was both researcher- and participant-led. The final 1.Familiarizing yourself and re-reading the data, noting down Assigning data to re- lecting data relevant to each code 1.Familiarizing yourself and re-reading the data, noting down Assigning data to re- Transcribing data (if abstraction to themes was researcher informed only. This phase was designed to allow the Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING fined concepts to por- with the data initial ideas. Import data into the NVivo Data Management with the data Phas initiae 3.C l idea as. I tegori mport zat dat ion of a in Cto odes:C the NV ollivo at- fined concepts to por- Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING necessary), reading and Data Management researchers to engage the participant in direct dialogue with a wider arena such as litera- Training [48] material. data management tool tray meaning (Open and hierar- Training [48] 1.Familiarizing material. yourself withing codes re-reading into potenti the data,a noting l themes, gather- Refining and distilling data management tool Data Management Assigning tray m data eanin to rg efined (Open and hierar- ture and policy or strategy for example. The fourth and final column illustrates the more 3. Searching for themes the data Phase 2.Open down Coding initial ideas. :Coding intere Import st- (Open chal co anddin hierar g thr chal ough coding concepts to portray meaning Analytical Process ing all data relevant to each potential more abstract concepts Phase 2.Open Coding:Coding interest- chal coding through iterative nature of the coding, analysis and reporting of proceedings that terminate in a Analytical Process Practical application of Braun and Strategic Objec- Iterative Process data into the NVivo data through NVIVO) 2. Generating initial ing features of the data in a systematic NVIVO) Practical application of Braun and Strategic Objec- Iterative Process (Braun and Clarke, 2006) theme 2. Generating initial ing features of the data in a systematic NVIVO) conclusion. (Braun and Clarke, 2006) Cmanagement larke in conj tool unction with NVivo tive Throughout Analysis codes fashion across the entire data set, col- Descriptive Ac- [47] Clarke in conjunction with NVivo tive Throughout Analysis Phase 4.Coding on:Checking if the codes fashion across the entire data set, col- [47] Phase 2. Open Coding:Coding counts lecting data relevant to each code Table 4. Stages and Process Transcr Involveid b in ing Qualitative data (if n Analysis. ecessary), Adapte readin d fro g m Braun and Clarke [47] and QDATRAINING themes work lecting data re in re levant lation to e to ach the coded ex code - interesting features of the data Transcribing data (if necessary), reading Training [48] material. (Reordering, ‘coding Phase 3.Categorization of Codes:Collat- Assigning data to re- 4. Rev 1.Fami ili ewin arizg themes ing yourself tra and re- cts (lr evel eadi 1) ng the da and the entire da ta, noting down ta set 2. Generating initial codes Phas in e 3.C a systematic ategorizat fashion ion of acr Coss odes:Collat- 1.Familiarizing yourself and re-reading the data, noting down Assigning data to re- on’ and annotating ing codes into potential themes, gather- Refining and distilling the entire data set, collecting with the data initial ideas. Import data into the NVivo fined concepts to por- (level 2), generating a thematic “map” of Data Management Assigning data to ing codes into potential themes, gather- Refining and distilling Refining and distilling more Analytical Process 3. Searching for themes with the data initial ideas. Import data into the NVivo fined concepts to por- Data Management 3. Searching for themes data relevant to each code Practical application of Braun and Str thro aug tegic Ob h NVIV jec- O) Iterative Process ing all data relevant to each potential more abstract concepts tray meaning the analy data mana sis gement tool themes/concepts to abstract concepts ing all data relevant to each potential (Open and hierar- more abstract concepts (Braun and Clarke, 2006) data management tool tray meaning (Open and hierar- Clarke in conjunction with NVivo Descriptive Accounts tive Throughout Analysis theme Phase 3. Categorization of Phas Phase e 5.D 2.Open ata Red Coding uction:O :Coding intere n-going ana st-ly- chal coding through portray meaning theme [47] Phase 2.Open Coding:Coding interest- chal coding through (Reordering, ‘coding on’ and Codes:Collating codes into Descriptive Ac- Phase 4.Coding on:Checking if the Descriptive Ac- 2. Generating initial ing features of the data in a systematic sis to refine the specifics of each theme, NVIVO) Phase 4.Coding on:Checking if the Transcribing data (if necessary), reading annotating through NVIVO) 2. Generat 3. Sear ing init ching ial for themes ing features o potential f the d themes, ata in gathering a systemall atic 5. Defining and naming NVIVO) counts themes work in relation to the coded ex- counts codes and t fashio hn e ove across the enti rall story (st re da oryline ta set, col s) the - themes work in relation to the coded ex- 1.Familiarizing yourself and re- data rea relevant ding the da to each ta potential , noting down Assigning data to re- codes fashion across the entire data set, col- themes (Reordering, ‘coding 4. Reviewing themes tracts (level 1) and the entire data set (Reordering, ‘coding lecting data relevant to each code analysis tells, theme generating clear defini- 4. Reviewing themes tracts (level 1) and the entire data set with the data initial ideas. Import data into the NVivo fined concepts to por- Data Management lecting data relevant to each code on’ and annotating (level 2), generating a thematic “map” of on’ and annotating Assigning data to tions Phase 3.C and n ata emes for gorization each t of C heme odes:Collat- (level 2), generating a thematic “map” of Phase 4.Coding on:Checking Assigning data to data management tool tray meaning Assigning data to (Open and hierar- Phase 3.Categorization of Codes:Collat- through NVIVO) the analysis themes/concepts to through NVIVO) if the themes work in relation Explanatory Ac- Phas ing codes e 6.Ge in ne to potenti rating An al themes, alytical Mga em ther- os. Refining and distilling the analysis themes/concepts themes/concepts to to portray Phase 2.Open Coding:Coding interest- chal coding through 3. Searching for themes ing codes into potential themes, gather- Refining and distilling to the coded extracts (level 1) Phase 5.Data Reduction:On-going analy- portray meaning 3. Searching for themes counts meaning ing all data relevant to each potential more abstract concepts Phase 7. —Testing and Validating.Phase 4. Reviewing themes Phase 5.Data Reduction:On-going analy- Ass portray me igning me an aning ing 2. Generating initial ing features of the data in a systematic NVIVO) ing all data relevant to each potential more abstract concepts and the entire data set (level sis to refine the specifics of each theme, (Extrapolating 8.Synthesizin theme g Analytical Memos. sis to refine the specifics of each theme, codes fashion across the entire data set, col- 5. Defining and naming 2), generating a thematic Explanatory Accounts theme 5. Defining and naming Descriptive Ac- and the overall story (storylines) the deeper meaning, The Phasfin e 4.C al opp oding on:C ortunity for a heckin ng i aly f t sh ise . Selec- and t “map” he ove ofrthe all st analysis ory (storylines) th (Extrapolating e deeper meaning, Descriptive Ac- lecting data relevant to each code themes Phase 4.Coding on:Checking if the themes counts analysis tells, generating clear defini- drafting summary drafting s statements ummary t themes work ion of vivid, compellin in relationg e to x the coded ex tract exam- - analysis tells, generating clear defini- Phase 5. Data counts Phase 3.Categorization of Codes:Collat- themes work in relation to the coded ex- 6. Producing the report and analytical (Reord memos ering, ‘cod throughing tions and names for each theme statements and ana- 4. Reviewing themes ples, tracts (l Reduction:On-going fina evel l ana 1)l ys anis o d the entire da f select analysis ed ext tar act set s, tions and names for each theme (Reordering, ‘coding ing codes into potential themes, gather- Refining and distilling 4. Reviewing themes tracts (level 1) and the entire data set NVIVO) 3. Searching for themes Explan on’ and anno atory Ac- tating Phase 6.Gto ene refine rating the An specifics alytica of l M each emos. lytical memos rela (levti elng b 2), gen ack eof rat the inga a n t al hysis to the re- ematic “map” of Assignin g data to Explanatory Ac- Phase 6.Generating Analytical Memos. Assigning meaning on’ and annotating 5. Defining and naming ing all data relevant to each potential more abstract concepts (level 2), generating a thematic “map” of Assigning data to theme, and the overall story counts throug h NVIVO) Phase 7. —Testing and Validating.Phase through NVIVO) Assigning meaning search the analy qusest is ion and literature, produc- counts themes/concepts to Generating themes themes Phase 7. —Testing and Validating.Phase through NVIVO) Assigning meaning theme the analys (storylines) is the analysis tells, themes/concepts to (Extrapolating 8.Synthesizing Analytical Memos. ing a scholarly Phase 5.Data Red report uction:O of thn e an -goin alys g ana is ly- portray me and concepts aning Descr (Extra ip polatin tive Ac- g 8.Synthesizing Analytical Memos. Phas generating e 4.Coding on:C clear definitions hecking if the portray meaning Phase 5.Data Reduction:On-going analy- deeper meaning, The final opportunity for analysis. Selec- [47,48] sis to refine the specifics of each theme, deeper meani counts ng, The and finanames l oppofor rtunit each y for a theme nalysis. Selec- themes work in relation to the coded ex- sis to refine the specifics of each theme, 5. Defining and naming drafting summary tion of vivid, compelling extract exam- 5. Defining and naming and the overall story (storylines) the (Reord drafting s ering, ‘cod ummary ing tion of vivid, compelling extract exam- Phase 6. Generating 4. Reviewing themes tracts (level 1) and the entire data set 6. Producing the report themes and the overall story (storylines) the 6. Producing the report statements and ana- ples, final analysis of selected extracts, themes analysis tells, Analytical Memos. generating clear defini- statements and ana- ples, final analysis of selected extracts, on’ and annotating (level 2), generating a thematic “map” of Assigning da ta to analysis tells, generating clear defini- Generating themes and lytical memos relating ba Phase ck of7. the —T aesting nalysis to the re- and tions and names for each theme thro lytical memos ugh NVIVO) relating back of the analysis to the re- the analysis concepts themes/concepts to tions and names for each theme Validating. through NVIVO) search question and literature, produc- Generating themes Explanatory Ac- Phase 6.Generating Analytical Memos. through NVIVO) search question and literature, produc- Generating themes Phase 5.Data Reduction:On-going analy- portray meaning Explanatory Ac- Phase 8. Synthesizing Phase 6.Generating Analytical Memos. ing a scholarly report of the analysis and concepts counts Phase 7. —Testing and Validating.Phase ing a scholarly report of the analysis Assand concepts igning meaning Analytical Memos. sis to refine the specifics of each theme, counts Phase 7. —Testing and Validating.Phase Assigning meaning 5. Defining and naming [47,48] (Extrapolating 8.Synthesizing Analytical Memos. [47,4 The 8] final opportunity for and the overall story (storylines) the (Extrapolating 8.Synthesizing Analytical Memos. themes 6. Producing the report analysis. Selection of vivid, deeper meaning, The final opportunity for analysis. Selec- analysis tells, generating clear defini- deeper meaning, The final opportunity for analysis. Selec- compelling extract examples, drafting summary tion of vivid, compelling extract exam- tions and names for each theme drafting summary final analysis of selected 6. Producing the report tion of vivid, compelling extract exam- 6. Producing the report statements and ana- ples, final analysis of selected extracts, extracts, relating back of the Explanatory Ac- Phase 6.Generating Analytical Memos. statements and ana- ples, final analysis of selected extracts, lytical memos relating back of the analysis to the re- analysis to the research counts Phase 7. —Testing and Validating.Phase lytical memos Assigning meaning relating back of the analysis to the re- question and literature, through NVIVO) search question and literature, produc- Generating themes (Extrapolating 8.Synthesizing Analytical Memos. through NVIVO) search question and literature, produc- Generating themes producing a scholarly report ing a scholarly report of the analysis and concepts deeper meaning, The final opportunity for analysis. Selec- ing a scholarly of the report analysis of [47 th ,48 e an ] alysis and concepts [47,48] drafting summary tion of vivid, compelling extract exam- [47,48] 6. Producing the report statements and ana- ples, final analysis of selected extracts, Phase 1 activity involves familiarizing oneself with the transcribed data. In this first lytical memos relating back of the analysis to the re- phase, the data were loaded into NVivo. It was checked and re-read several times to ensure through NVIVO) search question and literature, produc- Generating themes accuracy of the uploaded transcripts. At the end of the phase activity, initial codes were ing a scholarly report of the analysis and concepts noted down and retained. [47,48] Generating initial codes (open coding: phase 2)—According to Lincoln and Guba [49] (p. 345), a data unit can be defined as the “smallest piece of information about something that can stand by itself, that is, it must be interpretable in the absence of any additional Aerospace 2021, 8, 27 11 of 20 information other than a broad understanding of the context in which the inquiry is carried out”. The open coding is intended to systemically organize the data and uncover the essential ideas found in the data [50]. Each discrete unit of data is labelled in line with the phenomenon it represents. The second phase required broad participant-driven open coding of the interview transcripts recorded during the data gathering step of the research study. Features of interest were coded in a systematic way across the complete dataset where data relevant to each code were collected. Clear labels were allocated to these codes and definitions to serve as rules for inclusion [26]. A set of provisional categories was generated for the segmented data to be coded to. These categories were descriptions of concepts and themes in broad terms. They took two forms: researcher-driven and participant-driven. The former was derived from a theoretical framework underpinning the study and the latter from the knowledge gained of the participants’ language and customs. Hammersley and Atkinson [51] (p.153) consider the importance of participant-driven categories: “the actual words people use can be of considerable analytic importance as the ‘situated vocabularies’ employed provide valu- able information about the way in which members of a particular culture organize their perceptions of the world, and so engage in the social construction of reality”. Searching for themes—In phase 3, codes from phase 2 were collated into categories of codes by structuring all the data relevant to each potential category into a framework that could be used in support of further analysis. This phase also included distilling, re-labelling and merging common codes that were generated in phase 3 to ensure the labels and definitions for inclusion were an accurate reflection of the coded content. These first-round categories are best described as broad descriptions of concepts and themes. During the analytical process they underwent content and definition change and the existence of the two forms of category provides an important means of traversing between “natural” and “theoretical” discourses. Araujo [52] (p.68) suggests that “codes should be viewed in two ways: as part of the analyst’s wider theoretical framework and as grounded in the data.; the process of coding data should be regarded as an important intermediary step in translating social actors’ frames of meaning into the frame of theoretical discourse; coding frames therefore, mediate between the ‘natural’ everyday discourse and the theoretical discourses in social science”. Reviewing themes (coding on) in phase 4 required further decomposition of the study units of data identified in phase 1. This activity was intended to support a greater understanding of the highly qualitative elements and gain a deeper appreciation of the meanings contained within. It should be noted that not every task could be further broken down and this meant that the activity was performed only as required. Restructured codes were broken down into further sub-codes in order to augment a greater understanding of the meanings embedded within them. These distinctive aspects included communication with management, discovering latent issues, just culture, learning lessons, reporting, root causes and story of an incident. Defining and naming themes in phase 5 of the data analysis was concerned with ana- lyzing the tentative categories identified in phase 2 for their properties and characteristics. This is a pre-cursor to drafting a propositional statement for each category. Developing analytical memos moves the process beyond identification and description of broad cat- egories to a position of analyzing and fusing meanings in the data under each category. This progressed to drafting a statement that aspires to illustrate the concerted meaning of the segments of data coded to each category. Maykut and Morehouse [26] (p.140) defines a propositional statement as, “a statement of fact the researcher tentatively proposes, based on data”. This phase in addition to further data analysis to refine the specifics for each theme, generated clear definitions and a name for each theme. It also involved data reduc- tion by consolidating categories from all three cycles into a more abstract, philosophical and literature-based thematic framework and conceptually mapping and exploring their relationships with one another for reporting purposes. Aerospace 2021, 8, 27 12 of 20 Producing the report in phase 6 required analytical memos to be written against the higher-level themes to present an accurate summary of the content of each category and its codes and to also propose findings. The tasks associated with phase 6 included (i) generating analytical memos, (ii) testing and validating and (iii) synthesizing the memos coherently and cohesively, and were performed simultaneously. Writing the analytical memos against the higher-level codes (i.e., learning from incidents, learning process and learning product) required an accurate summary of each category and its codes and findings against categories. These memos considered a few key areas: 1. The content of the cluster of codes which were being reported on. 2. Patterns where relevant. 3. Considering background information noted against participants and examining any patterns relating to participants’ profiles. 4. Considering any relationship between codes and their importance in relation to the research questions. 5. Noting any primary sources relating to the context of the relationship with the litera- ture in addition to highlighting any gaps in the literature. Testing, validating and revising analytical memos was performed in phase 7. The purpose of this was to provide a self-audit of the proposed findings by soliciting evidence in the data beyond just textual quotes in support of the recorded findings and to also expand on deeper meanings within the data. This required the data to be interrogated, not only relying on relationships across and between categories, but also a degree of cross tabulation with demographics, observations and the literature. The outcome of this phase was evidence-based findings as each proposed finding was validated by being rooted in the data themselves and was reliant on the creation of reports in support of substantiated findings. The discipline of writing analytical memos was used during the data analysis process. Birks et al. [53] believe “memoing serves to assist the researcher in making conceptual leaps from raw data to those abstractions that explain research phenomena in the context of which it is examined”. In general, memos were employed at the “ideation” stage when the researcher was developing thought processes and early in the data capture phase. As decisions were made, the early processes and rationale for final analysis iterations were recorded using this medium. Memos were further employed to preserve an objective close- ness to the harvested data and to maintain the context of each semi-structured interview at the participating individuals’ level. Developing ideas, reasons for considering possible category relationships and connections was also possible through the application of the analytical memo process. The rigorous support memoing offered served to guide the anal- ysis of the data through different levels of abstraction [54]. The rule of this activity served to ensure a high degree of continuity between the outputs of ideation and the evolving interpretation that were honed through the researchers’ articulation, exploration and their iterations of the data. Overall, this drew out the meanings in the data through the increased sensitivity the researchers were offered by applying the memoing process [53]. In phase 8, the analytical memos were synthesized into a coherent and cohesive report with the findings well supported. The final phase involved the assembly of the narrative with the data extracts while appreciating the product of this amalgam in the context of the related literature. The example features the finding, clear links to the interview data and literature and an explanatory narrative in the form of a memo. This finally resulted in the compilation of the report which contained the results and discussion elements of the body of work. In summary, this study adopted an interpretative approach pivoting on the fact that it was of an exploratory nature. The study performed thirty-four interviews in eight aircraft maintenance and management organizations based in Ireland. An analysis of various potential research methods and means of data collection resulted in the following research design being implemented. A thematic analysis approach was employed as a research methodology: Aerospace 2021, 8, 27 13 of 20 In summary, this study adopted an interpretative approach pivoting on the fact that it was of an exploratory nature. The study performed thirty-four interviews in eight air- craft maintenance and management organizations based in Ireland. An analysis of various potential research methods and means of data collection resulted in the following research design being implemented. A thematic analysis approach was employed as a research Aerospace 2021, 8, 27 13 of 20 methodology: • Unit of analysis is an individual; • Semi-structured interview guide was constructed following a systematic analysis of Unit of analysis is an individual; literature and the use of a focus group; Semi-structured interview guide was constructed following a systematic analysis of literature and the use of a focus group; • Data were collected through qualitative interviews; Data were collected through qualitative interviews; • Thirty-four interviews were collected in locations endorsed by eight organizations; Thirty-four interviews were collected in locations endorsed by eight organizations; • Qualitative analysis based on the guidelines from Braun and Clarke [47] (thematic Qualitative analysis based on the guidelines from Braun and Clarke [47] (thematic analysis) employing a six-phase approach was used in the study. analysis) employing a six-phase approach was used in the study. 3. Results and Discussion 3. Results and Discussion 3.1. 3.1. Frame Framework work Figure 3 presents a framework that offers an insight into how the present study applied Figure 3 presents a framework that offers an insight into how the present study ap- the research inputs and produced the results. plied the research inputs and produced the results. Figure 3. Research study framework. Figure 3. Research study framework. The top layer reflects the five themes that formed the basis for the data gathering The top layer reflects the five themes that formed the basis for the data gathering template. These themes were developed through an iterative process of conducting focus template. These themes were developed through an iterative process of conducting focus group sessions with two themes emerging, i.e., root cause and reporting. Concurrently, a group sessions with two themes emerging, i.e., root cause and reporting. Concurrently, a systematic literature review was performed using NVivo software to assist the researchers systematic literature review was performed using NVivo software to assist the researchers manage over 1000 screened publications. Following a thematic analysis of the data, three manage over 1000 screened publications. Following a thematic analysis of the data, three main themes (Appendix B) emerged from a final cache of 18 publications, i.e., learning main themes (Appendix B) emerged from a final cache of 18 publications, i.e., learning from incidents, precursors and just culture. The five themes informed the structure of a data from gathering incidents, pr instrument ecursors that and supported just cu 34 lture. semi-str The uctur five edthemes in interviewsfor inm the ed the continuing structure of a airworthiness segment of the industry. Following transcription, the data were uploaded to data gathering instrument that supported 34 semi-structured interviews in the continuing NVivo where they were thematically analyzed using the Braun and Clarke [47] framework. airworthiness segment of the industry. Following transcription, the data were uploaded The outputs from the thematic analysis distilled the interview analysis into three main to NVivo where they were thematically analyzed using the Braun and Clarke [47] frame- outputs, i.e., learning from incidents, learning process and learning product. The lower work. The outputs from the thematic analysis distilled the interview analysis into three tier represents the elements the themes were comprised of and the findings are presented main outputs, i.e., learning from incidents, learning process and learning product. The under these headings (Table 5). lower tier represents the elements the themes were comprised of and the findings are pre- sented under these headings (Table 5). Aerospace 2021, 8, 27 14 of 20 Table 5. Summary of results. Learning from incidents (LFI) is a safety management activity with a desired outcome of preventing unwelcome event recurrence. A learning process facilitates a change in knowledge and behavior intended to support LFI. Safety related information arising from the LFI process. 1 2 3 Learning from Incidents Learning Process Learning Product The decision to report an incident can be impacted by the perceived commercial The release of a safe aviation product is In the organizations supporting the study, pressure and the potential for the primary goal all operational it was apparent that incidents are embarrassment associated with making a maintenance and management staff managed with the support of a consistent mistake, amongst front line espouse to. life-cycle methodology. maintenance staff. Learning products that arise from the Identifying and understanding Single-loop learning is a level of learning managed lifecycle of an incident are organizational behavioral and human that can exist in a dynamic operational intended to impart sufficient learning to factors are important elements affecting environment where a “find and fix” ethos prevent recurrence or occurrence of same decisions to report. exists. or similar events. While aircraft manufacturers generally Inadequately resourced investigation and The mandatory human factors provide feedback on notified incidents, follow up of incidents does not support continuation training program is component manufacturers provide less the determination of accurate event considered by study participants to be an feedback with little or no feedback arising causation and measures to prevent effective enabler of double-loop learning. from aviation authorities on submitted similar incidents reoccurring. reports in the jurisdiction of the study. The recognition of the extended impact of Evidence amongst study participants The cost of classroom delivered under-reporting on “levels of learning” is where a review of single and double-loop continuation training is a primary not always a priority in some learning within organizations was not consideration for most organizations. organizations. available during the study. The absence of a potential learning Computer based training is an option No formal requirement for competence in product that results from effective that is under trial by some organizations the areas of learning for managers and reporting is an impediment when but there are concerns amongst accountable persons exists in EU attempting to gauge the effectiveness of operational staff regarding its overall regulation 1321/2014. learning. effectiveness in its current form. Pressure to prematurely close incident No competence requirements for staff reports does not promote thorough event involved in the development or delivery Just culture has a positive impact on causation and measures to prevent of formal human factors continuation reporting rates. similar incidents reoccurring. training programs. Feedback to staff on incident causation factors from an information and learning perspective is important. Poorly designed continuation training syllabi do not support effective learning. Timely follow up to incident reports supports more effective learning outputs from the reporting process. 3.1.1. Learning form incidents—Acquiring, Processing and Storing Data Incident reporting is accepted as a worthwhile activity amongst those participating in the study. This is based on the collective notion that the initiative raises awareness of incidents and potential hazards and can therefore help prevent event recurrence. The authors recognize that awareness is an important component of learning from incidents. Situations do arise where due to lack of report data, it is questionable if all the necessary reports are being submitted as required. Amongst the constraints to making a report are perceived production pressures and the potential embarrassment that could arise from making a mistake and highlighting it [5]. There are just culture concerns amongst some staff because they do not always know what the impact for them personally will be if they submit an incident report [44]. Aerospace 2021, 8, 27 15 of 20 A dedicated focal point in organizations is essential for the systematic management of reported incidents. Where this discipline is applied, the process owner is responsible for highlighting reported issues and raising the necessary awareness amongst operational staff. Once an incident is acquired through the efforts of a reporting system, some form of processing and analysis is necessary. The availability of adequate resources for determining causation and implementing measures to prevent recurrence was identified as a primary point of concern. Perceived premature closure of reports was also highlighted amongst participants. There was a call for improved accountability and transparency on decisions relating to some closure actions. Respondents associate the practice of applying commercial key performance indicators to safety management as shallow efforts are sometimes made by organizations to expeditiously and prematurely close reports on occasion. Incident reporting and safety management initiatives have been in existence for some time. Large repositories of associated safety data are stored in many organizations. Although entities are mandated to inform key stakeholders, there is a strong opinion amongst some partic- ipants that the data repositories could be aggregated and put to better use in support of learning amongst all operators. 3.1.2. Learning Process—Single-Loop, Double-Loop and Deutero Learning The interview data confirms that safety is a primary underpinning value in the or- ganizations that participated in the study. The release of a safe product, i.e., an aircraft or component, is a formative pursuit and measure of learning. In organizational environ- ments where a “find and fix” ethos may prevail, single-loop learning [11] is evident in the examples presented. A desired outcome of double-loop learning [11] is often witnessed for example through the adjustment of environmental, behavioral and procedural norms. Instances of double- loop learning can be evident following unsuccessful attempts through single-loop learning where causation is then adequately understood and actioned. Continuation (mandatory in-service) training was considered by study participants as an effective mechanism that enables double-loop learning. During the study, it was apparent that single and double learning loops are recognized amongst many participants as having differing capabilities in terms of delivering an effective learning product. However, there was no evidence of formal reviews of single and double-loop learning being performed within the participants’ organizations. Although deutero-learning [11,55] may be considered as a natural extension of other levels of learning, the concept did not feature strongly amongst the participants. A review of the EU1321/2014 [6] implementing requirements confirms an absence of any mandatory requirement to review learning processes. 3.1.3. Learning Product—Effectiveness and Types of Knowledge Continuation training is a mandated European requirement [6] for all aircraft main- tenance and continuing airworthiness management organizations. It is a product as well as a medium for imparting learning from incidents and safety related hazards. It was identified during the study that the learning product is shared amongst staff through three primary means of distribution: formally delivered continuation training, tool-box talks and safety briefings and electronic, paper, notice board and “read and sign” safety publications. The study suggests a learning product can arise as a result of an output from an incident lifecycle. Feedback from submitted occurrences to stakeholders varies from very good to poor. Cost is seen as a major consideration in some of the participating organizations when planning continuation training delivery. Although computer-based training is being considered in some companies as a viable option to class-room delivery, concerns are evident in respect of effectiveness of this medium in its current form. Bedwell and Salas [56] suggest computer-based training (CBT) can be used as a methodology for providing, “systematic, structured learning; a useful tool when properly designed”. The perceived overburdening of operational staff with complex learning products and excessive cognitive loads was recorded as an impediment to learning during the study. Aerospace 2021, 8, 27 16 of 20 Participants suggested this can arise from poorly designed training syllabi delivered during periods of high operational activity. Four knowledge types were identified and relate to: conceptual, dispositional, pro- cedural and locative knowledge forms [57]. One of the key objectives of learning from incidents is to identify the type of knowledge needed to prevent an issue recurring. When a reportable issue, for example, is discovered, the submitted report will identify “what” happened. Subsequent follow up will set out to determine “why” the issue occurred. The guiding principles of “how” to perform the task or operation are often contained in proce- dures or data particular to the task. The information contained in procedures will enable a person to utilize other forms of knowledge. Prevailing safety culture within an organiza- tion will have an impact on learning from incidents. If a strong commercial/production culture exists, this may have an impact on, for example, the depth and breadth of learning from incidents within the company. Induction and initial training are important when accessing information for new staff. Accident data repositories contain well-documented human factor-related examples often relating to access to approved data and consequently resulting in potentially preventable incidents. Examining the limitations of each type of knowledge when continuation training programs are being developed was flagged as important by some participants. During the study, no discernible differences were recorded in how the types of knowledge were differentiated in participant organizations. A review of the EU 1321/2014 [6] human factors syllabus requirements did not highlight a need to appreciate or account for these human centered limitations when designing and delivering training lessons. Improved regulatory guidance on the design of effective human factor related material should therefore be developed. Information on how training should be structured in order to appreciate types of knowledge and capitalize on it as a minimum are required to ensure the most efficacious outcome from incident-related training. 4. Conclusions An ameliorating feature of learning from incidents is the potential to effect sustainable improvements in aviation safety. A review of safety from the perspective of maintenance and continuing airworthiness staff is key to understanding the relationship between safety and the concept of learning from incidents [31]. From the study’s qualitative data, we were able to identify how learning occurs in the airworthiness segment, and issues that sup- port and constrain learning. Recurrent mandated training initiatives such as continuation training were found to be pivotal in enabling learning. Aspects such as prevailing culture and poor event causation were noted to have a negative impact on learning. Our pro- posed incident learning process (Figure 2) offers a panoramic of where potential learning opportunities and procedural improvements can arise within the lifecycle of an incident. This perspective could be applied in support of developing regulatory working group specifications and validating continuation training initiatives. In addition, it could also be used to develop a holistic review approach to learning from incidents within other organizations both in the aviation industry and outside. Two notable limitations to our research arise. First, the scarcity of prior studies capable of supporting the basis for the research was pronounced. However, prior studies in parallel domains were successfully leveraged in support of the literature review. Second, the study’s population (n = 34) was relatively small. As the study participants were representative of all affected domain functions and a point of saturation was reached, it was deemed adequate. This research is capable of supporting other papers on additional benefits associated with learning from incidents (LFI). Notably, with the imminent implementation of a safety management (SMS) requirement for continuing airworthiness organizations, potential improvements to hazard identification arising from learning from incidents (LFI) could be highlighted. Author Contributions: Conceptualization, J.C. and K.I.K.; methodology, J.C.; formal analysis, J.C.; investigation, J.C.; validation, J.C and K.I.K.; data curation, J.C.; writing—original draft preparation, Aerospace 2021, 8, 27 17 of 20 J.C. and K.I.K.; writing—review and editing, J.C. and K.I.K. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Institutional Review Board Statement: The study was conducted according to the Institutional guidelines and approved by the Institutional Ethics Committee of University of Limerick (Research Ethics Approval Number: 2015_12_06) dated 6 December 2015. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study. Data Availability Statement: The data presented in this study are available on request from the corresponding author. Conflicts of Interest: The authors declare no conflict of interest. Appendix A Semi-Structured Interview Template Table A1. The Semi-Structured Interview Template used in this study Code 1 Code 2 Previous Positions Years in Previous Positions Position Years in position Qualification Type of organization a. Reporting Could you describe an occurrence/incident that happened recently? How is a report made? Who decides what events to report? Where does the requirement to report come from? How is the importance of reporting highlighted in the organization? What do you think the aim of reporting is? Have you received feedback from reports you have submitted? b. Just culture Do you think there is a good safety culture in the organization? Why is this? Is it easy to communicate with management on safety issues? Do you feel a just culture exists in the company? (Why is that?) How does just culture impact on reporting? c. Learning How are lessons that arise from occurrence/incident reporting delivered to staff in your area? How is learning achieved? (What is the process?) What obstacles to learning from incidents have you experienced in your position? In your opinion, what conditions or developments could improve learning from incidents/occurrences in your organization? d. Root Cause What is your opinion on efforts to establish a single root cause when an incident/occurrence is investigated? Is this approach always effective? What situations have you experienced where incident causes can be numerous and complex? e. Occurrence/Incident Pre-cursors How important is it to identify and report events not required by the mandatory occurrence reporting (MOR) schemes? (Why is this?) Is the organization’s occurrence/ incident reporting system capable of managing reports other than MOR’s? Aerospace 2021, 8, 27 18 of 20 Is there a better way of gathering and using the potential information from non- mandatory events? (What would you suggest?) Appendix B Defining and Naming Themes Table 2. Taxonomy used in defining and naming the themes used in this study. Phase 5—Categories Conceptually Interviews Units of Mapped and Collapsed into 3 Major Code Definitions for Coding Consistency Coded Meaning Coded Themes with 8 Sub-Themes This relates to the three levels of learning suggested LEARNING PROCESS by Bateson (1972) and applied by Argyris and 34 359 Schon (1996) This relates to when members of an organization Deutero-learning reflect on previous learning and thereby setting 26 65 about to improve its learning process. This relates to learning that takes place and Double-loop Learning 26 63 organizational norms and theory in use are changed. This relates to when an organizations’ members Single-loop Learning detect and correct errors but still maintain the 26 63 organizations theory in use. LEARNING PRODUCT This relates to what the learning process delivers 33 235 Effectiveness This relates to measuring effectiveness of learning 31 155 This relates to conceptual, procedural, dispositional Types of knowledge 23 74 and locative knowledge This relates to the inputs necessary to enable the LEARNING FROM INCIDENTS assembly of a learning material in support of 17 213 learning from incidents Processing This relates to how learning information is processed 17 82 This relates to the sources of information that Acquiring 16 55 support learning and how there are gathered Storing This relates to how learning information is stored 12 27 References 1. Drupsteen, L.; Hasle, P. Why do organizations not learn from incidents? Bottlenecks, causes and conditions for a failure to effectively learn. Accid. Anal. Prev. 2014, 72, 351–358. [CrossRef] [PubMed] 2. Chang, Y.-H.; Wang, Y.-C. Significant human risk factors in aircraft maintenance technicians. Saf. Sci. 2010, 48, 54–62. [CrossRef] 3. Silva, S.A.; Carvalho, H.; Oliveira, M.J.; Fialho, T.; Soares, C.G.; Jacinto, C. Organizational practices for learning with work accidents throughout their information cycle. Saf. Sci. 2017, 99, 102–114. [CrossRef] 4. Akselsson, R.; Jacobsson, A.; Bötjesson, M.; Ek, Å.; Enander, A. Efficient and effective learning for safety from incidents. Work 2012, 41, 3216–3222. [CrossRef] 5. Hobbs, A.N. Human Errors in Context: A Study of Unsafe Acts in Aircraft Maintenance. Ph.D. Thesis, University of New South Wales, Kensington, Australia, 2003. 6. European Commission (EU). Commission Regulation (EU) No 1321/2014 of 26 November 2014 on the Continuing Airworthiness of Aircraft and Aeronautical Products, Parts and Appliances, and on the Approval of Organisations and Personnel Involved in These Tasks; European Commission (EU): Brussels, Belgium, 2014. 7. Harvey, C.; Stanton, N.A. Safety in system-of-systems: Ten key challenges. Saf. Sci. 2014, 70, 358–366. [CrossRef] 8. European Commission (EU). Commission Regulation (EU) No 376/2014 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 3 April 2014 on the reporting, analysis and follow-up of occurrences in civil aviation, amending Regulation (EU) No 996/2010 of the European Parliament and of the Council and repealing Directive 2003/42/EC of the European Parliament and of the Council and Commission Regulations (EC) No 1321/2007 and (EC) No 1330/2007; European Commission (EU): Brussels, Belgium, 2014. 9. Gerede, E. A study of challenges to the success of the safety management system in aircraft maintenance organizations in Turkey. Saf. Sci. 2015, 73, 106–116. [CrossRef] 10. Drupsteen, L.; Wybo, J.-L. Assessing propensity to learn from safety-related events. Saf. Sci. 2015, 71, 28–38. [CrossRef] Aerospace 2021, 8, 27 19 of 20 11. Argyris, C.; Schön, D.A. Organizational Learning II: Theory, Method, and Practice; Addison-Wesley Publishing Company: Boston, MA, USA, 1996. 12. Drupsteen, L.; Guldenmund, F.W. What is learning? A review of the safety literature to define learning from incidents, accidents and disasters. J. Contingencies Crisis Manag. 2014, 22, 81–96. [CrossRef] 13. Fogarty, G.J.; Saunders, R.; Collyer, R. Developing a model to predict aircraft maintenance performance. In Proceedings of the 10th International Symposium on Aviation Psychology, Ohio State University, Columbus, OH, USA, 3–6 May 1999. 14. Sms, I. Safety Management Manual (SMM). In Doc 9859; ICAO: Montréal, QC, Canada, 2012. 15. Ward, M.; McDonald, N.; Morrison, R.; Gaynor, D.; Nugent, T. A performance improvement case study in aircraft maintenance and its implications for hazard identification. Ergonomics 2010, 53, 247–267. [CrossRef] 16. Jacobsson, A.; Ek, Å.; Akselsson, R. Learning from incidents–A method for assessing the effectiveness of the learning cycle. J. Loss Prev. Process Ind. 2012, 25, 561–570. [CrossRef] 17. Brunel, I. Sessional Papers Printed by Order of the House of Lords, or Presented by Royal Command; Government of Great Britain: London, UK, 1841. 18. Furniss, D.; Curzon, P.; Blandford, A. Using FRAM beyond safety: A case study to explore how sociotechnical systems can flourish or stall. Theor. Issues Ergon. Sci. 2016, 17, 507–532. [CrossRef] 19. Hollnagel, E. Barriers and Accident Prevention; Ashgate Publishing: Farnham, UK, 2004. 20. Okoli, C.; Schabram, K. A Guide to Conducting a Systematic Literature Review of Information Systems Research; Sprouts Farmers Market: Chandler, AZ, USA, 2010. 21. Houghton, C.; Murphy, K.; Meehan, B.; Thomas, J.; Brooker, D.; Casey, D. From screening to synthesis: Using nvivo to enhance transparency in qualitative evidence synthesis. J. Clin. Nurs. 2017, 26, 873–881. [CrossRef] [PubMed] 22. Bandara, W.; Furtmueller, E.; Gorbacheva, E.; Miskon, S.; Beekhuyzen, J. Achieving rigor in literature reviews: Insights from qualitative data analysis and tool-support. Commun. Assoc. Inf. Syst. 2015, 37, 154–204. [CrossRef] 23. Gough, D.; Oliver, S.; Thomas, J. An Introduction to Systematic Reviews, 2nd ed.; Sage: London, UK, 2017; pp. 83–106. 24. Meline, T. Selecting studies for systematic review: Inclusion and exclusion criteria. Contemp. Issues Commun. Sci. Disord. ASHA 2006, 33, 21–27. [CrossRef] 25. Wienen, H.C.A.; Bukhsh, F.A.; Vriezekolk, E.; Wieringa, R.J. Accident Analysis Methods and Models—A Systematic Literature Review; University of Twente, Centre for Telematics and Information Technology (CTIT): Enschede, The Netherlands, 2017. 26. Maykut, P.S.; Morehouse, R. Beginning Qualitative Research: A Philosophic and Practical Guide; Falmer Press: London, UK, 1994. 27. Cooke, D.L.; Rohleder, T.R. Learning from incidents: From normal accidents to high reliability. Syst. Dyn. Rev. 2006, 22, 213–239. [CrossRef] 28. Drupsteen, L.; Groeneweg, J.; Zwetsloot, G.I. Critical steps in learning from incidents: Using learning potential in the process from reporting an incident to accident prevention. Int. J. Occup. Saf. Ergon. 2013, 19, 63–77. [CrossRef] [PubMed] 29. European Commission (EU). (EU) 2018/1139 of the European Parliment and the Council of 4 July 2018 on common rules in trhe field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, EU376/2014 and Directives 2014/30/EU of the European Parliment and the Council, and repealing Regulation (EC) No 216/2008 of the European Parliment and of the Council and Council Regulation (EEC) No 3922/91; European Commission (EU): Brussels, Belgium, 2018. 30. Lindberg, A.-K.; Hansson, S.O.; Rollenhagen, C. Learning from accidents–What more do we need to know? Saf. Sci. 2010, 48, 714–721. [CrossRef] 31. Lukic, D.; Littlejohn, A.; Margaryan, A. A framework for learning from incidents in the workplace. Saf. Sci. 2012, 50, 950–957. [CrossRef] 32. Steiner, L. Organizational dilemmas as barriers to learning. Learn. Organ. 1998, 5, 193–201. [CrossRef] 33. Taylor, F.W. The Principles of Scientific Management; Harper & Brothers: New York, NY, USA, 1911. 34. Weber, R. Editor ’s Comments: The rhetoric of positivism versus interpretivism: A personal view. MIS Q. 2004, 28. [CrossRef] 35. Guba, E.G.; Lincoln, Y.S. Competing Paradigms in Qualitative Research; Sage Publications: Thousand Oaks, CA, USA, 1994. 36. Walsham, G. Interpretive case studies in IS research: Nature and method. Eur. J. Inf. Syst. 1995, 4, 74. [CrossRef] 37. Oates, B.J. Researching Information Systems and Computing; Sage Publications: Thousand Oaks, CA, USA, 2006. 38. Hirschheim, R. Information systems epistemology: An historical perspective. Res. Methods Inf. Syst. 1985, 9, 13–35. 39. Schwandt, T. Constructivist, interpretivist approaches to human inquiry. Handb. Qual. Res. 1994, 1, 118–137. 40. Kitzinger, J. The methodology of focus groups: The importance of interaction between research participants. Sociol. Health Illn. 1994, 16, 103–121. [CrossRef] 41. Bogardus, E.S. Social distance in the city. Proc. Publ. Am. Sociol. Soc. 1926, 20, 40–46. 42. Frey, J.H.; Fontana, A. The group interview in social research. Soc. Sci. J. 1991, 28, 175–187. [CrossRef] 43. Powell, R.A.; Single, H.M. Focus groups. Int. J. Qual. Health Care 1996, 8, 499–504. [CrossRef] 44. Chatzi, A.V. The diagnosis of communication and trust in aviation maintenance (DiCTAM) model. Aerospace 2019, 6, 120. [CrossRef] 45. Chatzi, A.V.; Martin, W.; Bates, P.; Murray, P. The unexplored link between communication and trust in aviation maintenance practice. Aerospace 2019, 6, 66. [CrossRef] Aerospace 2021, 8, 27 20 of 20 46. Patton, M.Q. Qualitative Evaluation and Research Methods, 2nd ed.; Patton, M.Q., Ed.; Sage Publications: Thousand Oaks, CA, USA, 1990. 47. Braun, V.; Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006, 3, 77–101. [CrossRef] 48. QDA Training. Working with NVivo; QDA Training: Dublin, Ireland, 2013. 49. Lincoln, Y.S.; Guba, E.G. Naturalistic Inquiry; Sage Publications: Thousand Oaks, CA, USA, 1985. 50. Baskerville, R.; Pries, J.H. Short cycle time systems development. Inf. Syst. J. 2004, 14, 237–264. [CrossRef] 51. Hammersley, M.; Atkinson, P. Ethnography: Principles in Practice, 3rd ed.; Routledge: London, UK, 2007. 52. Araujo, L. Designing and Refining Hierarchical Coding Frames. In Computer-Aided Qualitative Data Analysis: Theory, Methods and Practice; Udo, K., Ed.; Sage Publications: London, UK, 1995; pp. 96–104. 53. Birks, M.; Chapman, Y.; Francis, K. Memoing in qualitative research: Probing data and processes. J. Res. Nurs. 2008, 13, 68–75. [CrossRef] 54. Miles, M.B.; Huberman, A.M. Qualitative Data Analysis: An Expanded Sourcebook; Sage Publications: Thousand Oaks, CA, USA, 1994. 55. Bateson, G. The Logical Categories of Learning and Communication. In Steps to an Ecology of Mind; The University of Chicago Press: Chicago, IL, USA, 1972; pp. 279–308. 56. Bedwell, W.L.; Salas, E. Computer-based training: Capitalizing on lessons learned. Int. J. Train. Dev. 2010, 14, 239–249. [CrossRef] 57. Thorndike, E.L. Fundamental theorems in judging men. J. Appl. Psychol. 1918, 2, 67. [CrossRef] http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Aerospace Multidisciplinary Digital Publishing Institute

Learning from Incidents: A Qualitative Study in the Continuing Airworthiness Sector

Aerospace , Volume 8 (2) – Jan 22, 2021

Loading next page...
 
/lp/multidisciplinary-digital-publishing-institute/learning-from-incidents-a-qualitative-study-in-the-continuing-vdaaYu1cGu
Publisher
Multidisciplinary Digital Publishing Institute
Copyright
© 1996-2021 MDPI (Basel, Switzerland) unless otherwise stated Disclaimer The statements, opinions and data contained in the journals are solely those of the individual authors and contributors and not of the publisher and the editor(s). MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Terms and Conditions Privacy Policy
ISSN
2226-4310
DOI
10.3390/aerospace8020027
Publisher site
See Article on Publisher Site

Abstract

aerospace Article Learning from Incidents: A Qualitative Study in the Continuing Airworthiness Sector 1 1 , 2 , James Clare and Kyriakos I. Kourousis * School of Engineering, University of Limerick, V94 T9PX Limerick, Ireland; james.clare@ul.ie School of Engineering, RMIT University, Melbourne, VIC 3001, Australia * Correspondence: kyriakos.kourousis@ul.ie Abstract: Learning from incidents (LFI) is a useful approach when examining past events and developing measures to prevent ensuing recurrence. Although the reporting of incidents in the aircraft maintenance and continuing airworthiness domain is well appointed, it is often unclear how the maximum effect of safety data can be efficaciously applied in support of LFI in the area. From semi-structured interviews, with thirty-four participants, the gathered data were thematically analyzed with the support of NVivo software. This study establishes a relationship between an incident in its lifecycle and the learning process. The main aim of this work is to elucidate factors that enable LFI. The analysis of the data revealed, for example, the benefits of a just culture and the use of formal continuation training programs in this respect. Moreover, it identified limitations inherent in current processes such as poor event causation and poorly designed learning syllabi. Additionally, aspects such as a lack of regulatory requirements for competence in the areas of learning for managers and accountable persons currently exist. This thematic analysis could be used in support of organizations examining their own processes for learning from incidents. Additionally, it can support the development of terms of reference for a continuing airworthiness regulatory working group to examine, strengthen and better apply LFI in the aviation industry. Citation: Clare, J.; Kourousis, K.I Keywords: learning form incidents; airworthiness; aircraft maintenance; safety management Learning from Incidents: A Qualitative Study in the Continuing Airworthiness Sector. Aerospace 2021, 8, 27. https://doi.org/10.3390/ 1. Introduction aerospace8020027 If it were possible for all organizations to learn effective lessons from the past, the effects of future unwelcome events might be limited [1]. Aviation safety depends to a large Academic Editor: Rosario Pecora extent on the efficacious efforts of all involved in the system [2]. Research has acknowl- Received: 16 December 2020 edged the importance of event information when it comes to learning and preventing Accepted: 18 January 2021 recurrence [3]. Thankfully, major events such as accidents are becoming less frequent and Published: 22 January 2021 generate less points for learning [4]. In contrast, there are numerous incidents with less severe consequences and if appropriately considered, these could offer an earlier insight Publisher’s Note: MDPI stays neutral into the circumstances that enable unwelcome events. Predefined and relevant information with regard to jurisdictional claims in harvested from incident reporting systems is a major element of learning and preserving published maps and institutional affil- acceptable levels of safety. Hobbs and Williamson [5] highlight the importance of aircraft iations. maintenance staff being aware of the cumulative effect of “seemingly insignificant” inci- dents as this amplifies the need to be proactive when it comes to learning from incidents. This research undertook a qualitative examination of staff involved in aircraft maintenance and continuing airworthiness operations in order to identify factors that could augment Copyright: © 2021 by the authors. learning from incidents within this industry sector. Licensee MDPI, Basel, Switzerland. In the areas of continuing airworthiness and aircraft maintenance, safety manage- This article is an open access article ment systems include incident and occurrence reporting [6] as an obligation. It is common distributed under the terms and for incidents to be discovered within organizations and reported with the assistance of conditions of the Creative Commons such “systems of systems” [7]. On an operational level, initial training on human factors Attribution (CC BY) license (https:// and company procedures is intended to specify and re-affirm the category and type of creativecommons.org/licenses/by/ occurrence and incident that should be reported. Recent developments in European Union 4.0/). Aerospace 2021, 8, 27. https://doi.org/10.3390/aerospace8020027 https://www.mdpi.com/journal/aerospace Aerospace 2021, 8, 27 2 of 20 (EU) regulations [8] empower voluntary and confidential reporting and are independent of all other individual obligations. Detecting and identifying hazards highlighted through incident reporting systems is also recommended by the International Civil Aviation Organi- zation (ICAO) standards and recommended practices as an effective means of augmenting levels of safety. However, Gerede [9] strongly suggests that a failure to foster a just cul- ture is considered to have a negative impact upon effective data collection (reporting), organizational learning and the subsequent ability to learn from incidents. Drupsteen and Wybo [10] reaffirm organizations use experience gained from past events in order to improve safety. Effective learning can be considered as a successful translation of safety information into knowledge. Utilizing information from events with learning potential can actively improve the operating environment and help prevent recurrence. Learning in this context can often be experienced as modifying or implementing new knowledge where cultural, technical or procedural elements are integrated. Therefore, when learning is transformed into measures to prevent re-occurrence, an organization often has a reasonable means of mitigating future similar events. Argyris and Schön [11] highlight the importance of learning to detect and address effective responses to errors. Their “theory in action” concept is the focal point for this determination. The first of its two components, “theory in use” is one that guides a person’s behavior. It is often “tacit” and is how people behave routinely. Very often these observed “habits” are unknown to the specimen. The second element is known as “espoused theory”, namely what people say or think they do. Drupsteen and Guldenmund [12] mention that espoused theory comprises of “the words we use to convey what we do, or what we like others to think we do”. However, it is important to re-affirm the linkages that exist between individuals and organizational learning. The introduction of safety management systems (SMS) has initiated a shift in how organizational errors are viewed. Firstly, equipment has become increasingly more reliable, but the human form has not displayed the same response. In the second instance, the impact of complexities associated with an increasing cognitive load for staff is just beginning to be realized. The existence of a potential for blaming an individual is now being aligned with organizational responsibilities. Prior to this, event causation was often misrepresented or even over quantified the human input as organizational factors were not always considered. They offer an insight into the connection between individual actions and organizational initiatives designed to secure the best safety outcomes. Fogarty et al. [13] also recognize the role that both individual factors have on human error and the inputs both can have on preventing recurrence. ICAO Doc 9859 ICAO [14] defines a template for aviation operators and regulators to support the application of a variety of proactive, predictive and reactive oversight methodologies. In addition to routine monitoring schemes, voluntary and mandatory reporting, post incident follow-up, there are also regular safety oversight audits. These audits and inspections often set out to establish if there is a difference between espoused theory and the theory in use (e.g., is the task being correctly performed in accordance with the documented procedure/work instruction or is there a deviation from approved data and practice?). However, Drupsteen and Guldenmund [12] caution auditors not to “focus too much on the documentation of procedures” alone. In such cases, the oversight audit may be ineffective because of its sole focus on espoused theories of the organization only and not the theory-in-use. These authors translate this idea of poor focus on theory in action, into a valid learning component arising from incidents. They also highlight the “espoused” aspect where those attempting to learn from incidents often fail to experience the desired learning because outcomes are not fully aligned with the practical objectives of a learning from incidents (LFI) initiative. For learning to be most effective, espoused theory and theory in use should be reasonably well aligned. Ward et al. [15] propose it is necessary to further develop an operational model that can account for “what is meant to happen and what actually happens”. Continuing airworthiness and aircraft maintenance and activities performed in EU member states are subjected to rules that mandate reporting of defined issues. Repositories Aerospace 2021, 8, 27 3 of 20 of reported data tend to be populated by sources that are predominantly the subject of mandatory reporting requirements. Conventional safety oversight models also only verify the presence of reporting media and repositories in this segment of the industry. Jacobsson et al. [16] avow the degree of interest invested in learning from incidents but question its efficiency in some organizations. Although unwelcome events are less prevalent, less severe events still provide learning opportunities. There is often only a primary focus for organizations upon reporting in line with each state’s own reporting obligations. Unfor- tunately, a narrow focus on this single element of an incident in its lifecycle can negate the potential benefits of learning from incidents at an organizational level. The absence of clearly defined competency requirements [6] that support a pedagogy for learning from incidents for continuing airworthiness staff could also be considered an impediment to effective learning in the domain. The featured industry sector is regulated by the application and upkeep of numerous requirements in the jurisdictions of operation. In general, a costly regulatory overhead tends to be carried by regulating states and operators to support safe and viable activity. However, a growing tendency to increase regulatory requirements in pursuance of safer activity across the segments may not always offer the same returns as previously realized by states. Brunel [17] (p.45) suggests, “ . . . it is impossible to make men perfect: the men will always remain the same as they are now and no legislation will make him have more presence of mind . . . ”. Furniss et al. [18] reviewed the Hollnagel [19] Functional Resonance Analysis Method (FRAM) which explores how functional variability resonates within systems, i.e., how well comprising elements function in a system. They also consider how FRAM can be modified to support complex socio technical system improvements. Perhaps as the paradigm supporting the linearity of regulatory oversight shifts, proactive regulatory inputs will also influence more effective safety outputs as intricacy increases. 1.1. Systematic Literature Review The primary reason for conducting a systematic review was to examine how learning from incidents occurs in aircraft maintenance and continuing airworthiness management. Other sectors and the issues impacting learning in these areas were also considered. The literature review sets out to establish factors that contribute to or potentially constrain learning from incidents in the subject domain. Applying a qualitative research approach is advantageous as it can provide a deeper contextual understanding of the literature and can assist with better research integration. The application of rigor and comprehensiveness can assist with advancing knowledge and identifying research gaps and aspects for further research in this area. Okoli and Schabram [20] suggest “a dedicated methodological approach is necessary in any kind of literature review”. A preliminary search of literature highlighted a scarcity of best-practice guidelines for conducting systematic literature reviews in this area. Qualitative research involves handling considerable volumes of data and a degree of discipline is required so that search results, decisions regarding subject inclusions and exclusions are recorded and references are well managed. Endnote was used in support of the literature review during this research. An electronic database is useful for supporting a search strategy, arranging publications and storing references [21]. The qualitative data analysis software NVivo (NVivo 12, QSR International, Melbourne, Australia) was used to augment the data management, storage and analysis associated with the literature review. NVivo possesses many functions, such as facilitating the synthesis of a review [22]. A sys- tematic search of in excess of 1000 publications was performed in the following databases: Web of Science, Scopus, IEEE Xplore, ProQuest and EBSCO. The following predefined search terms were applied: “learning from incidents”, “learning from experience”, “aircraft maintenance”, “aircraft management” and “safety management systems”. A practical screening of title and abstract was applied to each manuscript using predefined terms (e.g., subject, setting, publication, year). This part of the process had to be broad enough to create a sufficient number of applicable publications but also had to be practically man- Aerospace 2021, 8, 27 4 of 20 ageable. The following criteria were implemented for the practical screening of the source bibliographic details, title and abstract: Subject—Related to learning from incidents and past experiences. Setting—Any high reliability industry or sector where learning from incidents is critical. Publication—Journal or peer-reviewed conference proceedings. Date range—published post 1992. The year 1992 was the starting point for the screen- ing process, since at the time of planning the research project, 25 years was considered to be a reasonable timespan to include material pertaining to learning from incidents. The output of the practical screen step produced a list of publications denoted as the screened set of publications. An Endnote library was then created to store and manage the full text of the retrieved publications. The next step involved filtering the publications into primary and secondary publication subsets using only primary research manuscripts in the next phase. Applying a set of criteria helps to reduce any researcher bias in the screening system. A set of inclusion and exclusion criteria [23] was developed in accordance with the guidelines included in [24,25], listed in Table 1. Two researchers were involved in the screening process. Table 1. Inclusion and exclusion criteria used for the filtering of the subset of primary publications. Included Excluded Research studies Literature reviews Qualitative and mixed methods Quantitative methods Perceptions and experiences Focused on decision-making and legislative requirements Reference to just culture Not about “no blame” or a punitive approach High reliability settings Non high reliability settings Published post 1992 Peer-reviewed publications Industry based settings Original studies The final set of 18 papers was imported into NVivo and the following analysis approach, as defined by Bandara et al. [22], was used for the selection of the codifica- tion themes: Deductive—themes reported on are predetermined to some extent. In this case, these predetermined themes were the output of a focus group process. Inductive—themes reported are derived from analysis of the literature. NVivo is limited in terms of providing thematic classifications based on the occurrence of key words but can assist with identifying relationships between words and phrases amongst publications. It also provided thematic classifications of data based on the oc- currence of key words and phrases. The coding process consisted of selecting relevant passages of text that were captured in one or several of the framework nodes. Maykut and Morehouse [26] defines a propositional statement as “a statement of fact the researcher tentatively proposes, based on the data”. Memos were used to draft these summary state- ments which formed part of the literature review. Central to the idea of learning is how an incident is generally moderated during its useful existence. Section 1.2 documents this approach. 1.2. The Notion of a Generic Incident Lifecycle Figure 1 illustrates how an incident tends to be managed through its quiddity. This view is one possible way of representing the elements comprising a lifecycle view. Cooke and Rohleder [27] suggest it should also be evident that an incident system will operate most effectively when a safety management system has already been put in place and avoid- able risks are addressed. They propose an effective system that addresses: identification Aerospace 2021, 8, 27 5 of 20 1.2. The Notion of a Generic Incident Lifecycle Figure 1 illustrates how an incident tends to be managed through its quiddity. This view is one possible way of representing the elements comprising a lifecycle view. Cooke Aerospace 2021, 8, 27 5 of 20 and Rohleder [27] suggest it should also be evident that an incident system will operate most effectively when a safety management system has already been put in place and avoidable risks are addressed. They propose an effective system that addresses: identifi- and response, reporting, investigation, identifying causal structure, making recommenda- cation and response, reporting, investigation, identifying causal structure, making recom- mendationstions, , communicat communicating ing and reca andll rin ecalling g incident incident learning, learning, and im and plement implementing ing correct corr ive ective actions. Drupsteen et al. [28] also consider an incident from a learning perspective in its cycle. actions. Drupsteen et al. [28] also consider an incident from a learning perspective in its Their main constituents are investigating and analyzing incidents, planning interventions, cycle. Their main constituents are investigating and analyzing incidents, planning inter- intervening and evaluating (each of these four stages are further sub-divided into eleven ventions, intervening and evaluating (each of these four stages are further sub-divided sub-components). Continuing airworthiness-related incidents are notified by way of a into eleven sub-components). Continuing airworthiness-related incidents are notified by formal mechanism of reporting. During the data gathering phase of this research, the steps way of a formal mechanism of reporting. During the data gathering phase of this research, outlined in Figure 1 were found to be dictated by regulatory requirements [6,8]. Once the the steps outlined in Figure 1 were found to be dictated by regulatory requirements [6,8]. incident enters its lifecycle, it ideally transverses a process that transforms the information Once the incident enters its lifecycle, it ideally transverses a process that transforms the gathered into knowledge. Figure 1 and the contiguous paragraph offer an overview of how information gathered into knowledge. Figure 1 and the contiguous paragraph offer an the capture and processing of the incident information occurs in practice. overview of how the capture and processing of the incident information occurs in practice. Figure 1. An example of an aviation incident lifecycle within the continuing airworthiness and Figure 1. An example of an aviation incident lifecycle within the continuing airworthiness and aircraft maintenance sector. aircraft maintenance sector. Continuing airworthiness-related serious incidents are rare but often due to environ- Continuing airworthiness-related serious incidents are rare but often due to environ- mental, cognitive and mechanical demands, reportable and unreportable events do occur. mental, cognitive and mechanical demands, reportable and unreportable events do occur. All organizations in the industry segment subscribe to a reporting system and reports can All organizations in be made the electr ind onically ustry segmen or in paper t subsform cribe to in smaller a reportor ing ganizations. system and The repmain orts can underpinning be made elect regulation ronically or i in Eur n paper form ope, EU regulation in smalle2018/1139 r organizat[ ions. T 29], refers he main to a unde management rpinning system and regulation in Europe, EU regulation 2018/1139 [29], refers to a management system and mandates an organization to implement and maintain such a system to ensure compliance mandates an with orga these nizati essential on to imr pl equir ement a ements. nd m In aint practice, ain such althoug a systh em t a reporter o ensure co can mplianc report events e directly with these essential to an aviation requir authority ements. In , all pr oractice ganizations , although are requir a reporter ed to can have report an internal events di- reporting system also. A focal point/gate keeper will process these reports either internally and/or inform rectly to an aviation authority, all organizations are required to have an internal reporting third party stakeholders such as aviation authority or aircraft manufacturer as required system also. A focal point/gate keeper will process these reports either internally and/or by procedure. Depending on the event, technical management may determine there are inform third party stakeholders such as aviation authority or aircraft manufacturer as re- immediate actions required to recover a situation or restore serviceability. While a small quired by procedure. Depending on the event, technical management may determine number of scenarios will require an event to be investigated fully before an aircraft returns there are immediate actions required to recover a situation or restore serviceability. While to service, many incidents are investigated post event. As soon as causation is established, a small number of scenarios will require an event to be investigated fully before an aircraft if accepted by the relevant technical function, the report is closed. This management returns to service, many incidents are investigated post event. As soon as causation is system is strongly influenced by regulatory requirements and procedural form and is a established, if accepted by the relevant technical function, the report is closed. This man- pre-eminent influence on how an incident and its actors behave from the time a report is agement system is strongly influenced by regulatory requirements and procedural form made to the time its impact has been terminated. One of the limitations inherent in this and is a pre-eminent influence on how an incident and its actors behave from the time a cycle is that lessons tend to be delivered at a later point in time mostly through the medium report is made to the time its impact has been terminated. One of the limitations inherent of recurrent training programs such as continuation and human factors training. Therefore, in this cycle is that lessons tend to be delivered at a later point in time mostly through the there is often a hiatus in the feedback cycle. However, the effectiveness of the process and medium of recurrent training programs such as continuation and human factors training. the perceived contribution to learning are not fully reflected in this view. Therefore, there is often a hiatus in the feedback cycle. However, the effectiveness of the process and the perceived contribution to learning are not fully reflected in this view. 1.3. A Potential Learning Cycle Emerges According to Lindberg et al. [30], in order to prevent accidents, it is essential to learn from previous accidents and incidents. Lukic et al. [31] suggest that in order to increase the effectiveness of learning from incidents, it is necessary to understand who should be included in the learning process. In Figure 2, the incident lifecycle is aligned with the learning process in order to highlight where potential improvements might be made. As the incident is managed and causation is established, there are potential avenues Aerospace 2021, 8, 27 6 of 20 1.3. A Potential Learning Cycle Emerges According to Lindberg et al. [30], in order to prevent accidents, it is essential to learn from previous accidents and incidents. Lukic et al. [31] suggest that in order to increase the effectiveness of learning from incidents, it is necessary to understand who should be Aerospace 2021, 8, 27 6 of 20 included in the learning process. In Figure 2, the incident lifecycle is aligned with the learning process in order to highlight where potential improvements might be made. As the incident is managed and causation is established, there are potential avenues open for learning. The ultimate desired outcome is that adequate measures are put in place to pre- open for learning. The ultimate desired outcome is that adequate measures are put in vent a recurrence of the event. However, the lessons available in a potential learning prod- place to prevent a recurrence of the event. However, the lessons available in a potential uct are not always used to best effect when considering the Figure 1 process. Drupsteen learning product are not always used to best effect when considering the Figure 1 process. et al. [28] state that “many incidents occur because organizations fail to learn from past Drupsteen et al. [28] state that “many incidents occur because organizations fail to learn lessons”, because the traditional approach often stops short of preventing future incidents. from past lessons”, because the traditional approach often stops short of preventing future Their research examines: investigating and analyzing incidents, planning and prevention, incidents. Their research examines: investigating and analyzing incidents, planning and intervening and evaluating steps in a learning process. Ward et al. [15] found that the prevention, intervening and evaluating steps in a learning process. Ward et al. [15] found resulting relationship between the individuals and the systems have a direct impact upon that the resulting relationship between the individuals and the systems have a direct impact the system and prevailing environment. Silva et al. [3] examined how organizations use upon the system and prevailing environment. Silva et al. [3] examined how organizations accident information to reduce the occurrence of unwelcome events. Drupsteen and Wybo use accident information to reduce the occurrence of unwelcome events. Drupsteen and [10] found that hindsight can determine if an organization did learn from an event but Wybo [10] found that hindsight can determine if an organization did learn from an event there are no models to assist with gauging the “propensity” of an organization to learn. but there are no models to assist with gauging the “propensity” of an organization to Drupsteen and Hasle [1] suggest that learning can be improved if limiting factors are ad- learn. Drupsteen and Hasle [1] suggest that learning can be improved if limiting factors ar dresse e addr d. essed. Figure 2. Incident learning product and process (broken line denotes iterative learning feedback). Figure 2. Incident learning product and process (broken line denotes iterative learning feedback). The proposed enhancement (shown in Figure 2) to the generic lifecycle in the “tradi- The proposed enhancement (shown in Figure 2) to the generic lifecycle in the “tradi- tional” tional” appr appr oach oach rep reprr esents esentsa a nove novell vi view ewand and b brings rings the the learn learning ing p prr oduct oductinto intofocus. focus. Th This is figure highlights the benefits of ensuring the feedback loop of an incident is centered on the figure highlights the benefits of ensuring the feedback loop of an incident is centered on learning product. Treating its development as an iterative process ensures all steps in the the learning product. Treating its development as an iterative process ensures all steps in cycle are included and where deficiencies are noted, they can be identified and communi- the cycle are included and where deficiencies are noted, they can be identified and com- cated during the iterations. This can assist with delivery of timely and sustainable learning municated during the iterations. This can assist with delivery of timely and sustainable and help prevent an inability to think, talk and see what actions are proper in specific learning and help prevent an inability to think, talk and see what actions are proper in situations [32]. According to Drupsteen et al. [28], it is necessary to gain an insight into the specific situations [32]. According to Drupsteen et al. [28], it is necessary to gain an insight steps of the process to identify factors that hinder learning in order to make improvements. into the steps of the process to identify factors that hinder learning in order to make im- The research suggests an emphasis on accessible, timely and appropriate learning content provements. The research suggests an emphasis on accessible, timely and appropriate could provide all stakeholders in the process with better value for their efforts. Perhaps learning content could provide all stakeholders in the process with better value for their one reason that the customary incident lifecycle and its limitations prevail is related to efforts. Perhaps one reason that the customary incident lifecycle and its limitations prevail management theory. While innovators like Taylor [33] are responsible for advances in is related to management theory. While innovators like Taylor [33] are responsible for management, such theories have not always fully considered safety and learning. The advances in management, such theories have not always fully considered safety and early 1900s witnessed a time when it was necessary to inaugurate efficiencies in produc- learning. The early 1900s witnessed a time when it was necessary to inaugurate efficien- tion by initially decomposing tasks in order to introduce liner efficiencies. The limitations cies in production by initially decomposing tasks in order to introduce liner efficiencies. experienced in incident learning processes today may relate to this circumscribed tradition. The limitations experienced in incident learning processes today may relate to this cir- cumscribed tradition. 2. Materials and Methods 2.1. Philosophical Underpinnings The fields of science and philosophy consider ontology and epistemology in terms of What is the nature of reality? and How is human knowledge constructed? The ascendant ideologies of positivism and interpretivism can be applied in support of these philosophical differences [34–37]. Hirschheim [38] puts forward the aim of positivism to, “seek to explain and predict what happens in the social world by searching for irregularities and causal relationships between its constituent elements”. In contrast to this stance, Schwandt [39] Aerospace 2021, 8, 27 7 of 20 suggests the aim of interpretivism is to gain understanding. Interpretive research seeks to develop a richer understanding of the complex world of lived experience from the point of view of those who live in it. “This goal is variously spoken of as an abiding concern for the life world, for the emic point of view, for understanding meaning, for grasping the actor’s definition of a situation, for verstehen” [39] (p.118). The intent of this qualitative study was to understand how various situations impact on learning from incidents in the aircraft maintenance and continuing airworthiness man- agement domain by interacting with the participants on a social plane. Thus, in order to gain an empathetic understanding of the participants and their actions, the pursuit of “ver- stehen” considers adopting an interpretive paragon as an approach. This approach is not initiated with the aid of a hypothesis intended for testing but rather using a lodestar that guides the researcher to a point of discovery supported with an inductive modus operandi. The study is unwavering in its support for the view that (individual and combined) qual- itative and quantitative approaches possess equal value in terms of their investigative potential in this area of focus. In summary, the project employs a qualitative research methodology in an effort to generate “rich” findings in support of gaining a good un- derstanding of the learning environment in the featured domain. According to Maykut and Morehouse [26], the purpose of qualitative research is to discover the inner world of perceptions and meaning-making in order to gain an understanding to describe and explain certain social phenomenon from participants’ perspectives. In order to accomplish this, focus group activity was managed concurrently with the literature review. These activities cumulatively generated five themes which were used as the basis for a semi-structured interview template. The project employed a qualitative research methodology in an effort to generate “rich” findings in support of gaining a good understanding of the learning environment in the featured domain. The outcome of a qualitative research initiative was contextual findings as opposed to broad generalizations. 2.2. Focus Group According to Kitzinger [40], “focus groups are group discussions organized to explore a specific set of issues such as people’s views and experiences . . . ”. The idea of conducting group interviews is not a new one. Bogardus [41] is an early example of a reference to utilizing the group interview. Frey and Fontana [42] suggest that group interviews can be formally structured for a specific purpose or can be performed in a more informal setting where a researcher can “stimulate a group discussion”. Specific examples in the literature of focus groups being developed systematically within the area of aircraft maintenance and management are scarce. Frey and Fontana [42] state that although group interviews have implicitly informed research, often they are not formally acknowledged as part of the process. Powell and Single [43] remind us that when recruiting focus group participants, one must be mindful of systemic biases. Averting this was ensured by being careful to enlist the participants from different organizations and different positions of responsibility. Three sessions comprising of three industry professionals within each group were successfully moderated by the researcher. During the three phases of working with the focus group, statements and terms were recorded as dialogue amongst the members and observed. The second meeting of the focus group developed four codes (safety, regulatory compliance, root cause, reporting) that had emerged from the group’s earlier outputs. These four codes were further distilled during the focus group activity and were consolidated into two themes (reporting, root cause) that were to eventually form part of the piloted semi- structured interview instrument. Reporting and root-cause themes were the result of the draft consolidation of the comments and emerging codes. In concert with the focus group activities, a literature review was performed by the researchers and this generated three further themes as reflected in Table 2. Aerospace 2021, 8, 27 8 of 20 Table 2. Codification themes used in the NVivo analysis of the final set of publications. Codification Theme Description Origin Root Cause Reason to establish causation Focus Group Reporting Value of reporting to learning from incidents Focus Group Learning from Incidents Outcomes of learning from incidents Literature Analysis Just culture Impact of just culture on learning from incidents Literature Analysis Precursors Contribution of precursors to learning from incidents Literature Analysis The resulting draft semi-structured template (provided in Appendix A) containing the five themes was scrutinized by the focus group. The constituent questions relating to each theme and the running order of the document was subject to many minor changes during the individual piloting of the instrument with the three group members. 2.3. Data Collection Data were gathered from seven organizations using a semi-structured interview tem- plate. The participating organizations were involved in aircraft maintenance and continuing airworthiness activities. Building trust and commitment, as proffered by Chatzi [44] and Chatz et al., [45] was deemed to be a necessary tenet of a successful data collection exercise. Managing the interview process with the support of senior staff complimented visible top-down support for the research and ensured there would be no confusion regarding access to what some organizations often classify as sensitive commercial data. The aim was to explore how learning from incidents occurs and what can constrain learning in the area of focus. The pilot phase ensured the desired outcome of the main data collection phase would be congruent with the aims of the study. The interviews were recorded, transcribed and participants could not be identified from the recordings or transcripts. Full ethical approval for the data gathering was granted by the University. 2.3.1. Instrument Data were collected using semi-structured interviews, lasting on average sixty minutes. The “aide memoir” was arranged so that participants could offer a flexible response and any emerging themes could be identified. The semi-structured approach facilitated emphasis being placed upon any points that warranted further focus or examination by the researchers. An example of the interview template is included in Appendix A. Interviewees were asked to give an example of a recent incident they were familiar with. The structure of the template, (a) probed process around reporting and (b) elicited the participants perception of learning from incidents within their organizations. Following on from the initial contact on reporting, the participants discussed just culture, learning, root-cause and incident precursors during their individual engagements with the researchers. 2.3.2. Participants The “key issue in selecting and making decisions about the appropriate unit of analysis is to decide what it is you want to be able to say something about at the end of the study” [46] (p.168). The objective of this study was to investigate individuals’ perceptions of how learning from incidents takes place and the obstacles present in the maintenance and continuing airworthiness management domain of the aviation industry segment. There were thirty-four (34) participants in total, as presented in Table 3. Each of the organizations maintained between 6 and 300 aircraft at the time of the study. While traditional reporting and learning themes were evident outputs from the focus group meetings, it was decided that the data would be collected through one-to-one semi-structured interviews. Semi-structured interviews permitted the researchers to get a deeper understanding of complex organizational and social interactions and at the same time follow a construct. The participating organizations were selected based upon them being accredited to perform aircraft maintenance and continuing airworthiness activities since the inception of EU regulation 1321/2014 [6]. Within this domain, there are categories Aerospace 2021, 8, 27 9 of 20 of staff that are required to be aware of incident reporting and make a report as necessary (e.g., technical managers, certifying staff, quality assurance staff, stores personnel, technical services). Each organization is required under legislation to employ a satisfactory level of staff regardless of their aviation activities. As a minimum, at least one of each of these roles was represented in the study. It was ensured that at least one staff member from each discipline was included in the study and had made a report in the previous twelve months. As certifying staff, technical managers and quality assurance staff are by virtue of their position active reporters (due to their exposure to active operations), staff in these disciplines were well represented in the study’s cohort. Participation in the study was on a voluntary basis and all who participated were acquainted with the project prior to performing the interviews. All participants signed consent forms. Table 3. Participants in the study (n = 34). Participant Roles Number Category B1 Engineer 4 Supervisor 3 Category A Mechanic 3 Quality Assurance Engineer 3 Category B2 Engineer 2 Shift Controller 2 Contract Composite Inspector 1 Inspector 1 Aeronautical Engineer 1 Category B1/B2 Engineer 1 Maintenance Manager 1 Technical Safety Manager 1 Technical Services Manager 1 Line Maintenance Manager 1 Deputy Quality Manager 1 Maintenance Control Manager 1 Maintenance Planner 1 Maintenance Safety Officer 1 Apprentice Technician 1 2.4. Data Analysis Thematic analysis was the method chosen to support the analysis of the study’s data. The Braun and Clarke [47] six-step proposition, which consists of eight discreet cycles, in conjunction with the QDA Training [48] material, formed the basis of the analysis technique. A practical iterative approach was adopted throughout the analysis where the data were formally arranged into discrete phases. The eight individual stages of analysis distributed over the six phases were designed to support a robust and rigorous analysis of the data. Table 4 below illustrates the stages and processes outlined and performed in NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six-step approach that supports the application of thematic analysis is shown in column one and the corresponding application in NVivo is shown in column two. The third column features the strategic elements of coding as the researcher moved from the initial participant-led descriptive coding, to the secondary coding which was more interpretative in nature indicating this phase of coding was both researcher- and participant-led. The final abstraction to themes was researcher informed only. This phase was designed to allow the researchers to engage the participant in direct dialogue with a wider arena such as literature and policy or strategy for example. The fourth and final column illustrates the more iterative nature of the coding, analysis and reporting of proceedings that terminate in a conclusion. Aerospace 2021, 8, 27 10 of 20 the data. Table 4 below illustrates the stages and processes outlined and performed in NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- step approach that supports the application of thematic analysis is shown in column one Aerospace 2021, 8, 27 10 of 20 and the corresponding application in NVivo is shown in column two. The third column Aerospace 2021, 8, 27 10 of 20 features the strategic elements of coding as the researcher moved from the initial partici- pant-led descriptive coding, to the secondary coding which was more interpretative in nature indicating this phase of coding was both researcher- and participant-led. The final the data. Table 4 below illustrates the stages and processes outlined and performed in the data. Table 4 below illustrates the stages and processes outlined and performed in abstraction to themes was researcher informed only. This phase was designed to allow the NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- researchers to engage the participant in direct dialogue with a wider arena such as litera- step approach that supports the application of thematic analysis is shown in column one step approach that supports the application of thematic analysis is shown in column one ture and policy or strategy for example. The fourth and final column illustrates the more and the corresponding application in NVivo is shown in column two. The third column and the corresponding application in NVivo is shown in column two. The third column iterative nature of the coding, analysis and reporting of proceedings that terminate in a features the strategic elements of coding as the researcher moved from the initial partici- Aerospace 2021, 8, 27 10 of 20 features the strategic elements of coding as the researcher moved from the initial partici- Aerospace 2021, 8, 27 10 of 20 conclusion. pant-led descriptive coding, to the secondary coding which was more interpretative in pant-led descriptive coding, to the secondary coding which was more interpretative in nature indicating this phase of coding was both researcher- and participant-led. The final nature indicating this phase of coding was both researcher- and participant-led. The final Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING abstraction to themes was researcher informed only. This phase was designed to allow the abstraction to themes was researcher informed only. This phase was designed to allow the Training [48] material. the data. Table 4 below illustrates the stages and processes outlined and performed in researchers to engage the participant in direct dialogue with a wider arena such as litera- the data. Table 4 below illustrates the stages and processes outlined and performed in researchers to engage the participant in direct dialogue with a wider arena such as litera- NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- ture and policy or strategy for example. The fourth and final column illustrates the more NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- Analytical Process ture and policy or strategy for example. The fourth and final column illustrates the more step approac Practical ah p t ph lic atat suppor ion of Br ts taun he appl andica tion of t Stra h tegic Ob ematic an jec- alysis is shown in Iterative Proce column one ss Aerospace 2021, 8, 27 10 of 20 iterative nature of the coding, analysis and reporting of proceedings that terminate in a step approach that supports the application of thematic analysis is shown in column one (Braun and Clarke, 2006) iterative nature of the coding, analysis and reporting of proceedings that terminate in a and the corresponding application in NVivo is shown in column two. The third column Clarke in conjunction with NVivo tive Throughout Analysis conclusion. Aerospace 2021, 8, 27 and the corresponding application in NVivo is shown in column two. The third column 10 of 20 [47] conclusion. features the strategic elements of coding as the researcher moved from the initial partici- features the strategic elements of coding as the researcher moved from the initial partici- Transcribing data (if necessary), reading pant-led descriptive coding, to the secondary coding which was more interpretative in Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING pant-led descriptive coding, to the secondary coding which was more interpretative in Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING the data. Table 4 below illustrates the stages and processes outlined and performed in 1.Familiarizing yourself and re-reading the data, noting down Assigning data to re- nature indicating this phase of coding was both researcher- and participant-led. The final Training [48] material. Training [48] material. nature indicating this phase of coding was both researcher- and participant-led. The final NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six- fined concepts to por- with the data initial ideas. Import data into the NVivo Data Management Table 4. Stages and Process abstraction Involved to in themes Qualitative was r Analysis. esearcher Adapted informed on from Braun ly. T and his phas Clarke e w [47 a]sand desQDA igned TRAINING to allow the abstraction to themes was researcher informed only. This phase was designed to allow the step approach that supports the application of thematic analysis is shown in column one Analytical Process data management tool tray meaning Analytical Process (Open and hierar- Training [48] material. researchers to engage the participant in direct dialogue with a wider arena such as litera- Practical application of Braun and Strategic Objec- Iterative Process researchers t Practic o engag al app elic that e p ion art of Br icipant aun in and direct dialo Str gue wit ategic Ob h a w jec- ider arenIt a s eu rat ch ive Proce as litera- ss and the corresponding application in NVivo is shown in column two. The third column (Braun and Clarke, 2006) Phase 2.Open Coding:Coding interest- (Braun and Clarke, 2006) ture and policy or strategy for example. The chal co fourtdin h an g thr d fiona ugh l column illustrates the more Clarke in conjunction with NVivo tive Throughout Analysis ture and policy or strategy for example. The fourth and final column illustrates the more features the strategic e Clarke in conjunction with NVi lements of cod vo ing as the rese tiv archer moved e Through from the init out Analys ial partic is i- Practical application of [47] iterative nature of the coding, analysis and reporting of proceedings that terminate in a 2. Generati[4 ng 7] init ial ing features of the data in a systematic NVIVO) Analytical Process (Braun Iterative Process iterative natuBraun re of tand he cod Clarke ing, in analysis and report Strategic ing Objective of proceedings that terminate in a pant-led descriptive coding, to the secondary coding which was more interpretative in Transcribing data (if necessary), reading and Clarke, 2006) [47] conclusion. Throughout Analysis codes fashion across the entire data set, col- Transcribing data (if necessary), reading Conjunction with NVivo conclusion. nature indicating this phase of coding was both researcher- and participant-led. The final 1.Familiarizing yourself and re-reading the data, noting down Assigning data to re- lecting data relevant to each code 1.Familiarizing yourself and re-reading the data, noting down Assigning data to re- Transcribing data (if abstraction to themes was researcher informed only. This phase was designed to allow the Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING fined concepts to por- with the data initial ideas. Import data into the NVivo Data Management with the data Phas initiae 3.C l idea as. I tegori mport zat dat ion of a in Cto odes:C the NV ollivo at- fined concepts to por- Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING necessary), reading and Data Management researchers to engage the participant in direct dialogue with a wider arena such as litera- Training [48] material. data management tool tray meaning (Open and hierar- Training [48] 1.Familiarizing material. yourself withing codes re-reading into potenti the data,a noting l themes, gather- Refining and distilling data management tool Data Management Assigning tray m data eanin to rg efined (Open and hierar- ture and policy or strategy for example. The fourth and final column illustrates the more 3. Searching for themes the data Phase 2.Open down Coding initial ideas. :Coding intere Import st- (Open chal co anddin hierar g thr chal ough coding concepts to portray meaning Analytical Process ing all data relevant to each potential more abstract concepts Phase 2.Open Coding:Coding interest- chal coding through iterative nature of the coding, analysis and reporting of proceedings that terminate in a Analytical Process Practical application of Braun and Strategic Objec- Iterative Process data into the NVivo data through NVIVO) 2. Generating initial ing features of the data in a systematic NVIVO) Practical application of Braun and Strategic Objec- Iterative Process (Braun and Clarke, 2006) theme 2. Generating initial ing features of the data in a systematic NVIVO) conclusion. (Braun and Clarke, 2006) Cmanagement larke in conj tool unction with NVivo tive Throughout Analysis codes fashion across the entire data set, col- Descriptive Ac- [47] Clarke in conjunction with NVivo tive Throughout Analysis Phase 4.Coding on:Checking if the codes fashion across the entire data set, col- [47] Phase 2. Open Coding:Coding counts lecting data relevant to each code Table 4. Stages and Process Transcr Involveid b in ing Qualitative data (if n Analysis. ecessary), Adapte readin d fro g m Braun and Clarke [47] and QDATRAINING themes work lecting data re in re levant lation to e to ach the coded ex code - interesting features of the data Transcribing data (if necessary), reading Training [48] material. (Reordering, ‘coding Phase 3.Categorization of Codes:Collat- Assigning data to re- 4. Rev 1.Fami ili ewin arizg themes ing yourself tra and re- cts (lr evel eadi 1) ng the da and the entire da ta, noting down ta set 2. Generating initial codes Phas in e 3.C a systematic ategorizat fashion ion of acr Coss odes:Collat- 1.Familiarizing yourself and re-reading the data, noting down Assigning data to re- on’ and annotating ing codes into potential themes, gather- Refining and distilling the entire data set, collecting with the data initial ideas. Import data into the NVivo fined concepts to por- (level 2), generating a thematic “map” of Data Management Assigning data to ing codes into potential themes, gather- Refining and distilling Refining and distilling more Analytical Process 3. Searching for themes with the data initial ideas. Import data into the NVivo fined concepts to por- Data Management 3. Searching for themes data relevant to each code Practical application of Braun and Str thro aug tegic Ob h NVIV jec- O) Iterative Process ing all data relevant to each potential more abstract concepts tray meaning the analy data mana sis gement tool themes/concepts to abstract concepts ing all data relevant to each potential (Open and hierar- more abstract concepts (Braun and Clarke, 2006) data management tool tray meaning (Open and hierar- Clarke in conjunction with NVivo Descriptive Accounts tive Throughout Analysis theme Phase 3. Categorization of Phas Phase e 5.D 2.Open ata Red Coding uction:O :Coding intere n-going ana st-ly- chal coding through portray meaning theme [47] Phase 2.Open Coding:Coding interest- chal coding through (Reordering, ‘coding on’ and Codes:Collating codes into Descriptive Ac- Phase 4.Coding on:Checking if the Descriptive Ac- 2. Generating initial ing features of the data in a systematic sis to refine the specifics of each theme, NVIVO) Phase 4.Coding on:Checking if the Transcribing data (if necessary), reading annotating through NVIVO) 2. Generat 3. Sear ing init ching ial for themes ing features o potential f the d themes, ata in gathering a systemall atic 5. Defining and naming NVIVO) counts themes work in relation to the coded ex- counts codes and t fashio hn e ove across the enti rall story (st re da oryline ta set, col s) the - themes work in relation to the coded ex- 1.Familiarizing yourself and re- data rea relevant ding the da to each ta potential , noting down Assigning data to re- codes fashion across the entire data set, col- themes (Reordering, ‘coding 4. Reviewing themes tracts (level 1) and the entire data set (Reordering, ‘coding lecting data relevant to each code analysis tells, theme generating clear defini- 4. Reviewing themes tracts (level 1) and the entire data set with the data initial ideas. Import data into the NVivo fined concepts to por- Data Management lecting data relevant to each code on’ and annotating (level 2), generating a thematic “map” of on’ and annotating Assigning data to tions Phase 3.C and n ata emes for gorization each t of C heme odes:Collat- (level 2), generating a thematic “map” of Phase 4.Coding on:Checking Assigning data to data management tool tray meaning Assigning data to (Open and hierar- Phase 3.Categorization of Codes:Collat- through NVIVO) the analysis themes/concepts to through NVIVO) if the themes work in relation Explanatory Ac- Phas ing codes e 6.Ge in ne to potenti rating An al themes, alytical Mga em ther- os. Refining and distilling the analysis themes/concepts themes/concepts to to portray Phase 2.Open Coding:Coding interest- chal coding through 3. Searching for themes ing codes into potential themes, gather- Refining and distilling to the coded extracts (level 1) Phase 5.Data Reduction:On-going analy- portray meaning 3. Searching for themes counts meaning ing all data relevant to each potential more abstract concepts Phase 7. —Testing and Validating.Phase 4. Reviewing themes Phase 5.Data Reduction:On-going analy- Ass portray me igning me an aning ing 2. Generating initial ing features of the data in a systematic NVIVO) ing all data relevant to each potential more abstract concepts and the entire data set (level sis to refine the specifics of each theme, (Extrapolating 8.Synthesizin theme g Analytical Memos. sis to refine the specifics of each theme, codes fashion across the entire data set, col- 5. Defining and naming 2), generating a thematic Explanatory Accounts theme 5. Defining and naming Descriptive Ac- and the overall story (storylines) the deeper meaning, The Phasfin e 4.C al opp oding on:C ortunity for a heckin ng i aly f t sh ise . Selec- and t “map” he ove ofrthe all st analysis ory (storylines) th (Extrapolating e deeper meaning, Descriptive Ac- lecting data relevant to each code themes Phase 4.Coding on:Checking if the themes counts analysis tells, generating clear defini- drafting summary drafting s statements ummary t themes work ion of vivid, compellin in relationg e to x the coded ex tract exam- - analysis tells, generating clear defini- Phase 5. Data counts Phase 3.Categorization of Codes:Collat- themes work in relation to the coded ex- 6. Producing the report and analytical (Reord memos ering, ‘cod throughing tions and names for each theme statements and ana- 4. Reviewing themes ples, tracts (l Reduction:On-going fina evel l ana 1)l ys anis o d the entire da f select analysis ed ext tar act set s, tions and names for each theme (Reordering, ‘coding ing codes into potential themes, gather- Refining and distilling 4. Reviewing themes tracts (level 1) and the entire data set NVIVO) 3. Searching for themes Explan on’ and anno atory Ac- tating Phase 6.Gto ene refine rating the An specifics alytica of l M each emos. lytical memos rela (levti elng b 2), gen ack eof rat the inga a n t al hysis to the re- ematic “map” of Assignin g data to Explanatory Ac- Phase 6.Generating Analytical Memos. Assigning meaning on’ and annotating 5. Defining and naming ing all data relevant to each potential more abstract concepts (level 2), generating a thematic “map” of Assigning data to theme, and the overall story counts throug h NVIVO) Phase 7. —Testing and Validating.Phase through NVIVO) Assigning meaning search the analy qusest is ion and literature, produc- counts themes/concepts to Generating themes themes Phase 7. —Testing and Validating.Phase through NVIVO) Assigning meaning theme the analys (storylines) is the analysis tells, themes/concepts to (Extrapolating 8.Synthesizing Analytical Memos. ing a scholarly Phase 5.Data Red report uction:O of thn e an -goin alys g ana is ly- portray me and concepts aning Descr (Extra ip polatin tive Ac- g 8.Synthesizing Analytical Memos. Phas generating e 4.Coding on:C clear definitions hecking if the portray meaning Phase 5.Data Reduction:On-going analy- deeper meaning, The final opportunity for analysis. Selec- [47,48] sis to refine the specifics of each theme, deeper meani counts ng, The and finanames l oppofor rtunit each y for a theme nalysis. Selec- themes work in relation to the coded ex- sis to refine the specifics of each theme, 5. Defining and naming drafting summary tion of vivid, compelling extract exam- 5. Defining and naming and the overall story (storylines) the (Reord drafting s ering, ‘cod ummary ing tion of vivid, compelling extract exam- Phase 6. Generating 4. Reviewing themes tracts (level 1) and the entire data set 6. Producing the report themes and the overall story (storylines) the 6. Producing the report statements and ana- ples, final analysis of selected extracts, themes analysis tells, Analytical Memos. generating clear defini- statements and ana- ples, final analysis of selected extracts, on’ and annotating (level 2), generating a thematic “map” of Assigning da ta to analysis tells, generating clear defini- Generating themes and lytical memos relating ba Phase ck of7. the —T aesting nalysis to the re- and tions and names for each theme thro lytical memos ugh NVIVO) relating back of the analysis to the re- the analysis concepts themes/concepts to tions and names for each theme Validating. through NVIVO) search question and literature, produc- Generating themes Explanatory Ac- Phase 6.Generating Analytical Memos. through NVIVO) search question and literature, produc- Generating themes Phase 5.Data Reduction:On-going analy- portray meaning Explanatory Ac- Phase 8. Synthesizing Phase 6.Generating Analytical Memos. ing a scholarly report of the analysis and concepts counts Phase 7. —Testing and Validating.Phase ing a scholarly report of the analysis Assand concepts igning meaning Analytical Memos. sis to refine the specifics of each theme, counts Phase 7. —Testing and Validating.Phase Assigning meaning 5. Defining and naming [47,48] (Extrapolating 8.Synthesizing Analytical Memos. [47,4 The 8] final opportunity for and the overall story (storylines) the (Extrapolating 8.Synthesizing Analytical Memos. themes 6. Producing the report analysis. Selection of vivid, deeper meaning, The final opportunity for analysis. Selec- analysis tells, generating clear defini- deeper meaning, The final opportunity for analysis. Selec- compelling extract examples, drafting summary tion of vivid, compelling extract exam- tions and names for each theme drafting summary final analysis of selected 6. Producing the report tion of vivid, compelling extract exam- 6. Producing the report statements and ana- ples, final analysis of selected extracts, extracts, relating back of the Explanatory Ac- Phase 6.Generating Analytical Memos. statements and ana- ples, final analysis of selected extracts, lytical memos relating back of the analysis to the re- analysis to the research counts Phase 7. —Testing and Validating.Phase lytical memos Assigning meaning relating back of the analysis to the re- question and literature, through NVIVO) search question and literature, produc- Generating themes (Extrapolating 8.Synthesizing Analytical Memos. through NVIVO) search question and literature, produc- Generating themes producing a scholarly report ing a scholarly report of the analysis and concepts deeper meaning, The final opportunity for analysis. Selec- ing a scholarly of the report analysis of [47 th ,48 e an ] alysis and concepts [47,48] drafting summary tion of vivid, compelling extract exam- [47,48] 6. Producing the report statements and ana- ples, final analysis of selected extracts, Phase 1 activity involves familiarizing oneself with the transcribed data. In this first lytical memos relating back of the analysis to the re- phase, the data were loaded into NVivo. It was checked and re-read several times to ensure through NVIVO) search question and literature, produc- Generating themes accuracy of the uploaded transcripts. At the end of the phase activity, initial codes were ing a scholarly report of the analysis and concepts noted down and retained. [47,48] Generating initial codes (open coding: phase 2)—According to Lincoln and Guba [49] (p. 345), a data unit can be defined as the “smallest piece of information about something that can stand by itself, that is, it must be interpretable in the absence of any additional Aerospace 2021, 8, 27 11 of 20 information other than a broad understanding of the context in which the inquiry is carried out”. The open coding is intended to systemically organize the data and uncover the essential ideas found in the data [50]. Each discrete unit of data is labelled in line with the phenomenon it represents. The second phase required broad participant-driven open coding of the interview transcripts recorded during the data gathering step of the research study. Features of interest were coded in a systematic way across the complete dataset where data relevant to each code were collected. Clear labels were allocated to these codes and definitions to serve as rules for inclusion [26]. A set of provisional categories was generated for the segmented data to be coded to. These categories were descriptions of concepts and themes in broad terms. They took two forms: researcher-driven and participant-driven. The former was derived from a theoretical framework underpinning the study and the latter from the knowledge gained of the participants’ language and customs. Hammersley and Atkinson [51] (p.153) consider the importance of participant-driven categories: “the actual words people use can be of considerable analytic importance as the ‘situated vocabularies’ employed provide valu- able information about the way in which members of a particular culture organize their perceptions of the world, and so engage in the social construction of reality”. Searching for themes—In phase 3, codes from phase 2 were collated into categories of codes by structuring all the data relevant to each potential category into a framework that could be used in support of further analysis. This phase also included distilling, re-labelling and merging common codes that were generated in phase 3 to ensure the labels and definitions for inclusion were an accurate reflection of the coded content. These first-round categories are best described as broad descriptions of concepts and themes. During the analytical process they underwent content and definition change and the existence of the two forms of category provides an important means of traversing between “natural” and “theoretical” discourses. Araujo [52] (p.68) suggests that “codes should be viewed in two ways: as part of the analyst’s wider theoretical framework and as grounded in the data.; the process of coding data should be regarded as an important intermediary step in translating social actors’ frames of meaning into the frame of theoretical discourse; coding frames therefore, mediate between the ‘natural’ everyday discourse and the theoretical discourses in social science”. Reviewing themes (coding on) in phase 4 required further decomposition of the study units of data identified in phase 1. This activity was intended to support a greater understanding of the highly qualitative elements and gain a deeper appreciation of the meanings contained within. It should be noted that not every task could be further broken down and this meant that the activity was performed only as required. Restructured codes were broken down into further sub-codes in order to augment a greater understanding of the meanings embedded within them. These distinctive aspects included communication with management, discovering latent issues, just culture, learning lessons, reporting, root causes and story of an incident. Defining and naming themes in phase 5 of the data analysis was concerned with ana- lyzing the tentative categories identified in phase 2 for their properties and characteristics. This is a pre-cursor to drafting a propositional statement for each category. Developing analytical memos moves the process beyond identification and description of broad cat- egories to a position of analyzing and fusing meanings in the data under each category. This progressed to drafting a statement that aspires to illustrate the concerted meaning of the segments of data coded to each category. Maykut and Morehouse [26] (p.140) defines a propositional statement as, “a statement of fact the researcher tentatively proposes, based on data”. This phase in addition to further data analysis to refine the specifics for each theme, generated clear definitions and a name for each theme. It also involved data reduc- tion by consolidating categories from all three cycles into a more abstract, philosophical and literature-based thematic framework and conceptually mapping and exploring their relationships with one another for reporting purposes. Aerospace 2021, 8, 27 12 of 20 Producing the report in phase 6 required analytical memos to be written against the higher-level themes to present an accurate summary of the content of each category and its codes and to also propose findings. The tasks associated with phase 6 included (i) generating analytical memos, (ii) testing and validating and (iii) synthesizing the memos coherently and cohesively, and were performed simultaneously. Writing the analytical memos against the higher-level codes (i.e., learning from incidents, learning process and learning product) required an accurate summary of each category and its codes and findings against categories. These memos considered a few key areas: 1. The content of the cluster of codes which were being reported on. 2. Patterns where relevant. 3. Considering background information noted against participants and examining any patterns relating to participants’ profiles. 4. Considering any relationship between codes and their importance in relation to the research questions. 5. Noting any primary sources relating to the context of the relationship with the litera- ture in addition to highlighting any gaps in the literature. Testing, validating and revising analytical memos was performed in phase 7. The purpose of this was to provide a self-audit of the proposed findings by soliciting evidence in the data beyond just textual quotes in support of the recorded findings and to also expand on deeper meanings within the data. This required the data to be interrogated, not only relying on relationships across and between categories, but also a degree of cross tabulation with demographics, observations and the literature. The outcome of this phase was evidence-based findings as each proposed finding was validated by being rooted in the data themselves and was reliant on the creation of reports in support of substantiated findings. The discipline of writing analytical memos was used during the data analysis process. Birks et al. [53] believe “memoing serves to assist the researcher in making conceptual leaps from raw data to those abstractions that explain research phenomena in the context of which it is examined”. In general, memos were employed at the “ideation” stage when the researcher was developing thought processes and early in the data capture phase. As decisions were made, the early processes and rationale for final analysis iterations were recorded using this medium. Memos were further employed to preserve an objective close- ness to the harvested data and to maintain the context of each semi-structured interview at the participating individuals’ level. Developing ideas, reasons for considering possible category relationships and connections was also possible through the application of the analytical memo process. The rigorous support memoing offered served to guide the anal- ysis of the data through different levels of abstraction [54]. The rule of this activity served to ensure a high degree of continuity between the outputs of ideation and the evolving interpretation that were honed through the researchers’ articulation, exploration and their iterations of the data. Overall, this drew out the meanings in the data through the increased sensitivity the researchers were offered by applying the memoing process [53]. In phase 8, the analytical memos were synthesized into a coherent and cohesive report with the findings well supported. The final phase involved the assembly of the narrative with the data extracts while appreciating the product of this amalgam in the context of the related literature. The example features the finding, clear links to the interview data and literature and an explanatory narrative in the form of a memo. This finally resulted in the compilation of the report which contained the results and discussion elements of the body of work. In summary, this study adopted an interpretative approach pivoting on the fact that it was of an exploratory nature. The study performed thirty-four interviews in eight aircraft maintenance and management organizations based in Ireland. An analysis of various potential research methods and means of data collection resulted in the following research design being implemented. A thematic analysis approach was employed as a research methodology: Aerospace 2021, 8, 27 13 of 20 In summary, this study adopted an interpretative approach pivoting on the fact that it was of an exploratory nature. The study performed thirty-four interviews in eight air- craft maintenance and management organizations based in Ireland. An analysis of various potential research methods and means of data collection resulted in the following research design being implemented. A thematic analysis approach was employed as a research Aerospace 2021, 8, 27 13 of 20 methodology: • Unit of analysis is an individual; • Semi-structured interview guide was constructed following a systematic analysis of Unit of analysis is an individual; literature and the use of a focus group; Semi-structured interview guide was constructed following a systematic analysis of literature and the use of a focus group; • Data were collected through qualitative interviews; Data were collected through qualitative interviews; • Thirty-four interviews were collected in locations endorsed by eight organizations; Thirty-four interviews were collected in locations endorsed by eight organizations; • Qualitative analysis based on the guidelines from Braun and Clarke [47] (thematic Qualitative analysis based on the guidelines from Braun and Clarke [47] (thematic analysis) employing a six-phase approach was used in the study. analysis) employing a six-phase approach was used in the study. 3. Results and Discussion 3. Results and Discussion 3.1. 3.1. Frame Framework work Figure 3 presents a framework that offers an insight into how the present study applied Figure 3 presents a framework that offers an insight into how the present study ap- the research inputs and produced the results. plied the research inputs and produced the results. Figure 3. Research study framework. Figure 3. Research study framework. The top layer reflects the five themes that formed the basis for the data gathering The top layer reflects the five themes that formed the basis for the data gathering template. These themes were developed through an iterative process of conducting focus template. These themes were developed through an iterative process of conducting focus group sessions with two themes emerging, i.e., root cause and reporting. Concurrently, a group sessions with two themes emerging, i.e., root cause and reporting. Concurrently, a systematic literature review was performed using NVivo software to assist the researchers systematic literature review was performed using NVivo software to assist the researchers manage over 1000 screened publications. Following a thematic analysis of the data, three manage over 1000 screened publications. Following a thematic analysis of the data, three main themes (Appendix B) emerged from a final cache of 18 publications, i.e., learning main themes (Appendix B) emerged from a final cache of 18 publications, i.e., learning from incidents, precursors and just culture. The five themes informed the structure of a data from gathering incidents, pr instrument ecursors that and supported just cu 34 lture. semi-str The uctur five edthemes in interviewsfor inm the ed the continuing structure of a airworthiness segment of the industry. Following transcription, the data were uploaded to data gathering instrument that supported 34 semi-structured interviews in the continuing NVivo where they were thematically analyzed using the Braun and Clarke [47] framework. airworthiness segment of the industry. Following transcription, the data were uploaded The outputs from the thematic analysis distilled the interview analysis into three main to NVivo where they were thematically analyzed using the Braun and Clarke [47] frame- outputs, i.e., learning from incidents, learning process and learning product. The lower work. The outputs from the thematic analysis distilled the interview analysis into three tier represents the elements the themes were comprised of and the findings are presented main outputs, i.e., learning from incidents, learning process and learning product. The under these headings (Table 5). lower tier represents the elements the themes were comprised of and the findings are pre- sented under these headings (Table 5). Aerospace 2021, 8, 27 14 of 20 Table 5. Summary of results. Learning from incidents (LFI) is a safety management activity with a desired outcome of preventing unwelcome event recurrence. A learning process facilitates a change in knowledge and behavior intended to support LFI. Safety related information arising from the LFI process. 1 2 3 Learning from Incidents Learning Process Learning Product The decision to report an incident can be impacted by the perceived commercial The release of a safe aviation product is In the organizations supporting the study, pressure and the potential for the primary goal all operational it was apparent that incidents are embarrassment associated with making a maintenance and management staff managed with the support of a consistent mistake, amongst front line espouse to. life-cycle methodology. maintenance staff. Learning products that arise from the Identifying and understanding Single-loop learning is a level of learning managed lifecycle of an incident are organizational behavioral and human that can exist in a dynamic operational intended to impart sufficient learning to factors are important elements affecting environment where a “find and fix” ethos prevent recurrence or occurrence of same decisions to report. exists. or similar events. While aircraft manufacturers generally Inadequately resourced investigation and The mandatory human factors provide feedback on notified incidents, follow up of incidents does not support continuation training program is component manufacturers provide less the determination of accurate event considered by study participants to be an feedback with little or no feedback arising causation and measures to prevent effective enabler of double-loop learning. from aviation authorities on submitted similar incidents reoccurring. reports in the jurisdiction of the study. The recognition of the extended impact of Evidence amongst study participants The cost of classroom delivered under-reporting on “levels of learning” is where a review of single and double-loop continuation training is a primary not always a priority in some learning within organizations was not consideration for most organizations. organizations. available during the study. The absence of a potential learning Computer based training is an option No formal requirement for competence in product that results from effective that is under trial by some organizations the areas of learning for managers and reporting is an impediment when but there are concerns amongst accountable persons exists in EU attempting to gauge the effectiveness of operational staff regarding its overall regulation 1321/2014. learning. effectiveness in its current form. Pressure to prematurely close incident No competence requirements for staff reports does not promote thorough event involved in the development or delivery Just culture has a positive impact on causation and measures to prevent of formal human factors continuation reporting rates. similar incidents reoccurring. training programs. Feedback to staff on incident causation factors from an information and learning perspective is important. Poorly designed continuation training syllabi do not support effective learning. Timely follow up to incident reports supports more effective learning outputs from the reporting process. 3.1.1. Learning form incidents—Acquiring, Processing and Storing Data Incident reporting is accepted as a worthwhile activity amongst those participating in the study. This is based on the collective notion that the initiative raises awareness of incidents and potential hazards and can therefore help prevent event recurrence. The authors recognize that awareness is an important component of learning from incidents. Situations do arise where due to lack of report data, it is questionable if all the necessary reports are being submitted as required. Amongst the constraints to making a report are perceived production pressures and the potential embarrassment that could arise from making a mistake and highlighting it [5]. There are just culture concerns amongst some staff because they do not always know what the impact for them personally will be if they submit an incident report [44]. Aerospace 2021, 8, 27 15 of 20 A dedicated focal point in organizations is essential for the systematic management of reported incidents. Where this discipline is applied, the process owner is responsible for highlighting reported issues and raising the necessary awareness amongst operational staff. Once an incident is acquired through the efforts of a reporting system, some form of processing and analysis is necessary. The availability of adequate resources for determining causation and implementing measures to prevent recurrence was identified as a primary point of concern. Perceived premature closure of reports was also highlighted amongst participants. There was a call for improved accountability and transparency on decisions relating to some closure actions. Respondents associate the practice of applying commercial key performance indicators to safety management as shallow efforts are sometimes made by organizations to expeditiously and prematurely close reports on occasion. Incident reporting and safety management initiatives have been in existence for some time. Large repositories of associated safety data are stored in many organizations. Although entities are mandated to inform key stakeholders, there is a strong opinion amongst some partic- ipants that the data repositories could be aggregated and put to better use in support of learning amongst all operators. 3.1.2. Learning Process—Single-Loop, Double-Loop and Deutero Learning The interview data confirms that safety is a primary underpinning value in the or- ganizations that participated in the study. The release of a safe product, i.e., an aircraft or component, is a formative pursuit and measure of learning. In organizational environ- ments where a “find and fix” ethos may prevail, single-loop learning [11] is evident in the examples presented. A desired outcome of double-loop learning [11] is often witnessed for example through the adjustment of environmental, behavioral and procedural norms. Instances of double- loop learning can be evident following unsuccessful attempts through single-loop learning where causation is then adequately understood and actioned. Continuation (mandatory in-service) training was considered by study participants as an effective mechanism that enables double-loop learning. During the study, it was apparent that single and double learning loops are recognized amongst many participants as having differing capabilities in terms of delivering an effective learning product. However, there was no evidence of formal reviews of single and double-loop learning being performed within the participants’ organizations. Although deutero-learning [11,55] may be considered as a natural extension of other levels of learning, the concept did not feature strongly amongst the participants. A review of the EU1321/2014 [6] implementing requirements confirms an absence of any mandatory requirement to review learning processes. 3.1.3. Learning Product—Effectiveness and Types of Knowledge Continuation training is a mandated European requirement [6] for all aircraft main- tenance and continuing airworthiness management organizations. It is a product as well as a medium for imparting learning from incidents and safety related hazards. It was identified during the study that the learning product is shared amongst staff through three primary means of distribution: formally delivered continuation training, tool-box talks and safety briefings and electronic, paper, notice board and “read and sign” safety publications. The study suggests a learning product can arise as a result of an output from an incident lifecycle. Feedback from submitted occurrences to stakeholders varies from very good to poor. Cost is seen as a major consideration in some of the participating organizations when planning continuation training delivery. Although computer-based training is being considered in some companies as a viable option to class-room delivery, concerns are evident in respect of effectiveness of this medium in its current form. Bedwell and Salas [56] suggest computer-based training (CBT) can be used as a methodology for providing, “systematic, structured learning; a useful tool when properly designed”. The perceived overburdening of operational staff with complex learning products and excessive cognitive loads was recorded as an impediment to learning during the study. Aerospace 2021, 8, 27 16 of 20 Participants suggested this can arise from poorly designed training syllabi delivered during periods of high operational activity. Four knowledge types were identified and relate to: conceptual, dispositional, pro- cedural and locative knowledge forms [57]. One of the key objectives of learning from incidents is to identify the type of knowledge needed to prevent an issue recurring. When a reportable issue, for example, is discovered, the submitted report will identify “what” happened. Subsequent follow up will set out to determine “why” the issue occurred. The guiding principles of “how” to perform the task or operation are often contained in proce- dures or data particular to the task. The information contained in procedures will enable a person to utilize other forms of knowledge. Prevailing safety culture within an organiza- tion will have an impact on learning from incidents. If a strong commercial/production culture exists, this may have an impact on, for example, the depth and breadth of learning from incidents within the company. Induction and initial training are important when accessing information for new staff. Accident data repositories contain well-documented human factor-related examples often relating to access to approved data and consequently resulting in potentially preventable incidents. Examining the limitations of each type of knowledge when continuation training programs are being developed was flagged as important by some participants. During the study, no discernible differences were recorded in how the types of knowledge were differentiated in participant organizations. A review of the EU 1321/2014 [6] human factors syllabus requirements did not highlight a need to appreciate or account for these human centered limitations when designing and delivering training lessons. Improved regulatory guidance on the design of effective human factor related material should therefore be developed. Information on how training should be structured in order to appreciate types of knowledge and capitalize on it as a minimum are required to ensure the most efficacious outcome from incident-related training. 4. Conclusions An ameliorating feature of learning from incidents is the potential to effect sustainable improvements in aviation safety. A review of safety from the perspective of maintenance and continuing airworthiness staff is key to understanding the relationship between safety and the concept of learning from incidents [31]. From the study’s qualitative data, we were able to identify how learning occurs in the airworthiness segment, and issues that sup- port and constrain learning. Recurrent mandated training initiatives such as continuation training were found to be pivotal in enabling learning. Aspects such as prevailing culture and poor event causation were noted to have a negative impact on learning. Our pro- posed incident learning process (Figure 2) offers a panoramic of where potential learning opportunities and procedural improvements can arise within the lifecycle of an incident. This perspective could be applied in support of developing regulatory working group specifications and validating continuation training initiatives. In addition, it could also be used to develop a holistic review approach to learning from incidents within other organizations both in the aviation industry and outside. Two notable limitations to our research arise. First, the scarcity of prior studies capable of supporting the basis for the research was pronounced. However, prior studies in parallel domains were successfully leveraged in support of the literature review. Second, the study’s population (n = 34) was relatively small. As the study participants were representative of all affected domain functions and a point of saturation was reached, it was deemed adequate. This research is capable of supporting other papers on additional benefits associated with learning from incidents (LFI). Notably, with the imminent implementation of a safety management (SMS) requirement for continuing airworthiness organizations, potential improvements to hazard identification arising from learning from incidents (LFI) could be highlighted. Author Contributions: Conceptualization, J.C. and K.I.K.; methodology, J.C.; formal analysis, J.C.; investigation, J.C.; validation, J.C and K.I.K.; data curation, J.C.; writing—original draft preparation, Aerospace 2021, 8, 27 17 of 20 J.C. and K.I.K.; writing—review and editing, J.C. and K.I.K. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Institutional Review Board Statement: The study was conducted according to the Institutional guidelines and approved by the Institutional Ethics Committee of University of Limerick (Research Ethics Approval Number: 2015_12_06) dated 6 December 2015. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study. Data Availability Statement: The data presented in this study are available on request from the corresponding author. Conflicts of Interest: The authors declare no conflict of interest. Appendix A Semi-Structured Interview Template Table A1. The Semi-Structured Interview Template used in this study Code 1 Code 2 Previous Positions Years in Previous Positions Position Years in position Qualification Type of organization a. Reporting Could you describe an occurrence/incident that happened recently? How is a report made? Who decides what events to report? Where does the requirement to report come from? How is the importance of reporting highlighted in the organization? What do you think the aim of reporting is? Have you received feedback from reports you have submitted? b. Just culture Do you think there is a good safety culture in the organization? Why is this? Is it easy to communicate with management on safety issues? Do you feel a just culture exists in the company? (Why is that?) How does just culture impact on reporting? c. Learning How are lessons that arise from occurrence/incident reporting delivered to staff in your area? How is learning achieved? (What is the process?) What obstacles to learning from incidents have you experienced in your position? In your opinion, what conditions or developments could improve learning from incidents/occurrences in your organization? d. Root Cause What is your opinion on efforts to establish a single root cause when an incident/occurrence is investigated? Is this approach always effective? What situations have you experienced where incident causes can be numerous and complex? e. Occurrence/Incident Pre-cursors How important is it to identify and report events not required by the mandatory occurrence reporting (MOR) schemes? (Why is this?) Is the organization’s occurrence/ incident reporting system capable of managing reports other than MOR’s? Aerospace 2021, 8, 27 18 of 20 Is there a better way of gathering and using the potential information from non- mandatory events? (What would you suggest?) Appendix B Defining and Naming Themes Table 2. Taxonomy used in defining and naming the themes used in this study. Phase 5—Categories Conceptually Interviews Units of Mapped and Collapsed into 3 Major Code Definitions for Coding Consistency Coded Meaning Coded Themes with 8 Sub-Themes This relates to the three levels of learning suggested LEARNING PROCESS by Bateson (1972) and applied by Argyris and 34 359 Schon (1996) This relates to when members of an organization Deutero-learning reflect on previous learning and thereby setting 26 65 about to improve its learning process. This relates to learning that takes place and Double-loop Learning 26 63 organizational norms and theory in use are changed. This relates to when an organizations’ members Single-loop Learning detect and correct errors but still maintain the 26 63 organizations theory in use. LEARNING PRODUCT This relates to what the learning process delivers 33 235 Effectiveness This relates to measuring effectiveness of learning 31 155 This relates to conceptual, procedural, dispositional Types of knowledge 23 74 and locative knowledge This relates to the inputs necessary to enable the LEARNING FROM INCIDENTS assembly of a learning material in support of 17 213 learning from incidents Processing This relates to how learning information is processed 17 82 This relates to the sources of information that Acquiring 16 55 support learning and how there are gathered Storing This relates to how learning information is stored 12 27 References 1. Drupsteen, L.; Hasle, P. Why do organizations not learn from incidents? Bottlenecks, causes and conditions for a failure to effectively learn. Accid. Anal. Prev. 2014, 72, 351–358. [CrossRef] [PubMed] 2. Chang, Y.-H.; Wang, Y.-C. Significant human risk factors in aircraft maintenance technicians. Saf. Sci. 2010, 48, 54–62. [CrossRef] 3. Silva, S.A.; Carvalho, H.; Oliveira, M.J.; Fialho, T.; Soares, C.G.; Jacinto, C. Organizational practices for learning with work accidents throughout their information cycle. Saf. Sci. 2017, 99, 102–114. [CrossRef] 4. Akselsson, R.; Jacobsson, A.; Bötjesson, M.; Ek, Å.; Enander, A. Efficient and effective learning for safety from incidents. Work 2012, 41, 3216–3222. [CrossRef] 5. Hobbs, A.N. Human Errors in Context: A Study of Unsafe Acts in Aircraft Maintenance. Ph.D. Thesis, University of New South Wales, Kensington, Australia, 2003. 6. European Commission (EU). Commission Regulation (EU) No 1321/2014 of 26 November 2014 on the Continuing Airworthiness of Aircraft and Aeronautical Products, Parts and Appliances, and on the Approval of Organisations and Personnel Involved in These Tasks; European Commission (EU): Brussels, Belgium, 2014. 7. Harvey, C.; Stanton, N.A. Safety in system-of-systems: Ten key challenges. Saf. Sci. 2014, 70, 358–366. [CrossRef] 8. European Commission (EU). Commission Regulation (EU) No 376/2014 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 3 April 2014 on the reporting, analysis and follow-up of occurrences in civil aviation, amending Regulation (EU) No 996/2010 of the European Parliament and of the Council and repealing Directive 2003/42/EC of the European Parliament and of the Council and Commission Regulations (EC) No 1321/2007 and (EC) No 1330/2007; European Commission (EU): Brussels, Belgium, 2014. 9. Gerede, E. A study of challenges to the success of the safety management system in aircraft maintenance organizations in Turkey. Saf. Sci. 2015, 73, 106–116. [CrossRef] 10. Drupsteen, L.; Wybo, J.-L. Assessing propensity to learn from safety-related events. Saf. Sci. 2015, 71, 28–38. [CrossRef] Aerospace 2021, 8, 27 19 of 20 11. Argyris, C.; Schön, D.A. Organizational Learning II: Theory, Method, and Practice; Addison-Wesley Publishing Company: Boston, MA, USA, 1996. 12. Drupsteen, L.; Guldenmund, F.W. What is learning? A review of the safety literature to define learning from incidents, accidents and disasters. J. Contingencies Crisis Manag. 2014, 22, 81–96. [CrossRef] 13. Fogarty, G.J.; Saunders, R.; Collyer, R. Developing a model to predict aircraft maintenance performance. In Proceedings of the 10th International Symposium on Aviation Psychology, Ohio State University, Columbus, OH, USA, 3–6 May 1999. 14. Sms, I. Safety Management Manual (SMM). In Doc 9859; ICAO: Montréal, QC, Canada, 2012. 15. Ward, M.; McDonald, N.; Morrison, R.; Gaynor, D.; Nugent, T. A performance improvement case study in aircraft maintenance and its implications for hazard identification. Ergonomics 2010, 53, 247–267. [CrossRef] 16. Jacobsson, A.; Ek, Å.; Akselsson, R. Learning from incidents–A method for assessing the effectiveness of the learning cycle. J. Loss Prev. Process Ind. 2012, 25, 561–570. [CrossRef] 17. Brunel, I. Sessional Papers Printed by Order of the House of Lords, or Presented by Royal Command; Government of Great Britain: London, UK, 1841. 18. Furniss, D.; Curzon, P.; Blandford, A. Using FRAM beyond safety: A case study to explore how sociotechnical systems can flourish or stall. Theor. Issues Ergon. Sci. 2016, 17, 507–532. [CrossRef] 19. Hollnagel, E. Barriers and Accident Prevention; Ashgate Publishing: Farnham, UK, 2004. 20. Okoli, C.; Schabram, K. A Guide to Conducting a Systematic Literature Review of Information Systems Research; Sprouts Farmers Market: Chandler, AZ, USA, 2010. 21. Houghton, C.; Murphy, K.; Meehan, B.; Thomas, J.; Brooker, D.; Casey, D. From screening to synthesis: Using nvivo to enhance transparency in qualitative evidence synthesis. J. Clin. Nurs. 2017, 26, 873–881. [CrossRef] [PubMed] 22. Bandara, W.; Furtmueller, E.; Gorbacheva, E.; Miskon, S.; Beekhuyzen, J. Achieving rigor in literature reviews: Insights from qualitative data analysis and tool-support. Commun. Assoc. Inf. Syst. 2015, 37, 154–204. [CrossRef] 23. Gough, D.; Oliver, S.; Thomas, J. An Introduction to Systematic Reviews, 2nd ed.; Sage: London, UK, 2017; pp. 83–106. 24. Meline, T. Selecting studies for systematic review: Inclusion and exclusion criteria. Contemp. Issues Commun. Sci. Disord. ASHA 2006, 33, 21–27. [CrossRef] 25. Wienen, H.C.A.; Bukhsh, F.A.; Vriezekolk, E.; Wieringa, R.J. Accident Analysis Methods and Models—A Systematic Literature Review; University of Twente, Centre for Telematics and Information Technology (CTIT): Enschede, The Netherlands, 2017. 26. Maykut, P.S.; Morehouse, R. Beginning Qualitative Research: A Philosophic and Practical Guide; Falmer Press: London, UK, 1994. 27. Cooke, D.L.; Rohleder, T.R. Learning from incidents: From normal accidents to high reliability. Syst. Dyn. Rev. 2006, 22, 213–239. [CrossRef] 28. Drupsteen, L.; Groeneweg, J.; Zwetsloot, G.I. Critical steps in learning from incidents: Using learning potential in the process from reporting an incident to accident prevention. Int. J. Occup. Saf. Ergon. 2013, 19, 63–77. [CrossRef] [PubMed] 29. European Commission (EU). (EU) 2018/1139 of the European Parliment and the Council of 4 July 2018 on common rules in trhe field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, EU376/2014 and Directives 2014/30/EU of the European Parliment and the Council, and repealing Regulation (EC) No 216/2008 of the European Parliment and of the Council and Council Regulation (EEC) No 3922/91; European Commission (EU): Brussels, Belgium, 2018. 30. Lindberg, A.-K.; Hansson, S.O.; Rollenhagen, C. Learning from accidents–What more do we need to know? Saf. Sci. 2010, 48, 714–721. [CrossRef] 31. Lukic, D.; Littlejohn, A.; Margaryan, A. A framework for learning from incidents in the workplace. Saf. Sci. 2012, 50, 950–957. [CrossRef] 32. Steiner, L. Organizational dilemmas as barriers to learning. Learn. Organ. 1998, 5, 193–201. [CrossRef] 33. Taylor, F.W. The Principles of Scientific Management; Harper & Brothers: New York, NY, USA, 1911. 34. Weber, R. Editor ’s Comments: The rhetoric of positivism versus interpretivism: A personal view. MIS Q. 2004, 28. [CrossRef] 35. Guba, E.G.; Lincoln, Y.S. Competing Paradigms in Qualitative Research; Sage Publications: Thousand Oaks, CA, USA, 1994. 36. Walsham, G. Interpretive case studies in IS research: Nature and method. Eur. J. Inf. Syst. 1995, 4, 74. [CrossRef] 37. Oates, B.J. Researching Information Systems and Computing; Sage Publications: Thousand Oaks, CA, USA, 2006. 38. Hirschheim, R. Information systems epistemology: An historical perspective. Res. Methods Inf. Syst. 1985, 9, 13–35. 39. Schwandt, T. Constructivist, interpretivist approaches to human inquiry. Handb. Qual. Res. 1994, 1, 118–137. 40. Kitzinger, J. The methodology of focus groups: The importance of interaction between research participants. Sociol. Health Illn. 1994, 16, 103–121. [CrossRef] 41. Bogardus, E.S. Social distance in the city. Proc. Publ. Am. Sociol. Soc. 1926, 20, 40–46. 42. Frey, J.H.; Fontana, A. The group interview in social research. Soc. Sci. J. 1991, 28, 175–187. [CrossRef] 43. Powell, R.A.; Single, H.M. Focus groups. Int. J. Qual. Health Care 1996, 8, 499–504. [CrossRef] 44. Chatzi, A.V. The diagnosis of communication and trust in aviation maintenance (DiCTAM) model. Aerospace 2019, 6, 120. [CrossRef] 45. Chatzi, A.V.; Martin, W.; Bates, P.; Murray, P. The unexplored link between communication and trust in aviation maintenance practice. Aerospace 2019, 6, 66. [CrossRef] Aerospace 2021, 8, 27 20 of 20 46. Patton, M.Q. Qualitative Evaluation and Research Methods, 2nd ed.; Patton, M.Q., Ed.; Sage Publications: Thousand Oaks, CA, USA, 1990. 47. Braun, V.; Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006, 3, 77–101. [CrossRef] 48. QDA Training. Working with NVivo; QDA Training: Dublin, Ireland, 2013. 49. Lincoln, Y.S.; Guba, E.G. Naturalistic Inquiry; Sage Publications: Thousand Oaks, CA, USA, 1985. 50. Baskerville, R.; Pries, J.H. Short cycle time systems development. Inf. Syst. J. 2004, 14, 237–264. [CrossRef] 51. Hammersley, M.; Atkinson, P. Ethnography: Principles in Practice, 3rd ed.; Routledge: London, UK, 2007. 52. Araujo, L. Designing and Refining Hierarchical Coding Frames. In Computer-Aided Qualitative Data Analysis: Theory, Methods and Practice; Udo, K., Ed.; Sage Publications: London, UK, 1995; pp. 96–104. 53. Birks, M.; Chapman, Y.; Francis, K. Memoing in qualitative research: Probing data and processes. J. Res. Nurs. 2008, 13, 68–75. [CrossRef] 54. Miles, M.B.; Huberman, A.M. Qualitative Data Analysis: An Expanded Sourcebook; Sage Publications: Thousand Oaks, CA, USA, 1994. 55. Bateson, G. The Logical Categories of Learning and Communication. In Steps to an Ecology of Mind; The University of Chicago Press: Chicago, IL, USA, 1972; pp. 279–308. 56. Bedwell, W.L.; Salas, E. Computer-based training: Capitalizing on lessons learned. Int. J. Train. Dev. 2010, 14, 239–249. [CrossRef] 57. Thorndike, E.L. Fundamental theorems in judging men. J. Appl. Psychol. 1918, 2, 67. [CrossRef]

Journal

AerospaceMultidisciplinary Digital Publishing Institute

Published: Jan 22, 2021

There are no references for this article.