University of Southern California Dissertations and Theses

Human error risk reduction in aviation: an evaluation study

(USC Thesis Other)

Human error risk reduction in aviation: an evaluation study

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: HUMAN ERROR RISK REDUCTION IN AVIATION 1

HUMAN ERROR RISK REDUCTION IN AVIATION:
AN EVALUATION STUDY

by
Matthew D. Virtue

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

December 2018

Copyright 2018 Matthew D. Virtue
HUMAN ERROR RISK REDUCTION IN AVIATION 2

TABLE OF CONTENTS
TABLE OF CONTENTS ................................................................................................................2
LIST OF TABLES ..........................................................................................................................4
LIST OF FIGURES ........................................................................................................................5
ABSTRACT ....................................................................................................................................6
Introduction to the Problem of Practice...............................................................................7
Organizational Context and Mission ..................................................................................7
Importance of Addressing the Problem...............................................................................8
Purpose of the Project and Questions..................................................................................8
Organizational Performance Goal.......................................................................................9
Stakeholder Group of Focus and Stakeholder Goal...........................................................10
Methodological Approach.................................................................................................11
Review of the Literature ...................................................................................................12
History and Development of Human Factors in Aviation.....................................12
Human Factors Contribution to Errors..................................................................15
Human Factor Error Reduction Techniques..........................................................15
Human Factors Training Program.........................................................................17
The Clark and Estes Gap Analytic Conceptual Framework..............................................18
Aircraft Technicians’ Knowledge, Motivation and Organizational Influences.................19
Knowledge Influences...........................................................................................19
Motivation Influences............................................................................................21
Organizational Influences......................................................................................25
Interactive Conceptual Framework....................................................................................28
HUMAN ERROR RISK REDUCTION IN AVIATION 3
Data Collection and Instrumentation.................................................................................32
Data Analysis ....................................................................................................................35
Results and Findings .........................................................................................................36
Results and Findings for Knowledge Causes ....................................................................38
Results and Findings for Motivation Causes ....................................................................47
Results and Findings for Organizational Causes ..............................................................51
Summary of Validated Gaps..............................................................................................59
Recommendations for Practice..........................................................................................64
Knowledge and Organization Recommendations.................................................66
Conclusion.........................................................................................................................74
References......................................................................................................................................76
Appendix A: Participating Stakeholders with Sampling Criteria for Interview and Survey.........88
Appendix B: Protocols...................................................................................................................93
Appendix C: Credibility and Trustworthiness...............................................................................98
Appendix D: Validity and Reliability............................................................................................99
Appendix E: Ethics......................................................................................................................101
Appendix F: Integrated Implementation and Evaluation Plan.....................................................103
Appendix G: Limitations and Delimitations................................................................................115
Appendix H: Sample Evaluation Immediately Following Program............................................117
Appendix I: Sample Delayed Follow-Up Evaluation..................................................................120

HUMAN ERROR RISK REDUCTION IN AVIATION 4
LIST OF TABLES
Table 1: Organizational mission, global goal, and stakeholder goals............................................11
Table 2: Knowledge Type and Knowledge Influence...................................................................21
Table 3: Motivational Influences...................................................................................................25
Table 4: Organizational Influences and Organizational Influence Assessments...........................28
Table 5: Knowledge Findings Validated and Not Validated as Gaps ..........................................46
Table 6: Motivation Findings Validated and Not Validated as Gaps............................................50
Table 7: Organizational Findings Validated and Not Validated as Gaps......................................59
Table 8: KMO Findings Validated as Gaps ..................................................................................63
Table 9: Summary of Validated Knowledge Organization Findings and Recommendations.......68
Table F1: Outcomes, Metrics, and Methods for External and Internal Outcomes......................105
Table F2: Critical Behaviors, Metrics, Methods, and Timing for Evaluation.............................106
Table F3: Required Drivers to Support Critical Behaviors.........................................................107
Table F4: Evaluation of the Components of Learning for the Program......................................110
Table F5: Components to Measure Reactions to the Program....................................................111

HUMAN ERROR RISK REDUCTION IN AVIATION 5
LIST OF FIGURES
Figure 1. 65th MXG Technicians’ KMO Conceptual Framework................................................31
Figure 2. Technicians belief in how much human factors training changed their behavior..........41
Figure 3. Technicians belief in human factors awareness training and its potential outcomes.....48
Figure 4. Technicians Confidence in Applying Human Factors Principles...................................50
Figure 5. Supervisor and Co-worker Encouragement to Report Unsafe Conditions.....................53
Figure 6. Technicians comfort level reporting errors within the system.......................................54
Figure 7. Technicians Belief That Leadership Would Act Upon Safety Recommendations........54
Figure 8. Technicians Belief That Leadership Would Not Compromise Safety for Production...57
Figure F1. Level 3 monitoring and accountability dashboard.....................................................114

HUMAN ERROR RISK REDUCTION IN AVIATION 6
ABSTRACT

Aviation safety relies heavily on maintenance technicians to provide safe, reliable aircraft. With
15-20% of all aviation accidents directly attributed to human error in maintenance, many
aviation organizations are trying to find a way to reduce this risk. Not only does this error
contribute to accidents, but there are also countless other related events such as aircraft damage,
personnel injury, and a reduced mission capable status, that can impede an organization’s overall
objective. The purpose of this study was to understand influences related to human error and
aircraft maintenance technicians through the Clark and Estes’ (2008) gap analysis framework.
This study’s assumed influences were the result of an extensive literature review and then
explored through surveys, interviews, and document analysis. The study participants were
aircraft technicians that work on large multi-purpose cargo aircraft within a USAF Maintenance
Group. Data demonstrated that the most significant barrier to reporting human safety concerns
was an organizational culture that endorsed a complacent attitude. The data also determined that
the organizational culture needs to support overwhelmed managers in their role of enforcing
standard safety practices. Additionally, this study found that aircraft technicians need to increase
their understanding of how human factors contribute to errors, how to properly incorporate
decision-making strategies to manage human error, and how to reflect on performance
limitations. The study provides recommendations developed using the New World Kirkpatrick
Model (Kirkpatrick & Kirkpatrick, 2016). The recommendations chosen will help close the
identified gaps, ensuring the technicians have the necessary tools to recognize and mitigate
human error in aircraft maintenance.

HUMAN ERROR RISK REDUCTION IN AVIATION 7
Introduction to the Problem of Practice

Human error is a contributing factor in 70-80% of all aviation accidents (Chen, Chen &
Lin, 2009), with 15-20% being directly attributable to error in maintenance (Begur & Babu,
2016). Aviation relies heavily on human-machine interaction, which invites human factor driven
errors. Although aircraft reliability improvements have reduced the number of accidents
attributed to mechanical failure, human reliability has not progressed to the same degree. With
this lack of progression, the Civil Aviation Authority (Authority, 2008) determined human error
is the primary risk to flight safety. The evidence highlights human error in aircraft maintenance
may lead to damaging consequences and possibly a devastating human toll (Chaturvedi, Craft, &
Kupfer, 2011; Walker, Dolruedee, & Lin 2005). This problem is important to address because
74% of fatal aviation accidents investigated by the National Transportation Safety Board are
directly related to human factors (NTSB, 2016).
Organizational Context and Mission
The 65
th
Maintenance Group (MXG) is assigned to Sterling Air Force Base (SAFB), both
pseudonyms. The mission of the 65
th
MXG is to provide safe and reliable airlift capability
worldwide by means of the C-17 Globemaster III, a heavy multipurpose cargo aircraft. The
mission of the 65
th
MXG ultimately supports the mission of the United States Air Force (USAF)
which is to “fly, fight and win in air, space and cyberspace” (“U.S. Air Force,” n.d.). In order to
ensure the C-17 is ready and capable of meeting its mission, SAFB has over 1,200 skilled aircraft
maintenance technicians assigned to the 65
th
MXG. These aircraft technicians work around the
clock inspecting, servicing, and repairing the 53 assigned C-17 aircraft to be ready at a moment’s
notice. The goal of an aircraft technician, whether associated with a military organization,
civilian airline, or general aviation, is to provide a safe, airworthy aircraft.
HUMAN ERROR RISK REDUCTION IN AVIATION 8
Importance of Addressing the Problem
Although aviation safety has improved tremendously over the past fifty years,
maintenance related errors have remained constant (Marais & Robichaud, 2012). These errors
are partially due to aviation maintenance being part of a complex system that has time pressures,
organizational challenges, and economic variables that contribute to error producing actions
(Latorella & Prabhu, 2000). Given the complexity of this industry, problems may lay dormant
for years after aircraft assembly or repair only to surface at the most inopportune time (Garrette,
Castañer & Dussauge, 2009). Aircraft damage is typically a result of human error and can also
be a precursor to a more serious event. If this problem of human error in aircraft maintenance is
not solved, an increase in aviation fatalities is inevitable (Borenstein & Rose, 2014; Findlay &
Harrison, 2002), tarnishing the aviation community’s reputation for safety (Andersen, McNay &
Peterson, 2012).
Purpose of the Project and Questions

The purpose of this project is to evaluate the degree to which the 65
th
MXG is meeting its
goal of reducing aircraft damage from $1.7 million to below $1.53 million. The 65
th
MXG
tracks aircraft damage and has found that over 70% can be considered preventable and a result of
human factor related errors. The 65
th
MXG recognizes that human error in aircraft maintenance
plays a significant role in aircraft damage and has instituted an awareness course to reduce these
human induced errors. The Clark and Estes (2008) gap analysis framework was used to identify
and analyze existing knowledge, motivation, or organizational influences preventing goal
achievement and to generate practical solutions to performance gaps. While a complete
performance evaluation would focus on all stakeholders, for practical purposes, the stakeholder
HUMAN ERROR RISK REDUCTION IN AVIATION 9
of interest in this analysis are the 65
th
MXG technicians who have direct contact with assigned
aircraft.
In an attempt to uncover safety minded behaviors within the 65
th
MXG, the research
questions guiding the study include the technicians’ knowledge, motivation, and organizational
influences related to demonstrating proficiency in human factor related error reduction
techniques and the interaction between the 65
th
MXG culture and context and technicians’
knowledge and motivation.
1. What is the 65
th
MXG technicians’ knowledge and motivation related to demonstrating
proficiency in human factor related error reduction techniques?
2. What is the interaction between the 65
th
MXG’s culture and context and technicians’
knowledge and motivation related to demonstrating proficiency in human factor related
error reduction techniques?
3. What are the recommendations for the 65
th
MXG’s organizational practice in the areas of
knowledge, motivation, and organizational resources?
Organizational Performance Goal

The 65
th
MXG’s goal for 2019 is to reduce aircraft damage from $1.7 million to below
$1.53 million. One of the largest contributing factors to aircraft damage at the 65
th
MXG is
human error in aircraft maintenance. Aircraft damage as a result of human error represents an
added cost to the 65
th
MXG, a possible gap in training, and an inability of technicians to
recognize work-related hazards. Furthermore, minor aircraft damage events resulting from
improperly mitigated work-related hazards can be a precursor to a more significant incident or
accident that would have a crippling effect on the 65
th
MXG meeting its mission. The 65
th

HUMAN ERROR RISK REDUCTION IN AVIATION 10
MXG’s quality assurance department determined that a 10% reduction in aircraft damage was
appropriate through the calculation of previous year’s damage events which totaled $1.7 million.
Stakeholder Group of Focus and Stakeholder Goal

To comprehend the effectiveness of human factors awareness training on reducing human
error in aircraft maintenance, a complete analysis would involve all stakeholder groups. The
successful acquisition, comprehension, and implementation of human factors training is
important in reducing aircraft damage; therefore, the stakeholders of focus for this study were the
65
th
MXG technicians who have direct contact with assigned aircraft. Understanding the
benefits of human factors awareness, the 65
th
MXG leadership team has established a goal that
by November 2018, 100% of the 65
th
MXG technicians will demonstrate proficiency in human
factor related error reduction techniques through performance-based testing. If the human
factors training principles are not understood and applied, the 65
th
MXG technicians may be
unable to recognize individual capabilities and limitations leading to aircraft damage or possibly
an incident or accident. Table 1 shows the organizational mission, global goal, and stakeholder
goals.

HUMAN ERROR RISK REDUCTION IN AVIATION 11
Table 1
Organizational Mission, Global Goal, and Stakeholder Goals
Organizational Mission
The mission of the 65
th
Maintenance Group is to provide safe, reliable
airlift capability worldwide.

Organizational Performance Goal
By December 2019, the 65
th
Maintenance Group will reduce aircraft damage
from $1.7 million to below $1.53 million.

65th MXG technicians having
direct contact with assigned
aircraft
By December 2019, 100% of 65
th

MXG technicians will demonstrate
proficiency in human factor related
error reduction techniques through
performance-based testing.

65
th
MXG training development
team
By July 2018, the training
development team will incorporate
a human factor based error
reduction training component into
100% of aviation related training
courses offered.
65
th
MXG leadership team
By November 2018 the 65
th

MXG leadership team will
ensure 100% of technicians
complete all required training
tracked through the 65
th
MXG
Learning Management System
(LMS).

Methodological Approach
Although the purpose of this study was to evaluate the degree to which the 65
th
MXG is
meeting its goal of reducing aircraft damage, a significant precursor of this goal is the
effectiveness of the human factors awareness training program. Increased effectiveness of
human factors awareness correlates directly to the reduction of human error in aircraft
maintenance, thus helping the 65
th
MXG achieve its goal of reducing aircraft damage below a
predetermined level. As this study attempts to gauge the degree of human factors awareness
training success in relation to aircraft damage caused by human induced error, a mixed-methods
approach helped gain the most understanding of the projects purpose (Creswell, 2014; Locke,
Silverman & Spirduso, 2010; Salkind, 2017). Using an explanatory sequential method, this
descriptive process-based research first established quantitative evidence followed by purposely
HUMAN ERROR RISK REDUCTION IN AVIATION 12
selected qualitative interviews to further understand the results (McEwan & McEwan, 2003;
Creswell, 2014). To ensure accuracy of the results, pre-selected members of the qualitative
study participated in a follow-up interview; this member-checking technique gave participants an
opportunity to comment on the findings (Creswell, 2014). Finally, this mixed methods approach
helped increase reliability and credibility through the triangulation of evidential results and also
helped develop a complete understanding of the technicians learning outcomes by interacting and
gathering data directly to better understand their perspective.
Review of the Literature

As aircraft have become more complex, the cause of aviation accidents has also changed.
Early on mechanical failure was the predominant factor in most incidents and accidents within
aviation, but as aircraft reliability improved, human error has become the leading cause. To help
minimize human error, the aviation industry, as well as most military aviation units,
implemented training programs to improve team interaction, communication, and to increase
individuals’ awareness of their role in safety (Glendon & Clarke, 2015). These training
programs focus on the organizations management and aircraft technicians working together to
use existing resources to eliminate human error and encourage safety. The overall training
success lies in both the technicians’ and managers’ ability to identify and reduce factors
contributing to potential accidents.
History and Development of Human Factors in Aviation

Although Orville and Wilbur Wright were the first to pioneer many human factors
considerations in aviation, it became critical during World War II when aircraft design had to
consider the limitations and capabilities of the human. Even as aircraft design incorporated the
human, more crews were lost in World War II to their own errors than from enemy fire (Batteau,
HUMAN ERROR RISK REDUCTION IN AVIATION 13
2001). After World War II this error trend continued into the jet age as a large number of errors
resulting in accidents were attributed to failures of aircrew coordination. More specifically, in
the 1980s, NASA discovered a significant portion of aviation accidents were directly related to
the captain’s decisions, a tone that had been set and reinforced in the cockpit for decades
(Helmreich, Merritt & Wilhelm, 1999). In an attempt to regulate some of these errors, the
aviation industry introduced Crew Resource Management (CRM) that focused on teamwork and
safety and taught aircrews to use all available resources to communicate and coordinate as a
team (Salas, Bowers & Edens, 2001). By reducing the once standard ranking in the cockpit and
learning how to communicate through open, authentic exchanges of information, the crew
together began to ensure safe operation of the aircraft (Hagen, 2018).
Maintenance Resource Management (MRM) marked the next logical step within human
factors based safety awareness training. MRM concentrated specifically on aircraft maintenance
and was a result of the Aloha Airlines accident in 1988 (Patankar & Taylor, 2008). It was during
the investigation of this accident that the NTSB found systemic maintenance issues, such as
human performance limitations, teamwork, communication, culture, and training deficiencies
that needed to be addressed to minimize future accidents (Airlines, 1989). Prior to this Aloha
Airlines accident, it was widely assumed that if a technician made a mistake, it was a matter of
individual failure and typically addressed by disciplining the individual. The maintenance
industry responded to the Aloha accident by acknowledging the need for systemic solutions and
focused development of MRM, awareness training that focused on human factors. MRM
training programs proved useful in raising awareness about human performance limitations,
individual professionalism and interpersonal trust (McKenna, 2002; Shanmugam & Paul 2015).
As MRM and CRM evolved, it became apparent that all errors could not be eliminated
HUMAN ERROR RISK REDUCTION IN AVIATION 14
and a focus on minimization and mitigation was needed. Threat and error management (TEM)
resolved this need through an all-encompassing safety model involving aviation operations and
human performance helping the user understand the relationship between safety and human
performance in a demanding aviation environment (Helmreich, Merritt & Wilhelm, 1999). The
goal of TEM is to reduce risk by training the user to detect and mitigate events likely to cause
damage or result in an error triggering an adverse outcome, a key component in today’s human
factors awareness philosophy.
Traditionally, aviation safety has been reactive in that analyzation occurs after a mishap
to recognize causal factors and predict future occurrences. While this reactive process has
worked well, the next iteration in aviation safety emphasizes a proactive and predictive approach
in the identification of hazards prior to the needs of a reactive technique (FAA, 2010). This next
step, commonly referred to as safety management system (SMS), is a structured approach to
managing safety which embodies the entire system to include the organization and its structure
(ICAO, 2009). SMS encompasses four pillars: safety policy, risk management, safety assurance,
and safety promotion. This active risk management system measures and moderates operational
risk and improves the overall safety culture (Stolzer, 2017). The SMS structured approach looks
at aviation safety through a human factors lens to achieve tolerable levels of risk through
management’s commitment, hazard identification, risk assessment and mitigation (Liou, Yen &
Tzeng, 2008).
On January 8th, 2015 the FAA issued a final rule requiring part 121 operators
(commercial airlines) to develop and implement a “formal, top-down organization-wide
approach to managing safety risk and assuring the effectiveness of safety risk controls” (Safety
Management System, 2016, “What is a SMS,” para. 1). This SMS requirement brings U.S.
HUMAN ERROR RISK REDUCTION IN AVIATION 15
carriers in line with the rest of the world's airlines, but more importantly, it allows these carriers
to better define and communicate their safety goal to their employees, enhance the identification
and correction of risks, and better define employee training to include human factors awareness
(Bottani, Monica & Vignali, 2009).
Human Factors Contribution to Errors

Tasks in safety-critical environments, such as aircraft maintenance, frequently require
high levels of coordination for decision-making to be effective (Bearman, Paletz, Orasanu &
Thomas, 2010). This requirement for coordinated decision-making is partly due to the
demanding environmental settings that contribute to procedural errors and may include
conditions that condone or encourage deliberate violations. Unlike skill-based slips, where the
technician may have a large number of steps in a process that cause cognitive overload, or rule-
based slips, where the task demands conscious and unconscious processes to determine a
response, violations depend on the factors influencing the behavior of the technician at that
moment. To eliminate these deliberate violations a recognition of the invisible demands and
pressures facing maintenance personnel must be understood (Chiu & Hsieh, 2016). One way to
determine the degree of risk and establish necessary methods needed to minimize the potential
for human error is through the efforts of everyone in the system employing risk identification
(Chang & Wang, 2010).
Human Factor Error Reduction Techniques
Integrating human factors into design. Designers tend to focus on the product and its
functions, seldom considering its use from the human perspective. An analysis of all significant
industrial incidents and accidents show that the complex interaction of technical design features,
individual hazardous actions, poor team performance, inappropriate management, and ill-
HUMAN ERROR RISK REDUCTION IN AVIATION 16
structured organizations lead to human error related accidents (Wilpert, 2008). The
understanding of design error in high-stakes domains has produced new knowledge that
increases our capability of diminishing error (Roesler, 2009). However, this new knowledge has
resulted in the establishment of more automated systems that provide capabilities that exceed
operators’ need for effective system operation providing interfaces that can hinder, rather than
enhance situational awareness (Endsley, 2017). Fortunately, Strauch (2017) found that
integrating automation with normal operator expertise levels, and within training programs that
provide operators the knowledge of automation required, can reduce opportunities for
automation-related operator errors.
Organizational safety culture. An organization's safety culture embodies both
individuals and the company, and must successfully address both attitudes and structure to
achieve high levels of safety compliance and participation. Zohar (1980) defined safety climate
as a summary of perceptions that employees share about their work environment and has become
one of the most commonly used leading indicators of safety (O’Connor, O’Dea, Kennedy &
Buttrey, 2011; Schwatka & Rosecrance, 2016). Kapp (2012) explained the impact an
organizations leadership has on its safety climate where higher levels of transformational and
contingent reward leadership are both associated with more significant levels of safety
compliance and safety participation behavior. Furthermore, Dekker (2009) speaks of a “just
culture” (p.1) where an incident should be considered a free lesson and a great opportunity to
focus and learn collectively. This idea of a just culture helps ensure trainees are aware of the
importance of reporting incidents and develops them to see that incidents are not something
shameful but good for the entire organization.
HUMAN ERROR RISK REDUCTION IN AVIATION 17
Human factors awareness training. Multidimensional team training that integrates
group interaction in complex, authentic situations and followed by after-action reviews can
facilitate teamwork and increase overall safety (Halpin, 2008; Littlepate et al., 2016;
Tannenbaum & Cerasoli, 2013). Patankar and Taylor (2017) described a typical human factors
awareness training that includes components such as the dirty dozen (to include safety nets),
accident case analysis (focusing on the chain of events), an organization-specific problem (top
driver needing immediate attention), and interactive exercises (illustration of learned concepts).
Human factor awareness training has been found useful at building awareness of performance
limitations, increasing assertiveness and stress management, reducing lost time due to injury,
decreasing aircraft damage, and improved morale resulting in increased organizational safety
climate. However, human factors awareness training is only as effective as the organization
implementing it wants it to be (Towler, Watson & Surface, 2014).
Human Factors Training Program
Training effectiveness. Evaluating training effectiveness in the absence of reliable data,
or weak organizational norms, will not give organizations the necessary tools to improve their
safety climate (O’Connor, O’Dea, Kennedy & Buttrey, 2011; Marquardt, Robelski, & Jenkins,
2011). The aviation industry is a rigid, highly organized infrastructure that relies heavily on
written rules, legal regulations and extensive task knowledge. Bowen (2013) explained that
attempts to address the human component of accidents and errors that do not address this highly
regulated environment will have negligible success. Additionally, if norms are present that
conflict with or discourage change intentions, those change intentions may not materialize even
with well-designed training (Bowen, 2013). Furthermore, Taylor (1998) measured a large group
of aircraft maintenance technicians and found they were extremely individualistic and that the
HUMAN ERROR RISK REDUCTION IN AVIATION 18
teamwork and communication elements of human factors training were not likely to be
accomplished through typical classroom instruction suggesting the use of behavior modeling and
skills training to improve the transfer of new and challenging habits. To ensure a human factors
awareness program is adequately developed and implemented, an organization must include a
component of evaluation to verify its compatibility with the organization's culture and setting.
The Clark and Estes Gap Analytic Conceptual Framework

Clark and Estes (2008) gap analysis framework is a proven problem-solving model that
generates practical solutions to performance gaps by identifying and analyzing existing
knowledge, motivation, or organizational influences preventing goal achievement. This gap
analysis framework was used throughout this study to evaluate and understand what gaps exist
between the 65
th
MXG performance goals and actual performance. Clark and Estes asserted that
organizational goals would only be reached when the gap between actual performance level and
the performance goal is closed.
To better understand what is influencing the maintenance technicians’ performance, it is
important to first understand what is meant by knowledge, motivation and organizational
influences. Knowledge and skills, as identified by Krathwohl (2002) within his two-dimensional
taxonomy, include four types of knowledge influences: factual, conceptual, procedural, and
metacognitive. Knowledge influences are used to determine stakeholders understanding of how
to achieve a performance goal. Motivation influences help propel success and are necessary to
accomplish goals. Goal attainment is reached by actively choosing to participate in the goal,
persisting in such activity, and exerting the necessary mental effort to accomplish the target.
(Clark & Estes, 2008; Rueda, 2011; Schunk, Meece & Pintrich, 2012). Guiding motivational
principles such as attributions, self-efficacy, values, and goals must also be considered when
HUMAN ERROR RISK REDUCTION IN AVIATION 19
analyzing performance gaps (Rueda, 2011). Finally, organizational influences on stakeholder
performance may include resources, work processes, and culture (Clark & Estes, 2008). Clark
and Estes (2008) stress that performance gaps will only be closed and goals will only be reached
when knowledge, motivation and organizational influences are adequately addressed.
Aircraft Technicians’ Knowledge, Motivation and Organizational Influences

Knowledge Influences
In order to be successful in goal attainment, it is important to understand how needed
knowledge fits into the 65
th
MXG technician’s goal and why it is important to demonstrate
knowledge of human factor related error reduction techniques. For 65
th
MXG technicians, it is
important to understand the relationship between human factors and safety, using Krathwohl’s
(2002) classification of conceptual knowledge. Conceptual knowledge is notable because it
connects basic components within a larger structure that allows them to work collectively. Once
the technicians understand the theoretical framework behind human factors, they need to learn
how to apply these concepts to operational situations using Krathwohl’s (2002) classification of
procedural knowledge. Procedural knowledge is important because it shows the technicians how
to do something. By categorizing knowledge influences into one of the four knowledge types, a
universally understood standardized structure across many disciplines is born (Krathwohl, 2002).
Human factors contribution to errors. The 65
th
MXG technicians need to understand
how human factors contribute to errors in their work environment. The integration of human
factor principles is necessary to reduce errors because it enhances the interface of individuals
within their work environment, aircraft availability, and overall operational safety (Harris & Li,
2011; Vogt, Leonhardt, Koper & Pennig, 2010; Ahmadi, Söderholm & Kumar, 2010). Human
factors principles help shape the safety culture of aviation maintenance and that the mere
application of human factor awareness has spread beyond just the man-machine interface
HUMAN ERROR RISK REDUCTION IN AVIATION 20
(Shanmugam & Paul, 2015). Without knowledge of human factors and human imperfection, a
65
th
MXG technician’s likelihood of committing errors is increased. This increase is primarily
due to not understanding how to predict, prevent, and eliminate active errors (Gluyas &
Morrison, 2014; Wachter & Yorio, 2013). The technicians require basic elements of human
factors to function safer within their environment; this conceptual knowledge helps them
understand and implement methods to optimize the interactions of humans within a system to
improve performance.
Decision-making strategies in human error. The 65
th
MXG technicians need to know
how to incorporate decision-making strategies to manage human error. Foundations of human
error are rooted in the limitations characteristic of the human cognitive process (Gluyas &
Morrison, 2014). When stressed, distracted, or pressured the technician’s cognitive processes
become overloaded and fail to incorporate decision-making strategies correctly, ultimately
leading to increased error (Blouin, Deaton, Richard & Buza, 2014; Dostaler, 2010; Gluyas &
Morrison, 2014). There are many safety nets available to combat these limitations; one is to
increase the technician’s awareness of human fallibility emphasizing situations where the
possibility of error is increased. Improving the technician’s knowledge of human deficiencies
improves their ability to incorporate decision-making strategies to manage human error (Reid-
Searl, Moxham & Happell, 2010; Tomás, Cheyne & Oliver, 2011). Technicians ability to
incorporate human error management strategies involves procedural knowledge required to
improve performance. This procedural knowledge encompasses the expectancy, avoidance, and
recovery from mistakes, especially when error-free execution is paramount (Krathwohl, 2002;
Wachter & Yorio, 2013).
Self-evaluation of performance limitations. 65
th
MXG technicians need to reflect on
HUMAN ERROR RISK REDUCTION IN AVIATION 21
their performance limitations on the job. This self-evaluation can be thought of as metacognition
giving technicians the ability to know what they did which is crucial for achieving personal goals
and making appropriate decisions (Flavell, 1979). Having the capacity to understand capabilities
and limitations is particularly important in critical time and safety sensitive decision-making
(Hauser, Allen, Rees, Dolan & NSPN Consortium, 2017). By using metacognitive strategies
technicians can increase learning performance, improve understanding, and enhance self-
regulation to better recognize limitations on the job (Brycz & Karasiewicz 2011; Cetin, Sendurur
& Sendurur, 2014; van & Veenman, 2014). Table 2 shows the knowledge influences identified.
Table 2
Knowledge Type and Knowledge Influence

Motivation Influences

Fundamental to the 65
th
MXG technician goal achievement is the motivation to
understand and apply decision-making strategies to manage human error. Motivation helps
propel success and is necessary for the technicians to accomplish their goals. Without
motivation, the technician’s commitment, determination, and capacity to execute error-reducing
skills are strained. According to Johnson (2015), employee performance encompasses ability
and motivation, where ability constitutes skills, training, and resources, combined with
motivation acting as the force driving the employee. Rueda (2011) explains that motivation
Knowledge Type Knowledge Influence

Declarative
(Conceptual)
Technicians need to understand how human factors contribute to errors in
their work environment.

Procedural
Technicians need to know how to incorporate decision-making strategies
to manage human error.

Metacognitive
Technicians need to know how to reflect on their performance limitations
on the job.
HUMAN ERROR RISK REDUCTION IN AVIATION 22
directed through instigation and sustainment influences both internal and external factors. These
factors can be further divided into; active choice (deciding to choose an activity over another),
persistence (commitment over time), and effort (mental work needed to accomplish a task)
(Schunk, Meece & Pintrich, 2012).
Motivation plays a significant role in the technician’s quest for goal attainment. A
potential factor affecting the 65
th
MXG technician’s motivation is a need to see the usefulness of
human factors awareness training. This need is highlighted within Eccles (2006) expectancy-
value theory which unravels the questions “Can I do the task?” and “Do I want to do the task?”
where answering no to either result in a lack of engagement. Another factor affecting motivation
is the technicians need to have confidence in their ability to understand and apply human factors
principles on the job. Bandura (1991) explains that this confidence, or self-efficacy, helps
advance the result one imagines. In other words, if an individual is confident in a task they will
expect the outcome to be successful (Bandura, 1997).
Expectancy value theory. Expectancy-value theory (EVT) demonstrates how
motivation of a learner influences how much value they place on a goal and if they expect to
accomplish that goal. As noted earlier, Eccles (2006) EVT queries “Can I do the task?” and “Do
I want to do the task?” where answering yes to the capability question ties into learner’s
confidence, and the desire question considers value. Eccles (2006) further described how an
individual can find value in a task through intrinsic value (the expected enjoyment while engaged
in a task), attainment value (relationship between a task and preference of an individual), utility
value (how well an individual's goals relate the task at hand), and cost value (how much time and
energy will be lost if engaged in other activities). Eccles (2006) two basic questions are
fundamental to learning, in that answering no to either results in reduced motivation and a lack
HUMAN ERROR RISK REDUCTION IN AVIATION 23
of engagement.
Technicians’ expectations and values. 65
th
MXG technicians need to see the
usefulness of human factors awareness training as it relates to behavior and performance on the
job. The goal of human factors awareness training is to influence individuals' skills,
understanding, and attitudes to improve the overall effectiveness of the individual and
organization (Shenge, 2014). The technician’s need to see value in the training which is
determined by how well the training fits into the technician’s goals, if not the work necessary to
achieve the desired results may not be enough to influence their effort (Eccles, 2006).
Technicians must perceive human factors training as a high priority as this engagement is
a crucial defense against recognizing latent organizational failures and conditions leading to error
(Wachter & Yorio, 2013). Research shows that increasing awareness of error probability and
human fallibility reduces the chance of error (Gluyas & Morrison, 2014; Habtoor, 2016; Reid-
Searl, Moxham & Happell, 2010). By providing the needed human factors awareness support at
the individual level, the technicians will better see the value in training not only for themselves
but the organization as a whole.
Self-efficacy theory. Self-efficacy is the belief people have about themselves that
influence internal ideas of oneself and how they operate (Pajares, 2006). These ideas individuals
hold about their ability to understand and implement particular concepts help shape their
capabilities (Pajares, 2006). Individuals with low self-efficacy tend to avoid activities that seem
difficult and challenging because they do not believe they will achieve the desired results. Low
self-efficacy also brings about a belief that things are harder than they are which creates anxiety
and stress that can lead to depression. One benefit of having high self-efficacy is that it helps
individuals approach difficult tasks with a feeling of confidence and a level of assurance not
HUMAN ERROR RISK REDUCTION IN AVIATION 24
experienced by someone with low self-efficacy (Bandura, 1997; Schunk & Pajares, 2009).
Technicians’ self-efficacy. Technicians need to have confidence in their ability to
understand and apply human factors principles on the job. Yeo and Neal (2006) found that
individuals with a high level of self-efficacy tend to learn faster than those with low self-
efficacy. These high levels of self-efficacy improve learning because individuals with high self-
efficacy set higher goals, use more effective problem-solving strategies, and endure in
challenging situations (Bandura & Locke, 2003). However, some argue that high levels of self-
efficacy can lead to excessively high optimistic performance capabilities, resulting in the
misperception of actual ability contributing to increased errors (Schmidt & DeShon, 2010;
Vancouver, Thompson, Tischner & Putka, 2002; Yeo & Neal 2006). Although considerable
research on self-efficacy as it relates to “self-debilitating” capabilities is available, it does not
change converging evidence that perceived self-efficacy increases motivation and improves
overall performance (Bandura & Locke, 2003). This increased motivation and performance
helps technicians understand complexities within human factors and will aid in the execution of
associated principles on the job.
65
th
MXG technicians will also be more intrinsically motivated if they find their training
useful and relevant. This perceived authenticity was recognized in Radovan and Makovec’s
(2015) correlation analysis which connected theory with application resulting in increased
motivation. Radovan and Makovec (2015) also found that by ensuring the learning environment
was relevant and practical that the students set personal intrinsic goals reporting higher levels of
self-efficacy thus increasing motivation and overall performance. Table 3 shows the
motivational influences.

HUMAN ERROR RISK REDUCTION IN AVIATION 25
Table 3

Motivational Influences

Organizational Influences

The achievement of performance goals depends on more than knowledge and skills and
motivation; organizational causes also influence an organization’s capacity to achieve goals
(Clark & Estes, 2008; Rueda, 2011). Organizational causes can include work processes,
employee interaction with equipment and resources, materials, and the supplies and equipment
needed to accomplish stated objectives (Clark & Estes, 2008). Not only do inadequate processes
and materials prevent the realization of performance goals but according to Clark and Estes
(2008), organizational culture influences can also affect performance. Organizational culture is
the identity of an organization and points to the events that are occurring beneath the surface
(Schein, 2006). Schein (2006) explained that an organizations culture forms as groups make
sense and manages their environment, solutions that worked so well they were endorsed and
passed on. Schein (2006) goes on to explain that an organizations culture can be broken down
into three levels: 1) artifacts (surface level aspects easily discernible yet difficult to understand),
2) espoused values (conscious goals and beliefs that are first shared then become assumed), and
3) basic assumptions and values (core of the culture that evolves as solutions are repeated
becoming unconscious functions that supersede individuals’ rights). By understanding these
many complexities associated within an organizations culture, the identification of missing or

Assumed Motivation Influences

Utility Value - Technicians need to see the usefulness of human factors awareness training as it
relates to behavior and performance on the job.

Self-Efficacy - Technicians need to have confidence in their ability to understand and apply
human factors principles on the job.
HUMAN ERROR RISK REDUCTION IN AVIATION 26
poorly defined processes and inadequate materials causing performance goal hindrance, even
when motivation and knowledge and skills are present it is necessary to adequately identify and
correct poor work processes that are resulting in a performance gap (Clark & Estes, 2008).
In the identification of organizational influences on performance, Rueda (2011) suggests
looking at both cultural settings and cultural models. Cultural models are the covert values,
beliefs, and attitudes that are typically automated whereas cultural settings are evident and
tangible indicators of cultural models that are shaped by individuals or groups. More
specifically, cultural models influence trust, feelings towards change, and accountability where
the cultural setting focuses on the presence of goals, incentives, feedback, and communication.
In looking at the cultural models and setting, the delineation of specific influences impeding goal
attainment at the 65
th
MXG can begin.
Organizational cultural model influences. It is important to understand the degree to
which the 65
th
MXG technicians resist existing practices, such as the following of aircraft
technical manuals and established standard safety protocols, as well as understand if there is an
unwillingness to highlight human factor related safety concerns that would affect their
performance. When considering cultural model influences at a safety critical organization such
as the 65
th
MXG, a crucial look into their safety culture is necessary. An organization’s safety
culture can be defined as a summary of perceptions that employees share about their work
environment (Zohar, 1980), it embodies both individuals and the company, and must
successfully address both attitudes and structure to achieve high levels of safety compliance and
participation. Traditionally, the 65
th
MXG has based their safety performance on lagging
indicators such as fatalities, or incident rates. However, a common leading indicator to consider
is safety climate (O’Connor, O’Dea, Kennedy & Buttrey, 2011). Based on preliminary data from
HUMAN ERROR RISK REDUCTION IN AVIATION 27
the organization, the 65
th
MXG has introduced a human factors awareness training course in an
attempt to improve their safety climate, which emphasizes the need for employees to highlight
error producing conditions. In order for this training to be effective, the 65
th
MXG employees
must see the usefulness behind highlighting human factor related safety concerns rather than
resist it. Grama and Todericiu (2016) explained that this resistance may be a result of the
employees not understanding the reason behind the change and are distrustful regarding the
motivation behind it. By failing to highlight safety concerns technicians and the organization
miss an opportunity to focus attention on unknown risks and the possibility of eliminating error
producing threats throughout the entire organization (Dekker, 2009).
Organizational cultural setting influences. It is important to understand the degree to
which managers are overwhelmed by schedule restraints brought on by flying schedule pressure
and reduced personnel levels that may keep them from enforcing standard safety practices. The
aviation industry has traditionally been managed through a rigid management philosophy,
entailing open loop, preplanned tactics and comprehensive procedures (Besnard & Hollnagel,
2014). Within this ‘‘engineering management’’ thinking, inconsistency in performance is treated
as a symptom and a concession in safety (Reason, 1990). Through the system’s life, various
modifications and obstacles are added to seemingly improve performance which has proved
effective to a certain level of complexity. However, in highly complicated work environments,
such as aircraft maintenance, combining schedule pressure with these modifications results in a
complex system causing disorganization and error (Pentland & Feldman, 2008). According to
Chang and Wang (2010), 69% of aircraft maintenance technicians felt that it was sometimes
necessary to bend the rules to complete a job. These departures from prescribed procedures are
conscious decisions and can be effectively represented as competing decision-shaping factors
HUMAN ERROR RISK REDUCTION IN AVIATION 28
that involve more than knowledge and adherence to prescribed procedures (Savioja, Liinasuo &
Koskinen, 2014). Research in the factors that shape decisions of aircraft maintenance
technicians indicate that 72% of process deviation can be attributed to schedule pressure
(Nathanael, Tsagkas & Marmaras, 2016). When management is driven by schedule pressure,
they allow such violations and deviations to occur indicating an artificial endorsement of the
technician’s actions which is evidence that violations are vulnerable to managements influence
(Alper & Karsh, 2009). Table 4 shows the assumed organizational influences.
Table 4
Organizational Influences and Organizational Influence Assessments

Interactive Conceptual Framework
A conceptual framework is used to understand concepts, ideas, models, or theories that
inform the study (Merriam & Tisdell, 2016). This framework draws from the identified literature
that helps focus the topic, pinpoint its importance and specify the purpose of the study (Merriam
& Tisdell, 2016). Maxwell (2013) explained that an important attribute of the conceptual
framework is its ability to highlight existing knowledge on the subject, how it is interacting with
current practices, and why. By capturing and understanding available information on the

Assumed Organizational Influences

Cultural Model Influence 1 - Organizational culture needs to be conducive to changing existing
processes and procedures as they evolve.

Cultural Model Influence 2 - Organizational culture needs to be supportive of technicians
highlighting human factor related safety concerns that could affect
their performance.

Cultural Setting Influence 1 - Organization culture needs to support managers that are
overwhelmed by schedule restraints that keep them from enforcing
standard safety practices.

HUMAN ERROR RISK REDUCTION IN AVIATION 29
particular topic, one is better able to evaluate goals, develop relevant questions, and justify their
research (Maxwell, 2013).
Although potential influences were presented independently, it is important to understand
that they do not remain in isolation. On the surface, it may seem that differing stakeholder
groups operate independently in an effort to accomplish separate goals. However, without
deeply rooted interaction between stakeholder groups within the 65
th
MXG, the overall
organizational goal of providing safe, reliable aircraft cannot be realized. Furthermore, in order
to meet this organizational goal, intermediate goals must be fulfilled as well. This inter-
organizational collaboration, as defined by Keyton, Ford, and Smith (2008), is the
communicative processes multiple stakeholders engage in when working to address problems.
When first approaching this study, the identification of knowledge and skills, motivation,
and organizational elements influencing the technicians became evident. After over two decades
of working in and around various aviation maintenance organizations of all types, the researcher
realized that the technicians in question needed to see the usefulness of human factors awareness
training as it relates to behavior and performance before being receptive to this instruction. If the
technicians do not see the usefulness or value, the desired results may not be enough to influence
their effort (Eccles, 2006). Technicians also need to have confidence in their ability to
understand and apply human factors principles on the job. Yeo and Neal (2006) found that
individuals with a high amount of self-efficacy tend to learn faster than those with low self-
efficacy. These high levels of self-efficacy improve learning because individuals with high self-
efficacy set higher goals, use more effective problem-solving strategies, and endure in
challenging situations (Bandura & Locke, 2003).
HUMAN ERROR RISK REDUCTION IN AVIATION 30
Once the technicians meet these motivational influences, they can better appreciate the
conceptual knowledge of human factors contribution to errors, and the procedural knowledge of
how to incorporate decision-making strategies to manage this human error (Krathwohl, 2002;
Wachter & Yorio, 2013). This interaction between the knowledge and motivational influences
would not be complete without the organization's support. Organizational influences affecting
goal attainment can be seen as cultural models and cultural settings that affect the safety effort
such as schedule restraints keeping management from investing energy into enforcing standard
safety practices, or the technicians’ unwillingness or managements discouragement in
highlighting human factor related safety concerns (Kapp, 2012; Wachter & Yorio, 2013). There
may also be a general resistance by employees to change existing practices as processes and
procedures evolve. This interplay between the knowledge, motivation, and organization
influences can contribute or interfere with the 65
th
MXG’s goal of reducing aircraft damage
(Dekker, 2009).
The conceptual framework found in Figure 1 visually illustrates the factors influencing
knowledge and motivation, the organization's cultural influence, and their relation to the
organizational goal of aircraft damage reduction. The large blue circle represents the 65
th
MXG
as a whole, while at the center a green circle represents the 65
th
MXG technicians. Just below
the blue circle is an arrow pointing to a yellow box which describes the organizational goal of
aircraft damage reduction. The green and the blue circles must work together to ensure the
yellow square is satisfied. A departure from either the organization’s support or the technician’s
application of human factors training could negatively affect the goal. Without meeting the
proficiency within the human factors goal, both the technicians and the organization fail to meet
the aircraft damage reduction goal. See the conceptual framework, below, in Figure 1.
HUMAN ERROR RISK REDUCTION IN AVIATION 31

Figure 1. 65
th
MXG Technicians’ KMO Conceptual Framework.

Data Collection and Instrumentation

The mixed-methods approach selected for this study included quantitative and qualitative
instruments that resulted in data that was concrete and focused. An interview protocol, survey
instrument, and a document review process were developed to gain an in-depth comprehension
Goal
By December 2019, 100% of the
65
th
MXG technicians will
demonstrate proficiency in human
factor related error reduction
techniques through performance-
based testing.

65
th
MXG
Cultural Settings and Cultural Models
(Resistance to change and schedule
restraints: existing procedures fail to evolve,
little effort into enforcing common safety
practices due to schedule, technicians
unwilling or discouraged to highlight human
factor related safety concerns)

65
th
MXG
Conceptual and procedural knowledge
and skills related to human factors
contribution to errors and how to
manage that error and the skills and
motivation (value) needed to see the
usefulness of human factors training

HUMAN ERROR RISK REDUCTION IN AVIATION 32
of the problem of practice through detailed analysis. The interview protocol (Appendix B)
served the useful purpose of systematically exploring the selected respondents as well as kept the
interview focused and on track (Patton, 2002). Furthermore, this qualitative interview method
was primarily used to understand the knowledge of the aircraft technicians, offering data
pertaining to the question of the technicians’ knowledge related to demonstrating proficiency in
human factor related error reduction techniques. The quantitative survey was primarily used to
understand the motivation of the aircraft technicians and the interaction between the 65
th
MXG’s
culture and context related to demonstrating proficiency in human factor related error reduction
techniques. Finally, a document review process helped the researcher understand the technicians
level of training after human factors awareness course completion. These multiple data points
were triangulated to produce a clear representation of the technicians' knowledge, motivation,
and organizational influences in relation to human error in aircraft maintenance. By examining
these different data sources in search of similar items or themes, validity was added to the study
(Creswell, 2014).
Surveys

Fink (2013) explains that the researcher must determine the type of data the survey
produces and its relationship to the analysis method needed to accomplish the research goal.
Aircraft maintenance technicians having direct contact with assigned aircraft, a minimum 5-skill
level, and completion of the human factors awareness introductory course completed the survey
instrument. The primary instrument of data collection was reliant on a web-based online survey
tool due to its ease of collection and use, flexibility, relatively low cost, and time required to
complete the survey (Fink, 2013). The instrument link was distributed to 254 technicians at the
65
th
MXG electronically through the Department of Defense Enterprise Email system; this is the
HUMAN ERROR RISK REDUCTION IN AVIATION 33
standard e-mail system used throughout USAF. In order to ensure only 65
th
MXG technicians
that meet the criteria accomplish the survey, the survey included a series of qualifying questions
that helped determine respondents’ characteristics. If the technician did not meet the criteria
noted above, their responses were not included in the data analysis.
According to Galesic and Bosnjak (2009), the survey instrument should take no more
than ten minutes to complete; otherwise, a reduction in completion rate and validity due to
inattention of the respondent may be experienced. The ten-minute goal is also important when
considering the surveys completion rate. Galesic and Bosnjak (2009) found that when declaring
a longer survey prior its start, it is less likely that a respondent will even open it. The survey
completion rate is necessary to consider since the instrument was voluntary, confidential, and
anonymous.
To ensure timely survey completion and high response rates while providing accurate and
measurable data, the survey design was user-friendly and straightforward. Once initial
development was complete, it was put through a series of what Creswell (2014) explained as
field tests. These field tests were given to 65
th
MXG technicians outside the sampled population
to enhance the surveys questions, format, and structure. These tests also helped determine if the
survey included vague terms, completion times, an evaluation of any regularly skipped
questions, and that the survey link launched properly.
Interviews
Seven aircraft maintenance technicians participated in a one-time interview held in a
conference room away from the noise and distraction laden maintenance environment. The
conducted interviews were in English and interpretation was not necessary, as a basic
requirement to join the USAF is to speak, read, and write English fluently. The interviews lasted
HUMAN ERROR RISK REDUCTION IN AVIATION 34
between 20 and 45 minutes and were informal in their conversation like approach, but followed
the interview protocol (Appendix B) to help guide each interview. This semi-structured format
was used to help distinguish the areas to be explored while also allowing the interview to
deviate, a benefit Patton (2002) explained as a way to investigate a particular idea that may
surface in more detail. The questions used throughout the interview included what Patton (2002)
described as experience and behavior questions (what participants do and how they behave),
opinion and value questions (participant’s opinion and beliefs), and knowledge questions (what
participants know). By using these question types, a distinction of whether the technicians have
the knowledge and motivation to understand and apply human factor principles and any
organizational influences shaping their efforts emerged. All interviews were recorded and
outsourced to an online transcription service, Rev.com, that provided transcripts of the conducted
interviews. Once received, the transcripts were coded for what Maxwell (2013) described as key
themes and specific topics.
Documents and Artifacts
Documents used as part of this study were the current human factors awareness course
materials. This course material, provided by the 65
th
MXG’s director of human factors, offered
information specific to the course objectives and content allowing the researcher to set a baseline
on what the technicians should know after completion of the course. Furthermore, this review
allowed the researcher to reflect back on course content during qualitative analysis, in
understanding the technicians level of training. This comprehensive document review also
allowed valuable interview time to be concentrated on determining if the technicians obtained
the presented knowledge and if so, how they applied it rather than discussing what was presented
during training. By using this document review strategy, the researcher can recognize if
HUMAN ERROR RISK REDUCTION IN AVIATION 35
technicians have received the tools necessary to understand and apply human factors principles
to manage human error on the job.
Data Analysis

Data analysis is a means of translating raw data into meaningful information that may
uncover implicit concerns. Since the researcher of this study was the medium used in all data
collection and analysis, reflexivity in the process was critical. Merriam and Tisdell (2016)
explained that this self-refection by the researcher and their connection to the study may alter the
research and analysis. To help the reflection process and to provide effective and unbiased
analysis a peer review strategy was used to establish solidarity of the findings.
Upon survey completion, all data was transferred from the web-based survey tool into a
Google sheet format. Then, a data set was created based on the technicians’ responses to
highlight frequency and measures of central tendencies. This analysis of the generated survey
data helped support the conceptual framework in the identification of the technicians’ motivation
and organizational influences impacting the goal of aircraft damage reduction.
Upon interview and transcription completion, analytic codes were created to connect the
KMO framework to responses provided by technicians in the validation of assumed influences.
The goal of coding the qualitative data was to highlight the knowledge technicians gained
through the human factors awareness course. Once coding of the data was complete, the
researcher pursued emergent themes to understand patterns among the technicians’ responses
related to the KMO influences. These emergent themes helped validate the assumed influences
contributing to the problem of practice.

HUMAN ERROR RISK REDUCTION IN AVIATION 36
Results and Findings
The objective of this study was to conduct a gap analysis of knowledge and skills,
motivation, and organizational influences essential in meeting the organizational goal of
reducing human error related aircraft damage. The stakeholder of focus for this study was the
65
th
MXG technicians who have direct contact with assigned aircraft. By identifying gaps that
prevent the technicians from demonstrating proficiency in human factor related error reduction
techniques, the organization is better equipped to achieve its goal of reducing aircraft damage.
As a reminder, the research questions that guided this study were:
1. What is the 65
th
MXG technicians’ knowledge and motivation related to demonstrating
proficiency in human factor related error reduction techniques?
2. What is the interaction between the 65
th
MXG’s culture and context and technicians’
knowledge and motivation related to demonstrating proficiency in human factor related
error reduction techniques?
3. What are the recommendations for the 65
th
MXG’s organizational practice in the areas of
knowledge, motivation, and organizational resources?
The findings from the surveys and interviews are presented below using the Clark and
Estes (2008) KMO framework. The findings were examined to determine what assets were
present or if gaps existed in KMO influences listed in the aircraft technicians’ knowledge,
motivation and organizational influences section. An existing gap was considered validated if
the evidence presented from the surveys or interviews confirmed that the technicians lacked the
knowledge, motivation, or organizational influence thought to have caused the gap. Assumed
causes were not validated and therefore considered an asset if the evidence presented from the
surveys or interviews determined that the technicians possessed the required knowledge,
HUMAN ERROR RISK REDUCTION IN AVIATION 37
motivation, or appropriate organizational support initially thought to have caused a gap. The
recommendations for practice section discuss significant findings from this analysis and provides
potential solutions for closing identified performance gaps.
Participating Stakeholders
The 65
th
MXG technicians that met the necessary criteria of having direct contact with
assigned aircraft, a minimum 5-skill level, completion of the current human factors awareness
course, and agreed to participate in the survey and interview were the stakeholders of this study.
Of the 254 technicians available, 108 elected to complete the cross-sectional survey resulting in a
43% response rate. Of the 108 that agreed to complete the survey, 17 (16%) were unionized
civilian employees that can participate in bargaining functions when needed. It is also important
to note that these civilian aircraft technicians, vital in base sustainment as military members
deploy, must adhere to Air Force Instruction 36-703 para 5.4.5. and “comply with safety and
health standards set for the job environment,” which includes human factors awareness training
requirements. The participating technicians that completed the survey, on average, held the rank
of a 7-level Staff Sergeant (SSgt), or civilian equivalent, with 12.7 years of aviation experience.
This aviation experience was predominantly (98%) through formal military technical training
with only 5% (5 technicians) having some commercial airline (Part 121) experience. This is
important to note because Part 121 operators have had various forms of human factors awareness
training for aircraft technicians since the late 1980’s. As the 65
th
MXG has a low number of
technicians with Part 121 experience, the majority of human factors awareness exposure for the
technicians has been internal to the organization. Of the 108 technicians that completed the
survey, seven volunteered to participate in a one-on-one interview. The technicians interviewed
consisted of four males and two females that, on average, held the rank of a 7-level Staff
HUMAN ERROR RISK REDUCTION IN AVIATION 38
Sergeant (SSgt), or civilian equivalent, with 11.3 years of aviation experience. This aviation
experience was 100% through formal military technical training. Additional participant
demographic information was not obtained to maintain the anonymity of responses. Interviews
were scheduled and accomplished at the convenience of the participants and did not correlate to
survey completion as survey participation was anonymous and not tracked.
Results and Findings for Knowledge Causes
The method used to explore the knowledge influences was primarily through qualitative
interviews of the participating volunteer stakeholders. This qualitative method was supported by
item 10 of the quantitative survey in finding 1 to help establish a comprehensive determination
of whether the influence was an asset or gap. The knowledge influence examination was also
complemented by a document analysis of the human factors awareness course material to
familiarize the researcher with specific course objectives and content. The review of these
documents also allowed the researcher to reflect on particular course details during the
qualitative analysis to help determine if the technicians received the tools necessary to
understand and apply human factors principles on the job. The document analysis revealed that
the human factors awareness course contained information pertaining to human factor principles
to include methods and theories of the behavioral and social sciences, engineering, and
physiology that would help participants understand and apply critical human factor skills and
considerations that enhance capability and safety. The awareness course identified several
human factor centered cognitive skills and provided essential characteristics of each. Some of
these skills include planning and preparation, communication, teamwork, task management,
situational awareness, decision making, and feedback. The validity rationale for each knowledge
HUMAN ERROR RISK REDUCTION IN AVIATION 39
influence is presented through the specific knowledge category of declarative knowledge,
procedural knowledge, and metacognition.
Declarative Knowledge
Finding 1. Technicians need to increase their understanding of how human factors
contribute to errors in their work environment. Interviews revealed that although technicians
may understand how human factors contribute to errors, their understanding of the influence
within their specific work environment may be incomplete. One interviewee explained the
contribution human factors has on an error and a possible strategy to combat it; he explained that
it is not just up to the individual to recognize an issue, others must intervene as well. He said, “if
somebody comes to work with the wrong attitude, the wrong frame of mind, hung over, tired, it
absolutely impacts their judgment…people typically ask, what’s up man, you all right, you okay,
are you ready to do some work?” This type of intervention and assertiveness is essential in
combating error. Another interviewee, one in a supervisory role, detailed how he needed to
understand what his “troops” were going through, he needed to ensure everyone was in a safe
environment. He stated that “too much stress and people start making mistakes, they start
rushing...also if someone is having personal issues or not feeling good…then I can’t trust him to
do a job that day.”
While only two technicians interviewed gave sufficient examples of the contribution
human factors have in relation to errors, five of the seven (71%) failed to recognize the role these
factors play in their specific work environment. One interviewee described how a lack of
coordination can lead to increased complications but did not provide how it could lead to errors
in his work environment. He said, “without proper teamwork, without proper
communication…you’re going to run into problems.” The interviewer then asked to elaborate on
HUMAN ERROR RISK REDUCTION IN AVIATION 40
problems he may run into when a lack of teamwork is present on the job to which the respondent
answered, “serious problems.” The interviewer also asked the interviewees about specific safety
nets present in their environment, looking for environmental error-producing conditions, and five
of the seven interviewees (71%) could not provide detailed information or explain what a safety
net was in relation to human error reduction. Specific safety net examples were found during the
document analysis but the interviewees could not demonstrate this knowledge during the
interviews. Additionally, when specifically asked how human factors can affect maintenance
operations, five of the seven interviewees (71%) did not cite typical error producing components.
These components, or contributing factors, were found integrated within the human factors
course during document analysis and include a lack of communication, a lack of teamwork,
fatigue, complacency, stress, resources, and many other contributing factors to human error. The
interviewer also asked what aspects of the human factors awareness course they have
incorporated within their daily routine and all but one (86%) revealed that they gained very little
from the course. The notion that technicians neglected to describe basic human factors concepts
and revealed that they took very little away from the course prompted the researcher to review
survey data, item 10, to determine how much the human factors awareness training changed
technicians’ behavior on the job. Survey results showed that 64% of technicians (M=2.10,
SD=0.99) felt the training provided either no change (35%) or a slight change (29%) to their
behavior on the job. The inability to convey fundamental human factor error producing
components taught within the awareness course coupled with little change in behavior may
indicate that they need to increase their understanding of how human factors contribute to errors
in their work environment. Figure 2 illustrates the technicians' belief in how human factors
training has changed on the job behavior.
HUMAN ERROR RISK REDUCTION IN AVIATION 41

Figure 2. Technicians belief in how much human factors training changed their behavior.
71% of the technicians interviewed failed to describe basic human factor concepts found
within the awareness course. Furthermore, these same technicians could not explain how human
factors contribute to errors in their specific work environment or cite typical error producing
components. Therefore, this influence can be considered a validated gap as most technicians did
not adequately explain how human factors tie into their daily on-the-job routine, which may
prevent the technicians from achieving their goal.
Procedural Knowledge
Finding 2. Technicians need to increase their understanding of how to incorporate
decision-making strategies to manage human error. Interviews found that technicians may
lack the ability to incorporate decision-making strategies to manage human error. Recognizing
human error conditions is important, but the application of strategies to manage the error is
critical in reducing aircraft damage. During the seven interviews, the researcher found a lack of
N = 106
HUMAN ERROR RISK REDUCTION IN AVIATION 42
compelling evidence to support a claim that technicians currently exhibit the skills necessary to
incorporate decision-making strategies to manage human error. When asked about specific
strategies used to help manage the consequences of human error, one interviewee stated that she
was “very good with cause and effect,” while another described his decision-making strategy as
“I try to be as safe as I can” and “I literally just try to make sure that everything’s the way that
it’s supposed to be.” Both of these responses, in relation to strategies employed to combat
human error, fail to describe adequate strategies found within the current awareness course.
Other interviewees did describe basic error reducing techniques that may garner better results but
still failed to exhibit the necessary decision-making strategies needed to adequately manage
human error. For example, one interviewee stated “if you don’t know what you’re doing, stop
and ask”, while another said “If I’m having a bad day or I’m distracted with something, I’ll make
sure to clear my mind and not rush.” Both of these responses still fail to capture the true essence
of the human factors awareness course that, discovered through the document analysis, outlines
effective techniques of error management such as the identification and mitigation of hazards.
The incorporation of decision making strategies in managing human error is more than
stopping to ask a question or simply clearing your head, it involves fortitude in assessing the
complete situation and recognizing and eliminating error producing conditions. For example,
one interviewee, a flying crew chief that travels with the aircraft while off-station, went into
detail about how he incorporates strategies to combat error. He said, “I take into consideration
how long I've been awake, the weather outside, my surroundings, can I get parts if needed, have I
eaten, do I have the necessary tools to do the job, everything goes into play.” His response is a
reasonable example of someone attuned to his situation and understands what could go wrong
and avoiding conditions that can sometimes lead to error. He went on to describe the pressure he
HUMAN ERROR RISK REDUCTION IN AVIATION 43
encounters while flying with the aircraft, “there are a number of people focused on our mission,
so there’s that added pressure, and I believe knowing and understanding that pressure and not
bowing down to it…helps me avoid costly mistakes.” Here he displays an understanding that
there is pressure coming from many directions, and that he is in control of regulating that
pressure, which ultimately helps him “avoid costly mistakes.”
Another question used to help determine if the technicians have the ability to incorporate
decision-making strategies to manage human error involves how they would explain specific
skills associated with human factors principles. Interviewees were asked to imagine they were
training a new technician and explain how they would describe the skills needed to apply human
factors principles. One interviewee explained that he provided limited support during training
saying, “I don’t think I make a conscious effort to do it…I really don’t explain the human
factors.” Another interviewee explained that when she does incorporate a human factor
component into her training she “tries to make sure they understand their responsibility on the
aircraft.” Both of these responses fail to incorporate course specific strategies recognized during
the document review, such as specific procedures to minimize error, loss and recovery of
situational awareness, or any one of the Dupont’s dirty dozen that degrade technicians’ ability to
perform effectively and safely. The next four interview responses, related to the training of
human factors principles to new technicians, all indicated that a lack of time would be the reason
for not including human factors within training. One interviewee stated that even when training
“I’m being pulled off to do other jobs, too many people pushing me even when training” while
another said, “when I usually train someone new it's under circumstances that I don't particularly
like, we usually don't have enough time.” Even when given a hypothetical question the
technicians revert to pressure and a lack of time as a reason for either not implementing human
HUMAN ERROR RISK REDUCTION IN AVIATION 44
factors or not incorporating error reduction principles. The final interviewee, the flying crew
chief that travels with the aircraft, also answered the training of a new technician question and
highlighted the pressure they may face. He said, “the biggest piece I try to get across is that the
pressure can really destroy your work…what you just fixed may not fail today, but eventually it
may break at an inopportune time and cause a bigger problem.”
As only one of seven technicians (14%) interviewed seemed to understand how to
implement decision-making strategies to manage human error, this influence is a validated gap
preventing the technicians from achieving their goal. This gap was validated as 86% of the
technicians interviewed were unable to describe the sufficient incorporation of decision-making
strategies effectively. Although most technicians did explain generic safety behaviors during the
interview portion of data collection, it was these conservative error reduction attempts that
confirmed their inability to incorporate appropriate decision-making strategies to manage human
error.
Metacognitive Knowledge
Finding 3. Technicians need to increase their understanding of how to properly
reflect on performance limitations. Interviews discovered that the technicians might not
understand how to reflect on their performance limitations on the job. Through reflection,
technicians are better able to understand their limitations and know when to stop and make
appropriate decisions to help mitigate any possible error producing actions. The interview
questions used to gain adequate perspective into this influence asked if the technician felt it was
important to understand how human factors affected their behavior and performance and what
aspects of training they implemented within their daily routine. These questions attempted to
uncover the importance the technicians placed on self-reflection and the strategies used in
HUMAN ERROR RISK REDUCTION IN AVIATION 45
determining their limitations. This practice is essential to incorporate as understanding
limitations helps avoid mistakes and also allows a technician to stop and assess the situation
when something is not right. Unfortunately, the interviews revealed that most technicians do
very little to reflect on their limitations on the job and also indicate that what they acquired from
human factors training would not prepare them to reflect. Interviewees also stated that they had
minimal gain from the human factors awareness training with comments such as “It's a good
reminder to keep doing the things we do,” and “I wouldn't say I took an awful lot away from the
course…” Additionally, when asked directly if reflection was used on the job when considering
performance limitations, a technician stated “we rarely have time to reflect on what we did or
how we did it, if someone screws something up that’s when we start trying to figure out what we
did wrong and who’s in trouble.”
Another strategy often used in aviation maintenance to avoid error is to conduct a debrief
at the completion of a job. A debrief can help those involved look back at work accomplished
and learn from what went right and not so right. Once again, those interviewed did not indicate
debriefs occurred following a critical task, most did state that there was a lack of time to
accomplish a debrief after a task.
During the human factors’ awareness course document analysis, it was discovered that
little to no time is afforded to discuss reflection, either through a debrief, pre-brief, or
independently, nor is there a discussion on performance limitations. So as the interviews
indicate, the human factors course is missing a key element to error reduction. Understanding
limitations and reflecting on performance helps technicians recognize their abilities. Without the
ability to reflect on performance limitations, whether due to time on the job or in training, the
technicians may be missing an opportunity to fully understand how human factors contribute to
HUMAN ERROR RISK REDUCTION IN AVIATION 46
errors in their work environment and how to incorporate decision-making strategies to manage
human error.
The technicians’ ability to adequately reflect on their performance limitations was not
found. Therefore, this influence is a validated gap preventing the technicians from achieving
their goal. This gap was validated as the technicians interviewed were unable to effectively
describe the necessary requirements to properly reflect on their performance limitations on the
job. Although some technicians did explain what little they felt they gained from the training,
most could not adequality explain how reflection could be used in determining their performance
limitations. Table 5 shows the validated and not validated knowledge gaps.
Table 5
Knowledge Findings Validated and Not Validated as Gaps
Category Finding Validated
Gap
Not a
Validated Gap
(Asset)

Declarative Knowledge

Procedural Knowledge

Metacognitive Knowledge

Technicians need to increase their
understanding of how human
factors contribute to errors in their
work environment.

Technicians need to increase their
understanding of how to
incorporate decision-making
strategies to manage human error.

Technicians need to increase their
understanding of how to properly
reflect on performance limitations.

X

X

X

HUMAN ERROR RISK REDUCTION IN AVIATION 47
Results and Findings for Motivation Causes

The method used to determine the validity of the motivation influences was primarily
through a survey completed by the participating volunteer stakeholders. The validity rational for
each motivation influence is presented through the specific motivation category of utility value
and self-efficacy.
Utility Value
Finding 4. Technicians report a high degree of value in human factors awareness
training as it relates to behavior and performance on the job. Survey results showed that
technicians see the usefulness of human factors awareness training as it relates to behavior and
performance on the job. Two value related questions were used to determine the value
technicians place on training. The first question, item 8, asked if the technicians thought human
factors awareness training had the potential to increase aviation safety and teamwork
effectiveness. The survey revealed that 82% of the technicians (M=3.02, SD=0.87) either
strongly agreed (29%) or somewhat agreed (53%) that human factors training had potential to
improve safety, which indicates they believe there is value in training. Item 9 asked if the
technicians found the training meaningful and not simply an act of compliance in which 75% of
the technicians (M=2.92, SD=0.87) either strongly agreed (26%) or somewhat agreed (49%) that
they found the training meaningful and was not presented to fulfill a requirement. This question
could be interpreted as either value in the specific course they attended or value in human factors
awareness as a concept overall. By finding human factors training meaningful, the technicians
understand how it fits into their goals and are better equipped to recognize conditions leading to
an error. As survey results showed, the technicians were able to express high levels of value
regarding the usefulness of human factors awareness training as it relates to behavior and
HUMAN ERROR RISK REDUCTION IN AVIATION 48
performance on the job and therefore can be considered an asset. Figure 3 presents the results
from survey question 8 where technicians were asked if they thought human factors awareness
training had the potential to increase aviation safety and teamwork effectiveness.

Figure 3. Technicians belief in human factors awareness training and its potential outcomes.
Self-Efficacy
Finding 5. Technicians report a high degree of confidence in their ability to
understand and apply human factors principles on the job. Survey results showed that
technicians have confidence in their ability to understand and apply human factors principles on
the job. In order to establish that technicians trust their competence in applying human factors
principles to solve atypical problems, item 11 on the survey asked if they were confident in their
ability to apply human factors principles in abnormal situations. The positive response was
significant as 93% (M=3.35, SD =0.69) strongly agreed that they were able to apply human
factors principles in abnormal situations indicating a high level of self-efficacy.
N = 106
HUMAN ERROR RISK REDUCTION IN AVIATION 49
To further understand the confidence technicians had in understanding and applying
human factors principles, the survey asked two questions focusing on fatigue that ultimately
centered on self-efficacy under abnormal situations. Item 12 asked how often the technicians
worked fatigued, followed by item 13 which asked how well they thought they performed when
fatigued. These two questions helped gain insight into the technicians’ ability in identifying and
employing strategies to reduce a fatigue-related error. When asked how often they worked
fatigued, over 52% (M=2.43, SD=0.89) indicated that they frequently (36%) or very frequently
(16%) worked while fatigued. Although this question does not directly coincide with the
confidence level of the technicians, it established a baseline for the next question, item 13, which
stated: “I perform effectively during critical phases of work, even when fatigued.” This question
looked directly at the confidence level of the technician in both critical phases of work as well as
while being fatigued in which 71% of the technicians (M=2.91, SD=0.84) either strongly agreed
(26%) or somewhat agreed (45%) that they felt they performed well under these conditions.
These results further reinforce the technician’s confidence level concerning the application of
human factors principles on the job. As survey results showed, the technicians expressed high
levels of confidence in relation to understanding and applying human factors principles on the
job and therefore can be considered an asset. Figure 4 presents the results from survey question
11 where technicians were asked of their confidence in applying human factors principles while
Table 6 shows the motivation findings that were not validated as gaps and can be considered
assets.
HUMAN ERROR RISK REDUCTION IN AVIATION 50

Figure 4. Technicians’ Confidence in Applying Human Factors Principles.
Table 6
Motivation Findings Validated and Not Validated as Gaps

Category Findings Validated
Gap
Not a
Validated Gap
(Asset)

Utility Value

Self-Efficacy

Technicians report a high degree of value
in human factors awareness training as it
relates to behavior and performance on the
job.

Technicians report a high degree of
confidence in their ability to understand
and apply human factors principles on the
job.

X

X

N = 104
HUMAN ERROR RISK REDUCTION IN AVIATION 51
Results and Findings for Organization Causes

The method used to determine the validity of the organizational influences was through
qualitative interviews and quantitative surveys of the participating stakeholders. The qualitative
interviews and quantitative surveys complemented each other as they helped illustrate the
organizational influences as a whole. The validity rationale for each assumed organizational gap
is presented through the cultural models and cultural setting below.
Cultural Models
Finding 6. Organizational culture fails to support technicians reporting human
factor related safety concerns that could affect their performance. Survey results showed
that technicians received support from supervision when highlighting human factor related safety
concerns that could affect their performance. Item 16 asked the technicians if their supervisor
encourages them to report unsafe conditions. In this item, 92% (M=3.55, SD=0.75) either
strongly agreed (66%) or somewhat agreed (26%) that they are encouraged to report unsafe
conditions. This indicates that supervisors do encourage hazard reporting, but the technicians
must feel comfortable reporting, and the organization’s leadership must also be supportive
through action.
Another support characteristic of the organization is how the technicians feel about
reporting errors through the system. Item 17 asked if the technicians felt comfortable reporting
errors within the system in which the respondents expressed that 60% (M=1.88, SD=0.87) either
felt slightly comfortable (37%), slightly uncomfortable (18%), or extremely uncomfortable (5%)
when reporting errors. With the results only indicating that 40% of the population felt extremely
comfortable with the reporting of errors within the system, the likelihood of eliminating error
producing threats throughout the entire organization is decreased.
HUMAN ERROR RISK REDUCTION IN AVIATION 52
The results from item 16 (supervisor support), and item 17 (comfortability of reporting)
may be rooted in item 18 of the survey which centered around how leadership would respond if
safety concerns were reported. Item 18 asked if technicians believed their leadership would act
upon safety recommendations if communicated properly. This question is significant because if
the technicians do not feel that their concerns will be addressed, they will not be submitted. If
the technicians do not submit safety concerns the organization loses their ability to identify
trends that represent risk, they lose the identification of precursors to accidents, and they lose the
ability to be proactive in relation to safety hazards, and accidents. The results of item 18 indicate
that 19% of the technicians (M=3.24, SD=0.86) either strongly disagree (5%) or somewhat
disagree (14%) that their leadership would act upon safety recommendations if communicated
properly. These results can be considered less than ideal when factoring in the safety
implications of technicians not reporting hazards. If technicians feel that the organization's
leadership will not act on their concerns, reporting will drop, and the primary hazard
identification tool is lost weakening the overall safety culture. Although 48% of the technicians
strongly agree that they received support from supervision when highlighting human factor
related safety concerns, this leaves far too many potential hazards left unreported because 52%
of the technicians are unsure if supervision will act on their concerns. Although 92% of
technicians reported that supervisors support the reporting of hazards, a large number of
technicians do not feel compelled to report due to either comfortability of the system or an
unwillingness to trust that leadership would act on their concerns. In a safety critical
environment such as aviation, leaving any know safety-related hazards unreported is an
invitation for disaster. Therefore, the evidence supports that the organizational culture has failed
HUMAN ERROR RISK REDUCTION IN AVIATION 53
to support technicians reporting human factor related safety concerns that could affect their
performance and thus can be considered a gap.
Figure 5 presents the results from survey question 16 that asked the technicians if their
supervisor encourages them to report unsafe conditions. Figure 6 presents the results from
question 17 which asked how the technicians feel about reporting errors through the system.
Figure 7 presents the results of question 18 that asked if technicians believed their leadership
would act upon safety recommendations if communicated properly.

Figure 5. Supervisor and Co-worker Encouragement to Report Unsafe Conditions.
N = 104
HUMAN ERROR RISK REDUCTION IN AVIATION 54

Figure 6. Technicians comfort level reporting errors within the system.

Figure 7. Technicians Belief That Leadership Would Act Upon Safety Recommendations.

N = 103
N = 100
HUMAN ERROR RISK REDUCTION IN AVIATION 55
Finding 7. Organizational culture was found to be supportive as existing processes
and procedures evolve. Interviews emphasized that the organizational culture is conducive to
changing existing processes and procedures as they evolve. This became apparent as the
technicians were asked during one on one interviews if they could share a specific example of
when a change in a policy or procedure affected how they accomplished a task. This question
was used to establish how the organization dealt with shifting requirements due to safety
concerns or when better ways of performing particular tasks were discovered. One interviewee
explained how a change in a specific procedure for an engine cowl removal resulted in not only a
more efficient way of removing the door but a substantial increase to the technician’s overall
safety. He said “yeah, the old way was dangerous, but you know it's something that we had
always done…the new procedure not only saves time but it saves someone from crushing their
head too.” When another interviewee shared a specific example of a policy or procedure
transformation, they described a change to fuel tank entry at higher ambient temperatures. This
policy change eliminated a heat-related risk to technicians but also increased the time needed to
perform the maintenance. He explained that this policy “slowed maintenance down a lot, but it
made the job safer as a whole", which is an indication that the organization not only understands
that heat exhaustion is a high price to pay to fix an aircraft but as they are made aware of safety
concerns they act on and provide guidance to ensure that problems are resolved. This is ever
more evident as one technician interviewed described a recent change to a complex procedure for
hoisting a life raft after a periodic inspection, she said “we knew a better way to do it, a more
efficient way than what was in the book…after doing it ‘the wrong way’ for years, someone
finally brought it up…much safer and now we don’t need to deviate [from the technical
guidance].” As interview results showed, the technicians expressed high levels of support from
HUMAN ERROR RISK REDUCTION IN AVIATION 56
the organization in relation to changing existing processes and procedures as they evolve and
therefore can be considered an asset.
Cultural Settings
Finding 8. Organizational culture fails to support managers that are overwhelmed
by schedule restraints, keeping them from enforcing standard safety practices. Managers
and supervisors must be supported by the organization to ensure schedule does not eclipse
standard safety practices. Survey results of question 22 show that 25% of technicians (M=3.08,
SD=0.96) either strongly disagree (9%) or somewhat disagree (16%) that supervisors and
leadership would not compromise safety for production. These results are significant because
this shows that a quarter of the technicians believe that supervision would forfeit safety to make
schedule. This survey question also reveals that 34% of technicians only somewhat agree that
supervisors and leadership would not compromise safety for production, leaving only 42% that
trust their leaders not to forfeit safety for schedule. Figure 8 presents the results from survey
question 22 that asked the technicians if they believed that supervisors/leadership would not
compromise safety for production.
HUMAN ERROR RISK REDUCTION IN AVIATION 57

Figure 8. Technicians Belief That Leadership Would Not Compromise Safety for Production.
With over half (58%) of the technicians skeptical of leadership intentions in regards to
safety as seen in survey item 22, the researcher turned to qualitative interviews to help solidify
this finding. Of the seven interviews conducted, each one indicated that the technicians were
under tremendous pressure. One interviewee stated “it’s all the time, just hurry it up and get it
done” and when asked where the pressure was coming from she indicated “mid-level
supervision.” Another interviewee explained that they take any means necessary to accomplish a
task, he said, “we violate safety protocol all the time, because the pressure from above, nobody
wants to get in trouble.” Similarly, another interviewee explained the added pressure when
trying to repair a specific aircraft that is on the flying schedule the next day, and he said “they
tell us to hurry up and get it done…we end up skipping a lot of safety protocols because of the
push from supervision.” With continual pressure, even when the time is allotted to do the work
properly, the technicians continue to deviate. One interviewee explained that the use of shortcuts
N = 101
HUMAN ERROR RISK REDUCTION IN AVIATION 58
in aircraft maintenance has become an established norm. He said, “I've been in a number of
situations where I had no pressure, the aircraft was not on the flying schedule, and we continued
to take the same shortcuts that we would before…it's just kind of the way it gets it done.” The
prevalence of pressure on technicians from supervision found throughout the seven interviews
indicates that the organization's culture does not support managers that are overwhelmed by
schedule restraints.
Atak and Kingma (2011) explained that one by-product of increased pressure from
supervision is that it can lead technicians to forgo required aircraft technical data, referred to as
technical orders (T.O.) in the USAF. These T.O.’s are similar to a pilot’s checklist in that they
must be followed step by step to ensure safe operation of the aircraft systems and related
maintenance procedures. One interviewee stated that she “chose to deviate from tech data” and
when asked how often this deviation occurs she said, “a lot, like 60%...this is the way I've always
done it, and it’s been successful, so I continue to do it." Another interviewee talked about
problem-solving in abnormal situations and that they “routinely…go outside of the T.O.
specifications for fixing an aircraft.” Throughout most interviews, the technicians talked about
T.O. violations as an established norm within the organization. The preponderance of deviations
and willingness to openly discuss them is a clear sign that managers have failed to enforce
standard safety practices.
As all interviewed technicians discussed the pressure they face daily from supervision
and the established organizational norm of technical order violations/deviations; this influence is
a validated gap preventing the technicians from achieving their goal. This gap is considered
validated as the technicians interviewed were unable to effectively describe an organizational
culture that supports managers that are overwhelmed by schedule restraints that may keep them
HUMAN ERROR RISK REDUCTION IN AVIATION 59
from enforcing standard safety practices. Although 76% of the technicians surveyed believed
that supervisors and leadership would not compromise safety for production, the overwhelming
evidence that surfaced through the qualitative interviews confirmed the organization's inability to
support managers as they enforced standard safety practices. Table 7 shows the organizational
findings that were validated and not validated as gaps.
Table 7
Organizational Findings Validated and Not Validated as Gaps

Category Findings Validated
Gap
Not a
Validated Gap
(Asset)

Cultural Models

Cultural Models

Cultural Settings

Organizational culture fails to support
technicians reporting human factor
related safety concerns that could
affect their performance.

Organizational culture was found to be
supportive as existing processes and
procedures evolve.

Organizational culture fails to support
managers that are overwhelmed by
schedule restraints, keeping them from
enforcing standard safety practices.

X

X

X

Summary of Validated Gaps
This study set out to evaluate the 65th MXG’s goal of reducing aircraft damage caused
by human error in aircraft maintenance. Through comprehensive data analysis, the validated
knowledge and organizational gaps are summarized below. Although specific findings were
highlighted, it is important to note a resounding theme that was found running throughout this
study. During this evaluation, it was noted on several occasions that the 65th MXG technicians
are dealing with high levels of pressure, enough to forgo standard safety practices, and limited
HUMAN ERROR RISK REDUCTION IN AVIATION 60
time to properly accomplish their job. As this lack of time is coupled with increasing pressure,
supervisors, as well as technicians, are placed in a precarious situation where they feel as though
they much choose to be safe by following established procedures or forgo standard safety
practices to ensure the aircraft makes its schedule. In this safety critical environment, no one
should have to choose safety over schedule, and it is ultimately the responsibility of the
organization to ensure that, without question, safety is chosen every time. By identifying the
65th MXG gaps and instituting appropriate context-specific recommendations, a strengthened
organizational safety culture resulting in a reduction of human error can be realized.
Knowledge
There were three knowledge influences that can be considered validated gaps based on
data collection and analysis. The first validated gap was declarative knowledge and found that
technicians needed to increase their understanding of how human factors contribute to errors in
their work environment. Without knowing how human factors contribute to errors in their work
environment, a 65
th
MXG technicians’ likelihood of committing an error is increased primarily
due to not understanding how to predict, prevent, and eliminate the error. Although most
technicians did explain how human factors contribute to errors, they did not adequately explain
how human factors tie into their daily on-the-job routine.
The second validated gap was procedural knowledge and found that technicians need to
increase their understanding of how to incorporate decision-making strategies to manage human
error. When a technician is stressed, distracted, or pressured, their cognitive process becomes
overloaded and fails to incorporate decision-making strategies correctly, ultimately leading to
increased error. Although most technicians did explain generic safety behaviors during the
interview portion of data collection, it was these conservative error reduction attempts that
HUMAN ERROR RISK REDUCTION IN AVIATION 61
confirmed their inability to incorporate appropriate decision-making strategies to manage human
error.
The third validated gap was metacognitive knowledge and found technicians needed to
increase their understanding of how to properly reflect on performance limitations on the job.
Through the use of metacognition, technicians have the ability to understand what they did
which is crucial for making appropriate decisions in critical time and safety-sensitive decision-
making. Although some technicians did explain what little they felt they gained from the
training, most could not adequality explain how reflection could be used in determining their
performance limitations.
Organization
There were two organizational influences that can be considered validated gaps based on
data collection and analysis. The first validated organizational gap was that the organization’s
culture failed to support technicians reporting human factor related safety concerns that could
affect their performance. The reporting of known hazardous conditions allows the organization
to eliminate potential issues that may lead to accidents or incidents ultimately enhancing aviation
safety. When technicians feel their voice will not be heard or if they think nothing will be done
about their concerns, they will fail to report. By failing to report safety concerns, technicians and
the organization miss an opportunity to focus attention on unknown risks and the possibility of
eliminating error producing threats. The results illustrate that a significant number of technicians
are not willing to report due to either feeling uncomfortable with the system or a lack of trust that
leadership would act on their concerns. Therefore, the overwhelming evidence that surfaced
through the quantitative survey supports the conclusion that the organizational culture has failed
HUMAN ERROR RISK REDUCTION IN AVIATION 62
to support technicians reporting human factor related safety concerns that could affect their
performance.
The second validated organizational gap was that the organization’s culture needs to
support managers that are overwhelmed by schedule restraints that keep them from enforcing
standard safety practices. When schedule pressure drives management, they allow violations and
deviations which may suggest an artificial endorsement of the technicians’ actions. These
actions turn violations into environmental norms that can lead to errors. Although the majority
of technicians surveyed believed that supervisors and leadership would not compromise safety
for production, the overwhelming evidence that surfaced through the qualitative interviews
confirmed the organizations inability to support managers as they enforced standard safety
practices. Table 8 shows the KMO findings validated as gaps.

HUMAN ERROR RISK REDUCTION IN AVIATION 63
Table 8
KMO Findings Validated as Gaps

Category Findings Validated
Gap
Not a
Validated Gap
(Asset)

Declarative Knowledge

Procedural Knowledge

Metacognitive Knowledge

Organizational Cultural
Models

Organizational Cultural
Settings

Technicians need to increase
their understanding of how
human factors contribute to
errors in their work
environment.

Technicians need to increase
their understanding of how to
incorporate decision-making
strategies to manage human
error.

Technicians need to increase
their understanding of how to
properly reflect on
performance limitations.

Organizational culture fails to
support technicians reporting
human factor related safety
concerns that could affect their
performance.

Organizational culture fails to
support managers that are
overwhelmed by schedule
restraints, keeping them from
enforcing standard safety
practices.

X

X

X

X

X

Conclusion

The survey, interviews, and document analysis highlighted several KMO gaps that may
prevent the 65
th
MXG from meeting its goal of reducing aircraft damage. As stated previously,
the organization's goals will only be reached when the difference between actual performance
HUMAN ERROR RISK REDUCTION IN AVIATION 64
and the performance goal is reduced. Through the identification of performance gaps within the
65
th
MXG, recommended solutions can now be given to address these findings adequately.
Through these proposed recommendations the gap between actual performance and the
performance goal can close helping the 65
th
MXG reduce human error in aircraft maintenance.
Recommendations for Practice
As the assumed influences have been analyzed and gaps established through Clark and
Estes’ (2008) gap analysis framework, context-specific recommendations can now be offered.
The presented recommendations were developed through an extensive step-by-step process and
incorporates a comprehensive evaluation and implementation plan discussed in Appendix G that
is grounded in the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016). This New
World Kirkpatrick Model coupled with Clark and Estes gap analysis framework is presented as a
program that seeks to strengthen the implementation and evaluation of the knowledge,
motivation, and organizational recommendations.
Program recommendation. Through the Clark and Estes’ (2008) gap analysis, it was
discovered that the 65
th
MXG technicians needed to increase their understanding of how human
factors contribute to errors, how to incorporate decision-making strategies to manage human
error, and how to properly reflect on performance limitations. In order to close the confirmed
knowledge gaps, it is recommended that the human factors awareness training program contain
the necessary elements to strengthen the 65
th
MXG technicians and its culture. This human
factors awareness course needs to provide the aircraft technicians with a comprehensive
understanding of the influences affecting their performance on the job and the role these
influences play in the organizations overall safety program. This training program could consist
of in-class, instructor led, learning with real-world examples and case studies to help facilitate
HUMAN ERROR RISK REDUCTION IN AVIATION 65
understanding. Additionally, the program could guide technicians through error theory, risk
management, and the importance of communication and teamwork. The in-class instruction can
also include ample opportunities to practice learned skills and provide continuous feedback of
performance. Furthermore, the technicians could also engage in hazard recognition, fatigue
management, and the complexities associated with deviations and blatant disregard for safety.
At the conclusion of the first day, students could be given a case study to review in
preparation for the next day’s class. This case study needs to be a complex real-world scenario
that includes much of what was learned on the first day. This case study will also help build on
the aircraft technicians’ non-technical capabilities, decision-making skills that complement
technical skills, focusing on the management of human error. At the conclusion of the class the
students could engage in guided self-monitoring and self-assessment to better understand their
competence and awareness of knowledge and skill in a given situation. The total time for
course completion is 16 hours (2 days).
This study also discovered that the organization’s culture failed to support technicians
reporting human factor related safety concerns and also failed to support frontline managers in
the enforcement of standard safety practices. In order to close the organizational gaps, it is
recommended that the organization develop and support a non-retribution error reporting system
that is promoted by all levels of management to collect and respond to technicians concerns in a
timely manner. This confidential, voluntary error reporting system needs to be simple for the
user to submit and contain an improvement suggestion area to help prevent a reoccurrence of a
hazard. The error reporting system could include a de-identification component to ensure
anonymity with a place to provide contact information if the technician is willing to provide
more detailed information. The key to this system should be to encourage technicians to
HUMAN ERROR RISK REDUCTION IN AVIATION 66
voluntarily report safety concerns even though they may involve violations or unauthorized
deviations. An error reporting system is at the heart of an organizations safety culture and will
only be successful if the technicians fear of reprisal is eliminated.
Once trust is built between the technicians and management, the second validated gap can
begin to close. It was discovered that the organization’s culture failed to support managers that
are overwhelmed by schedule restraints, keeping them from enforcing standard safety practices.
As stated earlier, this gap is the most concerning because if standard safety practices are not
being enforced, neither are other error reducing strategies in place to ensure the safety of
technicians on the job and flight crews in the air. Therefore, it is recommended that the 65
th

MXG works to cultivate a supported safety culture through robust hazard reporting, safety
awareness for all employees, enhanced safety communication, improved supervision-technician
relationships and ensuring all policies and procedures are formalized and enacted.
Knowledge and Organization Recommendations
The method of how these program based solutions were generated corresponding to Clark
and Estes KMO framework is reflected here in this step-by-step process. There were five
validated gaps found that could result in the 65
th
MXG not meeting its goal of reducing aircraft
damage. These five validated gaps, found on Table F1, were identified using the Clark and Estes
(2008) gap analysis framework and include three knowledge gaps and two organization gaps.
The three recommendations targeted at the knowledge gaps would benefit most if
implemented within the current human factors awareness training program. The first validated
gap is a declarative knowledge type and asserts that technicians need to increase their
understanding of how human factors contribute to errors in their work environment. The second
validated knowledge gap is procedural and suggests that technicians need to increase their
HUMAN ERROR RISK REDUCTION IN AVIATION 67
understanding of how to incorporate decision-making strategies to manage human error. The
third validated knowledge gap is metacognition and states that technicians need to increase their
understanding of how to properly reflect on performance limitations on the job.
The two organizational gaps discovered are centered on both cultural models and cultural
settings. Cultural models influence trust, feelings towards change, and accountability whereas
cultural settings focus on the presence of goals, incentives, feedback, and communication. The
cultural model gap identified relates to how the organization needs to be supportive of
technicians reporting human factor related safety concerns. The cultural setting gap identified
relates to how the organizations culture fails to support managers that are overwhelmed by
schedule restraints, keeping them from enforcing standard safety practices. Table 9 outlines the
validated knowledge and organization findings/gaps influencing the achievement of the
stakeholder goal and the theoretical principles supporting the potential recommendations.

HUMAN ERROR RISK REDUCTION IN AVIATION 68
Table 9
Summary of Validated Knowledge and Organization Findings/Gaps and Recommendations
Finding/Gap Principle and Citation Context-Specific
Recommendation
(D) Technicians need to increase
their understanding of how
human factors contribute to
errors in their work environment.
Increasing germane
cognitive load by engaging
the learner in meaningful
learning and schema
construction facilitates
effective learning
(Kirshner et al., 2006).
Provide information that
is structured around case
studies and worked
examples regarding how
human factors contribute
to errors.
(P) Technicians need to increase
their understanding of how to
incorporate decision-making
strategies to manage human
error.
To develop mastery,
individuals must acquire
component skills, practice
integrating them, and know
when to apply what they
have learned (Schraw &
McCrudden, 2006).
Provide opportunities to
practice skills, and
provide feedback of
performance.
(M) Technicians need to increase
their understanding of how to
properly reflect on performance
limitations.
The use of metacognitive
strategies facilitates
learning (Baker, 2006).
Provide a job aid to help
technicians engage in
guided self-monitoring
and self-assessment.
Cultural Model - 1
Organizational culture needs to
be supportive of technicians
reporting human factor related
safety concerns that could affect
their performance.

Organizational
effectiveness increases
when leaders encourage
open lines of
communication (Clark and
Estes, 2008).
Develop a non-retribution
error reporting system that
is promoted by all levels
of management to collect
and respond to technicians
concerns in a timely
manner.
Cultural Setting - 1
Organizational culture fails to
support managers that are
overwhelmed by schedule
restraints, keeping them from
enforcing standard safety
practices.
Effective organizations
ensure that organizational
messages, rewards, policies
and procedures that govern
the work of the
organization are aligned
with or are supportive of
organizational goals and
values (Clark and Estes,
2008).
The organization needs to
cultivate a supported
safety culture and ensure
all policies and procedures
are formalized and
enacted.

HUMAN ERROR RISK REDUCTION IN AVIATION 69
Declarative knowledge solutions. Aircraft technicians need to increase their
understanding of how human factors contribute to errors in their work environment. In
considering recommendations, cognitive research will be employed to increase the technicians
understanding of the contribution human factors has on their work environment. Kirschner,
Kirschner and Paas (2006) found that increasing germane cognitive load by engaging the learner
in meaningful learning and schema construction facilitates effective learning. This would
suggest that increasing germane cognitive load by providing only relevant information related to
error-producing conditions that technicians may encounter would improve transfer by engaging
the learner in meaningful learning. During document analysis it was discovered that the current
human factors course contained a large amount of extraneous information not related to the
course objective. For example, the course was used to introduce two different programs,
Voluntary Protection Program (VPP) and Air Force Smart Operations for the 21st Century
(AFSO21). VPP does have a goal of reducing incidents but is not directly focused on aviation
operations, while AFSO21 focuses on process improvement, again not specifically aviation
related. When the researcher inquired about the extraneous information, it was explained that the
material was placed within the course until a better training opportunity came up. Unfortunately,
the training material was never revised and has continued within the course since its inception.
The recommendation then for aircraft technicians is to reduce the extraneous cognitive load and
increase germane cognitive load during human factors awareness training. This can be
accomplished by using purposeful and intentional concrete examples and identifiable case
studies, allowing technicians to better understand the contribution human factors have on their
specific work environment.
HUMAN ERROR RISK REDUCTION IN AVIATION 70
Van Gog, Pass, and Van Merriënboer (2006) found that studying worked examples and
case studies is effective when teaching problem-solving skills by decreasing extraneous load
(elements not contributing to learning), and increasing germane cognitive load (resources
devoted to long-term memory storage). The benefit of amplifying germane cognitive load was
discovered while conducting a full factorial experiment of sixty-eight first year electrotechnics
students from three senior secondary vocational schools. The Van Gog et al. (2006) study
supports the recommendation of maximizing learning during human factors awareness training
by decreasing extraneous cognitive load, and increasing germane cognitive load through the use
of concrete examples and case studies to help technicians understand human factor concepts
more effectively.
Procedural knowledge solutions. Aircraft technicians need to increase their
understanding of how to incorporate decision-making strategies to manage human error. In
considering recommendations, research related to practice and feedback will be employed to
help technicians improve decision-making strategies. Schraw and McCrudden (2006) found that
to develop mastery, individuals must acquire component skills, practice integrating them, and
know when to apply what they have learned. This would suggest that training can help
technicians employ learned skills by applying proven error reducing techniques and receiving
feedback on performance. The recommendation then is to provide an opportunity to practice
error reducing skills, and give appropriate feedback on skills implementation within human
factors awareness training.
Duvivier et al., (2011) studied the effects of deliberate practice and its role in mastery
achievement. Through exploratory factor and reliability analysis of 875 medical students’
questionnaires, it was found that training activities that include deliberate practice are effective at
HUMAN ERROR RISK REDUCTION IN AVIATION 71
improving learned skills. Additionally, Medina, Conway, Davis-Maxwell and Webb (2013)
studied the impact of feedback used to improve students' problem-solving skills. Through an
analysis of 350 pharmaceutical students at a large University, it was found that providing written
and verbal feedback strengthened the student’s ability to prioritize information and enhance
overall problem-solving skills. Both studies support the recommendation of providing
opportunities to practice skills and providing feedback of performance help technicians better
understand how to incorporate and use appropriate decision-making strategies to manage human
error.
Metacognitive knowledge solutions. Aircraft technicians need to increase their
understanding of how to reflect on their performance limitations on the job. In considering
recommendations, research related to self-monitoring and self-assessment will be employed to
help technicians reflect on their performance limitations. Baker (2006) found that the use of
metacognitive strategies facilitates learning. This would suggest that by allowing technicians an
opportunity during the training process to engage in guided self-monitoring and self-assessment,
technicians will better understand how to reflect on their performance limitations. The
recommendation then for aircraft technicians is to provide them with a job aid to help the aircraft
technicians engage in guided self-monitoring and self-assessment.
Ford, Smith, Weissbein, Gully, and Salas (1998) found through a study of 93
undergraduate students that learners who employed metacognitive strategies, to include self-
monitoring and self-evaluation, demonstrated improved knowledge acquisition and higher self-
efficacy. Additionally, Nett, Goetz, Hall, and Frenzel (2012) found through experience sampling
of 70 eleventh grade students, that metacognitive strategies, such as self-monitoring,
significantly increased motivation, cognitive resources, and test performance. Both studies
HUMAN ERROR RISK REDUCTION IN AVIATION 72
support the recommendation of providing a job aid that helps aircraft technicians engage in
guided self-monitoring and self-assessment.
Organizations culture support of safety concerns reporting solutions. Organizational
culture needs to be supportive of technicians reporting human factor related safety concerns that
could affect their performance. Clark & Estes (2008) suggest that organizational effectiveness
increases when leaders encourage open lines of communication. By opening the lines of
communication through management's involvement, people feel comfortable highlighting
concerns without fear of punishment (Edmondson & Lei, 2014), and when the fear of retribution
is removed, performance improves, allowing critical discussions to occur (Bradley, et al.,
2012). This would indicate that management's support for error reporting through open lines of
communication would improve organizational effectiveness. Therefore, it is recommended that
a non-punitive error reporting system is installed and promoted by all levels of management to
collect and respond to technicians concerns in a timely manner.
In a study of petroleum producing installations, Milch and Laumann (2018) sought to
examine safety challenges and practices that help manage inter-organizational
complexity. Using 14 in-depth, semi-structured interviews, they found that management’s role
in the field and worker involvement enhance work relations and contribute to open
communication, resulting in a higher likelihood that employees would report errors. Reason
(1998) explains that an important aspect of any safety culture is that it is an informed culture
which is dependent on the creation of an effective reporting culture. In another study of
healthcare practitioners, Anderson, Kodate, Walters, and Dodds, (2013) sought to examine the
contribution incident reporting can have on improving safety. Using qualitative interviews of
sixty-two healthcare practitioners it was discovered that error reporting had a positive effect on
HUMAN ERROR RISK REDUCTION IN AVIATION 73
safety culture. More specifically, it was found that error reporting helped change long standing
processes and improved employee attitude and knowledge. These studies support the
recommendation of instituting a non-punitive error reporting system, promoted by all levels of
management, to collect and respond to technicians concerns in a timely manner.
Organizations culture support of managers solutions. The organization needs to
support managers that are overwhelmed by schedule restraints that keep them from enforcing
standard safety practices. Clark and Estes (2008) suggest that effective organizations ensure that
organizational messages, rewards, policies and procedures that govern the work of the
organization are aligned with or are supportive of organizational goals and values. This suggests
that by aligning an organization's safety climate with established policies and procedures,
managers will obtain the needed support to enforce standard safety practices. Therefore, it is
recommended that the organization works to cultivate a supported safety culture and ensure all
policies and procedures are formalized and enacted.
In a study of production workers, Zohar and Luria (2005) sought to examine the effect of
organizational climate on safety behavior. Through 3,952 climate questionnaires and three
months of observations, organization-level climate, group-level climate, and homogeneity of
climate perceptions were assessed. It was found that group-level safety behavior is independent
of the organizational-level policies and is reliant on supervisor’s desire to implement formal
procedures. Furthermore, this study showed that by improving an organization's safety climate
and ensuring policies and procedures are formalized, a supervisor’s judgment (safety versus
productivity) is reduced, increasing group climate strength. Therefore, this study supports the
recommendation that the organization works to cultivate a supported safety culture and ensure all
policies and procedures are formalized and enacted.
HUMAN ERROR RISK REDUCTION IN AVIATION 74
Program evaluation recommendation. The execution of the recommendations for
practice implemented within the 65
th
MXG’s current human factors awareness training program
can be evaluated using the Kirkpatrick New World (2016) model of evaluation. This model of
evaluation consists of four levels; Level one reaction (how trainees react to training), Level two
learning (how much trainees have learned), Level three behavior (how much trainees have
changed their behavior), and Level four results (analyzation of the final results of training).
These four levels of evaluation, if used correctly, will help to analyze and adjust the
effectiveness and influence of training. This comprehensive evaluation and implementation plan
will also evaluate those assets technicians already excel at, such as the highlighting of human
factor related safety concerns, validating the technicians’ value in training, and propelling their
confidence in the ability to understand and apply principles to help reduce human error. The
plan for evaluating training using the Kirkpatrick New World model can be found in Appendix
G.
Conclusion
This study set out to reduce human error in aircraft maintenance, specifically human error
induced aircraft damage at the 65
th
MXG. Through a comprehensive literature review and Clark
and Estes (2008) KMO framework, several assumed influences were presented. Seeking to
validate these influences and find existing gaps, data from surveys, interviews, and human
factors awareness training documents were collected and analyzed. Findings from this research
suggested that the technicians needed to increase their understanding of how human factors
contribute to errors, how to incorporate decision-making strategies to manage human error, and
how to reflect on performance limitations. It was also discovered that the organizational culture
needs to support technicians reporting human factor related safety concerns and support
HUMAN ERROR RISK REDUCTION IN AVIATION 75
overwhelmed managers in their role of enforcing standard safety practices. The
recommendations developed through the step-by-step process corresponding with the Clark and
Estes (2008) KMO framework, and the implementation and evaluation plan based on The New
World Kirkpatrick Model found in Appendix G, will help close identified gaps ensuring the
technicians have the necessary tools to recognize and mitigate human error in aircraft
maintenance.

HUMAN ERROR RISK REDUCTION IN AVIATION 76
References
Ahmadi, A., Söderholm, P., & Kumar, U. (2010). On aircraft scheduled maintenance program
development. Journal of Quality in Maintenance Engineering, 16(3), 229-255.
Airlines, A. (1989). Flight 243, Boeing 737-200, N73711, near Maui Hawaii, April 28, 1988.
NSTB/AAR-89/03, National Transportation Safety Board, Washington, DC 20594.
Alper, S. J., & Karsh, B. T. (2009). A systematic review of safety violations in industry.
Accident Analysis & Prevention, 41(4), 739-754.
Anderson, J. E., Kodate, N., Walters, R., & Dodds, A. (2013). Can incident reporting improve
safety? Healthcare practitioners' views of the effectiveness of incident reporting.
International journal for quality in health care, 25(2), 141-150.
Andersen, P. A., McNay, D., & Peterson, J. (2012). Business jet aircraft industry: structure and
factors affecting competitiveness. US International Trade Commission.
Atak, A., & Kingma, S. (2011). Safety culture in an aircraft maintenance organisation: A view
from the inside. Safety Science, 49(2), 268-278.
Authority, C. A. (2008). Global fatal accident review 1997–2006. London: The Stationery
Office.
Baker, L. (2006). Metacognition. Retrieved from http://www.education.com/reference/
article/metacognition/
Batteau, A. W. (2001). The anthropology of aviation and flight safety. Human Organization,
201-211.
Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and
Human Decision Processes, 50(2), 248-287.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman.
HUMAN ERROR RISK REDUCTION IN AVIATION 77
Bandura, A., & Locke, E. A. (2003). Negative self-efficacy and goal effects revisited. Journal of
Applied Psychology, 88(1), 87.
Bearman, C., Paletz, S. B., Orasanu, J., & Thomas, M. J. (2010). The breakdown of coordinated
decision making in distributed systems. Human Factors, 52(2), 173-188.
Begur, S. H., & Babu, J. A. (2016). Human factors in aircraft maintenance. Human Factors,
3(3).
Besnard, D., & Hollnagel, E. (2014). I want to believe: some myths about the management of
industrial safety. Cognition, Technology & Work, 16(1), 13-23.
Bogdan, R.C. & Biklen, S.K. (2007) Qualitative research for education: An introduction to
theories and methods. Boston, MA: Pearson.
Blouin, N., Deaton, J., Richard, E., & Buza, P. (2014). Effects of stress on perceived
performance of collegiate aviators. Aviation Psychology and Applied Human Factors,
4(1), 40-49.
Borenstein, S., & Rose, N. L. (2014). How airline markets work… or do they? Regulatory
reform in the airline industry. In Economic Regulation and Its Reform: What Have We
Learned? (pp. 63-135). University of Chicago Press.
Bottani, E., Monica, L., & Vignali, G. (2009). Safety management systems: Performance
differences between adopters and non-adopters. Safety Science, 47(2), 155-162.
Bowen, E. E. (2013). Predicting impact of maintenance resource management training utilizing
individual difference variables. Journal of Aviation/Aerospace Education & Research,
22(3), 9-17.

HUMAN ERROR RISK REDUCTION IN AVIATION 78
Bradley, B. H., Postlethwaite, B. E., Klotz, A. C., Hamdani, M. R., & Brown, K. G. (2012).
Reaping the benefits of task conflict in teams: The critical role of team psychological
safety climate. Journal of Applied Psychology, 97(1), 151.
Brycz, H., & Karasiewicz, K. (2011). Metacognition and self-regulation: The metacognitive self-
scale. Acta Neuropsychologica, 9(3), 263-281.
Cetin, I., Sendurur, E., & Sendurur, P. (2014). Assessing the impact of meta-cognitive training
on students' understanding of introductory programming concepts. Journal of
Educational Computing Research, 50(4), 507-524.
Chang, Y. H., & Wang, Y. C. (2010). Significant human risk factors in aircraft maintenance
technicians. Safety Science, 48(1), 54-62.
Chaturvedi, A. K., Craft, K. J., Kupfer, D. M., Burian, D., & Canfield, D. V. (2011).
Resolution of aviation forensic toxicology findings with the aid of DNA profiling.
Forensic Science International, 206(1), 81-86.
Chen, C. C., Chen, J., & Lin, P. C. (2009). Identification of significant threats and errors
affecting aviation safety in Taiwan using the analytical hierarchy process. Journal of Air
Transport Management, 15(5), 261-263.
Chiu, M. C., & Hsieh, M. C. (2016). Latent human error analysis and efficient improvement
strategies by fuzzy TOPSIS in aviation maintenance tasks. Applied Ergonomics, 54, 136-
147.
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. IAP.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches. Thousand Oaks, CA: Sage Publications.
HUMAN ERROR RISK REDUCTION IN AVIATION 79
Dekker, S. W., & A. (2009). Just culture: Who gets to draw the line? Cognition, Technology &
Work, 11(3), 177-185.
Dostaler, I. (2010). Avoiding rework in product design: Evidence from the aerospace
industry. The International Journal of Quality & Reliability Management, 27(1), 5-26.
Duvivier, R. J., van Dalen, J., Muijtjens, A. M., Moulaert, V. R., van der Vleuten, C. P., &
Scherpbier, A. J. (2011). The role of deliberate practice in the acquisition of clinical
skills. BMC Medical Education, 11(1), 101.
Eccles, J. (2006). Expectancy value motivational theory. Retrieved from
http://www.education.com/reference/article/expectancy-value-motivational-theory/.
Edmondson, A. C., & Lei, Z. (2014). Psychological safety: The history, renaissance, and future
of an interpersonal construct. Annu. Rev. Organ. Psychol. Organ. Behav., 1(1), 23-43.
Endsley, M. R. (2017). Direct measurement of situation awareness: Validity and use of SAGAT.
In Situational Awareness (pp. 129-156). Routledge.
FAA. (2010). Safety management systems for aviation service providers (Advisory Circular 120-
92a). Washington, DC: Author.
Fink, A. (2013). Chapter 2: The Survey Form. In How to conduct surveys: A step-by-step guide
(5th ed.) (pp. 29-56). Thousand Oaks, CA: SAGE Publications.
Findlay, S. J., & Harrison, N. D. (2002). Why aircraft fail. Materials Today, 5(11), 18-25.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–
developmental inquiry. American Psychologist, 34(10), 906.
Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998). Relationships of
goal orientation, metacognitive activity, and practice strategies with learning outcomes
and transfer. Journal of Applied Psychology, 83(2), 218.
HUMAN ERROR RISK REDUCTION IN AVIATION 80
Fowler Jr, F. J. (2013). Survey research methods. Sage publications.
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (1990). How to design and evaluate research in
education. New York. NY: McGraw Hall.
Galesic, M., & Bosnjak, M. (2009). Effects of questionnaire length on participation and
indicators of response quality in a web survey. Public Opinion Quarterly, 73(2), 349-360.
Garrette, B., Castañer, X., & Dussauge, P. (2009). Choosing between collaborative and
autonomous production in the worldwide aircraft industry, 1949–2000. Strategic
Management Journal, 30(8), 885-894.
Glendon, A. I., & Clarke, S. (2015). Human safety and risk management: A psychological
perspective. Crc Press.
Glesne, C. (2011). Becoming qualitative researchers: An introduction. Pearson.
Gluyas, H., & Morrison, P. (2014). Human factors and medication errors: a case study. Nursing
Standard (2014+), 29(15), 37.
Grama, B., & Todericiu, R. (2016). Change, resistance to change and organizational cynicism.
Studies in Business and Economics, 11(3), 47-54.
Habtoor, N. (2016). Influence of human factors on organisational performance. International
Journal of Productivity and Performance Management, 65(4), 460-484.
Hagen, J. U. (2018). Crew Resource Management Revisited. In How Could This Happen? (pp.
233-251). Palgrave Macmillan, Cham.
Halpin, S. M. (2008). Does team training improve team performance? A meta-analysis. Human
Factors, 50(6), 903-933.
HUMAN ERROR RISK REDUCTION IN AVIATION 81
Harris, D., & Li, W. C. (2011). An extension of the human factors analysis and classification
system for use in open systems. Theoretical Issues in Ergonomics Science, 12(2), 108-
128.
Hauser, T. U., Allen, M., Rees, G., Dolan, R. J., & NSPN Consortium. (2017). Metacognitive
impairments extend perceptual decision making weaknesses in compulsivity. bioRxiv,
098277.
Helmreich, R.L., A.C. Merritt, and J.A. Wilhelm. 1999. The evolution of Crew Resource
Management training in commercial aviation. International Journal of Aviation
Psychology 9 (1): 19–32.
International Civil Aviation Organization (ICAO). (2009). Safety Management Manual (SMM)
(2nd ed.). (ICAO Order No. 9859). Quebec, Montréal, Canada: Author.
Johnson, M. P. (2015). Sustainability management and small and medium-sized enterprises:
Managers' awareness and implementation of innovative tools. Corporate Social
Responsibility and Environmental Management, 22(5), 271-285.
Kapp, E. A. (2012). The influence of supervisor leadership practices and perceived group safety
climate on employee safety performance. Safety Science, 50(4), 1119-1124.
Keyton, J., Ford, D. J., & Smith, F. I. (2008). A mesolevel communicative model of
collaboration. Communication Theory, 18, 376–406.
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick’s four levels of training
evaluation.Alexandria, VA: ATD Press.
Kirschner, P., Kirschner, F., & Paas, F. (2006). Cognitive load theory.
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory Into Practice,
41(4), 212-218.
HUMAN ERROR RISK REDUCTION IN AVIATION 82
Latorella, K. A., & Prabhu, P. V. (2000). A review of human error in aviation maintenance and
inspection. International Journal of Industrial Ergonomics, 26(2), 133-161.
Liou, J. J., Yen, L., & Tzeng, G. H. (2008). Building an effective safety management system for
airlines. Journal of Air Transport Management, 14(1), 20-26.
Littlepage, G. E., Hein, M. B., Moffett, R. G. I.,II, Craig, P. A., & Georgiou, A. M. (2016). Team
training for dynamic cross-functional teams in aviation: Behavioral, cognitive, and
performance outcomes. Human Factors, 58(8), 1275-1288.
Locke, L. F., Silverman, S. J., & Spirduso, W. W. (2010). Reading and understanding research.
(3rd ed.). Thousand Oaks: SAGE.
Marais, K. B., & Robichaud, M. R. (2012). Analysis of trends in aviation maintenance risk: An
empirical approach. Reliability Engineering & System Safety, 106, 104-118.
Marquardt, N., Robelski, S., & Jenkins, G. G. (2011). Designing and evaluating a crew resource
management training for manufacturing industries. Human Factors and Ergonomics in
Manufacturing, 21(3), 287.
Maxwell, J. A. (2013). Qualitative research design: An interactive approach (3rd ed.). Thousand
Oaks, CA: SAGE Publications.
McEwan, E. K., & McEwan, P. J. (2003). Making sense of research. Thousand Oaks, CA: Sage
Publications.
McKenna, J. T. (2002, October). Maintenance resource management programs provide tools for
reducing human error. Flight Safety Digest, 21(10), 1-15
Medina, M. S., EdD., Conway, S. E., PharmD., Davis-Maxwell, T., & Webb, R., M.P.H. (2013).
The impact of problem-solving feedback on team-based learning case responses.
American Journal of Pharmaceutical Education, 77(9), 189.
HUMAN ERROR RISK REDUCTION IN AVIATION 83
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation (4th ed.). San Francisco, CA: Jossey-Bass.
Milch, V., & Laumann, K. (2018). Sustaining safety across organizational boundaries: A
qualitative study exploring how interorganizational complexity is managed on a
petroleum-producing installation. Cognition, Technology & Work, 20(2), 179-204.
Nathanael, D., Tsagkas, V., & Marmaras, N. (2016). Trade-offs among factors shaping operators
decision-making: The case of aircraft maintenance technicians. Cognition, Technology &
Work, 18(4), 807-820.
Nett, U. E., Goetz, T., Hall, N. C., & Frenzel, A. C. (2012). Metacognitive strategies and test
performance: An experience sampling analysis of students' learning behavior. Education
Research International, 2012.
NTSB (2016). Strategic Plans & Reports. Retrieved June 25, 2016, from
http://www.ntsb.gov/about/reports/Pages/default.aspx
O’Connor, P., O’Dea, A., Kennedy, Q., & Buttrey, S. E. (2011). Measuring safety climate in
aviation: A review and recommendations for the future. Safety Science, 49(2), 128-138.
Pajares, F. (2006). Self-efficacy theory. Retrieved from
http://www.education.com/reference/article/self-efficacy-theory/.
Patankar, M. S., & Taylor, J. C. (2008). MRM training, evaluation, and safety management. The
International Journal of Aviation Psychology, 18(1), 61-71.
Patankar, M. S., & Taylor, J. C. (2017). Applied Human Factors in Aviation Maintenance.
Routledge.
Patton, M. Q. (2002). Qualitative interviewing. Qualitative research and evaluation methods, 3,
344-347. Thousand Oaks, CA: SAGE Publications.
HUMAN ERROR RISK REDUCTION IN AVIATION 84
Pentland, B. T., & Feldman, M. S. (2008). Designing routines: On the folly of designing
artifacts, while hoping for patterns of action. Information and Organization, 18(4), 235-
250.
Radovan, M., & Makovec, D. (2015). Relations between students' motivation, and perceptions of
the learning environment. CEPS Journal: Center for Educational Policy Studies Journal,
5(2), 115-138.
Reason, J. (1998). Achieving a safe culture: theory and practice. Work & Stress, 12(3), 293-306.
Reason, J. (1990). Human Error. Cambridge university press.
Reason, J., & Hobbs, A. (2017). Managing Maintenance Error: a Practical Guide. CRC Press.
Reid-Searl, K., Moxham, L., & Happell, B. (2010). Enhancing patient safety: the importance of
direct supervision for avoiding medication errors and near misses by undergraduate
nursing students. International Journal of Nursing Practice, 16(3), 225-232.
Roesler, A. (2009). Lessons from Three Mile Island: The Design of Interactions in a High-Stakes
Environment. Visible Language, 43(2/3), 169.
Rueda, R. (2011). The 3 dimensions of improving student performance. New York: Teachers
College Press.
Safety Management System. (2016, July 05). Retrieved May 11, 2018, from
https://www.faa.gov/about/initiatives/sms/specifics_by_aviation_industry_type/121/
Salas, E., Bowers, C. A., & Edens, E. (Eds.). (2001). Improving Teamwork in Organizations:
Applications of Resource Management Training. CRC Press.
Saldana, J., & Omasta, M. (2017). Qualitative research: Analyzing life. SAGE Publications.
Salkind, N. J. (2017). Statistics for people who (think they) hate statistics: Using Microsoft Excel
2016 (4th ed.). Thousand Oaks, CA: SAGE.
HUMAN ERROR RISK REDUCTION IN AVIATION 85
Savioja, P., Liinasuo, M., & Koskinen, H. (2014). User experience: Does it matter in complex
systems? Cognition, Technology & Work, 16(4), 429-449.
Schein, E. H. (2006). Organizational Culture and Leadership (Vol. 356). John Wiley & Sons.
Schmidt, A. M., & DeShon, R. P. (2010). The moderating effects of performance ambiguity on
the relationship between self-efficacy and performance. Journal of Applied Psychology,
95(3), 572-581.
Schraw, G., & McCrudden, M. (2006). Information processing theory. Retrieved from
http://www.education.com/reference/article/information-processing-theory/
Schunk, D. H., & Pajares, F. (2009). Self-efficacy theory. Handbook of Motivation at School, 35-
53.
Schunk, D. H., Meece, J. R., & Pintrich, P. R. (2012). Motivation in Education: Theory,
Research, and Applications. Pearson Higher Ed.
Schwatka, N. V., & Rosecrance, J. C. (2016). Safety climate and safety behaviors in the
construction industry: The importance of co-workers commitment to safety. Work
(Reading, Mass.), 54(2), 401-413.
Shanmugam, A., & Paul Robert, T. (2015). Human factors engineering in aircraft maintenance:
A review. Journal of Quality in Maintenance Engineering, 21(4), 478-505.
Shenge, N. A., PhD. (2014). Training evaluation: Process, benefits, and issues. Ife Psychologia,
22(1), 50-58.
Stolzer, A. J. (2017). Safety Management Systems in Aviation. Routledge.
Strauch, B. (2017). The automation-by-expertise-by-training interaction: Why automation-
related accidents continue to occur in sociotechnical systems. Human Factors, 59(2),
204-228.
HUMAN ERROR RISK REDUCTION IN AVIATION 86
Tannenbaum, S. I., & Cerasoli, C. P. (2013). Do team and individual debriefs enhance
performance? A meta-analysis. Human Factors, 55(1), 231.
Tomás, J. M., Cheyne, A., & Oliver, A. (2011). The relationship between safety attitudes and
occupational accidents: The role of safety climate. European Psychologist, 16(3), 209-
219.
Taylor, J. C. (1998). Evaluating the effects of maintenance resource management (MRM)
interventions in airline safety.
Towler, A., Watson, A., & Surface, E. A. (2014). Signaling the importance of training. Journal of
Managerial Psychology, 29(7), 829-849.
U.S. Air Force - Mission. (n.d.). Retrieved July 10, 2018, from
https://www.airforce.com/mission
van, d. S., & Veenman, M. V. (2014). Metacognitive skills and intellectual ability of young
adolescents: A longitudinal study from a developmental perspective. European Journal
of Psychology of Education, 29(1), 117-137.
Van Gog, T., Paas, F., & Van Merriënboer, J. J. (2006). Effects of process-oriented worked
examples on troubleshooting transfer performance. Learning and Instruction, 16(2),
154-164.
Vancouver, J. B., Thompson, C. M., Tischner, E. C., & Putka, D. J. (2002). Two studies
examining the negative effect of self-efficacy on performance. Journal of Applied
Psychology, 87(3), 506-516.
Vogt, J., Leonhardt, J., Koper, B., & Pennig, S. (2010). Human factors in safety and business
management. Ergonomics, 53(2), 149-163.
HUMAN ERROR RISK REDUCTION IN AVIATION 87
Wachter, J. K., & Yorio, P. L. (2013). Human performance tools: Engaging workers as the best
defense against errors & error precursors. Professional Safety, 58(2), 54-64.
Walker, T. J., Dolruedee, J. T., & Lin, M. Y. (2005). On the performance of airlines and
airplane manufacturers following aviation disasters. Canadian Journal of Administrative
Sciences, 22(1), 21-34.
Wilpert, B. (2008). Psychology and human factors engineering. Cognition, Technology & Work,
10(1), 15-21.
Woody, C. (2018, March 05). The Air Force is still grappling with a personnel shortage - and
one part of the solution can't be rushed. Retrieved May 5, 2018, from
https://www.businessinsider.com/air-force-adding-maintainers-but-training-and-
experience-take-longer-2018-3
Yeo, G. B., & Neal, A. (2006). An examination of the dynamic relationship between self-
efficacy and performance across levels of analysis and levels of specificity. Journal of
Applied Psychology, 91(5), 1088-1101.
Zohar, D. (1980). Safety climate in industrial organizations: theoretical and applied implications.
Journal of Applied Psychology, 65(1), 96.
Zohar, D., & Luria, G. (2005). A multilevel model of safety climate: cross-level relationships
between organization and group-level climates. Journal of Applied Psychology, 90(4),
616.

HUMAN ERROR RISK REDUCTION IN AVIATION 88
Appendix A: Participating Stakeholders with Sampling Criteria
for Interview and Survey

While a complete gap analysis focuses on all stakeholders, for practical purposes the
stakeholder of interest in this study was the aircraft maintenance technicians assigned to the 65
th

MXG. While the FAA does not govern maintenance operations of military aircraft, nor human
factors awareness training requirements for military technicians, Headquarters Air Force
recommends that all aircraft technicians attend a human factors awareness training course.
The USAF uses “skill-levels” to ensure timely progression and adequate training of its
force, to include aircraft technicians. A 3-level apprentice, someone recently out of technical
training with less than a year on-station, may not complete a task without direct supervision. A
5-level journeyman, generally awarded after 12 months of on-the-job training, can only perform
routine, non-critical work and must rely on a higher skill level to sign-off any critical tasks. A 7-
level craftsman, generally obtained after five years of experience, can perform work and then
release or certify the aircraft for flight. The 9-level superintendent, typically achieved after 19+
years of experience, oversees the whole operation and rarely performs maintenance on the
aircraft. Therefore, it is 5- and 7-levels that make up the majority of aircraft technicians
performing maintenance on the aircraft assigned to the 65
th
MXG.
Of the over 18,000 employees assigned to SAFB, 1,253 are aircraft maintenance
technicians. Of the 1,253 technicians, 864 have direct contact with assigned aircraft. Of the 864
technicians having direct contact with assigned aircraft, 170 are classified as having a 3-skill
level and unable to perform maintenance without being supervised. Those within this group of
694 technician’s, those having direct contact with assigned aircraft, a minimum 5-skill level, and
have completed the human factors awareness course, were the primary population of focus.

HUMAN ERROR RISK REDUCTION IN AVIATION 89
Survey Sampling Criteria and Rationale
Criterion 1. 65
th
Maintenance Group technicians having direct contact, therefore a
responsibility of safety, and overall airworthiness of assigned aircraft.
Criterion 2. 65
th
Maintenance Group technicians that have a minimum 5-skill level,
therefore an opportunity to perform aircraft maintenance unsupervised.
Criterion 3. 65
th
Maintenance Group technicians that have completed the current human
factors awareness sixteen-hour introductory course.
Survey Sampling (Recruitment) Strategy and Rationale
Probability cluster random sampling method was used to determine who received the
human factors awareness training effectiveness questionnaire. This sampling method was
chosen to minimize the overall impact on the organization that surveying the entire technician
population would inflict. By using probability sampling to determine participants as discussed
by Creswell (2014), Maxwell (2013), and Merriam and Tisdell (2016), a representative sample of
the qualified population was obtained, giving the ability to generalize the 65
th
MXG technician
populace as a whole. Furthermore, a single stage cluster random sampling technique is clear and
uncomplicated allowing this study to allocate its resources to the few randomly selected clusters
(Creswell, 2014). The clusters chosen were two of four equally distributed aircraft maintenance
squadrons. This choice helped avoid skewed results from over or underrepresentation and
limited possible sampling error. Additionally, the selection of the two maintenance squadrons,
or “clusters,” resulted in an equal representative sample from each shift, each area of expertise,
and level of experience, preventing a significant proportion of the population un-sampled.
With 694 aircraft technicians within the pool of potential respondents, 345 were selected
to participate in this cross-sectional survey. With 345 being selected to participate it is important
HUMAN ERROR RISK REDUCTION IN AVIATION 90
to note that during the survey period, 28 were on leave, 57 were deployed, and six were out-
processing for a move to another base, leaving a final survey sampling population of 254. Using
Fowler’s (2013) table with a margin of error of +/–3.5%, a confidence error of 95%, and a 50/50
chance that the sample is characteristic, the desired sample size is 190. Stratification of the
population prior to distribution did not occur due to the unavailability and confidentiality of
specific characteristics of technicians (such as gender and race) and therefore were unable to
determine if the sample reflects the true proportion of the population. The distribution of the
survey occurred in the middle of the year as to avoid the typical high absentee rates that routinely
take place during the fourth quarter due to the holiday season. However, typical of a military
airlift organization, deployment requirements of both personnel and aircraft are high year-round
thus were not distributed to deployed employees. The primary instrument of data collection was
reliant on an online survey tool distributed through two of the four 65
th
MXG squadrons. The
director of human factors sent the survey instrument to the respective squadron commanders
down through the individual shift leaders who invited the appropriate technicians. This option
was chosen to meet all criterions and to achieve a higher completion rate.
The rationale for selecting a survey sampling is primarily based on the knowledge and
skills, motivation, and organizational elements that influenced the 65
th
MXG technicians of their
understanding and application of human factors principles. Merriam and Tisdell (2016)
illustrated that to thoroughly explain the attributes of a given relationship between events, a
survey may be most appropriate. Additionally, Maxwell (2013) explained that sampling is
essential with a large population where generalizability is important. Finally, Creswell (2014)
clarified that the use of surveys is central to the examination of relationships between variables
and they also provide a rapid turnaround in data collection from such a large population.
HUMAN ERROR RISK REDUCTION IN AVIATION 91
Interview Sampling (Recruitment) Criteria and Rationale Strategy and Rationale
Criterion 1. 65
th
MXG technicians having direct contact, therefore a responsibility of
safety, and overall airworthiness of assigned aircraft.
Criterion 2. 65
th
Maintenance Group technicians that have a minimum 5-skill level,
therefore an opportunity to perform aircraft maintenance unsupervised.
Criterion 3. 65
th
MXG technicians that have completed the current human factors
awareness 16-hour introductory course.
Interview Sampling (Recruitment) Criteria and Rationale
Technicians recruited to interview were based on purposeful convenience sampling.
Merriam and Tisdell (2016) explained that purposeful sampling enables the researcher to select
those that they expect will offer the most insight into the study. The interview participants
initially volunteered through the last question of the human factors awareness training
effectiveness survey. The last question of the survey pertained to whether they were willing to
participate in an interview or discussion on the topic of this study, at which point a separate link
was provided to the interview volunteers to offer their contact information outside of the survey.
Upon survey completion, the researcher chose from a pool of interview volunteers using a
maximum variation sampling strategy. This strategy allowed the researcher to maximize the
diversity of participants by selecting technicians who possess a broad range of perspectives and
possible responses (Merriam & Tisdell, 2016). The sample consisted of seven technician’s that
had direct contact with assigned aircraft, a minimum 5-skill level, and had completed the sixteen-
hour human factors awareness introductory course. The individual interviews helped gain useful
insight and context by allowing the technicians to describe what is important to them and the
level of knowledge gained through the human factors course. These interviews also provided
HUMAN ERROR RISK REDUCTION IN AVIATION 92
material that would not have otherwise surfaced during surveys and yielded a larger margin of
error but were necessary due to limited resources. The interviews were open-ended in that the
researcher asked general questions of the technicians allowing them to freely provide their views
(Creswell, 2014). The intent of this detailed “ethnographic research” was to obtain a holistic
picture of the 65
th
MXG technicians revealing their everyday experiences (Fraenkel, Wallen &
Hyun 1990).

HUMAN ERROR RISK REDUCTION IN AVIATION 93
Appendix B: Protocols
Survey Instrument
1. __ I agree to patriciate and understand all responses are confidential
__ I do not wish to participate

2. Have you attended Human Factors Awareness Training? Yes____ No____
3. Do you currently perform maintenance on aircraft? Yes____ No____
4. Current pay Grade ___
5. Years in Aviation ___
6. Past Experience or Training (check all that apply)
Military Technical Training ____ Trade School (Part 147) ____
Airline (Part 121) _____ On-demand/Commuter (Part 135) ____
Repair Station/MRO (Part 145) ____ General Aviation _____

7. What would you consider to be the top three most influential factors that contribute to human
error (please choose three)?

Lack of Communication
Lack of Awareness
Pressure
Stress
Lack of Assertiveness
Norms
Fatigue
Distraction
Lack of Resources
Complacency
Lack of Knowledge
Lack of Teamwork

8. Human factors awareness training has the potential to increase aviation safety and teamwork
effectiveness.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

9. Human factors awareness training is meaningful and not simply an act of compliance.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

10. How much has human factors awareness training changed your behavior on the job?

A. No change
B. Slight change
C. Moderate change
D. Considerable change

HUMAN ERROR RISK REDUCTION IN AVIATION

94

11. I am confident in my ability to apply human factors principles in abnormal situations.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

12. How often do you work fatigued?

A. Very frequently
B. Frequently
C. Seldom
D. Very seldom

13. I perform effectively during critical phases of work, even when fatigued.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

14. How frequently does your team engage in PRE-BRIEFS before critical/high-risk tasks?

A. Very frequently
B. Frequently
C. Seldom
D. Very seldom

15. I know the proper channels to route questions or concerns regarding safety.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

16. I am encouraged by my supervisor and co-workers to report any unsafe conditions I may
observe.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

HUMAN ERROR RISK REDUCTION IN AVIATION

95

17. I feel comfortable reporting errors through our current error reporting system.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

18. I believe my leadership would act upon my safety recommendations if I communicated them
properly.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

19. My supervisor actively supports maintenance human factors.

A. In actions
B. In words
C. In both words and actions
D. Not at all

20. My flight chief actively supports maintenance human factors.

A. In actions
B. In words
C. In both words and actions
D. Not at all

21. My leadership actively supports maintenance human factors.

A. In actions
B. In words
C. In both words and actions
D. Not at all

22. I believe that supervisors/leadership would NOT compromise safety for production.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

HUMAN ERROR RISK REDUCTION IN AVIATION

96
23. We receive detailed feedback regarding the organization’s performance.

A. Very frequently
B. Frequently
C. Seldom
D. Very seldom

24. The procedures we have in place for shift/task turnover are efficient at reducing errors.

1 Strongly Disagree
2 Slightly Disagree
3 Slightly Agree
4 Strongly Agree

25. Would you be willing to participate in a confidential interview regarding the topics of this
study in the future?

1 Yes (please provide contact information below)
2 No

If you are willing to participate in an interview or discussion on the topic of this study, please
enter the requested information at the following link*:

(*link provided within survey)

Interview Protocol

I would like to first begin with expressing my gratitude for agreeing to participate in my study.
Thank you taking some time out of your extremely busy schedule to meet with me and answer
some questions. This interview will take about thirty minutes, although we have allocated an
hour for some cushion on time.

I am currently enrolled in a doctoral program at USC and am conducting a study on human
factors awareness training effectiveness within aviation. I am focusing on the knowledge,
motivation, and organizational influences as they relate to the aircraft damage event reduction
goal at the 65
th
MXG.

Today, I am not here to make a professional assessment or judgment of your performance, I am
only here as a researcher collecting data for my study. The information you share with me will
be placed into my study as part of the data collection. In addition, this interview is completely
confidential and your name or responses will not be disclosed to anyone or anywhere outside the
scope of this study and will be known only to me specifically for this data collection. While I
may choose to utilize a direct quote from you in my study, I will not provide your name
specifically and will make the best effort possible to remove any potential identifying data
information. I will gladly provide you with a copy of my final product upon request.

HUMAN ERROR RISK REDUCTION IN AVIATION

97
During the interview, I will be utilizing a recording device to assist me in capturing all of your
responses accurately and completely. This recording will not be shared with anyone outside the
scope of this project. The recording will be transferred to my password-protected files on a
cloud file storage account and deleted from the recording device immediately upon transfer. The
recording will then be destroyed after two years from the date my dissertation defense is
approved.

With that, do you have any questions about the study before we get started? If not, I would like
your permission to begin the interview. May I also have your permission to record this
conversation?

1. How can human factors effect maintenance operations?

2. Describe your work practices in relation to safety.

3. What are some strategies you use to help manage consequences of human error?

4. What “safety nets” are present within your daily routine?

5. Do you feel it is important to understand how human factors affect your behavior and
performance, if so, why? If not, why not?

6. What aspects of human factors training (if any) have you implemented within your daily
routine?

7. Imagine you were training a new technician; how would you describe the skills needed to
apply human factors principles?

8. Tell me about a time you had to use problem solving skills in an abnormal situation.

9. Why do you think employees lose situational awareness in high workload situations?

10. Can you share a specific example of when a change in policy or procedure affected how you
accomplished a task?

HUMAN ERROR RISK REDUCTION IN AVIATION

98
Appendix C: Credibility and Trustworthiness
Merriam and Tisdell (2016) discussed the importance of maintaining credibility and
trustworthiness as the data collected is directly tied to the trustworthiness and credibility of the
researcher. Merriam and Tisdell explained that the design of the study and depth of analysis is
what makes it trustworthy and that using triangulation within the study increases its credibility.
Patton (2002) expressed that it is up to the researcher to show that the methods used “involve
rigor and skill” (p. 340). Furthermore, Bogdan and Biklen (2007) explained the researcher's
conduct helps develop trust which can result in information that was not available without it.
I maintained credibility and trustworthiness during the data collection phase through
critical self-reflection and triangulation throughout the entire process. This reflection and
triangulation were accomplished through the writing of descriptive memorandums and field
notes during the interviews and document reviews. These memorandums and field notes
developed into what Merriam and Tisdell (2016) described as an “audit trail” (p. 252) which
documented the development of the completed study. This audit trail consisted of the pre-
interview and pre-document review thoughts, actual transcripts, and analyses followed by an all-
encompassing reflection. This audit trail also helped identify researcher bias, the existence of
perceived power dynamics, personality, perceptions, and internal validity. Merriam and Tisdell
(2016) explained that this validity is how findings match reality and can offer evidence of
trustworthiness.

HUMAN ERROR RISK REDUCTION IN AVIATION

99
Appendix D: Validity and Reliability
According to Salkind (2017), validity is how we know that the instrument is measuring
what it purports to measure. By understanding that the elements of the survey are relevant and
demonstrative of our purpose indicates they are gauging appropriately to their intent. Validity is
also essential when looking at a small sample of the overall subject population. Attempting to
show that the sample is representative of the population, the validity of the instrument must be
established which allows the study to be generalized across groups.
Validity for this study has been maintained during data collection by creating questions
that center on the problem of practice, ensuring interrelated subjects are included, and verifying
the results are generalizable to the overall aircraft technician population. This approach of
maintaining validity was accomplished through the use of field tests. The strategies used to
support validity are important because if respondents do not understand what is being asked, the
data will not correctly represent their perspectives.
Salkind (2017) described reliability as the replicability of the processes and results,
meaning that each survey question means the same to everyone. It is also important to note that
a reliable instrument that produces the same outcome every time is verification that the approach
used is successful as the results are consistent. Stability when collecting and analyzing data is
desirable because it aids in drawing conclusions, making generalizability claims, and formulating
theories. Salkind (2017) explained further that if the tools that are used in the collection of data
are “unreliable or invalid” (p. 156), the results may be questionable and even problematic.
Reliability for this study was maintained during data collection by using what Salkind
(2017) describes as “internal consistency reliability” (p. 165). Internal consistency characterizes
the reliability of a survey by measuring the same idea through multiple constructs and verifying
HUMAN ERROR RISK REDUCTION IN AVIATION

100
their consistency. This method of validation was accomplished through the use of multiple
survey questions within the same subject manner but articulated differently to aid in the
identification of unreliable responses. This redundant questioning facilitates validity-checking
allowing the aberrant data set to be identified and reviewed for accuracy.

HUMAN ERROR RISK REDUCTION IN AVIATION

101
Appendix E: Ethics
In a mixed methods research study, the focus is on the relationship between specific
events and their outcome (Merriam & Tisdell, 2016). Even though the majority of data
collection is through survey and one-on-one interviews rather than observations, ethical
decisions are important because the goal is to gain a description of attitudes and opinions of the
65
th
MXG’s technicians. Glesne (2011) noted that the rights of respondents and their privacy are
the primary concerns. In order to respect the respondent’s rights, an information sheet for
exempt research was used to ensure technicians understood that participation was voluntary and
they may withdraw at any time. Each respondent read this information sheet found at the
beginning of the survey and interview that outlined the underlying principles of this study. The
respondents also received a reminder that their participation was voluntary, collected data is
confidential and secure, and that they may withdraw at any time. Furthermore, the interview was
conducted in a quiet conference room away from distraction of the maintenance environment
which also protected the participants identity. The survey participants’ identity has also been
protected as the survey could be completed wherever the participant elected with the IP
addresses of the chosen computer not being collected or recorded. Collected data was then saved
using Encapsulating Security Payload (ESP) for data encryption and stored on a password
protected server to safeguard it from an unlikely data breach. In order to protect the technicians’
rights and welfare and to verify that this study follows necessary rules and guidelines, it was
submitted to the University of Southern California Institutional Review Board (IRB).
It is also important to note that the researcher has no professional ties or affiliation with
Sterling Air Force Base, the 65
th
Maintenance Group or its employees; therefore, concerns of
dual roles/relationships, subordinate - supervisor influence, or coercion and pressure to
HUMAN ERROR RISK REDUCTION IN AVIATION

102
participate did not exist. Glesne (2011) explained that the familiarity of the researcher and
participants are directly proportional to the emergence of special obligations and expectations.
The possibility of obligations and expectations affecting this study are small due to the
relationship between researcher and participants.
The recognition of attitudes, unconscious biases, and other influences of the researcher
unrelated to the credentials, behaviors, and characteristics of respondents can manipulate data
evaluation. The researcher has worked within the aviation industry for over 25 years, has
developed, implemented, and administered multiple human factor training courses, and provided
ground up training solution consulting services to several industry-leading companies. This
experience may alter the overall perspective due to an elevated cognitive expectation relative to
human factor awareness training within aviation. Knowing and understanding that the researcher
must identify with the approaches technicians take to understand human factors during the
practical learning process allowed for proper and comprehensive data analysis.
Finally, the researcher reminded all respondents that they will not be incentivized for
their participation. It is also important to note that at no time was a respondent coerced to
provide answers, and their participation was given without outside influence from the researcher.
These ethical protocols help remind the respondents that they were freely choosing to participate
in the study and that they understood the procedures and any potential risks.

HUMAN ERROR RISK REDUCTION IN AVIATION

103
Appendix F: Integrated Implementation and Evaluation Plan
Implementation and Evaluation Framework
This implementation and evaluation plan, based on The New World Kirkpatrick Model,
honors and maintains the original Kirkpatrick Four-Level Model of Evaluation (Kirkpatrick &
Kirkpatrick, 2016). The New World Kirkpatrick Model still includes four levels; (1) Reaction;
(2) Learning; (3) Behavior; and (4) Results; but recommends that the evaluation plans start in
reverse from the original beginning with level four (Results). By starting with level four
(Results), the evaluation is able to predict the extent to which desired outcomes transpire as a
result of training. Leading indicators, such as employee engagement and quality, are also
defined within level four and help link organizational results to individual efforts. Level three
(Behavior) is the extent to which what is learned in training is applied on the job. Level three is
critical in that it includes a means of reinforcing, encouraging, and rewarding performance of
fundamental behaviors on the job. Level two (Learning) explores the degree to which
participants obtain the expected knowledge, skills, attitude, confidence and commitment based
on their contribution in training. Level two is significant because it helps connect learning to
behavior. Finally, level one (Reaction) measures the extent participants find the training
beneficial and relevant to their jobs. Level one is essential because it shows if the participants
are actively engaged in and contributing to learning and the likelihood they will apply what they
learned on the job (Kirkpatrick & Kirkpatrick, 2016).
Organizational Purpose, Need and Expectations
The 65
th
MXG, an aviation unit within the USAF, provides safe and reliable airlift
capability worldwide by means of the C-17 Globemaster III, a heavy multipurpose cargo aircraft.
A goal the 65
th
MXG has set for 2019 is to reduce human factor related aircraft damage events
HUMAN ERROR RISK REDUCTION IN AVIATION

104
by 10%. These human factors related events can be a precursor to a more significant incident or
accident that would have a crippling effect on the 65
th
MXG and the USAF. Therefore, the 65
th
MXG’s leadership team would like to see 100% of their technicians demonstrate knowledge of
human factor related error reduction techniques through performance-based testing. If the 65
th
MXG’s technicians are unable to recognize individual capabilities and limitations, it can lead to
increased aircraft damage and possibly an incident or accident.
This project examined the knowledge and skills, motivation, and organizational
influences of the aircraft technicians’ ability to recognize and mitigate error-producing
conditions in aviation. The proposed solution, a comprehensive training program and a
supported safety culture that ensures all policies and procedures are formalized, should produce
the desired outcome – a reduction of human factor related aircraft damage events that helps the
65
th
MXG provide safe and reliable airlift capability worldwide.
Level 4: Results and Leading Indicators
The proposed Level 4 Results and Leading Indicators are shown in Table F1 in the form
of outcomes, metrics, and methods for both external and internal results for the 65
th
MXG.
Leading indicators are short-term observations or measurements of an organization's behaviors,
which when producing desirable results, advances the organization towards their goals
(Kirkpatrick & Kirkpatrick, 2016). The leading indicators are divided between internal (within
an organization) and external (customers response to organizations behaviors). If the internal
outcomes are met as expected, then the external outcomes should also be realized. If both
internal and external outcomes show positive results, the initiative will then be regarded as a
success (Kirkpatrick & Kirkpatrick, 2016).

HUMAN ERROR RISK REDUCTION IN AVIATION

105
Table F1

Outcomes, Metrics, and Methods for External and Internal Outcomes

Outcome Metric(s) Method(s)
External Outcomes
Reduction of aircraft
incidents/accidents
Incident/accident rate per flying
hour
Ratio between number of
incidents/accidents and the
number of flight hours
Improved perception of
safety
Distributed perception survey Perception survey results year
over year
Aircraft Availability Aircraft Availability rate Mission Capable (MC) Hours /
Total Active Inventory (TAI)
Hours
Internal Outcomes
Increased 65
th
AW flight
crew satisfaction
65
th
AW flight crew satisfaction
scores
65
th
AW flight crew survey
Reduction of aircraft
damage
Class C mishap rates ($50K-
$500K)
Human factor related aircraft
damage report
Improved morale Climate survey Survey results

Level 3: Behavior
Critical behaviors. Table F2 lists the critical behaviors, metrics, methods, and timing
for evaluation. Level 3 behaviors are defined as how participants apply what they learned during
training on the job (Kirkpatrick & Kirkpatrick, 2016). Since the stakeholder group of focus are
the 65
th
MXG aircraft technicians who have direct contact with customer aircraft, the first critical
behavior is the technicians use of the error reporting system to highlight safety concerns. The
second critical behavior is the technicians’ identification of error producing conditions on the
job. Finally, the third critical behavior is the technicians’ elimination of identified error
HUMAN ERROR RISK REDUCTION IN AVIATION

106
producing conditions on the job. The specific metrics, methods, and timing for each of these
outcome behaviors appears in Table F2.
Table F2

Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s)
Method(s)
Timing
(1) Technicians use of
error reporting system
to highlight safety
concerns.
Number of accepted
error / hazard
identification reports
submitted to reporting
system.
Anonymous reports
submitted through
reporting system
monitored and managed
by program manager.
Reporting system
continuously
monitored with
monthly leadership
meeting and when
significant reports
are submitted.
(2) Technicians
identification of error
producing conditions.
Number of near-miss
situations technicians
identified.
Collect near-miss data
from technicians in
order to distribute and
share among other
workers.
Continuously
(3) Technicians
elimination of
identified error
producing conditions.
Number of identified
near-miss situations
technicians combated.
Collect near-miss data
from technicians in
order to distribute and
share among other
workers.
Continuously

Required drivers. The critical behaviors found in Table F2 cannot exist without the
necessary support and encouragement from the organization's environment. This support is often
referred to as required drivers, of which there are four types: reinforcing; encouraging;
rewarding; and monitoring (Kirkpatrick & Kirkpatrick, 2016). Reinforcing drivers stress the
importance of the transfer of new skills learned in training to implementation on the job.
Encouraging drivers provide constant stimulation for employees to continue the transfer of the
skills after training. Rewarding drivers recognize the applicable execution of the required skills.
And finally, monitoring of the support drivers is necessary to show success of a program
initiative (Kirkpatrick & Kirkpatrick, 2016). Table F3 outlines the reinforcing, encouraging,
HUMAN ERROR RISK REDUCTION IN AVIATION

107
rewarding, and monitoring drivers necessary for technicians to demonstrate knowledge of human
factor related error reduction techniques, and which critical behaviors they support.
Table F3
Required Drivers to Support Critical Behaviors
Methods Timing Critical Behaviors Supported
Reinforcing
Job Aid that details different
error reducing methods

Ongoing 1, 2, 3
Human Factors Awareness
Refresher Training

Biennial 1, 2, 3
Discuss human factors contributions
at morning meetings, roll call, and
during task assignments

Daily 1, 2, 3
Discuss human factors contributions
during daily debriefs to collect and
thoroughly understand experienced
limitations

Daily 1, 2, 3
Encouraging
Feedback and coaching from
Team Lead, Supervisor, and
Manager.
Ongoing 1, 2, 3
Promote the use of the error
reporting system for the
identification and elimination of
hazards
Ongoing 1, 2, 3
Rewarding
Public acknowledgement,
such as a safety award at
All-Hands meetings
Quarterly 1, 2, 3

Monitoring
Human Factors program
manager briefs success stories
at All-Hands meetings.
Quarterly 1, 2, 3
HUMAN ERROR RISK REDUCTION IN AVIATION

108
Organizational support. As Kirkpatrick and Kirkpatrick (2016) suggest, success in how
participants apply what they learned on the job is the key to how well the organization meets
their goals. This is why supporting critical behaviors through reinforcing, monitoring,
encouraging, and rewarding performance is essential. However, Level 3 is more than just
monitoring, as Kirkpatrick and Kirkpatrick (2016) indicate, it requires continuous organizational
support to ensure the critical behaviors on the job are accomplished. Furthermore, these
behaviors will not occur with training alone, the organizational support and accountability
towards these vital behaviors help cultivate success.
Level 2: Learning
Learning goals. Following completion of the recommended solutions through the
implemented human factors awareness training program, the technicians will be able to:
1. Articulate how human factors contribute to errors in their work environment.
(Declarative Knowledge).
2. Confidently implement decision-making strategies to manage human error.
(Procedural Knowledge)
3. See the usefulness of human factors awareness training as it relates to behavior and
performance on the job. (Utility-Value)
4. Reflect on their performance limitations on the job. (Metacognition)
5. Have confidence in their ability to understand and apply human factors principles.
(Self-Efficacy)
Evaluation of the components of learning. It is essential to evaluate declarative
knowledge, procedural knowledge, attitude, confidence, and commitment of the learner when
measuring the success of a training program (Kirkpatrick & Kirkpatrick, 2016). This evaluation
HUMAN ERROR RISK REDUCTION IN AVIATION

109
focuses on the effectiveness of training and its ability to provide learners with the necessary tools
needed on the job. Table F4 outlines methods used to evaluate the components of learning, to
include projected timing.

HUMAN ERROR RISK REDUCTION IN AVIATION

110
Table F4
Evaluation of the Components of Learning for the Program.
Methods and Activities Timing
Declarative Knowledge “I know it.”

Knowledge checks through discussions, and
other individual/group activities.

Periodically during the workshop and
documented via observation notes.
Knowledge check using multiple choice.

At the conclusion of the training course.
Procedural Skills “I can do it right now.”

Demonstration of individual use of job aid to
successfully implement error reducing
strategies.

During case study workshop.

Participant demonstrates proficiency.

During case study workshop.

Individual application of the error reducing
techniques used in real scenario.

During discussion of actual real-world
examples brought to class by students.
Attitude “I believe this is worthwhile.”

Instructor’s observation of participants’
statements and actions demonstrating that they
see the benefit of human factors awareness on
the job.

Periodically during class discussions and
during the case study workshop.

Discussions of the value of what they are being
asked to do on the job.

Periodically during class discussions.

Retrospective pre- and post-test assessment.

Upon course completion.
Confidence “I think I can do it on the job.”

Discussions following practice and feedback.

During the case study workshop, worked
examples in class, and real-world examples
brought to class by students.

Retrospective pre- and post-test assessment.

Upon course completion.
Commitment “I will do it on the job.”

Discussions following practice and feedback.

During the case study workshop, worked
examples in class, and real-world examples
brought to class by students.

Retrospective pre- and post-test assessment.

Upon course completion.
HUMAN ERROR RISK REDUCTION IN AVIATION

111
Level 1: Reaction

In Level One, Kirkpatrick and Kirkpatrick (2016) recommend measuring three reactions;
engagement, relevance, and customer satisfaction. Kirkpatrick and Kirkpatrick explain that by
concentrating on these three reactions, the trainer gets immediate feedback and can make the
necessary adjustments to meet the needs of the learner which in turn enhances the learner's
experience. Table F5 below outlines the components used to measure the technicians’
engagement, relevance, and satisfaction from the provided training.
Table F5
Components to Measure Reactions to the Program.
Methods Timing
Engagement

Observation by instructor/facilitator

During training

Discussions during practice and feedback

During training

Course evaluation

Four weeks following training
Relevance

Brief pulse-check with participants via
discussion (ongoing)

During training or when new strategies are
introduced

Course evaluation

Four weeks following training
Customer Satisfaction

Brief pulse-check with participants via
discussion (ongoing)

During training or when new strategies are
introduced

Course evaluation

Four weeks following training

HUMAN ERROR RISK REDUCTION IN AVIATION

112
Evaluation Tools
According to Kirkpatrick and Kirkpatrick (2016), evaluation helps to “maximize transfer
of learning to behavior and subsequent organizational results” (p. 5). This change in behavior
fostering an improvement of the organization can only occur through an appropriate evaluation
plan, one that results in a thorough understanding of the technicians’ experience and applicable
program improvements as necessary. The following sections discuss the evaluation tools used
during and immediately following the human factors awareness training program and the delayed
evaluation tools used based on the Kirkpatrick and Kirkpatrick (2016) suggested timeline.
Immediately following the program implementation. To effectively measure Level 1
Reaction data, the instructor will conduct periodic brief pulse-checks during in-class lectures on
day one of training and workshop case studies on day two. This reaction data is determined by
asking the technicians about the relevance of the content to their daily work routine and the
organizational environment.
To adequately capture Level 2 Learning data, the instructor will use discussions during
training to informally check the understanding of concepts, knowledge, skills, and attitude using
open-ended questions and scenarios drawn from the content. Additionally, a multiple-choice
knowledge check will be used at the end of training to ensure learning has occurred. This end of
course knowledge check will also include retrospective pre- and post-assessment items that ask
technicians about their proficiency before training and after. There will also be an end of course
evaluation instrument to capture Level 1 reaction and Level 2 learning data at the completion of
training. See Appendix H for the evaluation instrument used immediately following training.
Delayed for a period after the program implementation. Kirkpatrick and Kirkpatrick
(2016) explained that a follow-up evaluation is used to understand how learners have employed
HUMAN ERROR RISK REDUCTION IN AVIATION

113
what was learned, the support they are receiving, and what results they have realized. By
including all four levels of evaluation, referred to as “blended evaluation” which evaluates more
than one level at the same time, the experience perspective can be increased. This delayed
evaluation will occur four weeks after the completion of training to allow the information
provided to the technicians to develop and to allow ample time for error reduction opportunities
to emerge on the job. The blended evaluation instrument should comprise items to assess
engagement, relevance, satisfaction (Level 1), confidence and value in knowledge obtained
(Level 2), application of the learning (Level 3), and the extent training has impacted the
technician on the job (Level 4). See Appendix I for a sample blended evaluation instrument used
four weeks following training.
Data Analysis and Reporting
The Level 4 goals for the organization are contingent on Level 3 critical behaviors that
focus on the frequency and use of an error reporting system to highlight safety concerns and the
identification and elimination of error-producing conditions reported by the technicians. As
Reason and Hobbs (2017) explained, an active reporting system is a sign of a robust safety
culture through the cultivation of an environment where employees have the confidence to report
safety concerns without fear of retribution. If employees believe confidentiality will not be
maintained or concerns are not acted on, they will fail to report concerns. Each week, the
director of human factors will track the number of error reports inputted into the error reporting
system and update the number of open reports, closed reports, and reports in-work. Additionally,
the director of human factors will also distribute any significant concerns needing immediate
attention to the appropriate departments. Finally, the director of human factors will also ensure
success stories and significant procedural changes are communicated to all necessary employees.
HUMAN ERROR RISK REDUCTION IN AVIATION

114
The dashboard in Figure F1 reports the data on these measures as a monitoring and
accountability tool. Similar dashboards will be created to monitor Levels 1, 2 and 4.

Figure F1. Level 3 monitoring and accountability dashboard.

Summary
This evaluation and implementation plan use the New World Kirkpatrick Model
(Kirkpatrick, & Kirkpatrick, 2016) as its frame. This inverted model looks at the organizational
goals and connects them with training and evaluation components to meet those goals in a
deliberate and logical way. Specifically, this model focused on how the 65
th
MXG could best
apply the New World Kirkpatrick Model (Kirkpatrick, & Kirkpatrick, 2016) to human factors
awareness training to ensure their technicians have the necessary tools to recognize and mitigate
errors. This evaluation centers on the incorporated recommendations within a comprehensive
training program and a supported safety culture that ensures all policies and procedures are
formalized. By using the New World Kirkpatrick framework, the 65
th
MXG can address the
knowledge, motivation, and organizational influences to help the technicians meet their goal and
ensure safe and reliable airlift for their customer.

HUMAN ERROR RISK REDUCTION IN AVIATION

115
Appendix G: Limitations and Delimitations
The limitations of this study included the participant group selected, time constraints, and
a presumed power dynamic between the researcher and technician. Limiting the participants for
this study to aircraft technician’s working at SAFB was done so based on accessibility and
availability of participants, location, and time. By limiting the selected population, the results
may not reflect the actions of others within a similar environment. Therefore, additional research
may support the generalizability of the findings from this study in relation to similar
organizations.
Another limiting factor was participant time. In both the survey and interview, the
technicians may have felt they did have the time to complete the requested survey or may have
struggled with the amount of time an interview took. This hyper-focus on time is related to the
nature of the technicians’ work. Aircraft maintenance at SAFB comes with plenty of challenges,
manning levels are at an all-time low, and the mission’s pace and criticality are at an all-time
high (Woody, 2018). These challenges result in the technician's constant awareness of the clock
as their time is scrutinized by supervision when they are off the aircraft.
Finally, there may have been a power dynamic related to the respondents not fully
understanding the researcher’s role within the USAF. Even though the researcher explained that
they had no affiliation with the 65
th
MXG, Sterling Air Force Base, or the USAF, the technicians
may have still associated the researcher as someone who was either in a leadership role or
possessed influence in some way. The limitation associated with this power dynamic is based on
social desirability bias which indicates that the respondents may provide answers centered on
what they thought the researcher wanted to hear (Saldana & Omasta, 2017).
HUMAN ERROR RISK REDUCTION IN AVIATION

116
The delimitations of the study include aircraft technician’s currently working on Sterling
Air Force Base/65
th
MXG who wanted to contribute to the survey and interview process. Those
technician’s deployed, on leave, or on flight status (flight mechanics), were not surveyed or
interviewed. It is important to note that each group of technicians not included has their own set
of norms and habits not captured in this study. The decision to not include these populations was
chosen due to time constraints of the study and limited access to potential participants. The
study also did not survey or interview any managers, superintendents, squadron commanders, or
65
th
MXG senior leadership, all of which have an impact on human factor related error reduction.
These leadership positions were not explored due to the limited opportunity to gather data and
the relationship between them and the research purpose. Furthermore, since participation in the
study was voluntary, not all aircraft technicians selected chose to participate in surveys and
interviews.

HUMAN ERROR RISK REDUCTION IN AVIATION

117
Appendix H: Sample Evaluation Immediately Following Program
Please circle the rating for each section based on the following criteria:

Level 1
Engagement
1. The classroom environment assisted my learning.
1 2 3 4 5 6 7 8 9 10

2. My involvement was encouraged by the instructor.
1 2 3 4 5 6 7 8 9 10

3. This human factor training maintained my interest.
1 2 3 4 5 6 7 8 9 10
Open-ended comments:
4. Was there anything that interfered with your learning? If so, what?

Relevance
5. What I acquired from human factors training will help me on the job.
1 2 3 4 5 6 7 8 9 10

6. During training, we discussed how to apply what we learned.
1 2 3 4 5 6 7 8 9 10

7. I am clear about what is required of me back on the job.
1 2 3 4 5 6 7 8 9 10

Customer Satisfaction
8. I will recommend human factors training to my co-workers.
1 2 3 4 5 6 7 8 9 10

HUMAN ERROR RISK REDUCTION IN AVIATION

118
Open-ended comments:
9. How could human factors training be improved?

Level 2
Declarative Knowledge
(Knowledge will be measured with formative exercises during training and a quiz near the end.)
Open-ended comments:
10. What are the major concepts that you learned during human factors training?

Procedural Skills
(Skill is measured with activities and demonstrations during training)

Attitude
11. I believe it will be worthwhile for me to apply what I learned.
1 2 3 4 5 6 7 8 9 10
Open-ended comments:
12. Why is it important to apply what you learned on the job?

Confidence
13. I feel confident about applying what I learned back on the job.
1 2 3 4 5 6 7 8 9 10
14. I anticipate that I will receive needed support to successfully apply what I learned.
1 2 3 4 5 6 7 8 9 10

HUMAN ERROR RISK REDUCTION IN AVIATION

119
15. My confidence is not high because:
a) I do not have the necessary knowledge and skills.
1 2 3 4 5 6 7 8 9 10
b) I do not have a clear picture of what is expected of me.
1 2 3 4 5 6 7 8 9 10
c) I have other, higher priorities.
1 2 3 4 5 6 7 8 9 10
d) I do not have the necessary resources to apply what I learned.
1 2 3 4 5 6 7 8 9 10
e) I do not have the support to apply what I learned.
1 2 3 4 5 6 7 8 9 10
f) I don’t think what I learned will work.
1 2 3 4 5 6 7 8 9 10
g) There is not an adequate system of accountability to ensure application of what I learned.
1 2 3 4 5 6 7 8 9 10
h) Other (please explain):

Open-ended comments:
16. What additional support is needed to implement what you learned?

17. What barriers could limit your success at applying what you learned?

Commitment
18. I am committed to applying what I learned to my work.
1 2 3 4 5 6 7 8 9 10

HUMAN ERROR RISK REDUCTION IN AVIATION

120
Appendix I: Sample Delayed Follow-Up Evaluation
INSTRUCTIONS: Circle the rating that best describes your current level of on-the-job
application for each behavior

DELAYED LEVEL 1: REACTION
Relevance
1.I have used information from human factors training in my job.
1 2 3 4 5 6 7 8 9 10

2. The information provided in human factors training is applicable to my job.
1 2 3 4 5 6 7 8 9 10

Open-ended comments:
3. What information from human factors training has been the most relevant to your job?

4. Was there any information in human factors training that is NOT relevant to your job?

5. What information should be added to human factors training to make it more relevant?

HUMAN ERROR RISK REDUCTION IN AVIATION

121
Customer Satisfaction
6. Looking back, attending human factors training was a good use of my time.
1 2 3 4 5 6 7 8 9 10

Open-ended comments:
7. Looking back, how could human factors training be improved?

8. Looking back, what would you change about human factors training?

DELAYED LEVEL 3: BEHAVIOR
9. I have successfully applied on the job what I learned in human factors training.
1 2 3 4 5 6 7 8 9 10

10. I have been able to apply what I learned in class on the job.
1 2 3 4 5 6 7 8 9 10

Level 3: Behavior Rating Scale
11. I have applied what I learned to my work.
1 2 3 4 5 6 7 8 9 10

Open-ended comments:
12. How have you used what you learned in human factors training on the job?

HUMAN ERROR RISK REDUCTION IN AVIATION

122
13. Describe any challenges you are experiencing in applying what you learned

14. My supervisor and I set expectations for human factors training prior to the class.
1 2 3 4 5 6 7 8 9 10

15. My supervisor and I determined how I would apply what I learned after training.
1 2 3 4 5 6 7 8 9 10

16. I have received support in order to apply what I learned successfully.
1 2 3 4 5 6 7 8 9 10

Open-ended comments:
17. What else do you need to successfully perform the skills you learned in human factors
training while on the job?

18. What has helped you to implement what you learned?

LEVEL 4: RESULTS TRAINING AND EVALUATION
19. I am already seeing positive results from human factors training.
1 2 3 4 5 6 7 8 9 10

HUMAN ERROR RISK REDUCTION IN AVIATION

123
Open-ended comments:
20. Please give an example of a positive outcome you have experienced since attending human
factors training.

Desired Results
21. This program has positively impacted my department.
1 2 3 4 5 6 7 8 9 10

22. My efforts have contributed to the mission of this organization.
1 2 3 4 5 6 7 8 9 10

Open-ended comments:
23. What impact is this program having on the organization as a whole?

24. How has your participation in this program benefited the company?

Abstract (if available)

Abstract Aviation safety relies heavily on maintenance technicians to provide safe, reliable aircraft. With 15-20% of all aviation accidents directly attributed to human error in maintenance, many aviation organizations are trying to find a way to reduce this risk. Not only does this error contribute to accidents, but there are also countless other related events such as aircraft damage, personnel injury, and a reduced mission capable status, that can impede an organization’s overall objective. The purpose of this study was to understand influences related to human error and aircraft maintenance technicians through the Clark and Estes’ (2008) gap analysis framework. This study’s assumed influences were the result of an extensive literature review and then explored through surveys, interviews, and document analysis. The study participants were aircraft technicians that work on large multi-purpose cargo aircraft within a USAF Maintenance Group. Data demonstrated that the most significant barrier to reporting human safety concerns was an organizational culture that endorsed a complacent attitude. The data also determined that the organizational culture needs to support overwhelmed managers in their role of enforcing standard safety practices. Additionally, this study found that aircraft technicians need to increase their understanding of how human factors contribute to errors, how to properly incorporate decision-making strategies to manage human error, and how to reflect on performance limitations. The study provides recommendations developed using the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016). The recommendations chosen will help close the identified gaps, ensuring the technicians have the necessary tools to recognize and mitigate human error in aircraft maintenance.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

The board fundraising challenge after nonprofit mergers: an evaluation study

PDF

Fundraising in small health and human service nonprofit organizations: an evaluation study

PDF

Preventing excessive force incidents by improving police training: an evaluation study of a use-of-force training program

PDF

Implementing proactive safety strategies in place of reactive safety strategies at a manufacturing organization: an evaluation study

PDF

An evaluative study on implementing customer relationship management software through the perspective of first level managers

PDF

Advisor impact on student veterans at a post-secondary institution: an evaluation study

PDF

Adaptability characteristics: an evaluation study of a regional mortgage lender

PDF

Violence experienced by registered nurses working in hospitals: an evaluation study

PDF

Relationship between employee disengagement and employee performance among facilities employees in higher education: an evaluation study

PDF

Increasing strategic investments of philanthropic funding in nonprofit organizations

PDF

Pacific Coast University Police Department sworn officer staffing shortages: a gap analysis

PDF

Organizational agility and agile development methods: an evaluation study

PDF

A qualitative examination of the methods church leaders use to increase young adult attendance in Christian churches: an evaluation study

PDF

Customer satisfaction with information technology service quality in higher education: an evaluation study

PDF

The role of professional development and certification in technology worker turnover: An evaluation study

PDF

Retaining female field grade officers in the USAF: an evaluative study

PDF

Increasing workplace training transfer

PDF

The impact of faculty interactions on online student sense of belonging: an evaluation study

PDF

Civic learning program policy compliance by a state department of higher education: an evaluation study

PDF

Development of intraorganizational post-merger collaboration plan: an evaluation study

Asset Metadata

Creator Virtue, Matthew D. (author)

Core Title Human error risk reduction in aviation: an evaluation study

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Publication Date 10/17/2018

Defense Date 10/08/2018

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag aircraft maintenance,aircraft maintenance technicians,AMT,Aviation,aviation safety,human error,Human Factors,OAI-PMH Harvest

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Seli, Helena (committee chair), Hirabayashi, Kimberly (committee member), Miller, Mark (committee member)

Creator Email mvirtue@usc.edu,virtue1@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-81577

Unique identifier UC11672308

Identifier etd-VirtueMatt-6866.pdf (filename),usctheses-c89-81577 (legacy record id)

Legacy Identifier etd-VirtueMatt-6866.pdf

Dmrecord 81577

Document Type Dissertation

Format application/pdf (imt)

Rights Virtue, Matthew D.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA