University of Southern California Dissertations and Theses

Taking the pulse on accountability: an innovation study

(USC Thesis Other)

Taking the pulse on accountability: an innovation study

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

Taking the Pulse on Accountability: An Innovation Study
by
Martin Witt III

Rossier School of Education
University of Southern California
A dissertation submitted to the faculty
in partial fulfillment of the requirements for the degree of
Doctor of Education

May 2022

© Copyright by Martin Witt III 2022
All Rights Reserved

The Committee for Martin Witt III certifies the approval of this Dissertation

Anthony Maddox
Jennifer Phillips
Adrian Donato, Committee Chair

Rossier School of Education
University of Southern California
2022

iv
Abstract
This dissertation explores the concepts of leading operational indicators and accountability at a
corporation, a global contract manufacturer, with the intent to understand what the knowledge,
motivational, and organizational influences impacting overall performance. The study featured a
mixed-methods, quantitative and qualitative approach utilizing internal organizational
documents, surveys, and interviews to collect the necessary data. The results indicated that gaps
were present in the areas of conceptual knowledge, goal orientation motivation, and cultural
models and settings. As a result of the research and corresponding literature review, core
recommendations to improve the situation were determined. From a knowledge perspective, the
sharing of information and job aids to the operations leaders can be leveraged to close the
existing conceptual gap. In terms of motivation, the creation of a community of learners as well
as the establishment of task, reward, evaluation and management structure could reinforce the
necessary motivation to improve performance. Finally, to address the organizational gaps, the
reinforcement of accountability via assessments, the setting of goals that promote accountability,
and the creation of a learning organization that reinforces leading indicator development would
be key mechanisms to improve and address the problem of practice.

v
Dedication
To my family, my wife Michelle Witt, children Olivia, Sean, and Talia for their continued
support and sacrifice, and my parents and retired Chicago educators, Martin and Augustine Witt,
for laying the foundation.

vi
Acknowledgements
I wanted to thank my executive sponsors John Carlson and Paul Humphries for their
guidance and advocacy; friends; my dissertation committee of Professors Donato, Maddox, and
Phillips for their thought leadership, strong coaching, and impactful insights. Thank you to my
fellow cohort “lucky 13” members for the comradery and partnership throughout the curriculum.
These relationships will without question carry on well beyond our U.S.C. graduation.
Also, thank you to my executive coach, Thomas W. (Tom) Mason, PhD, for helping
further the evolution of my own thinking as a leader and executive.
And finally, thank you to my current and past colleagues for all the valuable input that
informed my areas of study with tremendous insights.

vii
Table of Contents
Abstract .......................................................................................................................................... iv
Dedication ........................................................................................................................................v
Acknowledgements ........................................................................................................................ vi
List of Tables ...................................................................................................................................x
List of Figures ................................................................................................................................ xi
Chapter One: Introduction of the Problem of Practice ....................................................................1
Organizational Context and Mission ...................................................................................2
Organizational Performance Status ......................................................................................3
Related Literature.................................................................................................................4
Importance of the Organizational Innovation ......................................................................5
Organizational Performance Goal ........................................................................................6
Description of Stakeholder Groups ......................................................................................6
Stakeholder Groups’ Performance Goals.............................................................................7
Stakeholder Group for the Study .........................................................................................8
Purpose of the Project and Questions ..................................................................................9
Methodological Framework .................................................................................................9
Definitions..........................................................................................................................10
Organization of the Study ..................................................................................................11
Chapter Two: Review of the Literature .........................................................................................12
Introduction to Problem of Lack of Leading Indicators and Accountability .....................12
Role of Stakeholder Group of Focus .................................................................................22
Knowledge, Motivation and Organizational Influences Framework .................................23
Stakeholder Knowledge, Motivation, and Organizational Influences ...............................25
Conceptual Framework: The Interaction of Stakeholders’ Knowledge and Motivation
and the Organizational Context .........................................................................................45

viii
Conclusion .........................................................................................................................47
Chapter Three: Methodology .........................................................................................................49
Methodological Approach and Rationale ..........................................................................49
Participating Stakeholders .................................................................................................50
Data Collection and Instrumentation .................................................................................53
Data Analysis .....................................................................................................................56
Credibility and Trustworthiness .........................................................................................57
Validity and Reliability ......................................................................................................58
Ethics..................................................................................................................................59
Limitations and Delimitations ............................................................................................61
Chapter Four: Results and Findings ...............................................................................................63
Participating Stakeholders .................................................................................................64
Data Validation ..................................................................................................................67
Results and Findings of Knowledge Needs .......................................................................69
Results and Findings of Motivation Needs ........................................................................79
Results and Findings of Organizational Needs ..................................................................94
Summary ..........................................................................................................................105
Organizational Context and Mission ...............................................................................110
Organizational Performance Goal ....................................................................................110
Description of Stakeholder Groups ..................................................................................111
Stakeholder Group for the Study .....................................................................................112
Purpose of the Project and Questions ..............................................................................112
Recommendations for Practice to Address KMO Influences ..........................................114
Integrated Implementation and Evaluation Framework and Plan....................................132
Strengths and Weaknesses of the Approach ....................................................................160

ix
Limitations and Delimitations ..........................................................................................161
Future Research ...............................................................................................................163
Conclusion .......................................................................................................................164
References ....................................................................................................................................167
Appendix A: Document Analysis Rubric ....................................................................................196
Appendix B: Survey Protocol, Scale 1–5 ....................................................................................198
Appendix C: Interview Protocol ..................................................................................................201
Appendix D: Immediate Evaluation Instrument for Levels 1 and 2 Post Program
Implementation ............................................................................................................................206
Appendix E: Evaluation Instrument for Levels 1, 2, 3, 4 Delayed Post Program
Implementation ............................................................................................................................208

x
List of Tables
Table 1: Organizational Mission, Performance Goal, and Stakeholder Performance Goals 8
Table 2: Key Components of Leading Indicator Development 19
Table 3: Critical Components for Operations Leaders Developing Leading Indicators 21
Table 4: Knowledge Influences, Types, and Assessments for Knowledge Gap Analysis 31
Table 5: Assumed Motivation Influence and Motivational Influence Assessments 38
Table 6: Organizational Influences and Organizational Influence Assessments 44
Table 7: Summary of Findings, KMO Assumed Influences - Validated Assets and Validated
Needs 106
Table 8: Summary of Knowledge Influences and Recommendations 115
Table 9: Summary of Motivation Influences and Recommendations 120
Table 10: Summary of Organization Influences and Recommendations 125
Table 11: Outcomes, Metrics, and Methods for External and Internal Outcomes 136
Table 12: Critical Behaviors, Metrics, Methods, and Timing for Evaluation of Operations
Leaders 138
Table 13: Required Drivers of Stakeholders to Support Critical Behaviors 142
Table 14: Evaluation of the Components of Learning for the Program 148
Table 15: Components to Measure Reactions to the Program 151
Table 16: Leading Indicator Dashboard 154
Table B1: Survey Questions 198
Table D1: Evaluation Instrument 206
Table E1: Evaluation Instrument 208

xi
List of Figures
Figure 1: Conceptual Framework Illustrating the Relationship Among Operations Leaders’
Knowledge and Organization and the Organizational Goals 46
Figure 2: Survey Group Demographics: Years in Company and Industry 65
Figure 3: Interview Group Demographics: Years in Company and Industry 66
Figure 4: Study Group Demographics: By Region 67
Figure 5: Declarative Knowledge Survey Results (Question 10) 71
Figure 6: Conceptual Knowledge Survey Results(Question 11) 74
Figure 7: Metacognitive Knowledge Survey Results (Question 12) 77
Figure 8: Operations Leader Task Value Survey Results (Question 1) 81
Figure 9: Operations Leader Task Value Survey Results (Question 13) 82
Figure 10: Operations Leader Task Value Survey Results (Question 14) 83
Figure 11: Operations Leader Task Value Survey Results (Question 15) 84
Figure 12: Operations Leader Task Value Survey Results (Question 16) 85
Figure 13: Operations Leader Task Value Survey Results (Question 17) 86
Figure 14: Operations Leader Goal Orientation Survey Results (Question 2) 88
Figure 15: Operations Leader Goal Orientation Survey Results (Question 3) 89
Figure 16: Operations Leader Goal Orientation Survey Results (Question 4) 90
Figure 17: Operations Leader Goal Orientation Survey Results (Question 5) 91
Figure 18: Operations Leader Goal Orientation Survey Results (Question 6) 92
Figure 19: Organizational Cultural Models: Level of Accountability of Leadership (Question
7) 97
Figure 20: Organizational Cultural Models: Level of Communication in Facilitating
Accountability (Question 8) 98
Figure 21: Organizational Cultural Settings: Autonomy in Developing Leading Indicators
(Question 9) 102

xii
Appendix F: Dashboard for Operations Leaders’ Evaluation for Levels 1–4 Delayed Post
Program Implementation 212

1

Chapter One: Introduction of the Problem of Practice
The performance problem of focus in the manufacturing industry is the underutilization
of leading indicators. At a macro level, the presence of leading indicator measures is a
considerable factor in overall performance in a wide range of industries. In the financial industry,
the lack of leading indicators was a key driver in organizational failures during the 2008 global
financial crisis (Chou, 2015). In manufacturing forecasting, a review of leading indicators of 15
industrial countries determined that key leading indicators reliably predicted 4 to 6 months ahead
regarding manufacturing needs for future months compared to countries without strong leading
indicator performance (Berk & Bikker, 1995). Frankel and Saravelos (2010) determined a similar
correlation between the use of leading indicators and subsequent financial crises across different
countries. Regarding organizational safety performance, research conducted on 26,000 mining
establishments between 2006 and 2017 determined that the lack of effective safety leading
indicators was directly correlated with fatal injuries (Yorio et al., 2020). Also, in the oil and gas
industries, similar research affirmed a significant difference in the performance of organizations
as a result of the lack of leading indicators (Naji et al., 2020).
The organization and area of focus in this study is the operations manufacturing function
of the medical segment at Plasticman Corporation, or PMC, which is a pseudonym for the
organization of focus in this study. The organization’s mission is to build strong connections in
the world through its contract manufacturing solutions. A core competency of the organization is
its ability to design and build intelligent and efficient products. This particular segment continues
to grow at an accelerating rate. Ultimately, the organization strives to create value for its
customers and improve the condition of their people’s lives. Key organizational values expected
of all employees are to challenge the status quo, move quickly, execute in a disciplined manner
2

with an overarching purpose, and perform with the highest level of integrity at all times. The
performance problem of the organization is the underutilization of leading indicators, which
results in the lack of achievement of “lagging” accountability measures connected to operational
results. Manuele (2009) defined sagging indicators as the outcome measures resulting from past
efforts while leading indicators are outcome measures that predict future performance. The
performance problem of the organization is that it is meeting 0% of its leading performance
measures that tie to its existing lagging operational performance indicators. The organization has
a goal to meet 100% of its leading indicators by 2022. The gap in performance is 100%.
Organizational Context and Mission
Headquartered in Asia, PMC comprises over 100 manufacturing facilities and over
160,000 employees across the world. For more than 50 years, the organization has designed,
built, and delivered products in partnership with some of the world’s most innovative companies.
The purpose of PMC as a company is to make great products that create market-leading value
and improve customers’ and patients’ lives. The mission of PMC is to create an intelligent and
more connected world by being a manufacturing solutions provider that designs and
manufactures products for an increasingly connected and growing world. In executing its
mission, the organization emphasizes the importance of living the organizational values expected
of all employees. The values define how the employees act, behave and conduct daily
interactions with fellow employees, customers, and partners daily. The values help define the
organization’s culture. The organization’s vision is to become the world’s most trusted global
technology, supply chain, and manufacturing solutions partner.

3

Organizational Performance Status
The organization’s performance problem is the underutilization of leading indicators
resulting in not achieving lagging accountability measures connected to operational results. The
specific performance problem of the organization is that it is meeting none of its leading
indicators, resulting in a gap in performance of 100%. Currently, PMC has a considerable
number of lagging accountability measures of operational performance, including product startup
performance, controllable operating profit, and customer satisfaction metrics. The purpose of
establishing leading indicators that influence and predict performance is to address the current
lagging accountability measures. The organization aims to meet 100% of its leading indicators
by 2022. Overall, the corporation expects to exceed customer expectations and deliver on
internal measures yet does not have leading indicators to ensure proactive delivery of these
results. The organizationally declared global approach to delivering the desired results at a macro
level includes aligning with the medical segment’s objectives, executing the standard operational
business practices, and adopting best practices from other factories in the global network while
leading an inclusive organization.
Results for the medical manufacturing segment have been lower than expected, and the
underutilization of leading indicators is a problem. The accountability plan does not include
leading indicators. Their absence has adversely impacted product ramp implementations
resulting in only 30% of products passing initial qualification versus a goal of 100%, controllable
operating profit of 5.5% versus a goal of 7.5%, and a subsequently overall downward trending
customer satisfaction metric score of 78% versus a target of 100%. The intent of being
accountable to leading indicators to improve performance is to address operational gaps,
including 70% for startup product qualifications, 40% for controllable operating profit, and 22%
4

for the overall customer satisfaction metric. The problem of the lack of utilization of leading
indicators is evidenced by the organization not choosing to develop and include them in the
organization’s accountability plan. This problem of underutilization impacts the organization’s
goals because its success largely rests on its ability to hold itself accountable to increasing
performance and market demands. To become the most trusted global technology, supply chain,
and manufacturing solutions partner, the organization must raise its execution level. This
problem of underutilizing leading indicators represents the larger business challenge of
increasing price pressure by customers, causing eroding profit margins for original equipment
medical manufacturers.
Related Literature
Establishing effective leading indicators with clear accountability mechanisms is
important in a wide range of business contexts. Globally, the impact of the lack of leading
indicators and accountability mechanisms was observed after the 2008 financial crisis, wherein
businesses focused on cost reduction. As a result, many did not reach their strategic goals (Chou,
2015). Similarly, with the assessment of federal agencies, research illustrated by Barrados and
Blain (2013) shows that organizations understand more how to measure performance with
“lagging” indicators that highlight things that have already happened versus focusing instead on
proactive leading indicators. However, Barrados’s (2013) research also affirmed that these same
organizations fail to effectively track leading indicators that provide an effective barometer for
activities that are currently happening that can improve their performance moving forward.
The problem of underutilizing leading indicators is also an opportunity for improvement
in particular business processes within organizations. From an organizational safety perspective,
Pawłowska (2015) conducted a study of 60 companies, which revealed that organizations largely
5

focused more on compliance. Those teams that focused on more leading indicators to manage
their day-to-day processes saw significantly improved results and a greater than 50% reduction
in incident rates (Pawłowska, 2015). A similar phenomenon is seen in the area of project
management as well, where performance measures centered largely on items related to costs and
schedules and were lacking on additional leading indicators such as those performance indicators
relative to customer product requirements such as on-time delivery and percentage of products
produced without defects (Zheng, 2019).
Importance of the Organizational Innovation
It is important for PMC to innovate its use of leading indicators for various reasons. The
problem of using leading indicators is evidenced by the organization not developing and
including them in its accountability plan. The absence of leading indicators in this plan manifests
itself as challenges in the organization’s accountability binary, including information asymmetry,
divergent objectives, limited decision rights, weak incentives, and legitimacy. This overall
absence of accountability with leading indicators can result in lower performance in
organizations (Han, 2020).
The current financial results for the global medical segment for the organization have
been lower than expected. The organization experienced a 75% earnings decline, a 10% sales
revenue decline, and an overall deterioration of relationships with critical customers over the last
2 years. The organization’s reputation has taken a considerable hit in recent years, resulting in
the appointment of an external candidate to the role of CEO over the past year. The new
organizational leader stressed the importance of improving operational execution as a focal point
to repair the damaged relationships with key customers while also placing importance on a
6

longer-term strategy on growing the organization in the areas that allow for a rapid expansion of
operating margin and bottom-line growth.
Organizational Performance Goal
The organization aims to meet 100% of its established lagging operational performance
indicators by 2022. The focus is for PMC to achieve the newly developed leading indicators by
2022. The PMC senior executives determined this goal based on environmental scanning at the
organization and via quantitative analysis. As an example, while the operations function has five
core lagging operational metrics, there are 143 total key operations indicators (KOIs) within each
operating site. The core lagging operational metrics represent 3.5% of the total lagging measures
tracked. The performance gap is 100% of leading indicators as they currently do not exist.
This goal is based on the 80/20 principle as explained by Koch (2000), which reinforces
the finding that 80% of performance results from 20% of the time one spends on the critical
focus areas. In addition, the concept of leading indicators explained by McChesney et al. (2012)
and demonstrated in real-world examples such as with the Oakland Athletics as described by
Hammonds (2003), establishes the standard that achieving lagging measures requires leading
measures to provide the necessary leverage. The organization has an overall expectation of
exceeding customer expectations in program startups, customer satisfaction, on-time delivery,
quality performance, and cost results, yet it does not have leading indicators to ensure proactive
delivery of these results to take advantage of leverage.
Description of Stakeholder Groups
The participants were recruited through email and represented three of the organization’s
layers: senior executives (five senior VPs and presidents), regional leaders (five VPs), and 10
operational leaders (site general managers and program directors), with a focus on the
7

operational leader group. The operational leaders manage sites’ overall production facilities and
lead their executional programs. This group of employees is at the front line, focused, and
responsible for delivering the bottom-line results at the sites. The regional leaders are more
horizontally focused and place their overall emphasis on consistency of approach across all of
the production sites of their designation via developing policy and removing obstacles at the sites
that hinder the rate of performance improvement relative to the internal targets. Finally, the
senior executives are accountable for the legislation, implementation of measures, and follow-
through to ensure clear accountability and delivery of the lagging operational performance
measures.
Stakeholder Groups’ Performance Goals
Table 1 describes the organizational mission and performance goal as well as the
cascading stakeholder performance goals in alignment with the organizational mission.

8

Table 1
Organizational Mission, Performance Goal, and Stakeholder Performance Goals
Organizational mission
The mission of the organization is to build great connections in the world through the contract
manufacturing solutions it provides. A core competency of the organization is its ability to
design and ultimately build products that are intelligent and efficient in an industry that
continues to grow at an accelerating rate. Ultimately, the organization strives to create value
for its customers and improve the condition of their people’s lives.
Organizational performance goal
The organization has a goal to meet 100% of its newly developed leading indicators by 2022.
Stakeholder 1 goal Stakeholder 2 goal Stakeholder 3 goal
Operational leaders

Operational leaders develop
leading operations
indicators in alignment
with organizational goals
by December 2021
Regional leaders

Regional VPs will place
leading operations
indicators into policy by
April 2022
Senior executives

Presidents will legislate and
measure leading operations
indicators by July 2022

Stakeholder Group for the Study
It is critically important to understand the needs of the stakeholder group of focus in this
study. The operational leaders must develop the key leading indicators that align with the lagging
indicators. All stakeholders’ joint efforts contribute to achieving the organizational goal of
developing 100% of all leading indicators by 2022. However, the focus of this study was the
operational leaders responsible for the bottom-line execution at the manufacturing sites and
closest to the leading activities that provide the leverage to develop the leading indicators. The
stakeholder’s goal, supported by the regional and senior leaders, is to develop leading indicators
aligned with organizational goals by December 2021. A review of organizational data revealed
there currently is a very high number of lagging measures at the operational level, with 143
9

counted and no leading indicators currently in place. The organizational goal is to meet 100% of
the newly developed leading indicators by 2022. The performance gap is 100%.
Purpose of the Project and Questions
1. What are the operational leaders’ knowledge and motivation needs related to their
development of leading operations indicators in alignment with organizational goals?
2. What is the interaction between organizational culture and context and operational
leaders’ knowledge and motivation to develop leading operations indicators in
alignment with organizational goals?
3. To what level do operational leaders currently utilize leading indicators?
4. What are recommended knowledge, motivation, and organizational solutions?
Methodological Framework
For the problem of practice in this study, Clark and Estes’s (2008) gap analysis model
was the framework used. The Clark and Estes model is an analytical and systems-based approach
that clearly defines an organization’s goals and identifies the gap between the current and desired
performance levels. The knowledge, motivation, and organization needs for closing the gap are
identified and developed as a result of personal experience and knowledge as well as literary
works related to the concepts of focus in the study. This study validated the needs for closing the
identified performance gap through targeted activities: interviews, focus groups, surveys, and a
review of literature along with its respective analysis. The philosophical worldview utilized in
this study is that of a critical and transformative model where there are existing power and social
structures within which one names whose knowledge is listened to and those whose knowledge
is ignored and silenced. The overall research approach focused on uncovering subjugated
knowledge via investigator/participant dialogue and centered on those facing injustice to create a
10

view of the problem, the people under study, and the changes required to meet the needs for gap
closure (Creswell & Creswell, 2018). Overall, for this problem of practice, the theoretical frame
consists of the characteristics of critical, transformative, presence of a soft reform space, and a
frame capable of institutional change.
In understanding the gap in developing effective leading indicators, the research approach
for the study would feature more of a participatory and supportive methodology (Aliyu et al.,
2015) to understand the perspective of the operations leaders relative to the existing power
structures at PMC and expose the issues preventing change. The change desired is improvement
in the organization, such as developing and implementing leading operational indicators by
diagnosing the knowledge, motivational, and organizational influences present. The approach to
be performed in this study features mixed methods with a combination of closed-end type
research and open-ended inquiry. This methodology features comprehensive research of the
planned study group of the operations leaders who are targeted due to the need to gain their
understanding of the current power structures at PMC and how leading indicators can effectively
drive positive change via the implementation of the necessary critical behaviors and the delivery
of the organizational desired outcomes.
Definitions
Accountability: Based on the contractual relationship between a director with the ability
to reward, punish, or replace, and the provider who provides a service and is held responsible by
the director. The contractual relationship is defined as the accountability binary and is influenced
by the values, decision rights, and information shared between the two parties, the director and
the provider (Hentschke & Wohlstetter, 2004).
11

Lagging Indicators: Measures that represent past performance. These measures typically
represent the key goals deemed most important by the organization (Duin et al., 2007).
Leading Indicators: measures of the most impactful actions the team must take to achieve
the key lagging organizational goals. They measure the new behaviors that will provide the
leverage to make it more likely to achieve the lagging measures (Frankel & Saravelos, 2012).
Organization of the Study
Five chapters are used to organize this study. This chapter provided the key concepts and
terminology commonly found in a discussion about lagging and leading indicators and
accountability. The chapter introduced the organization’s mission, goals, and stakeholders as
well as the initial concepts of gap analysis adapted to needs analysis. Chapter Two provides a
review of the current literature surrounding the scope of the study. That chapter addresses issues
of lagging and leading indicators and accountability. Chapter Three details the assumed needs for
this study and methodology in terms of choice of participants and data collection and analysis.
Chapter Four presents the data and assessment and analysis of the results. Chapter Five provides
solutions, based on data and literature, for addressing the needs and closing the performance gap
as well as recommendations for an implementation and evaluation plan for the solutions.

12

Chapter Two: Review of the Literature
This literature review addresses the literature surrounding the underutilization of leading
indicators that resulted in achievement gaps regarding lagging accountability measures
connected to operational results. The lack of leading indicators impacted businesses globally in
2008 after the financial crisis, and organizations subsequently were unable to achieve their
strategic goals (Chou, 2015). A qual/quant/mixed-methods study of 60 companies revealed that
while some organizations largely focused more on safety compliance, teams that focused on
more leading indicators to manage their day-to-day processes saw significantly improved results
and saw a greater than 50% reduction in incident rates (Pawlowska, 2015).
This chapter presents a review of the key core concepts of focus for the problem of
practice: leading indicators, lagging indicators, and accountability. Next, the chapter describes
the role of operations leaders, followed by explaining the knowledge, motivation, and
organizational lens used in this study. Third, this chapter will center on the operations leaders’
knowledge, motivation, and organizational influences that impact their ability to address the
problem of practice. Finally, the chapter will conclude with an overall presentation of the
conceptual framework that will be the basis for the study.
Introduction to Problem of Lack of Leading Indicators and Accountability
In a review of literature, several findings affirm the problem of the lack of leading
indicators from a global and sector perspective. Globally, the absence of leading indicators has a
negative impact on future performance (Chou, 2015). The lack of accountability mechanisms in
the corporate sector is a significant issue that hinders organizations’ ability to deliver business
outcomes (Molinaro, 2015), with only 37% of employees in organizations satisfied with the level
of leadership accountability present. Molinaro (2015) determined that the majority of
13

organizations do not have leaders who are addressing problems proactively and not allowing
performance to derail without intervention. Additionally, Faisal (2009) found in the
manufacturing supply chain industry that challenges in prioritization are also a limiting factor in
an organization’s ability to deliver on its commitments. These are key findings that reinforce the
presence of the problem of practice explored in this section.
Macro Industry Perspective of Leading Indicators
At the macro level, the absence of leading indicators affects future performance
negatively. Graff and Etter (2004) performed research focusing on Swiss manufacturing firms
that had utilized key manufacturing indicators such as production levels and inventory to drive
operational performance. A key conclusion realized from their research was that firms accurately
projected future quarterly performance estimates by focusing on key business tendency trends as
leading indicators. In addition, an alternate finding was that firms that leveraged these types of
indicators experienced continued gaps in firm performance. Yorio et al. (2020) performed a
similar quantitative analysis on over 24,000 establishments in the mining industry over 12 years,
focusing on key safety performance measures as both lagging and leading indicators. They
determined for each establishment that they were better able to predict and prevent future
negative safety incidents by carefully focusing on negative safety incidents, understanding the
key factors that contributed to the events occurring, and controlling the factors. Conversely,
establishments that did not control the key indicators saw continued re-occurrence of negative
safety events.
Quantitative research in the Indian construction industry, conducted by Habibi et al.
(2018), saw a similar correlation. Their research focused on determining the principal cause of
delays and cost overruns across numerous projects. Ultimately, they determined that the key
14

factors of design changes, resource availability, and price were key leading indicators of future
or lagging overall project performance. Projects that did not control these leading indicators or
variables resulted in lower-than-expected performance. Finally, Bert and Bikker (1995)
conducted a leading-indicators-focused quantitative analysis of 15 developed countries’
manufacturing business cycles and correlated their economies and the timing of production as a
result. The lack of performance of the leading indicators in the less developed countries resulted
in a delayed manufacturing business cycle of at least 4 to 6 months longer than desired, resulting
in lower-than-expected performance.
Accountability Mechanisms in Relation to Leading Indicators
The lack of accountability mechanisms from a macro perspective also has a negative
impact on future performance. Hopper and Westrup (2008) conducted a quantitative study of
agencies and state-level organizations to determine the impact of accountability mechanisms on
performance. A key finding was that instituting accountability programs at both the local and
national levels focused on governance and encouraging competition improved performance. A
quantitative study by Ullah (2016) that focused on the macro state of corporate governance and
its impact on the performance of individual firms affirmed a similar finding. With a focus on
accountability and transparency, a correlation and regression analysis determined that effective
mechanisms to reinforce accountability had a positive and significant impact on the firm’s
performance. Organizations that did not have these accountability mechanisms present
subsequently performed unfavorably in comparison to their peers. Finally, quantitative and
qualitative research by Vassili (2012) across organizations at a macro level determined that
accountability administered appropriately to deliver impact in its environment can be effective in
developing leading indicators. Accountability, detailed by Vassili (2012), requires a
15

comprehensive approach that effectively combats resistance to ideal mechanisms for
accountability. This approach, which includes establishing tangible day-to-day mechanisms in its
environment, such as leading indicators, will subsequently impact organizations’ performance
positively.
Lack of Prioritization in Relation to Leading Indicators
Not prioritizing leading indicator development from a macro perspective has a negative
impact on organizations’ future performance. Quantitative and qualitative research conducted by
Muthiah and Huang (2006) focused on the manufacturing sector and the impact of globalization
related to prioritization in organizations. One key finding was that manufacturing continues to
have difficulty establishing the key leading indicators or variables regarding what is required to
perform at the highest level (Gershwin, 2000). Ultimately, a core finding was that the inability to
measure the critical variables and focus on them has a significant negative impact on
organizations. Another key discovery was that overall, manufacturing organizations lack the
ability to measure and prioritize, via leading indicators, the critical quantitative measures to
effectively drive key lagging measures such as productivity.
An alternate analysis conducted by Alagumurthi et al. (2008) focused on manufacturing
organizations and determined that an overall lack of prioritization on improving critical leading
indicator quality parameters and associated conditions negatively impacted organizations’ ability
to stay competitive in the current marketplace. Alagumurthi et al. detailed the impact in the
manufacturing grinding industry, highlighting the differences in performance and subsequent
positive improvement seen when key leading process parameters are prioritized and given a
higher level of emphasis. In the manufacturing distribution sector, Yao and Carlson (1999)
detailed root causes of the poor performance of organizations today in that they cannot transact
16

data in real time related to movement and processing, which as a result hinders their ability to
develop the necessary leading indicators and prioritize work effectively. They affirmed the
improved performance of organizations with more integrated, focused systems and the much-
improved performance level with both domestic and global customers.
Finally, Kiatcharoenpol et al. (2015) studied the Thai manufacturing sector and focused
on the prioritization of the key success indicators that were predictive of future success. A key
category of focus was the level of lean manufacturing execution based on research and
corresponding input from subject matter experts. Sundar et al. (2014) defined lean manufacturing
execution as maximizing resourcing allocation in manufacturing operations through waste
reduction (Sundar et al., 2014). The research of Kiatcharoenpol et al. determined that effective
prioritization of the vital factors resulted in organizations in this sector delivering on their
expected business outcomes while those that did not focus on these areas missed delivering on
their expected business commitments. Overall, prioritization is key, and the lack of prioritization
of leading indicator development can have a considerable impact on organizations.
Overall, according to this review of literature, the lack of focus, prioritization, and
associated accountability in developing leading indicators at a macro level is a problem. The
absence of leading indicators has a negative impact on future performance (Chou, 2015).
Organizations’ inability to review trends and identify the critical and most impactful inputs
greatly hinders leading indicator development (Muthiah & Huang, 2006). Challenges in
administering accountability include not having the level of governance necessary to reinforce
follow through as well as not having the adequate level of countermeasures and mechanisms to
address the opposition that arises as organizations implement accountability and develop leading
indicators (Vassili, 2012). Finally, and more broadly, an overall lack of focus on developing
17

leading indicators at a macro level across manufacturing organizations is an issue that is
prevalent in multiple sectors. Many organizations’ lack of prioritization on the development of
key leading indicators that drive performance and limitations in their current systems and
corresponding measurements also hinder their ability to administer leading indicators once
developed.
Critical Components for Developing Leading Operations Indicators
The literature review of leading indicators identified key components critical to their
development: (a) that the leading indicators are process or behavior focused, (b) are predictive,
(c) are determined to be the most impactful in delivering the desired business outcomes, (d) have
visible accountability characteristics such as tracking in place, (e) are developed as a result of
collaborative input, and (f) the effort to track is understood to balance with the expected impact.
The following section further explores the references that affirmed the key components that were
most critical in developing leading indicators.
Bennett and Foster’s (2005) research on safety performance revealed the importance of
ensuring that leading indicators are process- or behavior-focused. Wolfe et al. (2006) reinforced
this concept in detailing the significance of targeting behaviors in developing leading measures.
Bernstein et al. (2017) referenced a similar approach with the pre-flight checklist that pilots
developed in World War I to eliminate the gaps in human behavior and subsequent errors that
continued to result in flight crashes and subsequent mass casualties. Continuing the importance
of the proactive measure approach in improving process and associated behaviors, Collins (1999)
emphasized the presence of mechanisms that are predictive and have “teeth” via a review of the
approach of a successful rock manufacturer that implemented a short pay measurement to
prevent quality defects proactively (p. 25). Research affirmed similar findings regarding the
18

effectiveness of predictive measures in the financial industry. Dempsey et al. (1997), in a review
of indicators related to firms in financial distress, reinforced the key principle of establishing
leading-indicator-focused measures that highlight risks early to apply countermeasures earlier in
the process to improve overall organizational outcomes.
Another key component of developing leading indicators is identifying the most
impactful measures, which Foley (2019) emphasized via research. Foley determined that when
developing leading indicators, there is a key need to identify the most impactful indicators that
can be affected at the present moment to allow the delivery of future targets. Another key
component is collaboration within the organization. Weber and Thomas (2005) highlighted the
research of Peter Drucker (2002), who determined that strong relationships between managers
and their people are key when developing leading indicators. Drucker emphasized the intent of
building strong relationships is to ensure that the appropriate level of knowledge and
collaboration is involved in identifying the best leading measures. As collaborative relationships
are developed and strengthened in an organization, Zwetsloot et al.’s (2020) research reinforced
the criticality of establishing dashboards when developing leading indicators to ensure visibility
throughout the organization. The intent of this approach is to ensure there is motivation-driven
score-tracking to reinforce accountability and ensure prompt action to address opportunities.
Finally, Grabowski et al. (2007) connected the core concepts of leading indicator development
together via research in identifying five key components detailed in Table 2.

19

Table 2
Key Components of Leading Indicator Development
Component number Description
Component 1 Defining predictive measures
Component 2 Aligning commitment for tracking and
communication
Component 3 Refining operations/systems/behavior
Component 4 Design indicators that are deployed, activate
result in outcome improvement
Component 5 Categorize the indicators developed into types
of effort involved (i.e., high/medium/low)

Note. Adapted from “Leading Indicators of Safety in Virtual Organizations” by M. Grabowski,
P. Ayyalasomayajula, J. Merrick, J. R. Harrald, J. R., & K. Roberts, 2007, Safety Science,
45(10), 1013–1043.

Best Practices Surrounding the Development of Leading Indicators
The goal of deploying the key components detailed in this section is to improve the
overall outcome after deploying the leading indicators. The literature on leading indicators
identified a number of best practices for developing these measures. First, Lawlor (2012)
researched student achievement in education and highlighted the best practices of (a) minimizing
the number of indicators, as well as (b) ensuring the goals related to the leading indicators are
specific, measurable, achievable, relevant, and time-bound (SMART). In a related study, Nollet
et al. (2008) also found that minimizing indicators is critical. Guo (2016) identified the core best
practice of ensuring the leading measures align with the responsibilities of the group that must
perform to ensure clarity and accountability. Sinelnikov (2015) identified an additional best
20

practice regarding establishing leading indicators. In researching organizations’ health and safety
performance in utilizing leading indicators, Sinelnikov (2015) determined that the best practice
of having visible, actionable data would be required to ensure the effectiveness of established
predictive measures.
Rajendran (2013), in researching safety performance in the construction industry, also
determined the core best practice that leading indicators must focus on behaviors and culture.
Rejandran’s (2013) research identified a strong correlation with organizations that emphasized
behavior-focused leading indicators, resulting in improved overall safety outcomes. Selby’s
(2005) research of large-scale projects revealed the criticality and best practice of utilizing
measurement-driven dashboards when implementing leading indicators. This approach affirmed
that projects that followed this approach resulted in lower occurrences of faults and reduced the
effort required of participants.
Kaplan and Norton (1998) also identified a core best practice when implementing leading
indicators of establishing transparency throughout the organization as a management practice.
Their research of the drivers of the leading indicators determined that a core principle is the need
to place primary importance on the measurement strategy as well as implementing the leading
indicators versus focusing on the measures alone. Ultimately, the holistic concept Kaplan and
Norton (1998) reinforced is the requirement of an established, balanced scorecard clearly aligned
with the leaders in that organization. Overall, there are a number of key concepts identified in
research that can effectively support organizations developing leading indicators. Table 3 aligns
the knowledge, motivation, and organizational influences with the critical components of
developing leading indicators.

21

Table 3
Critical Components for Operations Leaders Developing Leading Indicators
KMO framework influences Critical components for developing
leading indicators
Knowledge
Need to understand contents of job description
related to the underlying leading indicator
principles of developing them
Need to understand how to create effective
accountability mechanisms
Need to understand contents of job description
related to understanding the core skills and
procedures involved with establishing tangible
mechanisms to counteract resistance to
accountability when developing leading
indicators
Need to understand the contents of job description
related to having the ability to reflect on the
optimal process and create necessary skills and
plans to develop, measure, and prioritize effective
leading indicators
Need to monitor and adjust their own behavior to
prioritize the effective development of leading
indicators
Must be process or behavior focused
Predictive and Influenceable
Measures that are forward looking and
highlight risks earlier in the process
Focused on most impactful and critical
measures (80/20 rule)
Requires the building of relationships
with the respective colleagues and
how to involve the team in the
development and establishment of
leading indicators
Leading indicators needs to be
categorized in terms of difficulty to
obtain high/medium/low
Motivation
Need to understand the task value of implementing
leading indicators and have the confidence to
communicate effectively to motivate their teams

Need to possess goal orientation in terms of
mastery versus performance in the development
of effective leading indicators, including lean
concepts such as poka-yoke

Requires execution of processes that
facilitate the elimination of error (i.e.,
poka-yoke)

Requires willingness to build
relationships amongst the team and
involvement and engagement in the
leading indicator development process
Organization
22

KMO framework influences Critical components for developing
leading indicators
The organization needs to have a strong
accountability binary influence on the operations
leaders in developing leading indicators (cultural
models)
The organization has a clearly negotiated
accountability binary for operations leaders’
decision-making, including the expectation and
prioritization of the implementation of leading
indicators
Need to have the necessary structure and
organization to support the development and
implementation of leading indicators
Establishment of tools that eliminate
gaps and engineer out hurdles to
delivering the desired outcomes
Implementation of processes that
facilitate mechanisms with “teeth”
Support from the organization in the
establishment of systems that allow
the ability to measure forward looking
leading indicators
Establishment of dashboards or similar
at a regular cadence of leading
indicators

Note. The critical components to develop leading indicators are not intended to tie directly to
each knowledge, motivation, or organizational influence. “PokePoka-Yoke” is defined as error
proofing such that the design eliminates the possibility for error. Adapted from Developing
Predictive Quality Scorecards: A Futuristic Approach to Quality Management by J. M. Cheema,
M. J. Cheema, and M. A. Bajwa, n.d. European Organization for Quality.
(https://www.eoq.hu/iaq/wqf/papers/b5-4-cheema.pdf)

Role of Stakeholder Group of Focus
Studies have shown that several factors across the Clark and Estes (2008) conceptual
framework of knowledge, motivation, and organization influence operations leaders. The
operational leaders are the stakeholder group of focus because they are responsible for the
manufacturing sites’ bottom-line execution, ensure appropriate accountability mechanisms, and
are closest to the leading activities that provide the leverage to achieve the lagging targets. Jha
and Bhattacharyya (2020) detailed the important role of motivation and connectedness the
23

manufacturing operations leader plays in delivering performance through the employees they
manage. Jha and Bhattacharyya (2020) studied over 300 operations leaders with more than 15
years of experience and determined that the leader’s level of communication and inclusiveness
had a direct impact on employee and organizational performance. Worley and Doolen (2006)
recognized the importance of leadership and corresponding communication in improving the
success of program implementation and execution. Finally, Mehra et al. (2006) established the
connection of objective performance in an organization with the quality of the relationship the
operations leader established both internally with their team and externally through their
reporting hierarchy. Overall, the effectiveness of the operations leader will have a direct
correlation with the performance of the operation. As a result of the literature findings regarding
the problem of practice, the operations leaders were the key stakeholder of focus in this study.
Knowledge, Motivation and Organizational Influences Framework
Clark and Estes’s (2008) framework is based on clear guidelines developed to identify
the causes of performance gaps and develop solutions for organizational problems related to
knowledge, performance, and motivation. The intent of the framework is to provide an overview
of organizations’ improvement processes and the key factors that contribute to changes in
performance. As a result, this framework is suited to stakeholder performance within an
organization and is a problem-solving process based on (a) understanding stakeholder goals with
regard to the organizational goal and (b) identifying assumed performance influences in the areas
of knowledge, motivation, and organization based on general theory, context-specific literature,
and an existing understanding of the organization.
Clark and Estes’s framework (2008), focused on the areas of knowledge, motivation, and
organization, is a problem-solving process suited to analyze stakeholder performance in an
24

organization. Companies with the best talent management practices outperformed the industry’s
mean of return on investment for shareholders by 22% (Clark & Estes, 2008). A national survey
of 3,000 employers revealed that increases in training and knowledge capability correlated
directly with productivity increases (Zemsky, & Iannozzi, 1995). Employees reported they were
less motivated and work 10% to 20 % less in teams than when alone when individual
performance is not monitored. A survey of organizational change projects by Clark and Estes
determined that a set of conditions met would likely determine success for the change effort.
Overall, Clark and Estes reinforced the theory that with the necessary focus on the components
of knowledge, motivation, and organization, research has indicated that an organization can
realize a positive impact.
Clark and Estes’s (2008) framework is adapted as a needs analysis for innovation. The
status within the study’s setting for the innovation model is determined based on various types of
inquiry. Influences of the knowledge, motivation, and organization (KMO) framework are
assessments for knowledge, psychological constructs for motivation, and various types of
measures for organizations. Recommendations moving forward to address the problem of
practice via the KMO framework were determined via the literature. Verification of the
sustainability of the recommendations per the KMO framework is best performed via the four
levels of evaluation consisting of reactions, impact, transfer, and bottom line (Kirkpatrick &
Kirkpatrick, 2006). In summary, the KMO model provides an approach to perform the necessary
level of inquiry, establish recommendations via research, and evaluate their effectiveness. The
next category of focus is the operations leaders’ KMO influences.

25

Stakeholder Knowledge, Motivation, and Organizational Influences
Introduction of Knowledge Influences
Using Clark and Estes’s framework (2008), the initial category of knowledge will come
under additional focus for the purpose of understanding the influences on the operations leaders
and the subsequent lack of achievement of leading indicators. Assumed knowledge influences
supported by literature may contribute to the problem of practice and are defined as the gaps
relative to the facts the operations leaders know, their understanding of how to achieve the goal
and the kind of reflection they engage in. Krathwohl (2002) identified categorical influence types
that impact the problem of practice explored in this study. The influence types of focus related to
knowledge on this study are factual, conceptual, procedural, and metacognitive. Vukić et al.
(2020) defined factual as having a specific understanding of the role, conceptual having the grasp
of the underlying principles, procedural knowing the overall steps to accomplish, and
metacognitive encompasses having the ability to reflect and adjust. Understanding each influence
type is key to understanding the root cause behind the gaps in performance and determining the
appropriate innovative solution as a result of this study.
Declarative Knowledge: Understanding of the Job Description Related to Underlying
Principles to Develop Leading Indicators
Krathwohl (2002) defined declarative factual knowledge as the knowledge of basic facts,
information, and terminology related to the topic. For this study, operations leaders must
understand the contents of their job description related to the underlying principles to develop
effective leading indicators. The literature established a connection relative to this theory. A
study by Bernthal (2020) of marine pilots determined that those pilots who had a strong
connection between job description and understanding of the work experienced a higher level of
26

performance and subsequent overall retention. Gaseau (2007) researched the exercise of leaders
developing leading measures to predict complex systems’ performance. Leaders’ understanding
of leading indicators and ability to effectively implement them had a considerable impact on
engineering practice as a result of the subject matter experts working together to establish them.
Ultimately, Gaseau (2007) affirmed leading indicators were mechanisms that can reduce
preventable problems and corresponding impacts. Kaplan and Norton (1998) affirmed that
leaders’ understanding of their role to effectively develop leading indicators as a result of the
team’s consensus is an essential element of establishing leading indicators. Operations leaders
must have the knowledge to understand this fundamental principle that is central to the
knowledge of implementing leading indicators.
Procedural Knowledge: Understanding the Core Steps and Procedures Involved With
Establishing Effective Leading Indicators
Procedural knowledge is the knowledge of skills, techniques, and methods that determine
the steps to delivering a desired outcome (Vukić et al., 2020). For this study, operations leaders
must understand the contents of their job description related to understanding core skills and
procedures involved with establishing effective leading indicators. They detailed the concept of
the accountability cube to determine methods of measuring accountability. They determined that
there are clear consequences when gaps in accountability exist. In addition, they affirmed that
gaps exist in organizations between the accountability mechanisms in places versus theoretical
expectations of accountability itself. Guo (2016), via research on developing effective leading
indicators, determined that management’s understanding of the core procedures required were
key to successful implementation. Manuele (2009), in a study on leading and lagging indicators,
established that leaders’ understanding of the steps to improve safety processes and establishing
27

effective leading indicators were significant in terms of managing and impacting risk. In a
review of organizations’ financial performance, Ittner and Larcker (1998) discovered that
leaders’ focus on certain non-financial measures utilizing procedures identified as predictive,
such as customer satisfaction, did have an impact on firm performance. Grosfeld-Nir et al.
(2007) described leadership’s implementation of the Pareto managerial principle and determined
the impact of the Pareto theory and the utilization of Pareto charts as key steps to driving
performance improvement. Focusing on the most impactful factors in the appropriate sequence
was significantly impactful on business results in current management practice (Grosfeld-Nir et
al., 2007).
Metacognitive Knowledge: Having the Ability to Reflect On and Adjust Necessary Skills,
Behaviors, and Plans to Implement Effective Leading Indicators
Per Herrera and Hovden (2008), metacognitive knowledge is the ability to reflect on and
adjust necessary skills and knowledge, including general strategies, assessing demands, planning
one’s approach, and monitoring progress as researched by. They researched the approach of
metacognitive knowledge relative to leading indicators applied to maintenance in the framework
of resilience engineering and determined that leading indicator development must include careful
consideration of the total operations of the system from a reflective perspective. For this study,
operations leaders must have the ability to monitor and adjust their behavior to develop and
implement leading indicators. Sheehan et al. (2016) determined that leaders that exhibited the
necessary level of safety leadership behavior had a considerable positive impact on the behavior
of the teams and overall safety performance.
Research determined that metacognitive knowledge plays a key role in developing and
implementing leading indicators. Leading indicators were determined to be subject to inter-
28

subjectivity and, as a result, must be determined based on consensus between experts and
decision makers after collectively reflecting on the ideal measures to use. Leaders’ core
behavioral tasks like monitoring and reflecting, carrying out analyses of change or
“interpretation of information at hand to reveal what it is believed to be important” (Odewald &
Reiman, 2003) were determined to be necessary for interpretation and action in effective leading
indicator implementation.
Metacognitive knowledge also plays a critical role in reinforcing accountability as it
pertains to leading indicators. Royle and Hall (2012) explored the relationship among
McClelland’s theory of needs, feeling individually accountable, and informal accountability for
others. Their findings indicated that leadership reflection on needs for power, achievement, and
affiliation increased individual feelings of accountability amongst their teams. Latham and Locke
(2006) researched enhancing the benefits and overcoming the pitfalls of goal setting and
determined that leaders’ meta-cognition is particularly necessary in environments with minimal
structure. Ultimately, they determined that when leaders lack the knowledge and skill to attain a
performance goal, giving them a difficult goal may lead to poorer performance among their
teams, hindering their ability to implement leading indicators effectively. They discovered that
reflecting and adjusting skills via learning goals are more effective because they prompt people
to generate solutions to an impasse, implement them, and monitor their effectiveness.
Metacognitive knowledge plays a key role in developing and implementing leading
indicators through the ability to design and execute effective plans to improve performance. For
this study, operations leaders must understand the contents of their job description related to the
ability to reflect on and adjust skills and plans to implement effective leading indicators. As
Frankel and Saravelos (2012) affirmed, procedural knowledge is the knowledge of the skills and
29

procedures involved with the task, including techniques, methods, and necessary steps. Their
research leveraged evidence and literature regarding the 2008 global financial crisis and
determined several variables correlating with planning were consistently effective in predicting
future performance. Zheng et al. (2019) researched the practice of leaders developing plans to
implement leading indicators to improve project performance measurement and discovered that
the process of mapping and subsequent application of leading indicators did control overall
project execution performance. Rhodes et al. (2009) utilized systems engineering analysis and
literature to assess management’s planned implementation of leading indicators for improving
program and technical effectiveness. A key finding from this research was that management’s
ability to reflect and plan effectively in defining the correct measures and procedures to quantify
at an appropriate frequency is critical. Planning and making the necessary adjustments to
determine viability and reinforce accountability are core to leading indicator effectiveness.
Finally, there were several findings regarding the impacts of poor planning in developing
leading indicators. One key observation was that the lack of historical and baseline trend data
when developing an overall approach hinders effective leading indicators implementation. This
challenge of the absence of management developing adequate plans for leading indicators was
also affirmed with research by Rhodes et al. (2009) that many decisions that impacted results
were tribal knowledge and not properly documented. Ebrahim (2003), in a study of
accountability in practice, determined that leadership not understanding the importance of plans
to reinforce downward accountability mechanisms results in the under-development of leading
indicators. In addition, Ebrahim (2003) affirmed that effective accountability leadership skill
exhibited by management in organizations is about both leaders being held responsible while
also exhibiting the willingness to take accountability.
30

Neely et al. (2005) conducted research on performance measurement system design and
discovered that traditional measures of performance are inappropriate for manufacturing today
due to short-term focus. To ensure success, leadership’s performance measurement improvement
plans must incorporate more predictive or leading measures to drive improvement. Craft and
Leake (2002) elaborated further regarding the process of leveraging the Pareto principle by
leaders via organizational planning and decision making and determined that complex processes
can significantly hinder the speed of decision making and that a thorough assessment of project
impact in alignment with organizational priorities can improve the decision-making process and
corresponding allocation of resources. Operations leaders need to understand their job
description related to implementing effective leading indicators and monitor and adjust their
behaviors to develop and implement leading indicators as well.
Table 4 provides details relative to knowledge influences, types, and assessments to allow
for further analysis of the gap in performance. The assessments detailed in the table will be the
mechanisms utilized to understand the level of declarative, procedural, and metacognitive
knowledge that the operations leaders possess or will need to possess to develop and implement
effective leading indicators.

31

Table 4
Knowledge Influences, Types, and Assessments for Knowledge Gap Analysis
Knowledge influence Knowledge type Knowledge influence assessment
Operations leaders need to
understand the job description
related to underlying principles
to develop and establish leading
indicators

Operations leaders need to
understand contents of job
description related to
understanding the core skills
and procedures involved with
establishing tangible
mechanisms to counteract
resistance to accountability
when developing leading
indicators

Declarative

Survey
Parts of my job description
require me to know how to
develop leading indicators in
delivering lagging KOI’s
How would you rate the level of
clarity in your job description
as an operations leader
regarding the expectation in
establishing effective leading
indicators?

Document analysis
Operations leader job
descriptions, look for evidence
of knowledge
PMC improvement program
charters

Interviews
Tell me what you know about
how the operations leader role
relates to leading indicators?
Tell me what you know about
how the operations leader role
relates to creating effective
accountability mechanisms?
Please define what is a leading
indicator. Follow up: What are
some organizational leading
indicators you are aware of as
an operations leader?
What is the difference between a
leading indicator and a lagging
indicator?
What are key challenges to
driving accountability in your
organization?
32

Knowledge influence Knowledge type Knowledge influence assessment
If you had to explain leading
indicators to someone what
would you say?
Operations leaders need to know
the steps to develop effective
leading indicators

Operations leaders need to
understand how to create
effective accountability
mechanisms
Procedural Survey
What are the current steps of
leading indicator development
that you practice (select all that
apply from the following list):
Categorize indicators into
the levels of effort
needed to establish as
H/M/L
Develop a strategic plan
Defining predictive and
influenceable measures
Track additional indicators
Refine existing
systems/behaviors/operati
ons
Align commitment for
tracking and
communication

Please place in order the steps
of leading indicator
development (order the list
below):
Categorize indicators into
the levels of effort
needed to establish as
H/M/L
Design indicators that are
measurable, deployed,
and result in outcome
improvement
Defining predictive and
influenceable measures
Collect comprehensive
input from the team on
proposed leading
indicators
33

Knowledge influence Knowledge type Knowledge influence assessment
Refine existing
systems/behaviors/oper
ations
Align commitment for
tracking and
communication
Organize lead indicators
into two groups:
behavior and outcomes

Document analysis
Internal training documents,
continuous improvement
modules

Interviews
Tell me how you would approach
identifying the right leading
indicators for your
organization?
Tell me how you prioritize key
operations indicators in your
organization?
If you had to explain to someone
how to implement
accountability mechanisms,
what steps would you give
them?
If you had to explain to someone
how to implement leading
indicators, what steps would
you give them? What would
you do first? What would you
do next?
Operations leaders monitor and
adjust their behavior to
prioritize the development of
leading indicators

Operations leaders need to
understand the contents of job
description related to having the
Metacognitive Survey:
How do you adjust your plan and
approach to ensure you have
effective leading indicators in
place?

Document analysis
34

Knowledge influence Knowledge type Knowledge influence assessment
ability to reflect on the optimal
process and create necessary
skills and plans to develop,
measure, and prioritize effective
leading indicators

Performance summaries with
trends over time that show
adjustments to approach

Interview
Tell me about a time when you
improved on a leading
indicator?
Tell me about a time you had to
adjust your approach and
measures around leading
indicators to drive
improvement?

Motivation
Motivation is the second component Clark and Estes (2008) identified as key to
improving performance in organizations. Assumed motivation influences supported by literature
contribute to the problem of practice and are the gaps relative to what the operations leaders
value, believe, have confidence in, and prioritize. Pintrich (2000) researched a motivational
science perspective on the role of student motivation in learning and teaching contexts and
determined a correlation between the student’s level of learning and motivation and the teacher’s
effectiveness in providing the instruction. Pintrich emphasized the importance of establishing a
connection of the goal to materials and activities relevant to the learners. Simpkins et al. (2006)
determined that personal interest through opportunities and choice can increase motivation. From
a leadership perspective, Britner and Pajares (2006) established the connection between leaders
who model congruent values and creating a positive environment for those around them to
improve performance. Additionally, Bong and Clark (1999) established the relevance of student
35

self-efficacy in improving motivation. Overall, the operations leader’s active engagement in
motivating those for whom they are responsible is an assumed influence to be studied further.
Two theories discussed in this section are value theory and goal orientation as it relates to
operations leaders and leading indicators.
Operations Leader Task Value of Leading Indicators
Task value is when motivation, learning, and performance are enhanced if a person
places value on the task in the areas of intrinsic value, extrinsic value, attainment value, and/or
cost value. Bong (2001) determined that students’ exam scores and enrollment intentions had a
strong correlation with the level of task value. Research by Cole et al. (2008) similarly
determined that the level of task value had a significant impact on students and the level of effort
and test performance for all of the tests observed. Stuart (2003) determined that higher task value
amongst high school athletes resulted in positive achievement and experiences. Eccles (2005)
determined that learning and motivation are enhanced if the learner values the task. Finally,
Borgogni et al. (2011) highlighted a connection between feedback and individuals’ success on
challenging tasks and their overall perception of competence.
The operations leader must consider developing leading indicators as important to
execution at a high level. Dorfman (2017) highlighted the effectiveness of major league pitchers
due to the intense level of concentration developed in them due to their leader’s influence. Pink
(2009) affirmed the importance of the leader focusing the team on what is important and
providing for them the value of their work in allowing autonomy and ensuring they gain mastery.
Sinek’s (2009) research revealed that when leaders align their team behind a common higher
purpose, results increase at a much higher level than those leaders who do not. Additionally,
research by Ordóñez et al. (2009) illustrated the importance of the operations leader focusing
36

their teams on the vital, most impactful objectives versus overwhelming the team with a high
quantity of objectives that represent an overall diminishing task value to the employees who
must carry the tasks out. According to the research, leaders who effectively reinforce the task at
hand to the team and emphasize its importance also deliver higher performance levels.
Operations Leaders Goal Orientation Regarding Leading Indicators
Brett and VandeWalle (1999) defined goal orientation theory as an individual disposition
towards developing one’s own ability. Ford et al. (1998) revealed that goal orientation can
predict an individual’s achievement and performance level. Further, motivation, learning, and
performance are enhanced if a person places value on the task in the areas of intrinsic value,
extrinsic value, attainment value, and/or cost value (Ford et al., 1998). Operations leaders
(Pintrich, 2000) demonstrated the ability to focus on mastery orientation with respect to their
specific goals. By allowing team members to focus on planning their work more effectively and
corresponding timelines (Anderman et al., 2010), operations leaders drive higher motivation.
Yough and Anderman also identified the importance of the operations leaders reinforcing the
disposition for self-improvement to build a community of learners amongst their team that
supports one another. This, in turn, correlates with improvements in motivation (Pintrich, 2000).
Radosevich et al. (2007) determined that leaders’ self-efficacy had a positive impact on
goal orientation and that goal orientation subsequently positively influenced their performance
and that of their teams. Stevens et al. (2007) determined that a leader’s mastery orientation e and
their subsequent ability to communicate effectively improved the learners’ level of belonging
and motivation. Ford et al.’s (1998) research also affirmed that the leader’s mastery orientation
had a positive correlation with the learner’s metacognitive activity and motivation. Anderman et
al. (2006) highlighted the importance of operations leaders emphasizing, via communication, the
37

importance of improving one’s own ability via goal orientation and subsequently allowing their
people to take calculated risks to increase motivation and ultimately achieve breakthrough
performance. From a process improvement perspective, the application of lean concepts such as
poka-yoke in implementing leading indicators can have a positive impact. Bajjou et al. (2017)
found that the utilization of error-proofing tools such as poka-yoke resulted in the improvement
of construction safety performance. Similarly, in the UK meat industry, the utilization of lean
tools such as poka-yoke resulted in a performance improvement (Zokaei & Simons, 2006).
Thirdly, in the industrial segment, applying lean concepts in leading indicators implementation
can also have a positive impact on design-led initiatives’ effectiveness (Eifler et al., 2013). As a
result of the research, operations leaders must exhibit a goal orientation to effectively drive
change and implement leading indicators.
Table 5 identifies two motivational influences: task value and goal orientation. An
examination of these influences determined how motivation affects the operations leaders’
ability to develop and implement leading indicators. This effect was the focus of the research in
this study.

38

Table 5
Assumed Motivation Influence and Motivational Influence Assessments
Assumed motivation influences Motivational influence assessment
Task value: Operations leader values the
objective in developing leading indicators

Type of focus:
Extrinsic value (utility)

Survey
Rate the level of importance to you in
implementing leading indicators as an
operations leader for the following
categories: Accountability, customer
satisfaction, financial performance,
quality performance, inventory, startup
performance.

Document analysis
Organizational mission statements, value
declarations, vision statements

Interviews
How valuable is it to you to establish
effective leading indicators?
How important is it for you to
communicate to your teams in
delivering core KOIs?
What do you see as the value of
accountability mechanisms around
leading indicators?
Some would say it’s not valuable to look
at leading indicators – what are your
thoughts?

Goal orientation: Operations leaders’ possess
mastery orientation in developing leading
indicators, including lean concepts

Survey
Leverage Likert-type items:
Please rate your level of commitment in
developing leading indicators even if
you make a lot of mistakes.
Please rate the level of lean concept
utilization in your organization to
deliver core KOIs
Please rate the level of mastery you
believe is required for you to be
successful in your current role?

Document analysis
39

Assumed motivation influences Motivational influence assessment
Review examples of internal documents
and communications and likely impact
on goal orientation
Forms of praise or criticism public or
private
What praise is attributed to
Consequences of success or failure
Whether employees ask for help

Interviews
Tell me about a time when you didn’t
quite hit the mark? What did you do in
response to this?

Organizational Influences
General Theory
Organizational assumed influences supported by literature contribute to the problem of
practice and are the gaps relative to what resources, policies, procedures, and characteristics of
the environment support the operations leader in achieving the goal. Gallimore and Goldenberg
(2001) analyzed cultural models and settings to connect minority achievement and overall school
environment improvement. Schein (2010) researched the effect of organizational culture on
employers’ behavior and reinforced the importance of effective leadership. Clark and Estes
(2008) established the importance of the organization in improving performance and ensuring
employees have the tools, processes, and materials to ensure success. Cultural models are present
in organizations and represent the values, beliefs, attitudes that are generally invisible and
automated (Gallimore & Goldenberg, 2001). Cultural settings represent those visible, concrete
manifestations of cultural models that appear within activity settings (Gallimore & Goldenberg,
2001)
40

The Organization Needs to Have a Strong Accountability Binary Influence on the Operations
Leaders in Developing Leading Indicators (Cultural Settings)
As part of the review literature, an accountability culture can have a considerable impact
on the operations leader’s ability to develop leading indicators. Rajandran (2013) performed
research on safety behavior in organizations and determined that developing tools to reinforce
accountability, eliminate gaps, and address hurdles from an organizational perspective can
improve outcomes. Bernthal (2020), in research on marine pilots, affirmed that an accountability
culture provided the framework to create processes to improve job satisfaction and engineer out
hurdles. Research from Hentschke and Wohlstetter (2004) focused on the federal No Child Left
Behind Act of 2001 and provided a framework to determine key problems present when the
accountability binary is lacking. In addition, their research highlighted the need to create an
environment with a strong accountability relationship between values, decision rights, and
information exchange. In the area of communication, Berger (2014), via research on
organizations, determined that effective communication and information exchange that
subsequently strengthens the accountability relationship is critical to improving performance and
identifying effective leading indicators.
Challenges around the accountability binary in developing leading indicators include not
gaining value and subsequent ethical alignment between providers and directors. Weber’s (2014)
research offered support that the lack of value alignment results in lower trust in organizations
and a weaker accountability relationship, which greatly hinders leading indicator development.
Finally, Joo (2012) affirmed that limited decision rights, which also weaken the accountability
relationship, have a considerable impact on the leader and member exchange. To counter the
challenges in establishing the accountability binary, strong accountability, the leaders’ influence,
41

and decision rights are key factors in the quality of the relationship and subsequent development
of effective leading indicators.
Hentschke and Wohlstetter’s (2004) accountability framework identifies five common
problems in the environment that need to be addressed. Adverse selection takes into account
situations where the directors are not well informed and do not select the best choice possible for
the provider role. Divergent objectives reflect scenarios where the providers pursue their
objectives at the director’s expense. Information asymmetry references those situations where the
information within the accountability relationship is not shared evenly between both the director
and provider. Weak incentive scenarios are where the director does not have the necessary
decision rights to influence or cause the provider to share the same values as the director or
behave as such. Finally, accountability relationships with limited decision rights result in
situations where the providers are held accountable for practices and outcomes over which they
do not have full authority and responsibility. Overall, these problems create imbalances in the
accountability relationship between director and provider and must be addressed to hold
operations leaders accountable for developing leading indicators (Hentschke & Wohlstetter,
2004).
The Organization Has a Clearly Negotiated Accountability Binary for Operations Leaders’
Leading Indicator Decision-Making
Relationship Between Organization and Operations Leader. An accountability binary
between the organization and the operations leader can have a considerable impact on the
operations leader’s ability to exercise leading indicator decision-making. Foley and Mishook
(2012), in research on education, identified the importance of the organization enabling the
leaders to use data to drive improvement in improving teaching outcomes. Chernesky and Israel
42

(2009) researched the positive connection of organizational accountability, job expectations, and
the leaders’ decision-making in predicting outcomes. Sull et al. (2015) performed research on
why strategy execution unravels in organizations due to gaps in both the level of collaboration
and the accountability binary and what to do about it in the event it occurs. Their research
determined that only 9% of managers indicate they can rely on other functions all the time to
accomplish their goals, representing a weak presence of the accountability binary in the
organization. 30% of managers cite failure to coordinate across units as they cause for not
achieving organizational goals. 55% of middle managers could name one of the top five
priorities. Further, Viseu et al. (2016) demonstrated that past performance is two or three times
more likely than a track record of collaboration to be rewarded with a promotion.
Organizational Climate. Organizational culture has a critical relationship with leader
motivation because it determines how leaders behave and perform (Ipek, 2010) and determines
the strength of the accountability binary Nieminen et al. (2013). Different organizational climate
perceptions can also have a distinct effect on the level of motivation and strength of the
accountability binary (Griffin & Neal, 2000). Ruben et al.’s (2018) research of the healthcare
industry correlated the positive impact of leading indicators with the level of support of the
organizational culture present to reinforce. Similarly, research on organizational safety
determined a strong correlation between organizational culture and ultimate safety performance
(Givehchi et al., 2018). Leadership styles, adequate working conditions (e.g., access to recent
materials), and cooperation between different organizational actors were crucial to professional
motivation (Rad & Yarmohammadian, 2006). Nielsen et al. (2008) established a positive
accountability binary connection via research of leadership behavior and overall approach with
employee satisfaction in that organization. As reported in past literature reviews (Viseu, 2003),
43

variables related to leaders affect their motivation, as do organizational variables, which refer to
the accountability binary relationship established between the leader and the school or
organization.
Kaplan and Norton’s (1998) research determined that establishing dashboards at a regular
cadence to reinforce accountability had a positive impact on the leaders’ decision-making
regarding leading indicators. Cleary et al. (2013) explored the organizational environment further
with research focused on resources, attitudes, and culture in primary health care settings. Key
findings from Cleary et al.’s study were that bureaucratic accountability mechanisms often
constrain external accountability mechanisms and hinder the leader’s ability to establish a strong
foundation for accountability. As a result of these findings, a key conclusion was that it is of
considerable importance to address the challenges with bureaucracy very carefully to ensure an
environment with a strong accountability binary. In addition, assessing the effectiveness of the
accountability mechanisms can limit the potential negative impact of bureaucracy. In summary,
the design of accountability mechanisms to strengthen the accountability binary must take into
account the attitudes and perceptions of individuals involved, the values of the organization as a
whole, and the resources available to support.
Table 6 identifies two organizational influences around accountability that operations
leaders need to be successful and implement leading indicators. These influences were used to
better understand how PMC can affect operations leaders’ ability to identify, implement, and
execute effective leading indicators while leveraging strong accountability mechanisms. The
organization needs a clearly negotiated accountability binary for operations leaders, which was
explored in this study.

44

Table 6
Organizational Influences and Organizational Influence Assessments
Assumed organizational influences Organization influence assessment
Cultural models: The organization needs to
have a strong accountability influence on
the operations leaders in developing
leading indicators
Interviews and focus groups
How resistant to change is the organization
in adopting new approaches to drive
accountability and improve performance,
such as leading indicator development?
How well do we support one another in the
organization and hold each other
accountable while finding better ways to
improve performance around leading
indicators?

Document analysis/observations
Internal documents that demonstrate
accountability or lack of accountability

Surveys
On a scale of 1–5, with 1 being autonomous
and 5 being authoritarian, how would
score the current level of accountability
provided by the leadership of the
organization around development of
leading indicators?
On a scale of 1–5, with 1 being pessimistic
and 5 being optimistic, how would you
score the current level of communication
from leadership in creating an
environment that facilitates
accountability to leading indicator
development?
Cultural settings
The organization has a clearly negotiated
accountability binary for operations
leaders’ leading indicator decision
making, including the expectation of the
development of leading indicators
The organization has a clearly negotiated
accountability binary for operations
leaders’ leading decision-making,
Interviews
In what ways do you share the leading
indicator development goals that align
across functions from an accountability
perspective?
Can you talk about your freedom to use
decision rights as it relates to your
leading indicator development
accountability?
45

Assumed organizational influences Organization influence assessment
including the prioritization of the
development of leading indicators
Must have the necessary organizational
structure to support the development of
leading indicators
Must have the necessary organization to
support the development of leading
indicators
The organization provides job descriptions
that reinforce the importance of
developing leading indicators

How does the organizational environment
regarding accountability facilitate your
development of effective leading
indicators?

Is there anything else regarding leading
indicators that you would like to share
regarding the development of leading
indicators that we have not already
covered?

Document analysis
Internal documents that demonstrate lack of
accountability or acceptance of loafing

Surveys
How much autonomy are you given to
develop effective leading indicators for
your organization?

Conceptual Framework: The Interaction of Stakeholders’ Knowledge and Motivation and
the Organizational Context
The study’s conceptual framework is based on the premise of looking through the lens
from a critical/transformative perspective where all participants can be successful if they
understand the rules and that institutional change is possible under the right conditions. The
design study focuses on the foundational input from the participants, namely the middle
managers, regional leaders, and senior executives, to understand the problem of lack of
achievement of lagging organizational measures. The key concepts are the focus for this study,
which leveraged Clark and Estes’s (2008) KMO methodology to understand what would be more
effective leading metrics to reinforce with demonstrated accountability mechanisms (Hentschke
& Wohlstetter, 2004). The execution of the leading metrics per the soft reform space theory (de
46

Oliveira Andreotti et al., 2015) would result in achieving key operational metrics, including
controllable operating profit, customer satisfaction, program startup performance, and inventory
turnover. The model to be used to address the problem of practice, illustrated in Figure 1, is
Clark and Estes’s (2008) gap analysis model. The model is an analytical and systems-based
approach that clearly defines the organization’s goals and identifies the gap between the current
level of performance and the desired level of performance.

Figure 1
Conceptual Framework Illustrating the Relationship Among Operations Leaders’ Knowledge
and Organization and the Organizational Goals

47

Personal experience and knowledge as well as literary works related to the concepts of
focus helped to identify and develop the needs to close the gap in performance relative to KMO
influence. The identified needs were validated through interviews, focus groups, surveys, and
literature review. The philosophical worldview is critical and transformative, wherein there are
existing power and social structures where one names whose knowledge is listened to and who is
silenced. This approach focuses on uncovering subjugated knowledge via investigator/participant
dialogue and centers on those facing injustice to create a view of the problem, the people under
study, and the necessary changes required to meet the needs for gap closure (Creswell &
Creswell, 2018). For this problem of practice, the theoretical frame consists of
critical/transformative, soft reform space, and institutional change. To understand the gap in
developing effective leading indicators, the approach would be more participatory and supportive
methodology (Aliyu et al., 2015) and focus on the operations leaders’ perspective relative to the
existing power structures and expose the issues of change and marginalization. This
methodology aligns with the planned study group targeted to understanding their perspective of
the current power structures and how leading indicators can effectively drive positive change and
delivery of the organizational metrics.
Conclusion
Leading indicators can be effective predictors of further performance. As indicated as
part of the literature review, KMO influences are significant contributors in determining whether
organizations can achieve their current lagging goals or fail. Therefore, the purpose of this study
is to understand the needs of the operations leaders in the areas of knowledge and skill,
motivation, and organizational resources necessary to create effective leading indicators that
allow the organization to reach its performance goals. In the current conceptual framework,
48

operations leaders can successfully impact change if they understand the current parameters.
This framework stresses the potential for the operations leaders to drive change under the right
conditions. The literature review identified potential KMO influences that contribute to the
problem of practice. These influences were the focus of this study, and Chapter Three describes
the validation of their impact and presents the study’s overall methodological approach.

49

Chapter Three: Methodology
Chapter Three will present the overall research design and methods for data collection
and analysis used during the study to answer the following core three research questions:
1. What are the operational leaders’ knowledge and motivation needs related to their
development of leading operations indicators in alignment with organizational goals?
2. What is the interaction between the organizational culture and context and the
operations leaders’ knowledge and motivation?
3. To what level do operational leaders currently utilize leading indicators?
4. What are recommended knowledge, motivation, and organizational solutions?
For this study, the purpose was to explore to a deeper level the problem of practice. The
performance problem of PMC is the lack of achievement of lagging accountability measures
connected to operational results. The corporation is meeting 0% of its leading indicators as they
currently do not exist. As a result of their lack of existence, the focus of the study was to
understand the drivers for the performance gaps and ultimately develop an innovative solution
based on the results of the research study. In performing this study, the expected outcome was a
higher level of understanding relative to the needs of the operational leader group in the areas of
knowledge and skill, motivation, and organizational resources. This higher level of
understanding helped create recommended effective leading metrics reinforced by accountability
mechanisms to allow PMC to reach its performance goals.
Methodological Approach and Rationale
The approach to this study was mixed methods. This approach dates to the 1980s and has
evolved to be comprehensive (Creswell & Plano Clark, 2011). Qualitative and quantitative data
each have strengths and weaknesses (Creswell, 2010). Quantitative is focused on more closed-
50

end type research with a very specific focus, while qualitative is more open-ended specific
inquiry, placing more focus on the overall meaning (Patton, 2005). Fossey et al. (2002) affirmed
that the focus of the data collection and ensuring the participants’ subjective contexts, meanings,
and actions are brought to light are critical to effective qualitative research. By combining both
types of research, the overall strategy for this study aimed to gain deeper insight into the causes
of the problem of practice and integrate the data gathered. The mixed-methods approach is also
ideal when the researcher has access to both quantitative and qualitative data (Leech &
Onwuegbuzie, 2009), which applies in this study. Mixed methods research involves rigorous
methods such as data collection, data analysis, and interpretation of all data (Johnson et al.,
2007).
Participating Stakeholders
The target population for this study for both survey and interview is the subject matter
experts in the organization: the operations leaders at each of the manufacturing sites. The criteria
for the operations leaders required that they have been in the organization for an extended period
and understand the culture, are well respected, and can influence the organization in alignment
with the theoretical framework regarding qualitative research (Osanloo & Grant, 2016).
Survey Sampling Criteria and Rationale
The initial criterion was to ensure participants were subject matter experts at each
manufacturing site. Consistent with the methodological approach, one key criterion is choosing
operations leaders or lead subject matter experts for each manufacturing site as these are the key
individuals that would be accountable for driving organizational change at their site. An
additional core characteristic of this criteria was at least 2 years of experience in either an
operations leader role in day-to-day operational management or customer program management
51

leadership. Finally, a third core characteristic was for the participants to be affirmed by the
regional VPs and senior-level organizational executives.
The second criterion was to ensure the operations leader’s manufacturing site was high-
performing. A high-performing site is defined as an output of a review of existing organizational
documents and artifacts. Key characteristics of this criteria include exceeding core operational
and quantifiable lagging measures such as operating profit, customer satisfaction, on-time
product delivery, and overall sales revenue. The intent of targeting high-performing
manufacturing sites is to understand qualitatively what key components these particular sites
possess to drive higher performance levels versus the average from a leading indicator
perspective.
The third criterion was to ensure the operations leader’s manufacturing site was average-
performing. An average-performing site is defined as an output of a review of existing
organizational documents and artifacts. Key characteristics of this criteria include an average
level of performance on core operational and quantifiable lagging measures such as operating
profit, customer satisfaction, on-time product delivery, and overall sales revenue. The intent of
targeting average- or medium-level performing manufacturing sites was to understand
qualitatively what key components these particular sites possess to drive higher performance
levels versus the average from a leading indicator perspective.
The fourth criterion was to ensure the operations leader’s manufacturing site was low-
performing. A low-performing site is defined as an output of a review of existing organizational
documents and artifacts. Key characteristics of this criteria include exceeding core operational
and quantifiable lagging measures such as operating profit, customer satisfaction, on-time
product delivery, and overall sales revenue. The intent here was to understand qualitatively what
52

components these sites are not exhibiting compared to the higher-performing sites from a leading
indicator perspective. These findings will support the development of innovative solutions
moving forward as an outcome of the study.
Survey Sampling, Recruitment, Strategy, and Rationale
The overall sampling strategy is non-random purposive, with the focus on the operations
leaders who best fit the criteria per the conceptual framework. There are approximately 20
manufacturing sites in the medical manufacturing segment population versus over 120 PMC
manufacturing sites globally. In terms of the total number of operations leaders of the
approximately 20 manufacturing sites, there are approximately 30 to 50. Researchers sought at
least 30 participant site operations leaders for the survey. Greater or equal to 50% of the
population was the focus of this study, with an equitable balance of sites performing at a high,
average, and low level versus the existing lagging operational metrics. The population is defined
as an output of an initial quantitative review of existing organizational documents and artifacts.
The survey took place as a secondary step in the data collection to understand qualitatively what
potential performance gaps and subsequently provide actionable insights for the interviews to
follow.
Interview sampling criteria were the same as those for the survey. Participants were
required to be subject matter experts at each manufacturing site with at least 2 years of
experience in either an operations leader role in day-to-day operational management or customer
program management leadership. They had to be affirmed by the regional VPs and senior-level
organizational executives. The second criterion was to represent a high-performing
manufacturing. The third criterion was ensuring the representation of a manufacturing site
identified as average-performing. The fourth criterion required a manufacturing site identified as
53

low performing. These findings were sought to support the development of innovative solutions
moving forward as an outcome of the study.
Interview Sampling, Recruitment, Strategy, and Rationale
The sampling strategy for the interviews was purposeful and direct. I sought 10
operations leaders for direct interviews. This is appropriate as it represented approximately one-
third of those surveyed and featured an equitable range of the population of high, medium, and
low levels of performance versus the established and lagging quantifiable measures. The timing
of the interviews was at the middle of the study and consisted of a subset of survey participants.
The overall strategy to recruit was to set up an initial connection with the individuals via Zoom
in light of the current COVID pandemic. I provided disclosure to the interviewee, including the
intent and overall process, and addressed concerns. Key areas of emphasis were that transcripts
would be generated during the interview, the overall intent of the study, and the overall
encouragement of their participation. At the conclusion of each interview, there was a timely
review of transcripts to ensure the data’s accuracy. Finally, as affirmed via research (Patton,
2005), a prompt follow-up with the interviewees where needed for clarification was conducted to
prevent any uncertainty or vagueness in the information.
Data Collection and Instrumentation
This study featured a mixed-methods approach to gather data from a sample of operations
leaders as well as via organizational documents and artifacts. Quantitative data collection
performed initially via company artifacts and documents will be the core starting point of
gathering quantitative information and corresponding lagging operational measures to begin the
process. For qualitative data collection, the two methods to be utilized in this study were surveys
and interviews. Surveys are an effective method to gain more aggregate data from the larger
54

population while also enabling the researcher to perform in a shorter time. As aggregate
qualitative data are collected and analyzed, the framework for more intentional and focused
interview study forms. This aligns with the concept map framework detailed previously and
allowed the researcher to address the core research questions per the KMO framework (Clark &
Estes, 2008). The interview approach as the follow-up data collection method allowed the
researcher to determine the underlying meaning and root causes behind the problem of practice
and how to address the problem with an innovative solution.
Surveys
The survey consists of a mix of open and closed-ended questions. Qualtrics was used to
administer the survey and was distributed via email and sharing of a link. The overall strategy
was to utilize key lagging indicators of operational performance, such as direct labor utilization
and total manufacturing loss, as the focus of the surveys and connect these measures to the core
research questions. The overall feedback from these surveys provides a better qualitative
understanding (Patton, 2005) of the root causes of performance gaps. To maximize the survey’s
reliability and content validity, careful consideration was given to the participants and their
overall credibility. This was incorporated into the criterion for participant selection.
Maximization of the response rates, including simplification of the questions (Salkind, 2014) and
having the ability to complete in less than 15 minutes online was the core strategy to ensure the
ease of execution. The insights gained were key in understanding the vital leading indicators that
need to be established.

55

Interviews
The interviews were one-time interviews with each participant. The sessions were
targeted to last from 60 to 90 minutes. The overall approach was semi-structured to focus the
conversation while remaining open-ended to better understand the core factors driving
performance gaps. The questions leverage the KMO framework and were purposefully set as
open-ended to gain richer content from the interviewees (Patton, 2005). The key emphasis was to
gain more of a qualitative look (Bogdan & Biklen, 1997) relative to the performance gaps
regarding the lagging measures while also providing insights on what the appropriate leading
indicators need to be. Formally, the initial structure of the interviews was 10 to 12 questions
focused on the research questions relative to the problem of practice and then, where appropriate
additional questions to probe, with a focus on causation explaining the current situation. The
questions focused on the components within each KMO category to better gain insights and
understand the root causes of the gaps.
Documents and Artifacts
For the purpose of understanding the core drivers behind the gaps in achieving the
lagging operational measures, existing company artifacts such as performance reports,
operational summaries, and public financial documents were used to establish an initial
qualitative base of study. In alignment with the conceptual framework, the qualitative
organizational measures act as the initial starting point for analysis and provide evidence of facts,
information, and terminology (Merriam & Tisdell, 2016). Organizational documents were
analyzed as well to determine the current level of conceptual and metacognitive knowledge that
exists in the operational leader population relative to leading performance measures. Quantitative
data via artifacts as well, typically captured in researcher-generated material (Merriam & Tisdell,
56

2016) after the study has begun, can provide greater insights into the current situation at PMC
and will also be an area of focus as part of this study.
Data Analysis
Since this is a mixed-methods study, quantitative data were analyzed initially utilizing
organizational documents and artifacts. After gathering an overall summary of core operational
and quantifiable lagging measures, the manufacturing sites were grouped into the performance
categories of high, medium or average, and low. These categories identify the targeted and
minimum quantity of sites for each category that will require completed qualitative analysis to
ensure credibility and reliability.
Subsequently, a qualitative analysis was performed utilizing surveys and interviews. For
the quantitative surveys, responses feature Likert-type options. Surveys are assumed to be
representative of the population of the stakeholder group. Documents and artifacts are also
referenced as baseline information. Descriptive statistical analysis allows the researcher to gain a
richer overall understanding of what is happening in the environment being studied (Wolcott,
1994) and was conducted once all survey results were submitted.
For qualitative data analysis, interviews are a key enabler to perform coding and
synthesis of key themes. In both quantitative and qualitative approaches, there will be consistent
alignment with the conceptual framework of the study, ensuring thorough and insightful
learnings and subsequent hypotheses. For stakeholder groups of fewer than 20, the percentage of
stakeholders who strongly agreed or agreed is presented in relation to those who strongly
disagreed or disagreed. For interviews and observations, data analysis began during data
collection, where the researcher wrote analytic memos after each interview and observation. The
researcher documented observations, concerns, and initial conclusions about the data in relation
57

to my conceptual framework and research questions. Once the researcher left the field,
interviews were transcribed and coded.
Coding is a process of making notations alongside data that allows it to be grouped in
categories relevant to the study research questions (Merriam & Tisdell, 2016). To expand on the
theory for the study, the process encompasses mirroring the three-phase approach of coding
(Corbin & Strauss, 2015). In the first phase of analysis, the approach will be to use open coding,
looking for empirical codes, and applying a priori codes from the conceptual framework. A
second phase of analysis was conducted where empirical and prior codes were aggregated into
analytic/axial codes. In the third phase of data analysis, the overall approach was to identify
pattern codes and themes selectively that emerge in relation to the conceptual framework and
study questions. Finally, an analysis of documents and artifacts for evidence will be conducted
while remaining consistent with the concepts in the conceptual framework.
Credibility and Trustworthiness
Credibility and trustworthiness are key aspects of mixed methods research. Credibility
(Howie, 1996) references the overall believability and accuracy of the research performed.
Trustworthiness (Polit & Beck, 2012) is the level of confidence in the data, analysis, and
approach utilized to ensure the quality of the research performed (Connelly, 2016). Per Denzin
and Lincoln (2011), the researcher must have a high level of confidence in the process regarding
how the inquiry was conducted and the overall results. As a result, some core approaches were
utilized to maintain the study’s credibility and trustworthiness where both the quantitative and
qualitative parameters were reviewed to determine validity (Creswell, 2010). From a qualitative
perspective, an initial focus was on the quantity of the interviews. If there are inconsistencies in
the data collected, the targeted population for the interviews can be increased. The length of the
58

study can be extended as well with additional surveys and interviews based on the data collected
(Guba & Lincoln, 1994). Another alternate approach, if needed, is to obtain additional
organizational artifacts or documents to affirm the research findings. Leveraging additional
strategies to improve the overall credibility of the study will be critical in developing effective
innovative solutions focused on the utilization of leading indicators.
Validity and Reliability
To maintain validity and reliability throughout the study, a number of approaches were
employed. Validity is the level of accuracy of the research in measuring the focus of the study
(Roberts & Priest, 2016). Reliability references the stability of the measures and corresponding
data both internally and externally (Kimberlin & Winterstein, 2008). Lichtman (2013) provided
key guidelines to improve validity and reliability, including clarifying how the study is
conducted and affirming strongly that the topic of study is important. Tracy (2013) affirmed this
approach and added that the research methods must be transparent and that the study itself must
be important and ultimately make a considerable contribution. The first approach was to ensure
careful consideration of the participants and their overall credibility. This was explained
previously with the choice of selection criteria. An additional strategy to ensure validity and
reliability is having a larger sample (Guba & Lincoln, 1994). Triangulation, where more than one
type of data source is used and the information gathered is cross-checked, is a strong approach to
improve a study’s trustworthiness (Patton, 2005). The theory with triangulation is that it counters
the theory that the study is not credible because it collects data from multiple sources. There are
also additional internal documents that can be leveraged as needed with standardized internal
quantitative measures to ensure appropriate data interpretation. Finally, as the research is
59

conducted, the necessary level of transparency can be provided regarding the methods, samples,
and sites used as well.
Ethics
The core principle for involving human subjects in research is to ensure the subjects are
not harmed (Glesne, 2016). The participants signed an informed consent that ensured their
participation was voluntary, affirmed confidentiality of the data and the participants (Then et al.,
2014), and obtained their permission to record, store, and secure the data gathered. Lincoln
(1995) highlighted the need for the researcher to recognize their relationship to the subjects and
details standards that need to be followed to ensure validity and an ethical approach. As an
executive at PMC, the researcher has an interest in the study as it relates to integrating the
learnings of the study into the improvement plans for the organization moving forward. The
researcher’s incentive plan performance was impacted by the study’s effectiveness and the
researcher’s subsequent ability to put the findings into practice. For this study, the researcher was
not in a leadership role over the interviewees but conducted the inquiry in a peer-type role
relative to the subjects.
To address potential confusion among the organization’s other members regarding the
dual roles during the research, the ethical situational guidelines (Merriam & Tisdell, 2016) for
qualitative research were followed. These guidelines were also followed when participants were
in a subordinate role. To minimize the potential for the participants feeling coerced or pressured
to participate, a comprehensive IRB review ensured the informed consent process was executed
properly (Rubin & Rubin, 2012). In addition, the overall research questions and subsequent
additional probing questions were also reviewed and approved by the IRB to ensure the
participants are not impacted from a performance evaluation or job performance perspective. To
60

ensure that all employees understood the researcher’s role as an investigator, rather than as an
employer and that they could remove themselves from the study at any time (Krueger & Casey,
2002), the informed consent document was utilized throughout the process.
Assumptions and biases had to be accounted for while the research was conducted and
mitigated. As a result, a key approach employed during this study was forming the interview
questions so that they were devoid of any leading tendencies while also bearing no correlation
with the researcher’s own personal life experiences or preferences. Patton (2005) reinforced the
importance of utilizing this approach to ensure objective data collection. A key theory from
Merriam and Tisdell (2016) is that the researcher recognizes their biases objectively and then
proceeds forward in as ethical an approach as possible for the study. From a survey perspective
and in terms of positionality and power, the participants were selected as the interviewer’s peers
to minimize the influence exerted and to mitigate risks to data validity. As mentioned previously,
the participants were informed that senior leadership supported the study and the overall focus
was on ensuring alignment with the overarching purpose of the organization and PMC’s CEO.
The organization’s goal is to exceed the cost improvement lagging measures and expand
operating profit. As Patton (2005) articulated, to mitigate ethical risks, it is key to gain the
participants’ informed consent and reinforce confidentiality and the overall purpose of the
research.
From an interview perspective throughout this study, special care was taken to not target
individuals with a direct reporting relationship to the interviewer. Assurance of this was also
affirmed through the IRB approval process. Not taking heed to this guidance could result in
responses and corresponding data that is false, inaccurate, or less truthful than what is needed to
effectively determine the root causes of the gaps in performance, so the IRB approval was a key
61

gate to ensure this was not the case. The interviewer’s positionality, biases, and identity would
influence the data because the interviewer would likely incorporate it in his or her own
experiences. For this reason, to mitigate ethical risks throughout the interviews, the format of the
questions largely focused on being open-ended and not leading. As Patton (2015) affirmed, the
main focus of the interview is to gather data objectively. The researcher must recognize the
privacy potentially involved with the interviewing process, as Stake (2013) noted, and keep the
interview process disciplined and strict to the code of ethics. These components were core to the
overall strategy regarding the interviewing approach for the study in ensuring the methods were
in full alignment with the ethical guidelines required for IRB approval.
Limitations and Delimitations
This study features a mixed-methods approach, which has limitations as a result,
including the lack of a control group to compare to as well as the dependency on the level of
knowledge of the researcher regarding both qualitative and quantitative research designs
(Merriam & Tisdell, 2016) and the respondents’ truthfulness. While the interviewer can reinforce
that the information will remain confidential, there remains a likelihood that the participants will
not fully disclose during data collection. The interviewer has been at PMC only a year, so there
remains an opportunity to build a stronger foundation for trust as well. The bounds of the study
were created with consideration for ethics, credibility, and validity, as emphasized by Merriam
and Tisdell (2016), and are illustrated in the conceptual framework shared previously. The KMO
influences detailed in the literature review were framed into the organizational framework as
well. The study did not include potential influences outside of the organizational environment
that were not included in the overall soft reform space to narrow its focus. While the researcher
sees the organization within a soft reform space and ripe for change, there remains a possibility
62

that the organization and senior leaders themselves may not be truly open to the change that the
research can make possible. This may, in fact, limit the quality of the data collected during the
process, especially regarding organizational artifacts.
A key approach to delimit some of these concerns is to use both open and closed-end
questions to integrate the information in a convergent manner while also providing some level of
generalizations (Creswell & Creswell, 2018) leverages the advantage of the mixed methods
approach in achieving a richer level of understanding via the data collected. In addition, the
research questions asked throughout the interviews and surveys conducted were linked tightly
with the conceptual framework to ensure all necessary components of the research process were
incorporated. The concept of data saturation and triangulation, as well mitigating the influences
of the researcher’s biases were other delimitation approaches as well (Bogdan & Biklen, 2007).
As Weiss (1995) detailed, it is very difficult to know if the participants’ responses are
open and honest or heavily influenced by their positionality or personal biases. There is also a
potential influence of recency bias with all participants, which can have an unpredictable impact
on the responses and data. As a result, the focus on validity and credibility via the IRB approval
process was key to protecting against these factors. Ultimately, the purpose of this study was to
understand how the KMO influences the ability of operations leaders to develop effective
leading indicators and associated strong accountability mechanisms at PMC. The most
significant limitation of this study is that the research is within the organization itself and may
not apply to other organizations.

63

Chapter Four: Results and Findings
The purpose of this study was to identify the KMO influences on PMC’s medical
manufacturing segment operations leaders’ ability to develop effective leading operational
performance indicators to accomplish the core organizational objectives. The key organizational
objective was for these leaders to meet 100% of their leading performance indicators by 2022.
The assumed causes for the performance gaps were identified in Chapter Two and organized into
the categories of challenges related to KMO factors. This study was based on a gap analysis
framework utilizing a mixed-methods design that effectively addresses complex populations
(Palinkas et al., 2019). Quantitative data were gathered through a survey, and qualitative data
were collected through interviews and document analysis to validate the assumed causes for
gaps. Qualitative results were coded via a priori approach centered around categories aligned
with the study’s needs to maximize effectiveness (Blair, 2015). Quantitative results were
analyzed utilizing cut score heuristics, which has been identified as an effective method to
identify and subgroup certain characteristics in a population. This chapter consists of a review of
the organizational and stakeholder goals, followed by assertions determined by the findings. The
core elements of the study will be summarized in this chapter regarding the following:
stakeholders, survey results, interview results, and overall findings. Lastly, this chapter presents
a summary of the validated KMO influences that ground the recommendations covered in
Chapter Five. Three questions were addressed in the first four chapters:
1. What are the operational leaders’ knowledge and motivation needs related to their
development of leading operations indicators in alignment with organizational goals?
64

2. What is the interaction between organizational culture and context and operational
leaders’ knowledge and motivation to develop leading operations indicators in
alignment with organizational goals?
3. To what level do operational leaders currently utilize leading indicators?
The fourth directional question will be addressed in Chapter Five: What are recommended
knowledge, motivation, and organizational solutions?
Participating Stakeholders
As mentioned, the target population for this study was the subject matter experts at PMC:
the operations leaders at the manufacturing sites. This group consists of individuals who have
been in the organization for an extended period, as illustrated in Figures 2 and 3, understand the
culture, are well respected, and can influence the organization. Of the 26 operational leaders in
PMC’s medical manufacturing segment, 17 (65% response rate) completed the survey.
Subsequently, 12 of the 17 (71% response rate) agreed to participate in interviews where needed.
Of these 12, nine (75% response rate) were interviewed for 45 to 90 minutes. The criteria for
personnel selection for both surveys and interviews, in addition to what was previously
mentioned, ensured an adequate spread of manufacturing sites that were performing at high,
average, and low levels. Figure 4 illustrates the participants by region to emphasize the balanced
representation of the segments in this study. The spread in overall performance ensures data will
validate the assumed influences relative to developing leading indicators. As mentioned
previously in Chapter Three, the interviews were conducted by an independent interviewer, with
no direct reporting relationship to the participants. All interviews were conducted with the
participants’ prior consent.

65

Figure 2
Survey Group Demographics: Years in Company and Industry

66

Figure 3
Interview Group Demographics: Years in Company and Industry

67

Figure 4
Study Group Demographics: By Region

Data Validation
Mixed-methods research methodology was used in this study to answer the research
questions and determine the validity of the KMO influences assumed. The data sources were
surveys, interviews, and document analysis. Document analysis was performed on internal
organization intranet searches (appendix A) and internal files after the surveys and interviews
were completed. The surveys were administered by an independent interviewer through an
organizational survey platform (Appendix B) to pose 17 questions to understand the knowledge,
motivational, and organizational influences in developing leading indicators. The surveys were
administered using a 5-point Likert scale (1 = strongly agree to 5 = strongly disagree). The
68

Likert 5-point scale correlated with survey respondent options ranging from strongly agree,
moderately agree, neutral, moderately disagree, and strongly disagree, with the intent to leverage
the Likert scale to understand the perceptions of the operations leaders to a reasonable level of
reliability (Adelson, & McCoach, 2010). The interviews (Appendix C) were conducted similarly
to gather the necessary qualitative data to affirm the assumed influences as well. Based on the
data collection, the assumed influences were validated fully, partially, or not at all. Based on
statistical analysis, an influence was validated when the data were synthesized to concurrent
findings. Influences were partially validated when some of the data confirmed to a level of
significance that a gap exists within the assumed influences and as not validated when the data
converged on the findings that there were no gaps observed in the assumed influence. The level
of significance was determined by the cut score heuristics per the ETC cut score primer (Yates,
2019). Per that approach, the cut score was based on the participants’ opinions. With the fewer
participants in the study, a higher cut score would be used. Of the 26 operations leaders in the
overall population, 17 participated in the survey. For both survey and interview analysis of the
data collected, certain determinations were established in creating the thresholds to make
assertions. As the survey results were gathered on a Likert scale of 1 to 5, a cut score of 85% or
above determined whether a KMO influence was an asset. If the cut score was less than 85%, the
influence was determined to be a need. With the smaller population and the hypothesis that the
organization would benefit from each of the KMO influences being an asset, targeting a higher
cut score would reinforce this finding. The cut score was also used in triangulation with the other
data collection methods for this study of interviews and document analysis to affirm the level of
the need identified.
69

As stated, certain parameters were established to determine the findings’ validity and the
strength of the argument for surveys as well. Regarding the surveys, 68% was the parameter that
confirmed agreement with the assertion, while findings at 85% or more were considered strong
evidence that the assertion was valid. For document analysis, the presence of assertions was
determined to be evidence of agreement. Relative to the interviews, 50% presence of evidence in
alignment with the priori codes’ alignment with the research questions was determined to be a
parameter establishing an assertion, while 70% or greater was an indicator of a strong assertion.
Results and Findings of Knowledge Needs
Utilizing Clark and Estes’s (2008) framework, the study was based on the assumptions
that declarative, conceptual, procedural, and metacognitive influences impacted the operations
leaders’ ability to develop and implement effective leading indicators. Specifically, operations
leaders must understand the job description as it relates to the underlying principles to develop
leading indicators. Second, it was assumed that operations leaders understand the core skills and
procedures involved with establishing effective leading indicators. Third, it was also assumed
that operations leaders have the metacognitive ability to reflect on and adjust necessary skills and
plans to implement effective leading indicators. Operations leaders were assumed to have the
metacognitive knowledge to monitor and adjust behavior in developing and implementing
effective leading indicators. The following sections present the results and findings of the
research in alignment with the assumptions stated.

70

Knowledge Influence 1: Operations Leaders Understand Their Job Description Related to
Developing Leading Indicators
Survey Results
Overall, the survey data indicated that the operational leaders possessed a strong
understanding of their job description as it relates to developing leading indicators. As indicated
in Figure 5, 50% of the participants strongly agreed (with a 4.3 average on a 1–5 scale) that they
understood their job description. In addition, the respondents’ standard deviation placed all of the
data greater than average, which is statistically significant (> 68% of the population above
average). From a cut score perspective, 88% of the sample population either demonstrated or
strongly demonstrated an understanding of their job description as it pertained to developing
leading indicators as well.

71

Figure 5
Declarative Knowledge Survey Results (Question 10)

72

Interview Findings
As a result of the interviews, a key theme was that the operations leaders understand the
importance of developing leading operational indicators as a core expectation for their roles. All
interviewees (n = 9) demonstrated an overall awareness for this expectation, which was greater
than the 75% threshold, and communicated as such. Some consistent comments from the
participants included the need to be forward-looking as required for leading indicators with
references to acting predictive and focusing on the future. Respondent 9 stated that “the
operations leader role relates very directly to leading indicators and is a driving force in
managing my priorities and accountabilities.” Respondent 7 reinforced the presence of overall
awareness with the comment that “the operations leader of every site needs to have their finger
on the pulse of leading indicators at all times.” Finally, Respondent 2 indicated “that it is most
important as an operations leader to find indicators that show where you need to react now to get
the best results in the future.” Net, the operations leaders expressed an overall awareness that
their role encompassed the expectation that leading indicators are developed and utilized in
managing operations.
Document Analysis
A review of the organization’s intranet to assess the content of the job descriptions for the
operational leaders yielded a number of references to developing leading indicators. There were
frequent references to the importance of defining and executing operational programs, creating
and driving accountability, and providing proactive advice. An emphasis on innovation was also
present in the job descriptions with the expectation that the operations leaders demonstrate
“creativity, develop the necessary competencies, significantly improve operational metrics, and
streamline processes by leveraging best practices.”
73

Summary
Operations leaders were assumed to understand how their job description includes the
underlying principles that are relevant to developing leading indicators. Results showed that the
assumed influence is an asset. The operations leaders provided examples of how developing
leading indicators is expected as part of their roles. The internal job descriptions also reinforced
this overall theme with references to underlying principles regarding leading indicators. Finally,
the survey feedback also indicated that there was a predominant overall awareness of developing
leading indicators as a core job expectation. As a result, the declarative knowledge influence is
determined to be a strong asset.
Knowledge Influence 2: Operations Leaders Did Not Possess the Core Skills and
Procedures Involved With Establishing Effective Leading Indicators
Survey Results
Overall, the survey data indicated that the operational leaders have a need in
understanding conceptually how to develop and implement effective leading indicators. As
indicated in Figure 6, 40% of the participants strongly agreed that they understood while 20%
moderately agreed. However, 40% of the survey population were either neutral, moderately
disagreed, or strongly disagreed. As a result, only 60% of the population agreed, which was less
than the established threshold of 68% that would indicate statistical significance as a potential
asset. Therefore, conceptual knowledge is a need. In addition, from a cut score perspective, the
60% of the sample population that either moderately demonstrated or strongly demonstrated an
understanding of conceptual knowledge of how to develop leading indicators was also short of
the established 85% threshold that would indicate an asset. This finding reinforced conceptual
knowledge of operational leaders as a potential need. The initial survey protocol established
74

included additional questions regarding conceptual knowledge. The intent of these additional
questions was for the operations leaders to demonstrate their level of conceptual knowledge by
the development process for leading indicators in order. However, due to organizational
limitations in the internal organizational information system used to administer the data
collection, these additional questions were not included in the actual survey. Net, the survey
findings determined that conceptual knowledge was a need for the operations leaders.

Figure 6
Conceptual Knowledge Survey Results(Question 11)

75

Interview Findings
The operations leaders interviewed provided evidence of a neutral to moderate level of
awareness that they understood the core skills and procedures involved with establishing
effective leading indicators, indicating a need. An additional finding was that 33% of
interviewees (n = 3) demonstrated a strong overall awareness for this expectation, which was less
than both the 50% and 75% threshold. The participants exhibited some level of knowledge of the
skills required to accomplish with emphasis on the traits, such as developing impactful measures
without strong assertions. Respondent 1 mentioned that leading operational indicators might be
something that points to a direction that you are going and measures you want to pay attention to,
indicating a need for a course correction versus a measure that looks backward and provides
insights you can learn from.
Respondent 2 also demonstrated a lesser and more general understanding conceptually in
stating that leading indicators show you what will happen in the future based on certain data that
is available now. Respondent 4 provided a similar assertion at a macro level in commenting that
we need to have a measurement in every major area so that we can control the key factors that
allow us to reach our target. Respondent 7 indicated a general awareness of the concept where
you start by identifying those indicators that are actually important and predictive versus those
that are categorized as just more data.
The participants also demonstrated understanding of the procedures, including defining
those key priorities that prevent reacting and reinforcing the importance of the process to pre-
identify the necessary measures that are important and predictive of delivering the lagging
measures critical to the business. Overall, there was an indication that the participants possessed
76

a lower level of the core skills and procedures required, representing a need for the operational
leaders.
Document Analysis
There were no documents created by the participants used in this study.
Summary
Operations leaders were assumed to have a base understanding of the core skills and
procedures involved with developing effective leading indicators. The results showed the
assumed influence was a need. The operations leaders exhibited a lesser level of knowledge of
the skills and procedures required to develop leading indicators. And finally, the survey feedback
also indicated a weak overall awareness of the skills and procedures required to develop effective
leading indicators. As a result, the conceptual knowledge influence is determined to be a solid
need.
Knowledge Influence 3: Operations Leaders Demonstrated the Ability to Reflect On and
Adjust the Necessary Skills and Plans to Implement Effective Leading Indicators
Survey Results
Overall, the survey data indicated that the operational leaders had a strong understanding
of the need to reflect and adjust their plans and corresponding skills to implement effective
leading indicators. As indicated in Figure 7, 47% of the participants strongly agreed (with a 4.3
average on a 1–5 scale) that the core expectation was to have the willingness to adjust based on
the results. In addition, the respondents’ standard deviation placed all of the data at or greater
than the average, which is statistically significant (greater than 68% of the population above
average). From a cut score perspective, 87% of the sample population either demonstrated or
strongly demonstrated the ability to reflect on and adjust their skills and respective plans to
77

implement effective leading indicators, which was greater than the established 85% threshold,
indicating a potential asset.

Figure 7
Metacognitive Knowledge Survey Results (Question 12)

78

Interview Findings
All interviewees (n = 9) demonstrated an overall awareness and ability to understand and
demonstrate this expectation, which was greater than both the 50% and the 75% threshold
established previously as validation for the influences and indicates strong evidence. The
interviewees provided evidence of metacognitive knowledge, including the ability to reflect on
and adjust necessary skills and plans to develop effective leading indicators. Respondent 1
indicated operations leaders needed to really pay more attention down to the operator level and
find the best-defined skill and performance match to do the job. Respondent 2 indicated the
importance of defining a key process indicator “you can measure, make sure you are going in the
right direction, adjust your approach, make it very simple, and ensure it is understood by
everybody.” Respondent 3 stated the need for operations leaders to prove over time that the
measures keep everything under control: “focus on people over the process with the necessary
discipline, and have a rational plan to correct where needed.” Respondent 7 indicated awareness
with the assertion that
You need to have a mindset that you have to have good leading indicators that keep in
control of your business. You know where it is headed, and you see when it is time to
increase measurement in one area and decrease in another and, as a result, never get out
of control.
Net, the participants overall mentioned identifying difficult paths, pursuing multiple
ways, assessing at a higher frequency, organizing data, and subsequently communicating to
others. As a result of these actions, the respondents mentioned performance increases such as
improved quality and world-class efficiency. Overall, there was a clear indication that the
participants understood the core skills and procedures required.
79

Document Analysis
There were no internal documents provided by the participants regarding metacognition.
Summary
Operations leaders were assumed to have the ability to reflect on and adjust the necessary
skills and plans to implement effective leading indicators. As a result of the data collected via
surveys and interviews, the assumed influence stated above is deemed an asset. The operations
leaders demonstrated the capability to make the required adjustments to implement leading
indicators across several examples throughout the interviews. The surveys also provided a strong
indicator of the level of ability present. The internal documents were also able to reference the
need to possess the ability to behave in the desired manner to adjust as well. As a result, the
metacognitive knowledge influence is determined to be a strong asset.
Results and Findings of Motivation Needs
This study also assumed that motivational influences originating from task value theory
(Cole et al., 2008) and goal orientation (Pintrich, 2000) impact operations leaders’ ability to
develop and implement effective leading indicators. A core theory of this study is that the
operations leaders’ task value motivation and the strong desire to both develop and demonstrate
competency would have an impact on their ability to develop and implement effective leading
indicators. Based on the research of this study, the level of task value and goal orientation in the
operations leaders determined whether the assumed influences stated previously were an asset or
a gap. The higher the level, the more the assumed influence is an asset. The lower the level of
task value and goal orientation, the more the assumed influence is a liability and gap that needs
to be addressed. The following sections present the findings of the research.
80

Motivation Influence 1: Operations Leaders Possess a High Task Value of Developing and
Implementing Effective Leading Indicators
Survey Results
Overall, the survey data indicated that the operational leaders possessed a high task value
of developing and implementing effective leading indicators. As indicated in Figures 8 through
13, greater than 88% of the participants strongly agreed (with an aggregate 4.48 average on a 1–5
scale) that there was a high level of importance in implementing leading indicators across several
operational performance categories, including financial, inventory, and quality performance. In
addition, the respondents’ standard deviation placed all of the data at or greater than 3.75, which
is greater than an average level of task value (less than 68% of the population above average).
From a cut score perspective and across several questions focused on operations leaders’ task
value as indicated in Figures 8 through 13, all participants either demonstrated or strongly
demonstrated a resolute possession of high task value in developing and implementing effective
leading indicators.

81

Figure 8
Operations Leader Task Value Survey Results (Question 1)

82

Figure 9
Operations Leader Task Value Survey Results (Question 13)

83

Figure 10
Operations Leader Task Value Survey Results (Question 14)

84

Figure 11
Operations Leader Task Value Survey Results (Question 15)

85

Figure 12
Operations Leader Task Value Survey Results (Question 16)

86

Figure 13
Operations Leader Task Value Survey Results (Question 17)

Interview Findings
All interviewees (n = 9) demonstrated a high task value, which was greater than both the
50% and the 75% threshold established previously as validation for the influences and indicating
strong evidence. The operations leaders provided evidence of a high level of task value in
developing and implementing leading indicators. Some of the understood benefits shared by the
participants by completing the tasks related to leading indicator development included the ability
to drive synergy, take a balanced and strategic approach, and predict future performance.
Respondent 1 indicated that leading indicators allow one to be proactive and prioritize things that
will help improve in the future. Respondent 2 shared that as the operations leader it is critical to
87

have balance and work on the strategic topics that are important for the future: “If one does not
have leading indicators, it is something that will be missing for ensuring future success.”
Respondent 9 indicated that having that leading indicator allows one to be confident that the
operation is on track to deliver the expected level of performance.
Document Analysis
There were no internal documents provided by the participants regarding motivation.
Summary
Operations leaders were assumed to have a high task value of developing and
implementing effective leading indicators. As a result of the data collected via surveys,
interviews, and document analysis, the assumed influence stated above is deemed an asset. The
operations leaders demonstrated a high level of task value via the survey results by reinforcing
the importance with their responses. The interviews reinforced the influence with recurring
mention by the participants on the benefits brought to the organization by completing the tasks
that lead to developing leading indicators. As a result, the task value motivational influence of
the operations leaders is determined to be a solid asset.
Motivation Influence 2: Operations Leaders Demonstrated a Need for Goal Orientation for
Developing and Implementing Effective Leading Indicators
Survey Results
While the survey data indicated that the operational leaders possessed a high level of goal
orientation for developing and implementing effective leading indicators for some of the
characteristics, there was lower goal orientation relative to demonstrating competency in leading
indicators from a learning perspective. While Figures 14 through 16 established a solid level of
motivation in the study population, Figures 17 and 18 determined gaps in motivation were
88

present. As highlighted in Figure 18, the participants scored lower (with a 3.0 average, standard
deviation of 1.2 on a 1–5 scale) on the question related to demonstration of the competency. The
wide range of responses, in addition to the standard deviation, indicates an opportunity in
motivation for operations leaders with potentially below-average goal orientation and is
statistically significant (greater than 68% of population in this range). From a cut score
perspective and as detailed in Figures 17 and 18, 76% and 32% of the sample population either
demonstrated or strongly demonstrated a high level of goal orientation to develop effective
leading indicators. This result was short of the 85% threshold that indicates a strong presence,
indicating a need ranging from moderate to strong for operations leaders’ motivation.

Figure 14
Operations Leader Goal Orientation Survey Results (Question 2)

89

Figure 15
Operations Leader Goal Orientation Survey Results (Question 3)

90

Figure 16
Operations Leader Goal Orientation Survey Results (Question 4)

91

Figure 17
Operations Leader Goal Orientation Survey Results (Question 5)

92

Figure 18
Operations Leader Goal Orientation Survey Results (Question 6)

Interview Findings
Most (78%) interviewees (n = 7) indicated there was a weak level of motivational focus
relative to goal orientation, which was greater than both the 50% and the 75% threshold
established previously as validation for the influences and indicates strong evidence. Vandewalle
(1997) highlighted a high level of performance goal orientation correlating with a high degree at
which an individual or group works toward completing specific goals or objectives, whereas
mastery goal orientation is more centered around an individual increasing their level of
competence The participants similarly expressed lower levels of mastery goal orientation to learn
in developing leading indicators. Respondent 1 described the behavior among colleagues'
regarding their level of motivation where if they don't see indicators improving the ability to
93

achieve their goals, they tend to behave in a more transactional manner and do not pursue
obtaining further mastery. Respondent 7 indicated some of the gaps in performance goal
orientation where getting individuals to understand what their responsibility is and driving
towards a goal is challenging. The respondent mentioned that many of the leaders do not
complete the circle of connecting the work to goals, resulting in less motivation overall. From a
leading indicator utilization perspective, the respondent also indicated the deployment of leading
indicators was minimal, as the performance goal orientation focus of the operations leaders is the
status of a large number of lagging measures in regular monthly meetings. Respondent 9
indicated that the level of mastery goal orientation regarding leading indicators was minimal as
the focus of leaders had been to drive measures that are more lagging or have happened in the
past.
To pull it all together, a synthesis of the participants’ comments indicated an
unwillingness by the operations leaders to demonstrate individual capability regarding leading
indicator development and implementation. Overall, there was a clear indication from the
participants that a deficiency in goal orientation of the operations leaders in developing leading
operational indicators was present.
Document Analysis
A review of the organization’s intranet to assess the presence of internal documents or
communication relative to ensuring a high level of goal orientation showed some forms of praise
for certain categories, although not aligned with leading indicators development. Some
recognition was discovered, including CEO-level recognition for personnel nominated for going
above and beyond in alignment with company values, specific recognition for a senior-level
executive for leading corporate sustainability initiatives, and wide functional recognition
94

garnered for the organization from external resources for furthering the diversity and inclusion
efforts. While there is a presence of recognition in the organization shared broadly, there is no
specific reference to establishing goals regarding the development of leading operational
indicators to drive performance improvements.
Operations leaders were assumed to have a high goal orientation for developing and
implementing effective leading indicators. As the result of a triangulation of the data collected
via surveys, interviews, and document analysis, the assumed influence stated above is deemed a
gap or strong need. Both the survey results and specific interview comments showed a lower
goal orientation relative to demonstrating competency in leading indicators. Overall, the
participants scored lower collectively in demonstrating the competency in the organization
regarding leading indicators, and via the interviews, there was a low orientation to learn and gain
the competency from the majority of the participants. Finally, the internal organizational
documentation, while making a concerted effort to recognize internal achievements, did not
place a specific focus on promoting goal orientation as it pertains to developing and
implementing effective leading indicators. As a result, the goal orientation of the operations
leaders is determined to be a strong need.
Results and Findings of Organizational Needs
This research also assumed that organizational influences originating from cultural
models and cultural settings impact operations leaders’ ability to develop and implement
effective leading indicators. A core theory of this study is that organizational assumed influences
related to both cultural models and cultural settings contribute to the problem of practice and are
defined as the resources, policies, procedures, and characteristics of the environment that support
the operations leader in accomplishing the above. Cultural models are the beliefs, values, and
95

attitudes and cultural settings are the visible, concrete manifestations of the cultural models that
appear in organizational settings (Gallimore & Goldenberg, 2001). Based on the research of this
study, the operations leaders’ cultural models and settings would determine whether the assumed
influences will be either an asset or a gap. The more the models and settings facilitate an
environment for leading indicator development, the more the assumed influences will be an
asset. The less support the settings and models provide, the more the assumed influences will be
a need. The following sections present the results.
Organizational Influence 1: The Organization Does Not Have a Strong Accountability
Binary Influence on the Operations Leaders in Developing Leading Indicators (Cultural
Models)
Survey Results
The accountability binary is defined as the contractual relationship between the director,
who has the ability to reward, replace, or punish, and the provider of a good or service
(Hentschke & Wohlstetter, 2014). The strength of the accountability relationship is based on the
relationship across the three dimensions of values, decision rights, and information. For
information, communication is a key element in ensuring a healthy two-way exchange of
information is present between the director and provider. Gallimore and Goldenberg (2001)
describe the cultural model as the shared mental schema or cultural practices that are often
invisible and unnoticed (Rueda, 2011). So, from a cultural model perspective, what would be
expected is strong evidence of a firm relationship between operations leaders and either the
providers or directors in developing leading indicators via the existing practices in the
organization and the presence of a strong accountability binary.
96

From the operations leaders’ perspective, the survey data indicated that the organization’s
existing cultural model around communication and accountability resulted in a gap regarding the
operations leaders’ development and implementation of leading indicators. Research from
(Wößmann, 2007) indicated that an environment where more autonomy is provided regarding
accountability resulted in better outcomes. As in Figures 19 and 20, the participants scored lower
(with a 3.5 average, standard deviation of 1.2 on a 1–5 scale) on the questions related to whether
the organization provides an environment with high levels of accountability and communication.
In addition to the standard deviation, the wide range of the responses indicates an opportunity in
the organization. The specificity of the questions asked could also be improved for further
inquiry in the future. The survey results indicated an accountability binary environment for the
operations leaders with a potentially below-average cultural model and is statistically significant
(greater than 68% of the population in this range). From a cut score perspective and for the
questions pertaining to cultural models, results ranging from 44% to 63% of the sample
population either believed or strongly believed there was a strong accountability binary and
communication influence in developing leading indicators. The cut score results were less than
the established threshold of 85%, indicating this influence was a moderate to strong need for the
operations leaders.

97

Figure 19
Organizational Cultural Models: Level of Accountability of Leadership (Question 7)

98

Figure 20
Organizational Cultural Models: Level of Communication in Facilitating Accountability
(Question 8)

Interview Findings
Most (66%) interviewees (n = 6) indicated that the overall environment presented a
challenge for operations leaders regarding the effectiveness of the accountability binary
influence in the corporation. This finding was greater than the 50% threshold and less than the
75% threshold established previously, which indicated a moderate level of affirmation that the
gap existed. Hentschke and Wohlstetter (2004) highlighted that the accountability binary’s
effectiveness is based on the strength of the relationship between the director and the provider.
Regarding communication between accountable parties in an organization, Berger (2014)
99

determined that its presence is critical to improving performance. Respondent 1 mentioned that
the lack of communication in the organization has resulted in the lack of connectedness of
employees to the main business objectives, silos, and more transactional relationships.
Respondent 4 indicated that the effectiveness of the accountability binary is not a one-
person problem but rather a team problem and those that are not in the right direction need to get
in the same direction. Respondent 5 mentioned that the lack of accountability and
communication has resulted in old school thinking, not learning from lessons of the past, and
inhibiting the level of creativity present. Regarding leading indicator utilization, there were
indications of minimal leading indicators deployed, where commentary was provided that they
were used sparingly and when used they were utilized too late. Respondent 6 indicated that their
lack of communication and accountability has resulted in a hesitancy to change, mis-alignment
when establishing goals, divisiveness, and lack of execution. Respondent 7 highlighted
additional challenges in binary relationships with examples where the employees do not know
what to do or what not to do and do not know what to track. This issue has resulted in a minimal
level of acknowledgement or focus on accountability in developing leading indicators.
Respondent 9 indicated that there are a number of opportunities to improve, with the
organization going through a tremendous amount of change, and an excess of different initiatives
since 2014. There is a recognition that the organization needs to align and roll out a robust
strategy because frequently the organization adds measurement points and as a result sometimes
loses the merit of each point because of the excessive measurement present. In terms of leading
indicator utilization, the respondent indicated that leading indicators were a new concept, and
when used they drove actions only after the completion of an extended observational period of
100

multiple months at a minimum. This indicated a more passive usage of leading indicator decision
making.
Net, there was a clear indication from the participants that the cultural model of
accountability and communication in the organization was an area of need in support of leading
indicator development for the operations leaders.
Document Analysis
A review of the organization’s internal documentation and communication showed a
strong level of rigor around the financial accountability of manufacturing site performance.
Reinforcing this fact were regular financial performance reviews requiring wide participation
from the operations organization. Outside of the financial results, there was a void of consistent
documentation and internal communications relative to operational metrics and leading indicator
development. There is a renewed focus on creating and reinforcing core cultural methods of
working on a daily basis. However, within those methods, there is not a focus on leading
indicator development and execution.
Summary
The organization was assumed to have a strong binary influence on the operations leaders
in the development of leading indicators. As a result of the data collected via surveys, interviews,
and document analysis, the assumed influence stated above is deemed a solid gap. Survey results
from the operations leaders indicated deficiencies in accountability and communication from a
cultural model standpoint. The interview feedback from the participants provided an aligned
affirmation that overall gaps in communication and accountability exist. Also, PMC’s internal
documentation and communications were absent of references to the development of operations
101

leading indicators. As a result, the cultural model of a solid accountability binary influence on
the operations leaders in developing leading indicators is determined to be a need.
Organizational Influence 2: The Organization Does Not Have a Clearly Negotiated
Accountability Binary for Operations Leaders’ Leading Indicator Decision-Making
(Cultural Settings)
Survey Results
The accountability binary and the strength of the accountability relationship was detailed
previously. Gallimore and Goldenberg (2001) describe the cultural setting as the concrete
version of the social context (Rueda, 2011). So, from a cultural setting perspective, what would
be expected is strong visible evidence of a firm relationship between operations leaders and
either the providers or directors in making decisions relative to developing and implementing
leading indicators. The survey data indicated that the operational leaders had a higher level of
autonomy for accountability for developing and implementing effective leading indicators from a
cultural setting perspective. As indicated in Figure 21, the participants scored overall higher
(with a 4.1 average, standard deviation of .8 on a 1–5 scale) on the question related to whether
the operations leaders have the level of autonomy needed to develop effective leading indicators.
However, the wide range of the response from the participants, in addition to the standard
deviation, indicates an opportunity from a cultural setting perspective for the operations leaders,
with a portion of the survey responses at or below the average level of autonomy required to
develop leading indicators. As illustrated in Figure 21, 31% of the survey population did not
agree that they had the autonomy needed to develop effective leading indicators. The span of the
data collected within the standard deviation is statistically significant as well (greater than 68%
of the population in this range). From a cut score perspective, 69% of the sample population
102

either affirmed or strongly affirmed that the organization has a clearly negotiated accountability
binary for operations leaders’ leading indicator decision-making. As a result of the cut score
below the established threshold of 85%, this influence is identified as a solid need.

Figure 21
Organizational Cultural Settings: Autonomy in Developing Leading Indicators (Question 9)

103

Interview Findings
Most (85%) interviewees (n = 8), greater than both the 50% and the 75% threshold,
indicated that the overall cultural setting was a gap in reinforcing the accountability binary for
operations leaders’ leading indicator decision making. The interviewees suggested that a weak
level of clearly negotiated accountability binary was present as a cultural setting, although the
overall input provides a mixed result that required further exploration. Respondent 2 indicated
there is not much guidance or accountability regarding leading indicators from a global
perspective, nor is there an expectation for standardization of best practices such as leading
indicators. Respondent 4 highlighted that within the organization there is not a clear expectation
to measure leading indicators that we measure too many lagging indicators, where some of them
are duplicated.
Respondent 6 highlighted differences across functions where accountability is not
reinforced with shared and specific measures, and there is lack of accountability to change
existing processes for the better. Respondent 7 stated that the organization has roles and
responsibilities that are not clear in developing leading indicators, there is a limited bandwidth
due to the lack of support provided by leadership, and that employees are criticized and chastised
when lagging measures miss their mark. Respondent 8 indicated the organization looks more at
lagging indicators more versus leading indicators and are more at an infancy stage,
accountability is not reinforced visibly with clear goals and expectations regarding leading
indicators, and the currently siloed organizational structure reflects this. Respondent 9 indicated
the overall environment around accountability has the visual characteristics of the measurement
of many things, the tendency for leadership to add more points for measurement of lagging
indicators, and the need for a clear strategy that will feature leading indicators as a strategy that
104

will ensure success in forward years. Improving the cultural setting overall for leading indicator
decision making is an opportunity for the organization.
While the participants admitted there was some level of accountability binary present,
overall, there were several suggestions to the contrary that solid gaps are present from a cultural
setting perspective.
Document Analysis
A review of the organization’s intranet to assess the presence of internal documents or
communication relative to the accountability binary for the operations leaders revealed an
internal focus on improving the organization’s overall culture with a campaign focused on
improving the method by which employees work daily. The culture’s focus is regarding the
consistency of fulfilling the commitments made versus reinforcing accountability in the
organization. Upon further review of internal documentation, there is very little public criticism
with the sharing of insights and lessons learned in public forums more focused on the individual
learnings of the participants. The internal communications’ emphasis on the declared state of
constant change in the organization reflects the need to constantly be in a mode of learning and
adaptation.
Summary
The core assumption regarding the organization is that PMC has a clearly negotiated
accountability binary for operations leaders’ leading indicator decision making from a cultural
settings perspective. As a result of data analysis, the assumed influence presents a potential gap
in developing leading indicators. The comments from the interview participants pointed to
several concerns with the current cultural setting, hindering operations leaders’ ability to develop
and implement leading indicators. Overall, the participants scored a wider range collectively in
105

describing the current settings from a survey perspective, highlighting the opportunity. Finally,
internal organizational documentation, while emphasizing the importance of current strategic
global initiatives, did focus specifically on reinforcing accountability in support of developing
leading indicators. As a result, the cultural setting for accountability to influence operational
leaders decision making for leading indicator development is deemed a solid need.
Summary
Chapter Four focused on the overall findings from the mixed methods research conducted
relative to the overall study purpose to determine the performance gaps influencing the
operations leaders’ ability to develop and implement effective leading operational indicators. The
core intent of this chapter was to answer the initial three study questions by assessing the
knowledge, motivational, and organizational influences via research to determine whether they
were deemed assets or needs in developing leading indicators. To appropriately assess the
findings and subsequently affirm assertions regarding the influences, parameters were
established. For the surveys, 68% was the parameter that confirmed agreement with the
assertion, while findings at 85% or more were considered strong evidence that the assertion was
valid. For document analysis, the presence of assertions was determined to be evidence of
agreement. And regarding the interviews conducted for this study, 50% presence of evidence in
alignment with the a priori codes alignment with the research questions established a core
assertion, while 70% or greater was an indicator of a much stronger assertion. Table 7
summarizes the findings regarding the seven influences of focus in this study.

106

Table 7
Summary of Findings, KMO Assumed Influences, Validated Assets, and Validated Needs
KMO influences Assets Needs
Declarative knowledge
Operations leaders’
understanding of the job
description related to
underlying principles to
develop leading indicators.
4.3 average on 1–5 scale
survey result
88% of sample survey
population affirmed, strong
evidence
100% of those interviewed
demonstrated, strong
evidence as an asset

Conceptual knowledge
Operations leaders understand
the core skills and procedures
involved with establishing
effective leading indicators

40% of survey population did
not demonstrate conceptual
knowledge (need)
Only 60% of those
interviewed had an
understanding of
conceptual knowledge
(need)
Metacognitive knowledge
Operations leaders have the
ability to reflect on and
adjust the necessary skills
and plans to implement
effective leading indicators

4.3 average on 1–5 scale
survey result
87% of sample survey
population affirm, strong
evidence
100% of those interviewed
demonstrated, strong
evidence

Motivation, task value
Operations leaders possess a
high task value of
developing and
implementing effective
leading indicators
4.48 average on 1–5 scale
survey result
100% of population
demonstrated via survey,
strong evidence as asset

107

KMO influences Assets Needs
100% of those interviewed
demonstrated, strong
evidence as asset
Motivation, goal orientation
Operations leaders possess a
high level of goal
orientation for developing
and implementing effective
leading indicators
3.0 average on 1–5 scale
survey result - gap
76% and 32% of sample
population demonstrated
via survey question
responses goal orientation,
moderate to strong
evidence of need
78% of those interviewed
indicated a weak level of
goal orientation, moderate
evidence of need
Organization, cultural model
Organization needs to have a
strong accountability binary
influence on the operations
leaders in developing
leading indicators

3.5 average on 1–5 scale, gap
44% and 63% of sample
population believed via
survey question responses a
strong binary present,
moderate to strong
evidence of need
66% of those interviewed
indicated weakness in
organizational influence,
moderate evidence of need
Organization, cultural settings
Organization has a clearly
negotiated accountability
binary for operations
leaders’ leading indicator
decision making
4.1 average on 1–5 scale
survey result
69% of sample population
affirm binary present -
evidence of moderate need
85% of those interviewed
indicated weak level of
accountability binary,
strong evidence of need

108

The findings summarized here provide the foundation for the recommendations to
address the gaps detailed in Chapter Five. Overall, the evidence from the research indicates that
the operations leaders possessed the assets of declarative, conceptual, and metacognitive
knowledge as well as motivational task value. Conversely, the key opportunities identified as
needs were the operations leaders’ motivational goal orientation as well as the organizational
cultural models and cultural settings related to developing and implementing effective leading
indicators. The conceptual framework, as a soft reform space, focused on gaining the input of the
population (de Oliveira Andreotti et al., 2015) while being influenced by the environment, which
guided the inquiry. These influences included the cultural modes and settings that fell within the
soft reform space and were identified as opportunities for improvement, demonstrating the
characteristics of the conceptual framework. Net, the overall approach, direction, and findings of
the research demonstrated a strong correlation with the conceptual framework.

109

Chapter Five: Recommendations and Implementation and Evaluation Plan
Chapter Four focused on sharing the findings of the inquiry performed in this study. The
summation of the findings included an analysis of the knowledge, motivational, and
organizational influences of the operational leaders in developing effective leading indicators
and determining whether the influences were assets or needs. Utilizing quantitative and
qualitative data analysis as well as cut-score heuristics, the influences were either validated as
impactful to the operational leaders in developing and implementing effective leading indicators
or not validated.
Chapter Five centers on answering the final research question. The focus will center on
leveraging the research findings to answer Question 4 in developing evidence-based
recommendations that align to and address the validated knowledge, motivational, and
organizational influences. In addition, this chapter will detail the implementation of these
recommendations and how to evaluate the implementation for effectiveness. The
recommendations utilize the new world Kirkpatrick model (Kirkpatrick & Kirkpatrick, 2016) as
a framework for establishing the core objective or mission, determining the appropriate strategy,
and identifying the measures to determine whether the desired progress is made. The new world
model has similarities to the traditional four levels of evaluation; however, it works in reverse in
focusing on the desired outcomes initially and the behaviors required to achieve these results.
Subsequently, this model will then explore the learning progress of the intended audience and
their corresponding reactions. This chapter will start with an initial framing of the organizational
context and mission, organization performance goal, description of stakeholder groups, goal of
the stakeholder group, and purpose of the project and questions before moving into an overview
of the recommendations.
110

Organizational Context and Mission
For more than 50 years, PMC has designed, built, and delivered products in partnership
with some of the world’s most innovative companies. The purpose of PMC is for its products to
create market-leading value and improve customers’ and patients’ lives. The mission of PMC is
to create an intelligent and more connected world by designing and manufacturing products for a
world that is increasingly connected and continues to grow at an accelerating rate. In executing
its mission, the organization emphasizes the importance of living the values by challenging the
status quo, moving at a high rate of speed, executing in a disciplined manner with an overarching
purpose, and performing with the highest level of integrity at all times. The values define how
the employees act, behave and conduct daily interactions with fellow employees, customers, and
partners daily. The values help define the organization’s culture. The organization’s vision is to
become the world’s most trusted global technology, supply chain, and manufacturing solutions
partner.
Organizational Performance Goal
The organization aims to meet 100% of its established lagging operational performance
indicators by 2022. The focus of this dissertation was to achieve the newly developed leading
indicators by 2022. Senior executives determined this goal based on environmental scanning and
quantitative analysis. For example, while the operations function has five core lagging
operational metrics, there are 143 total KOIs at each operating site. The core lagging operational
metrics represent 3.5% of the total lagging measures tracked. The performance gap is 100% of
leading indicators as they currently do not exist.
This goal is based on the 80/20 principle, which reinforces the finding that 80% of
performance results from 20% of the time one spends on the critical few focus areas, as Koch
111

(2000) explained. In addition, the concept of leading indicators establishes the standard that for
lagging measures to be achieved, the leading measures must be in place to provide the necessary
leverage (Hammonds, 2003; McChesney et al., 2012). The PMC organization has an overall
expectation of exceeding customer expectations in the areas of program startups, customer
satisfaction, on-time delivery, quality performance, and cost results yet do not have leading
indicators to ensure delivery of the results proactively to take advantage of leverage.
Description of Stakeholder Groups
The participants in the study consisted of three representative layers of the organization:
senior executives (five senior VPs and presidents), regional leaders (five VPs), and 10
operational leaders (site general managers and program directors), with a focus on the
operational leaders. The operational leaders manage the overall production facilities and lead
sites’ executional programs. They are responsible for delivering bottom-line results. The regional
leaders are more horizontally focused, covering all their designated production sites. They are
responsible for policy development of and removing obstacles to performance improvement
relative to internal targets. The senior executives are accountable for the legislation, measures
implementation, and follow-through to ensure clear accountability and delivery of the lagging
operational performance measures.
Goal of the Stakeholder Group for the Study
The organization aims to meet 100% of its established lagging operational performance
indicators by 2022. The focus is for PMC to achieve the newly developed leading indicators by
2022.

112

Stakeholder Group for the Study
The operational leaders are expected to develop the key leading indicators that align with
the lagging indicators. Joint efforts of all stakeholders contribute to achieving the organizational
goal of developing all leading indicators by 2022. However, this study focused on the operational
leaders as they are responsible for the bottom-line execution at the manufacturing sites and are
closest to the leading activities that provide the leverage to develop the leading indicators. The
stakeholder’s goal, supported by the regional and senior leaders, is to develop leading indicators
aligned with organizational goals by December 2021. A review of organizational data revealed
that there currently is a high number of lagging measures at the operational level, with 143
counted with the latest tally with zero leading indicators currently in place. The organizational
goal is to meet 100% of the newly developed leading indicators by 2022. The performance gap is
100%.
Purpose of the Project and Questions
1. What are the operational leaders’ knowledge and motivation needs related to their
development of leading operations indicators in alignment with organizational goals?
2. What is the interaction between organizational culture and context and operational
leaders’ knowledge and motivation to develop leading operations indicators in
alignment with organizational goals?
3. To what level do operational leaders currently utilize leading indicators?
4. What are recommended knowledge, motivation, and organizational solutions?
The results presented in Chapter Four addressed the first three research questions. The
remaining question is centered on providing the recommended KMO solutions for PMC
determined by the validated influences and needs detailed in Chapter Four The structure of
113

Chapter Five will consist of the following: an introduction to the recommendations categorized
under KMO, a detailed description of how the recommendations will be implemented as a
program, a description of the program, and finally how the implementation of the program will
be evaluated using the Kirkpatrick new world (Kirkpatrick & Kirkpatrick, 2016) model of
evaluation with a brief description of the four levels. Traditionally, the Kirkpatrick model
reflected the consideration of four levels of learning in consecutive order of program
implementation to include the reactions, learning, behavior, and results of the targeted audience.
However, the environment expressed a need to evolve the model to allow leaders to
operationalize it effectively, resulting in a revised model. The new world (Kirkpatrick &
Kirkpatrick, 2016) model conversely features an approach that focuses on the desired results of
the program implementation followed by the behavior, learning, and reaction of the targeted
audience.
Chapter Five will progress through the plan development process with initial emphasis on
the overall results via establishing the desired outcomes and leading indicators that define the
degree to which the results reflect the training administered to the operations leaders. Second, the
focus of plan development will be on the critical behaviors of the operations leaders that will
have the most significant impact in developing effective leading indicators. Key elements of the
behavior level in the planning process include required key drivers, such as processes and
systems that reinforce, monitor, encourage, and reward performance of critical behaviors and on-
the-job learning. Research by Brinkerhoff (2006) determined that when organizations rely on
training alone to improve job performance, they are successful only 15% of the time. The third
level of the plan, learning, will focus on enhancing the degree to which the operations leaders
acquire the knowledge, skills, attitudes, confidence, and commitment to develop and implement
114

effective leading indicators. The final level of the plan, reaction, details the approach to
increasing the degree of the response of the operational leaders in the dimensions of customer
satisfaction, engagement, and relevance to the overall organizational performance goals. For the
levels of the model utilized, there will be a dual emphasis of implementation and evaluation to
ensure the execution of the plan is meeting the desired outcomes at each stage of the plan. The
establishment of key process indicators will act as a feedback loop for all stakeholders identified
in the study in alignment with the new world model (2016).
Recommendations for Practice to Address KMO Influences
Knowledge Recommendations
The assumed knowledge influences of declarative, conceptual, and metacognitive were
explored as a focal point of the study. The operational leaders demonstrated declarative and
metacognitive strengths as influences. Conceptual knowledge gaps in developing and
implementing effective leading operational indicators were validated through the data analysis.
By providing information, training, and education to the operational leaders, it is expected that
their conceptual knowledge would be enhanced and close the existing validated gap. Table 8
presents the summary of knowledge influences and recommendations to address the problem of
practice. The theoretical principles provided with corresponding references support the
recommendations.

115

Table 8
Summary of Knowledge Influences and Recommendations
Assumed
knowledge
influence
Validated as a
gap?

Priority? Principle and
citation
Context-specific
recommendation
Operations
leaders’
understanding
of the job
description
related to
underlying
principles to
develop
leading
indicators. (D)

No No None None
Operations
leaders need to
understand the
core skills and
procedures
involved with
establishing
effective
leading
indicators (C)

V Y Conceptual
knowledge
is
developed
from
educational
experiences
(Clark &
Estes,
2008).

Training
people
effectively
requires
giving them
accurate
procedures,
practice,
and
corrective
feedback to
allow for
gradual
automation
Information:
provide guidance in
the form of a
pamphlet for the
operations
leaders to
identify top three
most impactful
measures for
their respective
operations

Job aids provide
operations
leaders with
conceptual
principles for
developing
leading indicators
that allows the
operational
leaders to
perform on their
own, followed by
recurring practice
116

Assumed
knowledge
influence
Validated as a
gap?

Priority? Principle and
citation
Context-specific
recommendation
of the
knowledge
(Clark &
Estes,
2008).

To develop
mastery,
individuals
must
acquire
component
skills,
practice
integrating
them, and
know when
to apply
what they
have
learned
(Schraw et
al., 2006).

on actual
operational
performance
issues with
regular reviews
to provide timely
feedback.

Job aids provide
recurring
reminders via
electronic
communications
to operations
leaders regarding
how to apply
what they learned
regarding the
development and
implementation
of effective
leading
indicators.
Operations
leaders have
the ability to
reflect on and
adjust the
necessary
skills and
plans to
implement
effective
leading
indicators (M)
No No None None

Note. V stands for validated, HP stand for high probability, and N stands for no.

117

Increasing Operational Leaders’ Conceptual Knowledge of the Core Skills and Procedures
Involved With Establishing Effective Leading Indicators
The results and findings of this study indicated that 40% of the operational leaders
required more in-depth conceptual knowledge of the core skills and procedures involved with
establishing effective leading indicators. With the utilization of the framework developed by
Krathwohl (2002) relative to what the operational leaders are expected to learn, strategic
conceptual knowledge objectives centered on the cognitive process dimension around enhancing
understanding, creation, and evaluation would be the focus of the learning. Clark and Estes
(2008) determined that educational experiences develop conceptual knowledge. Their research
also affirmed that training people effectively requires giving students accurate procedures,
practice, and corrective feedback to allow for gradual knowledge automation. Schraw et al.
(2006) found that to develop mastery, individuals must acquire component skills, practice
integrating them, and know when to apply what they have learned. These overall findings would
suggest that providing the operations leaders with information, training, and education in
developing and implementing effective leading indicators would support their learning.
The recommendation consists of three strategies. The first is to provide operations leaders
with information in the form of a pamphlet with conceptual guidance regarding how to identify
the top three most impactful measures for their respective operations. The second is to distribute
job aids with conceptual principles for developing leading indicators, therefore enabling
operations leaders to perform repeatedly on actual operational performance issues with regular
reviews to provide timely feedback. The third is to further leverage job aids as recurring
reminders via electronic internal communication that reinforces how to apply what they learned
regarding developing and implementing effective leading operational indicators.
118

Clark and Estes (2008) indicated that when learners either do not know how to do
something or have to address future problems that require novel problem solving, these two
conditions will require knowledge enhancement brought about by information, training, and
continuing education. Weaver et al. (2018) studied STEM students in an undergraduate setting to
determine methods to improve student engagement and learning outcomes. Research determined
that sharing information before instruction increased students’ conceptual knowledge by utilizing
exploratory learning to better connect concepts and procedures. Similarly, Dong et al. (2017)
studied the levels of leadership in teams and the correlation of information sharing with team
leadership, creativity, and innovation. Results were that information sharing increased
individually focused leadership, skill development, and individual creativity.
In the area of training, Koponen et al. (2009) explored whether training would enhance
childrens’ conceptual knowledge in performing calculations. Their research found that the
children’s conceptual knowledge around calculations improved due to training interventions.
Relative to education, Rogers (2021) studied the effect of education on students’ conceptual
knowledge in a university setting. Key findings were that through a generative educational
approach that focused on students developing their ability to teach the concepts they were
learning, the students’ conceptual knowledge subsequently increased. Zhang et al. (2012) also
utilized a study on students in a civic educational setting to understand the impact on conceptual
knowledge. Their research determined that the students’ conceptual knowledge increased was
directly correlated with the level of traditional classroom educational activities, including an
open discussion climate and exposure to concepts. Therefore, the evidence supports the
recommendation to provide information, training, and education to the operations leaders to
improve their conceptual knowledge in developing and implementing leading indicators.
119

Motivation Recommendations
The assumed motivation influences of task value and goal orientation were explored as a
focal point of the study. The operational leaders demonstrated task value as a strength. Gaps in
the goal orientation of the operations leaders in developing and implementing effective leading
operational indicators were validated via the data analysis. By using task, reward, and evaluation
structures, providing organizational and management structures, and creating a community of
learners, it is expected that the operations leaders’ goal orientation would be enhanced and close
the existing validated gap. Table 9 presents the summary of motivation influences and
recommendations to address the problem of practice. The theoretical principles provided with
corresponding references support the recommendations.

120

Table 9
Summary of Motivation Influences and Recommendations
Assumed
motivation
influence
Validated as a
gap?
Priority? Principle and
citation
Context-specific
recommendation
Operations
leaders
possess a high
task value of
developing
and
implementing
effective
leading
indicators
(TV)
No No None None
Operations
leaders need a
high level of
goal
orientation for
developing
and
implementing
effective
leading
indicators
(GO)

V Yes Focusing on
mastery,
individual
improvement,
learning, and
progress
promotes
positive
motivation
(Yough &
Anderman,
2006)

Designing
learning tasks
that are novel,
varied,
diverse,
interesting,
and
reasonably
challenging
promotes
mastery
orientation
(Yough &
Use task, reward,
and evaluation
structures that
promote
mastery,
learning,
effort,
progress, and
self-
improvement
standards and
less reliance on
social
comparison

Provide
organizational
and
management
structures that
encourage
personal and
social
responsibility
and provide a
safe,
121

Assumed
motivation
influence
Validated as a
gap?
Priority? Principle and
citation
Context-specific
recommendation
Anderman,
2006).

Goals motivate
and direct
students
(Pintrich,
2000).
comfortable,
and predictable
environment

Create a
community of
learners where
everyone
supports
everyone’s
attempts to
learn.

Note. V stands for validated, HP stand for high probability, and N stands for no.

Operations Leaders With a High Level of Goal Orientation of Developing and Implementing
Leading Indicators Will Effectively Execute Them
The results and findings of this study indicated that 78% of the operational leaders
demonstrated a weak level of goal orientation in developing and implementing effective leading
indicators. Clark and Estes (2008) determined an overall framework that motivation can be
increased by creating value to get buy-in, adjusting belief about tasks of the targeted audience,
and reducing the challenge perceived by the audience by focusing on the task. DeGeest and
Brown (2011) indicated that individuals that focus on main goal motivation elements of mastery
and performance positively influence expected outcomes. The recommendations developed were
based on the goal motivation framework centered around three core principles: focusing on
mastery, designing learning tasks, and the establishment of goals. Yough and Anderman (2006)
asserted that focusing on mastery, individual improvement, learning, and progress promotes
122

positive motivation. Their research also determined that designing learning tasks that are novel,
varied, diverse, interesting, and reasonably challenging promotes mastery orientation. And
Pintrich (2003) via study asserted that establishing goals provides direction and improves
motivation. These overall findings suggest that providing operations leaders with structures that
focus on mastery, the designing of learning tasks, and establishing of goals in developing and
implementing effective leading indicators would support increasing their overall motivation
regarding goal orientation.
Clark and Estes (2008) indicated that learners who have motivational gaps see increases
in active choice, persistence, and mental effort when the emphasis is placed on increasing
individual and team confidence, trust, collaboration, and learners’ perceived value about the
work to be performed. Malik et al. (2019) studied goal orientation in leader-subordinate
relationships to understand the effect of rewards to promote creativity and improved
performance. In a sample of 220 leader and subordinate relationships, research affirmed a
positive correlation between increased rewards and learner higher goal orientation. Rivers (2008)
explored the relationship between the task and evaluation structures and goal orientation. In a
sample of 148 high school students, research determined parenting style, including task and
reward structures at home, had a positive correlation with students’ goal orientation. Relative to
the relationship of task difficulty on goal orientation, Steele-Johnson et al. (2000) performed
research on a sample population. Their findings in a sample of approximately 200 students were
that task difficulty and consistency directly correlated with goal orientation.
For goals, Brett and VandeWalle (1999) studied goals and goal orientation as predictors
of performance in a student training program. In a sample of 252 participants in a complex skill-
training program, research determined that goal orientation was related to goals with specific
123

skill improvement focus and had a positive relationship with student performance. Hsieh et al.
(2007) performed research on college students to understand the effect of goal setting on goal
orientation. A key finding from the study indicated that self-efficacy and mastery goals
associated with goal orientation were positively related to students’ academic performance and
were reliable predictors of scholastic achievement. Wolters et al. (1996) explored the relationship
between students’ motivation and beliefs in a learning environment to better understand their
goal orientation. In a sample of 434 seventh and eighth-grade students, research determined that
an environment promoting a learning goal orientation correlated with a positive pattern of
motivational beliefs as well as improved levels of task value, self-efficacy, cognition, and overall
academic performance.
Lerang et al. (2019) researched the correlation of the classroom environment and
students’ goal orientation. In a sample of 1975 students from grades 8 through 10, the findings
showed that the quality of the classroom interaction was significantly associated with the level of
students’ goal orientation and academic achievement. Overall, the evidence from the research
supports the recommendation to use task, reward, and evaluation structures, provide
organizational and management structures, and encourage an environment of learners that
supports all involved. These approaches suggest that successful implementation will increase the
level of goal orientation of the operational leaders in developing and implementing effective
leading indicators.

124

Organization Recommendations
The assumed organization influences of cultural models and cultural settings were
explored as focal points of the study. Clark and Estes (2008) described organization influences as
the organizational work processes and material resources as well as the organizational culture
itself. Gallimore and Goldenberg (2001) described cultural models as the patterns of behaviors
that distinguish a culture and cultural settings as the environment that ties to the behavior
patterns. Hentschke and Wohlstetter (2004) highlighted the importance of the accountability
binary relationship in organizations for it to work effectively in theory and practice. Cleary et al.
(2013) determined that culture has a considerable impact on a group’s or organization’s level of
accountability and the corresponding strength of the accountability binary.
In this study, the assumed organization influences of cultural models and cultural settings
were validated as gaps. By identifying the existing accountability issues and challenges in the
organization, developing strategies to promote and reinforce accountability, and creating goals to
drive performance and create a learning organization, it is expected that the cultural models and
cultural settings would be enhanced and close the existing validated gap. Table 10 presents the
summary of organization influences and recommendations to address the problem of practice.
The theoretical principles provided with corresponding references support the recommendations.

125

Table 10
Summary of Organization Influences and Recommendations
Assumed
organization
influence
Validated as a
gap?
Priority? Principle and
citation
Context-specific
recommendation
Organization
needs to have
a strong
accountability
binary
influence on
the operations
leaders in
developing
leading
indicators
(CM)

V Yes Leaders are more
accountable
when
accountability
is framed both
internally and
externally
(Hentschke &
Wohlstetter,
2004).
Provide leaders
internal and
external
frameworks of
existing major
accountability
issues and
challenges in the
organization
related to
developing
leading
indicators
Measurement of
learning and
performance
are essential
components of
an effective
accountability
system capable
of improving
organizational
performance
(Dowd &
Shieh, 2013;
Golden, 2006;
Marsh &
Farrell, 2015).
Provide leaders
examples of
accountability to
developing
leading
indicators that
utilize
assessments, and
align learning
that has
happened and
learning that
will be assessed.
Accountability is
increased when
organizations
adopt a
balanced
scorecard
approach to
Provide leaders the
organizational
goals and
indicators to
reflect various
measures of
performance in
126

Assumed
organization
influence
Validated as a
gap?
Priority? Principle and
citation
Context-specific
recommendation
assessing
performance
(Bensimon,
2007)
the development
and
implementation
of its leading
indicators for
incorporation
into a balanced
scorecard
Organization
needs to have
a clearly
negotiated
accountability
binary for
operations
leaders’
leading
indicator
decision
making (CS)

V Yes Leaders can
create an
effective
accountability
system when
they engage in
the challenging
but necessary
process of
analyzing the
complex social
and political
elements within
an organization
(Waters et al.,
2003).
Provide
Operations
Leaders a
strategy that
identifies a set
of skills they
need to promote
accountability
within their
organization and
improve
organizational
performance.
Building the
capacity of an
organization is
crucial in
improving the
institution and
its
accountability
systems
(Hentschke &
Wohlstetter,
2004; Norton et
al., 2005).
Provide
Operations
Leaders a
strategy for
promoting
accountability
among the
organizational
members in
developing
leading
indicators.
Training people
effectively
requires giving
Provide operations
leaders a
strategy to
127

Assumed
organization
influence
Validated as a
gap?
Priority? Principle and
citation
Context-specific
recommendation
them accurate
procedures,
practice, and
corrective
feedback to
allow for
gradual
automation of
the knowledge
(Clark & Estes,
2008).
create a learning
organization that
possesses key
components of
an
accountability
system that will
promote
organizational
improvement in
the development
and
implementation
of key leading
indicators.
To develop
mastery,
individuals
must acquire
component
skills, practice
integrating
them, and know
when to apply
what they have
learned
(Schraw et al.
2006).

Note. V stands for validated, HP stand for high probability, and N stands for no.

128

Assurance That a Cultural Model of a Strong Accountability Binary Influence Supports
Operations Leaders in the Development of Leading Indicators
The results and findings of this study indicated that the existing cultural model of PMC
represents a gap. The overall survey results determined that 56% of the operational leaders
believed that a strong accountability binary did not exist. In addition, 38% of the operational
leaders did not believe that the communication from leadership reinforced a cultural model of
accountability in developing and implementing effective leading indicators. Regarding the
influence of the organization, 66% of the interviewees stated that there was weakness in the
organization’s ability to influence the accountability binary positively.
A recommendation centered on accountability and organizational change and leadership
theory will be utilized to close the validated influence gap. Rueda (2011) described the cultural
model as a framework of the elements that are visible indicators of the beliefs and values of the
organization. A key aspect of this framework is to provide solutions matched to the specific
causes detailed and are focused on recommendations around organizational structure, practices,
and policies. Hentschke and Wohlstetter (2004) reinforced that leaders are more accountable
when accountability is framed structurally, both internally and externally, and takes many forms.
The measurement of learning and performance are essential components of an effective
accountability system in practice capable of improving organizational performance (Dowd &
Shieh, 2013; Golden, 2006; Marsh & Farrell, 2015). Also, Bensimon (2007) affirmed that
accountability is increased when organizations administer policy and adopt a balanced scorecard
practical approach to assessing performance. These overall findings would suggest identifying
the existing accountability issues and challenges in the organization, developing strategies to
promote and reinforce accountability via policies and practices and creating goals to drive
129

performance and create a learning organizational structure. For example, a team scoreboard that
incorporates measures shared across the organization could reinforce accountability while
reinforcing the learned behavior of establishing effective leading measures to drive the most
impactful outcomes.
Clark and Estes (2008) determined with the support of research of others such as Dixon
et al. (1994) that organizational performance improves when there is a clear vision with goals
and ways to measure progress, the structures of the organization are aligned with the goals,
communication is constant and candid to those involved regarding the plans and progress being
made, and the organizational improvements are integrated with knowledge and motivational
changes. Giessner et al. (2013) studied the effects of accountability on team-focused behavior to
reinforce the importance of addressing accountability gaps. Dowd and Shieh (2013) researched
the correlation between accountability and efficiency regarding financing for community
colleges. After researching 1,083 community colleges, a key finding was that systems of
accountability that measured learning were required if colleges expected to achieve efficiency in
performance funding.
Furthermore, Bensimon and Harris (2007) performed a study on how accountability and
practitioner learning can occur in an organization in improving performance. In a sample of 91
team discussions, the researchers determined that equity scorecards can be used effectively to
increase accountability, determine when an issue is present, and inform stakeholders so that
appropriate action can be taken. In utilizing the balanced scorecard, the practitioners reinforce
the institutional change to deliver the desired outcomes. The evidence supports the
recommendation to provide leaders internal and external frameworks of existing major
accountability issues and challenges in the organization related to developing leading indicators.
130

Leaders should also receive examples of accountability to develop leading indicators that utilize
assessments and align learning that has happened and learning that will be assessed. Lastly,
leaders need to know the organizational goals and indicators to reflect various performance
measures in developing and implementing leading indicators for incorporation into a balanced
scorecard. These approaches suggest that successful implementation will result in a cultural
model that features a strong accountability binary influence and supports the operations leaders
in developing and implementing effective leading indicators.
Implementing a Strategy That Provides a Cultural Setting of a Clearly Negotiated
Accountability Binary to Support Operations Leaders’ Leading Indicator Decision Making
The results of this study indicated that the existing cultural setting represents a gap. The
study data indicated that 85% of the operational leaders interviewed indicated a weak level of
accountability binary present to support operations leaders’ leading indicator decision making. In
addition, 31% of the operations leaders did not confirm that a negotiated accountability binary
was present in any manner within the organization. Rueda’s (2011) work will be the focus to
address the gap perceived by the operations leaders. Rueda described the cultural setting as the
specific work environment within which the cultural model develops, defined as a concrete,
visible representation of the social context in the organization. Key aspects of effectively
intervening in a cultural setting include emphasizing teaching and learning, incorporating tools to
improve the culture, and reinforcing collaborative learning to ensure gains. Four principles
centered on accountability and organizational change represent the foundation for the
improvement recommendations.
First, Waters et al. (2003) determined that leaders can create an effective accountability
system when they engage in the challenging but necessary process of analyzing the complex
131

social and political elements within an organization. Research from multiple sources (Hentschke
& Wohlstetter, 2004; Norton et al., 2005) found that building an organization’s capacity is
crucial in improving the institution and its accountability systems. Clark and Estes (2008)
affirmed via study that training people effectively requires giving them accurate procedures,
practice, and corrective feedback to allow for the gradual automation of the knowledge. Finally,
Schraw et al. (2006) asserted that to develop mastery, individuals must acquire component skills,
practice integrating them, and know when to apply what they have learned. These overall
findings suggest that providing the operations leaders with an effective accountability system and
a training program that builds their capability properly and ensures they acquire the necessary
skills and effective application will improve the cultural setting to ensure a clearly negotiated
accountability binary that supports operations leaders’ leading indicator decision making.
Clark and Estes (2008) stated that a change in performance can happen by changing the
culture of an organization and its existing work environment. The change in the work setting will
be through the framework of accountability and organizational change and leadership. Childress
et al. (2006) studied urban school districts to determine how school district offices can influence
accountability in the school system. In a sample of 15 different urban school districts, research
determined that a variety of strategies can produce positive results in a work setting, contingent
on a consistent focus on strengthening the level of teaching and learning in the environment,
having clear objectives, and establishing accountability. Mosonik (2017) researched the
influence of the promotion of accountability on the overall setting of a funding constituency. In a
sample of 94,105 residents, a key finding was that promoting accountability and overall
transparency increased accountability amongst the overall setting.
132

Norton et al. (2005) explored the level of accountability in California community
colleges and how this can impact the overall environment. Their research on two community
colleges determined that accountability systems implemented effectively can develop
accountability within the environment and subsequently build the capacity of the organization.
Overall, the evidence supports the recommendation to provide operations leaders with the
following strategies: a strategy that identifies the skills to promote accountability and improve
organizational performance, a strategy for promoting accountability among the organizational
members to develop leading indicators, and a strategy to create a learning organization with key
components of an accountability system to promote organizational improvement in developing
and implementing key leading indicators. These approaches suggest successful implementation
will improve the cultural setting with a strong accountability binary to support operations
leaders’ leading indicator decision making.
Integrated Implementation and Evaluation Framework and Plan
The model used to design the integrated implementation and evaluation plan is the new
world Kirkpatrick model (Kirkpatrick & Kirkpatrick, 2016). This model is based on the original
Kirkpatrick model (Kirkpatrick & Kirkpatrick, 2006), which centered around four key levels of
evaluation in order: reactions, learning, behavior, and results, when determining the effectiveness
of a training program and the resulting outcomes. The new world model (Kirkpatrick &
Kirkpatrick, 2016) emphasizes the results and reverses the original model to reinforce the focus
on what is most important for the organization. The core rationale behind reversing the order of
evaluation is that Kirkpatrick and Kirkpatrick (2016) determined that successful training
professionals organize their work by focusing on results first when planning, implementing, and
evaluating training programs.
133

Results (Level 4) are to the degree to which targeted outcomes occur due to the learning
events and are a combination of the organizational purpose and mission. Leading indicators help
bridge the gap between the activities and the desired organizational outcomes and are shorter-
term observations and behaviors that confirm behaviors are on the right track to achieve the
high-level results. Behavior (Level 3) refers to the degree to which the trainees apply what they
have learned back on the job and consists of critical behaviors, required drivers, and on-the-job
learning. Required drivers are processes and systems that reinforce the critical behavior such as
job aids, coaching, pay for performance systems, and recognition.
Learning (Level 2) is the degree to which the trainees acquire the intended knowledge,
skills, attitudes, confidence, and commitment to prevent the cycle of waste with repeat training
for capable people who fail to perform appropriately. Reaction (Level 1) is the degree to which
the trainees react favorably to the training across the three dimensions of customer satisfaction,
engagement, and relevance. The reverse usage of the four-level model (Kirkpatrick &
Kirkpatrick, 2016) provides a training and evaluation program that properly emphasizes the
primary outcomes as the key focus and dedicates resources to the most impactful activities that
deliver organizational value.
Organizational Purpose, Need, and Expectations
The purpose of PMC is to make products that create market-leading value and improve
customers’ and patients’ lives. The mission of PMC is to create an intelligent and more
connected world by being a manufacturing solutions provider that designs and manufactures
products for a world that is increasingly connected and continues to grow at an accelerating rate.
The performance problem of the organization is the underutilization of leading indicators
resulting in the lack of achievement of lagging accountability measures connected to operational
134

results. The specific performance problem of the organization is that it is meeting 0% of its
leading indicators resulting in a gap in performance of 100%. The corporation has a considerable
number of lagging accountability measures of operational performance, including product startup
performance, controllable operating profit, and customer satisfaction metrics. The purpose of
establishing leading indicators that influence and predict performance is to address the current
lagging accountability measures. The organization emphasizes the importance of its employees
living the values by challenging the status quo, moving at a high rate of speed, executing in a
disciplined manner with an overarching purpose, and performing with the highest level of
integrity at all times.
To meet the organization’s overall needs, the operations organization has a goal to meet
100% of its newly developed leading indicators by 2022. The corporation’s operational leaders
are the leaders in the sites who manage the overall production facilities and lead the sites’
executional programs. This group of employees is at the front line and focused and responsible
for delivering the bottom-line results at the manufacturing sites. As a result of this clear line of
accountability, the operations leaders are the stakeholder group of focus for the study. The
stakeholder’s goal, supported by the regional and senior leaders, is for leading indicators to align
with organizational goals developed by December 2021.
A review of organizational data revealed a very high number of lagging measures at the
operational level, with 143 counted and zero leading indicators in place. The focus of this study
was to understand the knowledge, motivational, and organizational influences that impact the
operations leaders’ ability to develop and implement effective leading indicators. After
understanding the influences on the operational leaders, a core objective of this stakeholder
group would be for all of the trainees to develop a strategy and corresponding plan in alignment
135

with the organizational goals. With the intent to enhance the operations leaders’ ability to
develop and implement effective leading indicators, this strategic plan would assess and identify
the needs, establish effective countermeasures to close the gaps, measure the overall
effectiveness and sustained growth, and increase overall stakeholder engagement.
Level 4: Results and Leading Indicators
Table 11 shows the Level 4 results and leading indicators for the operations leaders’
learning development plan regarding leading operational indicators, categorized as both external
and internal outcomes, as well as the defined metrics, and corresponding methods to implement
and evaluate the effectiveness of the training. Effective implementation of the shorter-term
recommended methods with measurement of the defined metrics will result in the achievement
of the internal outcomes. The internal outcomes are leading indicators as defined by Kirkpartick
and Kirkpatrick (2016) of the operations leaders’ ability to achieve the defined external
outcomes. The successful attainment of the internal outcomes will increase the likelihood that the
external outcomes will be achieved. As the external outcomes are attained, the probability is
subsequently increased that the organization achieves its longer-term goals. As detailed, the
longer-term organizational goal is to achieve 100% of the newly developed operational
indicators.

136

Table 11
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metrics Methods
External outcomes
PMC increases market share
versus competitors
% market share Quarterly market analysis
PMC improves profitability
from a global perspective
Percentage of increase in
corporate operating profit
Monthly corporate reports
Increased promotions within
operational leader pool
Number of role promotions Monthly personnel
promotions tracker
PMC increases in employee
engagement
Percentage of increase on
engagement score
Annual engagement survey
Increase in global customer
favorability
Customer “likely to
recommend” favorability
score
Quarterly customer
satisfaction score
Internal outcomes
Operations teams’ increased
profitability
Percentage of increase in
operations operating profit
Weekly operating profit
reports
Site customer favorability Overall favorability score Quarterly customer
satisfaction survey
Increases in operations
leaders’ engagement
Operations leader engagement
scores
Annual engagement survey
Operations leaders have
developed at least three
leading indicators for their
teams
Number of new leading
indicators developed by
manufacturing site
Weekly operations
performance scorecard
Leading Indicators correlate
with key lagging business
outcomes
Variance between the
percentage of leading
indicator measurement
outcomes and lagging
indicator improvement
Weekly operations
performance scorecard
Leading operational
indicators identified
consistently perform better
Number of leading indicators
identified by site at or
above target
Weekly operations
performance scorecard
137

Outcome Metrics Methods
than target
Provide the highest level of
quality performance
Quality internal and external
customer score
Monthly and quarterly quality
scorecard

Level 3: Behaviors
Critical Behaviors
The stakeholder group of focus for this study was PMC’s operations leaders. Three
critical behaviors had the most influence on these leaders’ ability to deliver Level 4 results and
accelerate their transition from learning to delivering the desired outcomes. The first critical
behavior is to establish the critical business and functional goals that must be achieved. The
second is to execute the process to identify, develop, and reinforce leading operational indicators.
The third is to establish a clear accountability process. Table 12 shows the Level 3 behaviors and
details the critical behaviors, metrics, methods, and timing required to effectively evaluate the
demonstration of the desired behaviors.

138

Table 12
Critical Behaviors, Metrics, Methods, and Timing for Evaluation of Operations Leaders
Critical behavior Metrics Methods Timing
Operations leaders
must establish
critical business
and functional
goals that must be
achieved
Number of business
critical goals
The operations leader
leads a kickoff
session with the
defined team where
each group
establishes no more
than two business
critical goals
Operations leader
assesses viability
quarterly with the
team.

Progress against
defined plans are
tracked and shared
on a quarterly basis.
Number of critical
goals by
functional team
The operations leader
holds a follow-up
meeting to ensure all
teams and their
respective
departments have
their two most
important business
goals
Operations leader
assesses viability
quarterly

Progress against
defined plans are
tracked and shared
on a quarterly basis.
Number of critical
goals with a
quantifiable
deliverable
The operations leader
leads an information
session highlighting
how to set
quantifiable
SMART goals.

Subsequently, the
operations leader in
a follow-up meeting
held timely after
initial session to
verify that all most
important goals have
a quantifiable result
Operations leader
assesses viability
quarterly

Operations leader
assesses strength of
goal adherence to
SMART format

Progress against
defined plans are
tracked and shared
on a quarterly basis.
Operations leaders
must execute
process to
implement leading
operational
Number of leading
indicators by
critical goal
The operations leaders
conduct ideation
sessions with their
teams to define most
important top two
Operations leader
assesses viability
quarterly to
determine whether
leading indicators
139

Critical behavior Metrics Methods Timing
indicators predictive and
influenceable
measures that will
drive the identified
most important
goals
are moving the
lagging measures

Progress against
defined plans are
tracked and shared
on a quarterly basis.
Percentage of
functional teams
with completed
leading indicator
development
sessions
Operations leaders
ensure sessions with
teams are completed
to generate lists of
potential leading
indicators as
outcomes or
behaviors and
ensure that the lists
are narrowed down
to the two that
provide the most
leverage
Operations leader
assesses viability
quarterly

Progress against
defined plans are
tracked and shared
on a quarterly basis
Number of
functional teams
with process to
track identified
leading indicators
The operations leaders
align with the
accountable teams to
establish a process
to track leading
indicators
Operations leaders
track progress with
each team against
indicators weekly at
a minimum

Operations leader
assesses viability
quarterly

Progress against
defined plans are
tracked and shared
on a quarterly basis
Operations leaders
must sustain a clear
accountability
program regarding
leading indicators
Percentage of
operations leaders
with leading
indicator
scorecards in
place
Operations leaders
lead the team to
create a scorecard to
be maintained by
team leaders to track
leading indicators
and drive
accountability
Monthly Results
Review
140

Critical behavior Metrics Methods Timing
Percentage
scorecard
compliance to
corporate
standard
Operations leaders
create and verify a
team scorecard is
simple, visual, has
both leading and
lagging indicators,
and clearly shows
where team is
winning or losing
Weekly at a minimum
in alignment with
management
cadence
Percentage
completion of
team scoreboard
standard review
checklist
Operations leaders
establish a regular
follow-up cadence
no less than weekly
with the following
characteristics: a
report on prior
commitments
review of scorecard
removal of obstacles
commitment to new
actions minutes
shared shortly
thereafter
Weekly at a minimum
in alignment with
management
cadence
Percentage
completion of
leader’s team
scoreboard
checklist
Leader provides
accountability to
scoreboard process
via consistent
demonstration of
consideration for
leaders involved in
the process,
including
recognition for
superior follow-up
and wins
encouragement of
new ideas to move
operational
indicators support of
fellow team
members, and
reinforcement of
Weekly at a minimum
in alignment with
management
cadence
141

Critical behavior Metrics Methods Timing
commitment by
acknowledging
contribution,
highlighting the
importance of
honoring
commitments, and
sustainment of
completing actions
in alignment with
the cadence

Required Drivers
Required drivers are grouped into the categories of support and accountability and are the
key mechanisms that ensure the desired behaviors happen and are sustained. Within the two
categories of support and accountability are four areas of focus that consist of reinforcing,
encouraging, rewarding, and monitoring (Kirkpatrick & Kirkpatrick, 2016). In the area of
reinforcement, operations leaders will complete self-directed training provided by the
organization, utilize tools such as job aids, observe executive modeling of the desired behavior,
and actively participate in a community of practice to share learnings gained. From an
encouragement perspective, strategic reviews with leadership to note the progress made, positive
recognition when leading indicator usage is demonstrated, and collective communication forums
such as town halls would be the desired drivers to increase motivation for the operations leaders
and minimize organizational hurdles.
In terms of rewarding, the organization recognizing operations leaders in the management
cadence forums, ensuring positive recognition when milestones regarding the defined goals are
achieved, and revising the existing compensation systems to highlight the importance of leading
142

indicators would be the key mechanisms for the organization to utilize. Regarding monitoring,
the drivers that will sustain the critical behaviors are establishing team-level scorecard reviews,
conducting strategic ad-hoc sessions with the operations leaders for key measures off track,
maintaining the discipline of regular reviews, conducting post-training assessments to ensure the
learned concepts are applied, and senior-level recognition of team accomplishment at global
forums. Table 13 shows the Level 3 behaviors and details the required drivers determined from
this study, the recommended implementation frequency, and the alignment with the previously
established critical behaviors.

Table 13
Required Drivers of Stakeholders to Support Critical Behaviors
Methods Timing Critical behaviors supported
1, 2, 3, etc.
Reinforcing
Operations leaders’
completion of self-directed
module of concepts for
developing leading
indicators
Quarterly 2
Operations leaders’
completion of self-directed
module on identifying most
important goals
Quarterly 1
Operations leaders’
completion of self-directed
module setting SMART
goals
Bi-annual 1, 2
Executive modeling by
regional leaders of weekly
accountability process
utilizing the leading
Weekly 1, 2, 3
143

Methods Timing Critical behaviors supported
1, 2, 3, etc.
indicator scoreboard
Regional leaders lead team
community of practice to
review leading indicator
scoreboard
Weekly 1, 2, 3
Regional leaders provide
information in the form of a
pamphlet for the operations
leaders to identify top three
most impactful measures
for their respective
operations
Ongoing 1
Regional leaders provide job
aid that details criteria to
identify most important
goals
Ongoing 1
Regional leaders provide job
aid that details the
operations leader role in
developing and
implementing leading
indicators
Ongoing 2
Regional leaders provide job
aid that details the steps in
developing leading
indicators
Ongoing 2
Regional leaders provide job
aid that outlines the process
of leading the
accountability process
Ongoing 1, 2, 3
Operations leader’ completion
of self-directed training on
reinforcing accountability
Quarterly 3
Regional leaders’ on the job
reviews with operational
leaders to review
understanding of the
Ongoing 1, 2, 3
144

Methods Timing Critical behaviors supported
1, 2, 3, etc.
leading indicator
development and execution
process
Operations leaders’
completion of operational
leader onboarding which
includes the importance of
leading indicator
development in the role
Ongoing 2
Encouraging
Regional leaders complete
strategic reviews with
operational leaders to deep
dive leading indicator
performance and problem
solve drivers for gaps from
either knowledge,
motivation, or organization
Ongoing 1, 2, 3
Regional leaders’ creation of
community of practice
program where team shares
progress in leading
indicator implementation
Weekly 1, 2, 3
Regional leaders provide
regular, specific feedback
to operational leaders
regarding expected usage of
leading indicators to drive
performance
Weekly 2
Regional leaders’ positive
recognition of operational
leaders utilizing their
strengths to develop and
implement
Daily 2
Regional leaders’ strategic
reviews with operational
leaders to present progress
versus objectives and
Quarterly 1, 2, 3
145

Methods Timing Critical behaviors supported
1, 2, 3, etc.
encourage creativity
improved levels of problem
solving
Regional leaders conduct
town halls with operational
leaders as a collective
group to remind of the
intent of leading
operational indicators and
positively reinforce
effective usage
Quarterly 1, 2, 3
Rewarding
Regional leaders recognize
operational leaders for
strong accountability and
leading indicator utilization
via the management
cadence process
Weekly, monthly, quarterly 2, 3
Regional leaders provide
positive recognition of
milestones,
accomplishments on
established goals
Weekly 1, 2, 3
Regional leaders’
establishment of
management structure that
encourages personal
responsibility of operations
leaders in implementing
leading indicators
Weekly 1, 2, 3
Regional leaders ensure
compensation systems are
constructed around
effective development and
implementation of leading
indicators
Annual 1, 2, 3
Monitoring
146

Methods Timing Critical behaviors supported
1, 2, 3, etc.
Regional leaders’
development of strategy for
promoting accountability in
implementing leading
indicators
Ongoing,
quarterly at a minimum
1, 2, 3
Regional leaders provide
accountability of
performance versus key
process indicators via team
scorecard reviews
Weekly 1, 2, 3
Regional leaders conduct ad
hoc sessions conducted
with operational leaders for
key process indicators off
track
Ongoing 3
Operations leaders’ regular
usage of team performance
scoreboard
Weekly 3
Regional leaders conduct post
training on operational
indicators an accountability
follow-up survey to
understand level of
application by operational
leaders and their respective
managers
Ongoing,
quarterly at a minimum
1, 2, 3
Team performance and
overall accomplishments in
developing leading
indicators are recognized
by regional leadership
publicly at global meetings
and via corporate electronic
communications
Quarterly 1, 2, 3

147

Organizational Support
For the operational leaders to effectively develop and implement leading indicators, the
organization will have to provide support. A key aspect of the support required is to provide for
the operations leaders the ability to effectively transition from a phase of learning to a successful
demonstration of the learnings in their current roles. The demonstration of the critical behaviors
detailed in Table 12 and the required drivers in Table 13 is contingent on the organization
providing an environment to address the knowledge, motivational, and organizational gaps
detailed previously. To close the knowledge gap, the operations leaders’ understanding of
leading indicators and accountability must be addressed via organization-provided training
resources.
To improve the overall motivational influence, the alignment of the organizational goals
with the goals established by the operational leaders and the establishment and reinforcement of
accountability systems to measure against the established goals will be key mechanisms for the
organization to support. Also, PMC’s cultural models and settings will have to evolve to create
an effective accountability binary with the operations leaders to support the sustainment of the
required drivers detailed that will result in the achievement of the desired outcomes. This
includes a continued and recurring assessment of the organizational environment to remove any
hindrances to operations leaders’ ability to develop and implement effective leading indicators in
the delivery of the desired business outcomes. This key characteristic of the desired organization
will need to be incorporated in the transition strategy for the operational leaders to ensure
success.

148

Level 2: Learning
Learning Goals
For the operational leaders to effectively develop and implement leading indicators, the
organization will have to provide support. A key aspect of the support required is to provide for
the operations leaders the ability to effectively transition from a phase of learning to a successful
demonstration of the learnings in their current roles (Table 14).

Table 14
Evaluation of the Components of Learning for the Program
Methods or activities Timing
Declarative knowledge: “I know it”
Knowledge checks of each concept via short
quizzes or live discussion
At the end of each module during the session
Knowledge checks during group activity
when teams present outcomes
At the end of each group exercise during the
session
Knowledge checks where teams present their
understanding of concepts and to the group
Regularly throughout the session for targeted
concepts
Knowledge checks via games where correct
responses connect to the training material
Periodically during the session when team is
having difficulty absorbing taught concepts
Post tests At the end of the session
Verification of understanding during
management cadence meetings
As the critical behavior and required drivers
action plan is implemented
Procedural skills: “I can do it right now”
Case studies with questions where participants
role play and facilitator/other students
provide feedback
Periodically throughout the session after
concepts are explained
Scenarios where participants select from
optimal to suboptimal options to
demonstrate ability to use concepts
Periodically throughout the session after
concepts are explained
149

Methods or activities Timing
Demonstrating of understanding regarding
desired training outcomes
As the critical behavior and required drivers
action plan is implemented
Observation by the facilitator of
demonstration by the students
During the session and documented by the
facilitator
Demonstration of application of training
concepts via the development of
improvement plans for their own teams
During the session and documented by the
facilitator
Attitude: “I believe this is worthwhile”
Facilitator holds open discussion with trainees
to discuss the positive outcomes of applying
concepts and the negative outcomes of not
applying them
At the end of each module during the session
Post session survey to gauge students’ attitude
about the training
Immediately after the session
Students share how they valued the training
and how it has supported their professional
development
30, 60, and 90 days after the session
Confidence: “I think I can do it on the job”
Survey to gauge student confidence level in
practicing concepts on the job
At the end of the training session and then 30,
60, and 90 days after
Observations regarding trainees’ application
of concepts as part of core daily
responsibilities
Ongoing
Facilitator engages trainees in discussions
regarding confidence
During training session
Commitment: “I will do it on the job”
Establishment of clear goals and deliverables
as a result of the training session
During the training session
Demonstrated progress against established
goals and deliverables from the session
Ongoing as part of the established
management cadence to include weekly,
monthly, and quarterly
Facilitator notes level of commitment via
observations
During training session
150

Methods or activities Timing
Commitment survey During and post session for comparison
purposes

Level 1: Reaction
Once the training is concluded, a key activity of Level 1 (reaction) is to gain feedback
from the trainees relative to their level of engagement, the strength of their belief that the content
administered was relevant to their current roles, and the extent to which the trainees were
satisfied with the training provided. Table 15 details the methods or tools utilized to evaluate the
participants’ reactions and the overall timing and designated frequency for each assessment.

151

Table 15
Components to Measure Reactions to the Program
Methods or tools Timing
Engagement
Completion of assignments and deliverables
during the training session
Ongoing during the course
Facilitator observations of team engagement During the session
Alignment by team on course concepts During the session
Quality of the goals developed by the trainees During the session
Assigned dedicated observer to observe team
dynamics and gauge engagement
During the session
Pulse checks of trainees by facilitator Ongoing during the session
Alignment with trainees to communicate
deliverables agreed upon to larger
organization
At the end of the session
Relevance
Facilitator observes trainees establishing
strategies of timely application of concepts
in their current roles
During the session
Survey Likert scale of level of application of
concepts in their current roles
At the end of the session
Customer Satisfaction
Pulse survey to identify barriers in need of
removal
At the end of each module within the session
Facilitator removal of identified behaviors by
trainees
Ongoing during the session
Session Evaluation At the end of the session

152

Evaluation Tools
Immediately Following the Program Implementation. Throughout the plan’s
implementation, there are methods to assess the participants’ reaction as and overall learning. As
illustrated in Tables 13 and 14, the facilitator will gain qualitative feedback via methods such as
knowledge checks throughout the session and at the end of each module, open discussions
regularly with the participants, and data collected from overall observations of the participants
throughout the program modules.
In addition to the feedback provided during the program’s administration, evaluation
assessments performed immediately after the conclusion of the training modules will determine
both reaction and learning. Level 1 questions are designed to gauge the participants’
engagement, belief in the relevance of the module material to their roles, and satisfaction with
the session material overall. Learning questions are designed to measure the participants’
knowledge, their overall attitude towards applying the concepts gained, and their overall
confidence level in changing their behavior in applying the approach in their current roles.
Appendix D references an example of the evaluation utilized featuring a Blended Evaluation
method (Kirkpatrick & Kirkpatrick, 2016) that will be distributed to the operations leaders
immediately at the conclusion of the program modules.
Delayed for a Period After the Program Implementation. Based on recommendation
from Kirkpatrick and Kirkpatrick (2016), the optimal timing to conduct post-program evaluation
is after the required drivers are in place and the participants have had an opportunity to practice
the desired critical behaviors. While typical post-program evaluations are conducted 90 days
after the event, the ideal approach Kirkpatrick and Kirkpatrick (2016) emphasize is to consider
when it is most appropriate to conduct and what would be the ideal tools to use to gather the
153

data. For this study, Appendix E provides a sample of the blended evaluation method
(Kirkpatrick & Kirkpatrick, 2016) utilized to perform the assessment with the intent to
administer it in 30-, 60-, and 90-day intervals.
The core objective of performing the delayed evaluation is to understand overall progress
against the organizational outcomes, defined as results (Level 4). The key indicator of eventually
delivering on the desired outcomes is the Level 3 results, defined as the critical behaviors that
participants demonstrate due to the required drivers being in place. The delayed evaluation
method will also provide a gauge of the participants’ learning (Level 2), defined as the
knowledge, skills, attitude, confidence, and commitment as well as their reaction (Level 1) and
whether the program is still relevant to their roles and satisfies individual needs. The evaluation
method will utilize a Likert scale 1–5 measurement approach to more effectively determine the
degree of response from the participants.
Data Analysis and Reporting
Kirkpatrick and Kirkpatrick (2016) describe dashboards as an effective tool to share
program information and visualize progress towards the desired outcomes. A key benefit to this
approach, as described, is a clear display of areas that are below expectation so all involved with
the initiative can review and develop the plans to intervene and ensure all items get back on
track. Ultimately, the dashboard approach can be leveraged to resolve problems before the
desired outcomes are compromised.
As detailed by Kirkpatrick and Kirkpatrick (2016), the majority of the dashboard is to be
comprised of results (Level 4), which are the program’s and the corresponding leading
indicators’ desired outcomes, and behavior (Level 3), meaning the critical behaviors the
operations leaders are to demonstrate and the drivers required to sustain these behaviors. This
154

approach also minimizes one of the pitfalls shared where the majority of resources in
organizations are typically focused on Levels 1 and 2. In addition and as reinforced by
Kirkpatrick and Kirpatrick (2016), the design is to be kept simple and devoid of multiple pages
while still utilizing the data gathered from the evaluation. Table 16 is an example of a sample
dashboard that would embody these characteristics. Appendix F also provides a comparison
dashboard of results gathered after the training program versus the information gathered from a
delayed perspective. This approach would be a sustained indicator of whether the operations
leaders are developing as a result of the program and subsequently delivering the results or are
requiring further intervention and support. The health of each of the measures would be defined
on a more directional red/yellow/green scale, where red constitutes off track, yellow corresponds
with off track with a timely plan to close the gap, and green would be defined as on track and
ahead of schedule.

Table 16
Leading Indicator Dashboard
Measure Frequency Target Actual RYG Status
Level 4 results
% PMC market
Share
Quarterly % improvement
vs prior year
Green
Operating profit Monthly % improvement
vs prior year
Green
# of role
promotions
Monthly % improvement
vs prior year
Yellow
Employee
Engagement
Survey Score
Annual % improvement
vs prior year
Green
155

Measure Frequency Target Actual RYG Status
Customer
Favorability
Score
Quarterly % improvement
vs prior
quarter
Red
Operations
Operating
Profit
Weekly Quarterly
financial
commit
Red
Site Customer
Favorability
Score
Quarterly % Improvement
vs prior
quarter
Green
Operations
Leader
Engagement
Scores
Annual % Improvement
vs prior
quarter
Green
# of new leading
indicators
developed
Weekly 2X each
functional
department
Yellow
Variance of
leading
indicator
improvement
vs lagging
indicators
Weekly <5% difference Green
# of leading
indicators at or
above target
Weekly 100% Yellow
Quality internal/
external score
Monthly % improvement
vs prior
quarter
Green
Level 3 behaviors
# of business of
critical goals
Quarterly 2X each
functional
department
Green
# of critical
goals by
functional
team
Quarterly Two per each
functional
department
Yellow
156

Measure Frequency Target Actual RYG Status
# of critical
goals that are
quantifiable
Quarterly 100% Yellow
# of leading
indicators by
critical goal
Quarterly Two per critical
goal
Green
% of functional
teams with
completed
leading
indicator
development
sessions
Quarterly 100 Green
% of functional
teams with
process to
track leading
indicators
Weekly 100 Yellow
% of operations
leaders with
leading
indicator
scorecards in
place
Monthly 100 Green
% completion of
scorecard
compliance
checklist
Weekly 100 Red
% completion of
leader’s
scoreboard
checklist
Weekly 100 Green
Level 2 learning
Knowledge
checks with
operations
leaders
Ongoing Performed to
schedule
Green
157

Measure Frequency Target Actual RYG Status
Operations
leaders
attitude score
Monthly 5 on 1–5 scale Yellow
Operations
leaders
confidence
score
Monthly 5 on 1–5 scale Green
Commitment
level
Monthly 5 on 1–5 Scale Green
Progress against
established
goals and
deliverables
from each
training
session
Ongoing On target Red
Level 1 reaction
Operations
leaders fluent
in deliverables
of the program
Ongoing Delayed
evaluation
results of 5
Yellow
Operations
leaders
applying the
concepts of the
material in
current role
Ongoing Delayed
evaluation
results of 5
Green
Presence of
barriers of
operations
leaders
Ongoing Delayed
evaluation
results of 5
Yellow

158

Summary
The new world Kirkpatrick model (Kirkpatrick & Kirkpatrick, 2016) was the approach
utilized in this study to plan, implement, and evaluate the recommendations determined from the
data analysis performed. The recommendations were developed as KMO solutions to the
problem that PMC is meeting 0% of its leading performance measures that tie to its existing
lagging operational performance indicators. The organization has a goal to meet 100% of its
newly developed leading indicators by 2022. In support of the organizational goal, the respective
stakeholders have goals consisting of developing leading indicators, placing leading indicators
into policy, and measuring and legislating them throughout the organization. The new world
model featured an approach that starts with the desired results and corresponding leading
indicators or Level 4 outcomes initially, then the required critical behaviors and the supporting
drivers to reinforce them, defined as Level 3 behaviors. The critical behaviors are detailed as the
operations leaders’ most important behaviors that provide the most impact to the Level 4
outcomes. The intent of the approach of the new world model (Kirkpatrick & Kirkpatrick, 2016)
was to increase the probability of achieving the organizational and stakeholder goals described
previously by influencing the necessary behaviors to deliver the desired results.
While Levels 3 and 4 are emphasized with the new world model, the Level 2 learning,
which is the degree to which the operations leaders acquire the intended knowledge, skills,
attitudes, confidence, and commitment, and Level 1 reaction, the extent to which the operations
leaders find the program engaging, relevant, and favorable, were also core to the implementation
and evaluation of the program. To reach the desired levels, ongoing pulse-checks and immediate
assessments after the program were conducted. Of particular focus for the operations leaders was
establishing the necessary motivation and cultural settings to develop and implement leading
159

indicators. While Levels 1 and 2 were not as critical to the success of the program as Levels 3
and 4, gathering the assessment data initially to ensure foundational elements were established to
effectively engage the operations leaders and provide an environment for the critical behaviors to
be established and sustained are key in ultimately delivering the expected results.
Kirkpatrick and Kirkpatrick (2016) highlighted the importance of filtering the data that
needs to be captured to make the best decisions for both the organization and the stakeholders.
One key aspect of this process is gathering data throughout the implementation period and in a
delayed manner to provide the necessary inputs to signal interventions where appropriate. The
data collection methods regarding the assessments of the program reflect this approach.
Appendix F provides a dashboard that would be utilized to signal trends that are either going in
the needed direction or require a course correction by the leader. Kirkpatrick and Kirkpatrick
detailed three key questions to determine the following with the intent to maximize the process:
(a) whether the outcome at each of the four levels of Kirkpatrick and Kirkpatrick (2016) meet the
expectations, (b) if the outcomes do not meet expectations and understanding of why, and (c) if
the outcome does meet the expectations, why it was able to accomplish. This approach builds
credibility with the operations leaders as well as the stakeholders, while also improving the
probability of success for future programs.
Kirkpatrick and Kirkpatrick (2016) describe return on expectation (ROE) as what a
successful training initiative delivers to key business stakeholders, demonstrating the degree to
which their expectations have been satisfied. The ROE is also the ultimate indicator of value
where the necessary leading indicators to delivering the Level 4 results are established. With
ROE, which is a key differentiator of this approach, the business and the training facilitator have
to partner throughout the entire process, from planning through implementation and evaluation,
160

as part of an effective Level 3 execution plan. This approach ensures that success is defined and
aligned up front cooperatively with the business. The corporation’s approach with the operations
leaders of the new world model in developing leading indicators has the characteristics of
maximizing ROE embedded in its process. By utilizing this model, PMC has a higher probability
of achieving the desired outcomes, including achieving all of its leading operational indicators in
2022.
Strengths and Weaknesses of the Approach
This study initially featured a mixed-methods data collection process focused on the
Clark and Estes (2008) gap analysis methodology to understand the KMO opportunities that
influence the problem of practice for the stakeholder group of focus for this study, the operations
leaders. This methodology was developed as a result of research of multiple organizations to
better understand the drivers for gaps in organizational performance and how best to solve them.
The case studies provided mirrored PMC’s characteristics. In corporate organizations, influences
regarding employees’ active choice, their knowledge of how to perform their roles effectively,
and their overall environment have a similar impact.
A specific case study provided by Clark and Estes (2008) focused on an organization
focused on improving manufacturing performance that correlated accurately with the current
problem of practice at PMC. The results showed clear knowledge, motivational, or
organizational influences, reinforcing the appropriateness of the approach. In addition, as the
operations leaders are the stakeholder group of focus, they are accountable for executing the
recommended solutions. One of the weaknesses of the KMO methodology is that once an
effective diagnosis is complete, the approach to address the gaps is more under the discretion of
the individual and subjectively what they believe is the optimal tool and process to apply. This
161

could be problematic if the individual performing the diagnosis is not as fluent in improvement
methodologies and an initiative is unsuccessful, bringing into the discussion potential suspicion
that an effective definition of the problem to solve was not performed nor was a clear objective
established. To address this potential weakness, the new world Kirkpatrick Model (Kirkpatrick &
Kirkpatrick, 2016) was introduced for this study as the implementation, execution, and
evaluation methodology.
The new world Kirkpatrick model (Kirkpatrick & Kirkpatrick, 2016) featured a clear
process around effectively developing a human capital development program centered around
developing clear outcomes and establishing the necessary behaviors to deliver the results. Clark
and Estes (2008) stated that for an improvement program to be successful, a solid evaluation and
implementation design must be developed to determine the trainees’ reaction, learning, and
transfer of learnings to their behaviors on the job. In addition, Clark and Estes (2008) also
emphasized the need to have a human analysis process that constantly detects issues as the
overall environment changes. The new world process emphasizes fewer resources dedicated
upfront to the traditional areas of reaction and learning of the operations leaders and more on
demonstrating and delivering the bottom-line business results. An opportunity exists regarding
the influences of the cultural model and settings because the sustained improvement of the
operations leaders will require this gap to be closed. However, by delivering the Level 4 results
via the implementation of leading operational indicators, the organization will be expected to
evolve in alignment with the performance improvement realized.
Limitations and Delimitations
In a review of the study overall, from the perspective of the selection of the data
collection methods of surveys, interviews, and reviews of internal documentation and artifacts as
162

well as the implementation of the selected methods, there were delimitations as well as some
flaws, problems, and limitations after the data collection concluded.
In the delimitation space, the mixed-methods approach was intended to balance the
quantitative data present for PMC via internal documents and artifacts with the qualitative data
gathered as a result of the surveys and interviews to provide a richer context of the KMO
influences present impacting the problem of practice. As the data determined opportunities in the
areas of motivation and cultural models/settings, a limitation of the study was the stakeholder
group. The stakeholders were the operations leaders. However, it would have been beneficial to
gather qualitative data from senior-level stakeholders such as regional leaders and functional
presidents who play an influential role in creating the operations leaders’ environment and
providing mechanisms to facilitate motivation. These more senior-level stakeholders contribute
to the creation of obstacles present for the operations leaders, so their input as to the drivers of
the gaps would have been beneficial. This approach also includes another limitation with the size
of the interview pool, with less than half of the survey participants consenting o be interviewed.
Another limitation was the central focus on operational leading indicators. The analysis
was performed through the lens of what factors could be contributing to the lack of leading
indicators that drive the lagging indicators. As data were analyzed, additional themes surfaced,
such as accountability and the corresponding structures to reinforce it or lack thereof impacting
the organization’s success. The accountability theme appeared to influence the performance of
some of the organizations in which the operations leaders operated or for which they were
responsible. However, this apparent opportunity requires additional study. As mentioned
previously, including the regional leaders in the sample population would provide additional
insights on accountability and the opportunities around strategic alignment that impact the
163

accountability environment. In terms of the evaluation plan, one limitation was the overall focus
on the operational function alone, with the operational leaders being the stakeholder group of
focus. The success of PMC’s operational leaders is highly dependent on the effectiveness of the
other supporting functions such as supply chain, human resources, and finance because PMC is a
matrixed organization. However, the analysis or the implementation and subsequent evaluation
plan did not include these functions. Therefore, a key limitation of the research was that the
population did not include the functional leaders within the organization that may have more
clearly defined the KMO influence present more broadly across the organization versus within
one competency alone.
Future Research
The focus of this study was to understand the KMO influences that contributed to the
operations leaders’ lack of leading operational indicators that would ensure delivery of the
lagging business measures. In establishing the influences, of particular inquiry was the level of
accountability present to address the perceived gaps. A key driver for the emphasis on
accountability was the current lack of performance on the lagging operational accountability
measures for many operational leaders gained from the analysis of existing internal artifacts and
documents. Post-review of this study, there are some areas of interest for further study. The first
area pertains to gathering additional qualitative data from a larger population, such as the senior-
level leaders, including the regional leadership and organizational presidents. The additional
perspectives gained would also increase the probability of the ROE as defined by Kirkpatrick
and Kirkpatrick (2016). Validated causes of organizational gaps were identified in data analysis
as the cultural models and cultural settings present. Including multiple levels in the organization
would allow the researcher to gain richer insights into the organizational barriers present.
164

A second area of further research would be additional diligence prior to the study to
better understand the drivers for gaps in performance at PMC. This study was determined on the
basis that leading operational indicators were missing due to the organization’s current level of
excessive measurement as determined from internal processes and documents. Pre-study
research with a larger stakeholder population to identify potential causes behind the themes
would provide deeper insights to inform the recommended solutions.
A third area of further research would be a broader population for the study to include the
functional departments that partner with the operations leaders in delivering the business
outcomes. The excessive level of measurement present in the organization and validated by
internal documents crossed multiple functions at PMC. Therefore, evidence is present that
recommendations focused on prioritization and accountability as leaders should be explored in
greater detail. In addition, a similar implementation and evaluation approach of ongoing and
post-assessments to note the degree of behavioral change of the stakeholders of focus as a result
of the programs would provide actionable insights for the organization to either reinforce or
intervene to ensure the desired ROE.
Conclusion
The performance problem of focus in the manufacturing industry is the underutilization
of leading indicators. At a macro level, leading indicator measures are important factors in
overall performance in a wide range of industries. In the financial industry, the lack of leading
indicators was a key driver of organizational failures during the 2008 global financial crisis
(Chou, 2015). The performance problem at PMC is the underutilization of leading indicators,
which results in the lack of achievement of lagging accountability measures connected to
operational results. The operational leaders were the stakeholder group of focus because they are
165

responsible for the bottom-line execution at the manufacturing sites and are closest to the leading
activities that provide the leverage to develop the leading indicators. Clark and Estes’s (2008)
gap analysis model was the framework used to define the organization’s goals and identify the
gap between the current level of performance and the desired level of performance relative to
operations leaders’ KMO influences. The gaps between the desired and current performance
were validated.
The validated findings determined that the operations leaders had a knowledge need for
conceptual understanding of developing and implementing effective leading indicators. From a
motivation perspective, they also demonstrated an opportunity to improve their goal orientation
regarding leading indicator development. Finally, organizationally, there were needs regarding
both the cultural model and cultural setting in support of the accountability binary to develop and
implement leading indicators.
As a result of the needs identified, several solutions emerged to close the current gaps.
The first is to increase operational leaders’ conceptual knowledge of the core skills and
procedures involved with establishing effective leading indicators via pamphlets and job aids
would-be drivers to reinforce the learning. The second is to drive a high level of goal orientation
with the operations leaders via task, evaluation, and reward structures and a community of
learners would reinforce the need to increase motivation in implementing leading indicators. The
third is to create a cultural model of a strong accountability binary influence that supports
operations leaders’ development of leading indicators with an accountability framework and
accountability mechanisms such as scorecards. Finally, to provide a cultural setting of a clearly
negotiated accountability binary, a strategy that develops the necessary skills and behaviors to
166

support operations leaders’ leading indicator decision making with the leveraging of mechanisms
such as predictive measures would improve the overall environment.
As a result of the findings, there are key implications that inform a wider practice that
could be helpful in other organizational contexts. As mentioned earlier, the opportunity to
expand the approaches across functions and levels of the organization. The organizational
influences consisting of the cultural models and settings indicate that the overall environment is
in need of improvement, so the stakeholders outside of the operations leaders, such as the
regional vice presidents and overall presidents who can impact the environment, would be targets
of focus for further gap analysis from a KMO framework. The literature review highlighted that
the approach of leading indicators positively impacts bottom-line performance industries, such as
manufacturing, finance, and occupational safety. The data affirmed the approach and
corresponding recommendations in this study would work more universally at PMC. With a
strong implementation and evaluation plan similar to what was recommended for the operations
leaders, PMC, across multiple functions and levels, will effectively implement leading
indicators, prioritize effectively, improve organizational performance, and ultimately become an
organization that inspires trust in its customers, employees, and the overall marketplace.

167

References
Adelson, J. L., & McCoach, D. B. (2010). Measuring the mathematical attitudes of elementary
students: The effects of a 4-point or 5-point Likert-type scale. Educational and
Psychological Measurement, 70(5), 796–807.
Aguayo, R. (1990). Dr. Deming : the American who taught the Japanese about quality (1st ed.).
Carol Publishing Group.
Ahlstrom, P. (2004). Lean service operations: Translating lean production principles to service
operations. International Journal of Services Technology and Management, 5(5/6), 545–
564. https://doi.org/10.1504/IJSTM.2004.006284
Alagumurthi, N., Palaniradja, K., & Soundararajan, V. (2008). Optimisation of process
parameters in grinding on different dimensions and perspectives. International Journal of
Industrial and Systems Engineering, 3(4), 447–473.
https://doi.org/10.1504/IJISE.2008.017554
Almost, J., Caicco Tett, L., VanDenKerkhof, E., Paré, G., Strahlendorf, P., Noonan, J., Hayes,
T., Van hulle, H., Holden, J., Silva e Silva, V., & Rochon, A. (2019). Leading indicators
in occupational health and safety management systems in healthcare: A quasi-
experimental longitudinal study. Journal of Occupational and Environmental Medicine.
Journal of Occupational and Environmental Medicine, 61(12), e486–e496.
https://doi.org/10.1097/JOM.0000000000001738
Anderman, E. M., Anderman, L. H., Yough, M. S., & Gimbert, B. G. (2010). Value-added
models of assessment: Implications for motivation and accountability. Educational
Psychologist, 45(2), 123–137. https://doi.org/10.1080/00461521003703045
168

Arias, J. D. (2004). Recent perspectives in the study of motivation: Goal orientation theory.
Revista Electronica de Investigacion Psicoeducativa, 2(1), 35–62.
Arvey, R. D., & Murphy, K. R. (n.d.). Performance evaluation in work settings. Annual Review
Of Psychology, 49(1), 141–168). https://doi.org/10.1146/annurev.psych.49.1.141
Auerbach, A. J. (1981). The index of leading indicators:” Measurement without theory,” twenty-
five years later. National Bureau of Economic Research.
https://www.nber.org/papers/w761 https://doi.org/10.3386/w0761
Bahhouth, V., Maysami, R., & Gonzalez, R. (2014). Are financial measures leading indicators to
firm performance? International Journal of Business, Accounting, & Finance, 8(2), 37.
Ball, A. J. (2015). Identification of leading indicators for producibility risk in early-stage
aerospace product development [Unpublished master’s thesis]. Massachusetts Institute of
Technology, Cambridge, MA. https://dspace.mit.edu/handle/1721.1/98976?show=full
Barrados, M., & Blain, J. S. (2013). Improving program results through the use of predictive
operational performance indicators: A Canadian case study. The American Journal of
Evaluation, 34(1), 45–56. https://doi.org/10.1177/1098214012464426
Beck, C. T. (1993). Qualitative research: The evaluation of its credibility, fittingness, and
auditability. Western Journal of Nursing Research, 15(2), 263–266.
https://doi.org/10.1177/019394599301500212
Bennett, J., & Foster, P. (2005). Predicting progress: The use of leading indicators in
occupational safety and health. Policy and Practice in Health and Safety, 3(2), 77–90.
Bensimon, E. M., & Harris, F., III. (2007). Accountability, equity, and practitioner learning and
change. Metropolitan Universities, 18(3), 28–45.
169

Berger, B. (2014). Read my lips: Leaders, supervisors, and culture are the foundations of
strategic employee communications. Research Journal of the Institute for Public
Relations, 1(1), 1–17.
Berk, J. M., & Bikker, J. A. (1995). International interdependence of business cycles in the
manufacturing industry: The use of leading indicators for forecasting and analysis.
Journal of Forecasting, 14(1), 1–23. https://doi.org/10.1002/for.3980140102
Bernthal, J. P. (2020). A qualitative descriptive study of job expectations, job satisfaction, and
retention among fixed-wing marine pilots (Publication No. 28262605) [Doctoral
dissertation]. ProQuest Dissertations and Theses Global.
Bhalerao, H., & Kumar, S. (2016). Role of emotional intelligence in leaders on the commitment
level of employees: A study in information technology and manufacturing sector in India.
Business Perspectives and Research, 4(1), 41–53.
https://doi.org/10.1177/2278533715605434
Blair, E. (2015). A reflexive exploration of two qualitative data coding techniques. Journal of
Methods and Measurement in the Social Sciences, 6(1), 14–29.
Bogdan, R., & Biklen, S. K. (1997). Qualitative research for education. Allyn & Bacon Boston,
MA.
Bong, M. (2001). Role of self-efficacy and task-value in predicting college students’ course
performance and future enrollment intentions. Contemporary Educational Psychology,
26(4), 553–570. https://doi.org/10.1006/ceps.2000.1048
Bong, M., & Clark, R. E. (1999). Comparison between self-concept and self-efficacy in
academic motivation research. Educational Psychologist, 34(3), 139–153.
https://doi.org/10.1207/s15326985ep3403_1
170

Borgogni, L., Dello Russo, S., & Latham, G. P. (2011). The Relationship of Employee
Perceptions of the Immediate Supervisor and Top Management With Collective Efficacy.
Journal of Leadership & Organizational Studies, 18(1), 5–13.
https://doi.org/10.1177/1548051810379799
Brandsma, G. J., & Schillemans, T. (2013). The accountability cube: Measuring accountability.
Journal of Public Administration: Research and Theory, 23(4), 953–975.
https://doi.org/10.1093/jopart/mus034
Brett, J. F., & VandeWalle, D. (1999). Goal orientation and goal content as predictors of
performance in a training program. The Journal of Applied Psychology, 84(6), 863–873.
Brinkerhoff, R. O. (2006). Increasing impact of training investments: an evaluation strategy for
building organizational learning capability. Industrial and Commercial Training, 38(6),
302–307.
Britner, S. L., & Pajares, F. (2006). Sources of science self-efficacy beliefs of middle school
students. Journal of Research in Science Teaching, 43(5), 485–499.
https://doi.org/10.1002/tea.20131
Camacho, M., & Perez-Quiros, G. (2002). This is what the leading indicators lead. Journal of
Applied Econometrics, 17(1), 61–80. https://doi.org/10.1002/jae.641
Cheema, J. M., Cheema, M. J., & Bajwa, M. A. (n.d.). Developing predictive quality
scorecards:A futuristic approach to quality management. European Organization for
Quality. https://www.eoq.hu/iaq/wqf/papers/b5-4-cheema.pdf
Chen, P.-S. (2011). Finding quality responses: The problem of low-quality survey responses and
its impact on accountability measures. Research in Higher Education, 52(7), 659–674.
https://doi.org/10.1007/s11162-011-9217-4
171

Chernesky, R. H., & Israel, M. K. (2009). Job expectations and intention to leave in a state child
welfare agency. Journal of Public Child Welfare, 3(1), 23–39.
Childress, S., & Elmore, R. (2006). How to manage urban school districts. Harvard Business
Review. https://www.clevelandmetroschools.org/cms/lib/OH01915844/Centricity/
Domain/6750/HQresources/Article%20-%20How%20to%20manage%20urban%20
school%20district.pdf
Chou, H.-H. (2015). Multiple-technique approach for improving a performance measurement
and management system: Action research in a mining company. Engineering
Management Journal, 27(4), 203–217. https://doi.org/10.1080/10429247.2015.1104204
Christensen, C. M., Cook, S., & Hall, T. (2006, January 16). What customers want from your
products. Harvard Business School Newsletter: Working Knowledge.
https://hbswk.hbs.edu/item/what-customers-want-from-your-products
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Information Age Publishing.
Cleary, S. M., Molyneux, S., & Gilson, L. (2013). Resources, attitudes and culture: An
understanding of the factors that influence the functioning of accountability mechanisms
in primary health care settings. BMC Health Services Research, 13(1), 320.
https://doi.org/10.1186/1472-6963-13-320
Cole, J. S., Bergin, D. A., & Whittaker, T. A. (2008). Predicting student achievement for low
stakes tests with effort and task value. Contemporary Educational Psychology, 33(4),
609–624. https://doi.org/10.1016/j.cedpsych.2007.10.002
Collins, J. (1999). Turning goals into results (Harvard Business Review classics): The power of
catalytic mechanisms. Harvard Business Review Press.
172

Collins, J. C. (2005). Good to great and the social sectors: Why business thinking is not the
answer. Harper
Compton, M. (n.d.). Personal accountability in a team. Women in Business, 59(4), 14–17.
Connelly, L. M. (2016). Trustworthiness in qualitative research. Medsurg Nursing, 25(6), 435–
436.
Corbin J., & Strauss A (2015). Basics of qualitative research: Techniques and procedures for
developing grounded theory (4th ed.) Sage.
Craft, R. C., & Leake, C. (2002). The Pareto principle in organizational decision making.
Management Decision, 40(8), 729–733. https://doi.org/10.1108/00251740210437699
Creswell, J. W. (2010). Mapping the developing landscape of mixed methods research. SAGE
handbook of mixed methods in social & behavioral research, 2, 45–68.
Creswell, J. W, Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed
methods approaches (5th ed). Sage Publications
Decoene, V., & Bruggeman, W. (2006). Strategic alignment and middle-level managers’
motivation in a balanced scorecard setting. International Journal of Operations &
Production Management, 26(4), 429–448. https://doi.org/10.1108/01443570610650576
DeGeest, D., & Brown, K. G. (2011). The role of goal orientation in leadership development.
Human Resource Development Quarterly, 22(2), 157–175.
https://doi.org/10.1002/hrdq.20072
Delves, D. P. (2004). Stock options and the new rules of corporate accountability : measuring,
managing, and rewarding executive performance. McGraw-Hill.
173

Dembo, M. H., & Eaton, M. J. (2000). Self-Regulation of Academic Learning in Middle-Level
Schools. The Elementary School Journal, 100(5), 473–490.
https://doi.org/10.1086/499651
Dempsey, S. J., Gatti, J. F., Grinnell, D. J., & Cats-Baril, W. L. (1997). The use of strategic
performance variables as leading indicators in financial analyst’s forecasts. Journal of
Financial Statement Analysis, 2(4), 61. https://doi.org/10.2139/ssrn.2346
Denzin, N. K., & Lincoln, Y. S. (2011). The SAGE Handbook of Qualitative Research. SAGE.
de Oliveira Andreotti, V., Stein, S., Ahenakew, C., & Hunt, D. (2015). Mapping interpretations
of decolonization in the context of higher education. Decolonization: Indigeneity,
Education & Society, 4(1).
https://jps.library.utoronto.ca/index.php/des/article/view/22168
DeShon, R. P., & Gillespie, J. Z. (2005). A motivated action theory account of goal orientation.
The Journal of Applied Psychology, 90(6), 1096–1127. https://doi.org/10.1037/0021-
9010.90.6.1096
Diebold, F. X., & Rudebusch, G. D. (1989). Scoring the Leading Indicators. The Journal of
Business, 62(3), 369–391. https://doi.org/10.1086/296467
Dixon, J. R., Arnold, P., Heineke, J., Kim, J. S., & Mulligan, P. (1994). Business process
reengineering: Improving in new strategic directions. California Management Review,
36(4), 93–108. https://doi.org/10.2307/41165768
Dong, Y., Bartol, K. M., Zhang, Z.-X., & Li, C. (2017). Enhancing employee creativity via
individual skill development and team knowledge sharing: Influences of dual-focused
transformational leadership. Journal of Organizational Behavior, 38(3), 439–458.
174

Dorfman, H. A. (2017). The mental ABCs of pitching: A Handbook for performance
enhancement. Rowman & Littlefield.
Dowd, A. C., & Shieh, L. T. (2013). Community college financing: Equity, efficiency, and
accountability. The NEA Almanac of Higher Education, 37–65.
Drucker, P. F. (2002). The effective executive. HarperBusiness Essentials.
Duin, A. H., Baer, L. L., & Ramaley, J. (2007, April 20–23). Leading vs. lagging: A framework
for smart change [Paper presentation]. Annual Conference of the Higher Learning
Commission, Chicago, IL, United States.
https://www.researchgate.net/profile/Judith_Ramaley/publication/265669188_Leading_v
s_Lagging_A_Framework_for_Smart_Change/links/5592cdff08ae16f493ee404b.pdf
Dwyer, R. J. (2009). “Keen to be green” organizations: A focused rules approach to
accountability. Management Decision, 47(7), 1200–1216.
https://doi.org/10.1108/00251740910978377
Ebrahim, A. (2003). Accountability in practice: Mechanisms for NGOs. World Development,
31(5), 813–829. https://doi.org/10.1016/S0305-750X(03)00014-7
Eccles, J. S. (2005). Subjective task value and the Eccles et al. model of achievement-related
choices. In A. J. Elliot & C. S. Dweck (Eds.), Handbook of Competence and Motivation,
105–121. The Guilford Press.
Eifler, T., Ebro, M., Howard, T. J., & Others. (2013). A classification of the industrial relevance
of robust design methods. DS 75-9: Proceedings of the 19th International Conference on
Engineering Design (ICED13), Design for Harmonies, Vol. 9: Design Methods and
Tools, Seoul, Korea, 19-22.08. 2013, 427–436.
175

Faisal, M. N. (2009). Prioritization of Risks in Supply Chains. In T. Wu & J. Blackhurst (Eds.),
Managing Supply Chain Risk and Vulnerability: Tools and Methods for Supply Chain
Decision Makers (pp. 41–66). Springer London.
Fichtner, F., Rueffer, R., & Schnatz, B. (2009). Leading indicators in a globalised world.
https://doi.org/10.2139/ssrn.1516168
Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998). Relationships of
goal orientation, metacognitive activity, and practice strategies with learning outcomes
and transfer. The Journal of Applied Psychology, 83(2), 218–233.
https://doi.org/10.1037/0021-9010.83.2.218
Fossey, E., Harvey, C., McDermott, F., & Davidson, L. (2002). Understanding and evaluating
qualitative research. The Australian and New Zealand Journal of Psychiatry, 36(6), 717–
732. https://doi.org/10.1046/j.1440-1614.2002.01100.x
Frankel, J., & Saravelos, G. (2012). Can leading indicators assess country vulnerability?
Evidence from the 2008–09 global financial crisis. Journal of International Economics,
87(2), 216–231. https://doi.org/10.1016/j.jinteco.2011.12.009
Frink, D. D., & Klimoski, R. J. (2004). Advancing accountability theory and practice:
Introduction to the human resource management review special edition. Human Resource
Management Review, 14(1), 1–17. https://doi.org/10.1016/j.hrmr.2004.02.001
Froschheiser, L. (n.d.). The accountability leader: Inspiring your team to exceed its goals.
Contract Management, 49(7), 10–12.
Galindo, L. (2010). The power of accountability. Leader to Leader, 2010(56), 17–20.
https://doi.org/10.1002/ltl.409
176

Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist,
36(1), 45–56. https://doi.org/10.1207/S15326985EP3601_5
Gaseau, M. (2007). MIT develops measures to predict performance of complex systems. MIT
News. Retrieved Oct, 22, 2009.
Gershwin, S. B. (2000). Design and operation of manufacturing systems: the control-point
policy. IIE Transactions, 32(10), 891–906.
Giessner, S. R., van Knippenberg, D., van Ginkel, W., & Sleebos, E. (2013). Team-oriented
leadership: The interactive effects of leader group prototypicality, accountability, and
team identification. The Journal of Applied Psychology, 98(4), 658–667.
Givehchi, S., Hemmativaghef, E., & Hoveidi, H. (2017). Association between safety leading
indicators and safety climate levels. Journal of Safety Research, 62, 23–32.
Glesne, C. (2016). Becoming qualitative researchers: An Introduction (5th ed).Pearson.
Golafshani, N. (2003). Understanding reliability and validity in qualitative research. Qualitative
Report, 8(4), 597–607.
Goldenberg, C., Gallimore, R., Reese, L., & Garnier, H. (2001). Cause or effect? A longitudinal
study of immigrant Latino parents’ aspirations and expectations, and their children’s
school performance. American Educational Research Journal, 38(3), 547–582.
https://doi.org/10.3102/00028312038003547
Goldratt, E. M., & Cox, J. (2004). The goal: A process of ongoing improvement (3rd ed.). North
River Press.
177

Grabowski, M., Ayyalasomayajula, P., Merrick, J., Harrald, J. R., & Roberts, K. (2007). Leading
indicators of safety in virtual organizations. Safety Science, 45(10), 1013–1043.
https://doi.org/10.1016/j.ssci.2006.09.007
Graff, M., & Etter, R. (2004). Coincident and leading indicators of manufacturing industry:
Sales, production, orders and inventories in Switzerland. OECD Journal of Business
Cycle Measurement and Analysis, 1(1), 109–131.
Greising, C. H. (2010). Accountability at the top. Trustee: The Journal for Hospital Governing
Boards, 63(9), 1–3.
Griffin, M. A., & Neal, A. (2000). Perceptions of safety at work: A framework for linking safety
climate to safety performance, knowledge, and motivation. Journal of Occupational
Health Psychology, 5(3), 347–358. https://doi.org/10.1037/1076-8998.5.3.347
Grosfeld-Nir, A., Ronen, B., & Kozlovsky, N. (2007). The Pareto managerial principle: When
does it apply? International Journal of Production Research, 45(10), 2317–2325.
https://doi.org/10.1080/00207540600818203
Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In N. K.
Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 105–117). Sage
Publications, Inc.
Guo, B. H. W., & Yiu, T. W. (2016). Developing leading indicators to monitor the safety
conditions of construction projects. Journal of Management in Engineering, 32(1),
04015016.
Habibi, M., Kermanshachi, S., & Safapour, E. (2018, April 2–4). Engineering, procurement and
construction cost and schedule performance leading indicators: State-of-the-art review
178

[Paper presentation] Construction Research Congress 2018, New Orleans, LA, United
States. https://doi.org/10.1061/9780784481271.037
Haight, J. M., & Thomas, R. E. (2003). Intervention effectiveness research: A review of the
literature on leading indicators. Chemical Health and Safety, 10(2), 21–25.
https://doi.org/10.1016/S1074-9098(02)00454-9
Hall, A. T., Bowen, M. G., Ferris, G. R., Royle, M. T., & Fitzgibbons, D. E. (2007). The
accountability lens: A new way to view management issues. Business Horizons, 50(5),
405–413.
Hammonds, K. H. (2003). How to play Beane ball. 70, 84.
Han, Y. (2020). The impact of accountability deficit on agency performance: performance-
accountability regime. Public Management Review, 22(6), 927–948.
Harris, F., & Bensimon, E. M. (2007). The equity scorecard: A collaborative approach to assess
and respond to racial/ethnic disparities in student outcomes. New Directions for Student
Services, 2007(120), 77–84.
Hentschke, G. C., & Wohlstetter, P. (2004). Cracking the code of accountability. University of
Southern California Urban Education, 17–19.
Herrera, I. A., & Hovden, J. (2008, October 28–30). Leading indicators applied to maintenance
in the framework of resilience engineering: A conceptual approach [Paper presentation]
Third Resilience Engineering Symposium, Antibes Juan-les-Pins, France.
Hickey, D. L. (2017). Supervisors as leading indicators of safety performance. Professional
Safety, 62(04), 41–44.
Hill, C. (2017). “Houston, we have a problem:” 3 steps toward achieving accountability. Aircraft
Maintenance Technology, 28(9), 58–59.
179

Hill, E. E., Nguyen, T. H., Shaha, M., Wenzel, J. A., DeForge, B. R., & Spellbring, A. M.
(2009). Person-environment interactions contributing to nursing home resident falls.
Research in Gerontological Nursing, 2(4), 287–296. https://doi.org/10.3928/19404921-
20090527-02
Hirst, G., Van Knippenberg, D., & Zhou, J. (2009). A cross-level perspective on employee
creativity: Goal orientation, team learning behavior, and individual creativity. Academy
of Management Journal. Academy of Management Journal, 52(2), 280–293.
https://doi.org/10.5465/amj.2009.37308035
Hoke, T. (2014). The Importance of Personal Accountability. Civil Engineering, 84(10), 42.
Hopper, T., & Westrup, C. (2008). World class manufacturing and accountability. Journal of
Accounting & Organizational Change, 4(2), 97–135.
https://doi.org/10.1108/18325910810878937
Howie, J. G. (1996). Addressing the credibility gap in general practice research: Better theory;
more feeling; less strategy. The British Journal of General Practice, 46(409), 479–481.
Husman, J., Derryberry, W. P., Crowson, H. M., & Lomax, R. (2004). Instrumentality, task
value, and intrinsic motivation: Making sense of their independent interdependence.
Contemporary Educational Psychology, 29(1), 63–76. https://doi.org/10.1016/S0361-
476X(03)00019-5
Hsieh, P.-H., Sullivan, J. R., & Guerra, N. S. (2007). A closer look at college students: Self-
efficacy and goal orientation. Journal of Advanced Academics, 18(3), 454–476.
İpek, C. (2010). Predicting organizational commitment from organizational culture in Turkish
primary schools. Asia Pacific Education Review, 11(3), 371–385.
180

Ittner, C. D., & Larcker, D. F. (1998). Are nonfinancial measures leading indicators of Financial
performance? An analysis of customer satisfaction. Journal of Accounting Research, 36,
1–35. https://doi.org/10.2307/2491304
Jha, S., & Bhattacharyya, S. S. (2020). Moderated mediation analysis of leader technology
orientation: A study of operations and manufacturing leaders of India. Journal of
Operations and Strategic Planning, 3(1), 58–80.
https://doi.org/10.1177/2516600X20930946
Joannides, V. (n.d.). Accounterability and the problematics of accountability. Critical
Perspectives on Accounting, 23(3), 244–257. https://doi.org/10.1016/j.cpa.2011.12.008
Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed
methods research. Journal of Mixed Methods Research, 1(2), 112–133.
https://doi.org/10.1177/1558689806298224
Jones, T. C., & Dugdale, D. (1995). Manufacturing accountability. In A. J. Berry, J. Broadbent,
& D. Otley (Eds.), Management control: Theories, issues and practices (pp. 299–323).
Macmillan Education UK. https://doi.org/10.1007/978-1-349-23912-2_19
Joo, B.-K. (2012). Leader–member exchange quality and in-role job performance: The
moderating role of learning organization culture. Journal of Leadership &
Organizational Studies, 19(1), 25–34. https://doi.org/10.1177/1548051811422233
Kaifi, B. A. (2012). The 4 disciplines of execution. Journal of Business Studies Quarterly, 3(4),
163.
Kamuf, P. (2007). Accounterability. Textual Practice, 21(2), 251–266.
https://doi.org/10.1080/09502360701264428
181

Kaplan, A., & Maehr, M. L. (2007). The contributions and prospects of goal orientation theory.
Educational Psychology Review, 19(2), 141–184. https://doi.org/10.1007/s10648-006-
9012-5
Kaplan, R. S., & Norton, D. P. (1998). Putting the balanced scorecard to work. The Economic
Impact of Knowledge, 27(4), 315–324.
Kiatcharoenpol, T., Laosirihongthong, T., Chaiyawong, P., & Glincha-em, C. (2015). A study of
critical success factors and prioritization by using analysis hierarchy process in lean
manufacturing implementation for Thai SMEs. In Proceedings of the 5th International
Asia Conference on Industrial Engineering and Management Innovation (IEMI2014) (pp.
295–298). Atlantis Press. https://doi.org/10.2991/978-94-6239-100-0_54
Kimberlin, C. L., & Winterstein, A. G. (2008). Validity and reliability of measurement
instruments used in research. American Journal of Health-System Pharmacy, 65(23),
2276–2284. https://doi.org/10.2146/ajhp070364
Kirkpatrick, D., & Kirkpatrick, J. (2006). Evaluating training programs: The four levels. Berrett-
Koehler Publishers.
Koch, R. (2000). The 80/20 principle: The secret of achieving more with less. Nicholas Brealey
Publishing.
Koch, R. (2013). The 80/20 manager: the secret to working less and achieving more (1st ed.).
Little, Brown.
Koponen, T., Aro, T., & Ahonen, T. (2009). Conceptual knowledge‐based strategy training in
single‐digit calculation: a single case intervention study in a child with specific language
impairment. European Journal of Special Needs Education, 24(3), 259–275.
182

Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice,
41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2
Kravchenko, M., Pigosso, D. C. A., & McAloone, T. C. (2019). Towards the ex-ante
sustainability screening of circular economy initiatives in manufacturing companies:
Consolidation of leading sustainability-related performance indicators. Journal of
Cleaner Production, 241, Article 118318. https://doi.org/10.1016/j.jclepro.2019.118318
Krueger, R. A., & Casey, M. A. (2002). Designing and conducting focus group interviews.
Citeseer. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.607.4701
&rep=rep1&type=pdf#page=10
Latham, G. P., & Locke, E. A. (2006). Enhancing the benefits and overcoming the pitfalls of
goal setting. Organizational Dynamics, 35(4), 332–340.
https://doi.org/10.1016/j.orgdyn.2006.08.008
Lawlor, K. B. (2012). Smart goals: How the application of smart goals can contribute to
achievement of student learning outcomes. Developments in Business Simulation and
Experiential Learning: Proceedings of the Annual ABSEL Conference, 39. https://absel-
ojs-ttu.tdl.org/absel/index.php/absel/article/view/90
Leech, N. L., & Onwuegbuzie, A. J. (2009). A typology of mixed methods research designs.
Quality & Quantity, 43(2), 265–275. https://doi.org/10.1007/s11135-007-9105-3
Lencioni, P. (2012). The advantage: Why organizational health trumps everything else in
business (1st ed.). Jossey-Bass.
Lerang, M. S., Ertesvåg, S. K., & Havik, T. (2019). Perceived classroom interaction, goal
orientation and their association with social and academic learning outcomes.
Scandinavian Journal of Educational Research, 63(6), 913–934.
183

Lichtman, M. (2013). Qualitative Research for the Social Sciences. SAGE Publications.
Liem, A. D., Lau, S., & Nie, Y. (2008). The role of self-efficacy, task value, and achievement
goals in predicting learning strategies, task disengagement, peer relationship, and
achievement outcome. Contemporary Educational Psychology, 33(4), 486–512.
https://doi.org/10.1016/j.cedpsych.2007.08.001
Lincoln, Y. S. (1995). Emerging criteria for quality in qualitative and interpretive research.
Qualitative Inquiry: QI, 1(3), 275–289.
Malik, M. A. R., Choi, J. N., & Butt, A. N. (2019). Distinct effects of intrinsic motivation and
extrinsic rewards on radical and incremental creativity: The moderating role of goal
orientations. Journal of Organizational Behavior, 40(9-10), 1013–1026.
Manuele, F. A. (2009). Leading & lagging indicators. Professional Safety, 54(12), 28.
Marcellino, M. (2005). Leading indicators: What have we learned? (Working paper no. 286).
Innocenzo Gasparini Institute for Economic Research.
https://doi.org/10.2139/ssrn.695721
Marx, L. M., & Squintani, F. (2009). Individual accountability in teams. Journal of Economic
Behavior & Organization, 72(1), 260–273. https://doi.org/10.1016/j.jebo.2009.05.009
McChesney, C., Covey, S., & Huling, J. (2012). The 4 disciplines of execution: Achieving your
wildly important goals. Simon and Schuster.
McCollum, D. L., & Kajs, L. T. (2007). Applying goal orientation theory in an exploration of
student motivations in the domain of educational leadership. Educational Research
Quarterly, 31(1), 45–59.
184

McCrudden, M. T., Schraw, G., & Hartley, K. (2006). The effect of general relevance
instructions on shallow and deeper learning and reading time. Journal of Experimental
Education, 74(4), 291–310.
Meece, J. L., Anderman, E. M., & Anderman, L. H. (2006). Classroom goal structure, student
motivation, and academic achievement. Annual Review of Psychology, 57, 487–503.
https://doi.org/10.1146/annurev.psych.56.091103.070258
Mehra, A., Dixon, A. L., Brass, D. J., & Robertson, B. (2006). The social network ties of group
leaders: Implications for group performance and leader reputation. Organization Science,
17(1), 64–79. https://doi.org/10.1287/orsc.1050.0158
Merchant, K. A., & Otley, D. T. (2006). A review of the literature on control and accountability.
Handbooks of Management Accounting Research, 2, 785–802.
https://doi.org/10.1016/S1751-3243(06)02013-X
Merriam, S. B., & Tisdell, E. J. (2016). Designing your study and selecting a sample. Qualitative
Research: A Guide to Design and Implementation, 67(1), 73–104.
Mohamad, M. M., Sulaiman, N. L., Sern, L. C., & Salleh, K. M. (2015). Measuring the Validity
and Reliability of Research Instruments. Procedia: Social and Behavioral Sciences, 204,
164–171. https://doi.org/10.1016/j.sbspro.2015.08.129
Molinaro, V. (2015). Driving leadership accountability: a critical business priority for HR
leaders. Journal of the Human Resource Planning Society, 38(3), 32.
Molnár, M. (2008). The accountability paradigm: Standards of excellence. Public Management
Review, 10(1), 127–137. https://doi.org/10.1080/14719030701763245
Moore, G. H. (1983). Business Cycles, Inflation, and Forecasting (2nd ed.). Ballinger.
185

Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling
psychology. Journal of Counseling Psychology, 52(2), 250–260.
https://doi.org/10.1037/0022-0167.52.2.250
Mosonik, J. C. (2017). The influence of participatory communication in promoting
accountability and transparency of Constituency Development Fund in Emurua Dikirr
Constituency [University of Nairobi].
http://erepository.uonbi.ac.ke/handle/11295/102571
Mousavi, S. S., Cudney, E. A., & Trucco, P. (2018). Towards a framework for steering safety
performance: A review of the literature on leading indicators. In P. Arezes (Ed.),
Advances in Safety Management and Human Factors (Vol. 604, pp. 195–204). Springer
International Publishing. https://doi.org/10.1007/978-3-319-60525-8_21
Muchiri, P., Pintelon, L., Gelders, L., & Martin, H. (2011). Development of maintenance
function performance measurement framework and indicators. International Journal of
Production Economics, 131(1), 295–302. https://doi.org/10.1016/j.ijpe.2010.04.039
Mulvaney, H. R., R., Rebecca, & R. (n.d.). The trend toward accountability: What does it mean
for HR managers? Human Resource Management Review, 16(3), 431–442.
https://doi.org/10.1016/j.hrmr.2006.06.003
Muthiah, K. M. N., & Huang, S. H. (2006). A review of literature on manufacturing systems
productivity measurement and improvement. International Journal of Industrial and
Systems Engineering, 1(4), 461–484. https://doi.org/10.1504/IJISE.2006.010387
Naji, G. M. A., Isha, A. S. N., Al-Mekhlafi, A.-B. A., Sharafaddin, O., & Ajmal, M. (2020).
Implementation of leading and lagging indicators to improve safety performance in the
186

upstream oil and gas industry. Journal of Toxicology and Environmental Health. Part B,
Critical Reviews, 7(14), 2020.
National Research Council. (1997). Enhancing organizational performance. National Academy
Press. https://doi.org/10.17226/5128
Neely, A., Gregory, M., & Platts, K. (2005). Performance measurement system design: A
literature review and research agenda. International Journal of Operations & Production
Management, 25(12), 1228.
Neuville, S., Frenay, M., & Bourgeois, E. (2007). Task value, self-efficacy and goal orientations:
Impact on self-regulated learning, choice and performance among university students.
Psychologica Belgica, 47(1), 95. https://doi.org/10.5334/pb-47-1-95
Newport, C. (2012). So good they can’t ignore you : why skills trump passion in the quest for
work you love (1st ed.). Business Plus.
Newport, C. (2016). Deep work: Rules for focused success in a distracted world. Grand Central
Publishing.
Nielsen, K., Yarker, J., Brenner, S.-O., Randall, R., & Borg, V. (2008). The importance of
transformational leadership style for the well-being of employees working with older
people. Journal of Advanced Nursing, 63(5), 465–475. https://doi.org/10.1111/j.1365-
2648.2008.04701.x
Nieminen, L., Biermeier-Hanson, B., & Denison, D. (2013). Aligning leadership and
organizational culture: The leader–culture fit framework for coaching organizational
leaders. Consulting Psychology Journal: Practice and Research, 65(3), 177–198.
Nir, A. (n.d.-b). The Pareto managerial principle: When does it apply? International Journal of
Production Research, 45(10), 2317–2325. https://doi.org/10.1080/00207540600818203
187

Nollet, J., Calvi, R., Audet, E., & Côté, M. (2008). When excessive cost savings measurement
drowns the objectives. Journal of Purchasing and Supply Management, 14(2), 125–135.
Oconnell, V., & Osullivan, D. (2016). Are nonfinancial metrics good leading indicators of Future
financial performance? MIT Sloan Management Review, 57(4), 21–23.
Oedewald, P., & Reiman, T. (2003). Core task modelling in cultural assessment: a case study in
nuclear power plant maintenance. Cognition, Technology & Work, 5(4), 283–293.
Ordóñez, L. D., Schweitzer, M. E., Galinsky, A. D., & Bazerman, M. H. (2009). Goals gone
wild: The systematic side effects of overprescribing goal setting. The Academy of
Management Perspectives, 23(1), 6–16. https://doi.org/10.5465/amp.2009.37007999
Osanloo, A., & Grant, C. (2016). Understanding, selecting, and integrating a theoretical
framework in dissertation research: Creating the blueprint for your “house.”
Administrative Issues Journal: Connecting Education, Practice, and Research, 4(2), 7.
Palinkas, L. A., Mendon, S. J., & Hamilton, A. B. (2019). Innovations in mixed methods
Evaluations. Annual Review of Public Health, 40, 423–442.
Parella, K. (2014). Outsourcing corporate accountability. Washington Law Review, 89(3), 747–
818.
Parida, A., Kumar, U., Galar, D., & Stenström, C. (2015). Performance measurement and
management for maintenance: A literature review. Journal of Quality in Maintenance
Engineering, 21(1), 2–33. https://doi.org/10.1108/JQME-10-2013-0067
Patton, M. Q. (2005). Qualitative research. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia
of Statistics in Behavioral Science. Wiley. https://doi.org/10.1002/0470013192.bsa514
188

Pawłowska, Z. (2015). Using lagging and leading indicators for the evaluation of occupational
safety and health performance in industry. International Journal of Occupational Safety
and Ergonomics, 21(3), 284–290. https://doi.org/10.1080/10803548.2015.1081769
Pink, D. H. (2009). Drive: The surprising truth about what motivates us. Riverhead Books.
Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientation in learning
and achievement. Journal of Educational Psychology, 92(3), 544–555.
https://doi.org/10.1037/0022-0663.92.3.544
Polit, D. F., & Beck, C. T. (2004). Nursing research: Principles and methods. Lippincott
Williams & Wilkins.
Polit, D. F., & Beck, C. T. (2012). Nursing research: Generating and assessing evidence for
nursing practice. Wolters Kluwer.
Rad, A. M. M., & Yarmohammadian, M. H. (2006). A study of relationship between managers’
leadership style and employees’ job satisfaction. International Journal of Health Care
Quality Assurance Incorporating Leadership in Health Services, 19(2-3), xi–xxviii.
Radosevich, D. J., Allyn, M. R., & Yun, S. (2007). Goal orientation and goal setting: Predicting
performance by integrating four-factor goal orientation theory with goal setting
processes. Seoul Journal of Business, 13(1), 21–47. http://s-
space.snu.ac.kr/bitstream/10371/1803/1/sjbv13n1_021.pdf
Rajendran, S. (2013). Enhancing construction worker safety performance using leading
indicators. Practice Periodical on Structural Design and Construction, 18(1), 45–51.
Rhodes, D. H., Valerdi, R., & Roedler, G. J. (2009). Systems engineering leading indicators for
assessing program and technical effectiveness. Journal of Systems Engineering and
Electronics, 12(1), 21–35.
189

Rivers, J. (2008). Relationship between parenting style and academic achievement and the
mediating influences of motivation, goal-orientation and academic self-efficacy.
https://fsu.digital.flvc.org/islandora/object/fsu%3A176346
Roberts, P., Priest, H., & Traynor, M. (2006). Reliability and validity in research. Nursing
Standard, 20(44), 41–45. https://doi.org/10.7748/ns2006.07.20.44.41.c6560
Roedler, G., Rhodes, D. H., Schimmoller, H., & Jones, C. (Eds.). (2007). Systems engineering
leading indicators guide: Version 2.0. y Massachusetts Institute of Technology, INCOSE,
and PSM. http://scripts.mit.edu/~seari/seari/documents/LAI-Leading-Indicators-Update-
20070618.pdf
Rogers, M. M. (2021). Teaching-to-learn: Its effects on conceptual knowledge learning in
university students. International Journal of Innovative Teaching and Learning in Higher
Education (IJITLHE), 2(1), 1–14.
Royle, M. T., & Hall, A. T. (2012). The relationship between McClelland’s theory of needs,
feeling individually accountable, and informal accountability for others. International
Journal of Management and Marketing Research, 5(1), 21–42.
Ruben, B., De Lisi, R., & Gigliotti, R. (2018). Academic leadership development programs:
Conceptual foundations, structural and pedagogical components, and operational
considerations. Journal of Leadership Education, 17(3), 241–254.
Rubin, H. J., & Rubin, I. S. (2012). Qualitative interviewing: The art of hearing data. SAGE
Publications.
Rueda, R. (2011). The 3 dimensions of improving student performance: Finding the right
solutions to the right problems. Teachers College Press.
Salkind, N. J. (2014). 100 questions (and answers) about statistics. SAGE Publications.
190

Scarpelli, J. A. (2015). Use of metrics selected based on lag correlation to provide leading
indicators of service performance degradation (USPTO Patent No. 9195563).
https://patentimages.storage.googleapis.com/ab/fd/4b/79aa42a70cbe49/US9195563.pdf
Schein, E. H. (2010). Organizational culture and leadership. John Wiley & Sons.
Schumpeter, J. A. (1949). Vilfredo Pareto (1848-1923). The Quarterly Journal of Economics,
63(2), 147–173. https://doi.org/10.2307/1883096
Selby, R. W. (2005). Measurement-driven dashboards enable leading indicators for requirements
and design of large-scale systems. 11th IEEE International Software Metrics Symposium
(METRICS’05), 10 pp. – 22.
Selwyn, L. (2014). Goleman, Daniel. Focus: The Hidden Driver of Excellence [Review of the
book Focus: The Hidden Driver of Excellence by D. Golman]. Library Journal, 139(3),
61.
Simpkins, S. D., Davis-Kean, P. E., & Eccles, J. S. (2006). Math and science motivation: A
longitudinal examination of the links between choices and beliefs. Developmental
Psychology, 42(1), 70–83. https://doi.org/10.1037/0012-1649.42.1.70
Sinek, S. (2009). Start with why: How great leaders inspire everyone to take action. Portfolio.
Sinelnikov, S., Inouye, J., & Kerper, S. (2015). Using leading indicators to measure occupational
health and safety performance. Safety Science, 72, 240–248.
Stake, R. E. (2013). Multiple case study analysis. Guilford Press.
Stavrou, N. A. M., Psychountaki, M., Georgiadis, E., Karteroliotis, K., & Zervas, Y. (2015).
Flow theory—goal orientation theory: Positive experience is related to athlete’s goal
orientation. Frontiers in Psychology, 6, Article 1499.
https://doi.org/10.3389/fpsyg.2015.01499
191

Steele-Johnson, D., Beauregard, R. S., Hoover, P. B., & Schmidt, A. M. (2000). Goal orientation
and task demand effects on motivation, affect, and performance. The Journal of Applied
Psychology, 85(5), 724–738.
Stevens, T., Hamman, D., & Olivarez, A., Jr. (2007). Hispanic students’ perception of White
teachers’ mastery goal orientation influences sense of school belonging. Journal of
Latinos and Education, 6(1), 55–70. https://doi.org/10.1080/15348430709336677
Stuart, M. (2003). Sources of subjective task value in sport: An examination of adolescents with
high or low value for sport. Journal of Applied Sport Psychology, 15(3), 239–255.
https://doi.org/10.1080/10413200305388
Sull, D., Homkes, R., & Sull, C. (2015). Why strategy execution unravels - and what to do about
it. Harvard Business Review, 93(3), 58–67.
Sundar, R., Balaji, A. N., & Kumar, R. M. S. (2014). A review on lean manufacturing
implementation techniques. Procedia Engineering, 97, 1875–1885.
Supovitz, J. A., Foley, E., & Mishook, J. (2012). In search of leading indicators in education.
Education Policy Analysis Archives, 20(19). https://doi.org/10.14507/epaa.v20n19.2012
Then, K. L., Rankin, J. A., & Ali, E. (2014). Focus group research: what is it and how can it be
used? Canadian Journal of Cardiovascular Nursing, 24(1), 16–22
Tulgan, B. (n.d.). Accountability should be the standard for employees as well as managers. In
Employment Relations Today, 34(2), p. 21–28. https://doi.org/10.1002/ert.20149
Tracy, S. J. (2019). Qualitative research methods: Collecting evidence, crafting analysis,
communicating impact. John Wiley & Sons.
192

Tyagi, P. K. (1982). Perceived organizational climate and the process of salesperson motivation.
JMR, Journal of Marketing Research, 19(2), 240–254.
https://doi.org/10.1177/002224378201900208
Ullah, Z. (2016). The impact of corporate accountability & transparency on the performance of
manufacturing sector firms listed on KSE. City University Research Journal, 6(1).
http://dx.doi.org/10.2139/ssrn.2756977
VandeWalle, D., Cron, W. L., & Slocum, J. W., Jr. (2001). The role of goal orientation following
performance feedback. The Journal of Applied Psychology, 86(4), 629–640.
https://doi.org/10.1037/0021-9010.86.4.629
Vassili, J. V. (2012). Accounterability and the problematics of accountability. Critical
perspectives on accounting, 23(3), 244–257.
Versteeg, K., Bigelow, P., Dale, A. M., & Chaurasia, A. (2019). Utilizing construction safety
leading and lagging indicators to measure project safety performance: A case study.
Safety Science, 120, 411–421. https://doi.org/10.1016/j.ssci.2019.06.035
Viseu, J., Jesus, S. N., Rus, C., & Canavarro, J. M. (2016). Relationship between teacher
motivation and organizational variables: A literature review. Paidéia.
https://www.scielo.br/j/paideia/a/3dNTP7Vq8dGnjncF3kLLw9R/abstract/?lang=en
Vukić, Đ., Martinčić-Ipšić, S., & Meštrović, A. (2020). Structural Analysis of Factual,
Conceptual, Procedural, and Metacognitive Knowledge in a Multidimensional
Knowledge Network. Complexity, 2020, 1–17. Advance online publication.
https://doi.org/10.1155/2020/9407162
193

Waters, T., Marzano, R. J., & McNulty, B. (2003). Balanced leadership: What 30 years of
research tells us about the effect of leadership on student achievement (A working paper).
ERIC. https://eric.ed.gov/?id=ED481972
Weaver, J. P., Chastain, R. J., DeCaro, D. A., & DeCaro, M. S. (2018). Reverse the routine:
Problem solving before instruction improves conceptual knowledge in undergraduate
physics. Contemporary Educational Psychology, 52, 36–47.
Weber, A., & Thomas, R. (2005). Key performance indicators. Measuring and Managing the
Maintenance Function Ivara Corporation, Burlington.
Weber, A. J. (2014). White paper: Compliance, safety, accountability: Assessing the new safety
measurement system and its implications--2013 update. https://trid.trb.org/view/1290398
Weiss, R. S. (1995). Learning from strangers: The art and method of qualitative interview
studies. Simon and Schuster.
Whitbeck, C. (1995). Truth and trustworthiness in research. Science and Engineering Ethics,
1(4), 403–416. https://doi.org/10.1007/BF02583258
Wiktorsson, M., Andersson, C., & Turunen, V. (2018). Leading towards high-performance
manufacturing—Enabling indicators in early R&D phases ensuring future KPI outcome.
Procedia Manufacturing, 25, 223–230. https://doi.org/10.1016/j.promfg.2018.06.077
Wößmann, L. (2007). International evidence on school competition, autonomy, and
accountability: A review. Peabody Journal of Education, 82(2-3), 473–497.
Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation.
SAGE.
Wolfe, R., Wright, P. M., & Smart, D. L. (2006). Radical HRM innovation and competitive
advantage: The Moneyball story. And in Alliance with the Society ….
194

https://onlinelibrary.wiley.com/doi/abs/10.1002/hrm.20100?casa_token=ythJduxrKgcAA
AAA:zaALmzwQEOg3u3UaaZYLcogOBHgY6zke_ktqrc-k4NThjhR3TDk-
r_XYrDkpTKHqC8q7bXIsKC_VfQ
Wolters, C. A., Yu, S. L., & Pintrich, P. R. (1996). The relation between goal orientation and
students’ motivational beliefs and self-regulated learning. Learning and Individual
Differences, 8(3), 211–238.
Worley, J. M., & Doolen, T. L. (2006). The role of communication and management support in a
lean manufacturing implementation. Management Decision, 44(2), 228–245.
https://doi.org/10.1108/00251740610650210
Yao, A. C., & Carlson, J. G. (1999). The impact of real-time data communication on inventory
management. International Journal of Production Economics, 59(1-3), 213–219.
https://doi.org/10.1016/S0925-5273(98)00234-5
Yates, W. B., & Keedwell, E. C. (2019). An analysis of heuristic subsequences for offline hyper-
heuristic learning. Journal of Heuristics, 25(3), 399–430.
Yorio, P. L., Haas, E. J., Bell, J. L., Moore, S. M., & Greenawald, L. A. (2020). Lagging or
leading? Exploring the temporal relationship among lagging indicators in mining
establishments 2006–2017. Journal of Safety Research, 74, 179–185. Advance online
publication. https://doi.org/10.1016/j.jsr.2020.06.018
Zemsky, R., & Iannozzi, M. (1995). [No title]. ERIC. https://eric.ed.gov/?id=ED382811
Zhang, T., Torney-Purta, J., & Barber, C. (2012). Students’ conceptual knowledge and process
skills in civic education: Identifying cognitive profiles and classroom correlates. Theory
& Research in Social Education, 40(1), 1–34.
195

Zheng, L., Baron, C., Esteban, P., Xue, R., & Zhang, Q. (2017). Considering the systems
engineering leading indicators to improve project performance measurement. IFAC-
PapersOnLine, 50(1), 13970–13975. https://doi.org/10.1016/j.ifacol.2017.08.2416
Zheng, L., Baron, C., Esteban, P., Xue, R., Zhang, Q., & Yang, S. (2019). Using leading
indicators to improve project performance measurement. Journal of Systems Science and
Systems Engineering, 28(5), 529–554. https://doi.org/10.1007/s11518-019-5414-z
Zokaei, K., & Simons, D. (2006). Performance improvements through implementation of lean
practices: a study of the UK red meat industry. International Food and Agribusiness
Management Review, 9, 30–53.
Zwetsloot, G., Leka, S., Kines, P., & Jain, A. (2020). Vision zero: Developing proactive leading
indicators for safety, health and wellbeing at work. Safety Science, 130, Article 104890.

196

Appendix A: Document Analysis Rubric
Knowledge influences
Assumed knowledge
influences
Document analysis
triangulation
Present Comments
Operations leader job
descriptions, look
for evidence of
requirement of
knowledge to
develop leading
indicators

PMC improvement
program charters

Internal training
documents,
continuous
improvement
modules

Performance
summaries with
trends over time
that show
adjustments to
approach

Motivational influences
Organizational
mission statements,
value declarations,
vision statements

197

Review examples of
internal documents
and
communications
and likely impact
on goal orientation
Forms of praise or
criticism public or
private
What praise is
attributed to
Consequences of
success or failure
Whether employees
ask for help

Organizational influences
Internal documents
that demonstrate
Accountability or
lack of
accountability

Internal documents
that demonstrate
Lack of
accountability
Acceptance of loafing

198

Appendix B: Survey Protocol, Scale 1–5
Table B1
Survey Questions
Question 1
Strongly
disagree
2 3 4 5
Strongly
agree
Parts of my job expectations
require me to know how to
develop leading indicators in
delivering lagging KOI’s (K-D)

How would you rate the level of
clarity in your job expectations
as an operations leader
regarding the development of
effective leading indicators? (K-
C)

Do you frequently adjust your
plans and approaches to ensure
you have effective leading
indicators in place? (K-M)

There is a high level of importance
in implementing leading
indicators as an operations
leader for the following
categories: Accountability (M-
T)

There is a high level of importance
in implementing leading
indicators as an operations
leader for the following
categories: Customer
satisfaction (M-T)

There is a high level of importance
in implementing leading
indicators as an operations
leader for the following
categories: Financial
performance (M-T)

199

Question 1
Strongly
disagree
2 3 4 5
Strongly
agree
There is a high level of importance
in implementing leading
indicators as an operations
leader for the following
categories:
Quality performance: (M, T)

There is a high level of importance
in implementing leading
indicators as an operations
leader for the following
categories: Inventory (M, T)

There is a high level of importance
in implementing leading
indicators as an operations
leader for the following
categories: Startup performance
(M, T)

I have a high level of commitment
in developing leading indicators
even if I make a lot of mistakes.
(M, GO)

In my job, I am highly motivated
to come up with different ways
of addressing any challenges.
(M, GO)

I like challenging problems and
difficult tasks, even if I make a
lot of mistakes. (M - GO)

I want to do better than my peers
in developing leading indicators.
(M, GO)

One of my main goals is to avoid
looking like I can’t do my work.
(M, GO)

On a scale of 1-5, with 1 being
autonomous and 5 being

200

Question 1
Strongly
disagree
2 3 4 5
Strongly
agree
authoritarian, how would score
the current level of
accountability provided by the
leadership of the organization?
(O, CM)
On a scale of 1-5, with 1 being
pessimistic and 5 being
optimistic, how would you score
the current level of
communication from leadership
in creating an environment that
facilitates accountability? (O,
CM)

How much autonomy are you
given to develop effective
leading indicators for my
organization? (O, CS)

What are the current steps of leading indicator development that you practice (select all that
apply from the following list): (K-P)
❏ Categorize indicators into the levels of effort needed to establish as H/M/L
❏ Develop a strategic plan
❏ Defining predictive and influenceable measures
❏ Track additional indicators
❏ Refine existing systems/behaviors/operations
❏ Align commitment for tracking and communication

Please place in order the steps of leading indicator development (order the list below):
❏ Categorize indicators into the levels of effort needed to establish as H/M/L
❏ Design indicators that are measurable, deployed, and result in outcome improvement
❏ Defining predictive and influenceable measures
❏ Collect comprehensive input from the team on proposed leading indicators
❏ Refine existing systems/behaviors/operations
❏ Align commitment for tracking and communication
❏ Organize lead indicators into two groups: behavior and outcomes

201

Appendix C: Interview Protocol
First, I wanted to thank you for your continued contributions as part of the PMC
Corporation. As we continue to see record-breaking growth in the medical segment, a key
opportunity of interest is to explore as a manufacturing function what are the challenges that are
hurdles in improving operations performance. This study represents us as an organization
continuing in our journey and seeking input from operations leaders such as yourself as to how
we can expedite the improvement. I also want to thank you for your willingness to take part in
the collection of information in support of this dissertation. I certainly understand that everyone
is very busy, especially as we recover from a global pandemic, so I greatly appreciate you setting
aside time for us to field questions relative to the improvement effort. This interview should take
no more than an hour. Will that still work for you or should we find an alternate day/time?
Before we move forward with the interview questions, I want to ensure you have an
understanding of the overall approach as well as specifics regarding what we are going to
discuss. I will be having conversations with operations leaders across the globe with the intent to
understand what are some of the challenges in implementing more leading operational measures.
The intent here is to understand across multiple manufacturing plants and regions what are some
of the barriers that are preventing us from reaching our full potential as an organization and
refining our overall focus so that our collective prioritization as a global team aligns with those
activities that are most impactful in a positive way. From our interview today, we want to ensure
we identify all knowledge, motivational, and organizational challenges that we will need to
overcome for the organization to move forward in improving operational performance. I will act
as the principal investigator for this study. This study is a part of the degree requirements for the
Doctor of Education Organizational Change and Leadership program offered at the University of
202

Southern California’s School of Education. Before we move forward, do you have any questions
regarding this particular study or the program itself?
This study will be strictly confidential and the findings will be coded accordingly. There
will be no particular names used or connected with the data collected. The data collected will not
be shared with anyone else in this organization. Information gathered will be captured via Zoom
and electronic transcript which will be redacted to protect confidentiality. As a reminder, this
study is voluntary and you are open to withdraw your consent at any time. In the event you are
uncomfortable or wish to withdraw your consent, please inform me directly or if needed please
feel free to escalate to our CHRO.
As a final note, I will be recording via Zoom for transcribing purposes. If at any point in
time you wish to comment “off the record”, please inform me and I will stop the recording. Your
participation is 100% voluntary. If no other concerns, can I gain your approval to start recording
and proceed with the interview?

Start Recording - Voice to subject current date and start time
Focus on Gaps of K, M, O

On the topic of leading indicators, we first would like to understand your knowledge of
this approach. Before we do, I would like to open with your role -
Opener: What is your primary responsibility in your current role?
1. Please define what is a leading indicator. (K-D)
a. Follow up: What are some organizational leading indicators you are aware of as
an operations leader?
203

2. Tell me what you know about how the operations leader role relates to leading
indicators? (K-D)
3. What is the difference between a leading indicator and a lagging indicator? (K-C)
4. If you had to explain leading indicators to someone what would you say? (K-C)
5. Tell me how you would approach identifying the right leading indicators for your
organization? (K-P)
6. If you had to explain to someone how to implement accountability mechanisms, what
would you say? (K-P)
7. If you had to explain to someone how to implement leading indicators, what would
you say? What would you do first? What would you do next? (K-P)
8. Tell me about a time, if at all, when you improved on a leading indicator? (K-M)
a. Follow-up question - How did you apply this experience to a similar case in the
future?
9. Tell me about a time you, if at all, when you had to adjust your approach and
measures to drive improvement? (K-M)
Now, I would like to understand what this concept means to you
10. How valuable is it to you to establish effective leading indicators? (M-T)
11. How important is it for you to communicate to your teams in delivering core KOI’s?
(M-T)
12. What do you see as the value of accountability mechanisms? (M-T)
13. Some would say it’s not valuable to look at leading indicators – what are your
thoughts? (M-T)
204

14. Tell me about a time when you didn’t hit the operations goal you had set? What did
you do in response to this? (M-G)
And finally, we would like to understand your perspective of the organization relative to leading
indicators
15. How resistant to change is the manufacturing organization in adopting alternative
approaches such as leading indicators to drive accountability and improve
performance? (O-CM)
16. How well do colleagues support one another in the organization and hold each other
accountable in developing leading indicators to drive improved operations
performance? (O-CM)
17. How well do you share the operations goals that align across functions such as
customer satisfaction, operating profit, startup performance from an accountability
perspective? (O-CS)
18. Can you talk about your freedom to use decision rights as it relates to your leading
indicator development accountability? (O-CS)
19. How does the organizational environment regarding accountability facilitate your
development of effective leading indicators, if at all? (O-CS)
20. Is there anything else regarding leading indicators that you would like to share
regarding the development of leading indicators that we have not already covered?
(O-CS)

205

Closing and follow-up
Thank you for your participation and time today. It is greatly appreciated and will be very
helpful in understanding how we can better accelerate our improvement as an organization. In
the event I have any follow-up questions, would I be able to reach back out to you?
Thanks again for your participation

206

Appendix D: Immediate Evaluation Instrument for Levels 1 and 2 Post Program
Implementation
The intent of this evaluation is to determine the effectiveness of the operations leader
leading indicator program training. Your feedback will be key in improving future program
training modules and is greatly appreciated. Thank you in advance for your participation.

Table D1
Evaluation Instrument
Strongly
disagree
Strongly
agree
Engagement (Level 1)
My engagement was
enhanced by the
program facilitator
1 2 3 4 5
My interest was held
by the activities I
performed in the
program
1 2 3 4 5
Relevance (Level 1)
What I learned will
help me on the job
1 2 3 4 5
I gained a better
understanding of
what is expected
of me in my role
1 2 3 4 5
Customer Satisfaction (Level 1)
I will recommend
this program to my
co-workers
1 2 3 4 5
Knowledge (Level 2)
207

The training I
received has
provided me the
skills to improve
my performance
1 2 3 4 5
I understand the
steps to follow to
implement leading
indicators
1 2 3 4 5
Attitude (Level 2)
I believe it is
worthwhile to
apply the concepts
learned in this
program
1 2 3 4 5
Confidence (Level 2)
I feel confident
about applying
what I learned on
the job
1 2 3 4 5
I anticipate that I
will receive the
necessary support
to confidently
apply what I have
learned
1 2 3 4 5
Commitment (Level 2)
I am committed to
applying what I
learned to my
work
1 2 3 4 5

208

Appendix E: Evaluation Instrument for Levels 1, 2, 3, 4 Delayed Post Program
Implementation
The intent of this evaluation is to determine the effectiveness of the operations leader
leading indicator program training. Your feedback will be key in improving future program
training modules and is greatly appreciated. Thank you in advance for your participation.

Table E1
Evaluation Instrument
Strongly
disagree
Strongly
agree
Relevance, delayed Level 1
I have had
occasion in
my job to use
what I learned
in this
program
1 2 3 4 5
Looking back,
my ability to
do my job has
been
enhanced by
this program
1 2 3 4 5
Customer Satisfaction, delayed Level 1
Looking back, I
will still
recommend
this program
to my co-
workers
1 2 3 4 5
Attitude, delayed Level 2
I still believe it 1 2 3 4 5
209

is worthwhile
to apply the
concepts
learned in this
program
Confidence, delayed Level 2
I still feel
confident
about
applying what
I have learned
on the job
1 2 3 4 5
Looking back, I
have received
the necessary
support to
successfully
apply what I
have learned
1 2 3 4 5
Behavior, delayed Level 3
I have been able
to apply on
the job what I
learned in this
program
1 2 3 4 5
I now apply my
newly
acquired
behaviors
from the
training to
meet the
expectations
that my
supervisor
and I have set
1 2 3 4 5
I am applying
my newly
acquired
critical
1 2 3 4 5
210

behaviors
from the
training in the
manner I
committed to
and aligned
with my
supervisor
I feel I have
been able to
implement
adopted
critical
behaviors
from the
program as a
result of the
support
received
1 2 3 4 5
Leading indicators, delayed Level 4
I develop
leading
indicators for
myself and
others that
directly
correlate to
business
outcomes
1 2 3 4 5
Desired results, delayed Level 4
The
development
and
implementati
on of leading
indicators has
positively
impacted my
department
1 2 3 4 5
I now produce
leading
1 2 3 4 5
211

indicators as a
consequence
of my newly
adopted
critical
behaviors
rooted in the
training I
have received
My efforts have
contributed to
the mission of
the
organization
1 2 3 4 5

212

Appendix F: Dashboard for Operations Leaders’ Evaluation for Levels 1–4 Delayed Post
Program Implementation

Abstract (if available)

Abstract This dissertation explores the concepts of leading operational indicators and accountability at a corporation, a global contract manufacturer, with the intent to understand what the knowledge, motivational, and organizational influences impacting overall performance. The study featured a mixed-methods, quantitative and qualitative approach utilizing internal organizational documents, surveys, and interviews to collect the necessary data. The results indicated that gaps were present in the areas of conceptual knowledge, goal orientation motivation, and cultural models and settings. As a result of the research and corresponding literature review, core recommendations to improve the situation were determined. From a knowledge perspective, the sharing of information and job aids to the operations leaders can be leveraged to close the existing conceptual gap. In terms of motivation, the creation of a community of learners as well as the establishment of task, reward, evaluation and management structure could reinforce the necessary motivation to improve performance. Finally, to address the organizational gaps, the reinforcement of accountability via assessments, the setting of goals that promote accountability, and the creation of a learning organization that reinforces leading indicator development would be key mechanisms to improve and address the problem of practice.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Lean construction through craftspeople engagement: an evaluation study

PDF

Enhancing socially responsible outcomes at a major North American zoo: an innovation study

PDF

ReQLes technology's this is your life: an innovation study

PDF

Examining felt accountability and uneven practice in dual organizational systems: a bioecological study toward improving organizational accountability

PDF

Recruiting police diversity

PDF

Mentoring as a capability development tool to increase gender balance on leadership teams: an innovation study

PDF

Factors influencing first-term naval aviator career continuation: a gap analysis

PDF

Employee satisfaction factors and influences: an evaluation study

PDF

Inclusionary practices of leaders in a biotechnology company: a gap analysis innovation study

PDF

Increasing organizational trust within financial services during times of change: an improvement study

PDF

Optimizing leadership and strategy to develop an expenditure-reduction plan: an improvement study

PDF

Corporate innovation labs: exploring the role of university research park innovation lab leaders

PDF

U.S. Navy SEALs resilience needs assessment: an innovation study

PDF

Understand how leaders use data to measure the effectiveness of leadership development programs

PDF

Creating a safety culture to decrease vehicle accidents with Sales Service Representatives

PDF

Exploring the experience of acquired employees within an organization following acquisition

PDF

A qualitative examination of the methods church leaders use to increase young adult attendance in Christian churches: an evaluation study

PDF

Employee standardization for interchangeability across states: an improvement study

PDF

Procedural justice and the impact on African Americans: an evaluation study

PDF

Facilitating hospital patient flow: an exploratory analysis on reducing patient bed turnaround time

Asset Metadata

Creator Witt, Martin III (author)

Core Title Taking the pulse on accountability: an innovation study

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Degree Conferral Date 2022-05

Publication Date 04/15/2022

Defense Date 03/25/2022

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag accountability binary,lagging indicators,leading indicators,OAI-PMH Harvest

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Donato, Adrian (committee chair), Maddox, Anthony (committee member), Phillips, Jennifer (committee member)

Creator Email drmartinwittiii@gmail.com,wittmart@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC110960499

Unique identifier UC110960499

Document Type Dissertation

Format application/pdf (imt)

Rights Witt, Martin III

Type texts

Source 20220415-usctheses-batch-924 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu