University of Southern California Dissertations and Theses

Knowledge, motivation, and organizational influences in aligning learning to business performance results: a study of a promising practice

(USC Thesis Other)

Knowledge, motivation, and organizational influences in aligning learning to business performance results: a study of a promising practice

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS

Knowledge, Motivation, and Organizational Influences in
Aligning Learning to Business Performance Results:
A Study of a Promising Practice

by
Marina A. Theodotou

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

May 2018

Copyright 2018 Marina A. Theodotou
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS

Dedication
To my beloved parents, Andreas Theodotou (1936-2012) and Julia Theodotou, whose wisdom,
love, integrity, and perseverance enabled and propelled me to this point.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS

Acknowledgments
This dissertation journey would not have been possible without the guidance, support,
and time that several professors provided, including my Dissertation Committee Chair, Dr.
Kimberly Hirabayashi, and my committee members, Dr. Krop and Dr. Murphy.
Additionally, I would like to thank Dr. Randy Palisoc for his guidance on my early
writing, as well as Dr. Kenneth Yates for his guidance towards the end of the program.
I would like to recognize the members of my Cohort for all the Saturdays we spent in
class for over 2 years.
Last, but not least, I would like to thank the Defense Acquisition University for allowing
me to study their best practice journey. In particular, I would like to thank Mr. James Wolsey for
responding to my initial inquiry, and Dr. Christopher Hardy for his wisdom, perspective, and
support in this effort.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS
i

Table of Contents
List of Tables .................................................................................................................................. v
List of Figures ................................................................................................................................ vi
Abstract ......................................................................................................................................... vii
CHAPTER ONE. Overview of the Study ....................................................................................... 1
Introduction to Problem of Practice .................................................................................... 1
Background of the Problem ................................................................................................ 2
Importance of a Promising Practice Project........................................................................ 5
Organizational Context and Mission .................................................................................. 6
Organizational Performance Status..................................................................................... 7
Description of Stakeholder Groups ..................................................................................... 9
Stakeholder Group for the Study ...................................................................................... 10
Purpose of the Project and Questions ............................................................................... 11
Conceptual and Methodological Framework .................................................................... 11
Definitions......................................................................................................................... 12
Organization of the Project ............................................................................................... 13
CHAPTER TWO. Literature Review ........................................................................................... 15
Influences on the Problem of Practice .............................................................................. 15
History of Learning Measurement ........................................................................ 16
Benefits of Aligning Learning to Business Performance Results ......................... 18
Challenges in Aligning Learning to Business Performance Results .................... 19
Clark and Estes’s (2008) Knowledge, Motivation, and Organizational Influences
Framework ........................................................................................................................ 22
Knowledge, Motivation, and Organizational Influences ...................................... 23
Summary ........................................................................................................................... 31
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS
ii

CHAPTER THREE. Research Methodology ............................................................................... 33
Conceptual Framework: The Interaction of Stakeholders’ Knowledge and Motivation
with the Organizational Context ....................................................................................... 33
Participating Stakeholders ................................................................................................ 39
Interview Sampling Criteria (Recruitment) Business Performance Results and
Rationale ............................................................................................................... 41
Document Data Mining Sampling Criteria and Rationale .................................... 42
Data Collection ................................................................................................................. 42
Interviews .............................................................................................................. 42
Documents and Artifacts....................................................................................... 43
Data Analysis .................................................................................................................... 45
Credibility and Trustworthiness ........................................................................................ 45
Role of the Researcher ...................................................................................................... 46
Ethics................................................................................................................................. 47
CHAPTER FOUR. Findings ......................................................................................................... 50
Participating Stakeholders ................................................................................................ 51
Knowledge Assets ............................................................................................................. 53
Assumed Knowledge Influence Assets ................................................................. 54
Emergent Knowledge Influence Findings ............................................................ 71
Motivation Influence Assets ............................................................................................. 78
Attributions ........................................................................................................... 78
Goals ..................................................................................................................... 81
Values ................................................................................................................... 82
Organizational Influence Assets ....................................................................................... 86
Culture................................................................................................................... 86
Alignment ............................................................................................................. 91
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS
iii

Resource Allocation .............................................................................................. 92
Findings Summary ............................................................................................................ 94
CHAPTER FIVE. Recommendations for Practice to Address KMO Influences ......................... 96
Knowledge, Motivation, and Organizational Influence Recommendations ..................... 97
Knowledge Recommendations ............................................................................. 97
Motivation Recommendations ............................................................................ 104
Organizational Recommendations ...................................................................... 110
Best Practices for Learning & Development Teams....................................................... 119
Best Practice 1: Foster Confidence, Creativity, and Commitment to your L&D
Team ................................................................................................................... 120
Best Practice 2: Align Metrics, Strategy, and Resources to the CEO Agenda ... 121
Best Practice 3: Cultivate Continuous Improvement .......................................... 122
Integrated Implementation and Evaluation Plan ............................................................. 123
Implementation and Evaluation Framework ....................................................... 123
Organizational Purpose, Need, and Expectations ............................................... 125
Level 4: Results and Leading Indicators ............................................................. 125
Level 3: Behavior ................................................................................................ 126
Level 2: Learning ................................................................................................ 130
Level 1: Reaction ................................................................................................ 132
Evaluation of the Components of Learning .................................................................... 133
Evaluation Tools ................................................................................................. 134
Data Analysis and Reporting .......................................................................................... 136
Summary ......................................................................................................................... 136
Strengths and Weaknesses of the Approach ................................................................... 138
Limitations and Delimitations......................................................................................... 139
Limitations .......................................................................................................... 139
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS
iv

Delimitations ....................................................................................................... 140
Future Research .............................................................................................................. 141
Conclusion ...................................................................................................................... 142
References ................................................................................................................................... 144
Appendix A. Interview Protocol ................................................................................................. 154
Appendix B. Documents and Artifacts Protocol......................................................................... 158
Appendix C. Levels 1 and 2 Evaluation Tool ............................................................................. 159
Appendix D. Levels 3 and 4 Evaluation Tool............................................................................. 160
Appendix E. Program.................................................................................................................. 161
Appendix F. Data Collection Matrix .......................................................................................... 164
Appendix G. Evaluation Tools ................................................................................................... 165
Appendix H. Dashboards ............................................................................................................ 168

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS
v

List of Tables
Table 1. The Eight Most Prevalent Models for Learning Evaluation and Measurement ............. 17
Table 2. General Factors Affecting Learning Alignment ............................................................. 21
Table 3. Knowledge Influences .................................................................................................... 25
Table 4. Motivational Influences .................................................................................................. 28
Table 5. Organizational Influences ............................................................................................... 31
Table 6. Research Questions, Assumed Influences, and Assessment Instruments ....................... 38
Table 7. Knowledge Assets........................................................................................................... 54
Table 8. Motivational Assets ........................................................................................................ 78
Table 9. Organizational Assets ..................................................................................................... 86
Table 10. Summary of Knowledge Influences and Recommendations ........................................ 98
Table 11. Summary of Motivation Influences and Recommendations....................................... 105
Table 12. Summary of Organization Influences and Recommendations ................................... 111
Table 13. Outcomes, Leading Indicators, and Methods for External and Internal Outcomes .... 126
Table 14. Critical Behaviors, Metrics, Methods, and Timing of Evaluation .............................. 128
Table 15. Required Drivers to Support Critical Behaviors ......................................................... 129
Table 16. Components to Measure Reactions to the Program .................................................... 133
Table 17. Evaluation of the Components of Learning for the Program...................................... 134
Table 18. Evaluation Tools ......................................................................................................... 135
Table 19. Data Collection Matrix ............................................................................................... 164

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS
vi

List of Figures
Figure 1. Conceptual framework of this study.............................................................................. 35
Figure 2. The acquisition learning model ..................................................................................... 56
Figure 3. Example of organizational goal ..................................................................................... 59
Figure 4. The Defense Acquisition University strategic planning process ................................... 61
Figure 5. New course design ......................................................................................................... 67
Figure 6. The New World Kirkpatrick Model ............................................................................ 124
Figure 7. External results ............................................................................................................ 168
Figure 8. Internal results ............................................................................................................. 169

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS
vii

Abstract
In this study, the researcher used the Clark and Estes’s (2008) gap analysis framework to examine
the alignment of learning to business performance results in the learning and development
industry. The purpose of this promising practice study was to investigate the knowledge,
motivation, and organizational assets used by the Defense Acquisition University to successfully
align learning to business performance results. The results and recommendations resulting from
this study provided a set of best practices that can be used across the learning and development
industry to better align learning to business performance results and positively impact
organizational performance and profitability. The researcher used a combination of literature
review, interviews, and document analysis to evaluate the alignment of learning to business
performance results. The researcher identified several best practices relating to knowledge,
motivation, and organizational influences. The most notable findings relate to three goals that
learning and development teams can aim for to successfully align learning to business performance
results: (a) Goal 1: Foster confidence, creativity, and commitment in your L&D team; (b) Goal 2:
Align metrics, strategy, and resources to the CEO agenda; and (c) Goal 3: Practice continuous
improvement. The researcher then designed a blueprint of best practices and presented
recommendations for implementation.
Keywords: learning and development, alignment, business performance, Defense
Acquisition University, best practices, knowledge, motivation, organization

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 1

CHAPTER ONE
Overview of the Study
Introduction to Problem of Practice
Aligning learning to business performance results is the number one approach that
learning and development (L&D) teams can utilize to positively impact their organizations'
performance. Ninety-six percent of Chief Executive Officers (CEOs) in the United States want
learning to provide business impact, yet only 8% of CEOs currently see the business impact of
learning in their organizations (Juan, Yao, Tamyrez, & Ayodele, 2016; Phillips & Phillips,
2007). CEOs want to see stronger links between the learning that their employees receive and the
business performance results that these same employees are tasked with achieving in their
organizations (Phillips & Phillips, 2007); however, organizations often lack access to best
practices in how to align learning to business performance results, which, in turn, hinders the
professional development of their employees and organizational performance and profitability
(Ben-Hur, Jaworski, & Gray, 2015; Juan et al., 2016).
The business performance results are set by the Chief Learning Officer and his or her
team (Bose, 2004). Such goals are essential to an organization because they set the
organizational vision; facilitate the allocation of resources such as funding, technology, and staff;
and guide the prioritization of specific tasks so that managers and employees can deliver
products and services to their customers (Bose, 2004; Murray, 2003). In parallel, in these same
organizations, the Chief Learning Officer (CLO) and his or her learning and development team
are responsible for overseeing the learning and development programs which include the
learning competencies which guide the employees of the organization to learn the necessary
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 2

skills to do their job well and to achieve the business performance results set by the organization
(Bonner, Hastie, Sprinkle, & Young, 2000; Bose, 2004, Juan et al., 2016).
Background of the Problem
The lack of best practices for ensuring learning and business performance results can be
detected when there is a lack of alignment between the skills that employees are instructed to
learn during such programs and the tasks they do on the job (Stuckey, 2012). This misalignment
creates confusion and disengagement for the employees and negatively influences their
performance (Rogers, 2013). According to a LinkedIn report by Taylor (2016) the cost of a
disengaged employee can be as high as $50,000 per year. If left unchecked, the misalignment of
learning to business performance results can negatively affect the organization’s attainment of
key strategic goals, profitability, and overall performance (Ben-Hur et al., 2015; Juan et al.,
2016; Rogers, 2013). In contrast, organizations with strong alignment of learning to business
performance results demonstrate better market performance by more than 20% (Kirkpatrick &
Kirkpatrick, 2016). While the measurement of the business impact of learning has proven elusive
and complex (Ben-Hur et al., 2015), organizations spend on average $1,200 per year in direct
learning expenditure per employee (Kirkpatrick & Kirkpatrick, 2016). A lack of alignment
between learning and business performance results negatively influences business performance
and long-term growth (Ben-Hur et al., 2015; Kirkpatrick & Kirkpatrick, 2016).
Few approaches exist to guide the alignment of learning to business performance results,
and even fewer have been successfully implemented (Ben-Hur et al., 2015; Juan et al., 2016). As
a result, only 8% of CEOs in the United States see a return on investment (ROI) of the learning
programs in their organizations (Juan et al., 2016). Researchers have revealed that the key factors
leading to the lack of alignment between learning and business performance results stems from
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 3

the size and structure of the L&D teams, lack of business acumen skills in learning and
development teams, measurement of the wrong metrics, and lack of alignment with the CEO’s
agenda (Ben-Hur et al., 2015; Murray, 2003; Stuckey, 2012).
The structure and size of L&D teams vary greatly across organizations (Alonso-
Gonzalez, Peris-Ortiz, & Mauri-Castello, 2017; Juan et al., 2016). In terms of structure, two
thirds of organizations have centralized L&D teams located in Human Resources or Business
Operations divisions in their organizations, whereas the remaining one third of organizations
have decentralized L&D teams which are co-located with lines of business such as human
resources, business operations, talent management, information technology, customer service,
sales, marketing, engineering, and finance, among others (Juan et al., 2016).
The L&D team size often dictates the type and learning used by each organization,
whereby larger L&D teams have the capacity and budgets to develop and deliver learning
programs internally in various modalities including instructor-led classes, peer to peer coaching
and online learning (Alonso-Gonzalez et al., 2017; Juan et al., 2016). Other organizations with
smaller L&D teams and smaller budgets rely on external vendors to develop and deliver learning
programs across the organization (Alonso-Gonzalez et al., 2017). Regardless of their structure or
size, 80% of L&D teams in organizations in the United States agree that developing talent
through learning is the number one priority at their organizations (Juan et al., 2016); however,
they also admit that they are unable to align learning with business performance results and
demonstrate the business impact of learning in their organizations (Ben-Hur et al., 2015; Juan et
al., 2016).
Furthermore, lack of knowledge in how to align learning to strategic goals stems from the
fact that most L&D organizational staff lack business training; this is because they traditionally
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 4

rise from the human resource field and not a business setting (Juan et al., 2016). According to
Taylor (2016), Bird (2008), and Rogers (2013), L&D division employees with only a human
resources background and lack hands-on business experience to understand the challenges, needs
and strategic goals of the lines of business in their organization. Lack of understanding of the
business needs and strategic goals can result in learning and development organizations
becoming training order takers, whereby they merely respond to requests for training—instead of
C-suite influencers, whereby they can advise the CEO and his or her team on how to best tackle
the business challenges the organization is facing (Bose, 2004; Juan et al., 2016).
Additionally, measuring the wrong metrics is a significant challenge preventing the
alignment of learning to strategic goals. Most organizations measure learner attendance, learner
reaction to the program, instructional environment, instructor effectiveness, and content
engagement, but do not tie these metrics to the critical strategic and business results of the
organization to evaluate how the employees and their teams do back on the job after the learning
program has concluded (Juan et al., 2016; Williams, 2002). Ben-Hur et al. (2015) and Stuckey
(2012) argued that lack of practical or useful mechanisms to measure L&D return on investment
(ROI) is another dimension of measuring the wrong things. Furthermore, Ben-Hur et al. (2015)
asserted that L&D organizations deploy pre-learning assessments and complex learning
competencies in order to measure the effectiveness of learning programs without consulting with
employees or incorporating their business needs and vital strategic results. This results in
learning programs that are on the one hand, structurally sound and on the other hand, irrelevant
to business needs. Thus, the alignment of learning to business performance results is a critical
challenge for learning and development professionals and organizations focusing on workforce
development, performance, and profitability growth.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 5

Importance of a Promising Practice Project
Failing to align learning objectives with business performance results in one in four
Fortune 500 CEOs being concerned about employee development. This failure also drives
employee disengagement and can negatively impact the overall organizational profitability and
growth (Ben-Hur et al., 2015; Juan et al., 2016; PriceWaterhouseCoopers, 2012; Stuckey, 2012).
Studying promising practices can help identify appropriate alignment methodologies and
strategies to alleviate these challenges faced by organizations. The methods that Ben-Hur et al.
(2015), Murray (2003), and Stuckey (2012) discussed are similar to those implemented by the
Defense Acquisition University (DAU) between 2000 and 2010, which led to the successful
alignment of learning to business performance results. In this promising practice study, the
researcher reviewed the journey of the DAU from 2010 to 2017 in order to examine how the
changes, improvements, and adaptations took place at DAU to further enhance the alignment of
learning to business performance results. While the DAU operates as corporate university, it
functions within the realm and context of the Department of Defense (DoD) of the United States
(DAU, 2015). As a government educational institution, the DAU operates under the legal and
regulatory requirements of the DoD and Congress. While such requirements are more
complicated than those of a private sector corporate university, the critical functions of providing
learning at the modality and point of need of the learner are the same.
The focus of this promising practice study was specifically and solely on the examination
of the knowledge, motivation, and organizational influences on the successful alignment of
learning to business performance results as implemented and further enhanced by the DAU
between 2008 and 2018. As such, the content of learning—which is predominantly defense
acquisition—and the profile of the learners—which consists of acquisition professionals from the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 6

DoD and the U.S. who deliver and sustain warfighting capabilities—were not examined in depth
in this analysis because they were beyond the scope of this study. By examining any changes in
terms of knowledge, motivation, and organizational influences that may have taken place
between 2010 and 2017 in the process of aligning learning to business performance results and
gleaning relevant best practices and possible solutions, the researcher aimed to help
organizations better align learning to business performance results, develop their employees
more efficiently, increase profitability and performance more steadily, and contribute to the
growth of their organizations and the vitality of the United States economy.
Organizational Context and Mission
An organization that has successfully aligned learning and business performance results
is the Defense Acquisition University, which serves as the Corporate University of the
Department of Defense of the United States. In 2017, DAU won the Best Learning Organization
of the Year Learning Elite Award from the Chief Learning Officer magazine. The Learning Elite
Award recognizes organizations for their L&D success overall and in specific initiative areas
(LearningElite, 2017).
According to the DAU (2015), the university’s mission is to provide a global learning
environment in order to develop qualified acquisition, requirements, and contingency
professionals who deliver and sustain efficient and affordable warfighting capabilities (DAU,
2015). Congress established the DAU in 1990 with the Defense Acquisition Workforce
Improvement Act (DAWIA). Indicatively, DAU graduates over 175,000 professionals, delivers
more than 7 million hours of learning, offers over 310 online learning modules, and registers
over 700,000 learning completions (DAU, 2015). DAU offers acquisition certification training
across all modalities (in-classroom, blended, and on-demand) through certified faculty and staff
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 7

(DAU, 2015). In 2016, DAU employed 591 faculty and staff across five campuses and several
departments. The primary role of DAU staff and faculty is to enable the Defense Acquisition
workforce to achieve better acquisition outcomes by aligning DAU learning content to the
strategic goals for national security defined by the Department of Defense of the United States
(DAU, 2017).
Organizational Performance Status
A critical metric in organizational performance for learning and development
organizations is learner satisfaction. If the learning is aligned well with business performance
results, learning satisfaction increases as employees learn the skills that they need to perform
well on the job (Kirkpatrick & Kirkpatrick, 2016). During 2015, the mean for learner satisfaction
for L&D organizations and ranged between 78 to 88% (Gunawerdena, Linder-VanBerschot,
LaPointe, & Rao, 2010; Metrics that Matter, 2015). In comparison, during 2015, DAU had a
learner approval rating of 92%, or 12 percentage points above the mean for corporate
universities.
The DAU uses the Kirkpatrick Model to evaluate learning and deploys surveys generated
by Metrics that Matter/Corporate Executive Board (CEB) immediately following a course to
assess the first two levels of Kirkpatrick's model. These two levels are: (a) Level 1: Reaction and
(b) Level 2: Learning (Kirkpatrick & Kirkpatrick, 2016). The DAU defines these two metrics as
consumptive metrics (DAU, 2017). The survey asks the following two questions: “To what
degree employees react favorably to the learning event?” for Level 1 evaluation and “To what
degree employees acquire the intended knowledge, skills, and attitudes based on their
participation in the learning event?” for Level 2 evaluation (LearningElite, 2017). After 60 days
from the completion of the learning course, DAU sends out another survey to check in with
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 8

learners and their supervisors to evaluate learning pertaining to (a) Level 3: Behavior and (b)
Level 4: Business Impact of the Kirkpatrick Model (DAU, 2017). These two levels ask a new set
of questions as follows: “To what degree employees apply what they learned during training
when they are back on the job?” for Level 3 and “To what degree targeted outcomes occur, as a
result of the learning event(s) and subsequent reinforcement?” for Level 4 (LearningElite, 2017).
DAU focuses on the impact metrics in Levels 3 and 4 of the Kirkpatrick Model (LearningElite,
2017).
By continuously evaluating the changes in the learner behavior and their impact and
outcomes on the learner performance on the job, by drilling deep into the survey data, DAU has
been able to continuously align learning to business performance results (DAU, 2015; 2017).
Additionally, DAU also applies text mining on more than 50,000 surveys at a time in order to
identify patterns in learners’ responses as they relate to specific courses (DAU, 2015;
LearningElite, 2017). Based on these results, DAU maps the root causes of low-performing
courses and focuses on addressing and resolving them to improve the learner experience. This
approach drives learning content improvement and strengthens the alignment of learning content
with the Department of Defense's strategic goals.
As a result of the successful implementation of these practices during the past 16 years,
DAU is recognized as a best-in-class organization for learning alignment and learning
satisfaction (LearningElite, 2017). Between 2000 and 2017, the DAU won 66 awards, including
the Best Overall Corporate University in the World by the Global Council of Corporate
Universities in 2013; the Corporate University of the Year for North America by CUBIC in
2015; the Brandon Hall Group Best in Strategy for a Corporate University Silver Award in 2016;
the Chief Learning Officer Learning and Practice Award in 2016; and the eLearning Top 100
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 9

Best in Government Strategic Planning and Learning Strategy Award in 2016. In 2017, DAU
won the Best Learning Organization of the Year with the LearningElite 2017 award
(LearningElite, 2017).
Description of Stakeholder Groups
The Defense Acquisition University has several stakeholder groups, including the
members of the Defense Acquisition Workforce, the DAU Faculty, the DAU Staff, the Board of
Visitors, Foreign Students, the Department of Defense, the learning and development industry
and society at large. In this study, the researcher described three of these stakeholder groups and
focused on one.
The first stakeholder group consisted of the Defense Acquisition Workforce learners. In
2015 there were 153,542 members of the Defense Acquisition Workforce: 37% from the Navy,
23% from the Army, 23% from the Air Force, and 17% from the Defense Agencies. The
members of the Defense Acquisition Workforce are the students of DAU. These learners benefit
from the Acquisition Learning Model (ALM) which was developed by DAU. ALM provides
foundational learning, performance learning and workflow learning to support the mission of
DAU of “providing a global learning environment to develop qualified acquisition, requirements,
and contingency professional who deliver and sustain efficient and affordable warfighting
capabilities” (DAU, 2015).
The second stakeholder group includes the DAU Faculty. The third stakeholder group,
the DAU Staff, includes the DAU Leadership team, consisting of the President, Vice President,
and Chief of Staff, as well as 10 subgroups: the Office of the President, the Foundational
Learning Directorate, the Performance Learning Directorate, the Workflow Learning Directorate,
the Operations Support Group, Information Technology, the Performance and Resource
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 10

Management Group, the Human Resources Group, the Acquisition Services Group, and the
Fourth Estate Director of Career Management. In 2015, there were over 300 teaching faculty and
over 250 staff, which includes learning faculty at DAU. In 2015, their combined efforts provided
4.6 million hours of training to 173,773 graduates in 137 foundational training courses, and 2.6
million hours of continuous learning through 310 modules. The DAU Faculty and Staff are
located on five DAU campuses and DAU two colleges in the United States. In this study, the
researcher focused on stakeholders from staff including the Office of the President, the
Foundational Learning Directorate, the Workflow Learning Directorate, and the Performance
Resource Management Group, which includes the Defense Systems Management College
(DSMC). These stakeholders represented the learning administrators who implement the
alignment of learning to business performance results. The learning administrators’ counterparts
in industry are the L&D teams.
Stakeholder Group for the Study
The joint efforts of all stakeholder groups described above contributed to the achievement
of the overall organizational goal of 92%, exceeding the 80% goal in learner satisfaction at DAU
in 2015 (DAU, 2015). This study included interviews with learning administrators from the
Office of the President, the Foundational Learning Directorate, the Workflow Learning
Directorate, and the Performance Resource Management Group, which includes the Defense
Systems Management College. In the early 2000, this group led the organizational change,
identified the challenges relating to the misalignment of learning development, and charted the
journey of aligning learning with organizational business performance results which continues
today (Anderson, Hardy, & Leeson, 2008). Between 2008 and 2018, these learning administrator
groups continued to lead the process by adapting to the evolution of acquisition practices, the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 11

changing needs of the learners, and new technology influences (DAU, 2017). In the next section,
the researcher will specifically discuss the purpose of the project and the anchoring research
questions guiding this analysis.
Purpose of the Project and Questions
The purpose of this project was to study DAU’s performance between 2010 and 2017 as
a promising practice related to the process of successful alignment of learning to business
performance results. While a complete study would focus on all stakeholders, for practical
purposes, this analysis focused on the learning administrators described above and their assets in
the areas of knowledge and skill, motivation, and organizational resources. As such, the
following three questions guided this promising practice study:
RQ1: What were the knowledge, motivational influences that enabled DAU to become a
best-practice organization in the alignment of learning to business performance goals?
RQ2: How did organizational influences impact the alignment of learning to business
performance goals?
RQ3: What solutions and recommendations in the areas of knowledge, motivation, and
organizational resources may be appropriate for aligning learning to business performance goals
at another organization?
Conceptual and Methodological Framework
Clark and Estes's (2008) created a gap analysis model, which is a systematic, analytical
method that helps researchers to understand organizational goal achievement. In this promising
practice study, the researcher used the Clark and Estes model on a broad basis to explain gaps in
performance by examining the interplay of knowledge, motivation, and organizational
influences. In this study, instead of a gap in performance, the focus was on the expanse in
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 12

performance and on the interdependencies and the influences of the organization on the
knowledge and motivation of the stakeholder group and their efforts in successfully aligning
learning to business performance results.
By further examining the impact of knowledge, motivation, and organizational influences
on the achievement of the organizational goal, the researcher generated specific, focused
elements of assumed influence based on related theory and literature across knowledge,
motivation, and organizational influence. Subsequently, these knowledge, motivation, and
organizational assumed influences were examined from the lens of the study research questions,
analyzed through the collection and analysis of data, and explained using a qualitative
methodological framework (Creswell & Poth, 2014; Merriam & Tisdell, 2015). This
methodological framework included qualitative research, the researcher conducted via the
implementation of in-depth, structured interviews and the examination of documents and
artifacts in Chapter Three. Specific open-ended interview questions were developed and mapped
one for one back to the knowledge, motivation and organizational influences as delineated by the
research questions that guide this study (Creswell & Poth, 2014). After the resulting data were
analyzed in Chapter Four, it was collated in Chapter Five, and the researcher articulated
research-based solutions as possible best practices to help organizations align learning to
business performance results.
Definitions
This section includes the definitions of key terms which will be used throughout this
study.
C-Suite. This refers to the members of the executive team which lead a publicly traded or
privately held corporation. This includes the chief executive officer (CEO), the chief financial
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 13

officer (CFO), the chief operations officer (COO), and the chief marketing officer (CMO),
among others (Investopedia, 2017).
Defense acquisition. This is the management and procurement process supporting the
investment in the technologies, learning programs, and product support required to support its
armed forces and achieve its national security business performance results (DAU, 2015).
Kirkpatrick Learning Measurement Model. The Kirkpatrick Model is a mechanism for
evaluating the effectiveness of learning programs using four levels. Level 1 Reaction assesses
how employees respond to the training. Level 2 Learning examines whether they learned the
material. Level 3 Behavior explores whether they are using what they learned on the job, and
Level 4 Results evaluates whether the learning program delivered results to the organization
(Kirkpatrick, 2006).
Learning and development. Learning and development refers to the area in an
organization that focuses on workforce capabilities, skills, and competencies needed, and how
these can be developed in order to ensure a sustainable and successful organization and that sets
out the means of establishing these capabilities to underpin organizational effectiveness
(Kirkpatrick & Kirkpatrick, 2016).
Learner satisfaction. This is a metric used to evaluate the overall satisfaction of a
participant in a professional development learning or training program (DAU, 2015).
Return on investment. ROI measures the benefit or loss generated on an investment
relative to the amount of money invested (Investopedia, 2017).
Organization of the Project
The current promising practice study is organized into five chapters. In Chapter One, the
researcher provided the problem of practice and the fundamental concepts and terminology
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 14

around it. Additionally, the organization's mission, goals, and stakeholders, as well as the initial
concepts of gap analysis were introduced in the context of successfully aligning learning to
business performance results implemented by the Defense Acquisition University between 2010
and 2017. In Chapter Two, the researcher provides a review of the current literature surrounding
the scope of the study. The topics in this chapter include training evaluation methods and
models, challenges, and opportunities in measuring and aligning learning to business
performance results are discussed, and the gap analysis model designed by Clark and Estes
(2008). In Chapter Three, the researcher details the assumed knowledge, motivation, and
organizational influence assets for this study, as well as the methodology of selecting employees,
data collection and analysis approach. In Chapter Four, the researcher assesses and analyzes the
data and results. In Chapter Five, the researcher collates the results based on the data and
literature as well as recommendations for an implementation and evaluation plan.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 15

CHAPTER TWO
Literature Review
The alignment of learning to business performance results is critical for both employee
and organizational growth (Ben-Hur et al., 2015; Bird, 2008; Bose, 2004). It is therefore
essential to study this alignment to uncover possible best practices that organizations can use to
align learning to business performance results to improve their profitability and long-term
growth and more broadly impact the performance of the economy of the United States (Juan et
al., 2016). In Chapter Two, the researcher outlines the progression of literature around the
alignment of learning to business performance results leading to high learner satisfaction and
organizational performance. In the first section, the researcher reviews the history of learning
measurement, the benefits of learning measurement and the challenges in achieving alignment of
learning to business performance results according to the literature. In the second section, the
researcher discusses the construct of organizational change concerning learner satisfaction
through the Clark and Estes (2008) gap analysis model. The chapter ends with an analysis of
conceptual framework, in which the researcher presents the essential organizational influences
on the stakeholder knowledge and motivation in the organizational context.
Influences on the Problem of Practice
In reviewing the literature, three prevailing streams emerged to describe the alignment of
learning to business performance results. In this section, the researcher will discuss each of the
three streams which are: first, the history and growth of learning measurement; second, the
benefits; and third, the challenges of aligning learning to business performance results.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 16

History of Learning Measurement
The first literature stream relates the history and growth of the field of learning
measurement. Measurement of learning is a crucial component in aligning learning to business
performance results because for alignment to take place learning metrics must correspond to
business performance metrics (Ben-Hur et al., 2015; Bird, 2008). The conduction of this
literature review revealed that the absence of learning measurement implies a lack of learning
metrics. Researchers have shown that L&D leaders typically think of and use just two models:
(a) the Kirkpatrick Model, which has dominated the learning evaluation narrative, and (b) the
Philips ROI model. Over the past 50 years, over 20 models have been developed to measure
learning (Tamkin, Yarnell, & Kerrin, 2002). Of these models, eight have been distilled as
different; however, all eight drew from the Kirkpatrick Model, which is the anchoring learning
measurement model (Tamkin et al., 2002). According to Tamkin et al., these models include the
Hamblin Model, The Organizational Elements Model (OEM Model), the Indiana University
Model, IS Carousel of Development Model (Carousel), Philips ROI, the Kearns, and Miller
Model (KPMT), and the Context, Input, Reaction, and Outcome (CIRO) Model.
Table 1 provides a high-level view of these models and shows the various components of
each level of evaluation. The first column shows the Kirkpatrick model which is anchored by four
levels of measurement: Level 1: reaction, which evaluates the immediate reaction of the learner
to the content; Level 2: learning, whereby the learner's cognitive processes are engaged, and she
or he is acquiring new knowledge; Level 3: behavior, denoting the changes in the behavior of the
learner as a result of the training received; and finally, Level 4: results, or the business impact of
the learning to the organization (Kirkpatrick & Kirkpatrick, 2016; Kirkpatrick Partners, 2016).

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 17

Table 1
The Eight Most Prevalent Models for Learning Evaluation and Measurement
Kirkpatrick
Model
Hamblin OEM
Model
Indiana
University
IS
Carousel
Phillip
s, ROI
KPMT CIRO
Activity
Accounting
Identify
business need;
define
development
objective design
learning process
Staged process
to examine
business
needs; design
solutions and
get buy-in
Context
Analysis
Input
Level 1:
Reaction
Reaction Input
Process
Reaction Experience the
learning process
Reaction
and Planned
Action
Reaction to
training
and
developme
nt
Reaction
Level 2:
Learning
Learning Micro
acquisition
Learning Use and
reinforce
learning
Learning Learning Outcome
immediat
e
Level 3:
Behavior
Job
Behavior
Micro
performance
Transfer of
learning
Job
application
Transfer to
the
workplace/
Behavior
Outcome
intermedi
ate
Level 4:
Results
Organization Macro Business
Impact
Evaluate the
benefits to the
organization
Business
Results
Essential
results
added
value
Outcome
Ultimate
Ultimate
Value
Mega
Societal
Outcomes
Social
Impact
Return on
Investment

Further examination of the models above reveals that four of these models (Indiana
University, IS Carousal, KPMT, and CIRO) have added a stage prior to Kirkpatrick's Level 1
“Reaction” level to evaluate the business need context. In addition, four models (Hamblin, OEM,
Indiana University, and Phillips ROI) added a fifth level after Kirkpatrick's last stage of “Results”
to evaluate broader societal value and impact (Tamkin et al., 2002). Most of these models expand
upon the Kirkpatrick model and attempt to address some of its limitations, including criticisms by
researchers that the interdependence and implied causality of each of the four levels had not been
proven by empirical research (Tamkin et al., 2002).

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 18

Benefits of Aligning Learning to Business Performance Results
In the second literature stream, the researcher focuses on benefits of aligning learning to
business performance results whereby learning that ties to measurable results and return on
investment (ROI) can help organizations improve business performance (Ben-Hur et al., 2015;
Kirkpatrick, 2006; LearningElite, 2017; Rogers, 2013).
When learning is aligned to business performance results, employees increase their
knowledge of concepts and skills that relate to their job, can change their behaviors, and do their
job better, which results in improved overall business performance (Juan et al., 2016; Kirkpatrick
& Kirkpatrick, 2016; Murray, 2003; Rogers, 2013). For instance, if an organization sets the
business performance goal to be an increase in sales by 20% in 1 year, it will need to provide
employees with the learning programs to support this performance goal, including training on
new products, new sales techniques, and new communication approaches to engage clients
(Nalborczyk & Sandelands, 2012).
The key to reaping the benefit of aligned learning hinges on creating learning modules
with learning outcomes that line up with the business strategies and goals of the organization
(Juan et al., 2016; Nalborczyk & Sandelands, 2012; Sadler-Smith, 2009). Furthermore, Lykins
(2012), Smith (2008), Stuckey (2012), and van Rooij and Merkebu (2015) emphasized that the
responsibility of developing such learning lies with the learning and development teams. L&D
teams must proactively seek to understand the industry context, business needs, critical strategic
drivers, external threats, financial, and business goals of the organization in order to develop and
make available appropriate learning at the time of need.
Moreover, Stuckey (2012) and Smith (2008) asserted that when a learning and
development organization can align learning to business performance results, it can deliver
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 19

measurable results, and can articulate the learning ROI to the broader organization.
Demonstrating the ROI as a result of learning programs, in turn, can empower L&D teams to
transform from training order takers to strategic advisors influencing strategy design and
implementation (Lykins, 2012; Sadler-Smith, 2009; Taylor, 2016). There is compelling evidence
in the literature asserting the benefits of aligning learning to business performance goals
underscored by the premise that learning which ties to measurable results facilitates the
measurement of the return on investment (ROI) and enables the achievement of the
organizational business goals (Juan et al., 2016; Lykins, 2012; van Rooij & Merkebu, 2015).
Challenges in Aligning Learning to Business Performance Results
In the third literature stream, the researcher focuses on the challenges organizations face
in aligning learning to business performance results. Specifically, three key challenges have been
identified: first, the lack of business skills and business acumen in learning and development
staff; second, the measurement of the wrong metrics, third; the lack in alignment with the CEO’s
agenda (Ben-Hur et al., 2015; Murray, 2003; Stuckey, 2012).
The first challenge according to the literature refers to the inability of the L&D
departments to “speak” the language of the line of business and articulate learning regarding the
broader business goals. Learning and development departments that are too insular or unable to
“speak the language” of the line of business are unable to articulate learning concerning the
broader business goals (Ben-Hur et al., 2015; Sadler-Smith, 2009; van Rooij & Merkebu, 2015).
Smith (2008) contended that learning and development departments are not as agile and
responsive as the change taking place in the organizations they serve due to lack in business
acumen skills, lack of understanding of the industry context and its complexities, and inability to
discuss financial and business goals in depth.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 20

Lack of business acumen influences the second challenge, which pertains to the
measurement of the wrong metrics (Smith, 2008; Stuckey, 2012; van Rooij & Merkebu 2015).
Specifically, van Rooij and Merkebu asserted that learning and development teams lacking in
business acumen are unable to “speak the language” of the various business units they support,
and as a result, they measure metrics that do not reflect the needs of the business unit. This
challenge underscores the importance of learning and development teams engaging closely with
the business units in their organization to understand the underlying culture, the needs, and
challenges of these units (Ben Hur et al., 2015).
The third challenge, according to Bird (2008), Ben-Hur et al., (2015), and van Rooij and
Merkebu (2015), relates to the distance that most learning and development teams have from the
CEO and his or her agenda. The further the distance, the greater the misalignment of learning to
business performance results (Sadler-Smith, 2009). The absence of this organizational influence
has been cited in the body of literature as one of the leading challenges in aligning learning to
strategic goals. Specifically, Ben Hur et al. (2015) asserted that if the learning and development
organization does not align with the CEO’s agenda and does not implement a robust strategic
plan, they will not be able to achieve organizational goals. Similarly, Bird (2008) affirmed that
when learning is not aligned with business performance goals, learning and development
organizations cannot articulate the value that they add to the organization.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 21

Table 2
General Factors Affecting Learning Alignment
General Related Literature Selected Citations
History of learning measurement and
alignment
During the past 50 years, over 20 models
have been developed to measure learning.
Of those about eight have been distilled as
different, however, all draw from the
Kirkpatrick Model

Most organizations use the Kirkpatrick
Model or the Phillips Model but have not
been able to efficiently tie learning to
business performance results
Tamkin et al. (2002)
Benefits of aligning learning to business
performance results
Organizational influences within Learning
and Development teams in Fortune 500
companies seem to be critical in driving
the alignment of learning to strategy

Good learning alignment is good for
business. Learning that ties to business
performance results and ROI can help
organizations improve business
performance

Bird (2008); Lin, Hitvens and Thomas
(2001), Lykins (2012); Nalborczyck and
Sandelands (2012);

Ben-Hur et al. (2015); Lykins (2012);
Sadler-Smith (2009)
Challenges in aligning learning to business
performance results
Learning and Development teams lack
business acumen skills which prevent
them from connecting with the lines of
business they support

Learning and Development teams
tend to measure the wrong metrics

Learning and Development teams in
Fortune 500 companies do not feel
empowered enough to align with
the CEO agenda and to focus on robust
process improvements goals
Ben-Hur et al. (2015), Stuckey (2012);
Williams (2002)

Ben-Hur et al. (2015), Smith (2008); Stuckey
(2012), Williams (2002)

Ben-Hur et al. (2015), Sadler-Smith (2009),
Stuckey (2012), Williams (2002)

The overarching themes in the literature revolve around the lack of business acumen, lack
of knowledge as to which metrics to measure, and lack of strategic planning and implementation.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 22

This results in the inability of L&D organizations to align with the CEO agenda and tie
learning—which resides with the L&D team—with business performance results—which reside
with the CEO and the management team. In the following sections, the researcher revisits the
role of the stakeholder group. The researcher will also examine the gap framework developed by
Clark and Estes (2008) as the path to understand how the DAU closed the knowledge and
motivation gaps by leveraging organizational influences and successfully aligned learning with
business performance results.
Clark and Estes’s (2008) Knowledge, Motivation, and Organizational Influences
Framework

Clark and Estes (2008) provided a systematic, analytic framework that clarifies
organizational and stakeholder performance goals and identifies the gap between the actual
performance level and the performance goal. The model begins with the identification of the gap
between the organizational business goals and stakeholder goals. Next, the model analyzes the
gap to determine the assumed performance influences in the areas of knowledge, motivation, and
organization based on general theory, prevailing literature specific to the context and a current
understanding of the organization. Finally, the model validates these influences and highlights
possible approaches to fine tune the system and align the individual and the business goals
improvement (Clark & Estes, 2008). In this promising practice study, the researcher
implemented a modified version of the Clark and Estes by looking at the expanse or overage in
performance, instead of a gap, in order to identify and validate the assumed critical drivers of the
successful alignment of learning to business performance results between 2008 and 2018.
Concerning the organizational influences on the performance goal gap analysis, Clark
and Estes (2008) defined three critical organizational drivers: culture, alignment, and
organizational change, which includes resource allocation. The current study involved an
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 23

examination the influence of culture, alignment and resource allocation on the knowledge and
motivation of the stakeholder group. Each of these elements of Clark and Estes’s gap analysis are
addressed below regarding the DAU stakeholder group. The next section includes a discussion of
assumed influences on the organizational goal in the context of knowledge, motivation, and
organizational influences.
Knowledge, Motivation, and Organizational Influences
In this section, the researcher analyzes the assumed knowledge, motivation and
organizational influences which drove the achievement of the organizational goal. First, the
researcher provides an overview of three knowledge influences, two of which are procedural and
one metacognitive, relevant to the stakeholder goal and categorize the relevant knowledge type.
Next, the researcher discusses three motivation-related influences and how they affected the
achievement of the stakeholder goal and established the foundation of a promising practice in
learning measurement and evaluation. Finally, the researcher examines three organizational
influences, with a focus on their impact on the knowledge and motivation of the stakeholder
group and the attainment of the organizational goal.
Knowledge. The knowledge and skills identified by Krathwohl (2002) are divided into
four types: (a) factual, (b) conceptual, (c) procedural, and (d) metacognitive. In this section, the
researcher will review the procedural and metacognitive knowledge skills definitions and focus
on specifically on the use of the four solutions provided by Clark and Estes (2008) in order to
determine which knowledge and skills and motivational elements were attained by the
stakeholders to achieve the alignment of learning to strategic goals. Social cognitive theory
model evidences that that knowledge and skills are critical to achieving organizational goals
(Wood & Bandura, 1989). Specifically, Wood and Bandura asserted that a bidirectional
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 24

relationship exists between the self-efficacy of individuals and the achievement of business goals
set by organizations.
Cognitive load theory gives a framework to facilitate learning design and optimize
learner performance (Kirschner, Paas, & Kirschner, 2006; Krathwohl, 2002). Factual knowledge
consists of discrete and specific content elements, whereas conceptual knowledge consists of
complex yet organized pieces of knowledge (Krathwohl, 2002). Procedural knowledge consists
of directions and instructions on how to do something. Lastly, metacognitive knowledge consists
of the awareness of one's way of learning (Krathwohl, 2002). This analysis focuses on two types
of knowledge: procedural and metacognitive and discusses how they influenced the stakeholder
at DAU to achieve their goal.
Procedural knowledge. According to Pintrich (2002) and Krathwohl (2002), procedural
knowledge pertains to knowing “how” something works and being able to efficiently implement
a process or a procedure to complete a task. In this study, procedural knowledge was prevalent in
the stakeholders' knowledge in data collection and analysis and strategic planning.
Stakeholders calibrated the processes around data collection and analysis. Kirkpatrick
(2006) stated that data collection is critical in learning measurement and alignment because what
is deemed necessary will drive the survey questions given the learners to evaluate each of the
four levels of the Kirkpatrick model: (a) reaction, (b) learning, (c) behavior, and (d) results.
Stakeholders adapted and expanded strategic planning processes. Strategic planning is a
critical ingredient in aligning learning to business performance results because it set the focus
and goals of the organization (Bird, 2008; Rogers, 2013). These goals drive the individual tasks
employees must perform. If employees lack the required skills to perform the tasks, they can
receive learning. In turn, learning must be tied to the specific business goals so that upon
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 25

learning the necessary skills, the employees can achieve the business performance goals set
earlier (Kirschner et al., 2006; Krathwohl, 2002; Pintrich, 2002).
Metacognitive knowledge. According to Pintrich (2002) and Krathwohl (2002),
metacognitive knowledge pertains to knowledge of specific strategic approaches that may be
useful in specific settings, knowledge of the context and parameters within which such strategies
may be useful, and self-knowledge and self-reflection. Metacognitive knowledge can help
stakeholders focus on the challenges that facilitate the achievement of the organizational goal
(Ben-Hur et al., 2015; Krathwohl, 2002; Stuckey, 2012).
Stakeholder self and team reflection. By reflecting in their experiences and journey, as
individuals and as a team, stakeholders can take stock of their successes, evaluate their failures,
and glean lessons learned which help calibrate the process there are building. Table 3 below
presents the knowledge influences described above and includes selected citations from related
literature.
Table 3
Knowledge Influences
Knowledge Influences Selected Citations
Procedural
Stakeholders, by calibrating the processes
around data collection and analysis to
respond to new learner needs and technology
changes, strengthened their business acumen
in which metrics to measure
Bandura, 2007; Kirkpatrick, 2006;
Wood and Bandura, 1989;
Stakeholders adapted and expanded specific
strategic planning and implementation
practices
Bird, 2008; Kirschner et al., 2006; Krathwohl,
2002; Pintrich, 2002; Rogers, 2013
Metacognitive
Stakeholders conducted ongoing self-
assessment and self-reflection
Ben-Hur et al., 2015; Krathwohl, 2002;
Pintrich, 2002; Stuckey, 2012
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 26

Motivation. Motivation to learn is defined as the internal state requiring the learner to
begin and maintain focus and exert effort until a learning goal is completed (Mayer, 2011).
Specifically, Mayer asserted that there are five cognitive theories of motivation. First, motivation
based on interest, whereby the stakeholders are motivated to align learning to business goals
because they feel alignment is more productive in the long run. Second, motivation is based on
beliefs, whereby the stakeholders are motivated to do all the required work because they believe
that their efforts will pay off. Third, motivation based on attributions, whereby the stakeholders
work harder to because they attribute their failure or success on the level of her effort. The fourth
type is motivation based on goals, whereby the stakeholders want to align learning with business
goals so that they can reach their goal of reaching a customer satisfaction goal at 80% or higher.
Finally, the fifth type of motivation is one based on partnership, whereby the stakeholders work
harder because they feel the camaraderie of their team and see other stakeholders as partners in
their efforts. While all five types of motivation were present at DAU according to Anderson et al.
(2008), the researcher will review only three: attribution, goals, and self-efficacy for individuals
and teams.
Attributions. Mayer (2011) and Pintrich (2002) asserted that attribution theory is how
and why people explain events in a particular way, whereby they “attribute” their behavior to
internal or external drivers. In the context of the stakeholder group, they explain their success by
attributing it to their efforts and hard work (Perry & Hamm, 2017). They believe that challenging
work and effort pays off in the achievement of goals. Additionally, Perry and Hamm expanded
upon Weiner (2005), claiming that competence is a critical internal attribution that individuals
use to explain success in the organizational setting.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 27

Stakeholders believed their challenging work is meaningful. As seen in Table 2, the
first motivational influence examined is stakeholder attributions whereby the stakeholders
explained their success by attributing it to their hard work and efforts (Mayer, 2011).
Goals. Researchers have shown that goals act as targets of personal achievement and are
crucial for motivation. In the context of motivation in this analysis, goals pertain to personal
development goals as well as strategic goals set by the organization. Goals help individuals and
teams concentrate on specific targets and encourage persistence and perseverance to reach that
target (Downes, Kristof-Brown, Judge, & Darnold, 2017; Locke & Latham, 2006). Goals must
be specific, measurable, achievable, realistic, and timely in order to empower the stakeholders to
achieve results (Bjerke & Renger, 2017). Also, the more difficult the goal, the more motivated
that the stakeholders are to achieve it. Locke and Latham (2006) asserted that there is a positive
linear relationship between the difficulty of a goal and the achievement of a task. As seen in
Table 4, the second motivational influence examined are the stakeholder goals, whereby
stakeholders continued to strive for the implementation of new methodologies and process
improvements in measuring learning alignment to strategic goals.
Stakeholders strive for process improvements. According to Mayer (2011), when
learning stakeholders set as their goal to align with business performance results, it is good for
the profitability of the organization, resulting in improved overall business performance.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 28

Table 4
Motivational Influences
Motivational Influences Selected Citations
Attributions
Stakeholders explained the organizational
success by attributing it to their hard work
and efforts
Mayer, 2011; Perry and Hamm, 2017; Pintrich,
2002; Weiner, 2005

Goals
Stakeholders continued to strive for the
implementation of new methodologies and
process improvements
Downes et al., 2017; Mayer, 2011; Lock and
Latham, 2006; Weiner, 2005

Values
Stakeholders, both as individuals and as
teams recognized and live their values
Mayer, 2011; Weiner, 2005; Lock and Latham,
2006; Rueda, 2011

Values. Bandura (2007) cited that self-efficacy relates to an individual's internal belief
that she or he will succeed when undertaking a difficult task by taking adaptive action. Team
efficacy relates to the same concept but for a broader group of individuals who function as a
team and their collective team belief that they can succeed if the show resilience and a ‘can-do’
cognition when the external circumstance becomes difficult. Additionally, researchers focusing
on self-efficacy have highlighted that individuals within teams tend to assess their environment
and seem to perform better within organizations that cultivate self-efficacy by setting clear goals
and provide support which strengthens team efficacy (Jaiswal & Dhar, 2016).
Stakeholders believed they would succeed. The stakeholders showed resilience and
believed that they could succeed and produce the desired result of achieving their goal. As
Bandura (2007) asserted, individuals with high self-efficacy believe they can succeed on what
they set out to do because they are intrinsically motivated. Additionally, Bandura posited that
prior experience is the most significant driver in self-efficacy, whereby the stakeholders were
motivated by the past successes they achieved.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 29

Organizational influences. Organizational influences are critical in explaining a
performance gap in an organization (Clark & Estes, 2008). In this section, the researcher will
examine these three organizational influences on the stakeholder group of the Defense
Acquisition University and how each influence affected the achievement of the organizational
goal. In the context of the Defense Acquisition University and the stakeholder goal, these three
organizational influences are a healthy organizational culture of accountability, a robust process
of strategic planning, and intelligent management and allocation of resources (Anderson et al.,
2008; Clark & Estes, 2008). Table 4 below depicts the organizational influences which follow.
The organization conveyed a healthy culture. Organizational culture is the compilation
of values, philosophy, skills, habits, models, paradigms, and group norms that exist within an
organization and defines how the organization behaves itself and within the broader business and
societal arena (Schein, 2010).
Researchers have shown that a healthy culture enhances accountability acts as a cultural
construct and hinges on individual accountability, collective accountability, and the interplay
between hierarchy and egalitarianism within an organization (Gelfand, Lim, & Raver, 2004).
Specifically, in their seminal work on the culture of accountability, Frink and Klimoski (1998)
asserted that a strong culture of accountability defines a bidirectional relationship between the
way in which the individual employee perceives their responsibility of being answerable for their
actions and decisions to organization and the way in which the organization and its leaders
encourage accountability through transparency and trust in the individual employees. According
to Schein (2010), Northouse (2016), and Pintrich (2002), organizations with an active and
influential culture of accountability provide clear and specific direction on business performance
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 30

results and goals, which, in turn, provide clarity to the stakeholders and motivate them to strive
harder to achieve the goal.
The organization fosters a robust process of alignment. Work processes are defined as
the “what,” or the articulation and implementation of work procedures, which constitute the
“how” to complete a task to achieve an organizational goal (Clark & Estes, 2008). Robust
processes define how people, technology, and other resources must connect and align with each
step of each task to achieve the organizational goal (Clark & Estes, 2008). Strong alignment is
fundamental to organizational performance (Clark & Estes, 2008). Specifically, Clark and Estes
underscored their importance by asserting that organizations which lack robust processes will
fail, even if knowledge and motivation are present. Researchers exploring process improvement
theories have provided numerous practical examples whereby strong focus on process
implementation by corporations across banking, healthcare, and other services industries resulted
in the achievement of organizational goals, increased performance, and profitability (Bright,
Ahmad, & Zahra 2017; Furtherer, 2016; Haines, 2016).
The organization concentrates on resource allocation. Resources refer to the tacit
supplies of talent, technology, time, materials, and other tangible supplies needed to implement
processes and procedures (Clark & Estes, 2008). Intelligent allocation of resources is
fundamental to the achievement of the organizational goal as they provide the fuel to the engine
of an organization (Clark & Estes, 2008). Resources include the people, technology, knowledge,
and materials required to implement tasks. In the knowledge organizations, resources are heavily
concentrated in knowledge and technology (Clark & Estes, 2008).

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 31

Table 5
Organizational Influences
Organizational Influences Selected Citations
Culture
The organization exhibited a healthy culture by
implementing a comprehensive system of
accountability and reporting
Clark and Estes, 2008; Northouse,
2016: Pintrich, 2002; Schein, 2010;
Weiner, 2005

Alignment
The organization developed, implemented, and
calibrated a rigorous strategic planning method
Ben-Hur et al., 2015; Clark and Estes,
2008; Mayer, 2011; Northouse, 2016;
Schein, 2010
Resource Allocation
The organization continuously optimized resources
around strategic planning, alignment, and reporting
to adapt to new learner needs, technology changes
and industry evolution
Ben-Hur et al., 2015; Clark and Estes,
2008; Northouse, 2016; Schein, 2010

Lack of resources or inadequate management of resources can prevent an organization
from achieving its goals (Clark & Estes, 2008). A ubiquitous and critical resource today is
technology and its use by organizations to improve productivity and performance (Bock, 2015).
For example, organizations which understand the use of technology to gather and intelligently
analyze data can find themselves ahead of the competition as they can increase their ability to
analyze the present and predict future trends (Bock, 2015).
Summary
In this study, the researcher aimed to address the misalignment of learning to business
performance results in Fortune 500 organizations. Three main streams of literature underscore
three key trends prevalent in this misalignment. In the first stream of literature, the researcher
focused on the historical development of learning measurement over the last 50 years and its
overreliance on the Kirkpatrick model, despite the development of over 20 different learning
measurement models. In the second stream of literature, the researcher highlighted that benefits
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 32

of aligning learning to business performance goals, whereby proper learning alignment is good
for business because learning that ties to measurable results and return on investment can help
organizations improve business performance (Murray, 2003; Rogers, 2013; Stuckey, 2012). In
the third stream of literature, the researcher underscored three challenges of aligning learning to
business performance goals: lack of business skills in L&D staff, measurement of the wrong
metrics, and failure to align with the CEO’s agenda within an organization (Ben-Hur et al., 2015;
Stuckey, 2012; Williams, 2002).
In this study, the researcher examined the promising practice model of the Defense
Acquisition University, which is a world-class organization in aligning learning to business
performance results. The researcher aimed to develop a set of recommendations, solutions, and
best practices for other organizations to adopt and adapt. The methodology, data collection, and
data validation approaches will be described in the next chapter.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 33

CHAPTER THREE
Research Methodology
As the researcher discussed in Chapter One, the purpose of this study was to examine and
glean best practices around the alignment of learning to business performance goals by
reviewing the performance of the Defense Acquisition University. Specifically, the researcher
aimed to respond to the following three research questions:
RQ1: What were the knowledge, motivational influences that enabled DAU to become a
best-practice organization in the alignment of learning to business performance goals?
RQ2: How did organizational influences impact the alignment of learning to business
performance goals?
RQ3: What solutions and recommendations in the areas of knowledge, motivation, and
organizational resources may be appropriate for aligning learning to business performance goals
at another organization?
In this chapter, the researcher describes the conceptual framework, the participating
stakeholders, and the rationale, methodology, and instruments that the researcher used for data
collection and analysis. Additionally, the researcher clarifies the approach used to enhance the
validity and credibility of the study, discusses the ethical principles that were followed, and
states the limitations and delimitations which confounded the study.
Conceptual Framework: The Interaction of Stakeholders’ Knowledge and Motivation with
the Organizational Context

A conceptual framework is a visual or written product which explains graphically or in
written form the key focal points of the study including the key factors, concepts and variables
and describes the influence and interactions amongst them (Maxwell, 2013). According to
Maxwell (2013), a conceptual framework is “the system of concepts, assumptions, expectations,
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 34

beliefs, and theories that support and informs the research,” and constitutes a critical component
of the research design for the analysis. Similarly, Punch (2013) described the conceptual
framework as construct of studying certain components and the interaction with one another.
The purpose of a conceptual framework is to ground the study on the foundational
knowledge base and support the problem statement and inquiry questions used in the study
(Rocco & Plakhotnik, 2009). Specifically, Leshem and Trafford (2007) stated that the purpose of
the conceptual framework is to give cohesiveness to the research by mapping the noticeable links
between the theoretical perspective, the research business performance results and design, and
the data collection and analysis, providing conceptual significance of the evidence. Furthermore,
Leshem and Trafford asserted that the purpose of the conceptual framework is to answer a
fundamental question about the research study pertaining to whether tthere a clear and explicit
connection between the theory, earlier findings, and the purpose of the present study. As such,
the conceptual framework acted as a bridge between the paradigms in order to explain the
problem of practice and the research conducted to investigate that problem of practice (Leshem
& Trafford, 2007).
In the analysis that follows, the researcher presents the conceptual framework in graphic
form and explains it in detail by referencing the Clark and Estes (2008) Knowledge, Motivation,
and Organizational (KMO) influences model. Drawing from the related literature, the researcher
will discuss linkages back to the research questions. While earlier in this analysis, the
Knowledge, Motivation, and Organization influencers were described independently, it is
essential to state that they did not operate in isolation; rather, they interacted with each other
within the conceptual framework of this study (Leshem & Trafford, 2007). Specifically, through
the conceptual framework, the analysis demonstrated how the knowledge and motivation of the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 35

stakeholders were influenced the organizational culture through leadership and brought about
organizational change which led DAU to achieve and surpass its goal of 80% customer
satisfaction by 12 percentage points in 2015 (Anderson et al., 2008). Because this study was a
promising practice research, the conceptual framework described how the influencers interacted
to achieve the organizational goal. These three influencers construct the conceptual framework
for this study, as presented in Figure 1 below:

Figure 1. Conceptual framework of this study.
Figure 1 provides a graphic representation of the conceptual framework for this study.
The graphic consists of two blue-colored, nested rectangles located to the left and connected to a
third, green rectangle with a green arrow pointing from left to right. Starting from the left, the
light blue, nested rectangle represents the stakeholder group at the Defense Acquisition
University; the largest rectangle represents the organization, the Defense Acquisition University,
and the green rectangle to the right, represents the organizational goal. The stakeholder group
Organization
Defense Acquisition University
Organizational Influences
Culture Alignment Resources
Stakeholder Group
Knowledge Motivation
• Goals
• Attributions
• Values
• Strategic Planning
• Aligning learning to
business performance
results
• Self and team reflection
Organizational
Goal
80% customer
satisfaction
Actual
performance
92% customer
satisfaction
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 36

knowledge influences were related to the process of strategic planning, the process of aligning
learning to business performance results and self and team reflection, as listed on the left side of
the stakeholder rectangle. The stakeholder motivation influences included goals, attributions, and
values, which are listed on the right side of the stakeholder group rectangle. The organizational
influences include three components: culture, alignment, and resource allocation, each relating to
the stakeholder group with a green arrow (Clark & Estes, 2008). In this context, a strong culture
is an organizational influence, which connects the organization and the stakeholder group with a
bidirectional arrow. The second organizational influence is alignment, which is a fundamental,
overarching influence affecting strategic planning, as well as the alignment of learning to
business performance results at DAU; this is depicted with a bidirectional arrow between the
organization and the stakeholder group. The third organizational influence is resource allocation,
which is denoted with an arrow pointing from the organization to the stakeholder group because
the organization is responsible for allocating the necessary and applicable resources to achieve
the organizational goal (Clark & Estes, 2008).
Three research questions drove this study to align with the conceptual framework.
Specifically, the focus of this study was to uncover and discuss whether and how the process of
aligning learning to business performance results had changed during the last 10 years after the
publication of the Anderson et al. (2008) book about the DAU’s success in leading a learning
revolution. The trifecta of knowledge, motivation, and organizational influences were examined
first individually and then concerning each other in order to uncover possible influences,
changes, and further improvements that may have influenced the continuation and acceleration
and recognition of DAU’s success in aligning learning to business performance results. Through
the lens of the Clark and Estes (2008) gap analysis model, the researcher examined the influence
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 37

of culture, alignment and resource allocation on the knowledge and motivation of the stakeholder
group. In closing, this conceptual framework articulated the link between the prevailing literature
around the history of learning measurement, the challenges, and opportunities of aligning
learning to business performance results, and the interplay of knowledge, motivation, and
organizational influences on the stakeholder group at the Defense Acquisition University. An in-
depth examination of this interplay amongst these three influences which enabled the stakeholder
group to reach the organizational goal was the focus of this promising practice study. As a result
of the examination of this interplay, the goal of this study was to glean and articulate best
practices and possible solutions which other organizations can adopt and adapt in their efforts to
also align learning to business performance results. This conceptual framework drove the
research methodology.
Table 6 below outlines the research questions, the assumed influences examined in this
study and the assessment methods used for each. The instrumentation coding is indicative and as
marked. It must be noted that the responses to the third research (RQ3) question above were
gleaned as best practices and as a result of the responses for Research Question 1 (RQ1) and
Research Question 2 (RQ2).

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 38

Table 6
Research Questions, Assumed Influences, and Assessment Instruments
Research Questions Alignment with Assumed Knowledge Influences and Assessment
Instruments

RQ1: What were the knowledge, motivational influences that enabled DAU to become and remain
a
best-practice organization in the alignment of learning to business performance results?

RQ2: How did organizational influences impact the alignment of learning to business
performance results?

RQ3 What solutions and recommendations in the areas of knowledge, motivation, and
organizational resources may be appropriate for aligning learning to business performance
results at another organization?

Research
Question

Assumed Knowledge Influences
Knowledge Influence Assessment

Indicative Core Interview
Questions
Knowledge
Influence Assessment

Indicative
Document and
artifact
reviews
RQ1

Procedural (A)
Stakeholders calibrated the
processes around data collection
and analysis to respond to new
learner needs and technology
changes
Interview Questions as noted in the
Interview Protocol
Review of
books, annual
reports,
strategic
plans, award
response
submissions
RQ1

Procedural (B)
Stakeholders adapted and
expanded specific strategic
planning and implementation
practices
Interview Questions as noted in the
Interview Protocol
Review of
books, annual
reports,
strategic plans,
award
response
submissions
RQ1

Metacognitive
Stakeholders conducted self-
assessment and self-reflection
Interview Questions as noted in the
Interview Protocol
Review of
books, annual
reports,
strategic plans,
award
response
submissions to
seek evidence
of
metacognitive
knowledge
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 39

Research
Questions

Assumed Motivation Influences

Motivation (M)
Influence Assessment
Indicative Core Interview Questions
Motivation
Influence
Assessment
Document and
artifact
reviews
RQ1

Attributions (A)
Stakeholders explained the
organizational success by
attributing it to their hard work
and efforts
Interview Questions as noted in the
Interview Protocol

RQ1

Goals (G)
Stakeholders explained the
organizational success by
attributing it to their hard work
and efforts
Interview Questions as noted in the
Interview Protocol

RQ1

Self (SE –P) and team efficacy
(SE-T)
Stakeholders, both as individuals
and as teams, believed they
would succeed
Interview Questions as noted in the
Interview Protocol

Research
Question
Assumed Organizational
Influences
Organizational Influence (O)
Assessment

RQ2

Culture (C)
The organization through a
strong
culture cultivated accountability
and
performance

Interview Questions as noted in the
Interview Protocol
Review of
books, annual
reports, strategic
plans, award
response
submissions
RQ2 Alignment (Al)
Stakeholders implemented a
comprehensive system of
alignment and reporting
Interview Questions as noted in the
Interview Protocol
Review of
books, annual
reports,
strategic plans,
award response
submissions
RQ2 Resource Allocation (RA)
The organization has adequate
resources including time,
personnel, and funding
Interview Questions as noted in
the Interview Protocol
Review of
books, annual
reports,
strategic plans,
award
response
submissions

Participating Stakeholders
As the researcher discussed earlier in this analysis, the stakeholder group for this study
included learning administrators from the Office of the President, the Foundational Learning
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 40

Directorate, the Workflow Learning Directorate, and the Defense Systems Management College
at the Defense Acquisition University. This stakeholder group represented the learning
administrators who drove the efforts of organizational change at DAU which resulted in aligning
learning with business performance results and surpassing the organizational goal of 80% by
12% in 2015, which established DAU as a world-class corporate university (Anderson et al.,
2008; DAU, 2015).
For this qualitative research analysis, the appropriate sampling business performance
results were nonprobability, purposeful sampling. The researcher aimed to explore, comprehend,
and investigate the problem of practice from a sample that could offer the most learnings
(Merriam & Tisdell, 2015). As such, the sample was mainly selected because of their specific
experience and expertise in leading the promising practice achievements at DAU (Anderson et
al., 2008; DAU, 2015; Merriam & Tisdell, 2015). Thus, it was a unique sample because its
selection is based on three criteria (Creswell & Poth, 2014; Maxwell, 2013): first, it included
staff that have worked at DAU since 2000 to ensure that they would be able to provide salient
details and background to DAU’s journey to excellence; second, it included staff from different
ranks in the organization to ensure a holistic perspective is provided in the analysis of policies,
procedures, and business performance results in DAU’s journey to excellence; and third, the
sample included both men and women in order to ensure inclusion and diversity in the
information provided for this analysis (Merriam & Tisdell, 2015).
Furthermore, there were three critical goals of this purposeful sample selection. The first
goal was to ensure the representativeness of the settings, individuals, and activities to best
capture salient information that would illuminate the purpose of this promising practice
(Creswell & Poth, 2014; Maxwell, 2013); the second was to focus on the criticality to the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 41

problem of practice (Maxwell, 2013); and the third was to leverage the robustness and
productivity in the business relationship with the select group of sample employees (Maxwell,
2013). Apart from conducting interviews with the people in the sample, the researcher also
performed a review of relevant organizational documents (Maxwell, 2013; Miles, Huberman, &
Saldana, 2013). In the sections that follow, the researcher discusses the sampling criteria,
sampling strategies, and rationale for the interviews, and documents and artifacts (Maxwell,
2013; Merriam & Tisdell, 2015) and provides a rationale for the choices made for this study.
Interview Sampling Criteria (Recruitment) Business Performance Results and Rationale
The interviewing sample was purposeful, consisting of an adequate number of people to
allow reasonable coverage of the topics around the study (Patton, 2015). The total population of
Faculty and Staff and DAU is about 700 employees. For practical reasons driven by limited
resources of time and access, the researcher conducted interviews with six staff members from
the Strategic Planning and Analytics group, the Workflow Learning Directorate, the
Foundational Learning Directorate, and the Defense Management Systems College, whom the
researcher selected based on the following three criteria and rationales.
Criterion 1. The DAU staff were required to have worked at DAU since 2000 to ensure
that they would be able to provide salient details and background to DAU’s journey to
excellence;
Criterion 2. The researcher sampled DAU staff from different ranks in the organization
in order to provide a holistic perspective in the analysis of policies, procedures, and business
performance results in DAU's journey to excellence
Criterion 3. The researcher sampled both male and female DAU Staff in order to ensure
inclusion and diversity in the information provided for this analysis (Merriam & Tisdell, 2015).
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 42

Document Data Mining Sampling Criteria and Rationale
To conduct document data mining, the researcher set the following three sampling
criteria with a common rationale of adequate coverage. These criteria ensured that the gathered
books, documents, and artifacts could provide the initial details to address the problem of
practice adequately and be complemented by unstructured interviews discussed earlier in this
section. The two sampling criteria are as follows:
Criterion 1. The researcher selected public and internal documents that describe DAU’s
journey to excellence including a book published in 2010, annual reports, strategic plans, best
practice winner ranking reports, and other documents used internally such as strategic planning
processes, applications submitted to learning and development industry award competitions,
policies and procedures, manuals, and website.
Criterion 2. The researcher selected artifacts including the website, trophies, and awards
earned by DAU.
Data Collection
For this study, the researcher used two data gathering methodologies. The researcher first
reviewed secondary research in the form of articles, books, journals, policy and procedure
documents, and strategic planning tools and methodologies. Next, the researcher conducted in-
depth interviews. Given the purpose of this study, these two methods served best the data
gathering driven by the research questions in this study.
Interviews
To address the research questions for this study, which required in-depth discussions with
members of the stakeholder group, the researcher conducted one-on-one, semi-structured
interviews with a purposeful sample of six members of the stakeholder group of learning
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 43

administrators described earlier (Maxwell, 2013; Merriam & Tisdell, 2015). Semi-structured
interviews were most suitable because all questions could be used flexibly, did not require a
predetermined wording or order, and allowed enough flexibility for the conversation to go more
in-depth as needed to uncover salient information about the DAU's journey to excellence in
aligning learning to business performance results (Anderson et al., 2008; Merriam & Tisdell,
2015).
Specifically, the question types that the researcher included in the interviews consisted of
knowledge questions to uncover the knowledge influences; values and opinion questions to
discover the motivational influences and experiences; and behavior questions to reveal the
organizational influences (Merriam & Tisdell, 2015). The interviews were conducted online and
were recorded for transcription and further analysis. In addition, written notes were taken by the
researcher during the interviews to ensure that salient points were not missed (Merriam &
Tisdell, 2015). The researcher selected six interview participants from the stakeholder group of
learning administrators based on the specific criteria discussed earlier to participate in the study.
The researcher utilized an online transcription service, rev.com, to capture the interviews. The
researcher used a quantitative data analysis tool (NVivo) to code the data and glean essential
content and topic threads to examine how they related back to the knowledge, motivational and
organizational influences at DAU. The interview transcripts were compared to the written notes
for consistency, and were shared with the interview participants for feedback (Merriam &
Tisdell, 2015). The interview protocol has been included in Appendix A of this document.
Documents and Artifacts
Documents are defined as the collection of written, visual, and digital materials to be
used in this analysis, whereas artifacts are three-dimensional objects including awards and
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 44

trophies (Merriam & Tisdell, 2015). Documents and artifacts come in various forms; they may
include public documents, personal documents, and other objects. These were used in
conjunction with the data collected from the interviews to analyze the critical organizational
influences on the knowledge and motivation of the stakeholders (Clark & Estes, 2008; Merriam
& Tisdell, 2015). Public documents may include books, annual reports, policy and procedure
manuals, and strategic plans. Personal documents which may convey the views and beliefs of an
individual on particular subjects, including individual records, notebooks, and emails. Third,
artifacts may include websites and online content, flyers, posters, handbooks, and agendas,
among others (Creswell & Poth, 2014).
For this study, the researcher used only select public documents and artifacts. The
interviews supplied additional confirmation to the data gleaned from the documents through the
personal views and perspectives of the stakeholders (Merriam & Tisdell, 2015). Specifically, the
researcher reviewed policies and procedures, annual reports, books, interim reports, and strategic
plans in order to discern how they documented and described the benchmarking process, the
types of data and data collection methods used by DAU, and the broader learning measurement
and alignment process DAU built and implemented. Additionally, the researcher collected
various artifacts, including the DAU website, strategic planning agendas, and photographs of the
awards and trophies DAU earned during the past 5 years. These artifacts offered additional
supporting documentation which helped complement the research effort in this study (Maxwell,
2013; Merriam & Tisdell, 2015). While the authenticity and the accuracy of documents and
artifacts could be considered as two critical downfalls of the documents review process, as
Merriam and Tisdell contended, the benefits offered by such data gathering methodology are
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 45

many, including ease of access, limited to no cost, rich in content, and stability, because such
documents are not subject to change considering the research or the presence of the researcher.
Data Analysis
After the data were collected through interviews, the researcher transcribed the interviews
using rev.com, a transcription service. Next, the data were coded using a qualitative data analysis
(QDA) online tool NVivo. The various codes were created and referenced against the
knowledge, motivation, and organizational influences that were classified in the literature review
in Chapter Two of this study (Maxwell, 2013; Merriam & Tisdell, 2015). Likely, the documents
and artifacts were reviewed, and salient data gleaned, coded, and referenced against the
knowledge, motivation, and organizational influences (Merriam & Tisdell, 2015). After coding
and analysis, the researcher collated, synthesized, and presented the best practice solutions and
recommendations, which may be used by other organizations to align learning to business
performance goals. This synthesis and discussion is found in Chapter Five of the current study.
Credibility and Trustworthiness
Credibility refers to the methods used by the researcher to carry out the study and
whether these methods are credible for the analysis (Merriam & Tisdell, 2015). Trustworthiness
pertains to whether the reader can trust the integrity of the researcher and the analysis conducted
(Maxwell, 2013). Five strategies to enhance credibility and trustworthiness were deployed in this
study: complementarity, member checks/respondent validation, reflexivity, audit trail, and rich
descriptions (Creswell & Poth, 2014; Maxwell, 2013; Merriam & Tisdell, 2015). Additionally,
the role of the researcher was examined as a research instrument (Creswell & Poth, 2014).
First, by using complementarity, which pertains to using two or more data analysis
methods together, the researcher aimed to gain complementing perspectives and different angles
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 46

to the same issue to enrich the data (Maxwell, 2013). Second, the researcher used member
checks by asking interview respondents to validate the emerging findings from the interviews
and the document analysis. Member checks ensured that the findings were not misinterpreted or
misrepresented in the analysis (Merriam & Tisdell, 2015).
A third strategy that the researcher used in this study was reflexivity, which pertains to
rigorous self-reflection by the researcher regarding any assumptions, biases, industry
perspectives and theoretical orientation that may influence the data collection and analysis
(Merriam & Tisdell, 2015). Reflexivity is a limitation that must be acknowledged here as it
implies some degree of influence the researcher brings to the data sampling and data collection
methodologies discussed as the researcher herself also becomes an instrument of the research
(Kuper, Lingard, & Levinson, 2008). Furthermore, per Kuper et al., reflexivity relates to the
broader issue of ethics, confidentiality, and public scrutiny which are discussed in the next
section. The fourth strategy used was an audit trail, which involves a detailed account of the
methodology followed, the processes implemented, and how decisions were taken throughout the
collection and analysis of data (Merriam & Tisdell, 2015).
Lastly, the fifth strategy involves rich descriptions, which required the researcher to
describe in detail the setting, the employees, and the findings by providing evidence gleaned
from interview respondent quotes and from field notes and documents examined during the study
(Merriam & Tisdell, 2015). The researcher gathered rich descriptions of the DAU setting, quotes
from the stakeholder interviews, and notes from the documents and artifacts examined.
Role of the Researcher
Given that in qualitative research the research is a human instrument of data collection, a
critical component of adhering to ethical principles is the role of the researcher thought the study,
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 47

given (Merriam & Tisdell, 2015). At the beginning of this study, the researcher was not
employed by the DAU or the federal government. After data collection was completed, however,
the researcher was hired by the DAU. The researcher is not receiving any monetary remuneration
for the completion of this study. Per Merriam and Tisdell, a credible way to recognize the role of
the researcher as an instrument is to ensure the researcher is aware of and should strive to limit
her inherent biases, predispositions, and attitudes. The researcher did this to the utmost extent
possible throughout the study, whereby she held a non-judgmental, sensitive, and respectful
stance towards the stakeholders and study employees at large.
Ethics
The focus of this promising practice study was the successful alignment of learning and
skills to the performance achieved by the Defense Acquisition University. To analyze such
performance, the researcher conducted qualitative research using two data gathering approaches:
carrying out semi-structured interviews and reviewing documents and multimedia artifacts
(Merriam & Tisdell, 2015). This research and the data gathering process followed three crucial
ethical principles: respect, beneficence, and justice to protect the interviewees involved in the
study and the data they provide (Beauchamp, 2008; Glesne, 2015; Merriam & Tisdell, 2015). To
apply these ethical principles, this study was submitted to the University of Southern California
Institutional Review Board (IRB). The researcher followed the IRB rules and guidelines
regarding the rights and the welfare of the employees in this study and the responsibilities of the
research. Additionally, a written exemption was obtained from the DAU based on the relevant
Federal Code relating to IRB requirements.
Next, after the IRB approval was received, the principles were applied throughout the
data collection and analysis process (Maxwell, 2013; Merriam & Tisdell, 2015). As discussed
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 48

earlier, the data collection instruments for this study included semi-structured interviews and
document review. Prior to the planning and implementation of the interviews, research
employees were invited to fill out an informed consent form which introduced the researcher and
the goal of the study, outlined their role and responsibilities, informed them about the logistics of
the study, ensured that they are aware that their participation is voluntary, protected the
confidentiality of all conversations, and underscored that their withdrawal at any time as possible
and did not bear a penalty (Glesne, 2015; Merriam & Tisdell, 2015). Furthermore, the consent
form delineated that the expected benefits of this study could be useful for the learning and
development industry and denoted that there were no known risks involved (Merriam & Tisdell,
2015). Additionally, employees were reminded that there were no incentives for their
participation to the interview so as they would not feel coerced to participate in the study which
could be considered in violation of the Belmont Report principle of respect (Beauchamp, 2008;
Merriam & Tisdell, 2015).
During the interviews, permission was requested to audio record the interviews. An
interview protocol included all the open-ended structured questions that the researcher developed
to examine the organizational influences on the knowledge and motivation of the study
employees regarding the attainment of the organizational goal (Maxwell, 2013; Merriam &
Tisdell, 2015). After conducting the interviews, the resulting transcripts were provided to the
employees and they ensured that their words were not changed (Rubin & Rubin, 2011). At the
end of the study, “thank you” note cards were sent to reiterate the researcher’s appreciation for
their participation, their time, and effort. To successfully conduct the review of documents and
objects, as noted per Glesne (2015), the researcher ensured that the documents were copied
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 49

verbatim, and were stored and secured properly in a computer accessible by password and a
locked office to protect the confidentiality of the participants.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 50

CHAPTER FOUR
Findings
The purpose of this promising practice study was to examine the knowledge, motivation,
and organizational influences present in the practices which enabled DAU to successfully align
learning to business performance results and excel as a learning and development provider.
Specifically, the study examines DAU’s performance between 2008 and 2018 as a promising
practice related to the process of successful alignment of learning to business performance
results. The following three questions guided the study:
RQ1: What were the knowledge, motivational influences that enabled DAU to become a
best-practice organization in the alignment of learning to business performance goals?
RQ2: How did organizational influences impact the alignment of learning to business
performance goals?
RQ3: What solutions and recommendations in the areas of knowledge, motivation, and
organizational resources may be appropriate for aligning learning to business performance goals
at another organization?
The assumed influences were identified and validated in the literature review provided in
Chapter Two. Additionally, in Chapter Three, the researcher provided the conceptual framework
and the analysis methodology plan for the study which is anchored in reviewing fifty documents
and three videos, as well as conducting online, semi-structured, interviews. Each interview was
recorded, and notes were taken during the online calls. After completion, the interviews were
sent to a transcription service. The recordings and the resulting transcriptions were sent to the
interviewees for member checking. Subsequently, both the documents and the interviews were
coded and analyzed for key themes and emergent themes (Merriam & Tisdell, 2015). In this
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 51

chapter, the researcher revisits the participating stakeholders and presents a discussion of the
findings organized by knowledge, motivation, and organizational influences identified in the
conceptual framework. The researcher also discusses applicable emergent influences that were
not previously identified in earlier analyses.
Participating Stakeholders
While a complete study would focus on all stakeholders, for practical purposes, the
researcher focused on DAU staff members from the Office of the President, the Foundational
Learning Directorate, the Workflow Learning Directorate, and the Defense Systems Management
College. These learning administrators are all part of the Defense Acquisition University, and are
both users and influencers in the alignment of learning to business performance results.
The Strategic Learning and Analytics group is responsible for the strategic planning
process at DAU, as well as for the measurement of learning analytics across over 2,000 earning
experiences which include courses, workshops, job support tools, and consulting engagements
offered by DAU. For this study, the researcher conducted an online interview with Dr.
Christopher Hardy, Director of Strategic Learning, and Analytics. Dr. Hardy is one of the three
authors of the book Leading a Learning Revolution: The Story Behind DAU's Reinvention of
Training, which inspired this study. Dr. Hardy has worked at DAU since 2001 in various roles, in
which he has been instrumental in driving the architecture and implementation of DAU's
excellence journey analyzed in this study. The United States Distance Learning Association has
awarded Dr. Hardy the Eagle Award in recognition of lifetime achievement in learning.
From the Foundational Learning Directorate, Dr. Kurt Stonerock is the Dean of the
College of Contract Management at DAU. Contract Management is one the significant areas of
acquisition. Dr. Stone rock's input provided the perspective of one of the key learning areas and
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 52

colleges in DAU which works closely with the Defense Contracts Management Agency (DCMA)
at the United States Department of Defense and trains over 60,000 learners per year across all
areas of contract management.
From the Workflow Learning Directorate, Ms. Brenda Sedlacek, PMP is the Director of
Operations and Analytics in the Workflow Learning Directorate of DAU. Prior to joining DAU
in 2009, Ms. Sedlacek was the Dean of Engineering and Technology at Tidewater Community
College in Virginia. Today, her team manages all analytics, operations, and communications for
the Workflow Directorate.
The Defense Systems Management College is a branch campus within DAU offering
global defense acquisition executive level courses and services to executive leaders in the
defense community. DSMC also houses the Acquisition Leadership Learning Center of
Excellence which has played a significant role in the excellence journey of DAU. For this study,
two members of DSMC participated in interviews. Mr. Patrick Wills, Dean of DSMC. Mr. Wills,
a retired United States Marine Corps commissioned officer, has over 40 years of experience in
acquisition from various sectors including the US military, private industry, and the Defense
Acquisition University. He developed and led the congressionally mandated Requirements
Certification Training Program. This Program has become the exemplar for all non-Defense
Acquisition Workforce communities. Mr. Wills has become DAU's preeminent authority on the
Joint Capabilities Integration and Development System (JCIDS), which is the formal United
States Department of Defense procedure governing acquisition requirements and evaluation
criteria for future defense programs (DSMC, 2017).
Ms. Maryann Watson is the Deputy Director of DSMC. Ms. Watson served in the United
States Airforce in acquisition leadership roles at the Pentagon for over 33 years, and joined DAU
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 53

as professor of Program Management in 2012. After 3 years, she became Associate Dean for
Academic Affairs at DSMC and later Deputy Director of DSMC. Within the DSMC, Dr. David
Gallop, PMP, CSEP was also interviewed. After serving in the United States Army for 21 years
in various acquisition and tactical leadership roles, Dr. Gallop worked in the private sector for 4
years and then joined DAU in 2009. He provided the faculty perspective in this study.
Knowledge Assets
In this section, the researcher discusses the knowledge findings resulting from the data
analysis. Specifically, three knowledge assets were encountered in alignment with the conceptual
framework discussed in Chapter 3 as seen in Table 7, and three additional findings emerged.
The assumed knowledge influence findings assets are: (a) procedural knowledge:
strategic planning process, (b) procedural knowledge: data collection and analysis, and (c)
metacognitive knowledge: self and team reflection. The three emergent knowledge influence
findings are: (a) benchmarking, (b) accreditation, and (c) learning and development industry
award submissions. In the following sections, the researcher will discuss each of these in detail.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 54

Table 7
Knowledge Assets
Knowledge Assets Selected Citations
Procedural
Stakeholders adapted and expanded specific
strategic planning and implementation
practices
Bird, 2008; Kirschner et al., 2006;
Krathwohl, 2002; Pintrich, 2002;
Rogers, 2013
Procedural
Stakeholders, by calibrating the processes
around data collection and analysis to
respond to new learner needs and
technology changes, strengthened their
business acumen in which metrics to
measure
Bandura, 2007; Kirkpatrick, 2006;
Wood and Bandura, 1989;
Metacognitive
Stakeholders conducted ongoing self-
assessment and self-reflection
Ben-Hur et al., 2015; Krathwohl, 2002;
Pintrich, 2002; Stuckey, 2012

Assumed Knowledge Influence Assets
Discussed in this section are the three assumed knowledge influence findings: (a)
strategic planning process, which includes strategic planning goals, architecture, communication,
and implementation; (b) collection and analysis; and (c) self and team reflection.
Strategic planning process. The DAU has developed a robust organizational strategic
planning process which incorporates DAU's learning strategy, the Acquisition Learning Model,
allowing DAU to set strategic goals and successfully align learning to the business performance
results expected of the learners by their organizations within the United States Department of
Defense. In the case of DAU, the learners are the acquisition workforce and the business
performance results they are expected to include the systems, services, and equipment necessary
to support the warfighter. The Acquisition Learning Model was developed by DAU in 2008, and
it is still used today to align all the learning experiences developed by DAU for the learners to
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 55

the business performance results they are required to deliver for their organizations. Four
knowledge factors drive the successful development and implementation of the ALM strategic
process at DAU: knowledge of organizational goals, strategic plan architecture, communication,
and implementation.
Strategic organizational goals. The DAU elevated learning to mission-critical status by
incorporating its learning strategy into the overall DAU strategic planning process. DAU did this
by creating three new strategic goals relating to foundational learning, workflow learning, and
performance learning, plus two additional supporting goals of people and infrastructure. All five
goals have metrics of success relating to business performance results. Specifically, the
organizational goals of the DAU are driven by the Defense Acquisition goals set by the United
States Department of Defense and related to the national security needs of the nation. Defense
Acquisition (DA) is implemented by the acquisition workforce of 160,000 learners who are
working spread across the Army, Navy, Marine Corps, Air Force, and numerous other agencies
within the United States Department of Defense. The Defense Acquisition process is a highly
regulated and extremely complicated process consisting of several correlated components
including human resources, funding, and technology. The defense acquisition process consists of
five very complex stages of evaluating, prototyping, developing, deploying, and maintaining
systems to support the warfighter. Each step of the defense acquisition process involves several
levels and layers of checks and balances to ensure that all the requirements are met before
proceeding to the next stage. The acquisition workforce is required by law to learn and certify in
acquisition before they can apply the process. The Defense Acquisition University provides all
the acquisition certifications training and all related learning, as required by law, to all the
acquisition workforce. During the interviews, Dr. Hardy said: “If our learning programs are not
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 56

aligned with business performance results, the national security is at stake.” Because the DAU
learning programs focus on efficiently teaching such a complicated and important process, the
DAU has recognized the need to ensure that all the learning programs it provides align to the
business performance results the learners are required to achieve. The stakeholders in this study
had to learn the acquisition process through training to be able to understand and then describe
the components of the ALM model, starting with the organizational goals.
Strategic planning architecture. DAU incorporated ALM, its learning strategy, as a key
driver within its broader organizational strategy. The DAU strategic plan is a 3-year
performance-based plan and includes five goals: (a) foundational learning, (b) workflow
learning, (c) performance learning, (d) people, and (e) infrastructure. The architecture of the
strategic plan is shown in Figure 2 below, from a photograph of page 23 of the DAU Strategic
Plan 2016-2019.

Figure 2. The acquisition learning model.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 57

The ALM consists of three diamonds, as seen in Figure 2. Each diamond represents a
business directorate or division within the DAU. Each directorate is responsible for the design,
development, and management of the learning assets it governs and includes teaching and
learning faculty and staff. The first element and directorate is Foundational Learning and
includes over 400 in-classroom foundational acquisition learning courses, targeted training, and
rapid deployment training; the second element and directorate is the Workflow Learning which
provides the technology that supports and maintains over 600 performance support tools,
including job aids, glossaries, checklists, and matrices developed by faculty and the workforce.
Finally, the third element and directorate is Performance Learning which includes individual
qualification, team training, and mission assistance whereby DAU faculty delivers customized
consulting in the field. Figure 2 depicts the ALM at the center of the DAU Strategic planning
process, the implementation of which will be discussed in a subsequent section.
In the DAU 2015 Annual Report President's letter, the ALM model is described as
follows: “The ALM is now the backbone of the University; it ensures that all parts of our
organization collaborate to incorporate the needs of the workforce into our learning assets.”
DAU built the procedural knowledge around the learning strategy of the organization with the
ALM model and subsequently incorporated it in the overall organization. By incorporating the
learning strategy within the overall strategic planning process architecture of the organization,
DAU successfully aligned learning to business performance results. This alignment was done by
engaging with each directorate and ensuring that they understand the procedures of how the
architecture functions by directorate, within the ALM, and within the organization as a whole.
Once the architecture was learned, a communications campaign was launched to inform the
DAU Faculty and staff.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 58

Strategic planning communication. To successfully incorporate the learning strategy
into DAU strategic plan, DAU created and implemented communications campaigns targeting
senior leaders within the Department of Defense, agency partners, internal staff, and faculty, as
well and the defense acquisition workforce. These campaigns consisted of presentations aimed at
increasing the procedural knowledge of the audiences of how the ALM worked within DAU. An
awards application document reviewed in this analysis confirms: “Also promoted the acceptance
of our ALM through aggressive communications campaigns that targeted senior leaders within
the Department of Defense, internal staff, and faculty, the learning and development community,
and the defense acquisition workforce.”
In addition, DAU generated written communication documents to enhance the procedural
knowledge of the stakeholders further: first, the strategic plan, which includes the mission,
vision, goals, objectives, measures, and outcomes for 3 years, updated annually; second, the
organizational performance plan which has the measurable performance tasks for the current
year for each of the strategic goals; third, the organizational performance assessment which
shows the actual versus planned accomplishments for the year as well as progress for long-term
measures for each of the strategic plan goals; fourth, the annual report which describes DAU's
achievements for the preceding year; and fifth; the performance-based individual contribution
plans for both faculty and staff. These plans will be discussed in a later section.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 59

Figure 3. Example of organizational goal.
Figure 3 above shows an example of Goal 2, Workflow learning in two consecutive pages
26 and 27 from the DAU Strategic Plan 2016-2019. Page 26 on the left describes Goal 2 at the
top, followed by three objectives, and a corresponding metric for each. At the bottom of the page,
the goal expected outcome is listed. The following page, on the right, shows a table for Goal 2,
consisting of five columns. The far-left column lists the three objectives seen on the left of
Figure 3. Columns 2-5, with the headings FY15, FY16, FY17, and FY18 show the fiscal year
operationalization plans for each objective.
As seen in Figure 3, DAU is very transparent with information and shares all these
reports online in a uniquely designed shared dashboard and data mart. These practices are
evidenced in the DAU Strategic Plan 2016-2019 and in three award applications 2015, 2016, and
2017:
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 60

Our CEO and stakeholders receive a formal Annual Report depicting DAU's high-level
accomplishments for the year. A separate, more detailed Annual Organizational
Performance Assessment document is also provided. Both lay out DAU's organizational
performance and report in detail on the value as a strategic asset to the enterprise.
Additionally, operational and performance metrics and outcomes are also reported
monthly and again aggregated quarterly on a data mart and dashboard.
Furthermore, communication around strategic planning which enhances the related
knowledge around the strategic planning process included informal conversations taking place
continuously through weekly meetings within teams and across cross-functional teams at the
leadership and tactical level. On this point, the Dean of DSMC said: “ALM has been a helpful
framework to get everybody to have a conversation about planning and move forward in an
enhanced way, versus how we started in our early years.” Here, the DSMC Dean describes the
ALM model as a framework which enables all DAU employees to communicate about goals and
progress ahead in their strategic planning process in a meaningful way which is more well-
rounded and developed today, than it was a decade ago. Furthermore, the DSMC Deputy
Director described the ALM model as such: “The ALM is a new way of thinking about our
delivery channels. It allows us to do our planning in a very constructive way so that we do not
forget any of the dimension.” Both the document analysis and the interviews confirm that
communication is central to enhancing the knowledge around the procedures of successfully
incorporating the learning strategy into the organizational strategy during implementation.
Strategic planning implementation. As discussed in the Strategic Planning Architecture
section, the DAU Strategic Plan, which is the overall organizational plan is anchored in the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 61

Acquisition Learning Model. Before reviewing the strategic planning implementation process at
DAU in this section, the researcher will examine the various implementation components.

Figure 4. The Defense Acquisition University strategic planning process.
As seen in Figure 4 above, which is taken from a learning and development award DAU
submission document, the downward arrow depicts the central location of the ALM learning
strategy and its incorporation into the DAU strategic planning process. According to two learning
and development industry award applications, and DAU annual reports for 2015 and 2016, the
strategic planning architecture consists of seven components. First, the process begins with the
mission and vision of DAU; it proceeds with a situational review of new policy, new legislation,
emerging best practices, customer feedback, and performance assessments. It then includes a
Strengths-Weaknesses-Opportunities-Threat (SWOT) analysis of DAU; fourth, the ALM
strategic goals; fifth, the DAU strategic plan; which cascades into the sixth component, which
includes the annual performance plans for each division of the DAU. These cascade into the
seventh component, the individual employee objectives, which comprise the seventh component
of the DAU strategy architecture. These are updated mid-year, and then annually. The annual
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 62

performance review feeds back into to the situational review to close the loop. Next, the
researcher will describe the strategic planning implementation process at DAU.
Two documents including the DAU 2016-2019 Strategic Plan, the DAU Board of Visitors
minutes from 2016, and the interview responses from the Director of Strategic Planning and
Learning Analytics who leads the strategic planning process at DAU, describes the strategic
planning process implementation at DAU as follows. Each year DAU reviews, updates, and
validates the strategic goals, objectives, and measures it has set for that year. The strategic plan is
a performance-based 3-year plan. The first year includes five strategic goals, three of which
pertain to the learning strategy model, ALM; the other two refer to people and resources. These
five goals are articulated by more than 100 performance tasks, which are allocated to each of the
three ALM Directorates to be completed during the year.
First, a SWOT analysis of DAU is conducted. The strategic planning process begins with
the Director of Strategic Planning and Learning Analytics who engages DAU senior leaders,
faculty, and staff to initiate the strategic planning process with a SWOT analysis. The SWOT
analysis employees, anchored in the mission and vision of DAU, conduct an environmental scan
to identify and analyze four components: (a) changes in the Department of Defense, warfighting,
and acquisition environments; (b) improvements in the learning and development industry; (c)
critical shifts in the competition, market, technology, and regulatory environments; and (d) long-
term DAU sustainability. These tasks reflect both organizational and learning strategy goals and
are cascaded down through our leadership team to individual faculty and staff via their individual
development plans. This ensures that each individual working at DAU is clear on what the
organizational goals are and how their specific assigned tasks and individual performance goals
contribute to the achievement of the organizational and learning strategy goals. In addition, the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 63

SWOT team reviews the preceding year mission and operational performance, customer
satisfaction ratings, stakeholder inputs, climate survey results, and annual performance tasks and
accomplishments. These components are analyzed and used to conduct the DAU SWOT
analysis.
Next, after the SWOT analysis is completed, the strategic plan ALM goals are updated
based on the results of the SWOT analysis and presented to two reviewing bodies: (a) the DAU
Strategic Planning Council (SPC), which consists of DAU’s senior leadership team, and (b) the
DAU Board of Visitors (BoV), which consists of DAU partners from the DoD, the defense
industry, as well as learning and development thought leaders and professors. After the review,
the SPC approves the new strategic plan.
The importance of the knowledge of the details around the practical application of the
ALM model in the implementation of the strategy and into the day-to-day operations of DAU
was discussed and referenced in four of the six interviews conducted in this study. The DSMC
Deputy Director said: “So, the ALM allowed us to visualize that and not lose focus that we can't
just be about teaching or delivery of curriculum in a classroom. It has allowed us to better focus
on our energy.” The DSMC Deputy Director’s words show that in the DSMC context, that
knowledge of the ALM model has practical value as it helps internal teams to visualize all the
avenues they need to leverage to reach their customer, the learner and to conduct strategic
planning in a well-rounded way, without omitting any critical components. Additionally, the
Deputy Director stated that the ALM helped the internal teams to focus on the essential goals,
not only as they pertain to their learner in the classroom, but also, as they pertain to continuous
internal improvement accomplished by channeling of the organization’s energies. Therefore,
procedural knowledge was instrumental in implementing the strategic planning process.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 64

Moreover, the Deputy Director’s use of the word focus in two instances “not to lose
focus” and “to better focus” demonstrates how individuals and teams increased their procedural
knowledge by internalizing the role of the ALM both as a medium for concentration on the most
important items as well as a path for continuous improvement in the present and the future
beyond just providing learning. The Professor of Program Management at DSMC said: “[we are
always reinforcing with our learners and beneficiaries] that we're not delivering something to
stand alone, that it fits into a larger framework.” The words “always reinforcing with our
learners” and “fits into a larger framework” used by the Director of at DSMC validate that the
ALM offers practical value during day-to-day interactions with learners and serves as the
explanation for the ‘why’ of the learning provided by DAU.
Data collection and analysis. The second assumed knowledge finding relates to data
collection and analysis. DAU developed a rigorous data collection and analysis process around
all learning provided by the university. This data collection and analysis facilitates the strategic
planning process described in the previous section, which in turn, drives the alignment of
learning to business performance results.
Data collection. At DAU, a rigorous data collection process is implemented to collect
information about the effectiveness of learning and the degree of alignment between the learning
and the expected business performance results. Data is primarily collected through electronic
surveying of over 160,000 learners annually across over 400 instructor-led courses and 300
online learning courses. Two surveys are administered: the first survey is sent out electronically
to the learners immediately after the learning program the attended ends, to gauge the learner's
reaction and learnings related to the course, in line with Kirkpatrick's Level 1 and 2. The second
survey is sent out electronically after 60 days. The learners are surveyed again; at this point, their
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 65

immediate supervisors are also surveyed separately, which allows DAU to collect data on any
behavioral changes resulting from the learning programs the learners attended, as well as data on
the results delivered on the job because of the learning the learners received. The 60-day survey
aligns with the Kirkpatrick Levels 3 and 4 of learning evaluation.
During the interviews, Dr. Hardy said this about the rigor of data collection: “A lot of
people asked me, they said, ‘How do you know what [to collect] and measure?' And I say, ‘Well,
the first thing is you might want to inventory what you are measuring or not measuring, and
what's important to your business. And then, [based on] what's important to your business goals,
you try to align those with your learning strategy.'“ Here, the Director of Learning analytics very
merely discusses the approach DAU has increased its procedural knowledge on data collection,
starting with the business performance results that are important to the business and tying those
back to the learning content. Data collection for customer feedback centers on what is essential
to the customer, which in the case of DAU, constitutes what is vital to the business. The next
section discusses how DAU analyzes the data collected.
Data analysis. DAU uses data analysis of millions of surveys of learners and their
supervisors at 60 days after the training course occurred to evaluate whether the course
contributed to improving job performance results. This data analysis process is DAU's most
potent source of analytics insight evidencing the alignment of learning to business performance
results. DAU has built a database of millions of surveys over the last decade. This 60-day
feedback data from learners and their supervisors about the course is analyzed using text mining
of the qualitative responses to uncover theme typicality to issues with the course content, course
delivery, and applicability on the job. The data can be aggregated and analyzed by course topic
category within acquisition such as contracting, program management across different time
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 66

frames. Also, that data can be examined at the course level, and essential feedback is shared with
the course developers, instructional designers, and learning asset managers. These individuals are
responsible for the quality and content of the learning assets and can implement changes to adapt
the course to learner needs to drive business performance results. On this point, a 2015 award
submission document says:
More than 89 percent of learners are using or will use DAU course content on the job.
However, another view of the same data would be that 11 percent never use the content
on the job (which is low compared to industry standard of 40 percent). This level of
analysis has changed the conversation during curricula and new course development
reviews.
To conduct text mining, DAU invested in stronger, in-depth learning analytics technology
and resources. First, the words are mined and counted; next, patterns or themes are identified
from the word groups; finally, themes are isolated using dimensional reduction to arrive at
critical concepts. DAU uses Statistica's Text Mining Module. A 2016 award submission
document describes how DAU analyzes the survey data: “To accomplish this we invested in
more in-depth learning analytics capabilities to focus our attention on business results as our key
measure of success.” Text mining allows DAU to search for root causes and to interpret for
“meaning” described in the qualitative responses of the learners and their supervisors. As a
result, DAU manages to maintain the currency and relevance of over 400 existing courses and to
create about 40 new courses every year. “New course designs [are now aligned with business
performance results and] have now improved business performance results scores by 6.75
percent per year,” according to a 2015 award application submission document. This is also seen
in Figure 5 below, which is a photograph of slide included in a 2015 DAU award application.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 67

Figure 5. New course design.
The table in Figure 5 on the left shows the results of a course survey. The course is
CMM210, which is a Production Planning and Control course within the acquisition process
offered at DAU. The first column is divided into four sections. The first section shows the overall
rating of the course and the business results. The second section shows metrics around Job
Impact. The third section shows metrics around job effectiveness and the last section shows
metrics around return on investment. The second column shows the average score where
applicable (in green). The third column shows the survey results for Level 2 for this course. The
fourth and fifth columns will show Level 3 results for the student and Level 3 results for the
manager respectively. This was a new course and surveys at the 60 days after course completion
had not been administered; therefore, the fourth and fifth columns are blank. This figure is a
good snapshot example used in the data analysis process and depicts how DAU tracks Level 2
and Level 3 survey results for new courses across ten different criteria, two levels and at 0 and 60
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 68

days after course completion. As seen in a 2016 award application: “Our strategic analysis and
review have changed the level and scope of discussions during senior leader meetings and
curricula reviews. This impact has changed policy-level decisions that drive requirements on
who must attend which course based on data, not irrational needs.” Because of the in-depth
procedural knowledge, DAU has built the rigorous data collection and analysis process of
learning outcomes and results. It has been able to influence decision making relating to funding
and development of new learning curricula.
Self and team reflection. Self-reflection and team reflection are practiced continuously
at DAU. Employees practice self-reflection while developing their individual annual
performance plans; team reflection after the completion of courses and during the strategic
planning process to reflect on short-term and long-term lessons learned.
Self-reflection. Individual DAU employees are encouraged to practice self-reflection in
weekly team meetings. Within the directorate teams at DAU, employees reflect on a weekly
basis on the tasks they are involved in and discuss the challenges with their team. Specifically,
each DAU employee is encouraged to practice self-reflection while drafting of their individual
performance plan which includes personal development goals for the year. During self-reflection,
employees reflect on their accomplishments and contributions from the preceding year and their
impact on the progress of their directorate and DAU as a whole. They also adapt their individual
development plan for the upcoming year. Employees draft their goals and discuss them with their
supervisor. Each goal must follow the SMART approach, so goals need to be specific,
measurable, attainable, relevant, and time specific. Most importantly, each goal must align with a
specific goal and measure for the employee's Directorate as defined in the ALM learning strategy
discussed earlier in this chapter. As a result of self-reflection, employees' responsibilities are
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 69

directly aligned with the learning strategy and the overall strategic plan of DAU. The DAU
Human Capital Plan 2013-2017 discussed this point:
All employees are responsible for managing their own career and individual
developmental needs. At the beginning of the annual evaluation cycle, each faculty and
staff member formulate specific learning and development plans that meet strategic and
near-term requirements for the individual and the University. All learning and
developmental activities should be included in the employee’s IDP prepared in the Talent
Management System.
Team reflection. At DAU, teams within directorates and colleges—such as the DSMC—
practice rigorous team reflection during the strategic planning process for short and long-term
planning, as well as after each learning program, for continuous improvement.
Team reflection during strategic planning. DAU senior leadership teams engage in team
reflection once a quarter through the implementation of their strategic planning process. These
teams took the time to reflect and learn together from the results of the past quarter. They analyze
what is useful in their practices and what is ineffective and why. They discuss the reasons for
success or failure and investigate supporting data to ensure that they are making data-driven
decisions. As a result of this metacognitive exercise, they emerge as a stronger team who can
articulate why and how is best to move forward. With this clarity, they are able to communicate
more credibly to the Department of Defense leadership team and, in turn, align with their
priorities. The DAU leadership team created a web portal to facilitate this communication better
and to track progress in accomplishing the tasks leading to the DoD priorities. Explaining this,
the Director of Learning and Analytics at DAU said:
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 70

Because we could reflect as a team about what worked and what did not,
we managed to work together and to communicate effectively with the
U.S. Dept. of Defense leadership team. We aligned with their priorities,
especially on the efficiencies initiative which the leadership team used
DAU's web portal to communicate and manage, and proving our ability to
deliver results.
As seen in this Director’s quote, team reflection and the metacognitive knowledge garnered from
it helped them prove their ability to drive results.
Team reflection after learning program completion. Additionally, DAU leaders and their
teams use team reflection after learning program completion for continuous improvement to see
beyond the day-to-day operations and not to lose sight of the why of their efforts. While
achieving organizational goals is required, it is not sufficient because the team cannot rest on its
laurels. At DAU, both the leaders and their teams recognize that they must continue to reflect on
how to improve internally, so that that they can, in turn, meet the needs of their acquisition
learners. Specifically, the DSMC Deputy Director said “The challenge is recognizing that
achieving our goals is necessary, but not sufficient. That we need to continue to develop the
[acquisition] workforce. Not just to achieve certification, but to continue to improve and be
better.” The Deputy Director references that team reflection is important in promoting
continuous improvement. Additionally, the DSMC Dean said: “We hot wash constantly.”
According to the U.S. Department of Defense Education Activity, the term hot wash originates
from the U.S. military were soldiers spray their weapons with extremely hot water to remove grit
and residue and ensure that the cleaning process goes more smoothly. The Dean who provided
the quote served for 22 years in the US Marine Corps. In this context, the Dean used this term to
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 71

define the action of team reflection conducted immediately after a project concludes to review
what worked well and what requires additional improvements. Additionally, the professor of
Program Management said: “I think in the last decade, certainly in the last years or so that I've
been here, we've come to a realization that one of the most critical resources that we have an
obligation to really make sure we get the most out of, is the time we have with student learners in
a classroom.” The words the professor uses “in the last decade” and “we've come to a
realization” denotes team-reflection on what are the most critical resources and how to manage
them to be more effective.
The three quotes referenced above present and validate the metacognitive knowledge
assumed influence of self and team reflection in aligning learning to business performance
results. In this context, metacognitive knowledge gained through the practice of team reflection
has positively influenced the ability of the Defense Acquisition University to align learning to
business performance results.
Emergent Knowledge Influence Findings
In addition to the assumed knowledge influences discussed in the previous
section, three additional procedural knowledge influences emerged during the analysis of
documents and interview transcripts: benchmarking, accreditation, and learning and
development industry award applications. This section presents each of these emergent
knowledge influences and how they impact the successful alignment of learning to
business performance results.
Benchmarking. While benchmarking is not a new practice, the systematic rigor with
which DAU has implemented benchmarking to harvest, adopt, and adapt best practices from
other high performing organizations in other industries has proven beneficial in DAU's journey
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 72

to excellence. DAU has systematically identified opportunities for improvement in its processes
and has sought other best practice organizations in other industries to learn from them. As
encountered in the DAU 2016 Benchmarking presentation and across four of the six interviews,
DAU has used benchmarking within and outside the learning and development industry for over
15 years. The organization has successfully adopted and adapted best practices across people,
resources, and technology.
Specifically, in 2015, DAU benchmarked with Qualcomm, a digital communications
technology company, to explore their enterprise-wide social learning platform provided by
Pathgather, a learning platform provider. The learnings from the pilot were applied with DAU to
enhance the DAU continuous learning center to enable learners to access, manage, and create
their own learning experience.
During 2016, DAU benchmarked with Deloitte University in order to better understand
how to focus on millennials. Through the benchmarking visit, DAU learned about flat,
reconfigurable classrooms, as well as specifics about engaging millennials, including the 4
seconds rule to get their attention, preferences on self-directed, fast training, and the need to
access information independently. As a result of this benchmarking visit, DAU modernized all its
main campus learning facilities at Ft. Belvoir by adding telepresence and 360-degree classroom
set ups, the opportunity for learners to bring their own device (BYOD), shortening courses,
chunking videos, and using job support tools as part of courses. These improvements around
people were driven and implemented by a specific task included in the DAU 2016 Strategic Plan.
During 2016, DAU benchmarked with Abbott Laboratories, the pharmaceutical company,
to learn about executive business simulation. Specifically, DAU learned how executive teams at
Abbott work through intense computer-based business simulations to practice real-world
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 73

decision making; explored the various simulation features and observed how Abbott uses
behavioral observation and coaching. As a result of this benchmarking visit, DAU developed and
incorporated business simulations into executive training courses. These improvements in
learning technology were driven by specific tasks included in the DAU 2016 Strategic Plan.
Moreover, also in 2016, DAU benchmarked with openSAP, a technology company, to
learn about designing, developing, and offering Enterprise MOOCs (massive online open
courses) to DAU learners. Specifically, DAU learned how openSAP broke down weekly video
lectures into consumable 15-minute segments, incorporated discussion forums, self-tests, weekly
assignments, and social learning by encouraging learners to interact with peers to enhance the
MOOC experience. As a result of this visit, DAU developed and offered MOOCs for specific
acquisition certification courses. These improvements were incorporated into the 2016 Strategic
Plan and driven by specific tasks.
Finally, in 2016, DAU benchmarked with Hilton Worldwide, a hotel and hospitality
company, to learn more about how they used learning analytics and to track and show Level 3
and Level 4 learning value. The Hilton Worldwide Chief Learning Officer is a member of the
DAU Board of Visitors (BoV) and provided his organization's best practices in a future BoV
meeting.The LearningElite 2017 Application notes this about how DAU uses benchmarking:
DAU also has a robust benchmarking program where we seek out our peer organizations
with transferable best practices to adopt/adapt. We average four visits per year and, in
return, we share our best practices with others. In the last ten years, over 80 organizations
have benchmarked DAU. Benchmarking is a powerful tool to gain and maintain
leadership in our field.
The Director of Learning Analytics said this about benchmarking:
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 74

And you bring that back, and you fix that area that you're not doing. Let us say you are
not doing strategic planning or you do not have a learning strategy or something like that,
or you do not have a mobile, it could be a technology chip. So, sector leadership to me
plays a huge role, and it is got to be nurtured. You can't just do it once.
Additionally, the Dean of DSMC commented: “We're ever moving forward, and we're trying to
benchmark other high performing organizations, and with the learning and development
community, and infuse those best practices.”
As an emergent procedural knowledge influence, benchmarking has proven useful for
DAU, because it has been pursued continuously and systematically and the learnings have been
incorporated into specific tasks in the DAU Strategic Plan through the ALM model.
Accreditation. A second emergent procedural knowledge influence is the process of
accreditation for educational institutions. As a corporate university, DAU pursued and earned
accreditation from the three most prominent educational bodies in the United States: the Council
on Occupational Education, the International Association for Continuing Education and Training
(IACET), and the American Council on Education (ACE). Application and admission to these
organizations require a rigorous and thorough review of all functions within a university. As a
result, it promotes excellence in the process and application of a university’s educational
practices. In the case of DAU, these application processes strengthened DAU's focus on aligning
learning with business performance results.
Specifically, the DAU is accredited by the Council of Occupational Education (COE). In
its most recent accreditation site visit, DAU earned three commendations for educational
programs, strategic planning, and facilities. Earning an accreditation is a complex and detailed
undertaking addressing all functions of a university. It is a rigorous evaluation process that
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 75

includes submissions of documentation and evidence, followed by a site visit from the
accrediting body for an in-person evaluation. Because of its excellence in strategic planning,
DAU was invited to present its strategic planning process to COE National conference to over
1200 COE accredited learning organizations. To this, Dr. Hardy said during the interviews:
And we use the Council of Occupational Education because it's training better than
higher ed. That involves a huge self-study, and it is the first time you go through it it is all
your back end, all your resources, all your record-keeping, all, all, all the stuff that maybe
does not get visibility all the time. And thus, if you find holes there, you also use that to
fix them. And if you submit a self-study document to the accreditation agency, they will
show up with a team of 20 experts and visit you and track down what whether you are
doing what you said you are doing, and they will accredit you or not accredit you. We
have gotten back-to-back 6-year accreditations. We have no findings, and we got three
commendables on the last one.
Additionally, DAU is an International Association for Continuing Education and Training
(IACET) provider awarding CEU's for continuing education. IACET is the only standard-setting
organization approved by the American Standards Institute (ANSI) and also requires a rigorous
application and review process and site visit. Lastly, the American Council on Education (ACE)
providing college credit for DAU courses which enables the acquisition workforce to obtain
undergraduate and graduate degrees from over 100 universities. Through accreditation and the
practice of rigorous review and examination of one's own process and practices resembles
holding a mirror and viewing one is self. For DAU, such exercise highlighted both the operations
and processes that worked well and those that required improvements through better strategic
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 76

alignment, focus, and resources. In addition to accreditation, DAU practiced such improvements
further through submitting applications to learning and development industry awards.
Industry award applications. A third emergent knowledge influence is the depth of
procedural knowledge cultivated through the process of learning and development industry
award applications. During the past 15 years, DAU has won over 70 learning and development
industry awards. This record of accomplishment implies at least 70 successful applications were
submitted. As Dr. Hardy explained:
We did very well during that time, and within a few years we were starting to be
recognized even though we were the government. In 2002, we won best corporate
university, best in class, from the CUBIC Awards, and then we started ... We got into the
awards, and I treated it like a capture strategy. That was very deliberate.
One of such applications describes DAU's record of accomplishment:
DAU has repeatedly been recognized in both the public and the private sector. In the last
few years, DAU has repeatedly swept many of the major learning and development best-
in-class awards: 2017- LearningElite Gold Award; 2016 CLO Learning Elite Award (Top
10 last six years); CLO Magazine Learning in Practice Awards – three in 2016; American
Society of Training and Development Best Awards: Training Magazine Top 100;
Corporate University Best-In-Class Awards; Brandon Hall Gold Awards for Excellence in
e-Learning; ELearning! 100 Awards, etc. DAU was also named Best Corporate
University in the World in 2013 by the Global Council of Corporate Universities and Best
Corporate University in North America in 2014 and 2015 during the CUBIC Awards.
The application process for such awards including the LearningElite Award, the CUBIC
Award, Brendon Hall, and others; the process is rigorous and quite thorough. Applications
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 77

include up to 20 questions which require specific responses about the organization's learning and
development processes, application of metrics, description of learning strategies among others.
As such, responses require significant procedural knowledge around crucial learning and
development processes including learning strategy, the learner satisfaction, technology use, and
implementation of learning and development practices to enhance the learner experience. The
questions also require supporting evidence and data for each response which makes an
application a complicated and time-intensive process. As Dr. Hardy explained, “They [the
applications] usually take some effort, and a lot of it isn't just writing fiction. You actually have
to provide evidence, and if you do not have the evidence, then I tell people, ‘Go ahead and get in
there because you learn a lot about yourself and you also learn about what others are doing.’”
This is also evidenced in 2016 award application, which cites:
DAU is committed to providing its internal and external partners and customers with
best-in-class learning assets delivered in the most efficient and effective manner possible.
The university strives to be a leader in the training sector. DAU assesses its results in this
area qualitatively through participation in national training award competitions.
As evidenced in this section, five procedural and one metacognitive knowledge influence
findings played a significant role in the successful alignment of learning to business performance
results for the Defense Acquisition University. The findings provide a detailed response to the
first research question which asks about the knowledge and motivational influences in aligning
learning to business performance results. In the following section, the researcher will review the
motivation influence findings.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 78

Motivation Influence Assets
In this section, the researcher discusses the assumed motivation influence assets
based on the data analysis. These assets, as seen in Table 8, include attributions, goals,
and values, as well as their influence on the alignment of learning to business
performance results at DAU.
Table 8
Motivational Assets
Motivational Assets Selected Citations
Attributions
Stakeholders explained the
organizational success by attributing it to
their hard work and efforts
Mayer, 2011; Perry and Hamm, 2017;
Pintrich, 2002; Weiner, 2005

Goals
Stakeholders continued to strive for the
implementation of new methodologies
and process improvements
Downes et al., 2017; Mayer, 2011; Lock and
Latham, 2006; Weiner, 2005;

Values
Stakeholders, both as individuals and as
teams recognized and live their values
Mayer, 2011; Weiner, 2005; Lock and Latham,
2006; Rueda, 2011

Attributions
During the document analysis and the interviews, two attributions were encountered as
motivational influencers in DAU’s journey of excellence in aligning learning to business
performance results: teamwork and the drive to innovate.
Teamwork. With teamwork, notions of a deep commitment to the team, helping each
other, and working relentlessly to achieve the specific task assigned were encountered. Though
not explicitly discussed, the deep commitment to the team could be rooted in military service and
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 79

working in the US Dept. of Defense for many years. All the interview employees either served in
the military or worked for the US Dep. of Defense for eight years or more.
Dr. Hardy explained that a strong team—especially during difficult times of changes in
DAU leadership and government budget cuts—could be credited for the subsequent success of
DAU. Specifically, he said: “Besides changing in leadership, we had a powerful leadership team
of deans and directors that Frank Anderson had hired and groomed, and we were a tight team.
And that served us well for those times without a permanent president with any tenure.”
Dean Wills also attributed a large part of the success DSMC to the quality of the teams, stating,
“The end part of it is, yes, we do team with teammates that have very key skill sets in areas that
we do bring them in, and they have opportunities to team with us, whether to do mission
assistance or, in some cases, to do blocks of instruction.” Assembling qualified teams and
teaming with them to deliver learning experiences to the learner at the point of need has proven a
pivotal attribution to the DAU's ability to successfully align learning to business performance
results.
Drive to innovate. During the past 10 years, DAU teams have been driven to innovate to
ensure they keep up with the changing needs of the acquisition learner and by extent the
warfighter; to ensure they are well prepared in case of possible future budget cuts; and to keep up
with technology to keep reinventing themselves in alignment with the ever-changing needs of the
learner/customer. This drive to innovate within the learning and development space was
encountered in numerous documents discussing DAU strategy and goals, as well as in four
interviews. In his message included in the 2016-2019 DAU Strategic Plan, DAU President John
Wolsey said: “DAU’s resources, people and the innate desire to improve puts us in a great
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 80

position to use technologies in innovative and powerful ways.” Here, the DAU President
discussed the innate drive of DAU faculty and staff to improve and innovate.
Additionally, regarding the changing learning needs of the acquisition learner, the Dr.
Stonerock said: “Innovation in relation to changing paradigms of how training has evolved into
learning; that is, the idea that training in and of itself is less effective than well-designed learning
which is meaningful, lasting, and relevant to the job role of the target audience.” Additionally,
Pat Wills said: “Innovation in relation to changing paradigms of how training has evolved into
learning; that is, the idea that training in and of itself is less effective than well-designed learning
which is meaningful, lasting, and relevant to the job role of the target audience.” These quotes
supported the assertion that drive to innovate is a pivotal attribution to DAU's journey of
excellence.
The drive to innovate was also cultivated during the government sequestration, which
resulted in DAU letting go of over 150 people. Since this time, DAU has focused on innovative
ways to organize DAU in order to still be agile under future budget cuts and able to fulfill their
mission successfully. Specifically, Dr. Hardy explained:
So, we went through that, a shutdown, and then at the same time they downsized the
department, and we lost 150 people, and that cost ... We did it off sites and how are we
going to still do our mission? I was one of the few that came forward with a restructuring
plan. Anyway, I took the whole heart and soul of our innovation and our curriculum
development, and I broke it up, and I kept our centers together under their people.
In terms of reinventing themselves as a team, Brenda Sedlacek stated, “So I think we are
looking at technology, we are open to reinventing ourselves. So right now, we are saying hey, let
us get focused on the customer and get close to our customer to find out what is it they need and
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 81

how can we best provide it to them?” DAU observes the customer and asks questions to explore
new and innovative solutions and to address the learner needs. Additionally, as Ms. Sedlacek
described, it is evident that the drive to innovate is a key attribution in DA's ability to
successfully align learning to business performance results: “And one of the things that I love
about my job, and what I see us doing at DAU, is that we are not afraid of questioning ourselves,
reinventing ourselves… So, I think we're gonna continue to evolve, and it's an exciting place to
be.”
Goals
At DAU, goals play a significant role in aligning learning to business performance
results. Earlier in this chapter, goals were reviewed under the strategic planning section from a
strategic planning and tactical perspective. In this section, the researcher will review goals as a
motivation influence on the ability of the stakeholders are DAU to successfully align learning to
business performance results.
In the DAU context, goals act as determinants of both intrinsic and extrinsic motivation.
As extrinsic motivation determinants, goals at DAU are articulated in order to drive the
implementation of specific tasks undertaken both by faculty and staff members as individual
contributors and contributors to a broader team. There are five types of goals, from a singular,
individual goal to a broader national security goal. First are leadership goals, which pertain to the
national security of the United States and emanate from the Department of Defense and cascade
into acquisition workforce goals which cascade into DAU goals, then into directorate and college
goals and finally to individual DAU employee goals.
For example, DAU Goal 2 –Workflow Learning from the DAU 2016-2019 Strategic Plan
is presented as follows: “Provide customer-focused learning assets and job support tools, at the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 82

point of need to help the Defense Acquisition Workforce succeed.” The goal is articulated with
action verbs, in simple, easy-to-understand language and a succinct format to facilitate
implementation. It is also tied to the overarching leadership goal through the word “mission.”
The mission of DAU is to “provide a global learning environment to develop qualified
acquisition, requirements, and contingency professionals who deliver and sustain effective and
affordable warfighting capabilities.” This goal is implemented by the Workflow Learning
Directorate and the Workflow Learning broader team, who are tasked with making learning tools
and job aids available in various modalities in order to meet the ever-changing needs of the
acquisition workforce.
As intrinsic motivation determinants, individual development goals are succinctly stated
with specific references to the next level up in the group of goals within the organization. For
example, for a Director of a team within a DAU directorate, his or her individual development
goals drive the accomplishment of specific tasks that cascade upwards to Goal 2 above. For
example, one such individual development goal would be to “attend ABC SharePoint Conference
in May 2017 and bring back best practices to apply in Workflow Learning Directorate efforts on
building out SharePoint 2018.” Next, the goal lists the amount of funding needed, the amount of
actual time necessary to accomplish the goal, the time frame in which the goal is expected to be
completed, and the expected outcome.
Values
Values can play a significant role as motivation influencers. DAU values are
discussed frequently and are listed in several significant documents including the DAU
2015 Annual Report, the DAU Strategic Plan 2016-2019, the DAU intranet, and 2017
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 83

strategy presentations. In reviewing the values from 2008, 2016, and 2017, the researcher
detected a continuum, as well as a refreshing, of these values.
Specifically, the DAU 2016-2019 Strategic Plan lists and describes the following
values: alignment and teamwork, customer focus, performance excellence, speed, and
agility. The values are described as actions in simple, easy-to-understand, and easy-to-
remember language. For example, alignment and teamwork are described as follows: “we
are a team aligned with our senior leadership. Through support, respect, and trust, we
work together to meet our leaders’ objectives.” As the Strategic Plan says, “These values
define how we approach our work and how we interact with our internal and external
partners, customers, partners, faculty, and staff. These values are not bumper stickers, but
how we behave every day.”
In late-mid-2017, DAU announced its strategic focus on customer intimacy and a
refreshed set of six values:
Excellence: we commit to the highest standards of competence and
professionalism, to be the provider of choice for Acquisition based knowledge,
lifelong learning, and professional growth; trust: we inspire confidence in
teammates and customers to perform at their best by demanding integrity and
accountability; teamwork: we foster collaborative and cooperative efforts to
achieve common goals; service: we listen to our customers to understand and
directly fulfill their needs in improving acquisition outcomes; ever forward: we
commit to evolve and develop ourselves, our organization and those around us,
using every challenge as an opportunity to innovate, adapt and improve; and
people matter: we value all people, providing opportunities to serve and
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 84

contribute to a balanced and respectful environment that supports the Acquisition
community.
In comparing the values from 2016 to 2017, it is evident that they are similar;
however, in 2017 they are articulated more crisply and centered on human interaction as
the foundation for excellence, results, contribution, and service in the acquisition learning
community.
Expressions of these values were revealed in the discussions during four of the six
interviews conducted, whereby words such as: “there is no “there,” we are always
striving for excellence,” as well as “we're always going to be moving ever forward” and
“I don't like being comfortable with status quo. I love being on the cutting-edge of what,
not only what we could push the envelope in learning and development, but also what are
they doing in other organizations.“
Through the document analysis, the researcher confirmed that the value system
was not always as robust and centered on excellence. The current value system appears to
be the result of a broad organizational change implemented at DAU during the presidency
of Frank Anderson, whose leadership from 2000-2008 laid the foundation of today’s
robust value system at DAU. As he describes in one of the videos reviewed in this
analysis:
Every leader is a key player in shaping and building the workplace culture in their
organization, so we all have a part of the department's cultural real estate, and the
key question is what am I as a leader doing to shape and create a world
class environment in the piece of the DoD real estate that I am responsible for, so
we've got to get out of the blame game we've got to show up with a positive
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 85

attitude and I keep coming back to that I believe that it's just so critical that the
leader shows up every day with a positive attitude and they are accepting
responsibility for shaping the future of the organization.
It is evident from this quote that the transformation began with specific requests
on empowerment and accountability of every leader at DAU to take responsibility and
help develop his or her team members for the future. As Anderson et al. (2008) discussed
during the Frank Anderson presidency and thereafter, by attaining industry awards and
recognition, DAU could shape its perception in both the minds of its customers, its
internal and external partners, and even its own faculty and staff. Especially the faculty
and staff exhibited a sense of pride about their highly recognized workplace and as a
result were more open to change and transformation because they want to be part of a
winning team. At the writing of this study, it is observed that the verbiage of the 2017
values does not even include the word ‘leadership,’ yet the notion of leadership is imbued
in the tone of the words used to describe the values such as “trust accountability,”
“teamwork,” and “we commit to develop ourselves, our organizations, and those around
us.”
During the interview, Dr. Hardy commented on values:
So, DAU has come up with some values out of this re-looking at ourselves, and
it’s sort of our internal values rather than external values. Customer intimacy is
going to be our main thing, and acquisition outcomes and alignment, but
internally, how do we want to work together, excellence, trust, teamwork, service,
ever forward and people matter.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 86

As such, the values transformation journey DAU intended seems to prove successful
through the virtuous circle created by the emergent knowledge influences examined
earlier: accreditation and industry awards applications.
Organizational Influence Assets
In this section, the researcher discusses the three organizational assets
encountered during the analysis of documents and interview content. As seen in Table 9,
these assets have had an impact on the ability of DAU to successfully align learning to
business performance results. The assets include organizational culture, alignment, and
resource allocation.

Table 9
Organizational Assets
Organizational Assets Selected Citations
Culture
The organization exhibited a healthy culture by
implementing a comprehensive system of
accountability and reporting
Clark and Estes, 2008; Northouse,
2016: Pintrich, 2002; Schein, 2010;
Weiner, 2005

Alignment
The organization developed, implemented, and
calibrated a rigorous strategic planning method
Ben-Hur et al., 2015; Clark and Estes,
2008; Mayer, 2011; Northouse, 2016;
Schein, 2010
Resource Allocation
The organization continuously optimized resources
around strategic planning, alignment, and reporting
to adapt to new learner needs, technology changes
and industry evolution
Ben-Hur et al., 2015; Clark and Estes,
2008; Northouse, 2016; Schein, 2010

Culture
Unsurprisingly, culture is a critical organizational influencer; at DAU, culture is
rooted in the core organizational mission, openness, and communication centering around
the Acquisition Learning Model. With the ALM in the background, DAU fostered a
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 87

cultural change by focusing on its mission, encouraging openness, and development to
influence the culture of the overall organization. Dr. Hardy explained:
So, we think that our acquisition learning model is perfect for that, not just the
foundational things they need and the workflow assets they needed on the job
when they needed to do their job, and the collaborative wisdom of the crowd, that
has some impact on the culture. Also, training and learning assets alone aren't
going to change your culture. Leadership has to be involved, and I think one of
the things that will help us change the culture is how do you influence the
leadership.
Core organizational mission. Over 80% of the documents reviewed during this analysis
and all six interview participants evidenced a strong commitment to the core mission of DAU
which is to “provide a global learning environment to develop qualified acquisition,
requirements and contingency professionals who deliver and sustain effective and affordable
warfighting capabilities.” This focus on the core organizational mission at DAU has shaped the
culture of the organization into a one of high urgency to deliver valuable learning to support the
warfighter. As such, DAU made learning mission critical. In his message included in the DAU
2016-2019 Strategic Plan, the DAU President, Mr. Wolsey said: “DAU will be a resource you
turn to every day and in all phases of your career, bringing you the help you want, and need. This
strategic plan and its successors will guide us, and help DAU continue to be the learning
institution that the workforce, the warfighter, and the nation deserve.”
In conjunction with this, Dr. Hardy said, “In this business, we cannot afford to be second
place. As our workforce is successful, so are the men and women of our Armed Forces. Their
success in training and on the job ultimately translates to the safety of our nation and the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 88

achievement of our national interests.” During the interview, Pat Wills described the focus on the
mission by the DAU faculty at DSMC as such: “Because [of] what we're focused on, better
acquisition outcomes, that will enable the young men and women that defend our freedom to be
properly equipped and armed with what they need. Our faculty is extremely focused on their
calling and why they're here. I think that's a unique thing.”
Openness. During the interviews, all six interview employees discussed how
openness to new ideas fostered new ways of organization and engaged new people into
their teams and organizations. Also, documents such as the Minutes of the biannual
Meetings of the Board of Visitors from 2012-2017 also evidenced openness in discussing
challenges facing DAU and seeking advice and input from the Board of Visitors. Such
openness facilitated exploration of new ideas for implementation, enhanced
organizational improvements, and fostered collaboration among DAU faculty and staff to
always focus forward. Pat Wills said:
We not only looked at all the processes, but we also looked at the people. How are
we organized for success? Did we have the capabilities within DSMC that we
needed to move forward in a thoughtful way to get that? How were each of our
centers? Were they aligned and what did they think they were doing?
Openness implies the willingness to practice inquiry and ask many, often difficult
questions, to drive improvement and cultural change.
The Board of Visitors presently consists of 12 learning and development thought leaders
from the private sector, academia, and the defense industry. These experts provide advice to the
Secretary of Defense on learning and development including curricula, methods of instruction,
facilities, and other topics of interest to the DAU. The Minutes of Meeting from each bi-annual
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 89

meeting that took place from 2012-2017 were reviewed in this analysis, and the researcher
determined that openness was an essential attribute evidenced in these discussions.
For example, in a January 2012 meeting, the President of DAU at the time sought the
advice and input from the Board of Visitors regarding the substantiation of proficiency for the
acquisition learner. During the May 2014 Board of Visitors meeting, the President of DAU
discussed the challenges facing DAU and generational learning, specifically around the
relevance, currency, cost, content, and utility of learning offered. During the 2016 meeting, the
DAU President discussed the challenges of leadership, budget, and legislation and sought advice
from the Board of Visitors on how to address these in terms of their impact on DAU’s mission
and learning. These three examples evidence the openness of critical issues discussed openly
during the Board of Visitors meetings. An additional layer of openness is supported by the fact
that all the minutes from the Board of Visitor meetings from 2012-2017 are available on the
DAU website Board of Visitors page. Openness is an essential ingredient of successful
communication, which is analyzed in the next section.
Communication. Between 2008-2018, DAU fostered open communication,
which is fundamental to a healthy organizational culture. DAU supports communication
externally and internally across various departments, functions, and layers of leadership
in a transparent and easy-to-engage-with style. Communication seems to be centered on
the customer needs—listening to the customer and then discussing internally these
customer needs and translating them into new products and services. In this analysis, the
researcher focused on the internal communication as part of the culture at DAU.
Communication between the DAU faculty and staff and the learner, communication
amongst DAU directorates and departments, and communication.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 90

Several interview comments support the importance of communication as a
critical driver of culture at DAU. For example, during the interviews, David Gallop, who
is a member of the faculty at DAU, said: “I think that one of our best practices is we
actually talk to our internal and external partners, people that are relying on us to do the
right thing for the force. We don't go forward just doing what we think they need; we
actually talk to them to figure out what they really need.” Moreover, Pat Wills said: “The
customer does have a voice with us. Their comments mean a great deal to us, and we're
focused on making it a better place for them to reap the best results.” In addition, another
way DAU is fostering communication between DAU faculty, staff, subject matter
experts, and learners is through communities of practice. There are over 40 communities
of practice at DAU serving over 160,000 learners by providing a platform to engage,
learn, and collaborate. One way Brenda Sedlacek's team, within the Workflow
Directorate at DAU, influences communication with the communities of practice is by
collecting analytics to help communities of practice refine their approach and offerings to
the learners. Specifically, she said:
So now we're looking at how can we utilize these new metrics that we never had
before to give more information, and is what my goal is in interacting with for
example the community of practice owners, is to give them information that can
help them make their tool better and give them feedback from the customers
themselves. To say here's what the customers are saying, they really like this,
there's no activity on this, or they don't like this kind of information.
The culture at DAU encourages faculty and staff to reach out to the customer—the
learner—in order to listen to their needs and to gather data analytics. It is then possible to
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 91

translate those needs into learning experiences and make evidence-based decisions about
learning offerings that add value and align with the DAU mission.
Alignment
A critical organizational influence at DAU is alignment across all levels of the
organization to the Department of Defense leadership goals. The implementation of the
ALM model discussed in the knowledge section of this chapter has facilitated the
alignment of connecting individual employee goals, to departmental learning goals to
overall organization goals to the DoD leadership goals. DAU has defined alignment as
“the understanding of the objectives of an organization or corporation and delivering the
training, services, and learning assets that support those objectives.” In 2008, Anderson et
al. discussed that alignment is about “keeping the main thing the main thing,” defining
the main thing as the understanding what goals senior leaders are trying to accomplish
and adopting the organizational structure and the internal processes to meet those goals.
Alignment at DAU implied organizational change at the cultural, process, and
even individual mindset level. At DAU, cultural change was implemented through
transparent communication conveying enthusiasm, composure, decisiveness, and respect.
DAU staff was embedded at the DoD leadership level to ensure that DoD leaders had a
reliable point of contact. Processes were changed to facilitate vertical alignment from the
middle upwards to senior leadership, as well as downwards to the individual employee
level. At the center of vertical alignment are middle managers and DAU
overcommunicated to middle managers to strengthen their commitment to alignment. In
addition, DAU embraced critics and involved them in the alignment process.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 92

In 2018, Dr. Hardy said: “The most important thing to me that a learning and
development organization can do is be in alignment with the senior leadership and their
business goals. If you don't do that the rest of it doesn't matter. You're only going to be a
reactive compliance training department.” Additionally, the DAU Human Capital Plan
2013-2017 discusses alignment as follows: “Strategic Alignment – DAU's human capital
management strategies are aligned with our mission, goals, and organizational objectives
and are integrated into our strategic plans, performance plans, and budgets.” Throughout
the analysis of over 50 documents, alignment was referenced at least once in each
document; thus, it is an anchoring influence of the culture at DAU.
Resource Allocation
A third organizational influence at DAU is the allocation of resources to facilitate the
alignment of learning to business performance results. Once again, the Acquisition Learning
Model, which the researcher discussed in detail in the Knowledge section of this chapter, also
acts as the process by which DAU prioritizes the organizational learning goals and facilitates
resource allocation in terms of people and technologies to the implement the DAU business
goals. In the following sections, the researcher will discuss people and systems infrastructure.
People. DAU is committed to allocating the right talent to the prioritized learning goals
which are aligned with the business goals. Working to serve the acquisition workforce and
ultimately the warfighter drives DAU's allocation of resources to the most critical areas of
acquisition learning. Successful allocation of people at DAU begins with recruiting, developing,
and retaining strong talent comprising of faculty and staff. Pat Wills expands on DAU’s hiring
practices for faculty: “We hire subjects matter experts and seasoned practitioners, and then if
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 93

they are already a teacher, that's even a plus. We'll give them the necessary tools, the learning,
our Fwd.’s, to round their skills out so they can be effective in the classrooms. That's important.”
New employees that join DAU are carefully selected for their expertise and their
commitment to serve the customer. The DAU has a detailed hiring process, which involves
interviewing and conducting meticulous reference checks to ensure that the candidates share the
motivation to serve and bring the required expertise to do so. As a result, DAU hires experienced
employees from the private sector, academia, and other Department of Defense agencies who are
committed to serving through their specialization and technical expertise. As David Gallop
explains: “The other resource that comes to mind is our human capital. One of the things that I
think has helped us in the recent years is our intermittent pull of the faculty.”
Additionally, DAU, using the ALM model, has a clear picture of the technical skills
needed to address the prioritized business goals. Based on this skills needs assessment, DAU has
established a technical coaching program to ensure that people are continuously engaged and
coached and allocated to tackle the acquisition business goals. Dr. Hardy explained:
DAU has a world-class executive coaching program that we have people that have held
senior positions in their careers are now coaches, and we have a certified coaching
program. And it's not a personality coaching. It's more performance-based technical
coaching. And we're very effective coaching senior acquisition leaders, and it's almost a
continuous thing.
Systems and infrastructure. At DAU, systems and infrastructure are an indispensable
aspect of resource allocation as the third organizational influence. As opposed to other
government organizations, DAU has harnessed the value the right systems and technology
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 94

infrastructure can bring to a learning organization by always exploring new technologies. As
Brenda Sedlacek explained:
So sometimes, I think especially in government and DOD, there's a lot of folks who are
really behind. But I feel like we're forward-leaning at DAU when it comes to technology.
So, we are able to, like I said, we have a whole group that's all about innovation and
seeing what's in the future, how can we take advantage of it?
DAU explore new technologies and selecting those that center on enhancing the learner
experience. For example, it has an innovation center which deploys the “try before you buy”
approach. As Brenda Sedlacek explained:
So, I think one thing that we're doing well at DAU is that we have an innovation center
that has their focus on looking at what are the new technologies that are coming out
there? “Try before you buy” I think is really cool as well is we have a try before you buy
program where if there is a new technology, we can bring it in and we have tested it on
standalone laptops to assess functionality.
As a result, DAU is able to adopt and adapt or develop technologies in-house to address
learner needs and facilitate learning inside and outside the classroom.
Findings Summary
The researcher analyzed data collected from the documents and artifacts, as well as six
semi-structured interviews, and generated several findings across the knowledge, motivation, and
organizational areas of influence. The findings point to how the Defense Acquisition University
successfully aligned learning to business performance results by analyzing the confluence of the
knowledge, motivation, and organizational influences at DAU.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 95

Specifically, under knowledge, three assumed influences were encountered in the data,
and three new influences emerged. The three assumed influences included: (a) procedural
knowledge pertaining to the strategic planning process, (b) procedural knowledge pertaining to
data collection and analysis, and (c) metacognitive knowledge pertaining to self and team
reflection. Moreover, three additional knowledge influence findings emerged, pertaining to
procedural knowledge, which included: (a) benchmarking, (b) accreditation, and (c) industry
award applications. Next, under motivation, the researcher encountered three assumed influences
in the data: (a) attributions, which include teamwork and the drive to innovate; (b) goals; and (c)
values. Finally, under organizational influences the three assumed influences were encountered,
which include: (a) culture, defined by the core organizational mission, openness, and
communication; (b) alignment; and (c) resource allocation, defined by people, systems, and
infrastructure.
These knowledge, motivation, and organizational influence findings highlight the
Defense Acquisition University’s journey of learning and subsequent excellence in aligning
learning to business performance results. In Chapter Five, the researcher provides solutions, best
practices, and recommendations on aligning learning to business performance results, based on
data and literature, which may be adopted and adapted by organizations for implementation in
their context.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 96

CHAPTER FIVE
Recommendations for Practice to Address KMO Influences
In this promising practice study, the researcher analyzed the knowledge, motivation, and
organizational influences that impacted the journey of excellence in successfully aligning
learning to business performance results at the Defense Acquisition University. In Chapters One,
Two, and Three, the researcher presented the problem of practice within the learning and
development industry, whereby organizations are not successful in aligning their learning
programs to their organization's business performance results. In Chapter Two, the researcher
presented the related literature. In Chapter Three, the researcher discussed the assumed KMO
influences and research approach. In Chapter Four, the researcher introduced the collection and
analysis of qualitative data from document reviews and six semi-structured interviews with
stakeholders from DAU. Three knowledge influences were confirmed by the data analysis:
strategic planning process, data collection and analysis, and self- and team-reflection. Moreover,
three additional knowledge influences emerged during the data analysis: benchmarking,
accreditation, and industry awards applications. The motivational influences analyzed in Chapter
Four included attributions, goals, and values. Finally, the organizational influences analyzed
included culture, alignment, and resource allocation.
In Chapter Five, the researcher explores how best practices from DAU can be gleaned
and adopted and adapted by other learning and development organizations, as well as corporate
universities to help them align their learning programs to their organizations’ business
performance results. A comprehensive blueprint of best practices is presented, based on the New
World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016). It is anticipated that application of
the recommendations included in Chapter Five will increase the ability of Chief Learning
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 97

Officers and their learning and development teams to successfully align learning to business
performance results in their organizations, demonstrate value to their CEOs and executive teams,
and improve the learning experiences of their organizations’ learners.
RQ1: What were the knowledge, motivational influences that enabled DAU to become a
best-practice organization in the alignment of learning to business performance goals?
RQ2: How did organizational influences impact the alignment of learning to business
performance goals?
RQ3: What solutions and recommendations in the areas of knowledge, motivation, and
organizational resources may be appropriate for aligning learning to business performance goals
at another organization?
Knowledge, Motivation, and Organizational Influence Recommendations
In the following sections, the researcher discusses the knowledge, motivation, and
organizational influence recommendations, and presents an implementation, evaluation, and
reporting program as a blueprint of best practices. The researcher will introduce the convergence
of the Clark and Estes (2008) KMO Model with the New World Kirkpatrick Model with an
individually designed program delineating the required results, behaviors, drivers, learning goals
and engagement criteria respectively for Level 4, Results; Level 3: Behavior; Level 2: Learning,
and Level 1: Behavior of the New World Kirkpatrick Model. This program is the blueprint which
chief learning officers and learning and development teams can use to adopt and adapt in their
contexts to successfully align learning to business performance results.
Knowledge Recommendations
The data for the knowledge influences were collected from documents and semi-
structured interviews and analyzed. Three assumed knowledge influences were confirmed, and
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 98

three additional knowledge influences emerged during the data analysis. As seen in Table 10
below, the assumed knowledge influences are: (a) strategic planning process (procedural
knowledge), (b) data collection and analysis (procedural knowledge), and (c) self and team
reflection (metacognitive knowledge.) In addition to the three assumed influences, three new
influences emerged: (d) benchmarking, (e) accreditation, and (f) industry awards. The three
emergent influences fall within procedural theory. The framework used in this analysis is based
on the work of Krathwohl (2002) and Pintrich (2002).
Table 10
Summary of Knowledge Influences and Recommendations
Assumed Knowledge Influence Principle and Citation Context-Specific Recommendation
1.Strategic Planning Process
(Procedural)

Stakeholders adapted and expanded specific
strategic planning and implementation
practices

To develop mastery,
individuals must acquire
component skills,
practice integrating
them, and know when to
apply what they have
learned (Schraw and
McCrudden, 2006).

1a. It is recommended that Learning
and Development Organizations
receive training on how to develop a
strategy, how to conduct an
environmental scan, and how to
conduct a SWOT, followed by an
opportunity to practice and then to
receive feedback on these skills.

1b. Additionally, it is recommended
that job aids are developed to assist
them in defining and integrating their
learning strategy into their
organizational strategic planning
process. These job aids may include:
A SWOT template, a strategy model
template, a communications strategy,
and a metrics dashboard to measure
and evaluate progress.
2. Data Collection and Analysis (Procedural)

Stakeholders calibrated the processes around
data collection and analysis to respond to new
learner needs and technology changes
How individuals
organize knowledge
influences how they
learn and apply what
they know (Schraw and
McCrudden, 2006)

Modeling to-be-learned
strategies or behaviors
improves self-efficacy,
learning, and
performance (Denler,
Wolters, & Benzon,
2012)
2a. It is recommended that Learning
and Development organizations
receive training on how to build
robust data collection and analysis
processes.
2b. In addition, they can receive
training on how to conduct
benchmarking visits with other
organizations that have developed
such rigorous processes.

Following this training, Learning and
Development organizations will
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 99

require practice sessions followed by
feedback on their skills.
3.Self and Team Reflection (Metacognitive)
Stakeholders conducted ongoing self-
assessment and self-reflection
The use of
metacognitive strategies
facilitates learning
(Baker, 2006).
3a.It is recommended that Learning
and Development organizations
provide their employees with training
and coaching sessions on self-
reflection and team-reflection tactics

3b. These training and coaching
sessions can be supplemented with
job aids such as apps available on the
market to remind and guide
employees to practice self-reflection.
4.Benchmarking
(Procedural)

This is an emergent influence not previously
identified in the conceptual framework

Stakeholders structured and engaged in
benchmarking including observation,
modeling behaviors and practices from other
best-in-class organizations
Effective observational
learning is achieved by
first organizing and
rehearsing modeled
behaviors, then enacting
them overtly (Mayer,
2011).
4a. Learning organizations
participate in benchmarking process
training to learn how to identify the
areas and the process they excel at
and the areas they would like to
develop and engage in benchmarking
visits with best in class organizations
within and outside their industry to
learn how to exchange best practices
and how to achieve breakthrough
performance.

4b.The training will need to be
followed by practicing a
benchmarking visit and with the
provision of feedback.
5.Accreditation (Procedural)

This is an emergent influence not previously
identified in the conceptual framework

Stakeholders underwent rigorous processes
relating to accreditation with three different
accrediting bodies which provided feedback
and guidance to enhance performance
Feedback that is private,
specific, and timely
improves performance
(Shute, 2008).
5a. Accreditation is useful for
corporate universities as it involves a
rigorous evaluation process by the
accreditation body and grants
gravitas to the functions of a
corporate university. It is
recommended that learning
organizations receive training and
models on how to apply for
accreditation, followed by an
opportunity to practice with mock
consulting teams and receiving
feedback prior to their accreditation
application.
6. Industry Awards (Procedural)

This is an emergent influence not previously
identified in the conceptual framework

Stakeholders participated in industry award
competitions by submitting detailed industry
award applications which described best
practices that can be modeled and learned by
other organizations in learning and
development.
Modeling to-be-learned
strategies or behaviors
improves self-efficacy,
learning, and
performance (Denler,
Wolters, & Benzon,
2012).
6a. It is recommended that Learning
and Development to receive training
and models of industry award
applications, to create teams who will
study such models and then submit
actual industry award applications.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 100

Procedural knowledge solutions, or description of needs or assets. Strategic planning
process (P). Learning and development organizations need to know how to implement a strategic
planning process which includes a learning strategy. In order to develop mastery, an individual
must acquire component skills, practice integrating them, and know when to apply what they
have learned (Ben-Hur et al., 2015). This is particularly applicable for learning and development
organizations whereby they need to master the components and the application of an
organization-wide strategic planning process to be able to align learning to business performance
results (Ben-Hur et al., 2015). As Ben-Hur et al., Lykins (2012), and Sadler-Smith (2009)
asserted, incorporating the learning strategy in the overall strategy of an organization is good for
business because learning that ties to business results can help organizations improve business
performance. It is therefore recommended that Learning and Development Organizations define
and integrate their learning strategy into their organizational strategic planning process. This can
be achieved by conducting an environmental scan, implementing a SWOT analysis; building a
learning strategy model that reflects their organizational structure; engaging the organization's
leaders and listening to their goals; implementing communications campaigns to educate all
employees; and defining metrics for success to evaluate their progress.
Data collection and analysis (P). To successfully implement a strategic planning
process, learning and development organizations must know what data to collect and how to
collect it (P). The way individuals organize knowledge impacts how they learn and apply what
they know (Schraw & McCrudden, 2006). In the learning and development theoretical context
this influence is critical as successfully demonstrated by the Defense Acquisition University
because most learning and development organizations tend to gather and measure to wrong
metrics (Ben-Hur et al., 2015; Stuckey, 2012; Williams, 2002).
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 101

According to Elbanna, Andrews, and Pollanen (2016), strategic planning has positively
influenced implementation and business performance. In their empirical study, Elbanna et al.
surveyed 150 public service organizations in Canada. The results of their findings showed that
formal strategic planning is important in driving strategy implementation and addressing
stakeholder uncertainty. In the context of learning and development, it is therefore recommended
that Learning and Development organizations build robust data collection and analysis processes
by benchmarking with other organizations that have developed such rigorous processes to
strengthen their ability to align learning to business performance results.
Self and team reflection. Learning and development employees and their teams need to
know how to conduct self and team reflection (M). Pintrich (2002) discussed the importance of
self-reflection in improving performance. In the context of the Defense Acquisition University,
self-reflection is practiced regularly both informally, prior to one-on-one discussions with one’s
manager, and formally during the development of the employee individual development plan.
Additionally, team reflection is practiced often. Some teams within DAU practice team reflection
weekly after the completion of a learning program to debrief on what worked well and what did
not. Therefore, from a theoretical angle, learning and development organizations would benefit
from offering training programs to encourage employee self and team reflection.
Moreover, Cho and Jung (2014) conducted an empirical investigation to examine the
relationship between metacognition and firm performance. In their analysis, they analyzed
survey data from 190 business owners and founders working in the United States. The results of
the study confirmed that the business owner’s metacognition from practicing self and team
reflection had a positive impact on firm performance. It is therefore recommended that Learning
and Development organizations provide their employees with training and coaching sessions on
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 102

how to practice self- reflection and team-reflection tactics. These training and coaching sessions
can be supplemented with job aids such as apps available on the market that can be customized
to the organization and used by employees and their teams to remind and guide employees to
practice self-reflection and team reflection.
Benchmarking (P). Learning and development organizations must understand how to
engage in benchmarking activities, including observation, data gathering, analyzing, adopting,
adapting, and modeling behaviors and practices from other best-in-class organizations to
improve their organizations (P). According to Mayer (2011), effective observational learning is
achieved by first organizing and rehearsing modeled behaviors, then enacting them overtly. From
a theoretical angle, learning and development organizations would benefit from using
benchmarking to improve their practices and behaviors.
Additionally, Yigitcanlar (2014) asserted that benchmarking is an essential prerequisite
for strategic vision and successful organizational performance. In this empirical analysis,
Yigitcanlar conducted an assessment model anchored in benchmarking that compared eleven
emerging knowledge cities including Birmingham, Boston, Brisbane, Helsinki, Istanbul,
Manchester, Melbourne, San Francisco, Sydney, Toronto, and Vancouver. The results of the
analysis confirmed the value of benchmarking in improving organizational performance at scale.
Thus, it is recommended that learning organizations should identify the areas they excel at and
the areas they would like to develop and engage in benchmarking with a best-in-class
organization outside their industry to achieve breakthrough performance.
Accreditation (P). Accreditation is useful for corporate universities and higher
educational institutions as it involves a rigorous evaluation process by the accreditation body and
grants gravitas to the functions of a corporate university (P). The accrediting body evaluates the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 103

educational institution and provides detailed private feedback across all functions, policies, and
procedures. As Shute (2008) asserted, feedback that is private, specific, and timely enhances
performance. Moreover, Alagaraja and Li (2015) contended that corporate universities can
strengthen their operations, exceed corporate limits, and provide university-level learning and
development services upon receiving accreditation. From a theoretical angle, learning and
development organizations which operate corporate universities can benefit from seeking
accreditation.
Additionally, Capiello and Pedrini (2017) conducted an empirical analysis to examine the
drivers of corporate university performance. In this study, the authors surveyed 20 corporate
universities in Italy, and the results confirmed the importance of measuring corporate university
performance through numerous approaches including seeking accreditation. Thus, it is
recommended that corporate universities and learning and development organizations focus on
accreditation as it involves a rigorous evaluation process by the accreditation body and grants
gravitas to the functions of the organization once accredited.
Industry award applications. In addition to accreditation, learning and development
organizations can improve their processes to align learning to business performance results by
applying to industry award competitions (P). Industry awards serve as an external assessment
with similar organizations across the industry. Such applications must describe learning best
practices evidenced by credible data. Such applications foster modeling of strategies which in
turn improves performance (Denler, Wolters, & Benzon, 2012). Additionally, Asif and Gouthier
(2014) and Lee and Ooi (2015) asserted that industry awards confirm the high commitment
which winning organizations dedicate in order to achieve quality excellence and strengthen
organizational memory and process innovation.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 104

Moreover, Wilson and Collier (2000) conducted an empirical investigation of the impact
of the Malcolm Baldrige National Quality Award to organizational performance. These
investigators surveyed 226 manufacturing firms. The results confirmed that “leadership drives
the system that causes the results” and that critical categories of the Baldrige award had a
positive impact on predicting customer satisfaction, financial results, and organizational
performance. It is therefore recommended that learning and development organizations that seek
to improve their efforts in aligning learning to business performance results to create teams who
study the processes and apply to such industry awards to improve their internal processes and
procedures
Motivation Recommendations
The data for the motivation influences were collected from documents and semi-
structured interviews and analyzed to glean the motivation recommendations. All three assumed
motivation influences were validated. As seen in Table 11 below, the assumed motivation
influences are: (a) attributions, (b) values, and (c) goals. The three emergent influences fall
within crucial motivation theory constructs. Specifically, the framework used in this analysis is
based on the research of Clark and Estes (2008), Locke and Latham (2006), Pintrich (2002), and
Anderman and Young (1994). As seen in column 3 of Table 8, these influences were prioritized
by three priority levels—high (H), medium (M), and low (L)—concerning their impact on
aligning learning to business performance results.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 105

Table 11
Summary of Motivation Influences and Recommendations

Assumed Motivation Influence

Principle and Citation
Context-Specific
Recommendation
7. Attributions

Stakeholders explained the
organizational success by attributing
it to their hard work and efforts
Adaptive attributions and control
beliefs motivate [individuals]
(Pintrich, 2002).
7a. It is recommended that
Learning and Development
organizations provide training
programs which foster a culture of
excellence by stating what
excellence means for the
organization regarding goals,
values, and actions;

7b. Also, it is recommended the
L&D organizations design
communications campaigns to
promote learning programs which
motivate, reward, and recognize
their employees by creating
rewards programs and recognition
programs that recognize and value
the actions and results which
employees achieve by practicing a
culture of excellence at their
organization.
7.1 Teamwork

This is an emergent attribution type
not previously identified explicitly
in the conceptual framework

Stakeholders formed functional
teams which fostered a positive
emotional environment

Positive emotional environments
support motivation (Clark & Estes,
2008).
7.1.a It is recommended that
Learning and development
organizations provide training to
and coaching to their employees
to work in small teams and
through brainstorming and
ideation to create clear definitions
and define ways to build to
exercise some choice and control
7.2 Drive to innovate

This is an emergent attribution type
not previously identified explicitly
in the conceptual framework

Stakeholders focused their efforts to
innovate by learning from successes
and failures
Learning and motivation are
enhanced when individuals attribute
success or failures to effort rather
than ability. (Anderman & Young,
1994).
7.2.a It is recommended that
Learning and Development
organizations provide
opportunities of choice and some
control to their employees both
individually and in team settings
to foster innovation and creativity
through training and engaging
with innovative teams in other
organizations.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 106

8. Goals

Stakeholders continued to strive for
the implementation of new
methodologies and process
improvements
Focusing on mastery, individual
improvement, learning, and
progress promotes positive
motivation (Locke & Latham, 2006;
Pintrich, 2002; Anderman &
Young, 1994).
8a. It is recommended that
learning and development
organizations create
organizational structures that
encourage personal responsibility
and provide safe environments
where employees can learn to
create and own SMART (specific,
measurable, achievable,
rewarding, and time-specific)
goals.
9. Values

Stakeholder values played a critical
role in driving for excellence and
motivating for continuous
improvement

Higher expectations for success and
perceptions of confidence can
positively influence learning and
motivation (Wigfield & Eccles,
2000).
9a. It is recommended the learning
and development organizations
conduct brainstorming and
training sessions to empower and
enable their teams to define their
organizational values (their
“why”) and to focus on the
importance of implementing those
values through tasks and activities
in their teams and their
organizations.

Attributions. Learning and development organizations need to know and understand
how individually and collectively they explain their improved performance. According to
Pintrich (2002), individuals are motivated by adaptive attributions and control beliefs.
Furthermore, Wiener (2012) asserted that providing feedback which highlights the process of
learning, including the importance of effort and strategies, strengthens attributions and the
confidence of employees in attributing their success to deliberate learning and hard work. In the
context of the Defense Acquisition University. It is recommended that learning and development
organizations provide training programs which foster a culture of excellence; design and help
employees learn how to build communications campaigns to promote learning programs which
to motivate, reward, and recognize their employees.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 107

Smith (2008) asserted that learning helps individuals grow and aspire towards higher
level needs, whereby employees feel that their human potential is enhanced both as individuals
and as employees of teams in their organizations. Additionally, Van Rooij and Merkebu (2015)
posited that learning organizations which focus first on their own employees' learning and
development through programs that inspire, motivate, and reward them, see positive results in
aligning the overall learning strategy to the organizational business goals. Therefore, from a
theoretical perspective, it appears that providing training opportunities, which is an
organizational influence, for employees may inspire and motivate them resulting in a positive
attribution impact to the overall goal of aligning learning to business performance results. This is
an example of the interplay between organizational influences and motivational influences.
Moskovsky, Alrabai, Paolini, and Ratcheva (2013) conducted a controlled investigation
to examine the effect of teacher motivational strategies on learners’ motivation. The
investigation included 14 teachers and 296 Saudi English language learners divided into an
experimental group and a control group. One of the eight constructs examined was positive
attributions to learning defined as “If I do well in my learning, it is because I try hard.” The study
results were overall positive and in particular for positive attributions to learning. Specifically,
the motivation of the experimental learner group declined less significantly than the motivation
of the control learner group over time on the attribute of positive attributions to learning. Thus,
the recommendation of this study that learning and development organizations provide training
programs to positively motivate their employees would be beneficial.
Values. Learning and development organizations’ values play a critical role in driving for
excellence and motivating for continuous improvement. According to Wigfield and Eccles
(2000), higher expectations for success and perceptions of confidence can positively influence
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 108

learning and motivation. Additionally, modeling through credibility and similarities can foster
and enhance positive values. In the context of the Defense Acquisition University, values were
posted in strategic plans and documents, listed on the university website, and most importantly
were modeled by leadership on a day-to-day basis.
Malachi and Leiter (1997) defined values as the coherence between the expectations of
an organization and the expectations of its employees. Specifically, when such coherence exists,
Maslach and Leiter and Kanter (1979) asserted that employees feel both encouraged that their
organizations are supportive and understanding of what drives them as individuals and
empowered to accomplish their tasks. Thus, from a theoretical perspective, it appears that
organizations that foster the precise definition of their values in conjunction and collaboration
with their employees will likely improve business performance.
Greco, Laschinger, and Wong (2006) conducted a study to examine the relationship
between leader empowerment behavior towards employees, perceptions of staff empowerment,
values, and areas of work and life using Kanter's (1979) theory of structural power in
organizations. These researchers surveyed a random sample of 322 staff nurses in intensive care
hospitals in Ontario. The results showed that the leader behaviors played an essential role in the
nurses' perception that their organization supported them, and their values aligned. The resultant
recommendation in this study that learning and development organizations must empower and
enable their teams to invest time and effort to define their organizational values and to focus on
the importance of implementing those values through tasks and activities in their teams and their
organizations would likely facilitate the alignment of learning to business performance results.
Goals. Learning and development teams need to know how to set meaningful and
measurable performance goals which drive proficiency. Focusing on mastery, individual
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 109

improvement, learning, and progress promotes positive motivation (Anderman & Young, 1994;
Locke & Latham, 2006; Pintrich, 2002). Clark and Estes (2008) and Dembo and Eaton (2000)
clarified that encouraging employees and teams to set specific and measurable performance goals
improves performance. These researchers recommended that learning and development
organizations : provide training to employees on how to create SMART (specific, measurable,
achievable, rewarding, and time-specific) goals; build organizational structures that encourage
personal responsibility and provide safe environments where employees can learn how to create
and own their goals; and offer job aids to allow their employees to continue setting, using, and
measuring their SMART goals.
Beausaert, Segers, and Gijselaers (2011) asserted that there are substantial positive
relationships between personal development goals and plans and organizational performance.
Specifically, Beausaert et al. contended that a supervisor that motivates and provides relevant
information and supporting feedback can positively impact the employee, allow him or her to
reflect and create personal development goals that are relevant to both individual and
organizational performance. From a theoretical perspective, it appears that an organization that
fosters the creation of appropriate personal development employee goals would improve the
alignment of learning to business performance results.
Personal development goals and their alignment to organizational goals has been the
central theme in research analysis conducted by van Rooij and Merkebu (2015). These scholars
conducted an empirical study of 15 learning and development leaders within professional
development firms with 8,000 employees or more. Van Rooij and Merkebu deconstructed
business goals and employee personal development goals by asking fundamental questions such
as “What observed behaviors indicate that an employee is applying new learning or new
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 110

knowledge?” and “What are the consequences of not addressing the performance gap identified
in this area?” Through their analysis, they found that well-articulated employee learning goals
can have a significant positive impact in reaching the business performance goals of professional
services firms. The resultant recommendation in this study that learning and development
organizations ought to provide safe and engaging environments to allow employees to develop
their individual learning goals would benefit the broader efforts of aligning learning to business
performance results.
Organizational Recommendations
The data for the organization influences were collected from documents and semi-
structured interviews and analyzed to glean the organization recommendations. Three assumed
organizational influences were validated, and three additional knowledge influences emerged
and were also validated. As seen in Table 12 below, the assumed motivation influences are: (a)
culture, (b) alignment, and (c) resource allocation. The framework used in this analysis is based
on the research of Clark and Estes (2008), and Schein (2010). As seen in column 3 of Table 12,
these influences were prioritized by three priority levels: high (H), medium (M), and low (L)
concerning their impact on aligning learning to business performance results.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 111

Table 12
Summary of Organization Influences and Recommendations
Assumed Organization Influence Principle and Citation Context-Specific Recommendation
10. Culture

The organization exhibited a
healthy culture by implementing a
comprehensive system of
accountability and reporting
through openness, communication,
and organizational mission clarity
Culture is defined by three levels:
visible and feelable structures and
processes; espoused beliefs and
values, and underlying assumptions
that determine behavior, perception,
thought, and feeling.
Schein (2010)

Leaders, supervisors, and culture are
the foundations of strategic
employee communications
Berger, B. (2014).
10a. It is recommended that
learning and development
organizations create foster a strong
culture that is anchored in
openness, goals, and
communication through training,
focus groups and communications
campaigns. This can serve as the
backbone to successfully align
learning to business performance
results.

11. Alignment
The organization focused and
implemented alignment of all goals
and resources to the organizational
priorities
Alignment of the visible aspects of
organizational culture (mission,
goals, and policies and procedures)
with the invisible aspects
(unconscious styles, attitudes and
behavior and most importantly
operational patterns at all levels) is
important in achieving performance
excellence.
(Clark & Estes, 2008)
11a. It is recommended that
learning and development
organizations foster alignment of
their agendas and their learning
strategies to the organizational
goals by
1) conducting training on how to
conduct a strategic planning
process engaging with 2) by
encouraging modeling of the
alignment through consistent,
periodic meetings with the
organization leaders, listening to
their vision and providing insight
on how to implement the vision
through the learning strategy.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 112

12. Resource allocation

The organization continuously
optimized resources
around strategic planning,
alignment, and reporting to adapt to
new learner needs, technology
changes and industry evolution
The design and availability of
material and staffing resources
enable organizations to achieve
goals. (Clark& Estes, 2008)

12.a It is recommended that
Learning and Development
organizations provide training to
foster the development of
appropriate management processes
which will enable employees to
best allocate and manage their
resources including people,
technology, and knowledge.

12.b Additionally, job aids, and
underlying technology systems can
be developed and for adoption to
facilitate training and
implementation in particular
around the resources of people,
systems, and infrastructure.

12.1.People

This is an emergent influence
specific to resource allocation not
previously identified explicitly in
the conceptual framework

Stakeholders focused significant
resources on recruiting, developing
and retaining the right talent
Organizational leaders embed and
transmit culture through their hiring
process of candidates with a similar
style, assumptions, values, and
beliefs (Schein, 2010).
12.1.a It is recommended that
Learning and Development
Organizations create training to
foster robust talent acquisition,
development, and retention
programs to ensure that the newly
hired employees are trained,
empowered, and encouraged to
successfully align learning to
business performance results.
12.2 Systems and Infrastructure

This is an emergent influence
specific to resource allocation not
previously identified explicitly in
the conceptual framework

Stakeholders ensured that systems
and infrastructure were allocated to
the highest priority learning goals
aligned with the organizational
priorities.
Leader beliefs about resource
allocation and budgets influence
their choice of goals, means to
accomplish them, and management
processes to be used (Schein, 2010)
12.2.a It is recommended that
learning and development
organizations develop and provide
training to their employees to
enable them to define their needs
for systems and infrastructure,
allocate the appropriate budgets,
and align the right people to
leverage the technology

Culture. The organization exhibited a healthy culture by implementing a comprehensive
system of accountability and reporting through openness, communication, and organizational
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 113

mission clarity. As Schein (2010) asserted, culture is defined by three levels: visible and feelable
structures and processes; espoused beliefs and values, and basic underlying assumptions that
determine behavior, perception, thought, and feeling. Additionally, Berger (2014) contended that
culture modeled by leaders and supervisors forms the foundation of strategic employee
communications. In the context of the Defense Acquisition University, a culture of openness and
focus on the organizational mission was fostered and cultivated over a decade primarily through
open communication and modeling by the organizational leadership. It is recommended that
learning and development organizations create and foster a strong culture that is anchored in
openness, goals, and communication through training, focus groups and communications
campaigns. Such activities can help set a foundational culture which can evolve and develop as
the organization grows and embraces change.
As Stuckey (2012) and Williams (2002) posited, most learning and development
organizations lack a culture of empowerment that allows their employees to seek and retain a
seat at the decision-making table of their organization. Furthermore, Ben-Hur et al. (2015)
asserted that such lack of empowerment prevents learning and development employees from
engaging with the chief learning officer and his or her team, resulting in learning and
development organizations acting as training order takers as opposed to organizational
performance advisors. From a theoretical perspective, therefore, it would appear that
strengthening the culture of learning and development organizations will have a positive impact
on their ability to engage with the organization leaders to align learning to business performance
results.
Ahmadi, Salamzadeh, and Akbari (2012) studied the relationship between organizational
culture and strategy implementation to examine possible correlations in typology and dimension,
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 114

and explain failures in strategy implementation. In this empirical study, the researchers surveyed
136 employees of Karafarin Bank in Iran. The study findings included positive correlations
between all the typologies and dimensions of the organizational culture and the components of
strategy implementation. Specifically, the authors concluded that the more flexible the culture,
including openness in policy formation, the more successful the implementation of the
organizational strategy. Thus, the recommendation provided in this study whereby learning and
development organizations provide training to their employees to help them build an
organizational culture that is strong and anchored on openness, strategic goals, and
communication, would enhance their ability to align learning to business performance results.
Alignment. Learning and development organizations need to focus on aligning their
mission, vision, goals, and resources to the organizational strategic priorities. As Clark and Estes
(2008) asserted, the visible aspects of organizational culture which include the mission, goals,
and policies and procedures of the organization must align with the invisible aspects of
organizational culture which include unconscious styles, attitudes and behavior, and most
importantly operational patterns at all levels. Such alignment fosters performance excellence. In
the context of the Defense Acquisition University, alignment serves as the foundation of their
performance excellence. As such, all learning strategy is aligned with the organizational strategy
through a series of annual and semi-annual activities, conducted by senior leaders and directors.
These activities include conducting an environmental scan of external changes, developing a
SWOT analysis, drafting a 3-year strategic plan, developing an annual performance plan, and
incorporating the respective goals and objectives in the individual personal performance plans of
the employees.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 115

It is recommended that learning and development organizations foster alignment of their
agendas and their learning strategies to the organizational goals by conducting training for their
employees on how to conduct strategic planning through specific baselining activities and
developing strategic plans, annual performance plans, and individual employee performance
plans. Additionally, Learning and Development organizations would benefit from modeling
alignment through consistent, periodic meetings with the organization leaders, to listen to their
vision and to provide insight on how to implement the vision through the learning strategy.
Additionally, job aids may be created to leverage related expertise and strengthen experience
around aligning learning to business performance results.
Luthans, Luthans, and Luthans (2015), Smith (2008), and Stuckey (2012) discussed the
importance of aligning learning and development organization goals to the broader business
goals of the organization. Most learning and development organization fail to plan and achieve
for such alignment which results in miscommunication including the measurement of the wrong
metrics to report learning impact to the organization. Ben-Hur et al. (2015) further pinpointed the
limitations of failing to align with the chief learning officer agenda. From a theoretical
perspective, it would appear that learning and development organizations can significantly
benefit themselves and their organizations if they align learning to the business performance
goals of such organizations.
Nalborczyk and Sandelands (2012) conducted a case study of Emerald Academy, the
corporate academy or university of Emerald Group Publishing. The investigators of this case
study described how the academy was able to align its strategy with the broader organization,
Emerald Group Publishing, using a structured action-learning process lead by the organization
chief learning officer, the academy dean, organizational directors, and academy learners. The
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 116

process is structured and takes place during six separate workshops over a 6-month period. This
structured process of aligning the learning goals of the academy to the business priorities of the
broader publishing group resulted in successful implementation of tens of revenue generating
projects per year and vertical or lateral job changes to over 44% of the academy learners. Thus,
the recommendation in this study to learning and development organizations focuses on aligning
their goals with the needs and business goals of their broader organization will likely deliver
similar positive outcomes.
Resource allocation. Learning and development organizations need to ensure that
systems and infrastructure are allocated to the highest priority learning goals that are aligned
with the organizational priorities. As such, learning and development organizations need to
continuously optimize resources to facilitate strategic planning, alignment and reporting to
enable the overall organization to adapt to new learner needs, technology changes and industry
evolution. As Clark and Estes (2008) asserted, the design and availability of material and staffing
resources enable organizations to achieve goals. Moreover, Schein (2010) discussed the pivotal
role leaders have in the allocation of resources through budgeting, whereby the leader's
assumptions and beliefs are revealed. These beliefs act as drivers for decision making on how
resources, including funding, are allocated. It is recommended that learning and development
organizations provide training to foster the development of appropriate management processes
which will enable employees to best allocate and manage their resources including people,
technology, and knowledge. Additionally, job aids and underlying technology systems can be
developed and for adoption to facilitate training and implementation in particular around the
resources of people, systems, and infrastructure.
People. Learning and development organizations need to allocate significant resources in
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 117

recruiting, developing and retaining the right talent. As Schein (2010) pointed out, organizational
leaders embed and transmit culture through their hiring process of candidates with a similar
style, assumptions, values, and beliefs. The researcher recommended that learning and
development organizations create training to foster robust talent acquisition, development, and
retention programs to ensure that the newly hired employees are trained, empowered, and
encouraged to successfully align learning to business performance results.
Sparrow and Makram (2015) contended that robust talent management architecture can
have a positive impact on organizational performance defined in four different constructs: value
creation, value capture, value leverage, and value protection. Additionally, Collings and Mellahi
(2009) described talent management in four constructs, one of which focuses on people as the
key drivers of talent management and ultimately organizational performance. In this construct,
organizations and their people experience two competitive pulls: the interpersonal competition
which takes place between different and the intrapersonal competition whereby the employee
strives to be their best. Interpersonal competition shifts the responsibility to the organization to
attract, hire, develop, nurture, and retain top talent. The intra-personal competition implies that
every employee can be supported to cultivate and improve their strengths through customized
learning and training programs. Both competitive pulls drive improved business performance
outcomes. Thus, from a theoretical perspective, it appears that organizations which invest in
hiring, developing, and retaining the right talent can achieve better performance results.
Glaister, Karacay, Demirbag, and Tatoglu (2018) conducted a study with organizations
that are members of the Union of Chambers of Commerce, Industry, Maritime Trade, and
Commodity Exchanges of Turkey. The survey respondents came from 198 organizations with
more than 50 employees. The study results that are relevant here include two points: first, an
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 118

organization that focuses on talent recruitment and selection, performance appraisal, and
workforce planning can build a robust talent management practice; in turn, this results in
improved organizational performance. Therefore, the recommendation in this study that learning
and development organizations foster robust talent acquisition, development, and retention
programs to ensure that the newly hired employees are trained, empowered, and encouraged will
likely be more successful in aligning learning to business performance results.
Systems and infrastructure. Learning and development organizations need to focus on
their leadership beliefs in conjunction with the appropriate allocation of resources to systems and
infrastructure to enhance performance and successful alignment of learning to business
performance results. According to Schein (2010), leader beliefs about resource allocation and
budgets influence their choice of goals, means to accomplish them, and the management
processes to be used. It is recommended that learning and development organizations develop
and provide training to their employees to enable them to define their needs for systems and
infrastructure, allocate the appropriate budgets and align the right people to leverage the
technology to successfully integrate the implementation and evaluation of their efforts to align
learning to business performance results.
Ben-Hur et al. (2015) and Stuckey (2012) posited that a significant challenge learning
and development organizations are facing in aligning learning to business performance results
stems from their employees' lack of business acumen and expertise in discerning business
challenges in the front lines of product and service production and procurement to customers.
Furthermore, Williams (2002) asserted that learning and development employees cannot “speak
the language” of business; as such, they cannot successfully allocate resources, including
budgets, and prioritize business goals. Ben-Hur et al. (2015) further postulated that the learning
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 119

and development organizations with employees that have the business acumen skills to align
with the CEO's agenda, can transform their organizations. From a theoretical perspective, it
would appear that increasing learning and development employees' understanding and focus on
resource allocation including the resources of people, systems, and infrastructure would benefit
learning and development organizations in their efforts to align learning to business performance
results.
From an empirical perspective, Taegoo, Lee, Paek, and Lee (2013) surveyed 14 top tier
five-star hotels in Seoul, Korea to discern the impact of enabling and sharing structural and
relational business knowledge among employees to performance. Using a sample of 486
employees, Taegoo et al. concluded that enabling employees' access to structural and relational
knowledge relating to the business operations, including resource allocation in the hotels resulted
in superior organizational performance. Thus, the recommendation from this study for learning
and development organizations to provide training that would enable their employees to acquire
and share business-related knowledge of their organizations business priorities and performance
will likely strengthen the employees’ ability to align learning to business performance results.

Best Practices for Learning & Development Teams
In this section, the researcher outlines three overarching best practices gleaned from
DAU’s excellence journey, which L&D teams can use to successfully align learning to business
performance results. These goals have been distilled from the 12 KMO assets, and respective
recommendations discussed in earlier sections and are based on the actions and success achieved
by the DAU. The following three goals include the recommended steps, resources, and learning
that Chief Learning Officers and their management teams will need to provide to their L&D
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 120

team to ensure they are successful in aligning learning to business performance results. The goals
are: (a) Best practice 1: Foster confidence, creativity, and commitment to your L&D team; (b)
Best Practice 2: Align metrics, strategy, and resources to the CEO agenda; and (c) Best Practice
3: Practice continuous improvement. While not every organization may be able to implement all
three of the recommended best practices outlined below due to the size of the team, and
availability of resources, it is recommended that the organization aims to implement at least best
practices 1 and 2.
Best Practice 1: Foster Confidence, Creativity, and Commitment to your L&D Team
Start with “Why.” As Simon Sinek's book recommends, before embarking on an
organizational change effort, the company must start at the beginning to ensure that everyone is
on board with the why. To do this, you can conduct brainstorming and training sessions to
empower and enable your teams to define their organizational values (their “why”). Next, you
will need to guide your teams to identify the “how” which includes the behaviors and
communication patterns that demonstrate how your organization will implement the why.
Finally, the team will have to define the “what” or the activities, tasks, and goals needed to
implement the “why” which in this case is the alignment of learning to business performance
results.
Encourage innovative and creative problem-solving. Empower your teams to think
innovatively and creatively through design thinking and agile training sessions. Design thinking
and agile process learning sessions will help L&D teams increase efficiencies, develop new
ideas, and empower them to embrace their own notions of choice and control over these ideas
through specific tasks and activities to implement them.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 121

Communicate openly. This begins with the Chief Learning Officer and cascades to
senior L&D leaders all the way to the individual team members. Openness means that the team
communicates openly about successes and challenges alike. Communication can be expressed
verbally, in writing, and non-verbally. Verbal communication may include internal meetings,
external meetings with clients and stakeholders; day to day coaching provided by managers to
team members; communication among team members and the practice of self and team
reflection. Written communication may include informal internal day to day such as email, chat,
and intranet instant messaging; in addition, it may include more formal external communications
and memos via email to clients, partners, vendors, and other stakeholders. Non-verbal
communication includes the physical environment, media kit, social media presence, logos, and
mottos of the organization.
Hire, develop and retain great people. The Chief Learning Officer and L&D
management team, using the values defined by the “why,” must define a talent acquisition,
development, and retention plan centering on hiring the right people, providing them with
learning opportunities and recognizing and rewarding them with specific rewards and
recognition programs that foster work and life balance.
Best Practice 2: Align Metrics, Strategy, and Resources to the CEO Agenda
Collect the right data. Engage the L&D team with the C-Suite through consistent,
periodic meetings with the organization leaders to listen to their vision and provide insight on
how to implement the vision through the learning strategy. From these meetings, draft a data
collection plan to ensure that the team is collecting the right data and set metrics that align with
the vital C-suite goals.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 122

Develop a strategic plan. Provide your team training sessions and tools on how to
develop a strategy, how to conduct an environmental scan externally of the industry and
internally the organization; how to conduct a SWOT analysis, and how to set SMART goals. The
data from all these components and the budget will be used to build the strategic plan. These
sessions can be followed by an opportunity to practice and then to receive feedback on these
skills during the weekly team reflection sessions.
Align your resources to the CEO’s agenda. Having set the metrics and the strategic
plan, the Chief Learning Officer, and the management team must align the budget, the people,
and the technology resources to the organizational priorities gleaned from the meetings with the
C-Suite, to ensure that all align to the CEO agenda. This alignment can be fostered through
training and tools to enable the L&D team to implement it correctly. Furthermore, the alignment
can be practiced and implemented in strategic planning sessions with the team. The output
resource allocation plan can be shared with the C-Suite team to ensure the alignment is on target.
Best Practice 3: Cultivate Continuous Improvement
Engage in benchmarking. The Chief Learning Officer and L&D management team will
need to provide learning resources, and training to the L&D team so that they can learn how to
conduct benchmarking, identify best practice organizations within and outside their industry, and
engage in benchmarking visits to collect, glean, adopt, and adapt best practices within the
learning and development division and the organization as a whole.
Seek accreditation. This step pertains to corporate universities. The Chief Learning
Officer and the L&D management team will need to provide guidance and learning opportunities
to the L&D team to prepare, seek, and obtain higher educational institution accreditation from
organizations such as the Council for Opportunity in Education (COE), Distance Education
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 123

Accrediting Commission (DEAF), and others. The accreditation process can be quite rigorous as
it involves detailed audits of all services, processes, and products developed and offered by an
educational entity. As such, the accreditation process can be resource heavy requiring significant
planning and preparation.
Apply for industry awards. Similar to accreditation, L&D teams can apply for industry
awards such as the Chief Learning Office LearningElite award, Brandon Hall, and Training
Industry, among others. In this case, the Chief Learning Office and the L&D Management team
will need to engage the L&D team with learning opportunities on how to seek, apply and win
industry awards through the specific, and often rigorous, application preparation, application,
and communication process required by such awards. Industry recognition not only helps foster
continuous improvement within the L&D team, but it also motivates the L&D team to work
harder in aligning learning to business performance results. More importantly, industry awards
demonstrate to the CEO and the C-Suite the excellence and the focus of L&D in aligning
learning to business performance results. In the following section, the researcher presents an
integrated implementation and evaluation plan to assist L&D teams in implementing and
evaluating these goals.

Integrated Implementation and Evaluation Plan
Implementation and Evaluation Framework
This analysis deploys the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick,
2016) as the framework for implementing and evaluating the recommendations listed in earlier
sections in this chapter. The New World Kirkpatrick Model, which was launched in 2010,
expands upon the teachings of D. Kirkpatrick (Kirkpatrick, 2006) and address the rapidly
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 124

changing business and learning landscape due to quickly changing technology. As seen in Figure
6 below, this model included the original four levels of evaluation, in reverse and reframed order
(Kirkpatrick & Kirkpatrick, 2016).

Figure 6. The New World Kirkpatrick Model.
Specifically, starting with Level 4, the model examines the degree to which targeted
outcomes are evidenced in the employees work as a result of the learning experience provided to
them along with support and accountability content also available to them (Kirkpatrick &
Kirkpatrick, 2016). Next, with Level 3, the model examines the degree to which employees
apply what the learned back on the job through monitoring, encouraging, rewarding, and
reinforcing from their supervisors and organizations at large (Kirkpatrick & Kirkpatrick, 2016).
With Level 2, the model examines the degree to which employees embrace the intended
knowledge, skills, attitude, confidence, and commitment to do the tasks in a new and enhanced
way (Kirkpatrick & Kirkpatrick, 2016). Finally, with Level 1, the model examines the degree to
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 125

which employees are engaged in the learning experience, find it relevant, and are satisfied
overall (Kirkpatrick & Kirkpatrick, 2016).
Organizational Purpose, Need, and Expectations
As discussed throughout this study, learning and development organizations are
challenged with aligning learning to business performance results due to limitations existing in
across their knowledge, motivation, and organizational contexts. Through this analysis, the
current researcher presented the promising practices of the Defense Acquisition University and
distilled the knowledge, motivation, and organizational influences that drove DAU’s journey of
excellence between 2008-2018 in successfully aligning learning to business performance results.
The goal of this implementation and evaluation plan was to integrate these best practices
and provide to Chief Learning Officers and their Learning and Development teams a blueprint to
adopt and adapt in their contexts. This implementation and evaluation blueprint includes
practical tools and tactics L&D teams can use to implement and evaluate the best practices they
chose to adopt and adapt so that they can to successfully align learning to business performance
results in their context. In the following sections, the researcher will describe in detail the levels
of the New Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016), as well as the steps required
within each level to enable L&D teams to successfully align their learning to business
performance results.
Level 4: Results and Leading Indicators
In Level 4, the New World Kirkpatrick Model examines the degree to which targeted
outcomes are evidenced in the employees work as a result of successfully aligning learning to
business performance results (Kirkpatrick & Kirkpatrick, 2016). Such results need to be visible
as outcomes, evidenced by leading indicators, and measured with specific methods. Specifically,
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 126

external outcomes are defined as the results visible to anyone externally or internally at the
organization. Internal outcomes are defined as the results visible and experienced internally at
the organization overall and by the learning and development team. Leading indicators are the
metrics that can assist in evidencing the outcome during and after implementation. Additionally,
a method is described which will be how the leading indicator will be evidenced. Table 13
describes the expected external and internal outcomes, as well as the leading indicator and
method for each one.
Table 13
Outcomes, Leading Indicators, and Methods for External and Internal Outcomes
Outcome Leading Indicators Method(s)
External Outcomes
Increased Industry
recognition for the
alignment of learning to
business performance
results
Number of Industry awards won
per year
Industry Award Winners List
Improved business services
and products
Number of positive customer
feedback comments on social
media per month
Monthly review of customer
ratings on social media
Renewed accreditation as a
corporate university
Reduce the number of
discrepancies in the accreditation
review
Accreditation Organization report
Internal Outcomes
All learning modules
designed internally to align
with the CEO agenda
Number of learning modules
aligning to CEO agenda
Crosswalk list aligning CEO
agenda goals and learning module
learning objectives
Improved business
performance results
achieved by employees
Number of employees confirming
applying learning on the job
Employee surveys 60 days after
training completion
Improved benchmarking
process applications within
an organization
Number of benchmarking gleaned
best practices adopted and adapted
to improve the alignment of
learning
Two benchmarking visits per
quarter

Level 3: Behavior
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 127

Critical behaviors. In Level 3, the New World Kirkpatrick model examines the degree
to which employees apply what the learned back on the job through monitoring, encouraging,
rewarding, and reinforcing from their supervisors and organizations at large (Kirkpatrick &
Kirkpatrick, 2016). As such, six behaviors are identified as necessary to be implemented by the
learning and development team for the L&D organization to successfully align learning to
business performance results.
The six critical behaviors are: (a) Learning and Development employees conduct self and
team reflection; (b) Learning and development employees conduct benchmarking visit with other
organizations outside industry; (c) Learning and development employees set SMART goals for
their individual development; (d) Learning and Development employees conduct a robust
strategic planning process; (e) Learning and Development employees collect and analyze the
right data regarding learning usage, and (f) Learning and Development employees prioritize
decisions regarding people and systems in line with business goals .
As seen in Table 14, each of the critical behaviors has a metric, describing what will be
used to measure the critical behavior implementation progress; a method, describing how the
data regarding the metric will be collected; and timing, which is a recommendation of how often
the data will need to be collected to evidence the critical behavior necessary to enable the L&D
team to successfully align learning to business performance results.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 128

Table 14
Critical Behaviors, Metrics, Methods, and Timing of Evaluation
Critical Behavior Metric(s) Method(s) Timing
1 Learning and Development
employees conduct self and team
reflection
Number of
learnings gleaned
from self and team
reflection
Weekly self-reflection
report and Weekly
team reflection report
Weekly
2 Learning and development
employees conduct benchmarking
visit with other organizations
outside the industry
Number of
benchmarking visits
completed per year
Quarterly
benchmarking visit
report
Quarterly
3 Learning and development
employees set SMART goals for
their individual development
Number of SMART
goals set per
employee
Annual individual
Performance Plan
reported completed
Goals are set
annually and
reviewed
quarterly
4 Learning and Development
employees conduct a robust
strategic planning process
Number of learning
strategy goals
incorporated and
measured in overall
organizational
strategy
Annual strategic
planning process with
quarterly report outs
Annual with
quarterly report-
outs
5 Learning and Development
employees collect and analyze the
right data regarding learning usage
Quantity and quality
of data collected
Weekly and monthly
data collection and
analytics report
Weekly and
Monthly
6 Learning and Development
employees prioritize decisions
regarding people and systems in line
with business goals
Number of newly
hired employees
whose skills align
with business needs
to achieve business
goals
Quarterly newly hired
report
Quarterly and
annually

Required drivers. Several vital drivers are required to be in place to successfully support
the implementation of the critical behaviors, both from the organizational context as well as the
motivational influences framework. These needed drivers are defined by four categories: (a)
reinforcing, (b) encouraging, (c) rewarding, and (d) monitoring. Of the 15 required drivers
identified, 14 fall in the reinforcing, rewarding, and monitoring categories and are anchored in
the organizational context; as such, they are implemented by managers and supervisors. One
encouraging driver emanates from the learning and development employees and in supporting
each other. Table 15 below consists of three columns. The first column lists each of the drivers
required to support the critical behaviors; the second column recommends the periodicity or
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 129

timing for implementation of each driver, and the third column specified the corresponding
critical behavior each of the required drivers' support.
Table 15
Required Drivers to Support Critical Behaviors
Method(s) Timing Critical Behaviors Supported
Reinforcing
Managers supply team reflection
report and job aid
Annually 1
Managers supply benchmarking
process and report job aid
Annually 2
Managers provide annual
performance plan and job aid on
how to complete it
Annually 3
Managers supply employees job
aid in defining SMART goals
Annually 3
Managers supply to employees
data collection and analytics
report with job aid on how to
complete it
The report is provided semi-
annually
5
Encouraging
Employees meet with their teams
to conduct team reflection
Weekly 1
Managers encourage and support
their employees in their effort to
set SMART goals their Individual
Development Plan
Annually 3
Rewarding
Managers publicly acknowledge
employee progress in conducting
quarterly benchmarking visits
Quarterly 2
Managers publicly recognize
employee progress in
incorporating learning goals in
the organizational learning
strategy
Annually 4
Monitoring
Managers review number of
newly hired employees whose
skills align with business needs to
achieve business goals
Quarterly 6
Managers review quantity and
quality of data collected
Monthly and quarterly 5
Managers review number of
learning strategy goals
incorporated and measured in
overall organizational strategy
Quarterly and annually 4
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 130

Managers review number of
SMART goals set per employee
Semiannually and annually 3
Managers review quarterly
benchmarking report
Quarterly 2
Managers review weekly team
reflection report
Weekly 1

Organizational support. As the researcher discussed earlier in this chapter, the
organization has an important role to play in the realization and implementation of the critical
behaviors necessary to enable learning and development organizations to successfully align
learning to business performance results. Specifically, the organization must foster a culture of
openness, where communication is transparent and continuously focusing on the broader
organizational mission. Additionally, the learning and development organization must ensure
that all initiatives in this recommended program are aligning with a more comprehensive strategy
of aligning learning to business performance results. Finally, the organization must support its
people by encouraging and rewarding employees and by prioritizing resources such as funding
and time to implement this organizational change program based on the implementation and
evaluation blueprint presented here. In the sections which follow, Level 2 and Level 1, the
remaining two levels of the New World Kirkpatrick model (Kirkpatrick & Kirkpatrick, 2016)
will be discussed.
Level 2: Learning
The second level in the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016)
refers to learning, or the degree to which the employees—in this case, the learning and
development team members—embrace the intended knowledge, skills, attitude, confidence, and
commitment to do the tasks in a new and enhanced way. Level 2 is defined by and measured by
learning goals discussed in the sections that follow.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 131

Learning goals. Below are the learning goals, gleaned from the best practices
implemented by DAU, for Learning and Development employees seeking to align learning to
business performance results:
1. Implement a rigorous strategic planning process (P)
2. Collect meaningful data and analyze it rigorously (P)
3. Conduct self and team reflection (M)
4. Conduct rigorous benchmarking visits (P)
5. Submit well-articulated Industry Award applications (P)
6. Attribute your success or failure to your own efforts (Attribution)
7. Form functional teams to foster a positive emotional environment (Teamwork)
8. Foster innovation (Drive to innovate)
9. Implement of new methodologies and process improvements (Goals)
10. Articulate the values, goals, and actions relating to your organizational “why” (Values)
Program. To engage learning and development team managers and employees in
collaborating and co-build their implementation plan, the researcher drafted a recommended
program to include a full day offsite to brainstorm, discuss, and agree upon critical goals and
metrics developed by the learning and development team. The goal of the program will be to
incorporate the activities and actions required to encourage the L&D team to learn and embrace
the knowledge and mindset of aligning learning to business performance results. A sample
program can be found in Appendix E.
Communications strategy. A key component for the success of the program is a
communication plan to inform and perform pulse checks of the learning and development
employees before, during and after the program, as well as 60 days and 90 days beyond program
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 132

completion. The proposed communication plan will aim to inform, engage, motivate, and
ultimately commit the learning and development team program employees to implement the
changes needed in their organization to successfully align learning to business performance
results. Specifically, the recommended communications strategy would include social media
engagement for employees with the creation of a hashtag such as #learning2results to engage
employees. The social media hashtag can then broaden the conversation to other organizations
seeking to align their learning with results. Next, a blog can be written by the Chief Learning
Officer championing the event using the hashtag #learning2results, inviting learning and
development employees in his or her organization and other organizations to weigh in and also
post their blogs with the same hashtag. Next, an infographic outlining the program
implementation using the Kirkpatrick New World Model can be developed. Additionally, a
webcast with the Chief Learning Officer discussing the initiative, describing the goals, and the
challenges and calling to action learning and development professionals to align learning to
business performance results can be included in the communications campaign. Social media
data analytics will be deployed to evaluate the communications strategy including the number of
blog post likes, reposts, and comments, new blogs created using the hashtag, as well as likes,
reposts, and shares of the infographic. The evaluation will be discussed in the next section.
Level 1: Reaction
Finally, with Level 1, New World Kirkpatrick Model examines the degree to which
employees are engaged in the learning experience, find it relevant, and are satisfied overall
(Kirkpatrick & Kirkpatrick, 2016). Table 14 below lists the recommended methods and tools the
employees, in this case, the Learning and Development team, can use to determine how the
employees react to the full-day offsite and upon return to the workplace for further program
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 133

implementation. Table 16 also consists of two columns. The column on the left lists the methods
or tools to evaluate the level of engagement among the learning and development employees; the
degree of relevance of the program content, and the level of satisfaction of the program
employees. The column on the right lists the recommended timing for each of the Level 1
evaluation methods or tools.
Table 16
Components to Measure Reactions to the Program
Methods or Tools Timing
Engagement
Tweeter posts using the hashtag
#Learning2Results
During the program, immediately after the
program, 60 and 90 days after program event

Attendance records

At the beginning and end of the program
Asking meaningful questions During
Pulse check of employees with an on-the-
spot poll using polltogo
Before, during and after completion of the
program

Participating in project teams and report-outs During
Relevance
Pulse check using polltogo 3x per day During coffee breaks and lunch
Anonymous survey End of the program
Customer Satisfaction
Facebook live sessions asking employees to
share live their impressions of the program
live online
During lunch and Immediately after
completion of the program

Anonymous survey After each program session

Evaluation of the Components of Learning
Table 13 describes the evaluation methods recommended for each of the components of
the learning recommended for the full day offsite program presented in the earlier section. Table
17 consists of two columns; the first column lists the methods or activities recommended for the
one-day offsite to support the procedural skills, the attitude, the confidence, and the commitment
of the learning and development team. Additionally, the second column of Table 17 specifies the
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 134

timing of the method or activity recommended in column one. As such, the evaluation
components are spread to take place during the actual offsite program, after the program
concludes, or both.
Table 17
Evaluation of the Components of Learning for the Program
Method(s) or Activity(ies) Timing
Procedural Skills “I can do it right now.”
‘Jeopardy’ Game with questions and answers
on the various procedural knowledge skills
During
Project teams draft an annual benchmarking
visits plan and report out.
During and refine/finalize after
Project teams draft the annual learning goals
and report out
During
Project teams select data and define collection
tools and methods
During
Attitude “I believe this is worthwhile.”
Discussions about the value and the rationale During
Self-reports of the progress During
Action Plans of how-to-implement new skills Before, during, after
Confidence “I think I can do it on the job.”
Discussion in teams with report outs to the
broader team about concerns, barriers.
During and report after
Teaming between mentors and mentee teams During and after
Peer check-ins During and after
Commitment “I will do it on the job.”
Self-reports of the progress After
Action Plans of how-to-implement new skills After

Evaluation Tools
Several tools can be deployed to evaluate the program at each level including social media,
anonymous surveys, and small group discussions. The tools and the timing of their use are listed
in Table 18 below.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 135

Table 18
Evaluation Tools

Level

1 Day before
the program
begins

During the
program

Immediately
after the
program ends

60 days after the
program ends

90 days after
the program
ends
Level 4
Results
Tabulation of
2 surveys
results and
review of all
data analytics
from Level 1,
2, and 3
Level 3
Behavior
Survey 5
questions of
employees and
their managers
Small group
discussions
with
employees
and their
managers to
discuss if and
how
behaviors
have changed
on the job as a
result of the
program
Level 2
Learning
Facebook live
sessions
Survey 5
questions
Small group
discussions
with
employees
and their
managers to
discuss if and
how the new
learning has
changed how
they do their
job as a result
of the
program
And review
products and
artifacts they
developed.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 136

Level 1
Reaction
Participant
pulse check
using polltogo
Twitter posts
using hashtag
#learning2results

Participant Pulse
check using
polltogo sent on
the participant
smartphone
Participant
Pulse check
on polltogo
sent on the
participant
smartphone
Twitter posts
using
#learning2results

Pulse check on
polltogo sent on
the participant
smartphone

Review of CLO
Blog post likes,
reposts, and
shares

Review of the
infographic data
analytics

Review of the
additional blog
posts’ data
analytics

Details about each tool and its respective timing have been included in the Appendix.
Data Analysis and Reporting
Data analysis and reporting are crucial steps to the successful alignment of learning to
business performance results. The researcher created a data collection matrix and two
dashboards, one for external results and one for internal results, in order to facilitate data
reporting and analysis. A sample data collection matrix and two dashboards have been included
in the Appendix E and F.
Summary
Learning and development teams within organizations are not able to efficiently align
learning to business performance results (Ben-Hur et al., 2015). The purpose of this promising
practice study was to use the Clark and Estes (2008) gap analysis model to evaluate and glean
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 137

the influence of knowledge, motivation, and organizational best practices implemented in the
alignment of learning to business performance results at the Defense Acquisition University. A
literature review presented this challenge facing the learning and development industry and
outlined the necessary elements for successful alignment of learning to business performance
results.
Assumed knowledge, motivation, and organizational influences were identified and
verified against related literature. Data collection through document review and semi-structured
interviews followed, and data analysis ensued to highlight best practices from the findings. All
three assumed knowledge influences were central to successfully aligning learning to business
performance results: strategic planning process, data collection and analysis, and self and team
reflection. Additionally, three more procedural knowledge influences emerged, including:
benchmarking, accreditation, and industry award applications. Moreover, all three assumed
motivation influences were gleaned during data analysis: attributions, goals, and values. Finally,
the researcher gleaned all three assumed organizational influences from data analysis, acting as
the glue that connected all the other influences: culture, alignment, and resource allocation.
Furthermore, the researcher leveraged the New World Kirkpatrick Model of Kirkpatrick
and Kirkpatrick (2016) as the framework for implementing and evaluating the recommendations
listed in earlier sections in this chapter. The New World Kirkpatrick Model, which was launched
in 2010, expands upon the teachings of D. Kirkpatrick (Kirkpatrick, 2006) in order to address the
rapidly changing business and learning landscape due to rapidly changing technology.
Starting with Level 4, the model examined the degree to which targeted outcomes are
evidenced in the employees work as a result of the learning experience provided to them along
with support and accountability content also available to them (Kirkpatrick & Kirkpatrick, 2016).
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 138

Next, with Level 3, the model examined the degree to which employees apply what the learned
back on the job through monitoring, encouraging, rewarding, and reinforcing from their
supervisors and organizations at large (Kirkpatrick & Kirkpatrick, 2016). With Level 2, the
model examined the degree to which employees embrace the intended knowledge, skills,
attitude, confidence, and commitment to do the tasks in a new and enhanced way (Kirkpatrick &
Kirkpatrick, 2016). Finally, with Level 1, the model examined the degree to which employees
are engaged in the learning experience, find it relevant, and are satisfied overall (Kirkpatrick &
Kirkpatrick, 2016).
The resulting blueprint of recommendations aspire to serve as a performance
enhancement framework available for adoption, adaptation, and implementation by Chief
Learning Officers, learning and development teams and organizations, and Corporate
Universities striving to successfully align learning to business performance results. It is
anticipated that deliberate and systematic application of the recommendations included in this
study can increase learner satisfaction by enhancing their learning experiences and improve the
alignment of learning to business performance results.
Strengths and Weaknesses of the Approach
While the Clark and Estes (2008) model provided a robust framework of the knowledge,
motivation, and organizational influences, it is designed to analyze performance gaps. In the case
of this analysis, which is a promising practice, there were no performance gaps but rather,
performance excellence to be examined and therefore, the application of the Clark and Estes
model was not a classic fit. The combination of these authors’ conceptual pillars of knowledge,
motivation, and organizational influences with the New World Kirkpatrick Model (Kirkpatrick
Partners, 2016), which is used at depth at the organization examined in this analysis, provided a
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 139

robust performance improvement framework with specific best practices that can be considered
for adoption and adaptation by learning and development organizations and corporate
universities.
Limitations and Delimitations
Limitations
To further protect the credibility and trustworthiness of the study, limitations to the study
must be disclosed (Maxwell, 2013). Limitations consist of elements in the study that cannot be
controlled by the researcher, including factors relating to sample size, sample representation,
potential confounding variables, limitations in the data collection process, and researcher bias
and subjectivity, among others (Glesne, 2015; Maxwell, 2013). For this study, four fundamental
limitations have been identified over which neither the researcher nor the study had control over:
first, the truthfulness in the responses of the participants; second, the veracity of the documents
and the artifacts collected for review; third, the reflexivity and subjectivity of the researcher; and,
fourth, time constraints (Merriam & Tisdell, 2015).
Regarding the truthfulness in the responses of the interview participants, the risk can be
mitigated given the fact that the interview participants are federal employees and are bound by
federal regulations regarding transparency and disclosure to the public. The veracity of the
documents acquired for review in this study is also a limitation which is addressed given that the
Defense Acquisition University is a federally funded institution and therefore is bound by federal
laws and regulations governing documentation transparency and veracity. The researcher noted
that all the information accessed for this study is available for public review and is not subject to
any clearance (DAU, 2016). Inherently, the subjectivity of the researcher is the third limitation to
be addressed in this study. In this study, as discussed earlier in the Role of the Researcher
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 140

section, the researcher is one of the instruments through which data collection, analysis, and
interpretation take place (Merriam & Tisdell, 2015). The researcher was deeply informed about
the excellence journey accomplished by the Defense Acquisition University before engaging in
this analysis and was hired by the Defense Acquisition University at the end of the data analysis
phase in Chapter 4. Although actions were taken to minimize bias as much as possible to provide
a meaningful and robust analysis, access to stakeholders and access to information were
influenced by this relationship in this analysis.
Additionally, because of the limited resource of time, only six stakeholders were
interviewed in an organization with over 725 employees. In addition to the interviews, the
researcher reviewed and analyzed 50 documents and three videos. Although the data from the
interviews was rich in content, as was the data from the document and artifact review, and all
assumed influences were validated, a larger group of interviewees and more documents could
have presented an even richer pool of content for analysis.
Delimitations
Study delimitations refer to the boundaries set in the study by the researcher and must
also be disclosed to protect the validity and reliability of the study (Maxwell, 2013; Merriam &
Tisdell, 2015). In this study, delimitations pertained to sampling and data collection, as well as
the methodology (Merriam & Tisdell, 2015). Regarding the sampling and data collection
approach, while there were several stakeholders that actively contributed in the achievement of
the organizational goal at DAU and could speak in detail about their role, due to time constraints
and cost limitations, only one group was selected to participate for sampling purposes in the data
collection process.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 141

Additionally, there are numerous documents, books, articles, curricula, reports, and other
artifacts available from the DAU discussing the journey of successfully aligning learning to
business performance goals, the researcher selected specific books, articles, reports, and award
questionnaire submissions because they were deemed most relevant to the objectives of the study
(Creswell & Poth, 2014; Merriam & Tisdell, 2015). While there are several methodological
approaches to conduct a promising practice study, the methodology and instrumentation selected
for this study is qualitative analysis using structured and in-depth interviews, as well as
document review and analysis. As per Creswell and Poth (2014) and Maxwell (2013), this
methodology was the most appropriate approach for a promising practice study to glean best
practices from an organization which has already achieved its goal of aligning learning to
strategy.
Future Research
In this study, the researcher analyzed the knowledge, motivation, and organizational
influences in successfully aligning learning to business performance results from the perspective
of the learning analytics, and academic staff at the Defense Acquisition University, a corporate
university within the United States Department of Defense during the timeframe 2008-2018.
Future research could include the exploration of an acquisition university learner stakeholder
group to analyze directly the learners’ expectations, perspectives, and on-the-job experiences in
the alignment of learning to business performance results and the impact on their learning
journeys and career growth as members of the acquisition workforce.
Also, because the Defense Acquisition University is now embarking on a journey of
customer intimacy, additional research may be warranted to examine the impact on aligning
learning to business performance results. In this case, the study could be conducted with three
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 142

distinct research stakeholder groups: (a) the acquisition professionals who are the learners and
are defined as the consumers of learning; (b) the customers, who are the Program Managers at
the various Program Management Offices within the Department of Defense; and (c) the
leadership at the Department of Defense Services (Air Force, Army, Navy, and the Marine
Corps) and other Department of Defense Agencies.
Conclusion
Learning and development teams within organizations are not able to efficiently align
learning to business performance results (Ben-Hur et al., 2015). The purpose of this promising
practice study was to use the Clark and Estes (2008) gap analysis model to evaluate and glean
the influence of knowledge, motivation, and organizational best practices implemented in the
alignment of learning to business performance results at the Defense Acquisition University. A
literature review presented this challenge facing the learning and development industry and
outlined the necessary elements for successful alignment of learning to business performance
results.
Assumed knowledge, motivation, and organizational influences were identified and
verified against related literature. Data collection through document review and semi-structured
interviews followed, and data analysis ensued to highlight best practices from the findings. All
three assumed knowledge influences were encountered during data analysis: strategic planning
process, data collection and analysis, and self and team reflection. Additionally, three more
procedural knowledge influences emerged: benchmarking, accreditation, and industry award
applications. Moreover, all three assumed motivation influences were encountered during data
analysis: attributions, goals, and values. Finally, all three assumed organizational influences were
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 143

also encountered during data analysis, acting as the glue that connected all the other influences:
culture, alignment, and resource allocation.
The current researcher compiled a blueprint of recommendations to serve as a
performance enhancement framework available for adoption, adaptation, and implementation by
Chief Learning Officers, learning and development teams and organizations, and Corporate
Universities striving to successfully align learning to business performance results. The
researcher anticipates that deliberate and systematic application of the recommendations
included in this study can increase learner satisfaction by enhancing their learning experiences
and improving the alignment of learning to business performance results.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 144

References
Ahmadi, S. A. A., Salamzadeh, Y., Daraei, M., & Akbari, J. (2012). Relationship between
organizational culture and strategy implementation: Typologies and dimensions. Global
Business and Management Research, 4(3/4), 286.
Alagaraja, M., & Li, J. (2015). Utilizing institutional perspectives to investigate the emergence,
rise, and (relative) decline of corporate universities. Human Resource Development
International, 18(1), 4-23. doi:10.1080/13678868.2014.979003
Alonso-Gonzalez, A., Peris-Ortiz, M., and Mauri-Castello, J. (2017). Collaborative networks
between corporate universities, customers, and SMEs: Integrating strategy towards value.
In Cooperative and networking strategies in small business (pp. 197-205).
Anderman, E. M., & Young, A. J. (1994). Motivation and strategy use in science: Individual
differences and classroom effects. Journal of Research in Science Teaching, 31(8), 811-
831.
Anderson, F. J., Hardy, C. R., & Leeson, J. (2008). Leading a learning revolution: The story
behind Defense Acquisition University's reinvention of training. Malden, MA: John
Wiley and Sons.
Asif, M., & Gouthier, M. H. (2014). What service excellence can learn from business excellence
models. Total Quality Management & Business Excellence, 25(5/6), 511 –531.
Bandura, A. (2007). Guide for constructing self-efficacy scales. Self-Efficacy Beliefs of
Adolescents, 5, 307-337.
Beauchamp, T. L. (2008). The Belmont report. The Oxford Textbook of Clinical Research Ethics,
21-28.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 145

Beausaert, S., Segers, M., & Gijselaers, W. (2011). The use of a personal development plan and
the undertaking of learning activities, expertise-growth, flexibility, and performance: The
role of supporting assessment conditions. Human Resource Development
International, 14(5), 527-543.
Ben-Hur, S., Jaworski, B., & Gray, D. (2015). Aligning corporate learning with strategy. MIT
Sloan Management Review, 57(1), 53.
Berger, J. (2014). Word of mouth and interpersonal communication: A review and directions for
future research. Journal of Consumer Psychology, 24(4), 586-607.
Bird, H. (2008). Articulating the value of training: Linking training programs to organizational
goals. Development and Learning in Organizations: An International Journal, 22(2), 20-
23.
Bjerke, M. B., and Renger, R. (2017). Being smart about writing SMART objectives. Evaluation
and Program Planning, 61, 125-127.
Bock, L. (2015). Work rules! Insights from inside Google that will transform how you live and
lead. London, UK: Hachette UK.
Bonner, S. E., Hastie, R., Sprinkle, G. B., & Young, S. M. (2000). A review of the effects of
financial incentives on performance in laboratory tasks: Implications for management
accounting. Journal of Management Accounting Research, 12(1), 19-64.
Bose, R. (2004). Knowledge management metrics. Industrial Management and Data Systems,
104(6), 457-468.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 146

Bright, P., Ahmad, W., & Zahra, U. (2017). An investigation into the relationship of strategic
planning practices and organizational performance using advanced data mining
techniques. Paper presented at the Asian Conference on Intelligent Information and
Database Systems, 676-687.
Cappiello, G., & Pedrini, G. (2017). The performance evaluation of corporate
universities. Tertiary Education and Management, 23(3), 304-317.
Cho, Y. S., & Jung, J. Y. (2014). The relationship between metacognition, entrepreneurial
orientation, and firm performance: An empirical investigation. Academy of
Entrepreneurship Journal, 20(2), 71-86.
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions (2nd ed.). Scottsdale, AZ: Information Age Publishing.
Collings, D. G., & Mellahi, K. (2009). Strategic talent management: A review and research
agenda. Human Resource Management Review, 19(4), 304-313.
Creswell, J. W., and Poth, C. N. (2017). Qualitative inquiry and research design: Choosing
among five approaches. Thousand Oaks, CA: SAGE.
Dembo, M. H., & Eaton, M. J. (2000). Self-regulation of academic learning in middle-level
schools. The Elementary School Journal, 100(5), 473-490.
Denler, H., Wolters, C., & Benzon, M. (2012). Social Cognitive Theory. Retrieved from
http://www.education.com/reference/article/social-cognitive-theory/
Defense Acquisition University. (2015). Annual report. Retrieved from
www.dau.mil/publications/ATLdocs/Annual_report.pdf
Defense Acquisition University. (2017). LearningElite 2017. Retrieved from
https://www.dau.mil/about/.../DAU%20CLO%202017%20Learning%20Elite.pdf
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 147

Downes, P. E., Kristof-Brown, A. L., Judge, T. A., & Darnold, T. C. (2017). Motivational
mechanisms of self-concordance theory: Goal-specific efficacy and person–organization
fit. Journal of Business and Psychology, 32(2), 197-215.
Elbanna, S., Andrews, R., & Pollanen, R. (2016). Strategic planning and implementation success
in public service organizations: Evidence from Canada. Public Management
Review, 18(7), 1017-1042. doi:10.1080/14719037.2015.1051576
Frink, D. D., & Klimoski, R. J. (1998). Toward a theory of accountability in organizations and
human resource management. In G. R. Ferris (Ed.), Research in personnel and human
resources management (Vol. 16, pp. 1–51). Greenwich, CT: JAI Press.
Furtherer, S. L. (2016). Lean six sigma in service: Applications and case studies. Boca Raton,
FL: CRC Press.
Glaister, A. J., Karacay, G., Demirbag, M., & Tatoglu, E. (2018). HRM and performance: The
role of talent management as a transmission mechanism in an emerging market
context. Human Resource Management Journal, 28(1), 148-166.
Gelfand, M. J., Lim, B., & Raver, J. L. (2004). Culture and accountability in organizations:
Variations in forms of social control across cultures. Human Resource Management
Review, 14(1), 135-160.
Glesne, C. (2015). Becoming qualitative researchers: An introduction. Upper Saddle River, NJ:
Pearson.
Greco, P., Laschinger, H. K. S., & Wong, C. (2006). Leader empowering behaviours, staff nurse
empowerment, and work engagement/burnout. Nursing Leadership, 19(4), 41-56.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 148

Gunawardena, C. N., Linder-VanBerschot, J. A., LaPointe, D. K., & Rao, L. (2010). Predictors
of learner satisfaction and transfer of learning in a corporate online education program.
American Journal of Distance Education, 24(4), 207-226.
Haines, S. (2016). The systems thinking approach to strategic planning and management. Boca
Raton, FL: CRC Press.
Jaiswal, N. K., & Dhar, R. L. (2016). Fostering employee creativity through transformational
leadership: Moderating role of creative self-efficacy. Creativity Research Journal, 28(3),
367-371.
Juan, S. H., Yao, L., Tamyez, P. F. B. M., & Ayodele, F. O. (2016). Review on knowledge
management and employee engagement. Proceedings of the National Conference for
Postgraduate Research (NCON-PGR 2016), 24-25 September 2016; 127-134.
Kanter, R. M. (1979). Power failure in management circuits. Classics of Organization
Theory, 342-351.
Kirkpatrick, D. L. (2006). Seven keys to unlock the four levels of evaluation. Performance
Improvement, 45(7), 5-8.
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick's four levels of training evaluation.
Alexandria, VA: Association for Talent Development.
Kirkpatrick Partners. (2016). The New World Kirkpatrick Model. Retrieved from
http://www.kirkpatrickpartners.com/Our-Philosophy/The-New-World-Kirkpatrick-Model
Kirschner, F., Paas, F., & Kirschner, P. A. (2009). A cognitive load approach to collaborative
learning: United brains for complex tasks. Educational Psychology Review, 21(1), 31-42.
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into Practice,
41(4), 212-218.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 149

Kuper, A., Lingard, L., & Levinson, W. (2008). Critically appraising qualitative research. BMJ,
2008, a1035. doi:10.1136/bmj.a1035
LearningElite (2017). Executive summary. Retrieved from http://download.clomedia.com/2017-
learningelite-executive-summary
Lee, V., & Ooi, K. (2015). Applying the Malcolm Baldrige national quality award criteria: An
approach to strengthen organizational memory and process innovation. Total Quality
Management & Business Excellence, 26(11-12), 1373-1386.
doi:10.1080/14783363.2014.934519
Leshem, S., & Trafford, V. (2007). Overlooking the conceptual framework. Innovations in
Education and Teaching International, 44(1), 93-105.
Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current Directions
in Psychological Science, 15(5), 265-268.
Luthans, F., Luthans, B. C., & Luthans, K. W. (2015). Organizational behavior: An evidence-
based approach. Charlotte, NC: Information Age Publishing.
Lykins, L (2012). Getting Results: Aligning Learning’s Goals with Business Performance. T+D
66(4), 28.
Maslach, C., & Leiter, M. P. (1997). The truth about burnout. San Francisco, CA: Jossey-Bass.
Maxwell, J. A. (2013). Qualitative research design: An interactive approach. Thousand Oaks,
CA: SAGE.
Mayer, R. E. (2011). Does styles research have useful implications for educational practice?
Learning and Individual Differences, 21(3), 319-320.
Merriam, S. B., & Tisdell, E. J. (2015). Qualitative research: A guide to design and
implementation. Malden, MA: John Wiley and Sons.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 150

Miles, M. B., Huberman, A. M., & Saldana, J. (2013). Qualitative data analysis. Thousand Oaks,
CA: SAGE.
Moskovsky, C., Alrabai, F., Paolini, S., & Ratcheva, S. (2013). The effects of teachers’
motivational strategies on learners’ motivation: A controlled investigation of second
language acquisition. Language Learning, 63(1), 34-62.
Murray, P. (2003). Organizational learning, competencies, and firm performance: Empirical
observations. The Learning Organization, 10(5), 305-316.
Nalborczyk, S., and Sandelands, E. (2012). Aligning learning and business results: A case study
of Emerald Academy. Training and Management Development Methods, 26(5), 111.
Northouse, P. G. (2016). Leadership: Theory and practice. Thousand Oaks, CA: SAGE.
Patton, M. Q. (2015). Qualitative research and methods: Integrating theory and practice.
Thousand Oaks, CA: SAGE.
Perry, R. P., and Hamm, J. M. (2017). An attribution perspective on competence and
motivation. Handbook of Competence and Motivation: Theory and Application, 2006, 61.
Phillips, J. J., & Phillips, P. P. (2007). The value of learning: How organizations capture value
and ROI and translate it into support, improvement, and funds. Malden, MA: John Wiley
& Sons.
Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing.
Theory into Practice, 41(4), 219-225.
PriceWaterhouseCoopers, (2012). 15th Annual Global CEO Survey 2012. Retrieved from
https://www.pwc.com/gx/en/ceo-survey/pdf/15th-global-pwc-ceo-survey.pdf
Punch, K. F. (2013). Introduction to social research: Quantitative and qualitative approaches.
Thousand Oaks, CA: SAGE.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 151

Rocco, T. S., & Plakhotnik, M. S. (2009). Literature reviews, conceptual frameworks, and
theoretical frameworks: Terms, functions, and distinctions. Human Resource
Development Review, 8(1), 120-130.
Rogers, S. S. (2013). Great expectations: Making ROI successfully work for you. Training and
Development, 40(1), 8.
Rubin, H. J., & Rubin, I. S. (2011). Qualitative interviewing: The art of hearing data. Thousand
Oaks, CA: SAGE.
Rueda, R. (2011). The 3 dimensions of improving student performance: Finding the right
solutions to the right problems. New York, NY: Teachers College Press.
Sadler-Smith, E. (2009). Learning and development for managers: Perspectives from research
and practice. Malden, MA: John Wiley & Sons.
Schein, E. H. (2010). Organizational culture and leadership. Malden, MA: John Wiley and
Sons.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-
189.
Smith, R. (2008). Aligning Learning with Business Strategy. TandD, 62(11), 40
Sparrow, P. R., & Makram, H. (2015). What is the value of talent management? Building value-
driven processes within a talent management architecture. Human Resource Management
Review, 25(3), 249-263.
Stuckey, S. E. (2012). Examining the impact of student-generated screencasts on middle school
science students' interactive modeling behaviors, inquiry learning, and conceptual
development: A dissertation. Retrieved from
https://libres.uncg.edu/ir/asu/f/Stuckey,%20Scott_2012_Dissertation.pdf
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 152

Taegoo, T. K., Lee, G., Paek, S., & Lee, S. (2013). Social capital, knowledge sharing, and
organizational performance: What structural relationship do they have in hotels?
International Journal of Contemporary Hospitality Management, 25(5), 683-704.
doi:10.1108/IJCHM-Jan-2012-0010
Tamkin, P., Yarnall, J., and Kerrin, M. (2002). Kirkpatrick and beyond: A review of models of
training evaluation. Brighton, UK: Institute for Employment Studies.
Taylor, R. (2016). The real cost of your disengaged workforce. LinkedIn Pulse. Retrieved from
https://www.linkedin.com/pulse/real-cost-your-disengaed-workforce-richelle-taylor-pmc/
Van Rooij, S. W., & Merkebu, J. (2015). Measuring the business impact of employee learning: A
view from the professional services sector. Human Resource Development Quarterly,
26(3), 275-297.
Weiner, B. (2005). Motivation from an attribution perspective and the social psychology of
perceived competence. Handbook of Competence and Motivation, 73-84.
Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement
motivation. Contemporary Educational Psychology, 25(1), 68-81.
Williams, S. L. (2002). Strategic planning and organizational values: Links to alignment. Human
Resource Development International, 5(2), 217-233.
Wilson, D. D., & Collier, D. A. (2000). An empirical investigation of the Malcolm Baldrige
national quality award causal model. Decision Sciences, 31(2), 361-383.
doi:10.1111/j.1540-5915.2000.tb01627.x
Wood, R., & Bandura, A. (1989). Social cognitive theory of organizational management.
Academy of Management Review, 14(3), 361-384.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 153

Yigitcanlar, T. (2014). Position paper: Benchmarking the performance of global and emerging
knowledge cities. Expert Systems with Applications, 41(10), 4680-4690.
doi:10.1016/j.eswa.2014.03.032

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 154

Appendix A
Interview Protocol
Hello, my name is Marina Theodotou, and I am a doctoral student the University of Southern
California. Thank you again for agreeing and taking the time to speak with me. As you know, I
am conducting a study on the alignment of learning to business performance results. I have selected
the Defense Acquisition University given its success in such alignment. My focus is to examine
the DAU's journey and processes in aligning learning during the past seven years after the
publication of the “Leading a Learning Revolution” book by Anderson, Hardy, and Leeson in
2008. Specifically, I would like to understand how the processes changed (if at all) since then,
what new technologies or methods are being used and how has the overall strategic planning and
alignment process evolved (if at all) since 2008. The goal of this study is to highlight the DAU as
a promising practice and to glean best practices and solutions that may be valuable to other
organizations who want to align learning to business performance results.

As part of this study, I will be conducting some interviews. To facilitate note taking, I would like
to record our conversation today. I will be the only person with access to the recording which will
be destroyed after transcription. Before we begin, you will need to sign a consent form drafted to
meet the USC human subject requirements. This document states that: 1) all information will be
kept confidential, 2) that your participation is voluntary and you may stop at any time you chose
to, and 3) that the interview does not intend any harm nor offer a reward.

The interview will last no more than an hour.
When you are ready, we can begin.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 155

Let me turn on the recording.
Semi-structured interview questions (to be followed as needed by additional probing questions)
Opening Comments
So, shall we begin? Thank you again for making time to speak with me today.
First, could you please describe your role at the Defense Acquisition University?
• What have been your main responsibilities vis a vis the journey of DAU to becoming the
Learning Organization of 2017?
• Has your role changed during the last seven years?
Core Questions
1-Can you please describe this journey of excellence from your vantage point?
Probe: What were the differentiating processes DAU used in strategic planning?
Probe: How did DAU set, implement, and assess goals?
2- During this time frame what else has changed in your strategic planning processes?
Probe: Are you using new software technologies?
Probe: Have you created new metrics and reports?
Probe: How has reporting to DoD evolved?
3- Since 2008, DAU has successfully maintained the alignment of learning to business
performance results through the ALM Model. How has the model evolved since then?
4- Looking back on the last ten years, what were the biggest lessons learned from this journey for
you and the team?
Probe: What was the biggest challenge? How did you overcome it?
Probe: What would you do differently?
5- What have been the biggest improvements during the past decade? Why?
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 156

Probe: What were the differentiating actions that pushed DAU to the next level?
Probe:
6- How did DAU manage to maintain such high level of performance?
Probe: How did you maintain the enthusiasm and commitment levels when things got
tough?
Probe: How did DAU manage to make learning alignment mission critical?
7- From your perspective, which are the key factors can DAU attribute its success to? Probe:
For example, was it people, process, strategy, technology? Something else?
A combination of these?
8- In the Anderson, Hardy, and Leeson book (2008) alignment was a major driver. What is the
role of alignment today at DAU? How has it changed since 2008? Why?
9- Another key driver in the book was culture. What are the key drivers of culture today? How
has culture changed or evolved since then?
Probe: Drucker said that culture eats strategy for breakfast. Is this the case at DAU?
Why or why not?

10 - How has DAU changed the allocation of resources to continue to align learning to business
performance results? What technologies are you using today? How has the team structure
changed in the past 7 years?

11- What are the biggest takeaways for you for the last decade?
(possible emerging influencers)
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 157

Probe: What should other L&D organizations’ focus on when embarking on the learning
alignment journey?
Probe: What should they brace for?

12- During the last decade the L&D industry has changed a lot and continues to change. What
lessons learned would you like to share with other organizations in this space? What are the 3-5
key things other organizations can learn from DAU?
Closing comments
• In closing, how would you describe DAU’s journey ahead?
• Is there anything you would like to add that I forgot to ask?
An actionable result of this study will be a blueprint for other organizations to adopt and adapt in
their efforts to align learning to business performance results. Once I build the draft, I would like
to work with you and your team to ensure it is meaningful and applicable. Would that be OK
with you?
Thank you so much for your time today, I appreciate it. And thank you for your work in ensuring
the national security of our country. Would it be OK if I reach back out to you if something does
not make sense when I review the work we did today?
Many thanks again!

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 158

Appendix B
Documents and Artifacts Protocol

Item
number
Document
Name
Document
Type
Document
Date
Document
Author(s) and
Content
Description
Corresponding

KMO
Influence
Content Code
and Comment

Item
number
Artifact
Name
Artifact
Type
Artifact
Date
Artifact Corresponding
KMO
influence
Content Code
and Comment

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 159

Appendix C
Levels 1 and 2 Evaluation Tool

2/21/2018 Evaluation Tool 1 - For Levels 1 & 2
https://docs.google.com/forms/d/1N1KkiAC-uz0mWQTqdKz80ewU_sMriuw1khXJshwKy3A/edit 1/2
Powered by
Evaluation Tool 1 For Levels 1 & 2
This 5 question survey must be administered immediately after the program ends directly via email to the
smartphone of the program participants
1.
Q1: I thought the program was well developed
Mark only one oval.
1 2 3 4 5
Completely Disagree Completely Agree
2.
Q2: I thought the program was engaging
Mark only one oval.
1 2 3 4 5
Completely Disagree Completely Agree
3.
Q3: I thought the program presenters were prepared
Mark only one oval.
1 2 3 4 5
Completely Disagree Completely Agree
4.
Q4: I thought the program materials were valueadding
Mark only one oval.
1 2 3 4 5
Completely Disagree Completely Agree
5.
Q5: I am satisfied with the program
Mark only one oval.
1 2 3 4 5
Completely Disagree Completely Agree
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 160

Appendix D
Levels 3 and 4 Evaluation Tool

2/21/2018 Evaluation Tool 2 - Level 3 and 4
https://docs.google.com/forms/d/19Rq2X3lWByY6DITJJA8OlkZaMrs1WJkVMxsGCTjKsrg/edit 1/2
Evaluation Tool 2 Level 3 and 4
This survey is to be completed by program participants and their managers 60 days after the program
completion
1.
Q1: Level of effort
Mark only one oval per row.
Poor Fair Satisfactory Very good Excellent
The level of effort I put in the
program is
2.
Q2: Value add on the job
Mark only one oval per row.
Poor Fair Satisfactory Very good Excellent
Level of skill/knowledge at start of
program
Level of skill/knowledge at end of
program
Contribution of program to my
skill/knowledge
The degree of impact of the
program on my day to day work
behavior
3.
Q3: Support and engagement from my manager
Mark only one oval per row.
Strongly
disagree
Disagree Neutral Agree
Strongly
agree
My Manager was supportive
throughout
My manager was available when I
needed help
My manager provided useful
feedback
My manager provided useful real
time examples
My manager provided additional
learning resources
4.
Q4: What aspects of this program were most useful or valuable?

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 161

Appendix E
Program
Specifically, the program will need to draw from the critical behaviors, and required
drivers stated in section Level 3: Behavior, as well as focus on engaging the teams to work and
plan on the implementation of the learning goals stated in the section Level 2: Learning also
described earlier in this chapter. The program will require adapting a process for the team to
embrace and implement the behaviors and learning goals using a pilot project. While the
duration of the pilot will depend on the size of the L&D team , whereby the larger the team, the
longer the pilot will likely need to be to ensure all team members are trained, for an L&D team
of 10-15 people, a six-week rigorous pilot period which begins with a full-day offsite to get the
team engaged is recommended. There are several different approaches to implement the pilot.
Two examples include 1) the agile process, which borrows techniques from software
development, and 2) the five moments of need approach, which focuses on providing learning
and tools at the exact moment of need. It is up to the L&D team to select an implementation
approach that best suits their needs, selecting from these or other approaches.
It is recommended that at the onset of the program, the L&D team is divided by into sub-
teams of 2 people or more. Each sub-team will own one of the learning goals and act as the
subject matter experts for that learning goal throughout the program. Training would be required
for each of the sub-teams to ensure they know the details, applications, tools, and metrics
required for the successful implementation of each of the Learning goals. It is recommended that
the program kicks-off with a full day offsite meeting. Such meetings help engage teams away
from the day-to-day working space and are known to increase team brainstorming and
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 162

collaboration capabilities. The activities and breakout sessions of the full-day onsite program
will need to be customized to address Level 3 behaviors and Level 2 learning goals.
Additionally, it is recommended that the program is championed by the Chief Learning
Officer and led by a team of 2 moderators, preferably external consultants, to the L&D team. The
program agenda will need to be developed and communicated to the L&D team at least two
weeks prior. A sample agenda has been developed as an example to help facilitate such program.
The sample offsite agenda lists the recommended sequence for the teams to work on the various
goals by three significant sessions listed as Project Team Session 1, 2 and 3. It is recommended
that the program starts by focusing on the critical motivation related learning goals 7, 8, 9 and
10; followed by the procedural learning goals 1, 2, 4 and 5, and closing with the self-reflection
and attribution learning goals. Depending on the size of the L&D organization, these sessions
can run linearly or concurrently. If the L&D organization is large, the three sessions can run
concurrently with the various sub-teams working on their specific learning goals. This sample
agenda is presented in Appendix E. This proposed agenda is indicative, and the timing could be
expanded to more than one full day depending on organizational resources and needs.

Full Day Offsite Proposed Agenda

8:30-9:00 am - Registration and Coffee
9:00-9:15 am - CLO Welcome and Opening Remarks
9:15-9:45 am - Program Overview and critical goals - Which Level 3 behaviors will we
implement and how?
9:45 - 10:45 am Jeopardy Game - testing Procedural Knowledge around strategic
planning, data collection, and analysis, benchmarking, and industry awards
10:45-11:00 am Break #1
11:00-12:00 am Project Teams Session 1 - Planning for Level 2 Learning goals 7, 8,9,10
and related Level 3 behaviors
Team 7 - Define how teams will work together
Team 8- Articulate how the L&D team will foster innovation
Team 9- defining goals, articulating values, and fostering innovation
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 163

Team 10- Define Values and the ‘why” of the LandD team - ensure it ties
to the overall organization

12:00-12:30 pm Team Report Out and QandA 1
12:30- 1:00 pm Lunch
1:00- 2:00 pm Project Team Session 2 -Planning for Level 2 Learning goals 1, 2, 4, 5
and related Level 3 behaviors – Each Team works concurrently and in
parallel for one hour each
Team 1 - Structure how to implement a rigorous strategic planning process
Team 2- Map how to collect meaningful data and analyze
Team 3 – Articulate how to conduct rigorous benchmarking visits
Team 4- List all the steps on how to submit well-articulated Industry
Award applications

2:00 - 2:30 pm Team Report Out and Q &A 2
2:30 - 2:45 pm Break #2
2:45- 3:45 pm Project Team Session 3- Planning for Level 2 Learning goals 3 and 6 and
related Level 3 behaviors
Team 3- Focusing on self and team reflection
Team 6 - attributing successes and failures of the day to own efforts

3:45- 4:15 pm Team Report Out 3- 3
4:15- 4:30 pm CLO Wrap Up and closing remarks
4:30 pm Adjourn

As seen in the sample program agenda, the goal of the program is to engage teams in working
together and identifying their plan of action for all ten learning goals.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 164

Appendix F
Data Collection Matrix
The matrix depicted in Table 19 below shows each of the quarterly goals for the external
and the internal results as well as corresponding columns for each quarter to report the actuals.
Similar data collection matrixes can be created for Level 3, 2 and 1.
Table 19
Data Collection Matrix
Results
1st Quarter
Goal
1st
Quarter
Actual
2nd Quarter
Goal
2nd
Quarter
Actual
3rd Quarter
Goal
3rd
Quarter
Actual
4th
Quarter
Goal
4th Quarter
Actual
External Results
Increase
Industry
recognition
for the
alignment of
learning to
business
performance
results
1 award
application
submitted
1 award
application
submitted
1 award
application
submitted
1 award
application
submitted

Improved
business
services and
products
80 percent
5-star rating
Customer
ratings on
social media
85 percent
5-star rating
Customer
ratings on
social media
90 percent
5-star rating
Customer
ratings on
social media
95 percent
5-star rating
Customer
ratings on
social media

Renewed
accreditation
as a corporate
university
Reduce the
number of
discrepancies
in the
accreditation
review
Accreditation
Organization
report

Internal Results
All learning
modules
designed
internally to
align with the
CEO agenda
70 percent of
learning
modules
aligning with
CEO agenda
80 percent of
learning
modules
aligning with
CEO agenda
90 percent of
learning
modules
aligning with
CEO agenda
100 percent
of learning
modules
aligning with
CEO agenda

Improved
business
performance
results
achieved by
employees
Employee
surveys 60
days after
training
completion
Employee
surveys 60
days after
training
completion
Employee
surveys 60
days after
training
completion
Employee
surveys 60
days after
training
completion

Improved
benchmarking
process
applications
within the
organization
2
benchmarking
visits
2
benchmarking
visits
2
benchmarking
visits
2
benchmarking
visits

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 165

Appendix G
Evaluation Tools
Level 1 Evaluation Tools

As seen in Table 15, two different tools are recommended to be used to evaluate Level 1
engagement, relevance, and customer satisfaction. First, Twitter posts can be used to evaluate
engagement and customer satisfaction. The number of tweets and retweets is used in data
analytics as an indicator of engagement. The content of the tweets can be text-mined to reveal
levels of customer satisfaction. Second, the on-the-spot pulse check of the employees using on
online poll service reaching the employees on their smartphones can reveal program relevance.
This type of poll will include one question such as: “How engaging is the program” with a 5
level Likert scale: not at all, somewhat engagement, neutral, quite engaging, and very engaging.
As seen in Table 15, both tools can be used immediately before, during and immediately after
the completion of the program.
Additionally, after the 60-day mark, the program organizers can review data analytics
resulting from the CLO Blog such as post likes, reposts, and shares; they can review the data
analytics resulting from the infographic such as post likes, reposts, and shares; finally, they can
review the data analytics from the webcast including number of people attending live, number of
people who viewed the recording, number of likes, shares, and repostings. The benefit of such
tools the zero-cost associated with their use, as they are free, and the robust analytics they deliver
to the user immediately and throughout the communication campaign including daily, weekly,
and monthly.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 166

Level 2 Evaluation - Tools, during and after the program completion
For Level 2 evaluation, the proposed tools include conducting: live Facebook sessions during
the program; conducting a 5-question survey immediately after the program completion, and
small group discussions with employees and their managers to discuss if and how the new
learning has changed the way they do their job as a result of the program. The managers play a
critical role here during and immediately after the program, whereby they will need to observe,
track and be ready to report on behavioral changes and improved work product resulting from
the program. Concerning metrics, the Facebook live sessions, employees can share what they are
learning during the program. The number of views, likes, and shares to the video resulting from
the Facebook live session will be an indication of both learning (based on the content of the
participant comments) and of engagement. The 5-question survey immediately after the program
can also capture Level 1 and Level 2 data from employees. Finally, the small group discussions
with employees and their managers to discuss if and how the new learning has changed the way
they do their job as a result of the program can be recorded and analyzed qualitatively for
overarching themes, possible concerns and/or benefits.
Level 3 Evaluation - 60 days and 90 days after the program
For Level 3 evaluation, two tools are recommended. An anonymous survey with five
questions of employees and their managers conducted 60 days after the program. The survey
results can be tabulated and analyzed to glean critical messages about changes in the behavior of
the employees on the job 60 days after program completion. Additionally, small group
discussions with employees and their managers to discuss if and how behaviors have changed on
the job as a result of the program can complement the quantitative results of the survey. During
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 167

these small group discussions, employees and their managers can review artifacts and products
they developed as a result of the program.
Level 4 Evaluation - 90 days after the program

For Level 4 evaluation it is recommended that all results from the two surveys results, as
well as all the data analytics, are tabulated, analyzed, and presented to the employees and their
managers in small group discussions about the business performance results they have achieved
as a result of the program.
All results can be collected in a data collection matrix and analyzed and reported in a
dashboard presented in the next section.

ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 168

Appendix H
Dashboards
The dashboards below is indicative and depict the external results (Figure 7) and the internal
results (Figure 8) respectively for Quarter 1. As the quarters progress, additional pie charts
dashboards can be created to allow for the visualization of the progress in the implementation
from one quarter to the next and finally for the overall annual performance. The percentages
show actual goal implementation which can be compared the quarterly goal for management and
reporting.

Figure 7. External results.
ALIGNING LEARNING TO BUSINESS PERFORMANCE RESULTS 169

Figure 8. Internal results.
Dashboards are useful for visualizing data and getting a different perspective on the progress or
lack thereof for goal implementation. While a pie chart is recommended, other types of charts
can be used, such as a bar chart, to depict the goal and program progress in successfully aligning
learning to business performance results.

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Knowledge, motivation and organizational influences impacting recruiting practices addressing the gender gap in the technology industry: an evaluation study

PDF

An evaluation of project based learning implementation in STEM

PDF

Analyzing the implementation of a learning management system into a post-merger & acquisition organization and its effects on sales performance (improvement model)

PDF

Social media and the decline of U.S. newspapers: the role of learning in digital media transformation

PDF

Prior learning assessment portfolios: an evaluation study

PDF

Organizational agility and agile development methods: an evaluation study

PDF

Knowledge, motivation, and organizational influences within leadership development: a study of a business unit in a prominent technology company

PDF

Reducing the environmental impact of mining - a promising practice study

PDF

Improving instructor skills (IIS): a Needs analysis

PDF

Completion in online learning: graduate students' perspectives

PDF

The successful implementation of diversity and inclusion efforts: a study of promising practice

PDF

Developing socially intelligent leaders through field education: an evaluation study of behavioral competency education methods

PDF

Development of employee well-being initiatives to improve engagement and performance: an innovative study

PDF

Modern corporate learning requires a modern design methodology: an innovation study

PDF

Improving pilot training by learning about learning: an innovation study

PDF

The role of professional development and certification in technology worker turnover: An evaluation study

PDF

Social work faculty practices in writing instruction: an exploratory study

PDF

Applying best practices to optimize racial and ethnic diversity on nonprofit boards: an improvement study

PDF

Financial stability and sustainability in online education: a study of promising practice

PDF

Building 21st century skills for school-age children in Colombia: lessons from a promising practice

Asset Metadata

Creator Theodotou, Marina A. (author)

Core Title Knowledge, motivation, and organizational influences in aligning learning to business performance results: a study of a promising practice

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Publication Date 04/26/2018

Defense Date 03/09/2018

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag best practices,business performance results,Defense Acquisition University,Knowledge,learning alignment,learning and development,Motivation,OAI-PMH Harvest,organization

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Hirabayashi, Kimberly (committee chair), Krop, Cathy Sloane (committee member), Murphy, Don (committee member)

Creator Email marinatheodotou1@gmail.com,theodoto@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-497439

Unique identifier UC11268172

Identifier etd-TheodotouM-6280.pdf (filename),usctheses-c40-497439 (legacy record id)

Legacy Identifier etd-TheodotouM-6280.pdf

Dmrecord 497439

Document Type Dissertation

Format application/pdf (imt)

Rights Theodotou, Marina A.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA