University of Southern California Dissertations and Theses

Modern corporate learning requires a modern design methodology: an innovation study

(USC Thesis Other)

Modern corporate learning requires a modern design methodology: an innovation study

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: MODERN LEARNING REQUIRES MODERN DESIGN 1

Modern Corporate Learning Requires a Modern Design Methodology: An Innovation Study

by

Summer Rivers Salomonsen

An Executive Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2019

Copyright 2019 Summer Rivers Salomonsen
MODERN LEARNING REQUIRES MODERN DESIGN 2
Dedication

Babe.
This has been one wild ride.
For your constant support, unfailing encouragement, and profound love; I thank you.
You are the blessing of my life.
MODERN LEARNING REQUIRES MODERN DESIGN 3
Acknowledgements
Dr. Monique Datta
Dr. Eric Canny
Dr. Lawrence Picus
------------------------
Ms. Anna Chiang
Dr. Jaimie Hoffman
Dr. Sarah Lillo
Mr. James McGee
Dr. Alison Muraszewski
Ms. Jasmine Nichols
------------------------
The amazing Content team at Grow Learning

MODERN LEARNING REQUIRES MODERN DESIGN 4
Table of Contents
Acknowledgements .....................................................................................................................3
Abstract ......................................................................................................................................6
Introduction to the Problem of Practice .......................................................................................7
Organizational Context and Mission ............................................................................................7
Importance of Addressing the Problem ........................................................................................8
Organizational Performance Goal................................................................................................9
Stakeholder Group of Focus and Stakeholder Goal .................................................................... 10
Purpose of the Project and Questions......................................................................................... 11
Methodological Approach ......................................................................................................... 12
Review of the Literature ............................................................................................................ 12
Content Team Knowledge, Motivation, and Organizational Influences ...................................... 16
Conceptual Framework: The Interaction of Knowledge, Motivation, and Organization ............. 27
Data Collection and Instrumentation ......................................................................................... 31
Results ...................................................................................................................................... 34
Findings .................................................................................................................................... 36
Recommendations for Practice .................................................................................................. 49
Conclusion ................................................................................................................................ 55
References ................................................................................................................................ 58
Appendix A: Participating Stakeholders with Sampling Criteria ................................................ 63
Appendix B: Interview Protocol ................................................................................................ 65
Appendix C: Credibility and Trustworthiness ............................................................................ 66
Appendix D: Ethics ................................................................................................................... 68
Appendix E: Integrated Implementation and Evaluation Plan .................................................... 70
MODERN LEARNING REQUIRES MODERN DESIGN 5
Appendix F: Program Evaluation Survey .................................................................................. 79
List of Tables
Table 1: Organizational Mission, Global Goal and Stakeholder Performance Goal .................... 10
Table 2: Four Values of the Agile Manifesto ............................................................................. 15
Table 3: Assumed Knowledge Influences and Types ................................................................. 19
Table 4: Assumed Motivation Influences .................................................................................. 23
Table 5: Assumed Organizational Influences ............................................................................. 27
Table 6: Critical Documents for Analysis .................................................................................. 34
Table 7: Summary of Motivation Influences and Recommendations.......................................... 50
Table 8: Summary of Organizational Influences and Recommendations .................................... 53
Table 9: Outcomes, Metrics, and Methods for External and Internal Outcomes ......................... 71
Table 10: Critical Behaviors, Metrics, Methods, and Timing for Evaluation .............................. 72
Table 11: Required Drivers to Support Critical Behaviors ......................................................... 73
Table 12: Evaluation of the Components of Learning for the Recommended Solutions ............. 76
Table 13: Components to Measure Reactions to the Program .................................................... 78
List of Figures
Figure 1. ADDIE Model ............................................................................................................ 14
Figure 2. Pre-Study State Interaction of Content Team’s Knowledge and Motivation within
Grow Learning .......................................................................................................................... 29
Figure 3. Future State of Content Team’s Knowledge and Motivation within Grow Learning.... 30

MODERN LEARNING REQUIRES MODERN DESIGN 6
Abstract
To keep pace with the rapidly changing workplace, corporate learning practitioners must break
with outdated models to innovate agile, responsive, and iterative design methodologies. Too
often, the creation of learning content is bound by antiquated, waterfall methods that protract
content development timelines, limit opportunities for innovation, and result in stale and
ineffective training. This study centered on the journey of the Content team at Grow Learning
1
, a
learning technology company, as they attempted to innovate their content creation process to
achieve their performance goal of building 1,000 Microlearning® lessons over the course of
2018. The study aligned to the Clark and Estes (2008) gap analytical framework to illuminate the
knowledge, motivation, and organizational influences impacting the team. Research questions
centered on: revealing stakeholder knowledge and motivation related to the goal, identifying the
interaction between organizational culture and team knowledge and motivation, and identifying
any resulting solutions. A qualitative inquiry method was employed, which included eight, one-
time interviews with members of the stakeholder group. While knowledge influences were not
validated in the study, significant motivation findings included the team’s profound reliance on a
blend of collaboration and autonomy to reach their goals, in addition to strong evidence that
psychological safety was a catalyst for the team to innovate on the established content creation
process. Recommendations were strategically crafted to dovetail within the organization’s
processes and procedures, including reinforcement at weekly 1:1s, team meetings, and all
working sessions. This ensured that recommended supports would both validate and perpetuate
the innovated practices developed by the team while also reinforcing the collaboration, trust, and
psychological safety so critical to the team’s success.

1
A pseudonym
MODERN LEARNING REQUIRES MODERN DESIGN 7
Introduction to the Problem of Practice
Companies of all sizes recognize that the most efficient and expedient way to deliver on-
demand learning to a modern corporate audience is through learning technology, but they have
not yet identified the best way to design and deploy that learning (The National Academies,
2012). Deloitte University Press (2016) reports that 84% of global executives identify their
corporate learning initiatives as important or very important when defining corporate strategic
goals. This prioritization is also reflected in the five-year trend of U.S. corporations steadily
increasing training technology expenditures, with consistent year-over-year gains, the most
recent data from 2014 showing a 10% increase (Bersin & Associates, 2015). Aligning with a
fast-paced business environment, organizations have widely accepted virtual learning technology
as a legitimate delivery system, offering the opportunity to scale learning across large and
geographically dispersed audiences rapidly and inexpensively (Brown, 2001). Yet, despite the
rapid adoption and subsequent growth of computer-based learning opportunities across
industries, research has yet to identify those factors that influence its successful design,
deployment, and ultimate acceptance among virtual learners (Aguinis & Kraiger, 2009).
Corporate learning that is not designed for the modern corporate learner, using modern learning
approaches, will adversely affect companies by further widening the talent gap, ill-preparing
employees for the realities of their jobs, and short-changing the next generation of corporate
leaders (Oxford Economics, 2015).
Organizational Context and Mission
Grow Learning is a learning technology company based dually in San Francisco, CA and
Manhattan, NY. Founded in 2010, Grow Learning’s Microlearning® approach makes it quick
and easy for employees to learn new skills through multi-modal, short-form content. Grow
MODERN LEARNING REQUIRES MODERN DESIGN 8
Learning offers a robust library of ~2,500 Microlearning® lessons housed in a modern
Microlearning® platform to nearly 400 customers across multiple industries. Though the early
years of Grow Learning’s inception marked a period of significant market success, the company
was not able to sustain its early momentum, and in early 2017 the board appointed a new CEO.
Thus, ensued a period of dramatic change, in which the workforce transitioned from 250 to 75
employees, content creation was prioritized over platform development, and a new executive
team was hired. The corporate rebirth of Grow Learning resulted in a massive cultural shift as
the remaining employees adopted a “start-up” mentality, moving away from the constraints of
“established company.” The new mission of Grow Learning is to make Microlearning® a core
component of every organization’s learning strategy. Grow Learning is now represented by six
teams, including: Content, Sales, Customer Success, Product, Marketing, and Engineering.
Despite these changes, and as a testament to its leadership and innovation in Microlearning®,
Grow Learning was awarded a registered trademark on the term in October of 2017. As the
leading provider of Microlearning® lessons, Grow Learning must innovate a rapid and valid
content creation process to expedite development and maintain its high standards.
Importance of Addressing the Problem
To achieve its mission of making Microlearning® a core component of every
organization’s learning strategy, Grow Learning must innovate a new learning design process
that shortens content creation timeframes, maintains content credibility, and aligns to the
company’s Microlearning® framework. In Grow Learning’s seven-year history, the Content
team was never led by a learning practitioner, which, over time, exposed deficits in the quality of
the created content. This deficit catalyzed the need for the new CEO to hire a Chief Learning
Officer (CLO) to lead the Content team and establish both internal and external learning
MODERN LEARNING REQUIRES MODERN DESIGN 9
strategies for the company. It is important for Grow Learning to analyze and iterate on its
established content creation process to rapidly produce content aligned with the expectations of
its customer-base and its shareholders. Principally, the creation of agile-like design
methodologies will set the company apart from its competitors, establishing Grow Learning as an
innovator in the learning technology industry and subsequently increasing its market-share. If
Grow Learning is unsuccessful in meeting its goal, it risks losing current and future customers,
the effect of which will be compounded by negative word-of-mouth, low NPS ratings, and loss
of repeat and new business.
Organizational Performance Goal
Grow Learning’s expectation is that by year end 2018 the Content team will create 1,000
new lessons utilizing a modern learning design approach. The innovated content creation process
will expedite content creation timelines, minimize resourcing requirement, and quicken the time
to deliver a minimal viable product (MVP). The CLO established this goal in December 2017 as
part of her presentation to the board. The achievement of this goal will be measured through
coordination with Content Leads, team discussion and review, and the establishment of a formal
feedback and review process. As the recognized leader of Microlearning® content, it is critical
for the organization to pioneer the latest learning development approaches as integral to the
creation of its content. Clients expect learning programs to adhere to modern learning principles,
but more importantly, they expect the development and delivery of such programs to also adhere
to these principles. It is critical that Grow Learning design learning programs in the most
expeditious way to meet its performance goal. If Grow Learning is unsuccessful, the
organization will be unable to maintain appropriate margins, will over-extend a limited content
staff, will be unable to retain current customers, and will experience difficulty in recruiting
MODERN LEARNING REQUIRES MODERN DESIGN 10
prospects. Table 1 details the organizational mission, organizational performance goal, and
stakeholder goal.
Table 1
Organizational Mission, Global Goal and Stakeholder Performance Goal
Organizational Mission
Make Microlearning® a core component of every organization’s learning strategy
Organizational Performance Goal
By year end 2018, 1,000 new lessons will be created utilizing a modern learning design approach
Content Team Goal
By year end 2018, the Content team will innovate a content design process that shortens creation
timeframes while maintaining credibility and alignment to the Microlearning® framework.
Stakeholder Group of Focus and Stakeholder Goal
Grow Learning’s ability to create 1,000 new lessons by year end 2018 utilizing a modern
design approach required a combined effort by all stakeholder groups, but the Content team was
uniquely suited to influence the successful outcome of this goal. The Content team is comprised
of three sub-groups, including: Strategy, Learning Experience, and Production. The Strategy
team, led by the Director of Content Strategy is responsible for setting the Content Roadmap,
maintaining the credibility and voice of the library, and analyzing all lesson data. The Learning
Experience Design team, led by the Director of Learning Experience, is comprised of five
Content Producers, whose job it is to create new Microlearning® lessons, based on the
determined Content Roadmap set by Strategy team. All lesson production is captured in house
and managed by the Production team, who dually support both Content and Creative
Departments. This group, led by the Director of Photography, are responsible for all video pre-
MODERN LEARNING REQUIRES MODERN DESIGN 11
production, film shoot, and post-production. Additionally, the Production team support Content
Producers in capturing all audio, animation, and offer design support for lessons. The Content
team, represented by roughly 13 individuals, were primarily responsible for innovating a new
approach that both expedited and modernized the content creation process. The CLO established
this goal and it was reviewed bi-weekly, when the heads of Strategy, LED, and Production meet
to discuss the content creation process. The Content team’s ability to reach this goal would result
in an expedited timeline to new content, improved content refresh cycles, quicker hand-offs
between Content teams, ultimately empowering the Content team to scale production. Further,
applying modern design will better result in modern learning.
Purpose of the Project and Questions
The purpose of this project was to conduct a needs’ analysis in the areas of knowledge
and skill, motivation, and organizational resources necessary to reach the organizational
performance goal of creating 1,000 new lessons by year end 2018 utilizing a modern learning
design. The analysis began by generating a list of possible needs and then moved to examining
these systematically to focus on actual or validated needs. While a complete needs’ analysis
would focus on all stakeholders, for practical purposes the stakeholder focused on in this analysis
was the Content team.
Therefore, the questions that guided this study were:
1. What is the stakeholder knowledge and motivation related to the creation of 1,000 new
lessons utilizing a modern learning design?
2. What is the interaction between organizational culture and context and stakeholder
knowledge and motivation?
3. What are the recommended knowledge, motivation, and organizational solutions?
MODERN LEARNING REQUIRES MODERN DESIGN 12
Methodological Approach
The gap analytic framework as defined by Clark and Estes (2008) formed the basis of the
needs analysis conducted for this study. This framework attempts to identify and define the gap
between actual and desired organizational performance by studying the knowledge, motivation,
and organizational assets that contribute to individual employee performance.
Review of the Literature
To compete in fast-paced, global markets, companies are prioritizing training as a
mechanism to benefit the individual employee, team, and corporation (Aguinis & Kraiger, 2009).
Further, companies are investing heavily in learning as part of their strategic goals and
objectives, with investment in leadership training and learning technology the biggest source of
spending within the category (Bersin, 2014; Carroll, Singaraju, & Park, 2015). The year 2015
marked a five-year trend increasing training expenditures in U.S. companies, with consistent
year-over-year gains since 2011 (Carroll et al., 2015).
Learning technology enables companies, both large and small, to deliver standardized,
on-demand, and globally-scaled training simply and efficiently (Schwartz et al., 2016; The
National Academies, 2012). As the adoption of learning technology becomes more prevalent
across industries, it has become recognized as a legitimate training methodology (Aguinis &
Kraiger, 2009; Brown, 2001). This prioritization and wide acceptance of corporate learning
conducted through technology has catalyzed the need for further research on factors that
influence its successful deployment, specifically design and development processes (Arthur,
Bennett, Edens, & Bell, 2003).
Though initially questioned as a viable medium for instruction, learning deployed
through learning technology has been consistently proven to be as effective, and in some cases
MODERN LEARNING REQUIRES MODERN DESIGN 13
more effective, than traditional, in-person training (Sitzmann, Kraiger, Stewart, & Wisher, 2006).
Sitzmann et al. (2006) also found that web-based instruction was 6% more effective for teaching
declarative knowledge skills, equally as effective for teaching procedural knowledge, and
resulted in equally satisfied participants. Bell and Kozlowski (2002) contend that learning
technology that enhances both perceived and actual learner control over the experience will
greatly enhance the effectiveness of the training. Further, Klein, Howard, Noe, Raymond, and
Wang (2006) recommend that learning outcomes are also enhanced when the virtual
environment is perceived by learners as an enabler and not a constraint.
In general, companies recognize the importance of virtual learning opportunities that are
both relevant and engaging to the learner but have not unilaterally identified the best learning
strategy that both interests and retains a high-performing workforce (Wentworth, 2016). By not
providing workplace training in an expeditious and timely way, companies are at risk of facing a
simultaneous talent and leadership gap. A 2015 study of over 6,000 global executives and
employees reveals that 51% of executives believe their leadership pool has the skills to
effectively manage talent and 34% believe their leaders are prepared to lead a diverse workforce
(Oxford Economics, 2015). In the same study, 40% of employees report concern that their job
will be obsolete in the next five years, yet only 13% have experienced meaningful professional
development through formal corporate training. By offering well-designed training that
addresses the learning needs of employees, companies will better position themselves to retain
high-performers and effectively prepare future company leaders.
The relationship between virtual training design and organizational effectiveness is
critical to the success of virtual learning and may be measured through learner reactions,
behavior change, and organizational results (Kraiger et al., 2004). Yet, learning practitioners in
MODERN LEARNING REQUIRES MODERN DESIGN 14
the corporate environment are largely unaware of the research surrounding impactful training
design and the underpinning psychological theory required to successfully create virtual learning
(Arthur et al., 2003; Kraiger et al., 2004). The success of virtual learning programs is, in part,
determined by the inclusion of relevant stakeholders, but ultimately is reliant on the chosen
design methodology (Arthur et al., 2003).
Since its inception in the 1970s, the five-stage, learning design model ADDIE, which
stands for: Analyze, Design, Develop, Implement, Evaluate, has been the most widely accepted
and utilized design approach in corporate learning (Aguinis & Kraiger, 2009). Suited to both
individualized and group instruction, the ADDIE model proposes a linear, staged learning design
process that is both time-consuming and expensive (www.trainingindustry.com).

Figure 1. ADDIE Model
Nguyen, Prasad, Alsadoon, Hoe, and Elchouemi (2016) acknowledge the long tenure of
ADDIE as a respected learning design approach, but purport that it is insufficient as a modern
learning design methodology, which requires both expedited and responsive development.
MODERN LEARNING REQUIRES MODERN DESIGN 15
Unlike the ADDIE model, modern learning design approaches align more closely with an
agile methodology, in which learning is designed in an iterative fashion, compressing stages, and
empowering designers to arrive more quickly at a minimal viable product. A common
methodology for technology development, agile was born out of the tech boom of the early
2000s. The Agile Manifesto which was the flagship document created by the Agile Alliance,
founders of the agile movement, espouses four critical values that define this revolutionary
project management methodology (www.agilemanifesto.org). Table 2 outlines each of the four
values. For the purposes of this study, the reader may substitute the term software with learning
solution(s).
Table 2
Four Values of the Agile Manifesto
Value 1 Individuals and interactions over processes and tools.
Value 2 Working software over comprehensive documentation.
Value 3 Customer collaboration over contract negotiation.
Value 4 Responding to change over following a plan.

Alqudah and Razali (2017) contend that technology companies will generally adopt an
agile methodology to ensure speed of production, minimize cost, and achieve high quality
results. However, they report that less than 50% of agile deployed projects are successful
because these companies overlook the primary metrics for success, which are: organizational
culture and the skills and competency of the primary development team. Similarly, Kochikar and
Ravindra (2007) recommend that organizations foster and develop an agile capability as part of
their established competency model to achieve both product differentiation and market
competitive advantage. Additionally, they suggest that agile design may be applied across
MODERN LEARNING REQUIRES MODERN DESIGN 16
industries as a valid and credible methodology and is an organization’s greatest defense in the
face of “rapid environmental change” (p. 129). Modern corporate learning, delivered through
technology, requires a revised, agile methodology that will expedite development timelines,
allow for flexibility of design, and improve the quality of the created learning solution.
Content Team Knowledge, Motivation, and Organizational Influences
To study stakeholder performance within an organization, Clark and Estes (2008)
propose a problem-solving framework to identify gaps through the lenses of knowledge,
motivation, and organizational influence. To understand knowledge influences, Krathwohl
(2002) proposes a revision to Bloom’s Taxonomy which includes four distinct types: factual
(e.g.: terminology), conceptual (e.g.: principles and structures), procedural (e.g.: how to do
something), and metacognitive (e.g.: strategic and self-knowledge). Clark and Estes (2008)
recognize three fundamental motivation indexes that influence the modern worker, they are:
active choice, persistence, and mental effort. Ultimately, knowledge and motivation are not
sufficient to achieve performance goals, as the organization must provide and sustain appropriate
assets and systems to support employees (Clark & Estes, 2008). The literature reviewed for this
section presents the assumed knowledge, motivation, and organizational influences that impacted
the performance goal of the Content team of Grow Learning.
Knowledge and Skills
At the center of every organizational initiative and strategy is the employee. Whether that
employee is skilled, knowledgeable, or engaged is a critical factor that may determine the
success or failure of an organization’s performance goals. Clark and Estes (2008) contend that
knowledge and skill enhancement is needed only when employees don’t know how to
accomplish their goals or when the organization’s future state requires more complex problem
MODERN LEARNING REQUIRES MODERN DESIGN 17
solving. The literature reviewed for this section responds, specifically, to the knowledge-related
influences that impacted the established stakeholder performance goal of the Content team of
Grow Learning. By the end of calendar year 2018, Grow Learning set out to create 1,000 new
lessons utilizing a modern learning design. However, this organizational goal was only attainable
if the Content team was successful in its goal of developing a new learning design approach that
both shortened the content creation timeline and delivered high-end modern learning. Mayer
(2011) purports that an employee’s ability to learn and transfer new knowledge to a goal or
assignment is beneficial both to the employee and the organization. Therefore, the Content team
must have acquired new information, aligned it with their existing understanding, and applied it
to create new processes that responded to the performance goal while also ensuring the continued
success of the company.
Knowledge influences. To successfully reach its goal, the Content team needed to
acquire new knowledge. In a revision of Bloom’s Taxonomy, Krathwohl (2002) proposes a
revised knowledge taxonomy, which includes four distinct types: factual (e.g.: terminology),
conceptual (e.g.: principles and structures), procedural (e.g.: how to do something), and
metacognitive (e.g.: strategic and self-knowledge). Mayer (2011) proposes that ultimately,
learners must possess all forms of knowledge to succeed in advanced tasks. For the purposes of
this study, the knowledge influences identified for the primary stakeholder group were
categorized into knowledge types. Specifically, the Content team must have attained mastery of
two conceptual knowledge influences to have achieved the performance goal.
The Content team must understand how learners learn in a virtual environment. The
virtual learning environment is distinct, in both experience and outcome, from in-person training,
and encompasses the myriad of ways employees may learn through use of technology, including:
MODERN LEARNING REQUIRES MODERN DESIGN 18
computers, simulators, self-directed modules, mobile devices, etc. Increasingly, companies large
and small are choosing virtual over in-person delivery to expedite training timelines, achieve
learning at scale, and offer employees a truly on-demand learning experience that transcends
traditional training (Bersin, 2016). Though the migration to virtual has increased significantly
over the past decade, research continues to be somewhat limited in ascertaining what increases or
decreases its effectiveness (Brown, 2001).
To reach its performance goal, the Content team needed to understand the impact of
virtual training and how employees learn in this new and evolving environment. Sitzmann et al.
(2006), in a landmark research study of over 19,000 trainees, found virtual training to be 6%
more effective than classroom-based teaching for declarative knowledge instruction and equally
as effective for procedural knowledge instruction. Further, Aguinis and Kraiger (2009) suggest
that virtual learning can be enhanced when learners are provided some type of adaptive guidance
to complete activities. Moreover, Bell and Kozlowski (2002) contend that in an organizational
context, the responsibility of learning has shifted from trainers to learners and the mark of
training effectiveness will be ultimately determined by learners and learner choice.
The Content team must recognize the critical components of the modern learning
design process. Since its creation in the 1970s, the five-stage ADDIE model: Analyze, Design,
Develop, Implement, Evaluate, has been the most widely accepted design approach in corporate
learning (Aguinis & Kraiger, 2009). Suited to both individualized and group instruction, it
proposes a linear, staged learning design process that is both expensive and time-consuming
(www.trainingindustry.com). Conversely, modern learning design approaches are more iterative,
and closely align to agile, appropriated from the technology industry, in which learning is
designed in an iterative fashion, compressing stages, and empowering designers to reach more
MODERN LEARNING REQUIRES MODERN DESIGN 19
quickly a minimal viable product. Aguinis and Kraiger (2009) identify a gap between applied
and scholarly literature that investigates the impact of shortened cycle time in developing and
deploying corporate learning. Specifically, they recognize that the needs of the business dictate
the learning design timeframe, and “work is needed regarding the factors that can accelerate the
realization of training” (p. 467).
Arthur et al. (2003) conducted a meta-analysis of the impact of training design on
organizational effectiveness, measuring employee reactions, learning, behavioral change, and
organizational results. The research showed a positive correlation between training and each
measure of effectiveness, but further showed that the training methodology magnified this effect.
Kraiger, McLinden, and Casper (2004) recommend that training professionals must link training
outcomes with business outcomes to be successful. Moreover, they must include strategic
stakeholders in the design process to achieve success. In short, training design is a critical
indicator of training effectiveness. Table 3 details the Content team’s assumed knowledge
influences, types, and related literature.
Table 3
Assumed Knowledge Influences and Types
Assumed Knowledge Influence Knowledge Type Related Literature
The Content team must
understand how employees learn
in a virtual learning environment.
Declarative:
Conceptual
Aguinis & Kraiger, 2009; Bell &
Kozlowski, 2002; Bersin, 2016;
Brown, 2001; Klein et al., 2006;
Shraw & Lehman, 2009; Sitzmann et
al., 2006
The Content team must recognize
the critical components of the
modern learning design process.
Declarative:
Conceptual
Aguinis & Kraiger, 2009; Arthur et
al., 2003; Brown, 2001; Klein et al.,
2006; Kraiger et al., 2004

Motivation Influences
MODERN LEARNING REQUIRES MODERN DESIGN 20
A motivated workforce can bring numerous, tangible benefits to a company. Indeed,
motivation is a critical driver of organizational success and is ultimately rooted in an employee’s
individual beliefs about his or her abilities, potential performance, co-workers, and even the
entire organization (Clark & Estes, 2008). In a learning context, motivation is responsible for
sustaining a learner through the cognitive process to acquire new skills, learn new information,
and apply new learning (Mayer, 2011). Understanding what motivates, and de-motivates,
employees is a key to achieving performance goals. Two motivation influences from expectancy
value theory have been identified for this section: utility value and cost belief.
Expectancy value theory: Utility value. Established as the result of decades-long social
and motivational research, expectancy value theory links achievement-related choices to two
factors: an individual’s decision to do the task and the value the individual attaches to the task
(Eccles, 2006). Essentially, an individual’s response to two questions: Can I do the task? and Do
I want to do the task? illustrates expectancy, a strong predictor of motivation, and value, a strong
indicator of active choice (Eccles, 2006). Wigfield and Cambria (2010) suggest that the two
constructs inherent in expectancy value theory – expectancy and value – are the most prominent
predictors of performance and choice. There are four beliefs inherent to expectancy goal theory,
including: attainment value, intrinsic value, utility value, and cost belief. Attainment value
describes the value an individual places on tasks that represent the attainment of personal identity
and preferences (Eccles, 2006). Conversely, intrinsic value refers to an individual’s internal
motivation, even joy, that is experienced while doing the task (Eccles, 2006). Utility value is an
individual’s belief around whether that task will be useful in achieving future goals. Moreover,
when employees recognize the importance of a task to meeting their goals, they are decidedly
motivated to start, endure, and complete the task (Clark & Estes, 2008). Cost belief is defined as
MODERN LEARNING REQUIRES MODERN DESIGN 21
the perceived cost of participation in the task (Eccles, 2006). For the purposes of this
dissertation, the theories of utility value and cost belief will be emphasized as critical motivation
influences.
Content team utility value. A team of 13 learning practitioners and production
specialists, the Content team is comprised of three primary teams: Strategy, Learning Experience
Design, and Production, and is led by the CLO. Perhaps the most critical motivation influence of
the Content team stakeholder group was the utility value it placed on creating new design
strategies to meet its performance goal. Colquitt, LePine, and Noe (2000) postulate that training
motivation is directly linked to an individual’s job involvement and career commitment. Further,
they clarify job involvement as the importance an individual places on his or her work, self-
image, and long-term goals. Members of the Content team may have been highly motivated
because the achievement of the goal will result in the creation of innovative work that is
important in the field of learning and development. As learning professionals, it is critical they
recognize the utility value in creating a new approach to solve emerging problems in the
industry.
The Content team’s success in reaching its performance goal also depended on the
support (e.g.: budgeted hours, resourcing, and collaborative work time) it received from Grow
Learning. Borgogni, Russo, and Latham (2011) contend that an employee’s perceived support of
the organization correlates positively to his or her attachment to the company. As Grow Learning
demonstrated its support of the Content team in achieving its performance goal, the Content team
would become more motivated to reach the goal. Further, Klein, Howard, Noe, Raymond, and
Wang (2006) purport that perceived barriers may affect performance by adversely impacting
motivation itself. Simply, the Content team would be less likely to achieve this performance goal
MODERN LEARNING REQUIRES MODERN DESIGN 22
if they believed their actions would be impeded and more likely to achieve this goal if they
believed they would be supported. Essentially, the Content team’s ability to meet its performance
goal would be determined, in part, by the tangible support of leadership. Klein et al. (2006)
further suggest that an individual’s attitude towards technology is influenced by its perceived
utility, usefulness, and availability. Because Grow Learning is a learning technology company,
many of the design strategies proposed by the Content team were appropriated from the
technology field. In designing agile approaches in response to the performance goal, the Content
team must recognize both the value of the approach and the value it brings to customers.
Expectancy value theory: Cost belief. One of four foundational beliefs that constitute
the expectancy value theory, cost belief is defined as the set of beliefs that an individual uses to
determine the cost of participating in an activity (Eccles, 2006). Many factors influence cost
belief, but most importantly it may be determined by an individual’s fear of failure and social
consequences, self-worth, and the time taken from other (more enjoyable) activities (Eccles,
2006).
Content team cost belief. The Content team must quantify and communicate the impact
to Grow Learning of not utilizing modern design approaches when creating virtual learning. The
quantification of this impact empowered the Content team to educate and equip colleagues and
customers in choosing modern learning that has been modernly designed. Wigfield and Cambria
(2010) suggest that the emphasis of cost is especially critical in issues of choice, because choice
includes both positive and negative task associations. The Content team must articulate the tasks
associated with choosing a modern design strategy and how those tasks differ from the tasks
associated with declining a modern design strategy. By assigning a cost value to these tasks, the
Content team would be more successful in innovating an appropriate solution.
MODERN LEARNING REQUIRES MODERN DESIGN 23
Aguinis and Kraiger (2009) contend that corporate training can lead to substantial
benefits to individuals, teams, and organizations. Further, their research shows that the benefits
of training are maximized when companies pay close attention to the training design process,
which may include adopting new strategies or forgoing old strategies. Heimbeck, Frese,
Sonnentag, and Keith (2003) conducted a research study on the use of error training in place of
the traditional correct-method only approach. The study followed two groups of undergraduate
students as they learned new spreadsheet applications. The group that was given the opportunity
to make errors, followed by guidance to correct the error, outperformed the group that was only
taught the correct way. This study demonstrates the value of an iterative design process, where
learning professionals are empowered to test designs, collaborate, and refine during the design
process. Traditional learning design promotes a linear and gated approach and once a stage has
passed (e.g.: the Analyze stage of ADDIE), there is no revision. Conversely, an agile design
approach promotes a circular flow, wherein aspects of the training may be revisited for correct
analysis as other portions are built out. This approach leads to a more robust and dynamic
learning experience, which shortens traditional content creation timelines and decreases
employee working hours per lesson. Tables 4 details the assumed motivation influences that
impacted the Content team’s goal and related literature.
Table 4
Assumed Motivation Influences
Assumed Motivation Influences Related Literature
Utility
Value
The Content team must recognize the
importance of creating new learning
approaches to meet performance
goals.
Arthur et al., 2003; Borgogni, Russo, &
Latham, 2011; Colquitt, LePine, &
Noe, 2000; Klein et al., 2006; Kraiger
et al., 2004; Sitzmann et al., 2006
MODERN LEARNING REQUIRES MODERN DESIGN 24
Cost Belief The Content team needs to
understand how a modern design
approach is more valuable both to
Grow Learning and its customers.
Recognizing that the pain of same is
greater than the pain of change.
Aguinis & Kraiger, 2009; Arthur et al.,
2003; Bell & Kozlowski, 2002; Carroll
et al., 2015; Heimbeck et al. 2003; The
National Academies, 2012; Wigfield &
Cambria, 2010

Organizational Influences
The final component of the Clark and Estes (2008) model identifies organizational assets
as a critical influence on a stakeholder group’s ability to achieve performance goals. The topic of
organizational culture is trending in current research, with significant focus on the distinctions
between cultural models and cultural settings. Gallimore and Goldenberg (2001) define cultural
models as the shared understanding of an organization. In other words, cultural models are
interpreted through organizational values, group meaning-making, shared experiences, and
reactions to change. Cultural settings are seen in the daily interactions between employees and
the process of how work is accomplished. Clark and Estes (2008) define culture in the broadest
terms, identifying work process and material resources as necessary components that must be
measured to complete a gap analysis. Further, they recommend that to best diagnose
performance gaps, organizational culture must be assessed and understood as it impacts
stakeholders individually, in groups, and as an organization.
The following section identifies research related to organizational influences as it pertains
to the ability of the Content team of Grow Learning to accomplish their performance goal of
developing 1,000 new lessons following a modern learning design by year-end 2018.
Specifically, this review emphasizes two possible primary organizational influences: a clearly
defined organizational strategy and the provision of training.
MODERN LEARNING REQUIRES MODERN DESIGN 25
Clearly defined organizational strategy. Rath and Conchie (2009) assert that successful
leadership is evident in the uniting of followers towards a clearly defined objective. As such, the
Content team will only be successful if Grow Learning established a clear organizational strategy
to embrace iterative, agile-like methodologies in place of the current content creation process.
While typical studies have focused on the leader in isolation, Rath and Conchie (2009) propose
that studying a leader in context of her/his followers may offer a more complete picture of
leadership. A critical factor that has contributed to the success of the team has been its ability to
collaborate and to disseminate new learning in a meaningful way. Members of the Content team
are budding leaders within Grow Learning and engaged employees within their organizational
structure. However, to be successful in their goal of creating new design approaches, the Content
team required a clearly defined organizational strategy that formalizes the approach.
Schwandt and Marquardt (1999) recommend a hands-on approach when organizations
adopt a new learning strategy. Specifically, they advocate for organizations to “wrestle” with
internal process to best determine the strategy that will close gaps and support stakeholders in
achieving their performance goals (p. 228). Grow Learning must adopt a similar approach as it
deploys a revised methodology to expedite its content creation process. This shift in strategy
required a significant internal reckoning of current process in addition to external facing
measures, in which customers are apprised of the enhancements to the Grow Learning library of
content.
Finally, the Content team’s performance goal is closely linked with the success of Grow
Learning as a company. As a company seeking to maintain its position within the learning
technology industry, Grow Learning placed the highest value on customer success, prioritizing
Net Promoter Scores (NPS), word-of-mouth, and customer satisfaction as evidence of its market
MODERN LEARNING REQUIRES MODERN DESIGN 26
growth. Boswell (2006) highlights the critical importance of establishing a connection between
the individual employee and organizational strategy. The alignment between these two entities
functions for the employee as a “line of sight” into the organization’s strategy, which in turn
supports the employee in knowing how to support the initiative. (p. 1490). To support the
Content team, Grow Learning must clearly define its strategy to innovate its content creation
process and offer visibility into the deployment process.
Provision of training. In a comprehensive literature review of training research, Aguinis
and Kraiger (2009) found that training, when conducted by an organization, can benefit both the
company and the individual. Once Grow Learning has established a clear organizational strategy
aligned with the performance goal, they must provide training to the Content team focused on
agile design.
In the seminal article on establishing a learning culture, Senge (1990) makes the case for
learning as the natural pre-cursor to performance. As true today as it was in the 1990s, the pace
of business is accelerating quickly. The establishment of a learning organization is a natural
response to the need for employees to adapt quickly to changing environments. However,
organizations can struggle to identify the right leadership skillsets that are aligned to a learning
organization. Typically, leaders in learning organizations are skilled at creating a shared vision,
challenging assumptions, and can build systems of trial and error to propel innovation (Senge,
1990). Pirjol and Maxim (2012) suggest that successful companies prioritize the impact to the
employee over other factors when deciding upon strategic initiatives. Specifically, they identify
professional training as a critical process that influences employees on a personal level. Grow
Learning is not just a learning technology company, it aspires to become a true learning
organization. Grow Learning values iterative design for both platform and content creation, but
MODERN LEARNING REQUIRES MODERN DESIGN 27
principally it recognizes the need to offer key stakeholder groups relevant training to drive
organizational strategy.
Kraiger et al. (2004) promote the idea that the only way to increase the adoption and
success of training in an organization is by connecting training outcomes to business goals. They
further recommend that this connection must be established at the outset by both the organization
and the impacted stakeholder group. Arthur et al. (2003) show that training effectiveness is
linked to methodology, skill to be trained, and evaluation criteria. To be successful in its
deployment of agile design training, Grow Learning must include the Content team in the
creation, deployment, and assessment of the training plan. Table 5 provides an overview of the
assumed organizational influences and related literature.
Table 5
Assumed Organizational Influences
Assumed Organizational Influences Related Literature
The organization needs to embrace an agile
methodology as its content creation strategy.
Aguinis & Kraiger, 2009; Boswell, 2006;
Kraiger et al., 2004; Rath & Conchie, 2009;
Schwandt & Marquardt, 1999
The organization needs to provide training to
the Content Team in agile design
methodology.
Aguinis & Kraiger, 2009; Arthur et al., 2003;
Kraiger et al., 2004; Pirjol and Maxim, 2012;
Senge, 1990

Conceptual Framework: The Interaction of Knowledge, Motivation, and Organization
The conceptual framework presents the factors, concepts, and variables to be studied,
emphasizing the presumed relationship between each as both a visual model and a written
narrative (Maxwell, 2013). The conceptual framework clarifies the researcher’s initial
assumptions and approach to the study, rooted in a research question or questions, and identifies
four critical elements needed to construct the conceptual framework, including: experiential
MODERN LEARNING REQUIRES MODERN DESIGN 28
knowledge, existing theory and research, pilot and exploratory research, and thought experiments
(Maxwell, 2013). Merriam and Tisdell (2016) propose that the conceptual, or theoretical,
framework provides the scaffolding of the research study and comprises the researcher’s unique
point of view. Essentially, the conceptual framework acts as a lens through which to study
various phenomena (Anfara & Mertz, 2015).
Creswell (2014) proposes that a researcher’s worldview will inherently influence the
research study. As a critical component of the conceptual framework, philosophical assumptions
must be acknowledged and identified to correctly position the approach of the study. This
researcher holds a pragmatic view of the challenge to innovate agile learning design models. A
hallmark of the pragmatist worldview is the prioritization of “what works” over “what is right”
(Creswell, 2014, p. 10). Essentially, the pragmatist values the practical solution, emphasizing
problem solving through research.
To this point, knowledge, motivation, and organization have been presented as individual
elements of the study. However, it is the interaction of these influences that will be researched to
illuminate the ways in which members of the Content team may innovate new, agile learning
design approaches. Clark and Estes (2008) contend that a gap analysis is only complete when it
includes a study of the three influences both individually and interconnected. The figure
presented below describes the pre-study, prior state interaction between the knowledge and
motivation of members of the Content team and the organization in which they work.
MODERN LEARNING REQUIRES MODERN DESIGN 29

Figure 2. Pre-Study State Interaction of Content Team’s Knowledge and Motivation within Grow
Learning
Figure 2 represents the continual, almost chaotic, interaction between the stakeholder
group and the organization. This relationship was defined as one of mutual influence; each
dependent upon the success of the other. The green circles represent the separate influences of
knowledge and motivation on the stakeholder group. Members of the Content team struggled to
align their understanding of the tenants of modern learning design (K) with their individual and
group believe in its value to customers (M). The knowledge influences represented here included
the Content team’s understanding of the influence of technology in the learning & development
industry (Carroll et al., 2015; Schwartz et al., 2016; The National Academies, 2012; Wentworth,
2016), the knowledge of how to create pedagogically sound virtual learning (Aguinis & Kraiger,
2009; Bell & Kozlowski, 2002; Klein et al., 2006; Shraw & Lehman, 2009; Sitzmann et al.,
2006), and the understanding of ways to utilize modern learning design approaches when
creating virtual learning (Arthur et al., 2003; Brown, 2001; Klein et al., 2006). The motivation
influences of the Content team members centered on their willingness to embrace agile
methodologies (Arthur et al., 2003; Bell & Kozlowski, 2002; Carroll et al., 2015; The National
MODERN LEARNING REQUIRES MODERN DESIGN 30
Academies, 2012), their belief that virtual learning requires a modern design approach (Arthur et
al., 2003; Kraiger et al., 2004; Sitzmann et al., 2006), and their openness in including customer
stakeholders in the design process (Arthur et al., 2003; Kraiger et al., 2004)
Though the stakeholder group was functionally positioned within the organizational
context, as depicted by the green circles (Content team) within the blue circle (Grow Learning),
these two entities were engaged in a continual process of mutual refinement, as depicted by the
multiple black arrows originating from both the green circles and blue circle. The organizational
influences centered on Grow Learning’s commitment to adopt the agile design approaches
recommended by the Content team (Aguinis & Kraiger, 2009; Kraiger et al., 2004) and the
company’s ability to provide requisite training to the Content team on agile design processes
(Aguinis & Kraiger, 2009; Kraiger et al., 2004).

Figure 3. Future State of Content Team’s Knowledge and Motivation within Grow Learning
MODERN LEARNING REQUIRES MODERN DESIGN 31
Figure 3 represents the desired future state of the Content team’s knowledge and
motivation within Grow Learning. As a pseudo start-up organization, Grow Learning values the
innovation of its stakeholder group and recognizes that the surest way to innovation is through
mutual collaboration and refinement. As the organization struggles to redefine its go-to-market
strategy and position itself strongly among other technology companies, the Content team is
engaged in a process of re-envisioning its content strategies to expedite the creation process and
modernize its learning design approaches. Ultimately, the knowledge and motivation of the
Content team should be a co-occurring process (single green circle) that is positioned within the
organization (blue circle) as both a connected part of the whole, while still engaging in a process
of refinement and iteration (depicted by the two black arrows). This conceptual framework and
representative figures propose that the Content team will be successful in its goal of creating and
implementing modern design approaches if it is able to maintain a consistent and controlled flow
of feedback from and influence on its surrounding organization.
Data Collection and Instrumentation
Qualitative research seeks to uncover the meaning that people ascribe to events,
experiences, and processes (Merriam & Tisdell, 2016). It emphasizes rich data collected in the
form of words that enhance understanding of the phenomenon (Merriam & Tisdell, 2016).
Malloy (2011) suggests that the process of inquiry must be “demystified” so organizations can
build internal capacity for understanding, conducting, and responding to research (p. 4). In the
fast-paced industry of learning technology, practitioners rely heavily on quantitative data, often
at the expense of qualitative findings. The servers of Grow Learning were replete with
quantitative data, including thousands of data points comprised of web metrics, user clicks,
platform navigation practices, helpdesk submissions, feature requests, and run-time information.
MODERN LEARNING REQUIRES MODERN DESIGN 32
However, this quantitative data lacked voice, and required a qualitative approach to tell the story
of its content, and how this content is created. McEwan and McEwan (2003) suggest that user-
driven research should seek to answer causal, process, and cost questions. Accordingly, this
study deployed a qualitative methodology to illuminate the available quantitative data.
This study utilized interviews and document analysis to better understand the knowledge
and motivation of the members of the Content team as it pertained to their ability to create 1,000
new Microlearning® lessons by year end 2018. The interviews sought to understand the
employees’ knowledge, motivation, and the influence of the organization on the content creation
process. The document analysis targeted the interaction between members of the Content team
and the established cadence, timeframes, and critical thresholds of the content creation process.
Interviews
As Grow Learning’s Chief Learning Officer (CLO) and the manager of the Content team,
which comprises the entire population group for this study, it was not appropriate for this
researcher to conduct the interviews personally. Rubin and Rubin (2012) recommend that
interviewers should not be in a position of authority over interviewees, which can make it
difficult for participants to opt out of the study or to choose not to answer certain questions.
Accordingly, this researcher asked James McGhee, a fellow student in Cohort 6 of the OCL
Program, to conduct the interviews. He conducted eight, one-time interviews with willing
members of the Content team, assigning each participant a random number. This number is used
to reference participants in the analysis section. The interviews were conducted at Grow
Learning’s corporate office in Midtown Manhattan. Interview times were scheduled two weeks
in advance and a corresponding calendar invite was placed on each participant’s calendar. A
MODERN LEARNING REQUIRES MODERN DESIGN 33
private conference room was reserved for the interviews, with participant’s arriving individually
at the top of each hour. Each interview ran approximately 60 minutes.
Patton (2002) proposes that interviews can be helpful in determining a participant’s
unique perspective on a process and Weiss (1994) suggests that qualitative interviewing can also
be useful for the integration of multiple perspectives on a single process. For this study, it was
critical that the perspective of each member of the Content team be determined so, as a team,
they may be successful in their goal of revising the content creation process to create 1,000 new
Microlearning® lessons by year end. Merriam & Tisdell (2016) recommend that interviewers ask
a variety of questions to discover different types of information and emphasize that good
questions must be clearly worded, free of jargon, and understandable to the interviewee. For this
study, the interviews were conducted in a semi-structured format, with an established interview
protocol of twelve questions that allowed for the interviewer to ask follow-up probing questions
based on interviewee response. The interview protocol for this study attempted to uncover the
Content team’s collective knowledge and understanding of agile design methodologies and
whether they possessed the knowledge and motivation to revise the current content creation
process. The Interview Protocol is included in Appendix A.
Documents and Artifacts
Document analysis is a recognized and important part of qualitative research (Merriam &
Tisdell, 2016). For this study, an in-process group of Microlearning® lessons were chosen to
form the basis of the document analysis. Specifically, several documents related to the inception,
creation, development, and tracking of these lessons were analyzed. Table 6 identifies each of
these critical documents, their business function, and the person responsible for their creation
and maintenance.
MODERN LEARNING REQUIRES MODERN DESIGN 34
Table 6
Critical Documents for Analysis
Document Title Business Function Person Responsible
LED Team Stuff The “Schedule” tab tracks the various
due dates of all in-process lessons
Director of Learning Experience
Program Brief Provides topical overview, personas,
learning objectives, learner questions,
and critical research for lessons.
Director of Content Strategy
Storyboard Step-by-step visualization of lesson. Assigned Content Producer
Shot list Components of a lesson that will be
captured in film, animation, or audio.
Assigned Content Producer

These documents were fully within the purview of this researcher’s role as the manager of the
Content team and therefore it was not necessary to request permission to access any of these
documents. However, as a courtesy, the Content team was advised that these documents would
be analyzed as part of this study.
Merriam and Tisdell (2016) recommend that researchers “keep an open mind” when
identifying documents to analyze, not limiting themselves unnecessarily to a pre-determined set
of documents (p. 175). Accordingly, though four documents were identified as integral to the
analyzing the content creation process, this researcher also considered how other documents may
enhance understanding of the Content team’s knowledge, motivation, and organizational
influences related to the performance goal of creating 1,000 Microlearning® lessons by year end.
Results
As Grow Learning re-positioned itself within the learning tech landscape, the Content
team was forced to reckon its existing content creation process to achieve the performance goal
of creating 1,000 new Microlearning® lessons by year end 2018. This required the innovation of
MODERN LEARNING REQUIRES MODERN DESIGN 35
modern design approaches that would expedite content creation while maintaining established
quality and learner experience expectations. Data collection efforts included document analysis
and qualitative interviews of eight members of the Content team. The research questions that
guided this study were:
1. What is the stakeholder knowledge and motivation related to the creation of 1,000 new
lessons utilizing a modern learning design?
2. What is the interaction between organizational culture and context and stakeholder
knowledge and motivation?
3. What are the recommended knowledge, motivation, and organizational solutions?
Knowledge Results
This study set out to validate two specific knowledge influences that would be critical to
the Content team's ability to reach its target of 1,000 new Microlearning® lessons by year end
2018. These influences included the Content team's understanding of how employees learn in a
virtual environment and their understanding of the critical components of the modern learning
design process. In a thorough review of all interview transcripts and related documents, it was
clear that these knowledge influences were not validated as an opportunity for innovation related
to the Content team's performance. Indeed, the team consistently demonstrated a clear
understanding of these influences across four interview questions that targeted this knowledge.
Motivation Results
This study assumed two motivation influences when considering the Content team's
ability to meet its performance goal. Contained within the umbrella of expectancy value, these
influences included the team's ability to recognize the utility value of creating new learning
approaches and their cost belief related to its impact to customers. Both influences were
MODERN LEARNING REQUIRES MODERN DESIGN 36
validated and the findings surrounding these influences identified an opportunity to build upon
the team's strong self-identify in order to further engender trust and promote productivity.
Perhaps more so than other findings, the participants’ strong connection to the team was evident
throughout all interviews.
Organizational Results
The organizational influences identified at the start of this study emphasized the necessity
of the company to both embrace an agile methodology as its content creation strategy and
provide related training to the Content team. Study results validated both influences and
identified clear opportunities for innovation. Simply, the Content team identifies strongly as a
team and therefore is highly concerned, if not preoccupied, by their position within the
organization and their influence across departments. These findings suggest that the Content
team will be best positioned to reach its performance goal when concerns about its value to the
company are assuaged in addition to providing clear training on agile processes.
Findings
This study set out to identify opportunities to innovate the content creation process of
Grow Learning's Content team as evidenced by critical knowledge, motivation, and
organizational influences. The team's ability to reach the requisite 1,000 new Microlearning®
lessons by year end 2018 was dependent upon their ability to innovate the creation process.
Across the study, motivation and organizational influences were validated, and knowledge
influences were not validated, but all played an important role in better understanding the team's
knowledge and motivation to reach their goal and the organization's influence on that process.
Accordingly, the findings section has been organized into three primary themes that cut across
the KMO influences and identified research questions, these are: Do We Create Learning or
MODERN LEARNING REQUIRES MODERN DESIGN 37
Entertainment?, The Value of "Team" in Building Knowledge and Motivation, and Company
Culture Impacts Productivity.
Do We Create Learning or Entertainment?
In today’s learning marketplace, it is not uncommon for buyers to demand and expect
corporate learning content that is both theoretically sound and entertaining. The challenge of
developing Microlearning® content that is both solidly designed, and engaging is a continual
point of discussion between members of the Content team and was evidenced across all
interviews. This theme aligns most readily with the first research question, which is: What is the
stakeholder knowledge and motivation related to the creation of 1,000 new lessons utilizing a
modern learning design? Participant 3222 summarized this unique challenge:
There are many, many aspects of our content creation process that are effective. I think
that one thing that's maybe under recognized, but I think speaks volumes, is our
sophisticated production value and by that, I mean that we are meeting the twenty first
century content consumer. So, we know how to engage people who are consuming videos
all the time for entertainment purposes, but we're not just here for entertainment, we're
here for learning. So, we are pulling them in that way but then keeping them with us with
really sound and actionable learning.
The Content team is a single team comprised of three smaller teams: Strategy, Learning
Experience, and Production. The Strategy team sets the content roadmap and the Learning
Experience team builds the content in partnership with the Production team. Every lesson created
by the team include at least one video, in addition to text, gifs, motion, and an assessment that
support the central learning objective. Participants referenced the tension between building
learning and entertainment multiple times across interviews. Participant 4111 confided “but the
MODERN LEARNING REQUIRES MODERN DESIGN 38
concepts related to learning and production, there’s some tension between them.” Participant
0004 added: “We’re trying to change behaviors. People have to be engaged in order for them to
consume your stuff.” And, Participant 1051 shared: “like how people are learning and
consuming things without it you know we also try to not to be like entertainment.” Participant
3455 attempted to strike a balance between the concepts stating: “I think right now I would say
we are trying to create the highest quality content for a balanced library that we can get.”
Similarly, Participant 3432 suggested that the Content team: “develops Microlearning® lessons
that are both highly efficient and useful and also highly immersive and engaging for target
learners.” The dialogue surrounding whether lessons are principally learning or entertainment is
an important one and demonstrates that the Content team is committed to challenging,
questioning, and improving their approaches to building the best Microlearning® on the market.
Ultimately, Participant 3455 suggested this compromise: “Now I think we're looking at creating
the right content, the best way possible, for our clients and our prospective clients.” This
response summarized the Content team’s desire to focus on the output of their collaboration,
which is building compelling content that positively impacts Grow Learning’s customers.
Customer feedback validates output. A critical influence identified at the start of the
study was the Content team’s utility value is recognizing the usefulness in innovating new
learning design approaches and the cost belief related to the impact of these designs on
customers. A key finding from the interviews highlighted the team’s desire to fully understand
the impact their created content had on customers. Indeed, the lack of formal feedback processes
between the Content team and customers has created some anxiety and doubt as to the success of
the created content. Participant 1051 mentioned: “I make this stuff and I don't really know how
people respond to it. I don't actually know how successful it is in the end like from somebody
MODERN LEARNING REQUIRES MODERN DESIGN 39
who has taken it.” This desire for customer feedback was echoed through several interviews.
Participant 3455 contended that with consistent feedback “we’ll just be able to see what people
really want” and without it “we’re doing a lot of connecting the dots.” Further, Participant 3432
identified a missed opportunity during the first week of the content creation process, “we have
the user story but we don't actually have a real person, it's just kind of like a pretend person that
we have in our brains.” This represents a missed opportunity to engage with customers at the
beginning of the creation process and, as Participant 1051 observed: “I feel like if we had more
[customer] information, it would help.”
The need for access to customer feedback extends beyond basic lesson creation and into
deeper design choices made daily by the team. Participant 1051 revealed: “If we find, you know,
there’s some technique we’ve been relying on that doesn’t really work, that would be very
helpful to know.” Additionally, Participant 4111 suggested: “like talking to a real learner within
a company and saying, what is hard about running a meeting to you?” Access to consistent
customer feedback and learner engagement data is critical part of the team’s ability to innovate
modern design approaches. This information supports them in streamlining efforts, aligning
across the strategy, learning experience, and production teams, and ultimately gives them clearer
information to both assess the utility value and cost belief of the new design models they are
creating to reach their performance goal.
Agile means more than “quick.” Schwandt and Marquardt (1999) advocate for a hands-
on approach when an organization adopts new learning strategies, specifically those that close
gaps and support the achievement of performance goals. Likewise, it is incumbent upon Grow
Learning to clearly identify and define what an agile approach to content creation may look like
while engaging principal stakeholders in the discovery, creation, and implementation processes.
MODERN LEARNING REQUIRES MODERN DESIGN 40
To gauge the Content team’s understanding of agile, at the midpoint of each interview,
participants were prompted with this statement: Please tell me in your own words what agile
development means in the context of learning design. Though all participants were quick to
respond with their unique definitions of this complex term, their collective and applied
understanding of agile development was less clear. Initial responses from each participant varied
wildly, including definitions of agile as: “a buzzword” (Participant 3432), “an iterative process”
(Participant 3455), “willing[ness] to experiment” (Participant 4111), and simply “paying
attention” (Participant 1051).
As the interviews progressed, participants were asked to expand on their initial
definitions of agile. It was then that their collective understanding of the design process became
clearer. Participant 3432 conceded that: “the current content creation process is fairly agile”
likening it to a “cyclical effort” that incorporates information to “continually optimize” the
content. Participant 3455 expanded on this concept by adding: “while you approach
development, you work into the cycle criticism, reflection, and iteration” so that the end product
“has gone through multiple rounds of development.” These descriptions align more closely with
the principle values of agile, including the prioritization of individuals over processes and
responding to changes over following a plan (Alqudah & Razali, 2017). Participant 1051 offered
a concrete example of agile within the current content creation process:
I did a program recently. It was all scoped out and you know I had all the information and
I was chugging along and then while I was doing it I was like, oh I need another lesson
here because there's this part of it that I didn't address that now I'm seeing it now that I
see it sort of laid out. People need to know this or people need to see this because this one
thing is going to illuminate all this other stuff. So, for me I think that's part of the
MODERN LEARNING REQUIRES MODERN DESIGN 41
agileness. Agility, whatever of learning design is that you have to know when you know
you have to be able to respond and like add something in that will illuminate everything
else or maybe take something out.
This example demonstrated both an applied understanding of the agile process and a willingness
to flex when responding to changes that occur throughout the 6-week content creation process.
Participant 1051 further described this agility in saying: “I have to be nimble enough to know
like that’s not really working…like I’m not making the point or I’ve made the point but it’s not
as strong.” Clearly, the participants are familiar with the concept of agile, and are able to identify
examples in their day-to-day work but were not as confident when it came to identifying agile as
an organizational goal. Participant 0004 asserted: “when you think of agile, you think of agile as
an adjective, but agile as a discipline, I don’t know that we are.” These responses signify a
disconnect between the team’s descriptions of the content creation process as agile and the
organization’s intentions to formally adopt an agile process to build its learning content.
Participant 3455 suggested that true agile design must be done “systematically” and that the team
must consider “not just how do we make this content better with every cycle, but how do we
make the process better with every cycle.” Indeed, to achieve true product differentiation,
Kochikar and Ravindra (2007) suggest that organizations and teams must develop specific agile
capabilities as part of an individual’s competency model. Additionally, Alqudah and Razali
(2017) contend that less than 50% of agile projects are successful because organizations do not
develop the necessary agile skillset in the primary development team. Concluding the discussion
on agile, Participant 3432 summarized the team and company’s current state:
So, I think like it raises interesting business questions for us and like organizational
design questions for us and I don't know that the team really understands it very well. So
MODERN LEARNING REQUIRES MODERN DESIGN 42
I think like we try to bake in aspects and themes from agile as much as we can to make
sure that we are really producing innovative work and I think we do a good job of that. I
think that there's still room to grow in terms of like upping the sophistication level.
The Value of “Team” in Building Knowledge and Motivation
The conceptual framework created for this study was designed with both a current and
desired future state for members of the Content team. Specifically, that the former, chaotic state
of the Content team’s knowledge and motivation influences, exacerbated by the company’s
reimagined go-to-market strategies, would be unified and streamlined as the team established its
own revised strategies for creating content. This cultural shift assumed that the Content team
would function as a team – and that its attempts to innovate the content creation processes would
both solidify its knowledge of those processes and pinpoint its motivation to start, endure, and
complete them (Clark & Estes, 2008). This theme illuminates findings related to the study’s
second research question, which is: What is the interaction between organizational culture and
context and stakeholder knowledge and motivation?
Surprisingly, the qualitative evidence showed that members of the Content team not only
rely upon the support of their peers but expect it as resource to reach their performance goals.
This need is heightened by the fact that the team believes they are building something unique and
important to both customers and the larger industry of learning. Participant 0004 pointed out: “I
think we’re doing something cool and yeah, exciting and different…and member of the team are
inspired by what we’re doing and that’s really inspiring to me.” Important to note is that this
team, of all other departments within Grow Learning, had endured the brunt of the organizational
change. Indeed, most of the team had experienced multiple changes in leadership, immediate
management, and organizational positioning over a span of two to three years. In response to the
MODERN LEARNING REQUIRES MODERN DESIGN 43
question of “What do you look forward to most in your role,” Participant 1000 said simply:
“collaboration.” In speaking of the team’s success, Participant 4111 confided: “I think so much
of the speed that we can do this with really comes from, it comes from having really good
support.” And Participant 3455 shared: “we all do group meetings about new content. We look at
it together.” This concept of team, then, is developed as not a cultural nice to have, but rather, a
necessity for performance. Participant 3432 identified this by sharing: “when I wake up in the
morning I look forward to just like working with other people in the company.” Participants did
not simply expound on their “like” of the team, nor did they conflate niceness with work. Each
participant identified that they both appreciate their team and that the team, together, is
instrumental is improving productivity and reaching performance goals. In summary, Participant
4111 shared: “…but our team, I think it's awesomely successful. You know, we've done stuff
that I would not have believed we could have done when I first started here.”
Let me be autonomous and collaborative. Employee motivation, according to Colquitt,
LePine, and Noe (2000), is directly connected to job involvement, and is typically evidenced
through individual perceptions of self-image, long-term goals, and the value employees place on
their work. Job involvement as a theme emerged across several interviews and served to further
illuminate Content team members’ motivation to iterate on known learning design models to reach
the 1,000 Microlearning® lesson mark by year end 2018. As a key descriptor of job involvement,
the concept of “autonomy” was referenced multiple times by two participants. These participants
hold the role of Content Producer and are the individuals with primary responsibility for ushering
a lesson from initial idea to final publication. When asked to describe the content creation process,
Participant 1051 asserted “I have a lot of autonomy while I'm actually doing the process” and
Participant 3222 explained “it's really from planning to publish, that autonomy.” Building on this
MODERN LEARNING REQUIRES MODERN DESIGN 44
concept, Participant 3432, not a Content Producer, characterized this autonomy as being “cut off
from everyone else,” because “it’s hard to find that time during the week” to complete required
tasks. Interestingly, Participant 1051 further developed the concept of autonomy in sharing:
“because you are so autonomous that it's nice to feel like people have your back.” In this instance,
Participant 1051 has conflated the concept of autonomy with the need for collaboration. These
participants, the only two Content Producers participating in the study, conceded the
“autonomous” nature of their work, yet they also seemed to clarify that their motivation comes not
only from autonomy, but from the confluence of autonomy and collaboration.
In general, participants are keenly motivated by the collaborative culture and expectation
of team work within the Content team. In referencing the day to day work, Participant 3222
reported: “I most look forward to collaboration. So, getting to really dig into a topic and then
connect with other people on it and get other kind of brains to engage and talk to them about why
things are important.” This expectation of collaboration seemed also to motivate Participant
3222, specifically the ability to “engage and talk” to other members of the team. Participant 4111
expanded the concept of collaboration in saying: “it really is a collaborative effort…and it
wouldn’t work without specialized people to help us do stuff.” And Participant 3432 revealed: “I
mean, when I wake up in the morning I look forward to…working with other people in the
company.” Across all interviews, the concept of teamwork and collaboration was pervasive,
emerging more than 22 times, which reinforces the organizational design and structure of the
Content team as a group of individual experts who own distinct parts of the content creation
process that must also collaborate to achieve their goals.
MODERN LEARNING REQUIRES MODERN DESIGN 45
Team trust fosters productivity. As a critical component of collaboration, the concept
of trust was emphasized by participants throughout the interviews. Participant 1000 summarized
the importance of the Content team’s collaborative spirit:
I'd say first and foremost, we just have really good team of people who trust each other
but also care about each other, which you know is huge if you're having a rough day. You
know, it's great to have people who you know will pick you up or even if you're not
having a rough day, it's good to have people who make your day better. I know that if I
do something great, then people will recognize me, and if I do something subpar I'll also
hear about it in way that's honest and fair and I can kind of put my trust there.
The correlation of collaboration to trust is striking in this excerpt. Book-ending the quote and
underlined for emphasis, “trust” is identified as a key ingredient to the team’s productivity and
success. Specifically, Participant 1000 recognized that the team is comprised of individuals “who
trust each other” and will support each other, while also providing “honest and fair” feedback.
Further developing this concept of trust, Participant 1051 said, “…it's a very open place.
We're also psychologically safe which is a concept that we've all learned” and “it's a really great
team to work with and there's a lot of really smart people here so that is something to look
forward to.” Participant 3222 insisted, “And that collaboration…is so rewarding because they
will always bring something that I didn't think of.” Participants derive motivation from
collaboration, both in culture and in day to day interactions. These team interactions seem to
develop both the individual’s knowledge of the content creation and motivation to build content,
while also serving to increase safety and trust.
The Content team’s expectation of trust extends beyond their group to both the company
and customers. In referencing the company’s reliance on the Content team, Participant 1051
MODERN LEARNING REQUIRES MODERN DESIGN 46
admitted, “they trust us to figure out what's going to be the best thing.” But this trust is always
coupled with high expectations as Participant 1000 revealed, “I'm trusted to do my job and
entrusted to surface things that don't work.” Here, the concept of trust is tested, as members of
the Content team both rely upon it to meet their productivity goals and see that they are likewise
trusted to perform their jobs well. Further, Participant 3222 extended the notion of trust to Grow
Learning’s customers saying, “We (the Content team) strive to be authentic, human, accessible,
and a trustworthy partner to our learners.” And Participant 0004 talked about earning learner
trust through content design, saying, “We talk about how we aren’t going to take a single click
for granted. That’s how we earn their trust, we need to keep their trust.” Ultimately, trust is both
motivator and required expectation for the team to function collaboratively. Participant 3222
linked trust with productivity in saying, “In general I do trust that the organization is made up of
good people who believe in making great work.” This ideal state of knowledge and motivation
corresponds to the initial conceptual framework for the study and reflects the desired future state
of the Content team. Specifically, that members of the Content team will align their knowledge
and motivation of learning design to reach the ambitious goal of 1,000 net new lessons by year
end 2018.
Company Culture Impacts Productivity
Across each of the eight interviews, the concept of change was referenced, discussed, and
detailed 74 times. Seventy-four times, participants identified change as a critical component in
understanding Grow Learning’s transition as a company over the past eight years. Participant
1051 shared, “I feel like there were changes, a lot of changes across the company” and
Participant 3222 conceded, “I’ve been here for nearly two years and it feels like at least five.”
Additionally, Participant 1000 noted, “My role has changed from the changes” and Participant
MODERN LEARNING REQUIRES MODERN DESIGN 47
4111 summed up the many changes by saying, “I see now the change has been really good and it
has been much more stable here.” The concept of change is a common point of discussion for a
team that endured multiple changes in leadership and team structure. This theme and related
findings correspond to the study’s second research question - What is the interaction between
organizational culture and context and stakeholder knowledge and motivation? - and further
suggest that members of the Content team exist in a state of high alert and are ultimately
sensitive to even more change.
In the final portion of each interview, participants were asked to describe Grow
Learning’s current culture. Participant responses reflected their preoccupation with change and
seemed to be predicated upon the company’s recent shift in leadership and go-to-market strategy.
Participant 3432 shared that “the company is very different now from the organization I joined in
2015.” Participant 4011, in response to the question, conceded, “you know I hate this cliché, but
change is the only constant.” Adopting a more positive approach to the question, Participant
1000 declared, “I think there's been a certain sense of maturity, I'd say which is in contrast to
maybe where we were a little while ago.” Participant 1051 confided that “it can feel like there
are these little pockets of people, but I feel like the company's trying to get everyone to be a little
bit more engaged and see where we can cross pollinate a little bit.” In their initial reactions,
participants described Grow Learning’s current culture in terms of its history and the years of
change in between then and now. However, they also cited the company’s “maturity” and efforts
on the part of leadership to “cross-pollinate” between teams.
Clark and Estes (2008) contend that organizational culture may be “the most important
work process” and speculate that an organization’s culture will have a critical influence on a
group’s ability to achieve performance goals (p. 107). Demonstrating the validity of this claim,
MODERN LEARNING REQUIRES MODERN DESIGN 48
several participants chose to elaborate on their initial description of Grow Learning’s culture,
identifying the ways it impacts their daily work. Participant 3432 admitted, “Now, [Grow
Learning] is a far more mature organization. People in their roles must be thoughtful and the
different teams must connect to each other in their efforts and work well with each other. That
did not happen before.” Further, Participant 3455 reported that the culture is a “collaborative and
open workspace.” Additionally, Participant 1000 admitted that he has grown because of the
change, stating, “As a person, I've sort of gotten a better sense of how to stay focused in times of
change and how to not overreact to things beyond my control.” This resolve suggests that
members of the Content team, though adversely affected by the vicissitudes at Grow Learning
over the previous several years, are now able to recognize the positive change instituted in the
past year and further recognize the impact that change has on their role and their team.
Where does the Content team fit in? An important sub-theme that emerged across the
interviews exposed the Content team’s preoccupation with its positioning, importance, and
influence within Grow Learning. Initially, when asked to describe the ways the culture of Grow
Learning impacts their role on the team, participants were cautiously optimistic. Participant 4111
commented, “The only thing I could say is I know that we have somehow weirdly become the
most functional team in the company” and “I think they treat us pretty well.” In revising its go-
to-market strategies, Grow Learning bet big on content and positioned its Microlearning® library
as its primary and key differentiator in the busy Learning & Development marketplace.
Participant 3455 described this shift by saying, “We look at Content as sort of the central heart of
Grow Learning” and Participant 3432 confirmed, “I think that the changes for the Content team
have mainly elevated us.” The decision to prioritize content creation over platform development
MODERN LEARNING REQUIRES MODERN DESIGN 49
both positioned the Content team as the engine that created the content and subsequently
elevated them to a place of recognition within the organization.
The Content team is cognizant of the changes throughout Grow Learning that have
placed them as a strategic driver of the company. Participant 3432 said, “I think that now we’re
seeing that we can, that Content can really steer the conversation.” And Participant 3455
explained the impact to day-to-day work in saying, “You have access to people in all
departments at all levels if you need them.” However, the repositioning of the Content team
comes at a price and participants explained the ways this shift has impacted them. Participant
1051 confided, “There's a certain level of pressure that comes with that of course. I mean if
you're on the team that's basically driving the company it's like oh well don't screw it up.” And
Participant 3222 commented, “I think that's something that people at startups really internalize.
There's no such thing as inertia. It's like purpose or failure.” Members of the Content team are
anxious to establish and appreciate their new position within the organization. Participant 3432
summarized the team’s newfound momentum saying, “The changes that we've had over the past
year have largely increased focus on content as a strategic aspect of our company, which I think
is very gratifying to the team. It's certainly gratifying to me.” Ultimately, the team appears
motivated by their increased ability to build meaningful learning that is validated, appreciated,
and respected by their peers.
Recommendations for Practice
This study set out to determine how knowledge, motivation, and organizational
influences impacted the Content team’s ability to devise new learning design models to reach
their performance goal of creating 1,000 new Microlearning® lessons by year end 2018.
Surprisingly, the assumed knowledge influences established at the outset of the study were not
MODERN LEARNING REQUIRES MODERN DESIGN 50
validated as opportunities for innovation. It was evident through analysis that Content team
members were able to demonstrate their knowledge of both assumed influences, which were 1)
The Content team must understand how employees learn in a virtual learning environment, and
2) The Content team must recognize the critical components of the modern learning design
process. Therefore, the recommendations for practice emphasize the motivation and
organizational influences that provided the greatest opportunity for improvement. In this section,
solutions and recommendations related to these findings will be discussed in greater detail.
Motivation Recommendations
The study validated both assumed motivation influences as critical to the Content team’s
ability to reach their performance goal. Table 7 presents both validated influences, identifies the
motivation type, details a related principle and citation, and offers a context-specific
recommendation.
Table 7
Summary of Motivation Influences and Recommendations
Validated Motivation
Influence
Motivation
Type
Principle and Citation Context-Specific
Recommendation
The Content team must
recognize the
usefulness of creating
new learning
approaches to meet
performance goals.
Utility
Value
An individual assigns
utility value to a task
when he/she believes it
may be useful in
achieving future goals,
thus being motivated to
start, endure, and
complete the task. (Clark
& Estes, 2008; Eccles,
2006; Pintrich, 2003).
Reinforce the value and
future impact of the
modern learning approach
the team has created
together to reach the
performance goal.
The Content team
needs to understand
how a modern design
approach is more
Cost Belief Feedback as well as
actual success on
challenging tasks
positively influences
Source and share relevant
customer feedback on the
impact of lessons
MODERN LEARNING REQUIRES MODERN DESIGN 51
valuable both to Grow
Learning and its
customers.
Recognizing that the
pain of same is greater
than the pain of
change.
people’s perceptions of
competence (Borgogni et
al., 2011).

Activating personal
interest through
opportunities for choice
and control can increase
motivation (Eccles,
2006).
designed in the new
approach.

Schedule “ride-along”
conference calls in which
Content team members
may listen in to customer
feedback sessions on
newly released content.

Utility Value. Utility value is an individual’s belief around whether that task will be
useful in achieving future goals (Eccles, 2006). Members of the Content team must recognize the
usefulness of creating new learning approaches in order to meet their performance goal. Pintrich
(2003) recommends that reinforcing the rationale and importance of a task will increase its utility
value. The recommendation then is to demonstrate and reinforce to the content team the value of
the modern design approach to its customers.
Clark & Estes (2008) document the employee’s lifecycle of motivation in terms of his/her
determination to start, endure, and complete a task. Simply put, members of the content team
will be more likely to value the modern learning design approaches needed to reach their
performance goals if they see it as an opportunity for future success at Grow Learning. Colquitt,
LePine, and Noe (2000) postulate that training motivation is directly linked to an individual’s job
involvement and career commitment while Klein et al (2006) suggest that perceived barriers
within an organization may adversely impact motivation itself and that an individual’s attitude
toward technology is influenced by its perceived utility. Members of the Content team place high
value on the design approaches they have created because they know that these will result in the
creation of innovative work that is critical to the success of Grow Learning.
MODERN LEARNING REQUIRES MODERN DESIGN 52
Cost Belief. The Content team must understand how a modern design approach is more
valuable to both Grow Learning and its customers than old design models. Borgogni et al. (2011)
suggest that feedback on challenging tasks may positively influence an individual’s perception of
competence. Further, Eccles (2006) recommend the activation of personal interest to increase
motivation through opportunities for choice and control. The recommendation then is to solicit
and share customer feedback with the content team to demonstrate the value of the revised and
evolving design process.
To quantify the value of the modern design approach, the Content team must believe that
the pain of same will ultimately be greater than the pain of change. Wigfield and Cambria (2010)
suggest that the emphasis of cost is especially critical in issues of choice, because choice
includes both positive and negative task associations. The Content team must articulate the tasks
associated with choosing a modern design strategy and how those tasks differ from the tasks
associated with choosing an outdated design approach. Aguinis and Kraiger (2009) contend that
the benefits of training are maximized when companies pay close attention to the training design
process, which may include adopting new strategies or forgoing old strategies. By assigning a
cost value to these tasks, the Content team will be more successful in innovating an appropriate
solution.
Organizational Recommendations
Both organizational influences, which detailed the need for Grow Learning to both
embrace agile design as its content strategy and provide the requisite training, were validated as a
result of the study. Table 8 provides greater detail on each validated influence, the type (e.g.:
model or setting), the corresponding principle and citation, and the context-specific
recommendation.
MODERN LEARNING REQUIRES MODERN DESIGN 53
Table 8
Summary of Organizational Influences and Recommendations
Validated
Organization
Influence
Type Principle and Citation

Context-Specific
Recommendation
The organization
needs to embrace
an agile
methodology as
its content
creation strategy.
Cultural
Model
Adults are more motivated to
participate (and learn) when they
see the relevance of information, a
request, or task (the “why”) to
their own circumstances. They are
goal oriented. (Knowles, 1980).

Organizations with high levels of
cultural trust tend to produce high
quality products and services at
less cost because they can recruit
and retain highly motivated
employees. These employees are
more likely to enjoy their work,
take the time to do their jobs
correctly; make their own
decisions; take risks; innovate;
embrace the organization’s vision,
mission, and values; and display
organizational citizenship
behavior (e.g., helping a co-
worker in need) (Colquitt, Scott &
LePine, 2007).
The organization,
represented by the
Executive team, must
internally acknowledge
that, due to the
innovation of the
Content team, the
current content creation
process includes
important components
of agile design.

The organization
needs to provide
training to the
Content Team in
agile design
methodology.
Cultural
Setting
Organizational effectiveness
increases when leaders ensure that
employees have the resources
needed to achieve the
organization’s goals. (Clark &
Estes, 2008).

Modeling to-be-learned strategies
or behaviors improves self-
efficacy, learning, and
performance (Denler, Wolters, &
Benzon, 2009).

The CLO must allocate
resources to provide
and model related agile
training to the Content
team, to include 3
types of content: new
knowledge and skills,
coordination of skills
across Content sub-
teams, and training.
MODERN LEARNING REQUIRES MODERN DESIGN 54

Cultural model. Adults will be more motivated to participate in a task when they can see
its relevance (Knowles, 1980). Similarly, Colquitt, Scott, and LePine (2007) suggest that
organizations with high levels of cultural trust, like Grow Learning, are more likely to employ
individuals that enjoy their work, embrace the organization’s vision, and innovate. To support
the Content team in achieving their performance goal, Grow Learning needs to embrace an agile
methodology as it official content creation strategy. This means that Grow Learning, represented
by the Executive team, must acknowledge that due to the innovation of the Content team, the
current content creation process includes important components of agile design.
Schwandt and Marquardt (1999) recommend a hands-on approach when organizations
adopt a new learning strategy, advocating that organizations “wrestle” with internal process to
best determine how to close gaps and support stakeholders (p. 228). Similarly, Rath and Conchie
(2009) purport that successful leadership is evidenced in the uniting of followers towards a
clearly defined objective. As such, to be successful in their goal of creating new design
approaches, Grow Learning must support the Content team by formalizing the agile approach as
the organization’s content creation strategy. Boswell (2006) highlights the connection between
employee and organizational strategy, recommending a clear “line of sight” into the process,
which reinforces the why and therefore encourages support of the new strategy (p. 1490). To
support the Content team, Grow Learning must internally acknowledge that the content creation
process in, indeed, agile.
Cultural setting. Clark and Estes (2008) contend that an organization’s effectiveness
increases when its leaders ensure that employees have the needed resources to achieve goals.
Likewise, Grow Learning needs to provide training to the Content team on agile design to
MODERN LEARNING REQUIRES MODERN DESIGN 55
support them in meeting their performance goal. The recommendation then is for the CLO to
allocate appropriate resources to provide and model related agile training to the Content team.
Denler, Wolters, and Benzon (2009) recommend the modeling of to-be-learned strategies
to improve self-efficacy, learning, and performance. Once Grow Learning has established a clear
organizational strategy aligned with the performance goal, they must provide training to the
Content team focused on agile design. Pirjol and Maxim (2012) identify training as a critical
organizational process that can influence employees on a personal level and Kraiger et al. (2004)
recommend connecting training to business goals as a way to increase adoption. In terms of agile
training, Alqudah and Razali (2017) warn that less than 50% of agile-deployed projects are
successful because companies overlook the need to delivery critical agile training to principal
stakeholders. Similarly, Kochikar and Ravindra (2007) propose that organizations develop
required agile skill sets for key employees in order to achieve intended business results.
Ultimately, to be successful in its deployment of agile design training, the literature suggests that
Grow Learning must create and deploy agile training to its Content team to support them in
achieving their performance goal.
Conclusion
For the better part of a year, it has been this researcher’s distinct privilege to investigate
the knowledge, motivation, and organizational influences of the dedicated team at Grow
Learning, who is responsible for building the world’s most compelling corporate learning
content. As both practitioner and scholar, this researcher led and studied the workings of this
amazing team, tracking their progress towards the established performance goal of creating 1,000
new Microlearning® lessons by year end 2018.
MODERN LEARNING REQUIRES MODERN DESIGN 56
Conceived through the Clark and Estes (2008) KMO framework, this study assumed six
influences that would either propel or impair the team’s progress. Additionally, a conceptual
framework was devised that integrated these influences, anticipating considerable growth for the
team as they strived towards their goal. Interestingly, it was not the knowledge influences that
were validated through research, but, overwhelmingly, the analysis pointed to the team’s intense
reliance on their culture of collaboration as a key motivator in their success. This finding,
coupled with insights about the team’s interaction with various organizational assets, strongly
informed the recommended solution that will propel the team to the successful completion of
their performance goal. The recommended solution, detailed in Appendix E, is further predicated
upon Clark and Estes (2008) contention that employee performance will rise or fall based on the
strength of an organization’s culture. Indeed, they are systems, processes, and cultural artifacts of
an organization that will ultimately support employees beyond their own cultivated knowledge
and motivation. Because of this, the recommended solutions have been designed to integrate into
Grow Learning’s established culture, reinforcing key motivation and organizational influences
through the defined processes that inform the day-to-day work of the Content team, including
standing team meetings, collaboration, recurring 1:1s, and company-wide communications.
Limitations to this study are inherent to its scope, as the entire population was limited to
members of the Content team of Grow Learning. Accordingly, any findings should be treated
carefully and represented within that context.
In the world of corporate learning, content has become king. It is the method through
which business can most expediently and profoundly impact its workforce in a time of rapid
change, innovation, and digital transformation. Indeed, the need to design and deliver high-
quality, relevant content is critical as practitioners seek to demonstrate the value of learning in
MODERN LEARNING REQUIRES MODERN DESIGN 57
the corporate space. The findings of this study proposed a solution to the challenge of building
content at pace with the workplace by appropriating learnings from agile design to create content
on rapid, iterative cycles. But the challenge remains of how this compelling content can be most
effectively delivered and consumed by a voracious learning audience. Corporate learners are
sophisticated consumers and their expectations for content delivery far outstrip current
approaches and solutions. More work must be done to devise strategic delivery methodologies by
which employees might search, select, engage with, and consume high-quality, relevant content.
Further, to source, nurture, and retain top talent, learning content must be strategically created,
expertly curated, and seamlessly delivered in intuitive and personalized ways. If modern learning
requires modern design, then the modern workplace demands modern strategies, and it is the
learning practitioner who is uniquely suited to overcome these challenges with nimble, flexible,
and agile solutions.

MODERN LEARNING REQUIRES MODERN DESIGN 58
References
Alqudah, M., & Razali, R. (2017). Key factors for selecting an Agile method: A systematic
literature review. International Journal on Advanced Science, Engineering and
Information Technology, 526-537
Aguinis, H., & Kraiger, K. (2009). Benefits of training and development for individuals and
teams, organizations, and society. Annual Review of Psychology, 451–474.
Arthur, W., Bennett, W., Edens, P. S., & Bell, S. T. (2003). Effectiveness of Training in
Organizations: A Meta-Analysis of Design and Evaluation Features. Journal of Applied
Psychology. Vol 88. No. 2, 234-245.
Bell, B. S., & Kozlowski, S. W. J. (2002). Adaptive guidance: Enhancing self-regulation,
knowledge, and performance in technology-based training. Personnel Psychology;
Summer, 55(2).
Bersin, J. (2016). Predictions for 2016: A bold new world of talent, learning, leadership, and HR
technology ahead. Retrieved from Bersin by Deloitte website: www.bersin.com/library
Bersin, J. (2014, February). Spending on corporate training soars: Employee capabilities now a
priority. Forbes. Retrieved from http://www.forbes.com/sites/joshbersin/2014/02/04/the-
recovery-arrives-corporate-training-spend-skyrockets/#578fab6a4ab7
Bogdan, R. C., & Biklen, S. K. (2007). Qualitative research for education: An introduction to
theories and methods (5th ed.) Boston, MA: Allyn and Bacon.
Borgogni, L., Russo S. D., & Latham, G. P. (2011). The relationship of employee perceptions of
the immediate supervisor and top management with collective efficacy. Journal of
Leadership and Organizational Studies, 18(1), 5–13.
MODERN LEARNING REQUIRES MODERN DESIGN 59
Boswell, W. (2006). Aligning employees with the organization’s strategic objectives: Out of
“line of sight”, out of mind. The International Journal of Human Resource Management,
17(9), 1489–1511.
Brown, K. G. (2001). Using computers to deliver training: Which employees learn and why?
Personnel Psychology; Summer, 54(2).
Carroll, B., Singaraju, R., Park, E. (2015). Corporate Learning Factbook 2015: Benchmarks,
trends, and analysis of the U.S. training market. Retrieved from Bersin by Deloitte
website: www.bersin.com/library
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Charlotte, NC: Information Age Publishing, Inc.
Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Towards an integrative theory of training
motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied
Psychology, 85, 678–807.
Colquitt, J., Scott, B., & Lepine, J. (n.d.). Trust, trustworthiness, and trust propensity: a meta-
analytic test of their unique relationships with risk taking and job performance. The
Journal of Applied Psychology, 92(4), 909–927.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches. Thousand Oaks, CA: Sage Publications.
Eccles, J. (2006). Expectancy value motivational theory. Retrieved
from http://www.education.com/reference/article/expectancy-value-motivational-theory/.
Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist, 36,
45–56. doi:10.1207/S15326985EP3601_5
MODERN LEARNING REQUIRES MODERN DESIGN 60
Glesne, C. (2011). Chapter 6: But is it ethical? Considering what is “right.” In Becoming
qualitative researchers: An introduction (4th ed.) (pp. 162-183). Boston, MA: Pearson.
Heimbeck, D., Frese, M., Sonnentag, S. & Keith, N. (2003). Integrating errors into the training
process: the function of error management instructions and the role of goal orientation.
Pers. Psychol. 56:333–61
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick’s four levels of training evaluation.
Alexandria, VA: ATD Press.
Klein, Howard, J., Noe, Raymond, A., & Wang, C. (2006). Motivation to learn and course
outcomes: The impact of delivery mode, learning goal orientation, and perceived barriers
and enablers.
Knowles, M. S. (1980). The modern practice of adult education: From pedagogy to andragogy.
Englewood Cliffs, NJ: Cambridge Adult Education.
Kochikar, V., & Ravindra, M. (2007). Developing the capability to be agile. Organization
Development Journal, 25(4), 127-134.
Kraiger, K., McLinden, D., Casper, W. J. (2004). Collaborative planning for training impact.
Human Resource Management.
Krathwohl, D. R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory Into
Practice, 41(4), 212–218.
Maxwell, J. A. (2013). Qualitative research design: An interactive approach. (3rd ed.).
Thousand Oaks: SAGE.
Mayer, R. E. (2011). How learning works. In Applying the science of learning. Boston, MA:
Pearson Education.
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
MODERN LEARNING REQUIRES MODERN DESIGN 61
implementation. (4th ed.). San Francisco: Jossey-Bass.
The National Academies (2012, November) The Informed Brain in a Digital World:
Interdisciplinary Team Summaries. Keck Future Initiative. ISBN 978-0-30926888-2.
Nguyen, M., Prasad, P., Alsadoon, A., Hoe, L., & Elchouemi, A. (2016). Cloud-based learning A
study on rapid learning content development with an Agile method. IEEE Global
Engineering Education Conference, EDUCON.
Oxford Economics (2014, September). Workforce 2020: Fact sheet: SMEs.
Retrieved from http://www.oxfordeconomics.com/publication/open/251378
Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA:
SAGE Publications.
Pirjol, F., & Maxim, R. G. (2012, December). Organizational culture and its way of expression
within the organization. Annals of the University of Oradea, Economic Science Series,
21(2), 371-376.
Rath, T., & Conchie, B. (2009). Strengths-based leadership. New York: Gallup, Inc., 79‐91.
Rubin, H. J., & Rubin, I. S. (2012). Chapter 6: Conversational partnerships. In Qualitative
interviewing: The art of hearing data (3rd ed.) (pp. 85-92). Thousand Oaks, CA: SAGE
Publications.
Schwandt, D., & Marquardt, M. (1999). Steps and strategies for bringing organizational learning
to your organization. Organizational learning from world-class theories to global best
practices. New York, NY: St. Lucie Press, 227-249.
Schwartz, J., Bohdal-Spiegelhoff, U., Gretczko, M., Sloan, N. (2016). Global Human Capital
Trends 2016. Retrieved from the Human Capital section of the Deloitte website:
MODERN LEARNING REQUIRES MODERN DESIGN 62
http://www2.deloitte.com/us/en/misc/search.html#qr=human%20capital#f=more%3Aarti
cles#p=1#fr=0#country=UnitedStates
Senge, P. (1990). The leader’s new work: Building learning organizations. Sloan Management
Review, 32(1), 7-23.
Shraw, G., & Lehman, S. (2009). Interest. Retrieved from http://www.education.com/
reference/article/interest/
Sitzmann, T., Kraiger, K., Stewart, D.; & Wisher, R. (2006). The comparative effectiveness
of web-based and classroom instruction: A M. Personnel Psychology; Autumn, 59(3).
Weiss, R. S. (1994). Learning from strangers: The art and method of qualitative interview studies
(pp. 51-59, 61-81, and 141-150). New York, NY: The Fre Press.
Wentworth, D. (2016). 2016 Learning technology study: Summary of top findings. Retrieved
from Brandon Hall Group website: http://go.brandonhall.com/l/8262/2016-04-25/5brswr
Wigfield, A., & Cambria, J. (2010). Students' achievement values, goal orientations, and
interest: Definitions, development, and relations to achievement outcomes.
Developmental Review, 30, 1–35.

MODERN LEARNING REQUIRES MODERN DESIGN 63
Appendix A: Participating Stakeholders with Sampling Criteria
The Content team of Grow Learning was primarily responsible for achieving the goal of
improving its content creation process to successfully publish 1,000 new Microlearning® lessons
by year end 2018. Though it is common to select either random or nonrandom sampling when
choosing representative participants for a study, in this case the identified stakeholder group was
self-contained and were treated as a total population (Fink, 2013). Therefore, study participants
were defined as current members of the Content team who contributed to the creation of new
Microlearning® lessons. Using these criteria, the population size for this study was 13.
Interview Sampling Criteria and Rationale
Criterion 1. Must be members of the Content team, which comprises three sub-groups,
including: Strategy, Learning Experience, and Production. The rationale for this criterion was
that members of the Content team were the only Grow Learning employees responsible for and
involved in building new Microlearning® content.
Criterion 2. Must be actively contributing to the content creation process in the current
content production cycle. The rationale for this criterion was that when interviews were
conducted, some members of the Content team were not be actively creating content in the
current production cycle.
Criterion 3. Must be willing and available to participate in a 45-minute interview. The
rationale for this criterion was that there were members of the team who were out on leave or
unavailable during the scheduled time.
Interview Sampling (Recruitment) Strategy and Rationale
A primary goal of qualitative research is to better understand how people “interpret their
experiences” (Merriam & Tisdell, 2016, p. 6). This supposes that words function as a type of
MODERN LEARNING REQUIRES MODERN DESIGN 64
data in qualitative study, much like numbers form the base of research in quantitative inquiry
(Merriam & Tisdell, 2016). Content team members were emailed an overview of the study and
were asked if they were willing to participate by submitting to a 45-minute interview. The
interviews were structured to better understand the knowledge, motivation, and organizational
influences of the Content team as it attempted to innovate new learning design approaches.

MODERN LEARNING REQUIRES MODERN DESIGN 65
Appendix B: Interview Protocol
1. Tell me about your role on the Content team, specifically in relation to the content creation
process.
2. Describe the current content creation process as you understand it.
3. Tell me about a lesson you recently created (or contributed to) that was especially unique,
challenging, or exciting.
4. Tell me about the choices you make within your role that deliberately increase learner
engagement.
5. In what ways are you enhancing your own learning through each production cycle?
6. Please tell me in your own words what 'agile' development means in the context of learning
design.
7. What about the current content creation process is optimized to create the most engaging
learning?
8. What, in your opinion, is your role in improving upon the current content creation process?
a. Probe: In the past six months, have you shared an idea or participated in a
conversation to improve the content creation process? Can you elaborate?
Transition: For the final few questions, I’d like to focus on the role of the Content team within
Grow Learning.
9. What are the characteristics of Grow Learning’s culture?
10. How do these characteristics affect you in your role on the Content team?
11. In what ways might Grow Learning or its customers influence the current content creation
process?
12. What do you understand to be the current goals of the Content team?
MODERN LEARNING REQUIRES MODERN DESIGN 66
Appendix C: Credibility and Trustworthiness
Merriam and Tisdell (2016) purport that qualitative research can only be as trustworthy
as the rigor applied in conducting it, which includes both quality measures and ethical
considerations. Further, Merriam and Tisdell (2016) recommend that a researcher may influence
a study’s validity with the creation of the conceptual framework and with careful and consistent
attention to the collection of qualitative data. Similarly, Creswell (2008) defines validity as the
diligence a researcher applies in validating findings and checking data for accuracy. As such,
Merriam and Tisdell (2016) recommend several strategies that a researcher may employee to
continually assess the credibility and trustworthiness of a study. Maxwell (2013) challenges
researchers to rule out researcher bias and reactivity not by simply “verifying conclusions, but by
testing the validity of your conclusions” (p. 125). Accordingly, I plan to adopt three of the
strategies proposed by Merriam and Tisdell (2016) that will promote the validity and reliability
of this study.
Triangulation, as defined by Merriam and Tisdell (2016) is a strategy used by researchers
which utilizes multiple sources of data to validate study findings. In this study, I intended to
check and confirm interview data with the findings from the document analysis. It was critical to
cross-check Content team member’s description of their role and participation in the content
creation process with the workflow that was established in the “LED Team Stuff” spreadsheet.
Secondly, I needed to self-reflect consistently and honestly through the data gathering
and data analysis phases. As the manager of the Content team, I recognize that I am biased
towards what I expect team members to understand and communicate. Merriam and Tisdell
(2016) challenge researchers to keep “a running record of your interaction with the data,” which
should illuminate the scope of the study, the way it was conducted, and how the data were
MODERN LEARNING REQUIRES MODERN DESIGN 67
analyzed (p. 253). To reign in my known biases, I intended to document each stage of the data
collection and analysis process, describing my approach, my decision-making process, and the
results. I also committed to reviewing these notes and altering my approach when I recognized
that my bias influenced my analysis of the data.
The third strategy I employed was to engage in a consistent, albeit informal, peer review
and reflection with both colleagues and classmates throughout the study. Merriam and Tisdell
(2016) recommend this approach to validate preliminary data and findings through the utilization
of a peer network. As a second-year graduate student, I have grown to respect the insight and
expertise of my cohort. I intended to utilize these professional relationships as a validity check to
my research, engaging in peer review at critical junctures of my study. I also engaged my
professional peer network, a group of experience practitioners, to gut-check my findings and
review my process. Further, as I did not conduct the interviews myself, I debriefed the interview
sessions with the interviewer, James McGee, immediately after they were conducted and shared
my findings with him later to confirm their credibility. Ultimately, the credibility and reliability
of a study must present findings that “ring true to readers, practitioners, and other researchers”
(Merriam & Tisdell, 2016, p. 238). These strategies aided in improving the validity of my
findings and supported me in keeping my biases in check.

MODERN LEARNING REQUIRES MODERN DESIGN 68
Appendix D: Ethics
Merriam and Tisdell (2016) contend that the validity and reliability of a study hinges
upon the ethics of the researcher. This includes both the rigor to which the investigator conducts
the study and the way in which he or she interacts with study participants. To promote the
likelihood that a study is conducted ethically, the federal government requires that institutions of
higher education establish Institutional Review Boards (IRB) to review and approve all new
proposed research, ensuring that ethical standards are met (Rubin & Rubin, 2012).
Glesne (2011) suggests that while the value of informed consent remains a debated topic
in qualitative research, a well-crafted informed consent should communicate that participation is
voluntary, identify components of the study that may affect participant’s personally, and allow
for participants to stop participation at any time. For the purposes of this study, participant’s will
be apprised of their right to privacy and confidentiality through a detailed informed consent.
Additionally, prior to the interviews, I will gain permission from each participant to audio record
interview sessions and will provide them with a digital copy of the transcription afterwards.
As the Chief Learning Officer (CLO) of Grow Learning, I both manage the Content team,
a division of roughly thirteen employees, and am a member of the Executive team. This position
assumes a high level of authority throughout the organization. This study sought to gain the
qualitative feedback of specific members of the Content team as they worked to innovate modern
learning design approaches that supported them in their goal of creating 1,000 new
Microlearning® lessons by the end of 2018. This performance goal was a critical component of
their current and continued employment and will determine the ultimate success of Grow
Learning as the undisputed leader in both the Microlearning® and Learning & Development
space.
MODERN LEARNING REQUIRES MODERN DESIGN 69
Rubin and Rubin (2012) recommend that a researcher be straightforward with study
participants and advocate against deception of any kind. Likewise, the members of the Content
team, who are two degrees of authority removed from the Chief Learning Officer, were fully
aware that the success of Grow Learning depended upon their ability to scale production and
create 1,000 new lessons in 2018. Their participation in the study functioned as an opportunity to
validate their success and confirm their employment directives. Above all, this researcher aimed
to be respectful of the contribution of each participant, honoring their contributions to both the
success of the company and the validity of this study.

MODERN LEARNING REQUIRES MODERN DESIGN 70

Appendix E: Integrated Implementation and Evaluation Plan
Implementation and Evaluation Framework
The development of this implementation and evaluation plan has been informed, entirely,
by the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016), which is based on the
original Kirkpatrick Four Level Model of Evaluation (Kirkpatrick & Kirkpatrick, 2006). This
model establishes four levels as the basis for training evaluation, which must be considered in
reverse for optimal programmatic impact, they are: Level 4 - Results, Level 3 - Behavior
(Transfer), Level 2 - Learning, and Level 1 - Reaction. In general, Levels 1 and 2 provide the
practitioner with internal measurements of quality, whereas Levels 3 and 4 offer data more
aligned to training effectiveness.
Organizational Purpose, Need, and Expectations
To achieve its mission of making Microlearning® a core component of every
organization’s learning strategy, Grow Learning must innovate a new learning design process
that shortens its content creation timeframe while maintaining content credibility and aligning to
the company’s Microlearning® framework. It is important for Grow Learning to analyze and
iterate on its established content creation process to rapidly produce content aligned with the
expectations of its customer-base and its stakeholders. Principally, the creation of agile-like
design methodologies will set the company apart from its competitors, establishing Grow
Learning as an innovator in the learning technology industry and subsequently increasing its
market-share.
Level 4: Results and Leading Indicators
MODERN LEARNING REQUIRES MODERN DESIGN 71
Table 9 details the Level 4 anticipated outcomes, metrics, and methods in terms of
leading indicators for the Content team and Grow Learning as they attempt to reach their
performance goal of creating 1,000 new Microlearning lessons by year end 2018.
Table 9
Outcomes, Metrics, and Methods for External and Internal Outcomes

Outcome Metric(s) Method(s)
External Outcomes
Content-specific customer
feedback collected through
multiple sources
Qualitative feedback
collected via Customer
Success teams
Data collected digitally, at
renewal, or via the platform
NPS data collected via the
platform
Data collected and shared
weekly by Director of
Customer Success

Qualitative feedback
collected by Sales staff
Data collected as part of
prospect demos, win/loss data
in SalesForce, or generated
through SDR function
Increased content
partnerships with leading IP
owners in broader L&D
industry, targeting specific
library use-cases (e.g.:
leadership, diversity,
customer service).
Successful kick-off of content
partnerships that span critical
library collections
Data gathered as part of joint
work with Biz Dev
Win awards based on
content quality and impact
to raise profile of Grow
leadership, board, and
backing venture capital
firms.
Submit for various awards
across L&D industry (e.g.:
Brandon Hall Learning
Excellence, CLO Learning in
Practice, etc.) to demonstrate
content viability and
credibility
Marketing department will
compare submissions with
win/loss data quarterly
Internal Outcomes
MODERN LEARNING REQUIRES MODERN DESIGN 72
Consistent number of
published lessons per
Content Producer per
quarter
The number of lessons
published and refreshed per
Content Producer per quarter
Aggregate data from: 1)
Content Production Manager
(as part of flywheel
documentation), and 2)
Director of Learning
Experience (as part of
publication schedule).
Increased individual and
team productivity because
of collaborative working
sessions
The number of hours spent on
lesson creation during heavy
writing weeks: scoping and
scripting
Aggregate of self-reported
hours worked per program by
Content Producers and as
documented by Director of
Learning Experience

The length of time Content
Producers spent
workshopping lessons with
peers in place of Director of
Learning Experience or
Editorial Manager
Aggregate of self-reported
hours by Content Producers,
Director of Learning
Experience, and Editorial
Manager
Decreased time per
publication cycle
Number of days from scoping
to publication
Data collected by Content
Production Manager, including
by-week and by-program

Level 3: Behavior
Critical behaviors. The critical behaviors required to achieve the performance goal are
directly aligned with the knowledge and motivation influences identified in this study. The
stakeholders of focus are the members of the Content team responsible for supporting the content
flywheel. Team members must demonstrate two critical behaviors, including: keeping a program
on schedule and participating in collaborative opportunities designed to support them throughout
the content creation process.
Table 10
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s) Method(s) Timing
MODERN LEARNING REQUIRES MODERN DESIGN 73
1. Content team
members demonstrate
that they can keep a
program on schedule
throughout the content
creation process
The number of “on-
time” programs,
adherent to the
established, 6-week
schedule
Content Production
Manager (supported by
the Director of
Learning Experience)
will monitor the
progress of each
program
Throughout each
content cycle (6
weeks) and at
weekly thresholds
2. Content team
members participate in
various collaborative,
workshop opportunities
as part of creation
process
CP attendance at
weekly events: CP
office hours, writing
workshop, and 1:1
with Director of
Learning Experience
CP will self-report
attendance at offered,
collaborative meetings.
Team leads will
share-out self-
reported data to
CLO at monthly
standing leads
meeting.

Content team members must rely upon the support and backing of their leaders to
consistently demonstrate the desired critical behaviors. The interactions Content team members
experience with their peers and their supervisors will function as support systems to reinforce
behaviors. Kirkpatrick and Kirkpatrick (2016) contend that successful behavioral drivers are able
to reinforce, encourage, reward, and monitor the individuals tasked with achieving critical
performance goals. Table 11 outlines the drivers anticipated to support new critical behaviors.
Table 11
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical
Behaviors
Supported
Reinforcing
Job aid reinforcing each stage
of content flywheel
Ongoing, reinforced at bi-weekly Content
Roundtable (led by CLO), and shared at
all-hands meeting with entire company.
1
Job aid reinforcing elements of
lesson design as established
collaboratively by Content team
Ongoing, reinforced at bi-weekly
Throwback Thursday events with content
team (led by Editorial Manager)
2
MODERN LEARNING REQUIRES MODERN DESIGN 74
Encouraging
Feedback and coaching from
team leads, Content Production
Manager, and CLO related to
flywheel and learning design
approach
Weekly 1:1s and ongoing 1, 2
Rewarding
Recognize team members that
embody the desired critical
behaviors
Ongoing: CLO may identify team member
by name in weekly “snippets” email to
CEO and in-person at bi-weekly, “all-
hands” meetings with entire company
1, 2
Monitoring
Standing item may be added to
recurring 1:1s and team
meetings
Ongoing: weekly, bi-weekly, monthly, and
quarterly cadence
1, 2

Organizational support. Clark and Estes (2008) contend that ultimately, performance
goals will prove successful or not based on early accounting for the impact of the organizational
culture. This suggests that the proposed recommendations must be supported by Grow Learning
to be effective and long-term impactful. Accordingly, the organization will support the adoption
of critical behaviors by acknowledging team member incremental success, by including
opportunity to demonstrate and discuss behavior into established processes and systems (e.g.:
1:1, all-hands meetings, etc.), and by supporting leaders in their attempts to cultivate new
behaviors in their teams.
Level 2: Learning
Learning goals. Following the implementation of the proposed solutions, members of
the Content team will be able to demonstrate the following:
1. Correctly describe the various stages of the content creation process. (D)
MODERN LEARNING REQUIRES MODERN DESIGN 75
2. Describe the learning principles inherent in the creation of Microlearning content. (D)
3. Contribute to the team’s collective knowledge of building virtual content. (D)
4. Plan and monitor their own strategies for keeping a program on schedule as it moves
through the content flywheel. (M)
5. Identify strategies for supporting their peers through the content creation process. (M)
6. Challenge themselves to try new approaches and tactics with each program. (M)
Program. Clark and Estes (2008) reinforce the power of process and procedure in their
categorization of the organization as one of three elements that either encourage or deter
employees from achieving their performance goals. Indeed, they contend that an organization is
propped up and propelled by its internal systems and processes and that the backbone of a
company may be found in its daily routines and interactions. Accordingly, rather than a single
learning event or program, the proposed solution for this study will fully integrate into the
established systems of Grow Learning so that the collective and collaborative work of the
Content team may achieve the greatest impact. The team will be responsible for generating,
documenting, and agreeing on all new approaches as they demonstrate their knowledge, skills,
abilities, confidence, and commitment to improve the content creation process.
The proposed solution will span a six-month period of time, during which time
prescribed activities, interactions, and assessments will take place. These methods will provide
immediate support for the Level 2 learning goals, in addition to anticipated learning outcomes
identified in Level 3, and ultimately the metrics for success detailed in Level 4. Specifically,
methods will be integrated into existing organizational systems to reinforce the value of the
solution and to deepen the team's commitment to learning, including:
MODERN LEARNING REQUIRES MODERN DESIGN 76
• Integration of activities into established team meetings (e.g.: Content Roundtable,
Throwback Thursday, LED Team meeting) to include various activities, knowledge
checks, and share-outs.
• Addition of agenda item to all 1:1 meetings between team members and their leads
• Inclusion of team ideas into company events wherein Content team members may
document, share out, and gain recognition for new team learnings, including: all-hands,
weekly Gazette sent by the CEO, and authored blog posts as part of the company blog.
• Post-mortem of each program publication during which team members conduct a
comprehensive review of the previous cycle, identifying new learnings, approaches, and
opportunity for further innovation.
At the conclusion of the six-month period, Content team members will be asked to
reflect on the impact of the program in a facilitated discussion led jointly by the CLO and sub-
team leads. Additionally, all team members will be asked to complete an end-of-program survey.
Evaluation of the Components of learning. To validate the impact of these solutions,
the knowledge, skills, and abilities of Content team members will be evaluated. Additionally,
proposed evaluation methods will measure their individual commitment to learn and apply new
concepts. Most importantly, team members must stay committed to experience the value of the
proposed solutions. Their commitment to each other and to their collective success emerged as a
significant finding in the analysis and thus, the proposed evaluation methods build from that
foundation, emphasizing further team collaboration to successfully achieve their performance
goal. Table 12 lists the various evaluation methods and recommended timing.
Table 12
Evaluation of the Components of Learning for the Recommended Solutions
MODERN LEARNING REQUIRES MODERN DESIGN 77
Methods or Activities Timing
Declarative Knowledge “I know it.”

Knowledge checks on key learning design elements
during standing team meetings.
Weekly at LED meetings, based on
stage of cycle.
Facilitated discussions of elements of agile learning
design, usage, success, and impact.
Bi-weekly at Throwback Thursdays
and Content Roundtable.
Attitude “I believe this is worthwhile.”

Discussion item added to weekly 1:1 to gauge attitude. Weekly at pre-scheduled 1:1s.
Confidence “I think I can do it on the job.”

Share out at standing team meetings on new strategies,
approaches, or tactics.
Weekly or bi-weekly at pre-
scheduled meetings.
Write-ups of new strategies for company-wide
dissemination resulting in presentation at all-hands, blog
post, formal documentation.
Review monthly
Pair proven CPs with newly hired CPs to demonstrate
approach and coach through first program
As new Content Producers are
hired
Impromptu “teach it” moments at standing meetings, in
which team members are asked to share specific
learning from previous week.
Bi-weekly Content Roundtable
Schedule consistent writing workshops, facilitated by
Editorial Manager, open to all team members to
collaborate and share ideas
Weekly
Commitment “I will do it on the job.”

Formal post-mortem and review after each program
publication - facilitated discussion of new approaches
and learnings
At the close of each 6-week cycle
End of program discussion and reflection At the end of the six-month period

Level 1: Reaction
Successful programs are not only architected thoughtfully, they are built with evaluation
in mind, giving participants frequent and easy opportunity to share their reactions, engagement,
MODERN LEARNING REQUIRES MODERN DESIGN 78
and perceived relevance. Table 13 details the components that will measure participant reactions
to the proposed program.
Table 13
Components to Measure Reactions to the Program.
Methods or Tools Timing
Engagement

Observation of team member involvement and interaction at
standing meetings
Bi-weekly
Relevance

Pulse checks via Content team slack channel Weekly
Team lead observation and check-ins during standing 1:1s Weekly
Customer Satisfaction

Survey Mid-program (3 months) and
End of program (6 months)

MODERN LEARNING REQUIRES MODERN DESIGN 79
Appendix F: Program Evaluation Survey
In addition to informal check-ins, meetings, and conversations, and to formally capture
the reactions and growth of Content team members throughout the program, the following survey
will be distributed at the mid-point of the program (3 months) and at the program’s close (6
months). The survey targets the needed engagement, attitude, skill, knowledge, and commitment
the Content team must demonstrate to persist in innovating new approaches to content creation.
The survey will be distributed using Google forms and collected anonymously. Participants will
have the opportunity to share their responses and feedback at standing meetings and weekly 1:1s.
Program Survey
Strongly
Agree
Agree Neutral Disagree
Strongly
Disagree
I am confident in my ability to correctly
define the stages of the content flywheel

I can describe the learning principles
inherent in our content creation

I am comfortable engaging with the
team to discuss the content flywheel

I am encouraged to ask questions about
the content flywheel

I have identified and developed
strategies to successfully perform the
functions of my role

I support my team members throughout
the content creation process

I challenge myself to innovate and share
new approaches to content creation

You are welcome to include any additional feedback here:

Abstract (if available)

Abstract To keep pace with the rapidly changing workplace, corporate learning practitioners must break with outdated models to innovate agile, responsive, and iterative design methodologies. Too often, the creation of learning content is bound by antiquated, waterfall methods that protract content development timelines, limit opportunities for innovation, and result in stale and ineffective training. This study centered on the journey of the Content team at Grow Learning, a learning technology company, as they attempted to innovate their content creation process to achieve their performance goal of building 1,000 Microlearning® lessons over the course of 2018. The study aligned to the Clark and Estes (2008) gap analytical framework to illuminate the knowledge, motivation, and organizational influences impacting the team. Research questions centered on: revealing stakeholder knowledge and motivation related to the goal, identifying the interaction between organizational culture and team knowledge and motivation, and identifying any resulting solutions. A qualitative inquiry method was employed, which included eight, one-time interviews with members of the stakeholder group. While knowledge influences were not validated in the study, significant motivation findings included the team’s profound reliance on a blend of collaboration and autonomy to reach their goals, in addition to strong evidence that psychological safety was a catalyst for the team to innovate on the established content creation process. Recommendations were strategically crafted to dovetail within the organization’s processes and procedures, including reinforcement at weekly 1:1s, team meetings, and all working sessions. This ensured that recommended supports would both validate and perpetuate the innovated practices developed by the team while also reinforcing the collaboration, trust, and psychological safety so critical to the team’s success.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Online graduate-level student learning and engagement: developing critical competencies for future leadership roles: an evaluation study

PDF

The moderating role of knowledge, motivation, and organizational influences on employee turnover: A gap analysis

PDF

Creating changemakers: integrating social innovation and service-learning to empower student voice and bolster college, career, and civic readiness

PDF

Improving pilot training by learning about learning: an innovation study

PDF

Quality literacy instruction in juvenile court schools: an evaluation study

PDF

Analyzing the implementation of a learning management system into a post-merger & acquisition organization and its effects on sales performance (improvement model)

PDF

The happiness design: an innovation study

PDF

A qualitative study of educators’ attitudes towards blended learning institutional adoption

PDF

Civic learning program policy compliance by a state department of higher education: an evaluation study

PDF

Managers’ learning transfer from the leadership challenge training to work setting: an evaluation study

PDF

Manager leadership skills in the context of a new business strategy initiative: an evaluative study

PDF

Enhancing transfer of harassment prevention training into practice

PDF

Judaic/Israel studies curriculum for the 21st-century learner – An innovation study

PDF

A gap analysis of course directors’ effective implementation of technology-enriched course designs: An innovation study

PDF

Supporting women business owners: the Inland Empire Women’s Business Center: an evaluation study

PDF

The role of international school teacher leaders in building leadership capacity within their teams

PDF

A framework for customer reputation evaluation: an innovation study

PDF

Sustained mentoring of early childhood education teachers: an innovation study

PDF

Application of professional learning outcomes into the classroom: an evaluation study

PDF

Building employers’ capacity to support competitive employment for adults with autism: a promising practice study

Asset Metadata

Creator Salomonsen, Summer Rivers (author)

Core Title Modern corporate learning requires a modern design methodology: an innovation study

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Publication Date 01/31/2019

Defense Date 11/12/2018

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag ADDIE,agile,content creation,learning and development,learning content,microlearning,modern learning design,OAI-PMH Harvest

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Datta, Monique (committee chair), Canny, Eric (committee member), Picus, Lawrence (committee member)

Creator Email srsalomo@usc.edu,summersalomonsen@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-115955

Unique identifier UC11675674

Identifier etd-Salomonsen-7036.pdf (filename),usctheses-c89-115955 (legacy record id)

Legacy Identifier etd-Salomonsen-7036.pdf

Dmrecord 115955

Document Type Dissertation

Format application/pdf (imt)

Rights Salomonsen, Summer Rivers

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA