University of Southern California Dissertations and Theses

Online graduate-level student learning and engagement: developing critical competencies for future leadership roles: an evaluation study

(USC Thesis Other)

Online graduate-level student learning and engagement: developing critical competencies for future leadership roles: an evaluation study

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

1

ONLINE GRADUATE-LEVEL STUDENT LEARNING AND ENGAGEMENT:
DEVELOPING CRITICAL COMPETENCIES FOR FUTURE LEADERSHIP ROLES:
AN EVALUATION STUDY

by

Cynthia McCloud

_______________________________________________________________________

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

May 2019

Copyright 2019 Cynthia McCloud

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

2
ACKNOWLEDGEMENTS
After spending hundreds of hours writing my dissertation, it seems now there are no
words to express the depth of my gratitude for all of those who supported me on this journey.
With a humble heart, I would like to start by thanking God for reminding me to be thankful
during the peaks, for carrying me through the valleys, and for giving me faith to reach my goals.
To the most remarkable man who walked into my life 35 years ago and convinced me
that, in fact, all of my dreams could come true. To this day, you have never missed an
opportunity to lift me up. You have traveled every minute with me on this dissertation journey
as my closest advisor, my greatest sounding-board, and my strongest ally. You managed our
home, our family, and our friends while I sat in my office with my head in the books, never once
complaining about the time I took away from you. My heart is full and my love for you has
never been greater. I am thankful for every minute that you served and for the ways you loved
me.
I would also like to thank the many people who supported me and cheered my on through
this process. Dr. Datta — as my committee chair you inspired me, you made me laugh, and you
held me accountable at the same time, which is the definition of a true servant leader. Dr.
Bhambri — you believed in me and, though a man of few words, your guidance and
contributions will be forever treasured. Dr. Green — you were recruited to my committee in the
most unusual way and, after saying yes, you graciously shared your wisdom and support. Dr.
Chrystal — you were my greatest cheerleader, you listened to my highly emotional stories, you
provided guidance at every turn, and you celebrated every victory with me as if they were your
own . . . thank you, my friend.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

3
Lastly, I would like to express my deepest appreciation to my personal board of directors
— all of the women in my life who encouraged my zest for life and who lifted my heart. To Sue
. . . who has been my greatest sponsor and most devoted confidant, and who always encouraged
me to have courageous conversations. To Kim . . . who loved me and cheered me on like only a
sister can and who celebrated my every success with pride, passion, and love. To Ria . . . who
protected my heart and filled my soul with motherly love and kindness. To Katie . . . who
prayed for me and patiently waited for me to complete this journey so we could spend more time
together in the next seasons of our lives. I am thankful that God put each of you in my life and
look forward to the next chapters when I can return all of the favors and hugs, serve you in
special ways, and celebrate our lives together.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

4
TABLE OF CONTENTS
Acknowledgements 2
List of Tables 6
List of Figures 7
Abstract 8
Chapter 1: Introduction 9
Introduction of the Problem of Practice 9
Organizational Context and Mission 10
Organizational Goal and Measures 14
Related Literature 14
Importance of the Evaluation 17
Description of Stakeholder Groups 18
Stakeholder Group for the Study 22
Purpose of the Project and Questions 23
Methodological Framework 24
Definitions 24
Organization of the Dissertation 25
Chapter 2: Review of the Literature 27
Graduate-Level Online Learning 28
Critical Competencies and Graduate-Level Students in Online Learning 33
Impactful Elements of Student Engagement in Online Learning 36
Role of Stakeholder Group of Focus 43
The Clark and Estes (2008) Gap Analysis and Conceptual Framework 43
Stakeholder Knowledge, Motivation, and Organizational Influences 45
Conceptual Framework: The Interaction of Stakeholders’ Knowledge and 64
Motivation and the Organizational Context
Summary 68
Chapter 3: Methodology 69
Participating Stakeholders 70
Data Collection and Instrumentation 73
Data Analysis 76
Credibility and Trustworthiness 77
Validity and Reliability 78
Ethics 78
Limitations and Delimitations 80

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

5
Chapter 4: Results and Findings 81
Participating Stakeholders 82
Findings 83
Research Question 1 84
Research Question 2 102
Research Question 3 106
Document Analysis and Findings 116
Synthesis 120
Chapter 5: Recommendations 126
Recommendations for Practice to Address KMO Influences 127
Integrated Implementation and Evaluation Plan 137
Strengths and Weaknesses of the Approach 156
Limitations 156
Future Research 157
Revisiting the Conceptual Framework 158
Conclusion 161
References 164
Appendices 177
Appendix A: Interview Protocol — UFL Online Learning Team 177
Appendix B: Interview Protocol — Dean’s Cabinet (Supplemental 179
Stakeholder Group)
Appendix C: Memorandum of Understanding (Sample) 181
Appendix D: Strategic Planning Process Evaluation And Assessment — 182
Post-Meeting Survey
Appendix E: Strategic Planning Process Evaluation And Assessment – 183
Follow-Up Survey

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

6
LIST OF TABLES
Table 1. Organization of Focus — Goals 21
Table 2. Knowledge Influences on Organization of Focus Goal 49
Table 3. Motivational Influences on Organization of Focus Goal 55
Table 4. Organizational Influences on Organization of Focus Goal 62
Table 5. Knowledge and Motivational Influences 84
Table 6. Organizational Influences 106
Table 7. UFL Online MBA Student Profiles by Cohort 125
Table 8. Summary of Knowledge Influences and Recommendations 127
Table 9. Summary of Motivation Influences and Recommendations 129
Table 10. Summary of Organization Influences and Recommendations 133
Table 11. Outcomes, Metrics, and Methods for External and Internal Outcomes 140
Table 12. Critical Behaviors, Metrics, Methods, and Timing for Evaluation 144
Table 13. Required Drivers to Support Critical Behaviors 145
Table 14. Evaluation of the Components of Learning for the Program 150
Table 15. Components to Measure Reactions to the Program 151

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

7
LIST OF FIGURES
Figure 1. UFL Online Learning Department organization chart 12
Figure 2. UFL Online Learning Team conceptual framework 65
Figure 3. Online Learning Organization strategic plan — progress and accountability 154
report
Figure 4. UFL Online Learning Team conceptual framework 159

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

8
ABSTRACT
Traditional business schools are facing a radical transformation as they serve the next generation
of students, face new technology-based competitors, and ensure graduates have all of the skills
and competencies they need to step into business roles. This evaluation study explored how the
Online Learning Team at the University of Future Leaders (UFL), a pseudonym for a large
private research university business school in the western United States, took an important step
forward in developing the next generation of business leaders. Using the Clark and Estes (2008)
gap analysis as the conceptual framework, this qualitative study evaluated the Online Learning
Team’s organizational performance and determined the knowledge, motivational, and
organizational influences that impacted the team’s ability to understand and impact student
learning and engagement in an online, graduate-level setting. The key findings from this study
revealed that the Online Learning Team was both knowledgeable and motivated about
developing new, innovative solutions for improving student learning and engagement, and
clearly understood the critical competencies their graduate students need for future success. The
findings also identified several opportunities for organizational growth including the need to
develop and communicate formal goals and key performance measures, create tracking systems,
build opportunities for faculty development, and improve awareness with university leaders. The
study concludes with a recommendation to develop a strategic plan that addresses each of these
areas and an evaluation plan which, once implemented, will help the Online Learning Team
assess strategic outcomes.
Keywords: online learning, critical competencies, student engagement

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

9
CHAPTER 1
INTRODUCTION
Introduction of the Problem of Practice
Traditional business schools are facing a radical transformation as they learn to serve the
next generation of students, to face new technology-based competitors, and to ensure graduates
have all of the skills they need to step into business roles. One of the main challenges for
business schools is that teaching methodologies and pedagogies have not kept pace with industry
growth and change (Cress, Yamashita, Duarte, & Burns, 2010). As a result, business leaders are
concerned that business schools are graduating students without the appropriate skills and
competencies required for positions in business today. In a Gallup (2014) study, only 10% of the
623 business leaders interviewed believed that graduates had the knowledge and competencies
they need for roles in business. Pfeffer and Fong (2002) referenced a large body of research
related to the issue that curriculum taught in business schools is not tied to the competencies
required for success in the business world. Institutions that do not transform may be under threat
(Thomas, Lee, Thomas, & Wilson, 2015), and other institutions that are not among the most
prestigious or well-funded may not survive (Carey, 2015; Friga, Bettis, & Sullivan, 2003).
The University of Future Leaders (UFL) is a pseudonym for the business school at a large
private research university in the western United States which focuses on developing the next
generation of business leaders. The online Master of Business Administration (MBA) degree
was launched at UFL in 2015 by a small team of people committed to serving the needs of future
students. They will be referred to as the UFL Online Learning Team. This study will evaluate
the UFL Online Learning Team’s organizational performance as it relates to the problem of
practice to break outdated traditional pedagogies and improve student learning. It will also
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

10
evaluate how well the team is preparing future leaders with the most relevant competencies for
future roles, so they are able to respond to the impending avalanche of change (Barber, Donnelly,
Rizvi, & Summers, 2013; Cress et al., 2010). To achieve these objectives, the UFL Online
Learning Team will need to identify, measure, and improve student learning and engagement for
the online MBA program. Then, the UFL will be in a position to grow new enrollment for the
program, retain and serve students who are currently enrolled, and reach its organizational goal
to prepare graduate students with the most relevant competencies for future leadership roles in
business. The results will be measured by how well the organization performs in order to be
recognized and ranked among the top five online MBA universities in the next few years.
This problem is important to address because business students who are hired for future
leadership positions will be required to have advanced skills to handle both domestic and global
challenges. Cornuel (2007) noted that students will not be able to compete solely with
theoretical knowledge; they must develop practical skills such as global problem solving, critical
thinking and analysis, and entrepreneurial capabilities. These students will look to the most
innovative business schools and student-centered faculty who offer new and creative options for
learning and are willing to help them prepare for the current and future disruptive and rapidly
growing markets. In order to both survive and thrive, business schools will be required to
engage these students by providing solutions to the increased demand for on-line learning
technologies, innovative classroom access options, and cost-effective accredited degrees
(Kimberly & Bouchikhi, 2016).
Organizational Context and Mission
The University of Future Leaders was founded in the early 1900s and has a long history
of developing innovative and entrepreneurial business leaders. The university offers several
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

11
graduate school options including full-time, part-time, and online MBA programs for both early
career and mid-career professionals. In addition, there are multiple learning environments
available to both undergraduate and graduate students including case competitions, advanced
leadership programs, and university presentations across multiple business departments and
disciplines. There are approximately 400 students enrolled in the full-time graduate-level
programs and just over 500 enrolled in the part-time programs (U.S. News Grad Compass,
2017). Just over 30% of the students are female and approximately 30% of the overall student
population is international, representing more than 100 countries (UFL website, 2017). The
vision of the UFL is to transform from an 18th century academic education model to a 21st
century business education model that differs dramatically in content, professors, delivery, and
how it addresses student needs. The mission of the UFL is to provide an innovative and
entrepreneurial learning environment that mirrors business practices, while expanding thought
leadership, identifying key global opportunities, and exploring critical issues (UFL Strategic
Plan, 2015).
In keeping with the vision and mission, the UFL decided to launch its first online MBA at
the business school in 2015, and a separate Online Learning Department was established. A
Vice Dean was assigned to the new department, with the team reporting directly to the Dean of
the business school. As a result, the department does not fall under the umbrella of the Graduate
School, where all other MBAs are managed, at least in the short term. They were encouraged to
invest in advanced technology and empowered to build a highly skilled and experienced team,
including an Assistant Dean of Online Learning, an Online Media and Technology team, an
Instructional Design Team, and a Student Advising and Services Team. Figure 1 reflects the
organization chart for the Online Learning Team.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

12

Figure 1. UFL Online Learning Department organization chart

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

13
The Online Learning Team had strong support from the university and the business
school leaders, along with the resources, processes, and equipment needed to launch this new
department and achieve its goals. The first cohort of students graduated in May 2017. At that
time, the school was ranked 12th (top 5%) in its first year of being eligible for evaluation (UFL
website, 2017). Key strategic principles included a focus on identifying global challenges and
innovative approaches, developing rigorous and impactful research, and making education
accessible to all qualified students. The UFL was dedicated to creating a diverse learning
environment in order to develop highly impactful, well-rounded professionals with a global
mindset (UFL website, 2017). If the UFL Online Learning Team members, who were
responsible for addressing the aforementioned problem of practice, were successful in
developing and executing new, competitive, and innovative online education strategies, they
would help the UFL achieve its mission to graduate prepared future global leaders.
Since the online MBA was a relatively new program, much of the planning by the Online
Learning Team (OLT) and many of the resources were appropriately focused on launch and
implementation. As the OLT looks to the future, it will need to address several issues and
opportunities, as well as implement new strategies that have been discussed in new academic
models for the future. The team will need to address issues associated with the pressures to
abandon ineffective traditional approaches and develop more customized and engaging learning
models (Kimberly & Bouchikhi, 2016). They will need to determine the best strategies for
hiring and developing a blended team of both clinical and research-based faculty who are
focused on strong student engagement and effective learning methods to achieve the learning
outcomes that serve the needs of global business leaders (Schoemaker, 2008). Finally, the team
will need to take advantage of innovative solutions to create a more engaging online MBA
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

14
program that aggressively competes in a high-growth market alongside non-traditional providers
(Cornuel, 2007). These strategies will also help the university reach its overall organizational
goal.
Organizational Goal and Measures
This study will evaluate the UFL Online Learning Team’s organizational performance as
it relates to the problem of practice to break outdated traditional pedagogies and improve student
learning and engagement. In order to serve the needs of business leaders and attract future
students, the organizational goal for the UFL is to develop a strategic plan to identity, measure,
and improve online student learning and engagement that results in preparing graduate students
with the most relevant competencies for future leadership roles in business. This goal relates
directly to the problem of practice and the UFL mission by being recognized among the top
United States universities as one of the leaders in providing an innovative and entrepreneurial
online learning environment. The achievement of the UFL’s goal will be measured by annual
growth in rankings across several respected publications including U.S. News & World Report,
the Princeton Review, and Forbes. The team’s specific objective is to be recognized in the top
five online MBA programs in the United States university rankings by the end of 2019 and
number one by the end of 2020. The UFL Dean and the Dean’s Cabinet established this measure
during a strategic planning session in 2015.
Related Literature
The 18-20th century education model of knowledge-creation through courses delivered
by traditional faculty in traditional on-campus classrooms is no longer the only viable model.
New technology-enabled competition with more effective student-engagement practices will
challenge the status-quo in the future (Bouchikhi & Kimberly, 2014). Carey (2015) noted that
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

15
“education . . . is far more complicated than anyone realized, and standard hybrid universities
aren’t equipped or motivated to do the extremely difficult work of helping students with vastly
different individual neural patterns to achieve the same rigorous learning goals” (p. 82). In order
to compete, these traditional universities will be challenged to use both technology and data to
customize a more relevant and engaging learning environment.
Roger Martin, former dean of the University of Toronto’s Rotman School of
Management, predicted that the current business school model will eventually collapse if it does
not evolve to compete in this new, rapidly changing market (Byrne, 2015). Leiber (2016) noted
that according to an ASCSB survey of 265 business schools, overall enrollment in United States
MBA programs has declined by 11% since 2009, and that current enrollment is being propped up
by an increase in the number of international students (Byrne, 2015).
One of the key issues is that some research-centric faculty are unwilling or unable to
engage students and help them develop relevant competencies, and this has led to new,
competitive, and technology-driven solutions in the market (Dua, 2013). In an interview with
Rosabeth Moss Kanter of Harvard Business School, Bisoux (2016) noted current trends that will
have the most impact on business schools in the future. Kantar suggested that schools that are
not the top-rated institutions will struggle to survive if they do not encourage faculty to
experiment with new customized learning models that include experiential learning and online
components to more effectively engage students. Kimberly and Bouchikhi (2016) noted the
important disruption associated with new learning technologies, such as Massive Open Online
Courses and their potential to create an environment with expanded and leveraged capacity and
reach for faculty, as well as access to new forms of engaging learning options for students.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

16
Numerous studies have suggested that business schools are under scrutiny and are being
challenged to reinvent themselves or face the consequences of becoming irrelevant (Kimberly &
Bouchikhi, 2016; Pfeffer & Fong, 2002; Thomas, Lorange, & Sheth, 2013; Wisbauer, 2017).
Traditional business schools have been criticized for faculty who are too focused on research
than on finding ways to engage their students, improve learning, and teach new skills required
for success in business. This is the result of an overall disconnect of the faculty who do not have
relevant experience in the business world or are not willing to work closely with employers to
understand the changing landscape (Lorange & Thomas, 2016). While this challenge exists,
employers who do collaborate directly with educators have greater success with recruiting
graduates who are prepared with the skills and competencies they require (Barton, Farrell, &
Mourshed, 2012).
Some business school leaders and faculty struggle to recognize the value of collaborating
directly with employers about their needs, however, those who do find a way to connect help
both the students and the employers, as well as the university. Barton et al. (2012) ran three
parallel surveys with 4,656 students, 908 education providers, and 2,832 employers in nine
countries. Overall, 40% of the employers who had open positions reported the most significant
issue was the lack of the necessary competencies among graduates. They also noted that 42% of
these employers had never worked with education providers and were disappointed with the
skills they observed in graduates joining their companies. Conversely, 31% of the surveyed
employers reported that they were actively engaged with education providers to help develop
relevant curriculum. Among this more active segment, 75% reported that their efforts were
highly effective at helping to hire graduates with the right skills. If educators do not find ways to
collaborate with employers to better address their needs for skilled graduates, traditional
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

17
academic business schools will continue to experience declines in enrollment and will begin
seeing more competition from technology-driven schools (Frey, 2013).
Rudestam and Schoenholtz-Read (2010) posited that the continued advancement and
availability of new technology in online education, along with a growing focus on advancing
new forms of pedagogy, help empower students and improve student engagement. In addition,
there is growing research about how to keep students engaged and enhance their learning so they
build the important competencies they need for future roles in business. This research includes
the impact of metacognitive control and motivation to learn (Mayer, 2011), social engagement
and collaboration among students (Dixson, 2010; Rudestam & Schoenholtz-Read, 2010), faculty
and student engagement (Baker, 2010; Dixson, 2010), and more interactive technology and
learning tools (Rueda, Benitez, & Braojos, 2017). The UFL Online Learning Team has an
important opportunity to expand and improve their use of new research and technology in the
online MBA program to enhance overall student engagement and create a more engaging
learning environment, while also strengthening educational outcomes (Carey, 2015).
Importance of the Evaluation
It is important to evaluate the organization’s performance in relationship to the
organizational performance goal of improving online student learning and engagement and
preparing graduate students with the most relevant competencies for several reasons. If the
organization does not respond to both the needs of future business students and business leaders
with new innovative learning solutions, it will see continued declines in enrollment and struggle
to compete in this rapidly changing and competitive market. Wilson and Thomas (2012) argue
that if business schools do not respond, they could become “irrelevant and unnecessary
institutions” (p. 11). By evaluating the UFL’s current online MBA program and assessing its
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

18
future needs, this study will help the UFL Online Learning Team gather formative data about
how to address the issues associated with the pressures to abandon ineffective traditional
approaches and grow more customized and engaging learning models (Kimberly & Bouchikhi,
2016). It will also assist in hiring and developing a blended team of both clinical and research-
based faculty who are focused on strong student engagement and encourage students to be “co-
creators of the educational content” and learning outcomes (Schoemaker, 2008, p. 128). Finally,
this study will help identify solutions to create a more future-focused and engaging online MBA
program that aggressively competes in a high-growth market alongside non-traditional providers
and global education solutions (Cornuel, 2007).
Description of Stakeholder Groups
A stakeholder is someone who benefits from the overall health of the organization and
who often contributes to the key decisions regarding growth and relevant deliverables for the
end-users or customers. Companies that include key stakeholders in both the development of the
vision and the leadership of the company will have a unique competitive advantage over those
that are not communicating with their stakeholders (Wheeler & Sillanpää, 1998). The key
stakeholder group assessed in this study will be referred to as the Online Learning Team (OLT).
The other four stakeholder groups that directly contribute to, and benefit from, the achievement
of the OLT’s goals are the Dean’s Cabinet (organizational support), the faculty (professors), the
employers (investor companies), and the students (targeted customers).
The organization of focus is the UFL Online Learning Team, which is part of the Online
Learning Department at the university’s business school. The UFL Online Learning Team is
made up of the Vice Dean for Online Graduate Programs and the supporting team of
administrators, many of whom have worked in both academia and the corporate world. The UFL
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

19
Online Learning Team contributes to the UFL’s goal by ensuring the UFL online value
proposition, products, and services address the needs of both the employers and the graduate-
level students, no matter what their backgrounds or where they are in their education journey.
The UFL Online Learning Team has the chief responsibility for achieving the organization’s
performance goal and will benefit directly from the evaluation of the UFL’s current online MBA
program. This team will also be responsible for making decisions using the evaluation data to
address the issues. While the UFL Online Learning Team is directly responsible for addressing
and managing all of the areas used to rank the best online MBA programs (e.g., student
engagement, faculty credentials and training, student services and technology, peer reputation,
and admissions selectivity), this study will focus specifically on student learning and
engagement. Improving student learning and engagement will help the UFL Online Learning
Team achieve its objective to be ranked at the top of all online MBA programs in the United
States. In addition, the UFL Online Learning Team will achieve its mission of providing an
innovative and entrepreneurial learning environment that mirrors business practices while
expanding thoughtful leadership, identifying key global opportunities, and exploring critical
issues (UFL Strategic Plan, 2015).
The second key stakeholder is the Dean’s Cabinet which is an organization that provides
the primary support and leadership for the Online Learning Team. This group includes the Dean
of the business school at the UFL and seven members on the Cabinet from multiple areas of the
business school. The Cabinet is responsible for making decisions about the overall strategic
direction of the business school and new growth opportunities, like the online learning programs.
This stakeholder will benefit from the evaluation of the online MBA program by learning the
most effective ways to make decisions about organizational resources and understand key
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

20
strategic issues and opportunities for the future. The Dean’s Cabinet members provide the
organizational leadership and support for the online MBA.
The third key stakeholder is the faculty. This stakeholder group is made up of all faculty
who currently teach in the UFL online MBA program. This stakeholder group contributes to the
UFL goals through executing the plans and delivering the online coursework associated with
each class. This stakeholder will benefit from the evaluation of the UFL’s current online MBA
program by learning: how to restructure traditional courses and develop relevant, non-traditional,
online courses, how to use technology to create better online learning environments, how to
improve and increase their own use of technology, and how to better engage students in their
learning experience.
The fourth key stakeholder is the employer. This stakeholder group is comprised of
senior level leaders who employ thousands of people across the country in multiple verticals
including retail, consumer packaged goods, and technology among others. This stakeholder
contributes to the UFL goals through guiding the UFL Online Learning Team on key
expectations for specific competencies required for students who will be groomed for both
domestic and global leadership roles in their companies. The employers benefit from the UFL
goals since they hire students who are immediately employable after graduation and have been
exposed to new, more relevant and engaging curriculum, and leading-edge technology. In the
case where these employers fund the tuition for executive students already employed, they will
benefit by gaining quick payback and a strong return on investment (Ayling, Price, Tucker, &
Vellner, 2015).
The fifth key stakeholder is the student, who is also considered one of the important
customers. The targeted student audience consists primarily of adult, mid-career, and executive-
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

21
level students returning to school for an advanced degree or looking to improve and grow their
professional skills. In addition to the development of both critical soft and hard skills, the
students also benefit from the strong network of alumni and associates. This, in turn, supports
strong rankings for the UFL through student engagement and strong feedback on student services
and technology.
Table 1 reflects the organizational mission, the performance goal, and the key
performance measure for the organization.

Table 1
Organization of Focus — Goals
Organizational Mission
The mission of the University of Future Leaders (UFL) is to transform from an 18th century
education model to a 21st century business education model that differs dramatically in
content, professors, delivery, and serving student needs. The mission of the UFL is to
provide an innovative and entrepreneurial learning environment that mirrors business
practices while expanding thought leadership, identifying key global opportunities, and
exploring critical issues.
Organizational Performance Goal
By September 2019 the UFL Online Learning Team will develop a strategic plan to identity,
measure, and improve online student learning and engagement that results in preparing
graduate students with the most relevant competencies for future leadership roles in
business.
Organizational Performance Measure
Be recognized in the top five online MBA programs in the U.S. university rankings by the
end of 2019, and reach number one by the end of 2020 as measured and reported by
respected publications such as U.S. News & World Report, Bloomberg, and others who
measure online university programs.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

22
Stakeholder Group for the Study
While an analysis of each of the four stakeholder groups would result in a more complete
assessment, the stakeholder group who will be the organization of focus for this study is the
Online Learning Team (OLT) at the University of Future Leaders. The team was created in 2014
when the new online MBA was being designed at the business school. This team includes the
Vice Dean for Online Graduate programs, who provides leadership and strategic direction for the
team, the Director/Assistant Dean, and three operational teams led by Associate Directors. The
overall team of 13 members is responsible for Instructional Design (five members), Online
Media and Technology (four members), and Student Services (two members). These members
are ultimately responsible for achieving both the organizational performance goal tied to
developing a strategic plan for improving student learning and engagement, and the stakeholder
group performance goal to be ranked in the top five online MBA programs in the U.S. by the end
of 2019.
The Online Learning Team was chosen as the key internal stakeholder because they have
front-line responsibility for the results and can directly affect the outcomes used to rank online
MBA programs: student engagement, admissions selectivity, peer reputation, faculty credentials
and training, and student services and technology. The Online Learning Team is responsible for
all of these areas including: working closely with the faculty to design curriculum and develop
syllabi that supports strong student engagement; leading all decisions about which student they
admit; working both internally and externally to promote and develop relationships with other
universities to build the university’s brand and credibility as an online education leader; making
recommendations regarding both the hiring and training of new faculty; and leading an internal
team who provides all student services and relevant technology for the online learning programs.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

23
The organizational performance goal was determined by the Vice Dean of Online
Learning and the Dean’s Cabinet. The performance goal was set based on the business school’s
overall strategic growth goals (UFL Strategic Plan, 2015). Evaluating the organization’s
performance and achieving these goals will help the Online Learning Team understand how to
best educate and serve future leaders by developing new online strategies to address evolving
student demands, attract highly effective and engaging faculty, and serve the ever-changing
needs of business partners and employers. The risk for not achieving this goal is that the UFL
would be at a competitive disadvantage and this would affect future growth in a highly
competitive market.
Purpose of the Project and Questions
The purpose of this study was to understand student learning and engagement in an
online, graduate-level setting and assess the impact these have on students working to develop
critical competencies for future leadership roles in business. In addition, this study evaluated the
degree to which the UFL was meeting its goal by assessing its objective to be recognized in the
top five online MBA programs in the United States university rankings by the end of 2019 and
reach number one by the end of 2020. The study focused on knowledge, motivation, and
organizational influences related to achieving the organizational goal. The research questions
that guided the evaluation study are listed as follows:
1. What are the UFL Online Learning Team’s knowledge and motivation related to
identifying, measuring and improving online student learning and engagement?
2. What are the UFL Online Learning Team’s knowledge and motivation related to
leadership competency requirements for graduate students?
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

24
3. What is the interaction between organizational culture and context and the UFL
Online Learning Team’s knowledge and motivation?
4. What are the recommendations for the UFL Online Learning Team’s organizational
practice in the areas of knowledge, motivation, and organizational resources?
Methodological Framework
The Clark and Estes (2008) gap analysis conceptual framework was used in this study to
understand and evaluate the gap between the current performance and the desired goals for the
key stakeholders. In addition, an evaluation model was used in conjunction with the Clark and
Estes (2008) conceptual framework to identify the knowledge, motivational, and organizational
influences that impacted the key stakeholders. The assumed influences were developed after
discussions with the Vice Dean of the Online Learning team and other related research. This
study employed qualitative data gathering and analysis. The UFL Online Learning Team’s
knowledge and motivation related to improving student learning and engagement, developing
critical competencies in graduate students, and achieving the organization goal was assessed
using interviews with key stakeholders. Research-based solutions were evaluated and
recommended at the completion of this study.
Definitions
The following operational definitions are provided for ease in reading and understanding
the concepts in this dissertation:
Asynchronous: An academic term that refers to learning and sharing knowledge through
online discussion boards, blogs and other media and used when students are not online or using
the same platform at the same time (Great Schools Partnership, n.d.).
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

25
Blended Learning: Uses both online media and traditional on-campus learning methods
to transfer knowledge (Great Schools Partnership, n.d.).
Disruptive Innovation Theory: When new disruptive innovation targeting an untapped
consumer base shows up in the market, the incumbents see it as inferior, ineffective, and lacking
in any real opportunity for growth; when this new innovation begins to grow, the incumbent
scrambles to respond (McHenry, 2016).
Massive Open Online Course (MOOC): A web-based distance learning program that is
often offered at no/low cost and is designed for the participation of large numbers of
geographically dispersed students (Educause, 2013).
Synchronous: An academic term that refers to learning and sharing knowledge through
online chats or a webinar or other form of a shared learning management system where students
are on the same platform at the same time; used to describe classes that take place at a designated
time (Great Schools Partnership, n.d.).
Organization of the Dissertation
This dissertation is organized in five chapters. This chapter provided the reader with the
key concepts and terminology commonly found in a discussion about the online learning and the
disruption taking place in both the business world and in academia. The University of Future
Leader’s organizational mission, goals, and stakeholders, as well as the framework for the study,
were introduced. Chapter 2 provides a review of current literature surrounding the scope of the
study. Topics that will be addressed include understanding student engagement in an online,
graduate-level environment and the impact it has on the students’ ability to develop critical
competencies for future leadership roles in business. Chapter 3 details the knowledge,
motivation, and organizational elements to be examined, as well as methodology, data collection,
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

26
and analysis. In Chapter 4, the data and results are assessed and analyzed. Chapter 5 provides
solutions, based on data and literature, for closing the perceived gaps, as well as
recommendations for an implementation and evaluation plan for the solutions.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

27
CHAPTER 2
REVIEW OF THE LITERATURE
In response to rapidly changing demands for new, innovative learning environments at
the graduate level, traditional business schools are evaluating, launching, and re-assessing online
learning programs (Meyer, 2014). Business school leaders recognize they must prepare graduate
students for the disruption in the business market. This includes rapid growth of new
technologies and digitization, globally focused strategies, non-traditional online and e-commerce
competitors, and increasingly complex sustainability issues and crises (Roobeek & De Ritter,
2016). At the same time, these business schools are working to meet the needs of both their
incoming graduate-level students, many of whom are adult learners, and the business leaders for
whom these students will be working. These employers are demanding that graduate-level
programs ensure students enter the workforce with the appropriate skills and competencies
needed for managing the highly disruptive and rapidly changing market in business today
(Wilson & Thomas, 2012).
The purpose of this chapter is to outline the research for understanding student learning
and engagement in an online graduate-level environment and assess the impact it has on students
working to develop critical competencies for future leadership roles in business. The first
section describes the disruption occurring in the online learning environment, along with the
types of models being offered and the characteristics of the target market — the adult learner.
The second section identifies factors, variables, and causes that influence the need for specific
competencies as graduate students prepare for future leadership roles in an increasingly
disruptive market. The next section examines the role of the University of Future Leaders (UFL)
Online Learning Team and the important knowledge, motivation, and organizational influences
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

28
that help ensure their graduate students are engaged throughout the program, achieve the desired
learning outcomes, and graduate feeling efficacious about what they have learned. The final
section includes the Clark and Estes (2008) conceptual framework used in this study to assess the
interaction of the UFL Online Learning Team’s knowledge, motivation, and the organizational
context which impact student engagement and learning outcomes.
Graduate-Level Online Learning
In order to understand how best to grow a new innovative program at the UFL, the study
must include an exploration of the current environment, the developing market, and the
consumer most likely to be impacted by online learning. The following section will focus
specifically on the disruption occurring in the online learning environment, the new models in
the online learning space, and the adult learner who is the consumer most likely to take
advantage of this innovative option for completing a graduate-level degree.
Disruption and the Online Learning Environment
As rapid technology growth and its acceptance among new generations of leaders impact
the speed of innovation, business education universities serving graduate-level students face
disruption like never before (Roobeek & De Ritter, 2016). Traditional business schools with
standard curricula and traditional delivery methods, like lecture-based learning, are being
challenged to meet the changing needs of their graduate-level students; what helped these
schools establish strength and credibility in the past will no longer be sufficient in the future
(Roobeek & De Ritter, 2016). They can no longer rest solely on their reputations as either
research institutions or teaching institutions; yet, many of them attempt to straddle the line and
focus on both sets of objectives, research, and relevant student outcomes (Wilson & Thomas,
2012). As a result, faculty and the business leaders they serve have conflicting goals. Wilson
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

29
and Thomas (2012) noted that faculty members are often required to focus more on research and
publishing for recognition and career progression than on teaching responsibilities and relevance
to pedagogy. As a result, faculty members are often disconnected from the needs of their
students and the quickly evolving innovation needs of future business leaders (Kimberly &
Bouchikhi, 2016).
As is often the case, when new challengers first enter the market, the incumbents see
them as non-threatening with their supposed inferior products (McHenry, 2016). This reaction is
discussed by McHenry (2016) as the disruptive innovation theory, whereby strong organizations
such as traditional universities have historically delayed their response to technology changes
until innovative challengers start becoming successful at answering the needs of consumers in
the market. McHenry (2016) suggested that reputation, accreditation, and the question of
legitimacy have protected traditional universities in the past, but these safety nets are no longer
stable.
More traditional universities are launching online MBA programs and the competition is
no longer constrained by local or national boundaries (McHenry, 2016). Enrollment in online
courses continues to grow while on-campus enrollment continues to decline (Allen & Seaman,
2017). The number of students enrolled in at least one online or distance education course in the
fall of 2015 exceeded six million students, representing an increase of 3.9% versus 2014, and
nearly 30% of all students enrolled in higher education courses were taking at least one online
course (Allen & Seaman, 2017). At the same time, the total number of students studying in
residential or on-campus programs was down by nearly one million participants between 2012
and 2015 (Allen & Seaman, 2017). In addition, less than half of the full-time MBA, part-time
MBA, and flexible MBA on-campus programs experienced growth (Graduate Management
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

30
Admission Council, 2016). Universities are recognizing that online learning, particularly at the
graduate level, is an important strategic tool in higher education and it is helping universities find
new paths for growth that address the challenges of pending disruption in both business and
academia (Meyer, 2014).
Evolving Models of Graduate-Level Online Learning
In order to explore the impact and outcomes of online learning, it is important to
understand the growth of new models and define the various forms of online learning at the
graduate level. There are several terms that refer to learning in an online environment such as
distance learning, MOOCs, fully online courses, blended classes, synchronous, and
asynchronous course work (Moore, Dickson-Deane, & Galyen, 2011). In 2002, more than 71%
of universities had some form of an online learning course, and by 2012, more than 86% were
actively engaged (Allen & Seaman, 2012). The number of universities who offered full
programs as opposed to just individual courses nearly doubled from 2002 to 2012 (Allen &
Seaman, 2012).
Early models of online learning were launched and labeled as Massive Online Open
Courses or MOOCs (Leire, McCormick, Richter, Arnfalk, & Rodhe, 2016). Universities are
reviewing the effectiveness of these models because of their ability to reach massive numbers of
students with an open enrollment format and decreased costs associated with reuse of curriculum
and recorded lectures; however, their disadvantages include concerns and issues with quality,
retention, and the potential for reduced faculty and peer interaction (Ferguson & Sharples, 2014;
Leire et al., 2016). As new models develop and new competition enters the market, those in
higher education will continue to review the outcomes to build student engagement, encourage
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

31
faculty focus, and improve overall learning while also addressing the concerns that naturally
arise when new models disrupt the status quo (Meyer, 2014).
The research about the value and outcomes of online learning compared to on-campus
learning is growing. In a study conducted by Ladyshewsky and Taplin (2014), several methods
of learning were assessed in a post-graduate level leadership course which included 550 adult
learners. The modes of learning included face-to-face, blended, and fully-online courses and
student performance was measured. In the blended class environment, students had access to
and were accountable to both the instructor and their peers for critical analysis of the course
materials and preparation for a synchronous class where they were front-and-center on a screen
with their associates. This form of accountability was similar to the environment for those in
face-to-face classrooms.
The study did not find any significant differences in the learning outcomes or
performance between the students who took face-to-face courses and those who participated in
blended learning (Ladyshewsky & Taplin, 2014). However, the researchers did find that while
the fully online learning system could be challenging with a lack of interaction with the faculty
who might be available to expand on concepts and encourage engagement in classroom
discussions, the students in this study who engaged in the fully online learning mode slightly
outperformed their counterparts in the other modes. Ladyshewsky and Taplin (2014) noted that
“in order to be intelligent and responsive, [fully] online students have greater accountability and
responsibility for understanding the course material as they have to debate, comment and share
their views on these concepts in a written public domain” (p. 19). The online students are often
self-starters who are comfortable discussing and debating their viewpoint online in a more public
forum with other student learners in their network (Ferguson & Sharples, 2014). While the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

32
research will continue, several of those who have begun studying the outcomes of online
learning conclude that there are multiple options for different learning environments and
pedagogical preferences, particularly at the graduate level and for adult learners, that offer more
flexibility and similar, or better, learning outcomes (Ladyshewsky & Taplin, 2014; McBrien,
Cheng, & Jones, 2009; Meyer, 2014).
Characteristics of Adult Learners in an Online Learning Environment
Adult learning, or andragogy, at the graduate level is significantly different from
traditional pedagogy in that it requires a focus on addressing the needs of non-traditional
students, particularly in an online learning environment (Pew, 2007). Graduate level education
no longer serves only traditional young college graduates. Pew (2007) noted a larger influx of
non-traditional students is requiring faculty and administrators in higher education to rethink the
environment and create new, innovative options for expanded learning. In addition, andragogy
targets adult learners who take responsibility for their own learning and motivation, many of
whom are focused on fulfilling intrinsic goals.
While the characteristics of the graduate-level online learner are evolving rapidly, the
majority of students are mature adults with family and job responsibilities, have a strong internal
locus of control, are intrinsically motivated, and are focused on achieving their goals (Dabbagh,
2007). This profile is continuing to evolve and other characteristics are beginning to emerge as
well. Dabbagh (2007) summarizes the most important
characteristics and skills [which] are perceived as critical to the success of the online
learner:
1. Having a strong academic self-concept
2. Exhibiting fluency in the use of online learning technologies
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

33
3. Possessing interpersonal and communication skills
4. Understanding and valuing interaction and collaborative learning
5. Possessing an internal locus of control
6. Exhibiting self-directed learning skills
7. Exhibiting a need for affiliation. (p. 220)
These characteristics and skills are directly aligned with the critical competencies needed for
future roles in the global business market; roles that business leaders expect will be filled by
graduates who have learned to finely-tuned their skills in business school (Wilson & Thomas,
2012).
Critical Competencies and Graduate-Level Students in Online Learning
As business schools compete for the best and brightest students, they will be required to
look beyond their prestigious brand names and previously successful recruiting practices that, in
the past, guaranteed large numbers of highly qualified applicants. Future recruiting will include
proof that students are leaving graduate school with the right competencies. The following
section will identify some of the most relevant competencies for future global leaders and note
recent research about how online learning can impact the development of these critical skills.
Critical Competencies Defined
Many business leaders are concerned that students are not developing the competencies
they need while they are in business school. The World Economic Forum (WEF, 2016) noted
that some of the jobs that are most in-demand today and requiring new, evolving skills did not
even exist five or 10 years ago. The demographic, socio-economic, and technological trends will
all affect the skills and competencies required for future leaders; new technologies that include
artificial intelligence and machine learning will continue to evolve and business leaders will be
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

34
seeking future leaders who are prepared to tackle this new, ever-changing environment (WEF,
2016).
Today’s leadership roles require more than just the hard skills of finance, accounting, and
marketing, which are still the focus of graduate-level curriculum. WEF (2016) contends that
on average, by 2020 . . . social skills — such as persuasion, emotional intelligence and
teaching others — will be in higher demand across industries than narrow technical skills,
such as programming or equipment operation and control. In essence, technical skills
will need to be supplemented with strong social and collaboration skills. (p. 20)
Researchers agree that future leaders will be required to develop what some researchers
refer to as soft skills, such as emotional intelligence, communication, and interpersonal skills
(Anthony & Garner, 2016; Bedwell, Fiore, & Salas, 2014). There are several skills that are
important for interpersonal effectiveness, but they all filter down to two main competencies:
communication and relationship building (Bedwell et al., 2014). Communication includes such
skills as verbal communication, non-verbal communication, and written skills, as well as active
listening and response (Bedwell et al., 2014). In addition, relationship building includes skills
such as building trust, demonstrating sensitivity and influence, reflecting an attitude of servant
leadership, and implementing healthy conflict resolution and negotiation when an opportunity
arises (Bedwell et al., 2014). Wilson and Thomas (2012) take these ideas about soft skills one
step further in their discussions about teaching students to think more broadly about society and
develop a view that learning has value. They note that: (1) valuing learning as a priority over
traditional business education that focuses primarily on management, (2) building knowledge
and accountability for the moral and ethical issues as they relate to corporate social responsibility
and how companies impact or support society, and (3) focusing on both local and global issues
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

35
and ensuring the curriculum addresses diversity of thought, language, culture, and religion helps
students develop non-technical skills that can be used in the workplace. Effectively, these
students will be required to develop critical thinking skills and the ability to take action.
How Online Learning Impacts the Development of Critical Competencies
Online learning organizations are beginning to respond to the demands of business
executives in developing the competencies they require in graduates who will become future
leaders. By their very nature to be innovative and responsive, online learning teams and faculty
continue to evolve the ways they serve the graduate-level students who are working to become
the next generation of leaders (Meyer, 2014). In the future, business executives will demand that
graduate students graduate with skills focused “on global and intercultural learning,
technological sophistication, collaborative problem-solving, transferable skills, and real-world
applications — both civic and job-related” (Kuh, 2008, p. 13).
Non-research opinions about the future of online education in trade journals and
education publication also note that more students are enrolled in online courses than ever before
and they are all looking for unique and different experiences (Bisoux, 2016; Dua, 2013;
Friedman, 2017). Some of the trends include: (1) more universities will become involved in
granting credentials that serve professionals seeking new and different learning experiences and
competencies, (2) more and better use of big data will allow faculty to track participation and
engagement, (3) more use of artificial intelligence in classes will provide student assistance and
tailor experiential learning, and (4) new online degrees in specialized disciplines will help
answer the needs for specific skills and competencies (Friedman, 2017).
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

36
Impactful Elements of Student Engagement in Online Learning
Student engagement is one of the key elements of effective graduate-level online learning
and is weighted most heavily in ranking university online learning effectiveness. It may be even
more important than when measured in an on-campus environment because the students depend
heavily on their interactions with both the faculty and the technology for learning, and have
fewer options to be connected to the university (Meyer, 2014). This section will expand on the
importance of student engagement in online learning in addition to demonstrating the importance
of metacognition, social connections, and faculty that impact online learning outcomes.
Student Engagement Impact on Online Learning Outcomes
There are several models or frameworks for understanding student engagement in online
learning and its impact on learning outcomes. Moore (1993) studied the teacher-learner
relationships in online environments or what is referred to as distance education. The research
relates specifically to the relationships between the two when they are “separated by space and/or
by time” (p. 24) and is often used by other researchers as a framework for understanding student
engagement and online learning. The requirement to make changes in teaching methods fall into
three areas: instructional dialogue (interaction and media), program structure (course design and
media), and autonomy of the learner (allowing the learner to share in the responsibility for
learning).
To complete their qualitative study, McBrien et al. (2009) refer to the framework of
transactional distance theory, a model developed by Moore (1993) to understand student
engagement in online learning. McBrien et al. (2009) explored the concepts of dialogue
(interaction between students and faculty), structure (how the course content is created and
delivered), and learner autonomy (ability and willingness to engage in both self-directed learning
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

37
and interaction with others) from the perspective of understanding and strengthening student
engagement. While the study uncovered several concerns including challenges with technology
not working consistently, the over-abundance of stimuli with video, audio, and chat, and the need
for more connection and non-verbal communication, the concerns about distance were found to
be less of a concern than expected. In addition, the researchers noted that as new synchronous
learning technology and platforms evolve, these the forms of dialogue, new course structures,
and opportunities to engage will evolve, and overall student engagement and interaction will
continue to be studied and compared to traditional learning models to ensure quality is
maintained (McBrien et al., 2009).
Prior to considering the importance of student engagement in online learning, Meyer
(2014) noted the five benchmarks of traditional on-campus student engagement from the
research done by the National Survey of Student Engagement (NSSE, n.d.) and outlined by Hu
and McCormick (2012). They include:
1. Level of academic challenge (measures the extent to which colleges emphasize
student effort and set high expectations), which includes questions about how many
hours per week students study and the amount of reading or writing required in the
courses.
2. Active and collaborative learning (measures student engagement with learning both
alone and with other students), which includes questions on asking questions in class,
making presentations, and working on group projects.
3. Student-faculty interaction (measures the extent to which students interact with
faculty in and out of class), which includes questions on how often students discuss
ideas with faculty or work with faculty on projects.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

38
4. Enriching educational experience (measures several educational activities), which
includes questions about interactions with diverse others and participation with
learning communities, service learning, internships, and research with faculty.
5. Supportive campus environment (measures the quality of student relationships with
peers, faculty, and staff), which includes questions that capture students’ perceptions
of campus support. (Meyer, 2014, p. 8).
These elements that are relevant for adults and build on the development of professional skills
also help support the outcomes for productive student engagement in online learning.
In Garrison, Anderson, and Archer’s study (2010, as cited by Meyer, 2014) the
Community of Inquiry (CoI) model is integrated into the discussion of student engagement in
online learning. The early research on the CoI framework by Garrison et al. (2010) focused on
three critical elements or areas of collaborative learning. The three areas include: cognitive
presence (refers to some of John Dewey’s early work and includes the balance of reflection and
critical thinking), social presence (the feelings and emotions tied to being a part of a community
or group), and teaching presence (the presence of an engaging and collaborative faculty
member). The authors of this original work have made several adjustments since 2000, but the
premise of their study still stands as a highly respected model for successful online learning and
student engagement. Meyer (2014) emphasized that it has been validated by several researchers
since its first introduction.
The Role of Metacognition in an Online Learning Environment
In understanding and measuring student engagement in online learning, this study will
explore the role of several important factors including metacognition, social connections, and
faculty interaction. Each of these is an element of Community of Inquiry (Garrison, Anderson,
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

39
& Archer, 2000; Garrison et al., 2010; Meyer, 2014) discussed earlier. The first is the role of
metacognition and the impact it has on student engagement in an online learning environment.
The definition of metacognition and its roots come from the awareness, understanding, and
control of one’s own cognitive or learning abilities. Baker (2011) defines metacognition as a
process of thinking about thinking. Students who have a strong awareness of themselves, their
ability to learn and monitor themselves, and their responsibilities and coursework can more
readily transfer their knowledge across other subjects and to the workforce (Baker, 2011). In an
online learning environment, strong metacognition skills help the learner take responsibility for
completing the required coursework on their own schedule as well as the expand their ability to
plan, monitor, and evaluate their own progress (Lehmann, Hähnlein & Ifenthaler, 2014).
Metacognition is both personal and social, and it becomes even more important in an online
environment where self-regulation can significantly impact success and growth (Akyol &
Garrison, 2011).
The Role of Social Connections in an Online Learning Environment
The second element of Community of Inquiry discussed earlier is the role and definition
of social connections in an online learning environment. There are various forms of learning
models, communication channels, and online communities that support these connections
(Garrison et al., 2000, 2010; Meyer, 2014).
Constructivist learning is a model that supports the importance of social learning, as well
as interactions with others and connections to faculty and peers in an online learning
environment (Meyer, 2014). The power in this model stems from the quality of the interaction
based on the student’s background, life or career experiences, and their ability to contribute to an
interactive discussion based on the ways they organize their knowledge (Meyer, 2014). The use
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

40
of interactive experiential learning centers and activities where students are engaged with each
other in solving problems and then reflecting on the outcomes support learning in a constructivist
learning environment (Meyer, 2014).
In understanding the importance of social connections on student engagement, Dixson
(2010) found that multiple communication channels may contribute to higher student
engagement including both student-student and instructor-student. Dixson’s (2010) research
determined there are three areas that support effective online instruction including active learning
strategies, collaborative and interactive activities, and active involvement on the part of
instructors. The active learning strategies include activities where concepts are applied in
problem-solving (important for competencies above), discussion forums, group projects, and
assignments that tie in current events (Dixson, 2010).
On the other hand, a lack of access and interaction in an online environment can result in
feelings of isolation and low student engagement; however, learning communities can help
reduce these feelings (Yuan & Kim, 2014). Chang’s (2003) definition of a learning community
includes individual learners bringing their own knowledge and experience to the community, all
members having the opportunity to share ideas and experiences, and comprising the community
of learners from various backgrounds. Yuan and Kim (2014) noted the key elements in these
communities include “trust, knowledge construction, information sharing, a feeling of being
connected, common goals, and a belief that learners’ needs would be fulfilled” (p. 2). Most
important, the learning community of individuals believes that each person in the group is
important as a valued member, and the group, as a whole, exists to ensure everyone has a voice
and can assert influence (Yuan & Kim, 2014). In the case of online learning, this voice and the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

41
various communications might be conveyed via technology in the form of texts or chat boxes.
Synchronous courses also support this form of social interaction and communication.
The Role of Faculty in an Online Learning Environment
Finally, the role of faculty or instructors in online learning, particularly at the graduate
level, is one of the most important elements of student engagement, particularly as it relates to
developing both new skills and confidence and ensuring the students stay enrolled until
completion (Meyer, 2014). The question among scholars and students alike is whether or not the
power of teaching presence and the power of motivation can transcend online learning
environments. Baker and Taylor (2012) cite several researchers in asserting that teaching
presence is an important element for “fostering a sense of classroom community” (p. 6). Being
aware of the importance of teaching presence in addition to measuring and improving it will help
faculty and administrators continue to raise the bar on this important factor in student
engagement (Baker & Taylor, 2012).
Another element of importance discussed in the research is related to teacher immediacy
and student learning (Baker, 2010). They discussed that improvements in technology and the
addition of synchronous and asynchronous learning options have improved and allowed for a
narrowing of this gap. Baker (2010) also noted the research on the importance of immediacy
behaviors which can be incorporated quickly in online environments. The study demonstrated
that having access to a responsive faculty member impacted the student’s overall learning,
cognition, and motivation (Baker, 2010). Elements of the work that supported the positive
impact included having a set curriculum, effective use of the medium, and establishing
“conventions of netiquette prior to the course commencing” (Baker, 2010, p. 24). The faculty
help students expand the learning and push students to think more critically (Garrison &
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

42
Cleveland-Innes, 2005, as cited in Meyer, 2014). The faculty also provide guidance and advice
for students who struggle to keep up with their assignments or who are feeling insecure. As a
result, faculty must often provide personalized attention and guidance based on the needs of each
student in an online learning environment (Meyer, 2014).
The online learning environment has become a platform for what Meyers (2008) refers to
as transformative pedagogy, a concept developed in the research by Mezirow and Associates
(1991). Important competencies such as critical thinking and decision-making begin to unfold in
the online learning environment while students develop new skills and work closely with their
faculty partners (Meyers, 2008). Several strategies for effective instruction and use of the
transformative pedagogy framework are outlined by Meyers (2008). They include helping
faculty to develop a safe and trust-based learning environment, inspiring students to use their
critical thinking skills to explore their own backgrounds, beliefs and unconscious biases, trying
new, innovative teaching methods to build student engagement, challenging students to consider
and solve real-world problems they might face when they leave graduate school, and working
with students to develop creative and focused solutions to future problems (Meyers, 2008). The
concept of transformative pedagogy supports growth and the development of new competencies
for both the faculty and the students, and it challenges both to try new models for knowledge-
exchange (Meyers, 2008). Garrison (2017) suggests that innovation in the online learning
environment directly challenges traditional approaches to teaching and creates new levels of
stress among seasoned faculty. However, supporting the faculty through the critical phases of
design and delivering content in a new model, helping them learn the value of a new
transformative pedagogical approach, and providing the important technical and institutional
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

43
support will help move online learning from “incubation to mainstream” in the coming years
(Garrison, 2017, p. 152).
Role of Stakeholder Group of Focus
The stakeholder group of focus was the UFL Online Learning Team which is part of the
Online Learning Department at the University of Future Leaders. This group was formed in
2014 when the new Online MBA program was being developed. Most of the team members
were involved prior the launch. The Vice Dean for Online Graduate programs is the Dean’s
Cabinet member who provides leadership and strategic direction for this team and the
Director/Assistant Dean has direct management responsibility for the deliverables. Other key
team members include the Online Media and Technology Team, the Instructional Design Team,
and the Student Services Team. This stakeholder group has primary responsibility for achieving
the organization’s performance goal and is the most directly impacted by the knowledge,
motivational, and organizational influences.
The Clark and Estes (2008) Gap Analysis and Conceptual Framework
In an effort to clearly evaluate an organization to understand its effectiveness in
achieving its organizational performance goals, Clark and Estes (2008) developed an inquiry
process to gather relevant information, assess progress, and analyze gaps in stakeholder
performance. This conceptual framework focuses on specific gaps in performance by comparing
the organization’s current performance against its desired goals. Once these gaps are identified,
the researcher uses the framework to understand the underlying causes of the gaps.
Clark and Estes (2008) organized the major causes of performance gaps into three
specific categories: what stakeholders know and which skills they possess (knowledge), how
they are driven to perform (motivation), and which organizational influences or barriers impact
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

44
performance (organization). Rueda (2011) described the knowledge dimension as what a
stakeholder needs to know in terms of what to do and how to do it in order to achieve their goals.
Krathwohl (2002) broke this down further into four different types of knowledge: factual
knowledge (the facts), conceptual knowledge (the theories or models), procedural knowledge
(the process), and metacognitive knowledge (the understanding of one’s own knowledge).
Motivation was described as one’s willingness to choose an activity (active choice), how likely
they are to stick with it through to completion (persistence), and how they continue to develop
and learn (mental effort) new information and knowledge (Clark & Estes, 2008; Rueda, 2011).
Finally, the organizational influences or barriers which may impact performance include the
overall structure of the organization, which rules and guidelines are in place, and the values that
are espoused as compared to those that are actually in play (Clark & Estes, 2008). Rueda (2011)
referred to organizational influences as “culture, structure, and policies and practices” (p. 53).
The Clark and Estes (2008) gap analysis conceptual framework was used in this study to
understand and evaluate the gap between the current performance and the desired goals for the
new online MBA at the University of Future Leaders (UFL). The UFL Online Learning Team is
the key stakeholder responsible for achieving the organization’s performance goal, which is to
improve online student learning and engagement and prepare graduate students with the most
relevant competencies. The organization is also focused on being recognized in the top five
online MBA programs in the U.S. university rankings by the end of 2019 and reach number one
by the end of 2020. Student engagement is the most heavily weighted factor driving rankings for
the online MBA. The first section that follows will focus on knowledge-related influences that
are relevant to understanding the critical factors for student engagement in an online learning
environment. The second section will evaluate how motivation is impacting the organization’s
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

45
ability to achieve its goal to develop a strategic plan to identify, measure and improve online
student learning and engagement and prepare students with the most relevant competencies. The
final section will address the UFL organizational structure, values, policies, and procedures to
better understand the organizational influences that affect the Online Learning Team’s
performance and its ability to achieve its goals. A discussion in Chapter 3 will explore these
areas of knowledge, motivation, and organizational influences on performance more deeply
using a defined methodological approach.
Stakeholder Knowledge, Motivation, and Organizational Influences
Knowledge and Skills
While educators and researchers debate the future changes in the world of higher
education, the fundamentals of learning remain relatively consistent. Mayer (2011) discussed
three fundamental elements of learning which include: some level of long-lasting change in the
learner, a change in what the learner knows now versus before the learning occurred, and a
change in how the learner interacts with the environment. Clark and Estes (2008) described
education as “any situation” in which learners acquire new knowledge and skills that help them
develop “current, research-based knowledge about why things happen and what causes them to
happen” (p. 59). They specifically noted that this education can occur in any environment and in
any setting. Rueda (2011) enhanced this view about the overall goal of education which is to
ensure the learner is “able to perform in a self-regulated fashion, is engaged, and has developed
expertise” (p. 11).
Each of these perspectives supports one of the goals of the University of Future Leaders
which is to develop a strategic plan to identify, measure and improve online student learning and
engagement. The following literature review will focus on several knowledge-related influences
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

46
that are relevant to understanding how to achieve learning in an online setting. Examining these
knowledge and skill influences will help determine how well the UFL stakeholders are doing in
achieving their goals, identifying problems or issues, diagnosing the cause of any performance
gaps, and aligning solutions to the problems (Clark & Estes, 2008; Rueda, 2011). The
organizations that will ultimately hire the students operate in a highly competitive market and
will depend on these graduates to have the ability to use their knowledge to build innovative new
solutions in the future (Markova & Ford, 2011).
Knowledge influences. There are several types of knowledge that a student will develop
through the process of academic learning, whether in a classroom or through an online platform.
The knowledge dimension includes four types of knowledge ranging on a spectrum from more
concrete factual and conceptual knowledge to more abstract procedural and metacognitive
knowledge (Krathwohl, 2002; Mayer, 2011; Rueda, 2011). All of the types of knowledge are
necessary to enhance effective learning (Mayer, 2011).
Factual knowledge is a clear understanding of the “what” needs learning. This form of
knowledge includes the basic details about a topic such as a terminology, statistics, elements, and
lists of needs or details about the environment (Krathwohl, 2002; Mayer, 2011; Rueda, 2011). In
the case of the UFL Online Learning Team, this factual knowledge would include an
understanding of the mission and goals for the department in addition to the goals of the students
who will be using the online platform.
Conceptual knowledge is what might be considered the “so what” about an area of
knowledge. This form of knowledge includes categories, models, or principles which encourage
the learner to think about and understand the facts in relation to something broader and more
comprehensive (Krathwohl, 2002; Mayer, 2011; Rueda, 2011). In addition, conceptual
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

47
knowledge is the type of knowledge that evolves to become automated and is often used in the
work environment (Clark & Estes, 2008). In the online learning environment, students who
develop strong conceptual knowledge and expand their cognitive abilities are more likely to
transfer their training and skills into the workplace (Grossman & Salas, 2011).
Procedural knowledge is the “how to” area of knowledge and includes the very specific
steps needed to accomplish a skill or a goal (Mayer, 2011; Rueda, 2011). This form of
knowledge also includes the “when” a particular process or procedure might be used in a
situation (Krathwohl, 2002). Students who work in an online environment need procedural
knowledge for performing required tasks (Mayer, 2011) because a large portion of their work is
accomplished in an environment where they do not have immediate access to a professor.
Metacognitive knowledge is the “how am I doing” area of knowledge. It includes both
the understanding of the best personal strategies for learning as well as the beliefs or reflections
about oneself and how well they are responding, learning, and developing new skills (Mayer,
2011). This type of knowledge is critical for online learners in that they must be self-regulated
for long periods of time and are required to take complete responsibility for “monitoring and
controlling their learning” (Mayer, 2011, p. 43).
Required competencies for online Masters of Business Administration) graduates. The
market options for how and where MBA students obtain their graduate degrees continue to
change (Friga et al., 2003). In addition, the demands for very specific competencies for success
in the business world are evolving (Bruce & Schoenfeld, 2006). In order for the UFL Online
Learning Team to assess the most important competencies required for leadership roles in
business, they need to grow their conceptual knowledge and learn more about the business
organizational needs through surveys and interviews. Business organizational constituents may
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

48
include senior executive leaders, organizational recruiters, and recent MBA graduates working in
both domestic and global companies. In addition, the UFL Online Learning Team needs to
review the current curriculum for competencies being taught in the online MBA program and
determine the gap between what the business constituents require and what is currently being
delivered in the online MBA program. While the UFL Online Learning Team gathers facts and
details about each of these required competencies, they will also be processing the information
and seeking to understand the relationships between what is needed and what is taught so that
students can transfer their learning to future roles. This conceptual knowledge will enable
thoughtful and reflective planning for how the UFL will close the gap and achieve both their own
goals and the goals of their students.
In a series of three surveys completed in 2004-2005 with MBA graduates, MBA alumni,
and corporate recruiters, Bruce and Schoenfeld (2006) discuss the competencies among MBA
graduates most appealing to companies. The outcomes showed that companies regarded both
strategic thinking and analytical skills as most important for the roles they were trying to fill. By
2020, the demands will change again and social skills such as inspiring others, teaching, and
coaching, and collaboration will become even more critical (WEF, 2016).
Methods for teaching transferable skills. The UFL Online Learning Team and the
instructional designers are responsible for designing and building an online curriculum that
improves student learning and engagement and ensures transfer skills to the workforce. As a
result, this team must have strong procedural knowledge about the most effective learning and
engagement strategies including critical engagement information, job aids, training, and
education (Clark & Estes, 2008), as well as the effect of various demands on cognitive capacity.
In addition, the UFL Online Learning Team must understand the spectrum of deep learning
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

49
outcomes that online students need to operate in future leadership roles. Darling-Hammond,
Wilhoit, and Pittenger (2014) recommend a learning approach that focuses on the same soft
skills and competencies being discussed in business today including “critical and creative
thinking, problem solving, collaboration . . . the capacity to learn to learn, and the social-
emotional intelligence that fosters a growth mindset and supports resilience and resourcefulness”
(p. 4).

Table 2
Knowledge Influences on Organization of Focus Goal
Organizational Mission
The mission of the University of Future Leaders (UFL) is to transform from an 18th century
education model to a 21st century business education model that differs dramatically in content,
professors, delivery, and serving student needs. The mission of the UFL is to provide an innovative
and entrepreneurial learning environment that mirrors business practices while expanding thought
leadership, identifying key global opportunities, and exploring critical issues.
Organizational Performance Goal
By September 2019 the UFL Online Learning Team will develop a strategic plan to identity, measure
and improve online student learning and engagement that results in preparing graduate students with
the most relevant competencies for future leadership roles in business.
Organization of Focus Goal
Knowledge Influence Knowledge Type Knowledge Influence Assessment
The Online Learning Team needed to
know how to implement the most
effective methods for keeping students
engaged and teaching these transferable
skills in an online environment.
Procedural The Online Learning Team was
asked how they can identify and
improve student engagement in
achieving learning outcomes.
The Online Learning Team needed to
know how the competencies that online
MBA graduates learn will help secure
leadership roles in a global business
environment.
Declarative
(Conceptual)
The Online Learning Team was
asked to identify the most critical
competencies required of MBA
graduates by business leaders and
how these competencies can be
applied in future roles.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

50
Table 2 reflects the organizational mission, the performance goal, and the two knowledge
influences discussed in this literature review.
Motivation
While both knowledge and motivation are critical for achieving goals, motivation ensures
organizational members get started, keep going, and stay focused on reaching their goals (Rueda,
2011). At one end of the spectrum, motivation is an important enabler for strong performance
and, at the other end, can be a critical inhibitor of performance (R. E. Clark, USC Rossier School
of Education online learning discussion, EDUC627, 2017). The online MBA is a relatively new
program at the UFL and the UFL Online Learning Team appears highly motivated to reach its
goals. As the department grows and new goals are established, the overall culture will evolve
and change. It will be important to understand and monitor the team performance and the
“factors that influence work goal choice, persistence, and mental effort” (Clark & Estes, 2008, p.
89). These factors include the level of confidence each team member has about their own
abilities to perform and their confidence in the overall team’s abilities, the expectations and
beliefs about the barriers they will face, and how the team members value their individual and
collective goals (Clark & Estes, 2008).
Research suggests that motivation stems from the “desire to be effective” and one’s
beliefs about the available resources, support, and opportunities to be successful (Clark & Estes,
2008, p. 83). The UFL Online Learning Team is part of a department that is growing a new,
innovative learning model in the university and has a strong desire to be effective while also
reaching new heights in a relatively short period of time. In order to understand how this team
might continue to be motivated, this study looks further at how motivation works.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

51
Mayer (2011) discussed the five most common conceptions of motivation: interest
(working harder because of personal value), self-efficacy beliefs (believing hard work will pay
off), attributions (believing success will happen with more effort or control), goals (working
harder to master the task), and social partnerships (working harder when they perceive they have
a partner). This study explores self-efficacy and goal orientation theory in more detail.
Self-efficacy theory. The theory of self-efficacy was originally developed as part of
Bandura’s social cognitive theory which focused on self-perceptions and beliefs about one’s
ability to perform affected outcomes (Pajares, 1997). These beliefs, or perceived efficacy, touch
every aspect of one’s life and affect their behavior, their perceptions of assumed or real barriers,
their expectations of outcomes, and their overall accomplishments (Bandura, 2000; Pajares,
1997). Individuals who have a high self-efficacy and believe they are capable of achieving
significant accomplishments are more highly motivated; they are more likely to begin a new
project, persist in overcoming difficult hurdles and apply greater effort in achieving their goals
(Rueda, 2011). Pajares (1997) noted that these individuals are also known to recover their
confidence more quickly after failures and setbacks and, as a result, are more likely to take on
more challenging tasks.
Self-efficacy affects both personal beliefs and group perspectives known as collective
efficacy (Bandura, 2000; Pajares, 1997). Groups or organizations who operate with collective
efficacy develop a shared sense of empowerment and belief about their impact on other
constituents (Pajares, 1997). These shared beliefs determine the organization’s outlook about its
collective abilities to achieve its goals, how much effort the members invest in supporting the
team, how dedicated they are to support other team members when failures occur, and how
committed they are to achieving the final goals (Bandura, 2000).
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

52
Self-efficacy theory and online instructional design. Self-efficacy beliefs develop based
on information from multiple sources including mastery perceptions (e.g., past success in
mastering a task), verbal or social persuasion (e.g., experiences from collaboration and
feedback), and physiological reactions (e.g., the feelings of fear, anxiety, or excitement) among
others (Bates & Khasawneh, 2007; Pajares, 1997). The UFL Online Learning Team is made up
of several people with different backgrounds and levels of experience. Several team members
were hired with significant experience in the area of online curriculum development and
instructional design, and others developed their expertise through trial and error during the start-
up mode in this department. In either case, both their individual and their collective efficacy is
determined by these past experiences; if their experiences were generally positive, their self-
efficacy will be high, and if they experienced many failures their self-efficacy will likely be low
(Bates & Khasawneh, 2007; Pajares, 1997). Understanding the background of each member is
important as this relatively new team works together to use new technologies, develop new
learning tasks, and handle intricate problems. Wood and Bandura (1989) posited that in
situations where complex decisions will be addressed, the beliefs one has about their own ability
to learn new skills can influence their overall performance. As a result, the UFL team members
who believe their learning is generally fixed will more likely have low self-efficacy about the
outcomes for use of new online learning technologies, whereas those who believe their skills can
be improved or that new learning can be acquired will have higher self-efficacy and will be more
comfortable with taking risks, collaborating on new ideas, and willing to incorporate supportive
feedback (Bates & Khasawneh, 2007; Pajares, 1997). The more people on the team who have
positive self-efficacy, the more likely the team will develop a strong collective efficacy which, in
turn, promotes an environment and culture for growth.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

53
Goal orientation theory. The theory of goal orientation differs from several social-
cognitive theories of motivation since it focuses more on the purpose or reasons one might
engage in any activity rather than the beliefs about one’s abilities (Yough & Anderman, 2006).
The theory is split into two main types of goals: mastery goals and performance goals. Research
suggests that stronger, more long-term outcomes are the result of individuals who develop and
seek to achieve mastery goals, which include a focus on self-improvement and a deep
understanding of how the goals will serve a broader purpose (Pintrich, 2003; Yough &
Anderman, 2006). While there are potentially several positive outcomes from performance
goals, particularly in a sales-focused organization, these goals are generally more short-term
focused and less impactful in the long run (Yough & Anderman, 2006).
Setting goals in business is a significant part of the organizational culture. In higher
education, this process for setting goals can increase motivation, improve achievement, and
enhance learning (Turkay, 2014). Latham and Locke’s (2006) research showed that setting goals
significantly improve success rates. In 1954, Peter Drucker coined the term SMART goals in his
1954 book The Practice of Management to describe the most effective elements in achieving
objectives. The definition of SMART goals is: Specific, Measurable, Action-oriented, Realistic,
and Timely. These SMART goals will be important contributors to the success of the future
growth of online education at the UFL.
Goal orientation and administrator mastery goals. As the UFL Online Learning Team
works to develop learning technology and design engaging curriculum for its students, the team
will need to ensure goals are measurable and achievable. Working to accomplish what is known
as mastery goals will help promote learning among the team. In addition, these goals tend to be
more task-involved and are often focused on learning, mastering, and self-improvement
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

54
(Pintrich, 2003). In order to truly design a program to serve the needs of their students and
motivate strong performance, the UFL Online Learning Team may want to design their mastery
goals around the seven key questions outlined by Pintrich (2003) including:
(1) What do students want? (2) What motivates students in classrooms? (3) How do
students get what they want? (4) Do students know what they want or what motivates
them? (5) How does motivation lead to cognition and cognition to motivation? (6) How
does motivation change and develop? (7) What is the role of context and culture?
(Pintrich, 2003, p. 667)
Goal setting has been shown to significantly improve academic achievement and
motivation in the classroom (Turkay, 2014). In addition, research suggests there are several
principles of goal orientation that promote positive motivation, including focusing on mastery
and individual improvement, as well as designing tasks that are novel, varied, diverse, interesting
and reasonably challenging (Pintrich, 2003). The practice of goal setting and accountability will
be important elements of growing the online MBA at the UFL in future years.
Table 3 reflects the organizational mission, the organizational performance goal, the
performance measure, and the two motivational influences discussed in this literature review.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

55
Table 3
Motivational Influences on Organization of Focus Goal
Organizational Mission
The mission of the University of Future Leaders (UFL) is to transform from an 18th century
education model to a 21st century business education model that differs dramatically in
content, professors, delivery, and serving student needs. The mission of the UFL is to
provide an innovative and entrepreneurial learning environment that mirrors business
practices while expanding thought leadership, identifying key global opportunities, and
exploring critical issues.
Organizational Performance Goal
By September 2019 the UFL Online Learning Team will develop a strategic plan to identity,
measure and improve online student learning and engagement that results in preparing
graduate students with the most relevant competencies for future leadership roles in
business.
Organizational Performance Measure
Be recognized in the top five online MBA programs in the U.S. university rankings by the
end of 2019 and reach number one by the end of 2020 as measured and reported by
respected publications such as U.S. News & World Report, Bloomberg, and others who
measure online university programs.
Assumed Motivation Influences Motivational Influence Assessment
1. Self-Efficacy: The Online Learning Team needed
to believe they are capable of effectively
developing online learning and instruction that
will engage online MBA students and ensure they
have learned the critical competencies for a
rapidly changing business market.
Interview: “How comfortable are
you with developing online learning
focused on student competencies?”
2. Goal Orientation: The Online Learning Team will
needed to set smart goals that focus on learning,
mastering, and self-improvement in order to
design online learning for MBA students that
engage and motivate students.
Interview: “Have you developed
smart goals that will improve online
learning and engagement for your
students?”

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

56
Organization
A third important factor in ensuring both the organizational goals can be achieved is
understanding the overall organizational structure, support, resources, and culture. Clark and
Estes (2008) discussed that an organization will struggle to achieve its goals if the people in the
organization do not have adequate resources, processes, or equipment (often obvious or explicit),
and if the culture is not inherently structured or motivated to support the organization (often
implicit or hidden). Even in organizations where the people have deep experience in their roles
(knowledge and skills) and are highly motivated (motivation) to achieve their goals, performance
gaps may occur if the organization is not structured or supported appropriately (Clark & Estes,
2008).
At the University of Future Leaders (UFL), the decision makers on the Dean’s Cabinet
recognized the importance of the organizational structure and support before the launch of the
online MBA. These decision makers understood that the UFL Online Learning Team would be
operating in a climate of great organizational change with high expectations to perform at a time
when there was also great uncertainty about this new model of education. The members of the
UFL Online Learning Team agreed to become part of this new department in 2015, and
understood their performance would be highly visible and their ability to hit their goals would be
tracked very closely by the rest of the organization. To understand how this team might continue
to grow and serve both the industry business leaders, as well as the students interested in an
online MBA, this study looked further at the cultural models and the cultural settings that impact
this organization.
Cultural models. Culture is dynamic, invisible, tied to values and beliefs of the
stakeholders, and it often changes and develops over time (Clark & Estes, 2008). While it is
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

57
relatively easy to observe in many organizations, it is difficult to decipher (Schein, 2017) and
culture is almost always different between groups in the same organization (Erez & Gati, 2004).
As a result, it is important to understand the operating cultural models that influence
performance and determine how the cultures across organizations and in different departments
can effectively coexist to help the overall organization grow.
Collaboration and acceptance of new learning models. The Online Learning Team at
the UFL has developed an internal culture that is perceived to be somewhat innovative, and even
radical, in its approach to change some of the paradigms tied to traditional residential, graduate-
level learning models. Schneider, Brief, and Guzzo (1996) discussed the need for teams to be
radical in their actions in order to make the necessary impactful change that will endure for years
to come. The UFL Online Learning Team understands the importance of this value as it works
to help the organization grow both its deliverables and its reputation in a relatively new area of
graduate learning. At the same time, UFL Online Learning Team must find ways to effectively
recognize and support the more traditional culture that is prevalent in other areas of the
organization, create an environment of mutual collaboration among other the departments, and
work to build acceptance of different, non-traditional learning models at the UFL.
Schneider et al. (1996) described several organizations that, like the UFL, created small,
focused, and nimble teams to launch a new division when they faced fast-paced competitors
entering the market. In the case of an organization like AT&T, the leaders were challenged to
change the mindset of their employees who had historically operated a monopoly and had both
the time and the resources to slowly react to market needs. GM, on the other hand, faced
significant losses in market share and had to make decisions to launch an innovative division at
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

58
Saturn that countered their traditional culture of slow, methodical, and minor reactions to the
market.
The business school at the UFL faced, and still faces, similar challenges. The more
traditional leaders at the UFL business school value the historically strong UFL brand, its steady
success at slow, continuous growth, and its reputation for consistent, high-quality graduate-level
education. However, the market is changing and the next generations of students and future
leaders are looking for something different. The UFL leadership recognized this growing need
and responded by creating a separate department for the Online MBA.
The culture in the Online MBA department and of the new UFL Online Learning Team is
described as having values and beliefs that closely parallel those of the Saturn employees. At
Saturn, the teams had a willingness to take risks and the courage to fail fast, held each other
accountable, learned multiple skills to backfill others when needed, responded to the needs of
each other and their customers, treated each other with respect, and valued partnerships (Hanna,
2010). If the UFL team can use these skills to bridge their culture with other organizations at the
UFL and collaborate with those in other departments, they may be able to close the gap to
achieve their goals.
Leadership and focus on overcoming perceived risks of online learning. In order to
begin bridging the cultural gaps at the UFL, and creating an environment of understanding and
support, there needs to be leadership with a focus on building trust across departments and with
other key leaders at the business school as they work to overcome the perceived risks of change
with this new model. Agocs (1997) suggested one of the first steps in creating this trust is to
diagnose and address the overall institutional resistance to organizational change. The most
effective leaders or change agents exercise personal power to manage institutional resistance.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

59
The personal power of the leader often stems from “their expertise (and reputation) . . ., their
knowledge of the organization, their personal and collective influence and political skills, their
ability to mobilize support . . ., their commitment and perseverance, and their personal courage”
(Agocs, 1997, p. 920). The leader or change agent becomes responsible for addressing any
denials of the need for change, handling the rest of the organization’s refusal to own it,
uncovering the sometimes undetectable refusal to implement the agreed-upon plan, and
countering any deliberate actions to block change that has been initiated (Agocs, 1997). The
leaders of the Online Learning Team are responsible for managing the institutional resistance to
organizational change at the UFL.
In addition, the leaders of the UFL Online Learning Team work in what Senge (1990)
calls a learning organization. As a result, the role of the leader is significantly different than
those in other organizations in that they are “designers, teachers, and stewards” (Senge, 1990,
p. 9). The leaders in learning organization are responsible for building a shared vision among the
team, changing previous mental models that are tied to traditional ways of doing things, and
ensuring the team members are growing their own capabilities to prepare for future needs of their
customers (Senge, 1990). At the UFL, the leaders of the Online Learning Team are also
responsible for providing clarity and support to achieve goals while educating others in the
organization and helping to reduce or eliminate the perceived risks about the methods that
support learning in an online environment.
Cultural setting. The cultural setting or climate, particularly in an organization facing
change, is of critical importance and includes many conditions, factors, and activities that are
specific to each organization. Clark and Estes (2008) describe the setting in the form of a
cultural profile which includes how the organization is different from other organizations, which
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

60
goals and aspects are most valued, what changes are made, who drives the change, who makes
the decisions, and what happens once the change is agreed upon. Particularly in times of change,
the people in the organization are both affected by, and participants in, the cultural setting; as a
result, any solutions recommended for closing performance gaps must be compatible with the
culture (Clark & Estes, 2008). Schneider et al. (1996) supported the idea that change has a
higher probability of taking place when those on the team are challenged and can participate in
both the vision development and the decision-making process. The researchers also noted that
mutual trust among team members and leaders is critical for change to be embraced (Schneider
et al., 1996). Building trust across organizations will help support the future changes that are
proposed.
Creating engaging new learning techniques. The UFL Online Learning Team is
responsible for continually improving online student learning and engagement. One of the
specific strategic deliverables for improving student engagement is a model that includes blended
learning courses that are highly engaging and that also support new, interactive learning
techniques. The first step toward improvement in a learning organization might include the
principle of creative tension, which is described by Senge (1990) as clearly understanding where
we would like to be (our vision) and honestly assessing where we are (our current reality). As
discussed earlier, Clark and Estes (2008) describe this as assessing and understanding the gap
between goals and performance. The resulting tension between these two realities creates an
energy and desire to develop a new plan or model for improvement.
Langley et al. (2009) suggested that for a team to create a new model for improvement,
the team will need to understand what they are trying to accomplish, how they know that the
changes they make constitute an improvement, and what other changes can be made to further
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

61
improve the program. To assist in implementing and assessing these changes, Langley et al.
(2009) further recommended the PDSA (Plan, Do, Study, Act) framework. This framework
would allow the UFL Online Learning Team to review the feedback from their students, create
plans for updating their learning designs and programs, test and analyze the outcomes to
determine whether or not they resulted in improvements, and then implement a more formal
rollout of new program elements to make them permanent (Langley et al., 2009).
Collaboration for designing effective curriculum in online learning. While the UFL
Online Learning Team, and specifically the instructional designers, are responsible for designing
and building an online curriculum for the graduate students in the online MBA, they must also
collaborate closely with the faculty. This collaboration will help ensure the curriculum for each
class includes creative opportunities for students to reflect, discuss, and debate concepts from
their readings and pre-work rather than solely listening to lectures. In a learning organization
where multiple groups are responsible for the deliverables to effect change, the concept of action
research, and communities-of-practice play an important role (Hendry, 1996). These
communities-of-practice are developed among groups of people, and sometimes across
organizations, that are focused on achieving the same goals. These specialized communities
bring together multiple perspectives, opinions, and data, and they challenge each other with new
ideas. They are responsible for effectively managing conflict that results in better outcomes and
solutions, and they are more likely to try new methods in experiential learning. Finally, these
communities-of-practice often have a significant impact on the overall development of the
organizational culture (Hendry, 1996). The more often the UFL Online Learning Team can
engage new groups or communities-of-practice in collaborative learning and creative design
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

62
thinking, the better their chances of creating engaging curriculum and impacting the evolving
culture in their organization.
Table 4 reflects the organizational mission, the performance goal, performance measure,
and four organizational influences discussed in this literature review.

Table 4
Organizational Influences on Organization of Focus Goal
Organizational Mission
The mission of the University of Future Leaders (UFL) is to transform from an 18th century
education model to a 21st century business education model that differs dramatically in content,
professors, delivery, and serving student needs. The mission of the UFL is to provide an innovative
and entrepreneurial learning environment that mirrors business practices while expanding thought
leadership, identifying key global opportunities, and exploring critical issues.
Organizational Performance Goal
By September 2019, the UFL Online Learning Team will develop a strategic plan to identity,
measure, and improve online student learning and engagement that results in preparing graduate
students with the most relevant competencies for future leadership roles in business.
Organizational Performance Measure
Be recognized in the top five online MBA programs in the U.S. university rankings by the end of
2019 and reach number one by the end of 2020 as measured and reported by respected publications
such as U.S. News & World Report, Bloomberg, and others who measure online university programs.
Assumed Organizational Influences Organization Influence Assessment
Cultural Model Influence 1:
The Online Learning Team needed
to work with the UFL business
school leaders and faculty to create
a culture that understands and
values different types of learning
models based on the needs of
different students and industry
business leaders.
Interviews: Dean’s Cabinet & Online Learning Team
Questions (sample):
Describe your experience with teaching online courses.
Do you believe this model is as effective for graduate-level
students as a residential model?
How would you describe student learning in general and
learning outcomes being met in online classes in comparison
to a residential learning experience?
Describe the level of engagement needed with online
students? Does this differ from residential?

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

63

Table 4, continued
Assumed Organizational Influences Organization Influence Assessment
Cultural Model Influence 2:
The Online Learning Team needed to
build a culture of trust among other
business school leaders and faculty to
overcome the perceived risks of
online learning methods.
Interviews: Dean’s Cabinet & Online Learning Team
Questions (Sample):
What do you believe are the key advantages to online
learning at the graduate level?
What do you believe are the key concerns?
Cultural Setting Influence 1:
The Online Learning Team needed to
work with faculty to ensure the
curriculum for each class includes
creative opportunities for students to
reflect, discuss, and debate concepts
from their readings and pre-work
rather than listen to lectures.
Interviews: Online Learning Team
Questions (sample):
What course components (readings, lectures, guest speakers,
individual assignments, collaborative projects, in-class
brainstorming, discussions, debate) most (least) contributed
to development of your understanding/skills?
Cultural Setting Influence 2:
The Online Learning Team at the
graduate level needed to understand
how their organization and goals
disrupt or fit in with the overall
strategy for the business school.
Interviews: Online Learning Team & Dean’s Cabinet
Questions (sample):
What is the strategic role of an online learning program at
the UFL business school?
What are top three opportunities for developing a program
that will ensure the UFL online MBA reaches its
organizational goal?
What are the top three disruptors for having an online MBA
at the UFL business school?
How will resources be managed for the online MBA relative
to the residential MBA programs?

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

64
Conceptual Framework: The Interaction of Stakeholders ’ Knowledge and Motivation and
the Organizational Context
The purpose of a conceptual framework is to provide a visual representation about what
might be going on in an organization, what issues might be prevalent that tie back to the setting
and the people, what influences are in play, and how these influences are interacting to help or
hinder the organization in achieving their goals (Maxwell, 2013). The conceptual framework
takes into account the researcher’s disciplinary orientation (also known as a theoretical
framework such as business, academic, medical, etc.), which is the lens through which the
researcher views an organization (Merriam & Tisdell, 2016). In addition, references to recent
literature and theories on the topic of study, as well as the personal experiences of the researcher
were included in the model (Maxwell, 2013; Merriam & Tisdell, 2016). This framework ties
back to the research questions and helps inform the overall design of the study (Maxwell, 2013).
Figure 2 is a visual representation of the conceptual framework for how the University of
Future Leaders Online Learning Team’s knowledge, motivation, and organizational influences
interact to help the team achieve its goals.
While knowledge, motivation, and organizational factors often operate independently
from each other to influence the stakeholders, they must interact closely to ensure the
organizational goal can be achieved (Clark & Estes, 2008). Figure 2 demonstrates how the
organization of focus influences of knowledge and motivation are represented together inside the
green rectangle. The green arrows express that these two influences closely interact and affect
one another. The organizational influences are separate from the organization of focus
influences but are still important influences, as shown with the purple arrows, in helping the
team achieve its goals.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

65

Figure 2. UFL Online Learning Team conceptual framework

The knowledge influences in Figure 2 are listed in orange and are tied to making sure the
organization of focus understands both what competencies need to be learned in an online MBA
program (conceptual) and how to implement the most effective learning designs to teach them
(procedural). Clark and Estes (2008) pointed out that providing an environment where people
come together to learn about what they need to accomplish and also how to work together to
execute a plan helps generate new conceptual knowledge to solve problems and handle novel
challenges.
The motivation influences in Figure 2 are listed in blue and describe important influences
tied to self-efficacy and goal-orientation. Once the UFL Online Learning Team members gain
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

66
both the conceptual and procedural knowledge to achieve their goals, they need to believe they
are capable of developing highly engaging online learning programs. This belief is impacted by
what Clark and Estes (2008) call a “desire to be effective” (p. 83), and in the case of the Online
Learning Team, this translates into graduate students who ultimately develop the critical
competencies they need to become future industry leaders. The Online Learning Team also
needs to set SMART goals that focus on learning, mastery, and self-improvement in order to
design impactful, engaging and motivating online learning programs for online MBA students.
The organizational influences in Figure 2 are represented in purple and fall inside the
purple rectangle (the UFL Business School), but outside the blue rectangle (the Online Learning
department). The purple arrows in Figure 2 represent the influences the leadership and the
organization at the business school have on the Online Learning team. These are organizational
influences that must be in place in order for the Online Learning Team to achieve the its goal.
Clark and Estes (2008) noted that even when team members have the appropriate amount of
knowledge and skills and are highly motivated, gaps in their organizational structure and culture
can impact their ability to achieve their goals. In the case of the UFL Online Learning Team, the
organizational influences included having support from, and collaborative relationships with,
other team members inside their department, as well as other administrators and faculty in the
UFL business school. The Online Learning Team also needed to believe the work they do to
develop new online learning models will be accepted and viewed as legitimate by others in the
organization. Separately, the Online Learning Team needed support and recognition from the
Dean’s Cabinet and the rest of the organization at the business school in order to create blended
learning courses that include engaging and interactive learning techniques. The Online Learning
Team also needed to work with the business school faculty to ensure the curriculum for each
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

67
class includes opportunities for students to reflect, discuss, and debate concepts from their
readings and pre-work rather than solely listening to lectures.
Finally, while the overall organizational global measure is to be recognized in the top five
online MBA programs in the United States university rankings by the end of 2019 and reach
number one by the end of 2020, there were several key measures that ensured the university hit
this objective. The most critical measure, and the one weighted most heavily, is tied to student
engagement. The organization of focus goal in Figure 2 is identified with a red arrow and it is to
develop a strategic plan to identify, measure, and improve online student learning and
engagement that results in preparing graduate students with the most relevant competencies for
future leadership roles in business.
In summary, the organizational influences outlined above are critical for ensuring the
UFL Online Learning Team can close any gaps and improve online student learning and
engagement in the online MBA program. Specifically, the Online Learning Team needed to
know what to teach in order that their outcomes deliver the competencies required for their
graduate students. They needed to know how to design programs that are engaging in order to
manage student retention. They also needed to be motivated to work toward their goals and
believe they were capable of achieving them.
In addition, understanding and managing any gaps between the organizational culture and
the UFL Online Learning Team was critical. The team needed support from both the leaders and
other stakeholders in the business school; they also needed to experience a culture of
collaboration and acceptance of the new learning models they develop. While the UFL Online
Learning Team’s department culture was strong and supportive, and they had the resources they
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

68
needed, they also needed to believe they have the support and respect from the leaders, the
administrators, and the faculty in other parts of the UFL business school organization.
Summary
The purpose of this study was to understand student learning and engagement in an
online, graduate-level setting and assess the impact these have on students working to develop
critical competencies for future leadership roles in business. In addition, this study evaluated the
degree to which the UFL was meeting its objective to be recognized in the top five online MBA
programs in the United States university rankings by the end of 2019 and reach number one by
the end of 2020. The study focused on knowledge, motivation, and organizational influences
related to achieving the organizational goals. The organization of focus in this study was the
UFL Online Learning Team. Chapter 2 presented important previous research in the area of
graduate-level online learning and student engagement. The chapter also helped set the stage for
how the stakeholders at the UFL business school can improve performance by understanding and
closing the gaps between their current progress and their overall organizational goal (Clark &
Estes, 2008). Finally, this chapter presented several knowledge, motivational, and organizational
influences that impact the UFL Online Learning Team’s ability to achieve their performance
goal and ensure the online MBA graduate students are engaged throughout their program,
achieve the desired learning outcomes, and graduate feeling efficacious about what they have
learned. The conceptual framework for this study was outlined and the validation process will be
described in the Methodology section in Chapter 3.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

69
CHAPTER 3
METHODOLOGY
The purpose of this study was to understand student learning and engagement in an
online, graduate-level setting and assess the impact these have on students working to develop
critical competencies for future leadership roles in business. In addition, this study evaluated the
degree to which the UFL is meeting its objective to be recognized in the top five online MBA
programs in the United States university rankings by the end of 2019 and reach number one by
the end of 2020. The study focused on knowledge, motivation, and organizational influences
related to achieving the organizational goal. The organization of focus in this study was the UFL
Online Learning Team. This chapter will outline the research design and methods for data
collection and analysis.
To evaluate the influences that impact the organization of focus’ ability to achieve its
goal, the following research questions guided the study:
1. What are the UFL Online Learning Team’s knowledge and motivation related to
identifying, measuring and improving online student learning and engagement?
2. What are the UFL Online Learning Team’s knowledge and motivation related to
leadership competency requirements for graduate students?
3. What is the interaction between organizational culture and context and the UFL
Online Learning Team’s knowledge and motivation?
4. What are the recommendations for the UFL Online Learning Team’s organizational
practice in the areas of knowledge, motivation, and organizational resources?
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

70
Participating Stakeholders
While there are multiple stakeholder groups who could contribute valuable information to
this study, the primary focus was on the UFL Online Learning Team and additional data was
collected from one supporting stakeholder group. Interviews were conducted with both the
Online Learning Team and the UFL Dean of the business school and his Cabinet. The data
collected from the interviews with the Dean and his cabinet were included as supplemental
information for the purpose of enhancing the researcher’s understanding of the organizational
influences including the cultural models and cultural settings for this study.
As a member of the UFL Business School Executive Education Department, the
researcher has been working closely with the Vice Dean of Online Learning to develop a
separate new online, specialized masters for the business school. The Vice Dean developed and
launched the new online MBA in 2014-2015 and offered the researcher open access to the
Online Learning Team and to the alumni for this study. In addition, the researcher has been
working with the UFL Dean of the business school and his Cabinet. The Cabinet is highly
interested in knowing more about how to engage students in online learning primarily because it
is expected that online courses will become an important source of revenue for the business
school in the future.
The organization of focus includes the associates in the Online Learning Department at
the University of Future Leaders (UFL). This team was created in 2014 when the new online
MBA was being designed at the UFL business school. This team currently includes 13
members: the Vice Dean, an Assistant Dean, and three operational teams led by Associate
Directors. The team is responsible for Instructional Design (five members), Online Media and
Technology (four members), and Student Services (two members). These are the associates who
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

71
are ultimately responsible for achieving both the performance goal of strong student engagement,
learning, and the development of key competencies, as well as the objectives tied to ranking. All
employees in the Online Learning Department were invited to participate in the interviews for
this study.
The key stakeholder group is the Dean’s Cabinet, an organization who supports the
Online Learning Team. This support group includes the Dean of the business school at the UFL
and seven members on his Cabinet. They work in multiple areas of the business school and
impact the cultural models outlined in the conceptual framework. The Cabinet is responsible for
making decisions about the overall strategic direction of the business school at the UFL and
about new growth opportunities like the online learning programs. The Dean’s Cabinet members
provide the organizational leadership support for the online MBA.
Interview Sampling Criteria and Rationale
Criterion 1. Leaders and associates in the UFL Online Learning Department. All
leaders and members (those reporting to each of the leaders) of the department were invited to be
interviewed. They provided insights about the goals of the department and how the members of
the department were performing against the goals to improve student learning and engagement,
and serve the needs of the students and the business leaders.
Criterion 2. The interviews included both the members who have been in the
department since the launch of the online MBA in 2015 and those who have joined the
department since then. The members who have been in the department since the launch provided
history and context regarding their performance goals and the newer members enhanced this data
with different perspectives about the current state and possible new solutions.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

72
Interview Sampling Strategy and Rationale
Several non-random techniques were used as part of the strategy for this first round of
interviews. This strategy was used to answer the research question about the UFL Online
Learning Team’s knowledge and motivation related to understanding how to improve student
learning and engagement. The non-random techniques included convenience and purposeful
sampling and resulted in a typical sample group of respondents (Johnson & Christensen, 2014;
Merriam & Tisdell, 2016). The sampling was convenient as a result of having both access and
permission to interview the small group of 13 primary stakeholders in this study. In addition, the
sampling was purposeful in that the individuals who were interviewed were fundamentally at the
center of the conceptual framework and made up the department who is directly responsible for
achieving the stakeholder goal. Emails were sent to each associate to set up interview times.
Supplemental Interviews and Data — Sampling Criteria and Rationale
Criterion 1. Dean and members of the Dean’s Cabinet at the UFL Business School.
Four specific members of the Cabinet were invited to participate in an interview. These four
members are the leaders who are most closely connected to graduate-level programs in the
business school. They included the Dean of the UFL business school, the Senior Associate Dean
and Chief Financial Officer, the Vice Dean of Graduate Programs, and the Vice Dean for Faculty
and Academic Affairs. These members directly influence the cultural settings across the
business school at the graduate level. They also support the Online Learning Department for
tactical allocation of resources such as faculty, technology, and financing.
Supplemental Interviews and Data — Sampling Strategy and Rationale
The second round of interviews incorporated a strategy that also included non-random,
purposeful sampling. However, this round of sampling resulted in what Johnson and Christensen
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

73
(2014) called a unique sample group of respondents. This strategy was used to answer the
research question about the interaction between organizational culture and context and how it
impacts the UFL Online Learning Team’s knowledge and motivation. The Cabinet members
were invited via email to a short “live” one-on-one meeting with the researcher where the
researcher shared the overall goals, the research questions, and purpose of the study. The
Cabinet member were then invited to participate in an interview. Once they agreed, a separate
interview meeting was scheduled. The interviews with the Cabinet members took place after
completion of the first round of interviews with the Online Learning Team.
Data Collection and Instrumentation
The data collection approach for this qualitative study was focused on the use of both
document analysis and participant interviews. The documents which were analyzed were the
letters, memos, and articles published by the most senior leadership in the university. Several
documents were created by the UFL Dean of the Business School and several articles were
written by both the Provost of the university and also the university President. The document
analysis method was selected because it allowed the researcher to analyze actual published
perspectives about the strategic vision for online learning at the time the program was being
planned and designed. In addition, the document analysis provided an opportunity for the
researcher to compare and contrast the organizational stakeholder goals to the overall mission of
the university.
The participant interviews helped the researcher collect data from the stakeholders
directly involved in and responsible for delivering the overall goals of the Online Learning
department at the UFL. The interview method was selected because it allowed the researcher to
collect and analyze the data from the perspective of the actual participants (Patton, 2002). It also
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

74
helped the researcher gain an understanding of the online MBA program as it stands today,
where it should be heading in the future, and the challenges associated with growth.
This overall approach allowed specific data to be gathered to answer the first research
question about the UFL Online Learning Team’s knowledge and motivation relates to how
improving student learning and engagement in this relatively new online graduate program. It
also supported an analysis of how improving student engagement will impact the team’s ability
to achieve the performance and organizational goals. Once the data was gathered and analyzed,
the researcher better understood how knowledge and motivation impacted the team’s ability and
willingness to develop a program that resulted in outcomes that include the desired competencies
for future leadership roles. This data, in turn, helped answer the second research question.
Finally, the interviews helped the researcher develop a deeper understanding of the impact of the
broader organization and the meaning of culture at the UFL. The approach exposed how cultural
models and settings influenced the UFL Online Learning Team’s ability (knowledge) and
willingness (motivation) to make significant progress toward improving student engagement and
growing the online learning programs.
Interviews
Interview protocol. The interview protocol used in this study included a list of 15 semi-
structured, open-ended questions (see Appendix A and B). This type of inquiry format allowed
for the greatest flexibility during the interview, supported an environment where the researcher
could probe and ask follow-up questions, and allowed the conversation to evolve so that new
ideas might be uncovered. Merriam and Tisdell (2016) noted that a qualitative study with
interviews helps the researcher to uncover: “1) how people interpret their experiences, 2) how
they construct their worlds, and 3) what meaning they attribute to their experiences” (p. 24).
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

75
While questions in this study were listed in particular order in the protocol (see Appendix A and
B), the semi-structured approach, as described by Merriam and Tisdell (2016), allowed for
questions to be asked in any order and for issues to be explored as they developed in the
discussion. The questions also closely aligned with the conceptual framework by demonstrating
how the UFL Online Learning Team was able (with the appropriate knowledge influences),
willing (with the right motivational influences), and supported (with the right organizational
structure and resources) to achieve its performance goal.
Interview procedures. Once the researcher received approval from the Institutional
Review Board (IRB), the UFL Online Learning Team was invited to participate in this study.
Invitations to participate were sent to each member via email. Formal interviews were
conducted in either private offices or private conference rooms. The setting for these interviews
was selected to ensure the participants felt comfortable and trusted that the discussions would be
kept confidential. The interviews lasted approximately 45-60 minutes and were recorded with
permission from the participants. The interviews were later transcribed by a professional service
and kept confidential as agreed.
The stakeholder or organization of focus was the Online Learning Team, which included
13 members. The participants were interviewed one time. The supplemental stakeholder group
was the Dean’s Cabinet which included eight members. Three of these members were
interviewed once.
Documents and Artifacts
The approach for document analysis in this study was the collection and analysis of the
letters, memos, and articles published by the most senior leadership in the university, including
several from the Dean of the business school, the university Provost, and the President of the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

76
university. These documents reflected the published perspectives about the strategic vision for
online learning at the time the online MBA was being planned and designed. In addition, the
document analysis provided an opportunity for the researcher to compare and contrast the
organizational stakeholder goals to the overall mission of the university. The researcher was not
able to gain access to any published performance goals for the members in the department
because the department did not use formal objectives. The researcher received the university
documents directly from one of the cabinet members who is responsible for leading the Online
Learning Team. These documents were created and published at the university. As a result, they
are considered reliable, authentic, and relevant which are all important factors outlined by
Merriam and Tisdell (2016) for use of documents in data collection methods.
Data Analysis
Once the researcher received IRB approval, interviews were scheduled with 15
participants; 11 team members from the Online Learning Team at the UFL, three leaders from
the Dean’s Cabinet, and one faculty member. Each interview was held in a closed office and
recorded with permission from the participants. Once the interviews were completed, the
recordings were transcribed by a professional service provider. The transcriptions were then
filed as word documents in various organized folders and also uploaded into Atlas.TI, an online
data analysis tool. The researcher also kept field notes using Evernote to capture thoughts,
impressions, and other key observations for reference in the future.
A codebook was developed to help categorize the topics that evolved from the interviews
and the field notes. The process described by Harding (2013) was used for developing these
codes including:
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

77
1) identifying the conceptual theme and creating a category, 2) bringing together codes
from different illustrative issues into the category, 3) creating sub-categories to reflect
different elements of the conceptual theme, and 4) using the conceptual theme to explain
relationships between different parts of the data and to build theory. (p. 112)
Axial codes were created based the overall themes, and later, additional emergent codes were
created as the participant responses were reviewed. The use of a codebook helped to identify the
frequency and/or typicality of topics or themes that occurred across the various interviews.
During the last steps in the data collection process, the researcher reviewed and analyzed several
documents and artifacts that applied to the study.
The final phase in the process was to analyze the data collected, looking for important
themes and key findings that addressed both the research questions and explored the knowledge,
motivation, and organizational influences that were outlined in the conceptual framework for this
study. The overall results and findings were then developed using the outcomes from both the
qualitative interviews and the document analysis to answer the research questions in this study.
Credibility and Trustworthiness
To establish credibility and trustworthiness in this study, the researcher utilized several
strategies. The first strategy was to interview respondents from different departments in the
university, specifically the Online Learning Team and the Dean’s Cabinet. This strategy created
multiple sources of data that could be compared, contrasted, and cross-checked (Merriam &
Tisdell, 2016). A second strategy was respondent validation, or member checks, which allowed
the researcher to validate the data and seek feedback about the outcomes in follow-up interviews
(Maxwell, 2013). As previously mentioned, the Online Learning Team was interviewed first,
and several of these respondents were asked if they would be willing to do a follow-up interview
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

78
after the Dean’s Cabinet interviews are completed. A third strategy was the use of peer reviews
which was described by Merriam and Tisdell (2016) as a method to reduce bias and help increase
validity. For this study, the data collected during the interviews was compared with other studies
that described student engagement in online learning, and the data was shared with peers and
other experts in online learning for additional feedback. This process was done during the data
analysis phase of the study.
Validity and Reliability
As in any research, the overall credibility of the study will depend on both the validity
and the reliability. Internal validity is used in constructivist qualitative research to describe how
closely the research maps reality (Merriam & Tisdell, 2016). The reliability of a study is a
measure of how well something can measured consistently (Salkind, 2017) and how likely a
study, if repeated, might result in the same findings (Merriam & Tisdell, 2016). For this
qualitative study, the overall use of triangulation helped establish and maintain validity. Data
was collected and then checked against current and past research. In addition, the findings were
member checked and peer reviewed for independent analysis and interpretation. Reliability was
assessed during the data analysis phase to determine whether or not they were consistent with the
expectations and data collected (Merriam & Tisdell, 2016).
Ethics
The process of discovery for this study, along with its acceptance as credible and
trustworthy research, was heavily dependent upon the perceived ethics of the researcher and the
process that governed the collection, analysis, and management of data. For this form of study,
which involved human subjects in a university setting, the federal government established an
IRB to review and monitor all research (Glesne, 2011). The guidelines set out by the IRB
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

79
require that participants are fully informed about the study and its purpose, know they can leave
the study at any time without repercussion and are informed about any potential risks (Glesne,
2011). The researcher in this study went through the IRB process and followed the guidelines
set forth. The participants were provided an information sheet which outlined these
commitments along with an assurance that the data collected will remain confidential and stored
in a secure environment.
The University of Future Leaders (UFL) Online Learning Team is an organization that
supports all online education in the UFL School of Business where the researcher is employed.
Until recently, the researcher had no relationship with the members of this team. However, a
new program that was proposed in 2015 was approved in 2018. The researcher began working
with the UFL Online Learning Team to build the new specialized online masters for executives
in a targeted industry. The observations, reflections, and outcomes from this study will be used
to help strengthen all of the online programs at the UFL. The outcomes will also help the team
develop a strategy for strong student engagement in the new graduate degree.
The researcher worked in a separate department than the Online Learning Team in the
UFL School of Business. The benefits of the study for the department will be shared to help
support strong outcomes for other new online masters programs at the UFL. Finally, the
researcher ensured the Online Learning Team knew their participation in the study was
voluntary. As a result of clear, consistent communication, the participants appeared to
understand the researcher’s role and did not appear to feel pressured or coerced to participate.
The participants were also assured that what they shared was valued and would be kept
confidential, and the risks were eliminated as much as possible (Rubin & Rubin, 2012).
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

80
Limitations and Delimitations
It is important to recognize and address some of the limitations that may have been
present during the data collection, analysis, and reporting phases of this study. The researcher
had a strong vested interest in understanding all areas of student engagement to launch a separate
online masters program for future students. The researcher was also a student in an online
graduate-level program and needed to be highly conscious of personal perceptions and biases, as
well as being cautious about influencing participants. In addition, there had been some hesitancy
among UFL senior cabinet members to fully embrace graduate-level learning in an online
environment. For all of these reasons, the researcher was careful to avoid the inclination to lead
the interviews in any way. Creating a very specific questioning route for both sets of interviews
helped ensure the data collection and analysis study stayed on track.
The researcher has a background in both business and academia. As a result, the
outcomes from this online MBA program impacted both the researcher’s role and also the clients
who supported the program. To counter any potential bias, the researcher followed the protocol
closely and worked to remain neutral during the interviews. In addition, the researcher engaged
in critical self-reflection to ensure the interviews were conducted without researcher guidance or
response (Merriam & Tisdell, 2016), and the data analysis was carefully constructed to avoid any
previous assumptions or biases.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

81
CHAPTER 4
RESULTS AND FINDINGS
The purpose of this study was to understand student learning and engagement in an
online, graduate-level setting and assess the impact these have had on students working to
develop critical competencies for future leadership roles in business. In addition, this study
evaluated the degree to which the University of Future Leaders (UFL) was meeting its goal to be
recognized in the top five online MBA programs in the United States university rankings by the
end of 2019 and reach number one by the end of 2020. This problem was important to address
because business students who are hired for future leadership positions will be required to have
advanced skills to handle both domestic and global challenges. These students look to the most
innovative business schools and student-centered faculty who offer new and creative options for
learning and are willing to help them prepare for the current and future disruptive and rapidly
growing markets. The study focused on knowledge, motivation, and organizational influences
related to achieving the organizational goal.
While there are several important stakeholder groups who contribute to helping the
organization achieve its goals, the organization of focus in this study was the UFL Online
Learning Team. The research questions listed below were used to guide the evaluation study:
1. What are the UFL Online Learning Team’s knowledge and motivation related to
identifying, measuring and improving online student learning and engagement?
2. What are the UFL Online Learning Team’s knowledge and motivation related to
leadership competency requirements for graduate students?
3. What is the interaction between organizational culture and context and the UFL
Online Learning Team’s knowledge and motivation?
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

82
4. What are the recommendations for the UFL Online Learning Team’s organizational
practice in the areas of knowledge, motivation, and organizational resources?
Data was collected during formal interviews with participants in the Online Learning Department
at the University of Future Leaders (UFL). In addition to the leaders and members on the Online
Learning Team, other groups who participated in the interviews included the Dean of the
business school and several other leaders from the Dean’s Cabinet. Finally, the researcher
completed a document analysis of several departmental reports to further validate and triangulate
the data gathered during the interviews.
Participating Stakeholders
The internal stakeholder or organization of focus for this study was the UFL Online
Learning Team (OLT) which is part of the Online Learning Department at the University of
Future Leaders (UFL). This group was formed in 2014 when the new online MBA was first
being developed. With the exception of one team member, all of the Online Learning Team
members were involved prior to the launch of the new Online MBA program in 2015.
The Dean leads the overall business school at the UFL, including both on-campus and
online undergraduate and graduate programs. The Vice Dean for Online Graduate programs is
the Dean’s Cabinet member who provides leadership and strategic direction for the OLT. The
Director/Assistant Dean leads the department and has direct management responsibility for the
deliverables, and the other key teams include the Student Services Team, the Instructional
Design Team, and the Online Media and Technology Team. The Online Learning Team has
primary responsibility for achieving the organization’s performance goal. In order to fully
explore the organization, collect relevant data, and answer the research questions, qualitative
interviews were completed with a total of 15 participants. A total of 11 of the 12 team members
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

83
from the Online Learning Team were interviewed for this study. In addition, interviews with one
online faculty member and three team leaders from the UFL Business School Dean’s Cabinet
were completed. To protect the identity of the participants, the following pseudonyms were
used: Dean, Cabinet Member (One, Two, Three), Team Leader (One, Two, Three) and Team
Member (One through Eleven).
Findings
The findings for this study are presented in this section using the Clark and Estes (2008)
conceptual framework which helped the researcher evaluate the gap between the current
organizational performance and the desired goals for the internal stakeholders in the organization
of focus. Specifically, these findings enabled the researcher to identify the gaps in both the
knowledge and motivation related to identifying, measuring, and improving online student
learning and engagement in the Online Learning Team. They aided the researcher in identifying
the knowledge and motivational gaps in Online Learning Team’s ability and willingness to build
a program that ensures that graduate students have the competencies they need to be successful
in the global workforce. Finally, the results and findings helped the researcher identify the
organizational gaps in support and the type of cultural environment needed to continue growing
the program. Chapter 4 outlines the findings of this study which are organized by research
question. Several findings that tie back to the knowledge, motivational, and organizational
influences will be identified and analyzed, and key findings will be discussed in terms of how
they help answer the research questions identified in this study.
Research question 1 and research question 2 refer to the knowledge and motivational
influences for this study. These influences are reflected in Table 5.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

84
Table 5
Knowledge and Motivational Influences
Knowledge Influences Motivational Influences
1. The Online Learning Team needs to
know how to implement the most
effective methods for keeping students
engaged and teaching transferable skills
in an online environment. (Procedural)
1. Self-Efficacy: The Online Learning Team need
to believe they are capable of effectively
developing online learning and instruction that
will engage online MBA students and ensure they
have learned the critical competencies for a
rapidly changing business market.
2. The Online Learning Team needs to
know how the competencies that online
MBA graduates learn will help secure
leadership roles in a global business
environment. (Declarative Conceptual)
2. Goal Orientation: The Online Learning Team
will need to set smart goals that focus on learning,
mastering, and self-improvement in order to
design online learning for MBA students that
engage and motivate students.

Research Question 1
The first research question in the study asked if the Online Learning Team (OLT) knows
how to implement the most effective methods for keeping students engaged in an online
environment, along with how motivated they are to identify, measure, and improve student
engagement in the future. Knowledge Influence #1, which is related to the critical elements of
student learning and engagement, was partially validated during the study to answer the first
research question. In addition, Motivational Influence #1, which is related to self-efficacy, was
partially validated and Motivational Influence #2, which is related to goal orientation, was
validated to answer the first research question. Each of these influences will be clearly outlined
in the following sections. The following findings emerged which helped the researcher
understand the extent to which knowledge and motivation influenced the organization and
provided answers to the first research question.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

85
 The OLT leaders and instructional designers were knowledgeable about developing
the critical elements of student learning and engagement, but still needed to develop
tracking and measures.
 The OLT was motivated to improve student engagement by building relationships
with faculty and leaders, but needed more recognition from leaders.
 The OLT members did not know the formal goals for the department related to
student learning and engagement, but had some general knowledge and a high level
of motivation for building a high-quality program
Team had Knowledge about Student Engagement, but Limited Tracking or Measures
Knowledge Influence #1 was partially validated during the study in order to answer the
first research question. This knowledge influence stated that the Online Learning Team needed
to know how to implement the most effective methods for keeping students engaged and
teaching transferable skills in an online environment. The researcher learned that the Online
Learning Team members were clearly able to identify several critical elements of student
engagement including the importance of the faculty, the impact of student peers, and the use of
multiple forms of learning. However, the team had limited quantitative data that measured and
tracked the effectiveness of the OLT methods for actually keeping students engaged. As a result,
this knowledge influence was only partially validated.
In the research about student engagement in the online environment, Meyer (2014)
referred to the Community of Inquiry (CoI) model developed and discussed in the study by
Garrison et al. (2000). This early research on the CoI framework focused on three critical areas
of collaborative learning: teaching presence (the presence of an engaging and collaborative
faculty member), social presence (the feelings and emotions tied to being a part of a community
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

86
or group), and cognitive presence (referring to some of John Dewey’s early work and includes
the balance of reflection and critical thinking). Through the discussions with the participants in
this study, observations in the two areas of teaching presence and social presence were discussed
at length and are outlined in this section under research question 1. The focus on cognitive
presence including the importance of critical thinking will be discussed in the section under
research question 2.
Each of the participants in the study were asked several questions about the most
effective methods in online learning for building strong student engagement. In addition, the
participants were asked about the conditions when students were particularly engaged. In each
of the 15 interviews, the participants had clear responses and ideas about how to engage students.
Team Leader Two, who has responsibility for the instructional designers and works most closely
with the online faculty, shared that the overall framework for engaging students in the online
MBA was multi-dimensional; students simultaneously engage with the content, with the faculty
(teaching presence), and with other students (social presence). However, the tracking and
measurement of these program elements are still in the very early stages and would need a great
deal more focus in the future. The following sections outline the findings related to the three
critical elements of teaching presence, social presence, and measures.
Critical element one: The Online Learning Team understood the importance of
teaching presence. The Online Learning Team had clear knowledge about the first concept of
importance which is faculty or teaching presence. Vaughan and Garrison (2006) discussed three
important elements for consideration in teaching presence: (1) course design and organization;
(2) facilitation of discourse to engage students; and (3) direct instruction during the interactions
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

87
with students. All of these elements were found to be instrumental in improving overall student
engagement and learning (Garrison et al., 2010).
All 11 Online Learning Team members in this study talked extensively about the
importance of having the faculty and the instructional designers work closely together to develop
and design the coursework. Once the course content was developed, they worked together to
create faculty scripts, videotape their recordings, and then work with the students to help them
gain knowledge prior to class by reading case studies, watching videos, and participating on
blogs. The three important areas of teaching presence discussed above included instructional
design, facilitating discourse with instructional materials, and direct instruction and leadership in
the virtual classroom. Each of these will be explored further in the following sub-sections to
demonstrate how the UFL Online Learning Team incorporated these concepts into their
curriculum.
Instructional design and organization. According to the participants in this study, the
focus on collaboration between the instructional designers and the faculty was one of the most
important factors in the development of the program. While this type of collaboration was not
the normal process for on-campus or residential courses, the faculty learned that the instructional
designers were critical subject-matter experts who had a great deal of knowledge and experience
in developing the most engaging pedagogy. As a result, processes were implemented to help
facilitate future development. Cabinet Leader Two discussed the significant commitment, as
well as the extensive process, required to develop an online course. The Online Learning Team
was required to meet with each of the faculty members over a four to six month period of time
prior to launching the new course. In addition, the faculty had to sign a contract, or
Memorandum of Understanding (see Appendix C), prior to the launch where they agreed to work
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

88
closely with the instructional designers and the production team for a minimum of four to six
hours every week. These meetings set the stage for productive meetings between the faculty and
the subject matter experts where the faculty could brainstorm with the subject-matter experts
about the most effective methods for learning and engagement, train with both the instructional
designers and the video production team, and then develop course material that would be highly
effective in the online learning environment. This change in course development methodology
was both new and, at times, controversial when working with tenured faculty who had been
using some of the same content for many years.
Facilitating discourse with instructional materials. In the UFL online MBA program,
according to Cabinet Member Two, the students engage in a unique program unlike any others in
the online MBA learning space. After one three-unit hybrid course (both online and on-campus)
and a residential session, all of the other courses are combined in a series of five courses that
made up 9-10 units each. The faculty worked together with the instructional designers to
develop the courses, and with other faculty to co-teach the integrated courses that were designed
based on important global business themes. The students participate in both synchronous and
asynchronous activities and the course structure helps the faculty create a feeling of continuous
presence. Team Leader Two discussed teaching presence as the ability to create an environment
where the students feel the presence of the professor even when they are not physically present.
The OLT works with the faculty to ensure every assignment is created with important pre-work
that includes reading a case and/or watching a recorded lecture where the students get to know
the faculty prior to meeting them each week in the virtual classroom. Team Leader Two went on
to describe a specific faculty member who has achieved a strong presence in the online
classroom using multiple forms of pedagogy:
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

89
I’ll give you an example . . . the idea being that in [the faculty’s] discussion forums, he’ll
post a problem. The students will work on it. After they’ve worked on it and clicked
submit, a video of him comes up talking about the problem [and] explaining the solution.
The genius of this, of course, is that it creates the illusion of him being there. That took a
lot of time initially for him to develop, but now he can sit back and he has that presence
. . . and seems to be responding directly to students.
The combination of the unique integrated courses, the teamwork between the instructional
designers and the faculty, and the combined faculty teams have helped the overall program
continue to grow its strong teaching presence in this program, according to the participants.
Direct instruction and leadership in the virtual classroom. An additional area that
supports teaching presence is the strategic format for the online MBA at the UFL. It is set up in
a flipped classroom environment described by Team Leader Two as “the inverted classroom
model [where] students acquire knowledge prior to attending the synchronous class session, and
[then] during that synchronous class session they apply that knowledge.” The idea of a flipped
classroom strategy along with several other elements has helped the faculty engage students by
preparing them ahead of class so they can contribute to the class discussion, according to Team
Leader Two. In addition, the faculty are able to actually see all of the students in the classroom
on their computer monitors; this adds to the teaching presence. Cabinet Leader Two said it best:
“. . . we also know that when the faculty is seeing every face on the screen, they are seeing more
faces than they can see in a physical classroom.” The students understand that they will be
missed if they are not in attendance.
The OLT faculty were also using another tool for engaging students in the UFL online
MBA learning environment. The instructional designers were working with the faculty to help
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

90
them understand the power of, and get more comfortable with, cold-calling. The instructional
designers were helping even the most experienced faculty members set the stage at the beginning
of the course by letting the students know that they will be cold-calling on students throughout
the semester. After a bit of practice, the response was strong and the students tended to come to
class prepared and ready to engage, according to the Team Leaders. Team Member Two shared
the following thoughts about this strategy in the synchronous virtual classroom:
The professors I know [who] get the most student engagement are the ones who do cold
calling . . . [the students] probably need to have actually talked about the material with
other students during the week so that they’re prepared to come in for these questions the
professor’s going to ask and to [not] be embarrassed in front of the rest of the class. Cold
calling is a really interesting thing.
The participants in the Online Learning Team were all in agreement that the use of cold-calling
was effective as long as the faculty member prepared the students ahead of time and also
delivered the questions with both respect and a purpose to engage in further discussion rather
than catching the student off-guard.
The concept of developing and growing teaching presence that supports ongoing student
engagement in online learning has many layers. The final topic that was discussed as critical for
the most effective teaching presence, direction, and leadership in a virtual classroom
environment was the importance of feedback, both consistent and timely. Six of the participants
discussed this topic during their interviews. Team Leader One described the most effective
faculty as those who were open, welcoming, and willing to provide honest feedback so the
students always knew how they were performing. According to the three of the participants,
both the type of feedback and the timeliness of feedback was very important to the students. The
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

91
participants commented on the fact that in the online environment the professors seemed more
responsive because they were already using electronic media. One of the participants mentioned
that the professors often hold e-office hours. All of these elements combine to create a teaching
presence that ultimately helps promote student engagement and learning.
Critical element two: The Online Learning Team demonstrated knowledge about
the importance of social presence. The second concept that has been described as important
for improving student engagement is what Garrison et al. (2010) called social presence.
Vaughan and Garrison (2006) broke this concept into three distinct areas: (1) affective
expression where students express thoughts, emotions or beliefs; (2) open communication where
students express agreement and trust; and (3) overall group cohesion including collaboration and
teamwork.
The Online Learning Team launched several important social presence initiatives to
support their online adult learners and these initiatives are further described in the following sub-
sections.
Affective expression. In the first area of affective expression where learners have an
opportunity to share their thoughts and emotions, the students in the Online MBA at the UFL
created their own communication platform using social media and personal technology apps such
as WhatsApp. Team Leader One shared that WhatsApp has become “the most popular form of
communication within the cohort.” Team Leaders One added that students talk about this app
often and say it is pinging them all through the day and night with chat group discussions,
notifications about upcoming events, and information about upcoming assignments and projects
in class. This form of communication provides instant free access to students anywhere in the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

92
world, which is highly effective for students who live outside of the market and as part of an
international online program.
Open communication and group commitment. In the areas of open communication and
group commitment, both the program administrators and the students worked together to build
engagement. One of the formal programs mentioned by three of the participants was the
Ambassador Program. Team Member One described the Ambassador Program as one that used
a small group of students from each cohort to help with both the admissions onboarding process
and also the overall general outreach to new students. The ambassadors are “those who . . . are
willing to share their personal experience in the program with people who are interested in
joining [while providing] a more personal picture of what the program would look like.” Team
Leader One also described the importance of the Ambassador Program and its impact on student
engagement for the ambassadors themselves. These are students who take “ownership of their
educational experience and they’re truly engaged,” said Team Leader One. The Ambassador
Program recruits students who enjoy talking to other new prospective students and who are
willing and eager to discuss their experiences. The ambassadors become the point of contact for
incoming students and help build the feeling of community.
Group cohesion. There is an important focus in the Online MBA on group cohesion
where students have an opportunity to engage in tasks and group work. This is an area where the
participants said the Online MBA program excels. They described the learning management
system as one that allows the students to interact online with blogs and virtual video
conferencing sessions when they are outside of the virtual classroom. The participants also
touted the use of breakout sessions when the students are in their synchronous sessions and
described a new resource called a student resource lounge.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

93
The student resource lounge, according to Team Member One, was designed to be a
“one-stop shop” where students can go to get questions answered just like they might do in a
visit to a student services center on campus. The students can inquire about multiple topics such
as financial aid, professional resources, and library resources. Team Member One shared that
they have been ensuring this resource is known to all of the stakeholders who might have
valuable information for the students. The OLT is “really trying to foster and engage that so that
[the students] feel supported.” The Online Learning Team participants described many support
services such as these with a critical focus on student engagement. The team’s knowledge,
energy, and focus on providing an extraordinary experience for the students were both obvious
and present in all discussions about the available services in the program.
Critical element three: The Online Learning Team had limited tracking and
measurement of student engagement. The participants in the study were also asked about how
they measured and tracked student engagement. As a group, they consistently communicated
that this was an area where there had been a limited focus in the first few years of the program.
However, each of the participants interviewed recognized the need for measurement and
described some of the plans for tracking and measuring the program in the future. Team Leader
Two shared that the OLT was beginning to work on analytics and tracking to measure student
engagement. They were beginning to incorporate a program called Quality Matters to have their
courses reviewed by other universities in addition to implementing several other measures. They
were beginning to work with the online faculty to incorporate the use of rubrics to ensure
consistent measures and feedback in the coursework. They were planning to ask for
participation in a faculty peer program where faculty members would assess other faculty in their
virtual classrooms. Cabinet Leader Two discussed this program as “an important self-
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

94
assessment and collaborative effort to make all of our teaching better.” Finally, the OLT was
beginning to use the learning management system to analyze which students were completing the
assignments and mastering the learning objectives. Cabinet Leader Two concluded with the
following, “There’s going to be an assessment every five years of all of the [business school]
programs.” The Online MBA is now in its third year and, while the development of these
measures is new, this strategic initiative has been described as an important area of focus in the
future for the OLT.
Team is Motivated about Student Learning and Engagement, but Needs Additional
Recognition
In addition to understanding the level of knowledge among the Online Learning Team for
the most effective methods for keeping students engaged and teaching transferable skills in the
Online MBA, the researcher asked questions about how motivated the Online Learning Team
was to identify, measure, and improve student engagement in the future. Motivational Influence
#1 was related to self-efficacy and was partially validated during the study in order to answer the
first research question. This motivational influence identified that it was important to understand
the level of self-efficacy among the team members to determine whether or not they believed
they were capable of effectively developing online learning and instruction that engaged online
MBA students.
The team was found to be motivated to improve student engagement by building
relationships with faculty and leaders, but needed more recognition from the business school’s
most senior leaders. The researcher explored three areas described in the sub-sections below
relating to measuring the motivation and self-efficacy among the participants including the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

95
importance of relationships with the faculty, the importance of recognition by the most senior
leaders, and the importance of continued professional development.
Importance of relationships with the faculty. The Online Learning Team at the UFL
demonstrated a high sense of self-confidence during the interviews. They communicated that,
during the initial early years, they were clear on their goals to develop and launch their program.
Clark and Estes (2008) note that motivation includes three important processes: the choice to
work toward a goal, the willingness to persist until it is achieved, and the ability to apply the
needed mental effort to find solutions and stay the course toward completing the goal. As
evidenced by the team’s ability to get their program up and running in a very short period of time
(under six months), the motivation to achieve their early goals was strong.
Another important element of motivation was the culture of teamwork in the Online
Learning department. The team was highly group-oriented where the team members operated
together and supported one another to build their program. Clark and Estes (2008) describe this
as a “we” culture where the team tends “to be cooperative and invest(s) maximum effort in group
settings when the group determines that a goal is important” (p. 84). Inside the department, this
culture was easy to detect after interviews with the 11 Online Learning Team member
participants. However, there was one topic that came up in three of the interviews that
negatively impacted motivation on this team.
While the culture of teamwork with the faculty had improved tremendously, the
relationships had not always been mutually respectful. The OLT members shared that early on,
the perception of the Online Learning Team (OLT) among the faculty was skeptical, at best.
Two OLT members mentioned that, in the early days, the perceptions of the faculty were that
only faculty could be experts in teaching, and they often shared that they did not need any help
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

96
from those whom they perceived as staff people. One of the OLT members shared, “the whole
idea that my ideas might count for less because I’m not faculty or I’m not tenured” is difficult to
understand. This perceived lack of respect was de-motivating for the instructional designers who
were effectively the pedagogical experts when it came to developing the most effective ways to
teach online. One of the participants asked the rhetorical question, “How can I be effective in
this situation?” The same participant went on to share that, “from the get-go, we’ve been trying
to win the hearts and minds of faculty to develop this online MBA.”
A second participant shared similar thoughts and demonstrated how the instructional
designers were eventually able to gain the respect of the faculty. They worked hard to keep the
professors involved, engaged, and interested in each aspect of preparing and executing the
program. The OLT members recognized that, at first, the faculty needed to get used to having a
partner and many refused help during their first year. However, most of the faculty eventually
recognized that the suggestions being made by the OLT would actually make the experience
better for the students and also for themselves. Team Member Two shared, “I don’t know if
that’s a resistance against changing. I think it’s probably just a resistance that they have to
rethink something.” Even with the challenges of gaining respect from the faculty, the Online
Learning Team was able to apply the needed mental effort to find solutions and stay the course
toward completing their goals. They worked very hard to overcome these early challenges and
build relationships with the faculty. Cabinet Leader Two shared the following about faculty
perceptions:
Five years ago when we first talked about the online business, there were many who were
skeptical, who chided us, who said, “This isn’t really learning. I will never teach in those
programs.” [Today], we . . . have at least 30 professors who have been trained (and
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

97
supported) with instructional designers on how to teach online . . . and I believe you
could interview all 30 and they will all say it’s a great way of learning and that they have
learned during the process themselves.
The occasional elements of work environments that destroy motivation described by
Clark and Estes (2008) such as negative feedback, unnecessary rules, work barriers, or constant
competition were not detected in the Online Learning department. The potential organizational
barriers and emotional climate tied to the early relationships with the faculty appear to be behind
them. However, as new faculty join the team of online professors at the business school, this is
an area to keep in mind.
Importance of recognition by the Dean and other business school leaders. Another
important element that impacted motivation in the Online Learning Team is recognition. This
recognition came in many forms, both externally and internally. The external recognition
eventually showed up in the form of ranking and awards. Externally, the Online Learning Team
was recognized in several ways. First, the UFL Online MBA was ranked number 12 by U.S.
News & World Report in the first year after it launched, and one year later in 2017, had moved
up to number eight. In addition, the video production team was honored with the Davey Award
by the Academy of Interactive and Visual Arts for their work in online web and video design; the
curriculum designers were honored by the Online Learning Consortium in 2015 for their creative
instructional curriculum; and the Online Learning Team won two People’s Telly awards which
recognized the best in online video production in the online education category. The work in the
Online Learning Team was being recognized in academic respected circles and this helped
strengthen motivation.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

98
Another important motivator and confidence-builder was the internal recognition from
senior leaders within the business school and at the overall university level. In the early years,
the Dean of the business school had the inspiration, the foresight, and the courage to become a
true pioneer. Cabinet Leader Two shared the Dean’s vision: “To be an innovative business
school, we have to be representing what’s happening in the business world. The global business
world is on digital.” The Cabinet Leaders recognized that the business world was changing and
the needs of adult learners would require learning that fundamentally paralleled the business
environment. Cabinet Leader One shared, “the University has been very, very bullish on online
learning. This is a good way to increase reputation and revenue while at the same time not
having to increase bricks and mortar.”
Today, the Dean is still one of two of the most bullish leaders in the business school
when it comes to online learning. However, three of the Online Learning Team participants
discussed that there is an opportunity to expand the level of recognition among other top leaders
in the business school, and at the university level. There is a perception among some of the OLT
members that the university has not publicly recognized the contribution and rankings growth of
the Online MBA. One OLT member shared, “I still believe that the school doesn’t know the
Online MBA is available. We don’t see it, we don’t hear about it, nobody talks about it.” The
same member went on to share, “Even within the [business school], we’re still [seen as being] in
the testing phase . . . it doesn’t inspire me to keep going at . . . times.”
Cabinet Leader Two noted that the online MBA at the UFL was purposely designed,
developed, and launched by what some might call intrapreneurs at the university. The
organization was set up in a separate building away from the traditional graduate programs so
they could focus on getting a new, innovative business initiative off the ground. This structure
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

99
was highly rewarding for the participants in the early stages in that they were given a great deal
of autonomy and encouraged to take risks, try new ideas, and fail fast. However, three years into
the new business, it appears different kinds of rewards may be needed in order to continue
motivating the team. De Villiers-Scheepers (2011) noted that the types of rewards for
entrepreneurs differ from those in traditional department settings. Social incentives and rewards
such as recognition and increased job responsibilities can be far more impactful. Changing the
form of motivation and focusing on recognition may be an important growth opportunity for the
UFL business school leadership in the future.
Team has Limited Knowledge about Formal Organizational Goals, Yet is Highly Motivated
Motivational Influence #2 was related to goal orientation and understanding whether or
not the participants knew their performance goals. This motivational influence examined
whether there was a gap between what the participants knew about their performance goals and
their knowledge about how to achieve them. The researcher found that the Online Learning
Team members did not have clear knowledge about their organizational goals, yet they were
eager to learn more about them. This motivational influence was validated and will also be used
to answer research question 1.
During the study, the researcher asked the Online Learning Team participants to discuss
their goals to determine what they knew about the goals of their department. The participants
were also asked which goals that were tied to online learning were most important and how to
implement these methods to keep students engaged and teach transferable skills. Finally, the
researcher asked the participants to discuss which competencies were most important for
students to secure leadership roles in the global business environment. Clark and Estes (2008)
noted that responses to these types of questions help the researcher understand if the work goals
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

100
have been clearly communicated by the leaders to the team members, if the team members know
how to achieve their goals, and if they have any experience achieving the same types of goals in
the past.
The Online Learning Team members were eager to talk about the evolution of their
department and its objectives. Their knowledge about the performance goals and
implementation varied among the participants, as did their knowledge about important elements
of student engagement. Each of these areas is explored more fully in the following sections.
Team members did not have clear knowledge about organizational goals. The
Online MBA was developed in a relatively short period of time during the second half of 2014
and launched in 2015. The early participants, now Team Leaders, helped develop and design the
program that was eventually launched. Each of the three Team Leaders shared that while no
formal goals were written or communicated, it was common knowledge that the objectives were
to develop and launch a quality program by January 2015. Team Leader Two noted that
“Initially the mandate was . . . [to] develop the best classes you can . . . just develop the classes.
Design, develop, [and] deliver. [The goals] are not written.” While the team leaders understood
the value of communicating goals, they shared that the environment had been, “so dynamic,
there’s no way to put it together. Process, procedures, manual . . . it’s been ever-changing.”
Each of the Team Leaders made similar comments about the goals being somewhat fluid through
the start-up phase and during the first few years of the program. The Cabinet Leaders echoed
this overall perspective as well.
The one goal that was mentioned by every one of the participants in the study, however,
was the goal to build and improve quality. The Cabinet Leaders discussed the importance of
quality being even more important than program growth. Cabinet Leader Three mentioned
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

101
quality a total of 16 times during the interview, and made the following comment about
consistency:
Quality has multiple aspects to it. The quality of the technology, the quality of the
teaching materials, the quality of the pedagogical tools. It has to be consistent, across
faculty, across courses, across various environments. And I would rather trade-off
revenue in favor of quality.
While every participant in the study agreed that quality was an important goal, the results from
these discussions indicate that the team members did not clearly understand the rest of the goals
for the department. In addition, none of the goals mentioned were specifically focused on
identifying, measuring, and improving online student learning and engagement. In order to grow
the department and the overall graduate-level online business, there is an opportunity to close
this knowledge gap by engaging the team members in a discussion about short- and long-term
goals, debating and then agreeing on them, and clearly communicating a cohesive plan for
achieving the goals to the team.
Team members were highly engaged, eager to set goals, and motivated to grow the
business. As discussed in Chapter 2, setting goals in business is an important part of the
organizational culture. In higher education, setting goals can increase motivation, improve
achievement, and enhance learning (Turkay, 2014). Peter Drucker’s model of SMART goals
describes the most effective elements in goals for teams. Drucker (1954) defines SMART goals
as Specific, Measurable, Action-oriented, Realistic, and Timely. For this study, the researcher
asked each of the participants about their goals and how motivated they were to achieve them.
The researcher found the team to be poised and willing to set new, specific smart goals that focus
on learning, mastering, and self-improvement to design online learning for MBA students that
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

102
engage and motivate students. However, the team was unsure about what these goals are or
should be.
Among the seven team members who participated in the study, each of them listed one to
six goals which they thought were important for the department. While none of the goals listed
by the participants specifically articulated the goal of improving student engagement and
learning, several of the goals listed do enhance the experience for the students. The participants
listed goals such as adding content for a better user experience, developing new tools, and
improving quality. They also listed goals such as expanding the number of online programs,
improving online MBA rankings, making money, and creating policies for improved department
efficiencies.
Team Leader Four articulated the department goals, several of which ultimately support
improved student engagement and learning. They included improving staff development,
enhancing the Quality Matters certification process, building community between the faculty and
instructional designers, delivering value through “leading-edge” knowledge, and remaining in
the top 5 on a regular basis in the U.S. News & World Report rankings. While the goals listed by
the Team Leader Four appear to be on target, the goals do not appear to have all of the elements
of SMART goals (i.e., specific, measurable, attainable, relevant, and timely). While all of the
participants were engaged and eager to build the program, the need to formalize and
communicate one set of focused goals will be an area of focus in the future.
Research Question 2
The second research question in the study asked if the Online Learning Team has both
knowledge and motivation to understand the most important leadership competency
requirements for graduate students. Knowledge Influence #2 is related to critical competencies
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

103
and was not validated. The researcher found that the Online Learning Team and the UFL
business school leaders had strong knowledge about the competencies that online MBA
graduates needed to learn to help secure leadership roles in a global business environment.
Knowledge and Motivation about Critical Competencies
The researcher found the Online Learning Team to be both knowledgeable and articulate
about the competencies required for students who graduate from the program and search for roles
in the global business environment. In addition, the Online Learning Team was highly motivated
and confident that they had the knowledge, the tools, and the support to effectively develop high-
quality online learning and instruction that engages online MBA students. All 15 study
participants discussed the importance of soft skills needed for graduate students to be successful
in future roles as global leaders. As previously mentioned, Cornuel (2007) noted that students
will not be able to compete solely with theoretical knowledge; they must develop other skills that
will help them in a global market. The participants in this study agreed with the research, and
mentioned skills or competencies that were tied, in some way, to three overall themes: (1) data
analytics, (2) critical thinking, and (3) interpersonal skills. Each of these is explored more fully
in the following sections.
Data analytics. The first competency was data analytics. This competency included the
ability to read and synthesize great amounts of data, and report out key findings in a way that the
targeted audience can comprehend. This competency was mentioned by eight of the participants.
Team Leader Two specifically noted this important skill and shared that the team had
incorporated it into the curriculum: “In fact, the initial curriculum had analytics and
communication components specifically in every single class. That wasn’t practical in the [early
course] delivery so we changed it up a little bit.” Cabinet Leader Three added, “you’ve got to
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

104
understand the world of numbers, whether it is financial data, accounting data, core basic
statistics, analytics.” In these excerpts, the participants expanded on the concept of numerical
literacy and the importance of developing an ability to comb through, combine, and synthesize
large amounts of data to develop a story. “You may not know how to manipulate it, you may not
be the best statistician on the planet, but you should be able to look at the data and ask intelligent
questions,” shared Cabinet Leader Three. The participants in this study believed that future
leaders will need this competency more than ever before, particularly in the global business
world where “big data” is both more prevalent and more accessible through ever more
sophisticated technologies.
Critical thinking. The second competency that was mentioned as important for future
leaders was critical thinking. This competency was described in different ways by the
participants as the ability to evaluate information, consider all of the options through reflection,
and make decisions in order to take action. Critical thinking and reflection were also mentioned
earlier under research question 1 as part of the early research by Garrison et al. (2010) on
cognitive presence and the Community of Inquiry framework. To help graduate students
develop this competency, the Online Learning Team and the online faculty discussed the need to
develop a safe and trust-based learning environment. Meyers (2008) posited that online learning
teams need to ensure the faculty and the curriculum are inspiring students to use their critical
thinking skills to explore their own backgrounds, beliefs and unconscious biases, challenging
students to consider and solve real world problems they might face when they leave graduate
school, and working with students to develop creative and focused solutions to future problems.
During the interviews, Cabinet Leader Three mentioned how important it was to have an “ability
to ask the right questions, ability to frame alternatives, ability to get a lot of data and information
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

105
. . . and create a set of options which have multiple implications.” These abilities are
fundamentally driven by an overall skill to think critically about every situation, according to
Cabinet Leader Three.
In each of the interviews, the participants spent the most time discussing the competency
of critical thinking. The participants described a new way of teaching where the professors
become more facilitators than lecturers in the virtual classroom. They also discussed the
importance of using this valuable time with the students to help them expand and practice their
critical thinking skills, which included teaching them how to ask the right questions and think
more like strategists or entrepreneurs.
Interpersonal skills. This third competency discussed by the participants included
several interpersonal skills such as communication, listening, and collaboration among others.
Researchers agree that future leaders will be required to develop soft skills (Anthony & Garner,
2016; Bedwell et al., 2014). The participants discussed the importance of relationship building
including such skills as building trust and Cabinet Leader Three added:
Team skills, negotiation skills, empathy, listening skills, communication, the whole . . . .
You know, because you could have the best strategy, you could understand the data, and
you can be terrible at implementation because you didn’t get buy-in. You didn’t listen to
people’s diverse viewpoints. So, interpersonal skills, and ability to work with diverse
backgrounds [and] . . . diverse experiences. [It’s important] to be able to harness that.
One of the most profound statements in this discussion was the use of the words “the whole,”
which spoke to the importance of developing all of these skills as part of an overall leadership
competency in dealing with people. Cabinet Leader Three goes on to define the skill of listening
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

106
in great depth which comes up in all of the other interviews as well, closing with “Leadership is
mainly about listening.”
In each of these discussions, the participants were able to identify the most important
critical competencies for students who graduate and wish to pursue future leadership roles. Each
of the groups including the cabinet leaders, the team leaders, and the team members had some
level of knowledge about the critical competencies that graduate students need to be successful
in the global business environment. These participants were also eager to discuss the
competencies and demonstrated a strong motivation to continue finding ways to help their
students develop these skills. The findings demonstrated that the team was both knowledgeable
and motivated to ensure the highest quality learning outcomes were delivered. As a result, there
did not appear to be a knowledge or motivation gap in this area.
Research Question 3
Research question 3 refers to the organizational influences for this study. These
influences are reflected in Table 6.

Table 6
Organizational Influences
Organizational Influences (Cultural Model) Organizational Influences (Cultural Setting)
1. The Online Learning Team needs to work with
the UFL business school leaders and faculty to
create a culture that understands and values
different types of learning models based on the
needs of different students and industry business
leaders.
3. The Online Learning Team needs to work
with faculty to ensure the curriculum for each
class includes creative opportunities for students
to reflect, discuss, and debate concepts from
their readings and pre-work rather than listen to
lectures.
2. The Online Learning Team needs to build a
culture of trust among other business school
leaders and faculty to overcome the perceived
risks of online learning methods.
4. The Online Learning Team at the graduate
level needs to understand how their organization
and goals disrupt or fit in with the overall
strategy for the business school.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

107
The third research question in the study asked about the interaction between the
organizational culture and the UFL Online Learning Team’s knowledge and motivation. Rueda
(2011) suggested that while there is little agreement on the definition of culture, it is an
important factor in truly understanding organizations, and the fact that it is often transparent,
automated, and ever-changing makes it important to recognize and address. Being a relatively
new organization in the university and in the business school, the Online Learning Team faced
several important challenges as they launched a new program in 2015. Since the program
launch, the team has developed its own culture and has been continuously seeking ways to
balance the need to be a part of the business school while, at the same time, remaining agile and
entrepreneurial. Three of the four of the organizational influences were validated and one of the
four organizational influences was partially validated during the interviews in this study in order
to answer research question 3. Each of the influences will be clearly outlined in the following
sections. Through analysis of the data, four important findings emerged:
 The business school leaders were willing to work together with the Online Learning
Team to align values and develop new models of learning to meet the needs of future
adult learners.
 The Online Learning Team was just beginning to build trust among faculty and
business school leaders in order to move into the next phase of growth.
 While the overall program curriculum was strong and had won several awards, some
of the faculty still have some opportunities for professional growth and development.
 The Dean and the Vice Dean clearly understood how the Online Learning Department
fit into the overall business school strategy and the rest of the Online Learning Team
did not understand the fit but was eager to learn.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

108
Willingness to Align Values and Develop New Models of Learning
The first organizational influence that was explored during the study in order to answer
the third research question was tied to aligned values and new learning models. Organizational
Influence #1 identified the need for the Online Learning Team to work with the UFL business
school leaders and faculty to create a culture that understands and values different types of
learning models based on the needs of different students and industry business leaders. This
organizational influence was validated during the study. There still exists a gap between the
Online Learning Team and the business school leaders and faculty about the value of an online
model at the graduate level.
The Online Learning Team participants shared some information and background about
the early development of the new Online Learning Team organization. The OLT was tasked
with creating an innovative model in an area of education that was relatively new. The Dean
communicated that this model would help create a competitive advantage for the UFL business
school. The Online Learning Team communicated that the support from the Dean was strong
from the very beginning. A separate organization with its own dedicated budget was created by
the Dean and marketed separately from traditional graduate programs. A senior leader was
identified to build the program, a new team was put in place, and the team was challenged to
develop and launch a new, online MBA program in less than six months.
Through this process, the Online Learning Team faced several challenges. There was
resource competition since the new organization would be using the same graduate school
faculty. There was also a potential for cannibalization and encroachment on traditional
programs. There were several challenges with a new pedagogy using a flipped classroom model,
and there were many unknowns, fears, and misperceptions about the use of a new mode of
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

109
learning with virtual classrooms. In addition, the Online Learning Team members were
considered outsiders and not part of the mainstream MBA organization. While much of the early
concern has abated, the participants discussed that the UFL business school and the overall
university are just now beginning to recognize the Online MBA as a credible program. One of
the study participants shared that, “Not all of them are totally sold [on online learning], but
they’re certainly sold on the money and on our [growth in] ranking.” The leaders in the business
school are beginning to see the positive results as the Online Learning Team grows in the
rankings and is recognized with awards among respected academic institutions. However, the
perception among the Online Learning Team was that the leaders are still at the beginning of this
cultural transformation.
In working to create a culture that understands and values different types of learning
models based on the needs of different students and industry business leaders, the Online
Learning Team was challenged to balance traditional learning methods with new, innovative
learning models. Traditional teaching methods with lectures, mid-terms, and finals have been
prevalent in the university settings for hundreds of years. Students have been expected to learn
what the professor is teaching, no matter what the student needs or learning styles.
Three of the participants in this study referred to this model as the “sage on the stage”
and spoke of how the model is different in online learning. The professor in the online learning
classroom becomes more of a facilitator. Team Leader Two shared that traditional classroom
interactions revolve primarily about lectures versus the online, interactive classroom where the
professor is helping to facilitate discussion and debate. Team Member Two described this as
moving from “teacher-centered to student-centered . . . where the students are contributing as
much if not more to the totality of the knowledge of the class and the professor.” Two of the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

110
participants shared that this new classroom environment might be a bit more uncomfortable to
some of the professors “because they like to be the expert(s).” However, after the faculty get
used to the new format, they “seem to prefer it.”
In addition, three of the participants discussed how the online environment helped
address a more targeted learning environment for the students. Cabinet Leader Three shared:
You think about learning in the classroom, learning outside the classroom. People learn
at different rates. The ability to be able to go back and revisit materials, to re-engage
with the instructor in your own time, in your own space . . . [is valuable].
In each of these perspectives, the researcher identified both a recognition of different values and
a respect for the potential that values are beginning to shift. The participants discussed a number
of transitions that are beginning to occur during what was described by one participant as a
“cultural transformation.” The business school leaders and faculty were still working through
how to treasure the old model while being inspired by the new model, but many still were not
convinced that the new model would be effective. These leaders and faculty continued to respect
the proven processes and were very cautious about challenging accepted practices. The OLT
was working to help the leaders and faculty continue to use physical, face-to-face classrooms
while they tested a new form of face-to-face, virtual classrooms. Some of the leaders and faculty
recognized there may be value in moving from an environment that is one-for-all learners to one
that focuses on targeting adult learners based on their own needs, but many were still
unconvinced. Effectively, the OLT participants in this study were convinced that a shift from a
solely teacher-centered, “sage on the stage” methodology to a student-centered, “guide on the
side” pedagogy would be a significant part of adult education in the future. However, they also
recognized that there was still work to do to legitimize this new model and build their brand with
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

111
others in the business school. The OLT shared that this transition would require both proof of
concept and trust in order for new values to be incorporated into the future model at the UFL
business school.
Building a Culture of Trust
Organizational Influence #2 was also studied to help answer the third research question.
In this organizational influence, the Online Learning Team needed to build a culture of trust
among other business school leaders and faculty to overcome the perceived risks of online
learning methods. This influence was also validated during the study.
The topic of trust was discussed by seven OLT Team Members and also the three Team
Leaders. They each addressed the culture of trust a bit differently, but the participants all
understood the importance of building a culture of trust among other business school leaders and
faculty to overcome the perceived risks of online learning methods. This development of trust
started with the Online Learning Team themselves in how they focused on creating a bridge to
other business school leaders and the faculty members. Team Member Three said it best: “We
had to prove ourselves . . . build trust and credibility.” The Online Learning Team understood
that this trust could not be directed by the business school leaders and that they would have to
take responsibility for building trust in order to gain credibility. Team Leader Three discussed
the challenge that each department has their own desires to be successful and “anything that is an
obstacle to that, they’re going to push back against.” The other departments had their own
objectives and sometimes these goals conflicted with the goals of the Online Learning Team.
The participants agreed that building trust with these other departments was going to take time,
patience, and tenacity as the new model began to develop and gain success.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

112
In addition, the participants communicated that the faculty were very hesitant at first.
They did not want to have to deal with the changing model or have to learn new pedagogies.
Having the ability to build trust was a critical component of recruiting new faculty members to
teach in this new environment. Team Member Two described the faculty challenges:
They’re uncomfortable with the online environment, they’re uncomfortable with doing
videos, and they’re uncomfortable because their expectations are that they’re going to
examine their teaching practices and make changes. That’s a triple whammy, and that is
very, very hard on the professors. Huge, huge amounts of pushback.
The OLT members reflected a level of pride in how they worked with each of the professors,
individually, to help them overcome their concerns and develop their curriculum. The OLT
participants discussed the final results and the fact that the Online Learning Team would go to
great lengths to help. One participant said, “Once they see how much work [we] will do for
them, once they accept our expertise, and see the video product, [it] is non-trivial for them. It
looks good.”
Team Leader Three added an additional perspective on how the faculty engaged with the
Online Learning Team by describing the experiences they encounter in the studio where they did
the video recording of their online lectures. Faculty were generally uncomfortable during their
first few sessions and the OLT members were sensitive to this situation and the need to build
trust. Team Leader Three shared, “The idea is to get the instructor to feel as comfortable as
possible, and not think about who’s looking at the video so much.” According to each of the
participants, it took time to help the faculty members get over their fears of this new teaching
model, and the relationships they built were critical for building trust. “Our faculty and our
instructional designers have created real relationships,” shared Cabinet Leader Two.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

113
The elements described here include important relationship building by the Online
Learning Team with the faculty in the UFL business school. The participants described their
award-winning infrastructure with the Online Media and Technology Team to help make the
faculty feel comfortable in this new environment, an attitude of servant leadership among the
Online Student Services Team to help the faculty focus on the student needs rather than their
own concerns, and a level of professionalism among the highly respected Instructional Design
Team to help the faculty overcome their fears of the unknown and learn new, more effective
pedagogies for online teaching. Cabinet Member Two described how these groups of
professionals helped the faculty begin to trust and ultimately get focused again on effective
student engagement and learning. Most important, the most “effective faculty trust themselves
. . . even online,” concluded Team Member Five.
Need for Stronger Faculty Collaboration and Development
One other finding emerged from this study that helped validate the Organizational
Influence #3. While there was recognition among the participants in this study that the
curriculum was strong, and trust between the Online Learning Team and the faculty was
growing, there was consensus across the Online Learning Team that the team needed to continue
developing stronger curriculum and even more collaboration with the faculty. In addition, the
area of growth discussed most often among the Online Learning Team was the need for more
faculty development. They described the importance of further refining skills that helped them
effectively engage students in an online environment, even though “faculty complete online
teaching and learning training for four-six months prior to the delivery of an online course and
are supported by the online team during the course delivery” according to one of the internal
department documents.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

114
Meyer (2014) posited the importance of ensuring faculty understand “where students are
and what they need to learn, what technologies or software programs are available and what their
limits are, what pedagogies work and when, and what the course is supposed to accomplish in
terms of student learning” (p. 56). The planning and decision-making in an online environment
are more complex and requires faculty to be even more efficient in their planning than when they
design more traditional on-campus courses (Meyer, 2014). This latter opportunity for growth
was echoed most strongly by the Online Learning Team leaders and one of the Team Members.
The Online Learning Team shared that there are several highly successful faculty
members; however, the overall team of faculty are at different levels of knowledge and skills.
Getting all of the faculty to invest time and effort in learning new skills for innovative and
effective student engagement was described as “very difficult.” The Team Leaders discussed
that faculty are often unwilling to engage in any professional development in this relatively new
learning environment. While the Online Learning Team knew the key elements for strong
student engagement, they described the challenge of building a larger pool of motivated online
faculty who know how to specifically implement pedagogy that promotes strong student
engagement. The OLT recognized the need to continue working with faculty to ensure the
curriculum for each class includes creative opportunities for students to reflect, discuss, and
debate concepts from their readings and pre-work rather than listen to lectures.
Some other findings in this area were more positive. In the early years of the Online
MBA, the organization spent a great deal of time and effort helping faculty develop new skills
including meeting with them regularly, creating a faculty advisory committee, changing their
goals, and helping them learn through demonstrations of online classes for the entire faculty
council. Cabinet Leader Two shared that the professors who teach in the Online MBA program
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

115
were getting more comfortable with the pedagogy developed for their online courses and using
the “materials they created for the online classes in their residential classes.” In addition, the
professors who originally started with the Online Learning Team have agreed to renew their
contracts. Cabinet Leader Two shared:
Without exception so far, every single one of [the faculty members] has asked to teach
for another three years. They were people who were cautious about signing up for the
first time. Skeptical in some cases, afraid in some cases. But we encouraged them and
they did it and they’re loving it so much. I think that’s another way of assessing the
program.
These findings indicate that once the faculty overcome some of their concerns and fears, they
might be more willing to learn new skills. However, the findings indicate there is still an
important need for continued professional development among the faculty.
Fit or Disruption in the Business School Strategy
The final organizational influence, Organizational Influence #4, was partially validated
during this study and it also helps address the third research question. Organizational Influence
#4 outlined that the Online Learning Team at the graduate level needed to understand how their
organization and goals disrupted of fit in with the overall strategy for the business school. The
researcher found that the Online Learning Team members were not familiar with the overall
strategy of the UFL and did not have perspectives about how their organization fit into the
organizational strategy. However, the Dean and the Vice Dean had clear perspectives about the
strong fit of the Online Learning initiatives and also the importance of online learning in the
future growth of the UFL. The following section introduces several of the documents shared by
the OLT Leaders which help validate Organizational Influence #4.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

116
Document Analysis and Findings
The approach of reviewing and analyzing data from published documents was planned
and implemented for this study for the purpose of triangulating the data collected from the
interviews. While the data from the documents helped accomplish this goal of triangulation, the
analysis of the documents also helped the researcher validate the fourth organizational influence
for the study. In addition, the document analysis process helped expose an important
motivational gap for the Online Learning Team. Merriam and Tisdell (2016) posited that
documentary materials such as letters, memos, and published articles include some of the most
dependable data, partially because of their stability, and also because they are most often
published for reasons other than for research. The documents included in this study appear to fit
this description in that they have been published for the purposes of communicating a vision and
ensuring the rest of the organization understood the organizational mission and plan for
developing an innovative new learning model.
The seven documents reviewed and analyzed for this study include several internal
documents about the status and future of the Online Learning organization, as well as several
published articles and speeches by the most senior university leaders. They are listed as follows:
 UFL Online MBA Cohort Profiles (2015-2018)
 Annual Performance Report for Online Learning (2017)
 Business School Dean’s Memo and Update on the New Online MBA (2015)
 Annual Provost Report and Review of Online Learning Accomplishments (2014)
 University President Published Speech to the World Affairs Council (2014)
 University President’s Page Article — Online Education and Academic Mission
(2013)
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

117
 University Provost Memo to Faculty and Staff — Digital Initiatives (2013)
Each of these documents was published and available to the faculty and staff at the university.
As a result, they are considered reliable, authentic, and relevant, which are all important factors
outlined by Merriam and Tisdell (2016) for use of documents in data collection methods.
Three important themes were developed through the review and analysis of these
documents: (1) quality first and at all levels, (2) innovation and leadership in student
engagement, and (3) strategic fit and growth mentality for online learning.
Quality First and At All Levels
First, both the university and the business school talk extensively about the importance of
maintaining quality at every level and through the execution of every program, no matter what
the cost. This parallels the strategic focus discussed earlier in this chapter, where every
participant mentioned quality in some form. The Cabinet Leaders even stated that quality is a
value that is more important than program or revenue growth.
In describing the Online MBA at the UFL, one of the documents discussed the
accomplishments of the organization and how they continue to provide quality learning in each
of the courses. They are taught by residential professors, conceptualized and produced by
internal experts with “degrees and experience in instructional design, quality matters, cinema
production, and pedagogy.” In addition, the Dean described the other benefits that result from
having an online MBA program which includes developing knowledge and expertise about the
online educational products that will also support the residential programs, and help them
incorporate the latest technologies. The Dean closes this discussion with yet another comment
about quality, “And by launching a high-quality new program in a visible part of the higher
education space, we add luster to the [business school’s] brand overall.”
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

118
Innovation and Leadership in Student Engagement
The second theme that is consistent across all of the documents is the focus on innovation
and leadership in developing methodologies and pedagogy that focus on improving student
engagement. This theme is directly aligned with the findings from the interviews where the
Online Learning Team leaders and department members had clear knowledge about the specific
tools that most effectively engage students in an online learning environment. The Vice Dean
and the Dean discussed the innovative strategy that was developed for the launch of the online
MBA in several documents. They describe the following key elements in the online MBA
program: (1) one-week residential kickoff where students and faculty meet for the first time,
(2) online classes over five semesters, (3) in-house production at a professional studio owned and
managed by the Online Learning Team, (4) professional, experienced “instructional designers
and production staff with expertise running online pedagogy to film production to work with
faculty in producing the classes,” (5) access for the students to “prepared video material, which
can include lectures by the professor, interactive exercises, cases, interviews with other experts,
and guest speakers,” and (6) weekly, online virtual classes with the faculty and other student
peers “using [an] on online platform that allows students and faculty to see and interact with each
other in real time.” The Vice Dean writes that “this instructional model reflects best practices
and allows students the flexibility to work through most of the material according to the demands
of their schedules, but also provides them with direct small-group interactions with faculty
instructors each week.” At the time of the program launch, no other universities were executing
their online MBAs in this format. Student engagement was one of the specific goals of the
organization in developing this model.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

119
Strategic Fit and Growth Mentality
The third theme that was prevalent across all of the documents was the importance of
online learning technologies and online program growth for both the University of Future
Leaders and the business school. In addition, the fourth influence for this study was determining
how the Online Learning Organization disrupts or fits in with the overall strategy for the
business school. The influence was validated through the document analysis process and helped
answer research question 3. There is strong evidence that online learning is an important
element in the vision and mission of both the university and the business school; however, the
Online Learning Organization is concerned about the lack of recent recognition from the
university and business school leaders. This is an organizational gap that, with the right
solutions, can be resolved at a relatively low cost to the organization. Clark and Estes (2008)
once said that “upper management vision and commitment, clearly and candidly communicated
to everyone and demonstrated with visible management involvement in the process, is a critical
success factor” (p. 118).
As early as 2013, both the UFL President and Provost wrote extensively about the
importance of new technologies for academia, particularly at the graduate level. The President
talked about how important it was for the UFL to continue developing “the newest digital media
technologies” and shared that, “our goal is to provide an online education that is just as powerful
as the classroom one.” The President continued in 2013 by saying, “without question, [UFL’s]
faculty has developed a global online graduate education enterprise that blazes a new path and
draws on timeless values and timely innovations.” The Provost described the future of education
in 2013 saying, “our teaching and scholarship are enhanced by new avenues of evocative
multimedia communication.” On the topic of strategic growth, the President closed an article
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

120
with, “[online graduate education at the UFL] expands educational access broadly, while
maintaining our all-important standards of academic rigor, integrity, and quality.” The Dean at
the business school both supported and executed against this vision and by 2017 published the
following in a memo to the faculty and staff:
With the rapid pace of technological change, no one can see all of the twists and turns
that may be ahead for higher education. The one thing that is clear is that online
technologies will become an increasingly important part of the toolkit that is used for
teaching our students. Our strategy is to move aggressively into this space with careful
calculation and at the same time a spirit of experimentation. Our initial steps in this
direction have us well positioned to learn how to use these technologies and maintain our
reputation as a leader in higher education.
In every case, the documentation clearly communicated a powerful and focused vision for the
future of online learning at the graduate level. The most senior leaders are aligned on their
vision and their mission to grow this critical strategic initiative until it is consistently woven into
the fabric of the strategy and becomes an important pillar for education.
Synthesis
The following discussion synthesizes the results in the data analysis and findings from
the qualitative interviews and document analysis for this study. It also demonstrates how these
results helped identify the knowledge, motivational, and organizational influences that impacted
the key stakeholders as they worked to improve student learning and engagement, develop
critical competencies in graduate students, and achieve their organization goal.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

121
Knowledge Influences
Two knowledge influences were explored for this study. In the first knowledge
influence, the researcher asked questions during the interviews to determine if the Online
Learning Team knew how to implement the most effective methods for keeping students
engaged and teaching transferable skills in an online environment. The findings indicated that
each of the members of the team had knowledge about their own roles and responsibilities for
enhancing student engagement and teaching transferable skills. In addition, most of the
participants also had knowledge of several of the other roles in the department. The only gap
that was uncovered was how they will measure student engagement and the actual transfer of
skills from the graduate skills to the business world where the students will take global
leadership roles. There are currently no formal tracking procedures and only a few measures for
reporting in these areas.
The second knowledge influence that was explored was whether the Online Learning
Team knew the most important competencies for graduates and how the competencies that
online MBA graduates learned will help secure leadership roles in a global business
environment. The findings demonstrated that each of the Team Leaders, the Team Members,
and the Cabinet Leaders had some level of knowledge about the critical competencies required
for a future role in the global market. The participants were able to identify the most important
critical competencies for students who graduate and wish to pursue future leadership roles and
were highly motivated to continue helping their students develop these skills. As a result, there
did not appear to be a knowledge or motivation gap in this area.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

122
Motivational Influences
In the area of motivational influences, two influences were examined. For the first
motivation influence, the researcher asked questions to determine whether the Online Learning
Team had strong self-efficacy, and believed they were capable of developing online learning and
instruction that effectively engaged online MBA students and ensured they have learned the
critical competencies for a rapidly changing business market. The results indicated that self-
efficacy was strong across all of the team members. The participants were both articulate and
informed about the importance of the faculty, the content, and the student-peers in implementing
a highly engaging program. There was, however, one gap that was uncovered during the
discussions. Several members of the team did not believe their department was being recognized
appropriately for some of their more important accomplishments by the more senior leaders in
both the business school and the university. The only exception was the recognition from the
Vice Dean of Online Learning who was called out by each of the Team Leaders and the Team
Members as a truly great sponsor and the organization’s greatest cheerleader. The gap in this
area was with the more senior-level leaders including the Dean, whom the team respected a great
deal, and the University President, who was perceived to have been somewhat silent about online
learning in recent years.
The second motivational influence focused on goal orientation and the researcher
examined whether the Online Learning Team had set SMART goals that focused on learning,
mastery, and self-improvement to design online learning for MBA students that engage and
motivate students. The findings indicated that during the early years of launching the program,
the Team Leaders and Team Members focused primarily on the implementation of the new
online MBA program and the goals changed often, sometimes even daily. However, as the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

123
organization enters its third year, the findings indicate that there is a gap and a need for more
formal SMART goals that are clearly articulated and communicated to all members of the team.
Organizational Influences
Finally, there were four organizational influences that were explored during the study.
The first two influences were described in the conceptual framework as part of the cultural
model. For the first organizational influence, the researcher evaluated how the Online Learning
Team worked with the UFL business school leaders and faculty to create a culture that
understands and values different types of learning models based on the needs of different
students and industry business leaders. The OLT was found to be highly motivated to work with
anyone outside of their department and especially with their leaders and faculty to develop a
highly-targeted and effective learning model. The second influence focused on understanding if
the Online Learning Team was able to build a culture of trust among other business school
leaders and faculty in an effort to overcome the perceived risks of online learning methods. Both
of these influences were important and the Online Learning Team had worked very closely with
the faculty and the school leadership to achieve a collaborative working relationship. No gap
was found in either of these areas.
The other two influences were described in the conceptual framework as part of the
cultural setting. The first focused on how well the Online Learning Team worked with faculty to
ensure the curriculum for each class included creative opportunities for students to reflect,
discuss, and debate concepts from their readings and pre-work rather than listen to lectures. The
findings indicated that while the team has made some progress in this area, there was still a gap
in both the level of development needed and continued collaboration with the faculty to achieve
the performance goals. The second influence focused on whether the Online Learning Team
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

124
understood how their organization and their goals either disrupted or fit in with the overall
strategy for the business school. Based on the findings from the document review and analysis,
the Online Learning Organization clearly has a strategic fit in the business school and the overall
university. However, there is a need for more communication to the organization about the
vision, the mission, and how they fit into the strategic goals for the business school and the
university. The gap in this area is also tied to the gap in understanding their overall goals as an
organization.
As the Online Learning Team looks to the future, the researcher discovered that the Team
Leaders in the organization recognized the importance of nurturing the evolving culture while
also taking advantage of the opportunities and addressing several issues associated future
growth. The Team Leaders were aware of the need to develop engaging learning models, build a
team of faculty who are experienced in online teaching and willing to grow and develop, and
implement solutions to create a more engaging online MBA program that aggressively competes
in a high-growth market.
As discussed earlier in this study, the online MBA at the University of Future Leaders is
a relatively new program, launched in 2015, and much of the planning by the Online Learning
Team and many of the resources were appropriately focused on launch and implementation.
Since 2015, data from the document review show that the program has been growing each year.
When the program started in Fall of 2015, it admitted 16 students with an average age of 31 and
average experience of six years. By 2018, the program accepted between 40-50 students per
cohort and the number of years of experience almost doubled to 11 years. This growth begins to
demonstrate the importance of online learning for adult learners. Table 7 reflects the growth in
both the quantity and quality of students over the past few years.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

125
Table 7
UFL Online MBA Student Profiles by Cohort

FA 2015 SP 2016 FA 2016 SP 2017 FA 2017 SP 2018 FA 2018
Enrolled Students 16 20 28 26 48 39 49
Average GPA 3.1 3.2 3.2 3.1 3.1 3.1 3.1
Average GMAT 546 630 613 630 657 625 624
Average GRE 310 313 309 302 304 310 305
Average Age 31 31 35 34 34 36 34
Work Exp (years) 6 8 9 10 11 12 11
Corp Sponsored 43% 40% 39% 42% 27% 30% 38%
Minority Students 6% 10% 4% 25% 22% 30% 24%
Women 25% 50% 32% 29% 38% 23% 36%

The future for the Online Learning Organization at the University of Future Leaders
looks positive and the overall sentiment of the participants in this study is that they have built
and launched a solid foundation for the growth of online learning in the business school. They
are poised and ready for raising the bar. Several recommended solutions for continued growth
will be outlined in Chapter 5.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

126
CHAPTER 5
RECOMMENDATIONS
Chapter 4 was dedicated to completing the data analysis and reporting the findings that
resulted from qualitative interviews with the Online Learning Team at the University of Future
Leaders. Several members of the Dean’s Cabinet at the business school were interviewed as
well. The findings were presented in the form of themes that developed from the data and helped
answer three of the four research questions. The results also helped validate the knowledge,
motivational, and organizational influences that impacted the key stakeholders as they work to
improve student learning and engagement, develop critical competencies in graduate students,
and achieve their organizational goals.
Chapter 5 addresses the final research question about the recommendations for the UFL
Online Learning Team’s organizational practice. The final recommendation described in this
chapter is for the Online Learning Team to launch an offsite planning meeting which results in
the development of a strategic implementation plan. Once implemented, the plan will guide the
growth of the online programs at the UFL business school over the next three years. The
strategic implementation plan will clearly identify the overall organizational objective for the
year, list the quantitative annual goals, identify the key strategies or initiatives that include the
other recommendations from this study, and designate the measures that will be tracked to
determine progress against the goals.
The framework used to develop the recommendation for the strategic implementation
plan and the evaluation plan for this study is the New World Kirkpatrick Model (Kirkpatrick &
Kirkpatrick, 2016). This model is the updated and enhanced version of the last version of the
Four Level Model of Evaluation. The newly updated version still recognizes the four original
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

127
levels used to evaluate outcomes; however, the New World Kirkpatrick Model focuses on the
levels in reverse order. The model encourages the organization to start first by identifying the
desired results (level 4) and then addresses important behaviors (level 3), relevant learning (level
2) and finally, team member reactions (level 1) to the strategic implementation plan. Each of
these levels will be described in detail later in this chapter. The New World Kirkpatrick Model
will also be used to evaluate the outcomes from the final implemented plan.
Recommendations for Practice to Address KMO Influences
Knowledge Recommendations
Introduction. The researcher explored assumed knowledge influences for this study
using the Anderson et al. (2001) framework of the knowledge dimension. One of the knowledge
influences was validated through data collected during qualitative interviews with the UFL
Online Learning Team (OLT), the Dean’s Cabinet, and one online MBA program faculty
member. The knowledge influence is listed in Table 8.

Table 8
Summary of Knowledge Influences and Recommendations
Knowledge Influences Principle and Citation Context-Specific Recommendation
The Online Learning
Team (OLT) needs to
know how to implement
the most effective
methods for keeping
students engaged and
teaching transferable
skills in an online
environment. (P)
Enhancing procedural
knowledge through the use
of a job aid when
organizations are
responsible for performing
tasks in an area where they
already have declarative
knowledge and experience
(Clark & Estes, 2008).
Provide a strategic roadmap that
reminds the team about their
department goals, provides a
checklist for how to implement
best practices, and lays out how
they will consistently measure
outcomes related to student
learning and engagement methods
across all courses and faculty.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

128
Procedural knowledge solutions. Procedural knowledge is the “how to” area of
knowledge and includes the very specific steps needed to accomplish a skill or a goal (Mayer,
2011; Rueda, 2011). The Online Learning Team needed to know how to implement the most
effective methods for keeping students engaged and teaching transferable skills in an online
environment as well as measuring and tracking the outcomes. Clark and Estes (2008) discussed
the value of enhancing procedural knowledge through the use of a job aid when organizations are
responsible for performing tasks in an area where they already have declarative knowledge and
experience. The recommendation is to provide a strategic roadmap that reminds the team about
their department goals, provides a checklist for how to implement best practices, and lays out
how they will consistently measure outcomes related to student learning and engagement
methods across all courses and faculty.
McCrudden, Schraw, and Hartley (2006) discussed the importance of helping team
members model the use of plans to break down complex tasks and determine how and when to
use specific tools and methods. The development and use of strategic plans in this form had not
been a focus for the OLT during the start-up years. Other researchers noted the importance of
applying knowledge to develop mastery noting that once team members acquire new skills, they
must have the opportunity to practice using them and learning when and how to apply them for
the most effective outcomes (Ambrose, Bridges, Lovett, DiPietro, & Norman, 2010). The
evidence supports providing the OLT with a strategic map in order to remind them to focus on
goals, implement best practices, and develop key measures and tracking.
Motivation Recommendations
Introduction. The researcher explored two assumed motivation influences for this
study. Motivation has been described as one’s willingness to choose an activity (active choice),
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

129
how likely they are to stick with it through to completion (persistence), and how they continue to
develop and learn (mental effort) new information and knowledge (Clark & Estes, 2008; Rueda,
2011). Both motivation influences were validated through data collected during qualitative
interviews with the UFL Online Learning Team (OLT), the Dean’s Cabinet, and one online
MBA program faculty member. These influences were prioritized based on proposed short-term
versus long-term solutions. The motivation influences are listed in Table 9.

Table 9
Summary of Motivation Influences and Recommendations
Motivation Influences Principle and Citation Context-Specific Recommendation
Self-Efficacy: The Online
Learning Team needs to
believe they are capable of
effectively developing online
learning and instruction that
will engage online MBA
students and ensure they have
learned the critical
competencies for a rapidly
changing business market.
Individuals who have a high self-
efficacy and believe they are
capable of achieving significant
accomplishments are highly
motivated; they are more likely to
begin a new project, persist in
overcoming difficult hurdles and
apply greater effort in achieving
their goals (Rueda, 2011).
Interdependent teams are more
motivated when they developed
trust for the level of expertise and
willingness to collaborate from
other team members in addition to
knowing that less experienced
team members were doing their
best to put in the time and effort to
build their own expertise (Clark,
2005).
Provide opportunities for the OLT
to meet together regularly with a
department Director for goal-
directed practice in brainstorming
new ideas, collaborating to solve
problems, and publishing results.
Work together as a team to develop
a culture of requesting feedback
from the Dean, the faculty, and
other business school leaders as
well as department-level leaders
and peers about performance and
progress toward goals in order to
build trust, gain recognition, and
strengthen the team.
Goal Orientation: The Online
Learning Team needs to set
smart goals that focus on
learning, mastering, and self-
improvement in order to
design online learning for
MBA students that engage
and motivate students.
Focusing on mastery, individual
improvement, learning, and
progress promotes positive
motivation (Yough & Anderman,
2006).
Develop annual goals as a team
that cascade from the university
and organizational goals; ensure
the goals are specific, measurable,
attainable, relevant, and timely
(SMART). Discuss progress
quarterly.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

130
Self-efficacy. Self-efficacy was originally developed as part of Bandura’s social
cognitive theory which focused on self-perceptions and beliefs about one’s ability to perform
affected outcomes (Pajares, 1997). Individuals who have a high self-efficacy and believe they
are capable of achieving significant accomplishments are more highly motivated, they are more
likely to begin a new project, persist in overcoming difficult hurdles, and apply greater effort in
achieving their goals (Rueda, 2011). In addition, interdependent teams are more highly
motivated “when they trust both the expertise and collaborativeness of other team members as
well as the determination of weaker members on their team to invest maximum effort to build
their expertise” (Clark, 2005, p. 16). The Online Learning Team needs to believe they are
capable of effectively developing online learning and instruction that will engage online MBA
students and ensure they have learned the critical competencies for a rapidly changing business
market. The recommended solutions for helping the team build self-efficacy include providing
opportunities for the OLT team to meet together regularly with the department Director for goal-
directed practice in brainstorming new ideas, collaborating to solve problems, and publishing
results. In addition, the team should develop a culture of requesting feedback from the Dean, the
faculty, and other business school leaders as well as department-level leaders and peers about
performance and progress toward goals in order to build trust, gain recognition, and strengthen
the team.
Self-efficacy affects both personal beliefs and group perspectives, known as collective
efficacy (Bandura, 2000; Pajares, 1997). Groups or organizations that operate with collective
efficacy develop a shared sense of empowerment and belief about their impact on other
constituents (Pajares, 1997). These shared beliefs determine the organization’s outlook about its
collective abilities to achieve its goals, how much effort the members invest in supporting the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

131
team, how dedicated they are to supporting other team members when failures occur, and how
committed they are to achieving the final goals (Bandura, 2000). Theoretically, it appears that
meeting together regularly to help build both individual self-efficacy and collective efficacy will
help the OLT focus on the most important objectives and work together to ensure the entire team
reaches their goals.
Goal orientation. Goal orientation theory is split into two main types of goals: mastery
goals and performance goals. Research suggests that stronger, more long-term outcomes are the
result of individuals who develop and seek to achieve mastery goals, which include a focus on
self-improvement and a deep understanding of how the goals will serve a broader purpose
(Pintrich, 2003; Yough & Anderman, 2006). The Online Learning Team needs to set SMART
goals that focus on learning, mastering, and self-improvement to design online learning for MBA
students that engage and motivate students. Focusing on mastery, individual improvement,
learning, and progress promotes positive motivation (Yough & Anderman, 2006). The
recommendation is for the team to work together to develop annual SMART goals that cascade
from the university and organizational goals, and are discussed by the team at least quarterly.
Setting goals in business is a significant part of the organizational culture. In higher
education, this process for setting goals can increase motivation, improve achievement, and
enhance learning (Turkay, 2014). Latham and Locke’s (2006) research showed that setting goals
significantly improve success rates. Setting SMART goals will be important contributors to the
success of the future growth of online education at the UFL.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

132
Organization Recommendations
Introduction. The researcher explored four assumed organizational influences for this
study. The organizational influences or barriers are described as those which may impact
performance including the overall structure of the organization, which rules and guidelines are in
place, and the values that are espoused as compared to those that are actually in play (Clark &
Estes, 2008). Rueda (2011) referred to organizational influences as “culture, structure, and
policies and practices” (p. 53). All four organizational influences were validated through data
collected during qualitative interviews with the UFL Online Learning Team (OLT), the Dean’s
Cabinet, and one online MBA program faculty member. These influences were prioritized based
most urgent organizational needs. The organizational influences are listed in Table 10.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

133
Table 10
Summary of Organization Influences and Recommendations
Organizational Influences Principle and Citation
Context-Specific
Recommendation
The Online Learning Team
needs to work with the UFL
business school leaders and
faculty to create a culture that
understands and values
different types of learning
models based on the needs of
different students and industry
business leaders. (Cultural
Model)
Organizational effectiveness
increases when leaders identify,
articulate, focus the organization’s
effort on and reinforce the
organization’s vision; they lead from
the why.
Adults are more motivated to
participate (and learn) when they see
the relevance of information, a
request, or task (the “why”) to their
own circumstances. They are goal
oriented (Knowles, Holton, &
Swanson, 2012).
In order to continue building
the internal brand and further
enhance legitimacy, develop
organizational
communication and relevance
campaigns tied to different
types of learning models.
The Online Learning Team
(OLT) needs to build a culture
of trust among other business
school leaders and faculty to
overcome the perceived risks of
online learning methods.
(Cultural Model)
Organizational effectiveness
increases when leaders are
trustworthy and, in turn, trust their
team (Colquitt, Scott, & LePine,
2007).
Establish and agree on
methods to reduce perceived
risks and create transparency
around data related to OLT
online program performance.
The Online Learning Team
needs to work with faculty to
ensure the curriculum for each
class includes creative
opportunities for students to
reflect, discuss, and debate
concepts from their readings
and pre-work rather than listen
to lectures. (Cultural Setting)
Effective leaders are aware of
various types of engagement and
communication, non-verbal
communication, storytelling, person-
centered communication, and how
these communication modalities
influence change and the
environment within the organization
(Conger, 1991; Denning, 2005;
Lewis, 2011).
Ensure faculty are aware of
how various types of
engagement and
communication approaches
can influence students in an
online learning environment
and create opportunities for
faculty to participate in
further professional
development.
The Online Learning Team at
the graduate level needs to
understand how their
organization and goals disrupt
or fit in with the overall
strategy for the business school.
(Cultural Setting)
Effective leaders have the skills and
strategies needed to redesign their
organization for effective
communication (Denning, 2005;
Lewis, 2011).
Implement a communication
model that inspires and
informs the OLT and also
works well for the UFL
business leaders.
Implement a communication
model that works with UFL
business school leaders.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

134
Cultural models: New learning models. Cultural models are those accepted and widely
shared influences in an organization which impact both the perceptions and the behavior of the
organization’s team members (Holland & Quinn, 1987). Culture is dynamic, invisible, tied to
the values and beliefs of the stakeholders, and it often changes and develops over time (Clark &
Estes, 2008). The Online Learning Team needs to work with the UFL business school leaders
and faculty to create a culture that understands and values different types of learning models
based on the needs of different students and industry business leaders. Organizational
effectiveness increases when leaders identify, articulate, focus the organization’s effort on, and
reinforce the organization’s vision; they lead from the why (Knowles et al., 2012). Adults are
more motivated to participate (and learn) when they see the relevance of information, a request,
or task (the “why”) to their own circumstances; and, they are goal oriented (Knowles et al.,
2012). In order to continue building the internal brand and further enhance legitimacy, the
recommendation is for the OLT leaders to develop organizational communication and relevance
campaigns around different types of learning models.
While culture is relatively easy to observe in many organizations, it is difficult to
decipher (Schein, 2017), and culture is almost always different between groups in the same
organization (Erez & Gati, 2004). The Online Learning team at the UFL has developed an
internal culture that is perceived to be somewhat innovative, and even radical, in its approach to
change some of the paradigms tied to traditional residential, graduate-level learning models.
Schneider et al. (1996) discussed the need for teams to be radical in their actions to make the
necessary impactful change that will endure for years to come. The UFL Online Learning Team
understands the importance of this value as it works to help the organization grow both its
deliverables and its reputation in a relatively new area of graduate learning. As a result, the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

135
recommendation is to ensure all stakeholders in the organization are aligned on the vision,
priorities, and desired outcomes for developing online learning to help the organization build
brand equity and help further legitimize this relatively new learning model.
Cultural models: Building trust. To begin bridging the cultural gaps at the UFL and
creating an environment of understanding and support, there also needs to be a focus on building
trust across departments, with faculty, and with other key leaders at the business school as they
work to overcome the perceived risks of change with this new model. Agocs (1997) suggested
one of the first steps in creating this trust is to diagnose and address the overall institutional
resistance to organizational change. Organizational effectiveness increases when leaders are
trustworthy and, in turn, trust their teams (Colquitt et al., 2007). When working with the
business school leaders and faculty, the recommendation for the OLT is to establish and agree on
methods to reduce perceived risks and create transparency around data related to OLT online
program performance.
The leaders in learning organizations are responsible for building a shared vision among
the team, changing previous mental models that are tied to traditional ways of doing things, and
ensuring the team members are growing their own capabilities to prepare for future needs of their
customers (Senge, 1990). At the UFL, the leaders of the Online Learning Team are also
responsible for providing clarity and support to achieve goals while educating others in the
organization and helping to reduce or eliminate the perceived risks about the methods that
support learning in an online environment. As a result, the focus on reducing perceived risks and
creating transparency about outcomes and performance will enable the organization to continue
building trust across all stakeholders.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

136
Cultural settings: New learning techniques and faculty development. Clark and Estes
(2008) describe the cultural setting in the form of a cultural profile, which includes how the
organization is different from other organizations, which goals and aspects are most valued, what
changes are made, who drives the change, who makes the decisions, and what happens once the
change is agreed upon. The Online Learning Team needs to work with faculty to ensure the
curriculum for each class includes creative opportunities for students to reflect, discuss, and
debate concepts from their readings and pre-work rather than listen to lectures. Effective leaders
are aware of various types of engagement and communication, non-verbal communication,
storytelling, person-centered communication, and how these communication modalities influence
change and the environment within the organization (Conger, 1991; Denning, 2005; Lewis,
2011). The recommendation here is to provide professional development for faculty to ensure
they are aware of how various types of learning, engagement, and communication approaches
can influence students in an online learning environment.
In a learning organization where multiple groups are responsible for the deliverables to
affect change, the concept of action research and communities-of-practice play an important role
(Hendry, 1996). These communities-of-practice are developed among groups of people and
sometimes across organizations that are focused on achieving the same goals. Communities-of-
practice often have a significant impact on the overall development of the organizational culture
(Hendry, 1996). Theoretically, the more often the Online Learning Team can engage in
communities-of-practice and professional development with faculty in collaborative learning and
creative design thinking, the better their chances of developing engaging curriculum and
impacting the evolving culture in their online learning organization.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

137
Cultural settings: Strategic fit. The cultural setting or climate, particularly in an
organization facing change, is of critical importance and includes many conditions, factors, and
activities that are specific to each organization. The Online Learning Team at the graduate level
needs to understand how their organization and goals disrupt, or fit in, with the overall strategy
for the business school. Effective leaders have the skills and strategies needed to redesign their
organization for effective communication (Denning, 2005; Lewis, 2011). The recommendation
for helping the OLT understand how they fit into the overall strategy is to implement a
communication model that both inspires and informs the team members in the OLT and works
well for the UFL business leaders.
Several researchers have suggested that in order for a team to create a new model for
improvement, the team will need to understand what they are trying to accomplish, how they
know that the changes they make constitute an improvement, and what other changes can be
made to further improve the program (Langley et al., 2009). The UFL Online Learning Team
has been challenged to provide an innovative and entrepreneurial learning environment that
mirrors business practices while expanding thoughtful leadership, identifying key global
opportunities, and exploring critical issues (UFL Strategic Plan, 2015). The literature would
support the recommendation to implement a communication model that inspires and informs in
order to successfully meet this challenge.
Integrated Implementation and Evaluation Plan
Implementation and Evaluation Framework
The framework used to develop an implementation and evaluation plan for this study is
the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016). This model is the updated
and enhanced version of the original model introduced in the book Evaluating Training
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

138
Programs: The Four Levels, written by Dr. Don Kirkpatrick (Kirkpatrick & Kirkpatrick, 2016).
The newly updated version still recognizes the four original levels used to evaluate outcomes
which include Reaction (level 1), Learning (level 2), Behavior (level 3), and Results (level 4).
However, the New World Kirkpatrick Model focuses on the levels in reverse order, starting first
with the results. The result is defined as that which is most important in the organization
(Kirkpatrick & Kirkpatrick, 2016). One important element of this model is that an organization
has only one Level 4 result. While the result, or strategic outcome, ties back to the
organizational purpose, the number of people, projects, and initiatives that contribute to the final
business result are vast. Kirkpatrick and Kirkpatrick (2016) noted that “leading indicators” can
“help bridge the gap between individual initiatives and organizational results” (p. 13).
Once the organizational goal is determined and communicated, the organization develops
expectations and measures for the three other levels in the model. For Level 3, Behavior,
organizations develop and communicate the critical behaviors, the required drivers (such as
coaching, recognition, and performance reviews), and the on-the-job learning that will help the
team members reach their goals. For Level 2, Learning, the model outlines how organizations
measure the degree to which team members have acquired “the intended knowledge, skills,
attitude, confidence, and commitment” that will ultimately help them achieve their goals
(Kirkpatrick & Kirkpatrick, 2016, p. 15). Finally, for Level 1, Reaction, the level of response
can be measured and applied. Response includes team member engagement, training relevance,
and overall satisfaction with training and development. Using this framework to create an
evaluation plan helps the organization focus, first, on the overall desired result or program
outcome. The organization can then design a plan that helps all members on the team take
responsibility and be accountable for achieving the organizational goal.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

139
Organizational Purpose, Need, and Expectations
The vision of the University of Future Leaders is to transform from an 18th-century
academic education model to a 21st-century business education model that differs dramatically
in content, professors, delivery, and how it addresses student needs. The mission of the UFL is
to provide an innovative and entrepreneurial learning environment that mirrors business practices
while expanding thought leadership, identifying key global opportunities, and exploring critical
issues (UFL Strategic Plan, 2015). This study evaluated the UFL Online Learning Team’s
organizational performance as it relates to the problem of practice to break outdated traditional
pedagogies and improve student learning and engagement. In order to serve the needs of
business leaders and attract future students, the organizational goal for the UFL is to develop a
strategic plan to identify, measure, and improve online student learning and engagement that
results in preparing graduate students with the most relevant competencies for future leadership
roles in business.
This goal relates directly to the problem of practice and the UFL mission by being
recognized among the top United States universities as one of the leaders in providing innovative
and entrepreneurial online learning environment. The achievement of the UFL’s goal will be
measured by annual growth in rankings across several respected publications including U.S.
News & World Report, the Princeton Review, and Forbes. The team’s specific objective is to be
recognized in the top five online MBA programs in the United States university rankings by the
end of 2019 and number one by the end of 2021. The UFL Dean and the Dean’s Cabinet
established this measure during a strategic planning session in 2015.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

140
Level 4: Results and Leading Indicators
Table 11 outlines Level 4: Results and leading indicators for the Online Learning Team
plan, including the specific outcomes, metrics, and methods that are recommended for measuring
success. If the internal recommendations are implemented and the outcomes are achieved, the
organization will have a greater chance at successfully achieving the external outcomes.

Table 11
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
External Outcomes
1. Annual improvement in
reputation among
academic colleagues.
1a. Annual growth in rankings
among online MBAs.
Compare rankings in U.S. News and Poets
and Quants; annually.
1b. Recognized in the top five by
the end of 2019 and as #1 by the
end of 2021 according to
respected rankings organizations.
Track annual results published in respected
industry publications such as U.S. News &
World Report, the Princeton Review, and
Forbes among others.
2. Annual improvement in
reputation among
business leaders.
2a. Improved scores on key
questions.
Annual business leader survey; tracked
annually
2b. Improved relationships with
key business partners.
Compare YOY recruiting and placement of
students from the Online MBA program.
3. Annual growth in new
student applications.
The number of new applications. Track the total number of applicants
annually.
4. Annual improvement in
the quality and diversity
of students admitted.
The improvement in important
KPIs including:
- GPA and GRE scores
- GMAT scores
- professional student titles
- number of minority students
- number of female students
Track each measure annually; store and track
student information in the Salesforce
database.
5. Growth in recognition
among colleagues and
respected industry
research organizations.
The number of invitations to
speak and/or participate in
industry forums and conferences.
Track number and quality of speaking
opportunities and invitations for research
articles.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

141

Table 11, continued
Outcome Metric(s) Method(s)
Internal Outcomes
6. The “next phase”
vision and goals of the
Online Learning
Organization are aligned
and clear to all
stakeholders.
Communicated and published
annual formal goals for the
Online Learning Organization
that cascade from the business
school goals and are aligned with
the overall business school
strategy.
Offsite planning session with the team is
launched and annual goals are agreed upon
and published to the team and the UFL
business school leaders.
Goals are published annually.
7. Published strategic plan
that identifies, measures
and improves online
student learning and
engagement.
7a. The written strategic plan that
has been presented to the Dean’s
Cabinet and approved for
execution.
OLT Strategic Plan presented and approved
by the Dean’s Cabinet as indicated by the
approval memo from the Vice Dean, Online
Learning.
7b. The published strategic plan
that is being executed, reviewed,
and adjusted quarterly in year
one; annually starting in year two.
Annual report from the Vice Dean, Online
Learning about progress and completion of
each of the tactics and how they continue to
move the strategic initiatives forward.
8. Improved relationships
with faculty.
8a. The number of faculty who
agree to participate in ongoing
faculty development training.
Track and compare annual demonstrated best
practices by faculty for the most engaging
online learning methods and tools for
creating engaging curriculum.
8b. The number of faculty who
return to teach each year.
Compare number of participating faculty,
length of service, and effective outcomes
YOY.
8c. The number of faculty who
agree to participate in peer
feedback sessions.
Track semester feedback from faculty that
helps break down barriers, quickly solve
problems, and collect best practices to build
trust and credibility as partners.
8d. The number of faculty who
promote the online programs
among their faculty peers.
Track responses to end of semester survey
about the faculty’s willingness to promote
teaching in the online programs and number
of faculty who were introduced to the
programs by respected faculty peers.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

142

Table 11, continued
Outcome Metric(s) Method(s)
Internal Outcomes
9. Improved student
engagement and student
satisfaction.
9a. The growth in key
competencies for securing roles in
the business world.
Solicit and track feedback from students at the
end of the program; track success of their e-
Portfolios; track outcomes of individual and
collaborative projects; track outcomes of final
integrated capstone project.
9b. The student participation in
social activities.
Solicit and track feedback from students at the
end of each semester.
9c. The growth and maintenance of
both retention and graduation rates
according to the team goals.
Track and compare both retention and
graduation rates YOY and assess performance
by comparing to team goals.
9d. Improved scores on key
questions related to engagement
and student satisfaction.
Track data on annual student survey and
compare student engagement and satisfaction
scores.
10. Improved brand and
relationships among
business school
partners.
10a. Public recognition by
university and business school
leaders on the Dean’s cabinet.
Track and discuss number of times the Online
Learning Team’s successes and
accomplishments are recognized by the Dean
at the staff and faculty meetings as well as in
state-of-the-business publications.
10b. Inclusion in the UFL graduate
school activities and events.
Track and discuss number of times the OLT
receives invitations from the Vice Deans on
the Dean’s Cabinet to participate in industry
events as contributors.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

143
Level 3: Behavior
Critical behaviors. The stakeholder of focus for this study is the Online Learning Team,
who are responsible for online learning at the University of Future Leaders. There are five
critical behaviors that the OLT will need to demonstrate to achieve the outcomes. The first is to
implement a strategic planning process. The second is to execute their individual goals based on
the overall organizational goals. The third critical behavior is to build a system for continuous
measuring and tracking of goals, as well as communicating the outcomes. The fourth critical
behavior is to build a relationship with faculty to develop trust and ultimately improve student
engagement. The fifth critical behavior is to build relationships with the business school leaders
and other important university leaders. The critical behaviors, metrics, methods, and timing for
each of the behaviors are outlined in Table 12.
Required drivers. In order to successfully implement the recommendations in this plan,
the Online Learning Team will need the support of their team members, their direct managers,
and the other leaders inside the organization to reinforce the importance of developing and
achieving goals as a team. They will also need to support and recognition from other leaders in
the business school in order to build self-efficacy and confidence in their contributions to the
university. Rewards should be provided based on the needs of the individual stakeholders as
goals are achieved. Table 13 will further describe the recommended drivers for supporting the
Online Learning Team’s critical behaviors.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

144
Table 12
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s) Method(s) Timing
1. Implement the
Strategic Planning
Process.
The completion of an
offsite planning session
with the team to establish
and agree on goals.
The Vice Dean and the
Director will plan, launch,
and facilitate the first offsite
planning session.
Progress against plan will
be published to the team.
In the first 90 days
after approval for the
meeting.
Progress against plan
will be tracked and
published quarterly.
2. Execute against
individual goals that
cascade from the
organizational goals.
The number of goals that
are achieved.
2a. The Team Members will
initiate a meeting with their
Supervisors to review and
document progress against
goals.
Monthly in year one.
Quarterly starting in
year two of the plan.
2b. The OLT Director will
identify barriers and
provide resources to help
Team Members achieve
their goals.
Monthly in year one.
Quarterly starting in
year two of the plan.
3. Build a system for
continuous tracking of
goals, measuring results,
and publishing outcomes.
An approved and
accepted system with
examples of tracking,
measuring, and
publishing results.
The OLT Director will
work with each department
Supervisor to vet and agree
on a system.
Progress will be
reported monthly to
both internal and
external stakeholders.
4. Build relationships
with faculty to build trust
and improve student
engagement.
The number of faculty
who respond to
invitations to collaborate
including:
- faculty training and
development
- peer feedback sessions
- promoting the program
w/ peers
Each group on the Online
Learning Team will take
responsibility for an
element of building
relationships with the
faculty and then report back
to the team on progress.
Teams will report on
progress quarterly.
5. Build relationships
with business school
leaders and other
university leaders.
The number of times the
Online Learning
Organization is supported
and recognized as
contributors to the overall
business school
successes.
The Vice Dean and the
Director of the OLT will
work together to develop a
plan for building the brand
and growing credibility
among other UFL business
school leaders.
Leaders will share
outcomes with the
team on a quarterly
basis.
Dean of the Business
School will meet bi-
annually with the team
to discuss progress.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

145
Table 13
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical
Behaviors
Supported
1, 2, 3 Etc.
Reinforcing
Job aid that outlines the one organizational goal for the year taken from the
Strategic Plan.
Ongoing 1, 2, 3
Job aid that reflects specific quarterly departmental goals. Ongoing 1, 2, 3
Job aid that outlines each individual’s quarterly goals. Ongoing 2
Team meeting to discuss progress against the strategic plan. Quarterly 1
Meeting or call with each active faculty member to build trust, solve
problems, and encourage faculty development.
Monthly 4
Meeting with at least one business school leader to build relationships and
seek guidance on key issues.
Monthly 5
Team meeting to discuss weekly progress against individual goals and ensure
measures are in place and being tracked.
Weekly 3
Encouraging
Internal “Communities-of-Inquiry” is launched and led by OLT Instructional
Designers to enhance and encourage teaching presence among faculty.
Bi-
annually
4
Faculty are recognized for creative student engagement activities and
outcomes.
Quarterly 4
Brainstorming sessions are scheduled and led by different team members to
build interdependent teams and creative problem-solving.
Quarterly 1-5
Rewarding
Vice Dean and Director recognize key team member accomplishments. Quarterly 1-5
Team members invited to attend conferences with leaders as performance
incentives for achievement of individual goals.
Bi-
annually
1-3
One team member has the opportunity to present a new, innovative learning
idea to the Dean’s Cabinet.
Annually 5
Dean of the business school publicly recognizes the OLT for key
accomplishments at All-staff meetings and during interviews about the future.
Quarterly 1, 5
Monitoring
Vice Dean reports back to the team on Dean’s Cabinet discussions about
online learning.
Bi-weekly 1-3
Team members share success stories at team meetings. Weekly 1-3
KPI’s reviewed at team meetings. Weekly 1-5

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

146
Organizational support. In order to support the Online Learning Team’s critical
behaviors, the recommendation is for the organization to help the Online Learning Team move
into the next phase of growth. The OLT will transition from their “program launch phase” to a
“strategic growth phase” in online learning. This transition will include setting and managing
strategic goals, creating systems with tracking and measures that help them track against their
goals, and building relationships with both internal stakeholders (e.g., faculty, other business
school leaders, other online learning teams) and external stakeholders (e.g., business leaders,
other universities, other industry organizations). The Online Learning Team needs support from
the organization to launch an offsite strategic planning meeting in order to develop a plan that
will guide the growth of the business over the next three years.
Level 2: Learning
Learning goals. After the recommendations have been implemented, most notably the
three-year strategic plan, the OLT will be able to:
1. Implement the most effective methods for keeping students engaged in an online
program. (P)
2. Develop online curriculum that provides specific steps and deliverables for the
faculty to teach transferable skills that tie back to critical competencies needed in
business. (P)
3. Demonstrate confidence that they are capable of effectively developing online
learning and instruction the will engage online students. (Confidence)
4. Identify the one organizational goal from the strategic plan that is the focus for the
team during the year. (P)
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

147
5. Recognize how their individual goals fit into the overall strategic goal for the
organization and also for the UFL business school. (Conceptual)
6. Identify and effectively communicate resources needed to achieve their goals. (P)
7. Create a trusted system for continuous tracking of goals, measuring results, and
publishing outcomes. (P)
8. Reflect on new and novel ways to improve the overall credibility of online learning
models across all stakeholders. (M)
9. Demonstrate that they have developed a culture of trust with internal stakeholders
including faculty and other business school leaders by creating specific opportunities
for the groups to share best practices, recent successes, and innovative solutions. (P)
Strategic implementation plan. The learning goals listed in the previous section will be
achieved with the implementation of a strategic plan that will be developed and communicated
by the Online Learning Team. The objective of the strategic plan will be to identify, measure,
and improve online student learning and engagement that results in preparing graduate students
with the most relevant competencies for future leadership roles in business. The Online
Learning Team including a facilitator, all of the OLT team members, the department Director,
and the Vice Dean of Online Learning will participate in a one-and-a-half day, offsite planning
meeting. The meeting will include three modules, each with specific discussion topics and
deliverables.
During the first module (morning of Day 1), the participants will receive three strategic
documents or job aids. The team will start by reviewing and discussing a job aid that
communicates the current the University of Future Leaders overall mission, vision, and strategic
initiatives. The group will also receive a job aid listing the strategic initiatives of the UFL
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

148
Business School. Finally, the team will review a SWOT (Strengths, Weaknesses, Opportunities,
and Threats) analysis for the organization and make adjustments as desired. Following this
review and discussion, the team will participate in a brainstorming session about specific
organizational goals, strategies, and measures for the Online Learning Organization growth over
the next three years. As the final deliverable for this module, the group will create the first draft
of the year one department’s quantifiable organizational goals, the strategic initiatives to ensure
the goals can be achieved, and the specific measures to be used to ensure progress is being made.
The module will conclude with the group agreeing on the most important broad objective for the
first year in the plan.
The second module (afternoon of Day 1) of the planning meeting will include a focus on
each of the strategic initiatives. The group will be separated into smaller breakout teams and
each team will tackle one of the strategies to create specific tactics and measures, including
milestones for success and time frames. In addition, the teams will outline departmental key
performance indicators (preferably, no more than four) and discuss action plans, risks and
barriers, and critical resources that are needed for successful implementation of their strategic
initiatives. As the final deliverable for this module, each team will present their strategic
initiative, tactics, and measures to the larger team.
For the third and final module (morning of Day 2), the Online Learning Team will focus
on the implementation, evaluation, and communication elements for each of the strategic
initiatives. The deliverables for this module will include detailed action plans for each of the
strategic initiatives. Teams will work together to help each other determine priorities. The role
of the facilitator is to ensure all of the tactics are manageable and will also help simplify the plan
where appropriate. In addition, the team will agree on how often they will meet to discuss
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

149
progress against the plan over the next year, so they are able to review progress, obtain
resources, and make adjustments where needed. As the final deliverable for the strategic
planning meeting, the team will revisit the most important departmental objective for the first
year in the plan and agree that this is the one goal they will all strive to achieve in year one. The
team will also decide whether to meet annually to revisit strategic initiatives and decide on the
next year’s goal.
Evaluation of the components of learning. For many of the participants in the
department, this exercise will be the first time they have participated in a strategic planning
meeting. Learning will take place in a live setting while they are actually applying what they
learn in real time. In order for the Online Learning Team to apply this new knowledge to solve
problems and create plans, they will need to demonstrate that they have both declarative and
procedural knowledge. As a result, it will be important to evaluate the learning for both of these
components. In addition, the participants will need to feel confident that they can apply their
newly acquired knowledge and skills to execute against the department’s final strategic plan.
Table 14 outlines the evaluation methods and timing for each of these learning components.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

150
Table 14
Evaluation of the Components of Learning for the Program
Method(s) or Activity(ies) Timing
Declarative Knowledge “I know it.”
Knowledge checks by the facilitator during the planning
session.
From time-to-time during the
planning session as teams are
brainstorming ideas and solutions.
Knowledge checks while the teams are presenting their
deliverables.
At the end of each module during the
strategic planning session.
Checks for understanding in follow-up calls and meetings. During the implementation phases of
the plan.
Procedural Skills “I can do it right now.”
Directed questions by the facilitator to check in with team
members about how they will implement a tactic.
During the planning meeting.
Facilitator’s observation of those who are leading and those
who are participating in each team discussion.
During the planning meeting.
Demonstration that they can use the job aids to develop
strategic initiatives and tactics for their team.
During the planning meeting.
Individual demonstration of understanding about the
outcomes.
End of each module during the
planning meeting.
The attitude “I believe this is worthwhile.”
Facilitator observations about engagement and actions to
help create the plan.
During the planning meeting.
Participant comments about the value of the meeting and
how they have grown as a result.
After the planning meeting, during
breaks, during meal-breaks, and
before the 2nd-day session.
Reflective comments from a post-meeting survey. One week after the planning meeting.
Confidence “I think I can do it on the job.”
Observations about how the team members use the plans to
guide their day-to-day job tasks.
Over the course of the next quarter.
Discussions following application and feedback. After the planning meeting.
Reflective comments and response rates from a post-
meeting survey.
One month after the meeting.
Commitment “I will do it on the job.”
Discussions about how this plan will help them in their jobs. During the planning meeting.
Discussions when participants return to their jobs. After the planning meeting.
Progress against goals using metrics agreed upon. Quarterly after planning meeting.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

151
Level 1: Reaction
After the strategic planning meeting, it is critical to gather feedback and reactions about
the value of the meeting, the outcomes of the plan, and the impact of the work that was
completed. Table 15 discusses the methods and tools that will be used to evaluate the
components of engagement, relevance, and satisfaction with the learning and outcomes while
creating a strategic plan for the organization.

Table 15
Components to Measure Reactions to the Program
Method(s) or Tool(s) Timing
Engagement
Participation and contributions in large group. During the planning meeting.
Participation and contributions in subgroups. During the planning meeting.
Completion and agreement about the publication
of the strategic initiatives to the rest of the team.
During the planning meeting.
Agreement on key tactics. During the planning meeting.
Director observations of team activities. Ongoing during the meeting.
Survey feedback. At the end of the meeting.
Relevance
Director observations of team members making
decisions based on the plan.
During the planning meeting.
Live brief pulse-survey with team members.
Brief pulse-survey.
During the planning meeting.
At the end of the planning meeting.
Survey feedback. At the end of each assigned monthly meeting.
Customer Satisfaction
Team meeting discussions. At the end of each assigned weekly meeting.
Live brief pulse-survey with team members.
Brief pulse-survey.
During the planning meeting.
At the end of the planning meeting.
Survey feedback. At the end of each assigned weekly meeting.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

152
Evaluation Tools
Immediately following the plan implementation. During the strategic planning
meeting, qualitative data will be collected by the facilitator and the organization’s director in the
form of observations and note-taking. Both the participation and contributions of the participants
in the larger group setting and in the sub-group, breakout sessions will be observed. Data will be
collected during the discussions and at the end of each module during the planning meeting to
check for engagement.
Level 1 Reaction data will be collected during the strategic planning session by the
facilitator in the form of brief questions to check for understanding about both the relevance of
the content and also the satisfaction about the process during the meeting. Participants will be
asked at the end of each module about whether the sessions achieved the desired outcomes.
Level 2 Learning data will also be collected during the strategic planning session by the
facilitator. The facilitator will engage in conversations with each of the participants to check for
understanding about their roles in implementing the plan and to confirm they have some ideas
about how they will achieve their own goals. The facilitator will also ask different participants
to lead conversations during the planning sessions to help them begin building confidence that
they can achieve their goals. Appendix D is a sample of a survey using a Blended Evaluation
approach (Kirkpatrick & Kirkpatrick, 2016), which will be given to all participants at the end of
the strategic planning session to check for Level 1 and Level 2 outcomes.
Delayed for a period after the plan implementation. Approximately eight weeks after
the strategic planning session, an additional brief survey using the Kirkpatrick and Kirkpatrick
(2016) Blended Evaluation approach will be administered to further assess outcomes (see
Appendix E). The survey will be used to confirm that the Online Learning Team has begun to
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

153
see progress against their goals (Level 4 Results) and that they have begun demonstrating new
and improved behaviors (Level 3 Behaviors). The survey will also check for an increase in
overall knowledge, skills, confidence, and commitment (Level 2 Learning) and confirm that the
planning sessions were relevant and engaging. Finally, the team members will be asked if the
new strategic plan is helping them achieve their goals. Progress against individual goals will be
assessed by both the director and the facilitator, and a decision will be made about whether an
additional strategic planning session should be scheduled.
Data Analysis and Reporting
Once the Online Learning Team has developed a detailed strategic plan for the Level 4
goal, it will need to measure and track all elements of the plan to determine how it is improving
online student learning and engagement. Figure 3 shows an example report listing some of the
goals in each of the four levels of the Kirkpatrick and Kirkpatrick New World Model (2016)
including key performance indicators, tracking frequency, targeted outcomes, actual outcomes,
and a color-coded progress indicator. This report is one of several reports which may be used to
measure, track, and report on progress against organizational goals.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

154

Figure 3. Online Learning Organization strategic plan — progress and accountability report

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

155
Summary
The framework used for this evaluation was the New World Kirkpatrick Model
(Kirkpatrick & Kirkpatrick, 2016). The model focuses on the development of proposed
solutions, the creation of implementation plans, and a roadmap for an overall evaluation of the
plan. The Online Learning Team’s organizational goal is an executable strategic implementation
plan for improving student learning and engagement in the Online MBA at the University of
Student Leaders. The New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) begins
with the end result or desired outcome to ensure everyone on the team is striving to achieve the
same goal. Next, the model helps the organization determine and monitor critical behaviors and
required drivers that are needed once the team members are back at their jobs and beginning to
implement the program or the strategic plan. This element of the model is called Level 3
Behavior. “Active execution and monitoring of required drivers is perhaps the biggest indicator
of program success for any initiative” (Kirkpatrick & Kirkpatrick, 2016, p. 14).
The next focus is on Level 2 Learning, which measures the degree to which participants
develop additional knowledge, improved skills, positive attitudes, and increased confidence and
commitment to achieve their goals. In the case of the Online Learning Team, the process of
formal goal setting and tracking will be new to many of the team members. Helping the team
members learn how to close the learning gap in these areas will be critical success factors in
order to ensure the team achieves their organizational goal. Finally, the evaluation process
focuses on Level 1 Reaction which assesses engagement, relevance, and overall customer
satisfaction. While this level is less critical than the other levels, it is still an important element
of the plan to ensure all of the team members are actively engaged and focused on achieving
both their own goals and the goals of the organization.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

156
Strengths and Weaknesses of the Approach
This study employed a qualitative data gathering and analysis approach. The UFL Online
Learning Team’s knowledge and motivation related to improving student learning and
engagement, developing critical competencies in graduate students, and achieving the
organizational goal was assessed using interviews with key stakeholders and document analysis.
Merriam and Tisdell (2016) noted that a qualitative study with interviews helps the researcher to
uncover: “(1) how people interpret their experiences, (2) how they construct their worlds, and (3)
what meaning they attribute to their experiences” (p. 24). Important and relevant organizational
effects were evaluated based on the Clark and Estes (2008) gap analysis framework, focusing on
knowledge, motivational, and organizational influences in the organization. Research-based
solutions were then recommended at the completion of the study. Both the qualitative approach
and the gap analysis framework helped strengthen this study as they helped the researcher
identify specific gaps in the knowledge and motivation of the stakeholders and how the
organization supported the Online Learning Team. In addition, the use of a qualitative approach
helped the researcher focus on the perspectives from key stakeholders responsible for managing
the department. These are also the stakeholders who will be responsible for implementing the
solutions. The primary weakness in the approach used in this study was the use of only two
methods for data collection. With more time, the use of a mixed-method approach that included
both qualitative and quantitative data would have been valuable and would have enhanced the
overall credibility of the study with even stronger triangulation of the data.
Limitations
In reviewing the limitations for this study, the researcher recognized that the study would
have been enhanced with perspectives from other key stakeholders. With more time, several
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

157
additional stakeholders could have been interviewed, including the faculty who are partially
responsible for some of the most critical components of student engagement. The faculty could
also provide additional perspectives about their needs and issues with continuing development.
In addition, the study could have included interviews from the business leaders who hire students
graduating from the Online MBA program. Finally, the students themselves have a great deal of
knowledge, experience, and relevant opinions about the research questions in this study. The
student perspective could have been collected using a quantitative survey and administered to
each of the first five cohorts who have already graduated from the Online MBA program to
inquire about what they think are critical elements of student engagement. The feedback from
each cohort could have been compared and contrasted for a more in-depth look at online student
engagement and critical competency development. Each of these stakeholder groups would have
added additional depth to the study.
Future Research
As more universities explore the growth and demand for online learning, particularly at
the graduate level, additional research opportunities emerge. First, this study introduced the
importance of student engagement for effective online learning and outcomes to ensure graduate
students leave graduate school with the critical competencies they need to obtain global business
roles and grow in their careers. In light of these important requirements, the first
recommendation for future research is to conduct a longitudinal study using a mixed-methods
approach for each of the graduating cohorts in the Online MBA program at the UFL business
school. The combination of an ongoing survey and qualitative interviews with each of these
student groups would provide additional knowledge in at least four areas: (1) whether the
students developed new competencies as a result of attending the Online MBA program at the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

158
UFL, (2) which of the competencies are valued most and why, (3) whether these new
competencies were effective at helping the students obtain global roles and become more
impactful in the business world, and (4) how the level and forms of student engagement
enhanced the experience during the Online MBA program.
The second opportunity for additional research would involve the faculty at the UFL
school of business. As discussed in Chapter 4, some of the faculty at the UFL business school
are hesitant to explore teaching opportunities in online learning. In addition, while the study
recommended introducing programs for additional faculty development, some faculty who are
already participating in online teaching will not take advantage of this valuable resource to
further develop their skills. Separately, understanding why some of the more experienced
faculty choose not to teach in online graduate-level programs, and also why others who do teach
are unwilling to attend programs to further develop their own skills would help expand the
knowledge and the resources in this rapidly growing area of graduate-level learning. The
broader benefits of engaging faculty would result in having them leading the research about new
and innovative student engagement strategies, particularly as technology evolves. Faculty
research about specific and targeted online student engagement methods would likely result in
new ideas to further inspire students and open doors for those students who, in the past, did not
have an option to pursue a graduate degree due to location or time constraints.
Revisiting the Conceptual Framework
A conceptual framework was introduced at the beginning of this study to be used as a
model for exploring the operations, the culture, and the organizational environment in both the
Online Learning Organization (OLT) and the business school at the University of Future
Leaders. Using the framework, the findings from the study identified which internal and external
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

159
influences impacted the OLT and how these influences were interacting to help or hinder the
organization achieve its goals.
Figure 4 demonstrates the internal knowledge and motivational influences that
specifically impact the OLT. These are represented together inside the green rectangle. The
external organizational influences are separate and are shown with the purple arrows.

Figure 4. UFL Online Learning Team conceptual framework

The findings from this study demonstrated that the internal influences were both valid
and instrumental. The OLT, who launched the online MBA program, moved from a test
initiative to a significant, contributing growth organization in the business school at the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

160
University of Future Leaders. Over the course of just a few years, the organization developed
both strong conceptual and procedural knowledge. All of the participants in this study were able
to identify the most important competencies of data analytics, critical thinking, and interpersonal
skills (conceptual knowledge). In addition, the team designed a training program to help faculty
develop and learn the most effective methods for building student engagement (procedural
knowledge). The only major knowledge gap that revealed itself during the study was the need
for a formal system of tracking and measures.
The study also revealed that there is some opportunity in the area of motivational
influences. The OLT demonstrated strong drive, powerful passion, and a high sense of urgency
to help their graduate students develop the critical competencies they need to become future
industry leaders and, as a result, had a strong sense of self-efficacy about the specific operations
of the organization. However, the OLT expressed some concern about the lack of recognition
among the business school leaders about the value of online learning. In addition, there was
some lack of clarity around the organizational goals for the department. By themselves, both of
these gaps appear to be relatively minor. They reflect common opportunities found in
organizations moving from a start-up phase of implementation to the next phase of strategic
growth.
Perhaps the most important observation during this study was how several minor gaps in
the knowledge, motivational, and organization influences, when combined together, can result in
a significant impact on the OLT’s self-efficacy and the team’s ability to achieve the
organizational goal. Most importantly, the team members needed to believe that work they were
doing to develop new online learning models would be accepted and viewed as legitimate by
others in the organization. “Legitimacy is a generalized perception or assumption that the
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

161
actions of an entity are desirable, proper, or appropriate within some socially constructed system
of norms, values, beliefs, and definitions” (Suchman, 1995, p. 574). When the minor gaps that
include a lack of tracking and metrics, a lack of clarity around organizational goals, and a
perceived lack of recognition from the business school leaders are combined with the OLT’s
objective to break outdated traditional pedagogies and make important changes to improve
student learning, the study revealed that the organization may still be facing several significant
internal challenges.
As the OLT drives change and challenges the current set of norms at the UFL, they will
transform from an early-stage start-up to a fully operating revenue contributor. This
transformation may represent a threat to those in the organization who operate and lead more
traditional departments or programs. Each decision the OLT makes will likely be questioned
until the team can demonstrate it is adding value. Requests for resources will likely be
scrutinized until the organization can produce facts and data about positive results. Getting
faculty to take risks and spend time learning a completely new model will require a continued
focus on getting testimonies from both students and faculty. Being recognized in the top five
online MBA programs in the United States university rankings by the end of 2019 and reaching
number one by the end of 2020 will become an important factor in establishing legitimacy for the
Online Learning Team at the University of Future Leaders.
Conclusion
Universities are recognizing that online learning, particularly at the graduate level, is an
important strategic tool in higher education. In addition, online learning is helping universities
find new paths for growth that address the challenges of declining enrollment and retention in
addition to technological disruption in both business and academia (Meyer, 2014). This study
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

162
focused on understanding how to improve online student learning and engagement that results in
preparing graduate students with the most relevant competencies for future leadership roles in
business. The Online Learning Team at the University of Future Leaders was selected as the
organization of focus for the study because they have responsibility for the overall growth in
online learning in the business school and accountability for assessing and implementing the
recommended solutions in this study. The Clark and Estes (2008) framework was used for this
study to gather relevant information about the stakeholder knowledge, motivation, and
organizational influences. Once the data was gathered from document analysis and interviews
with the Online Learning Team, the Dean of the business school, and several members of the
Dean’s Cabinet, the data was assessed and analyzed for gaps between the current stakeholder
performance and the organizational goals.
The data from this study revealed that the Online Learning Team leaders and
instructional designers were both knowledgeable and motivated about developing the critical
elements of student learning and engagement. They also had strong knowledge about the
competencies that Online MBA graduates need to help secure leadership roles in a global
business environment. In addition, all of the team members were motivated to improve student
engagement by building relationships with faculty and other university leaders to align values
and develop new models of learning to meet the needs of future adult learners. The Online
Learning Team also had strong support in the form of financial and operational resources from
business school leaders.
As the Online Learning Team transitions from a “program launch phase” to a “strategic
growth phase” in online learning, the data also revealed several gaps in the knowledge and
organizational influences. Most important, the OLT members did not know the formal goals for
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

163
the department related to student learning and engagement even though they had some
knowledge of general goals and a high level of motivation for building a high-quality program.
The OLT also recognized the gap in reporting and the need to develop measures and ongoing
tracking for understanding overall student engagement over time. At the organizational level,
there were two important growth opportunities identified: finding ways to create momentum
around faculty development and ensuring the senior business school leaders were both aware and
willing to formally recognize the accomplishments of the Online Learning Team.
The implications of these findings identify the opportunities associated with being able to
grow new enrollment, retain and serve students who are currently enrolled, and reach the
organizational goal to prepare graduate students with the most relevant competencies for future
leadership roles in business. The recommendations for the OLT at the University of Future
Leaders include: (1) developing a program to identify, measure, and improve online student
learning and engagement, (2) launching a strategic planning process for setting, communicating,
and achieving organizational goals, (3) building relationships with faculty and creating a
program for continuous faculty development, (4) building a system for continuous tracking of
goals, measuring results, and publishing outcomes, and (5) building relationships with business
school leaders and other university leaders to help enhance the OLT brand and grow awareness
of the program’s key accomplishments. By implementing the recommended solutions in this
study, the Online Learning Team at the UFL will begin to achieve the vision to transform from
an 18th-century academic education model to a 21st-century business education model that
differs dramatically in content, professors, delivery, and how it addresses student needs.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

164
REFERENCES
Agocs, C. (1997). Institutionalized resistance to organizational change: Denial, inaction and
repression. Journal of Business Ethics, 16(9), 917–931.
Akyol, Z., & Garrison, D. R. (2011). Assessing metacognition in an online community of
inquiry. The Internet and Higher Education, 14(3), 183–190.
Allen, I. E., & Seaman, J. (2012). Conflicted: Faculty and online education, 2012. Babson Park,
MA and Washington, D.C.: Babson Survey Research Group and Inside Higher Ed.
Allen, I. E., & Seaman, J. (2017). Digital learning compass: Distance education enrollment
report 2017. Babson Park, MA: Babson Survey Research Group, e-Literate, and WCET.
Ambrose, S. A., Bridges, M. A., Lovett, M. C., DiPietro, M., & Norman, M. K. (2010). How
learning works: Seven research-based principles for smart teaching. San Francisco, CA:
Jossey-Bass.
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich,
P. R., . . . Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A
revision of Bloom ’s taxonomy of educational objectives (Abridged edition). White Plains,
NY: Longman.
Anthony, S., & Garner, B. (2016). Teaching soft skills to business students: An analysis of
multiple pedagogical methods. Business and Professional Communication Quarterly,
79(3), 360–370.
Ayling, A., Price, M., Tucker, B., & Vellner, T. (2015). Reimagining business education: A
world of ideas. Boston, MA: Boston University.
Baker, C. (2010). The impact of instructor immediacy and presence for online student affective
learning, cognition, and motivation. The Journal of Educators Online, 7(1), 1–30.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

165
Baker, C., & Taylor, S. L. (2012, February). The importance of teaching presence in an online
course. In Faculty Focus: Special Report: Online Student Engagement Tools and
Strategies. Madison, WI: Magna Publications.
Baker, L. (2011). Metacognition. In V. G. Aukrust (Ed.), Learning and cognition in education
(pp. 204–210). Oxford, UK: Academic Press.
Bandura, A. (2000). Exercise of human agency through collective efficacy. Current Directions in
Psychological Science, 9(3), 75–78.
Barber, M., Donnelly, K., Rizvi, S., & Summers, L. (2013). An avalanche is coming: Higher
education and the revolution ahead. London, UK: Institute for Public Policy Research.
Barton, D., Farrell, D., & Mourshed, M. (2012). Education to employment: Designing a system
that works. New York, NY: McKinsey. Retrieved from
http://www.mckinsey.com/industries/social-sector/our-insights/education-to-
employment-designing-a-system-that-works
Bates, R., & Khasawneh, S. (2007). Self-efficacy and college students’ perceptions and use of
online learning systems. Computers in Human Behavior, 23(1), 175–191.
doi:10.1016/j.chb.2004.04.004
Bedwell, W. L., Fiore, S. M., & Salas, E. (2014). Developing the future workforce: An approach
for integrating interpersonal skills into the MBA classroom. Academy of Management
Learning & Education, 13(2), 171–186.
Bisoux, T. (2016, February). Business schools must adapt to an ever-changing world. BizEd
Magazine. Tampa, FL: AACSB.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

166
Bouchikhi, H., & Kimberly, J. (2014, July 20). Time to evolve or perish in a changing world.
Financial Times. Retrieved from https://www.ft.com/content/f383467e-f705-11e3-8ed6-
00144feabdc0?mhq5j=e3
Bruce, G., & Schoenfeld, G. (2006, June 7). Marketers with MBAs: Bridging the thinking-doing
divide (GMAC Research Reports RR-06-06). Reston, VA: Graduate Management
Admission Council. Retrieved from https://www.gmac.com/-
/media/files/gmac/research/research-report-series/marketers-with-mbas.pdf
Byrne, J. A. (2015). Are American business schools headed for a GM-like catastrophe? Fortune.
Retrieved from http://fortune.com/2015/08/17/business-schools-future/
Carey, K. (2015). The end of college: Creating the future of learning and the university of
everywhere. New York, NY: Riverhead Books.
Chang, C.-C. (2003). Towards a distributed web-based learning community. Innovations in
Education and Teaching International, 40, 27–42.
Clark, R. E. (2005). Research-tested team motivation strategies. Performance Improvement, 4(1),
13–16.
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Charlotte, NC: Information Age.
Colquitt, J., Scott, B., & Lepine, J. (2007). Trust, trustworthiness, and trust propensity: A meta-
analytic test of their unique relationships with risk taking and job performance. The
Journal of Applied Psychology, 92(4), 909–927. doi:10.1037/0021-9010.92.4.909
Conger, J. A. (1991). Inspiring others: The language of leadership. Academy of Management
Perspectives, 5(1), 31–45.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

167
Cornuel, E. (2007). Challenges facing business schools in the future. Journal of Management
Development, 26(1), 87–92.
Cress, C. M., Yamashita, M., Duarte, R., & Burns, H. (2010). A transnational comparison of
service-learning as a tool for leadership development. International Journal of
Organizational Analysis, 18(2), 228–244. doi:10.1108/19348831011046281
Dabbagh, N. (2007). The online learner: Characteristics and pedagogical implications.
Contemporary Issues in Technology and Teacher Education, 7(3), 217–226.
Darling-Hammond, L., Wilhoit, G., & Pittenger, L. (2014). Accountability for college and career
readiness: Developing a new paradigm. Education Policy Analysis Archives, 22(86).
doi:10.14507/epaa.v22n86.2014
Denning, S. (2005). The leader ’s guide to storytelling: Mastering the art and discipline of
business narrative. San Francisco, CA: John Wiley & Sons.
De Villiers-Scheepers, M. J. (2011). Motivating intrapreneurs: The relevance of rewards.
Industry and Higher Education, 25(4), 249–263.
Dixson, M. D. (2010). Creating effective student engagement in online courses: What do
students find engaging? Journal of the Scholarship of Teaching and Learning, 10(2), 1–
13.
Drucker, P. (1954). The practice of management. New York, NY: Harper & Row.
Dua, A. (2013). College for all. New York, NY: McKinsey. Retrieved from
https://www.mckinsey.com/industries/social-sector/our-insights/college-for-all
Educause. (2013). Massive Open Online Course: What is a MOOC? Educause. Retrieved from
https://library.educause.edu/topics/teaching-and-learning/massive-open-online-course-
mooc
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

168
Erez, M., & Gati, E. (2004). A dynamic, multi-level model of culture: From the micro level of
the individual to the macro level of a global culture. Applied Psychology: An
International Review, 53, 583–598. doi:10.1111/j.1464-0597.2004.00190.x
Ferguson, R., & Sharples, M. (2014). Innovative pedagogy at massive scale: Teaching and
learning in MOOCs. In C. Rensing, S. de Freitas, T. Ley, & P. J. Muñoz-Merino (Eds.),
Open learning and teaching in educational communities (EC-TEL 2014) (pp. 98–111).
Switzerland: Springer International. doi:10.1007/978-3-319-11200-8_8
Frey, T. (2013, July 5). By 2030 over 50% of colleges will collapse. FuturistSpeaker.com.
Retrieved from https://www.futuristspeaker.com/business-trends/by-2030-over-50-of-
colleges-will-collapse/
Friedman, J. (2017, January 5). Five online education trends to watch in 2017. U.S. News &
World Report. Retrieved from https://www.usnews.com/higher-education/online-
education/articles/2017-01-05/5-online-education-trends-to-expect-in-2017
Friga, P., Bettis, R., & Sullivan, R. (2003). Changes in graduate management education and new
business school strategies for the 21st century. Academy of Management Learning &
Education, 2(3), 233–249.
Gallup. (2014). What America needs to know about higher education redesign: Lumina Study of
the American Public ’s Opinion on Higher Education. Indianapolis, IN: Lumina
Foundation. Retrieved from https://www.luminafoundation.org/files/resources/2013-
gallup-lumina-foundation-report.pdf
Garrison, D. R. (2017). E-learning in the 21st century: A community of inquiry framework for
research and practice (3rd ed.). London, UK: Routledge/Taylor and Francis.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

169
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based
environment: Computer conferencing in higher education model. The Internet and
Higher Education, 2(2-3), 87–105.
Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of
inquiry framework: A retrospective. The Internet and Higher Education, 13(1), 5–9.
Garrison, D. R., & Cleveland-Innes, M. (2005). Facilitating cognitive presence in online
learning: Interaction is not enough. American Journal of Distance Education, 19(3), 133–
148.
Glesne, C. (2011). Becoming qualitative researchers: An introduction (4th ed.). Boston, MA:
Pearson.
Graduate Management Admission Council. (2016). 2016 Application Trends Survey report.
Reston, VA: Author. Retrieved from
https://www.gmac.com/~/media/Files/gmac/Research/admissions-and-application-
trends/2016-gmac-application-trends-web-release-v2.pdf
Great Schools Partnership. (n.d.). The glossary of education reform. Portland, ME: Author.
Retrieved from https://www.edglossary.org/
Grossman, R., & Salas, E. (2011). The transfer of training: What really matters. International
Journal of Training and Development, 15(2), 103–120.
Hanna, D. (2010, March 8). How GM destroyed its Saturn success. Forbes. Retrieved from
https://www.forbes.com/2010/03/08/saturn-gm-innovation-leadership-managing-
failure.html
Harding, J. (2013). Qualitative data analysis from start to finish. Thousand Oaks, CA: Sage.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

170
Hendry, C. (1996). Understanding and creating whole organizational change through learning
theory. Human Relations, 49(5), 621–641.
Holland, D., & Quinn, N. (Eds.). (1987). Cultural models in language and thought. Cambridge,
UK: Cambridge University Press.
Hu, S., & McCormick, A. C. (2012). An engagement-based student typology and its relationship
to college outcomes. Research in Higher Education, 53, 738–754.
Johnson, R. B., & Christensen, L. B. (2014). Educational research: Quantitative, qualitative,
and mixed approaches (5th ed.). Thousand Oaks, CA: Sage.
Kimberly, J., & Bouchikhi, H. (2016). Disruption on steroids: Sea change in the worlds of higher
education in general and business education in particular. Journal of Leadership &
Organizational Studies, 23(1), 5–12.
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick ’s four levels of training evaluation.
Alexandria, VA: ATD Press.
Knowles, M. S., Holton III, E. F., & Swanson, R. A. (2012). The adult learner (7th ed.). New
York, NY: Routledge.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice,
41(4), 212–218.
Kuh, G. D. (2008). High-impact educational practices: What they are, who has access to them,
and why they matter. Washington, D.C.: Association of American Colleges and
Universities.
Ladyshewsky, R., & Taplin, R. (2014). Leadership development using three modes of
educational delivery: Online, blended and face to face. International Journal on E-
Learning, 13(3), 273–290.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

171
Langley, G. J., Moen, R. D., Nolan, K. M., Nolan, T. W., Norman, C. L., & Provost, L. P.
(2009). The improvement guide: a practical approach to enhancing organizational
performance. San Francisco, CA: Jossey-Bass.
Latham, G. P., & Locke, E. A. (2006). Enhancing the benefits and overcoming the pitfalls of
goal setting. Organizational Dynamics, 35(4), 332–340.
doi:10.1016/j.orgdyn.2006.08.008
Lehmann, T., Hähnlein, I., & Ifenthaler, D. (2014). Cognitive, metacognitive and motivational
perspectives on preflection in self-regulated online learning. Computers in Human
Behavior, 32, 313–323.
Leire, C., McCormick, K., Richter, J. L., Arnfalk, P., & Rodhe, H. (2016). Online teaching going
massive: Input and outcomes. Journal of Cleaner Production, 123, 230–233.
Lewis, L. (2011). Organizational change: Creating change through strategic communication
(Vol. 4). West Sussex, UK: John Wiley & Sons.
Lieber, N. (2016, March 18). The selling of the American MBA. Bloomberg Business Week.
Retrieved from http://www.bloomberg.com/news/articles/2016-03-18/the-selling-of-the-
american-mba
Lorange, P., & Thomas, H. (2016). Pedagogical advances in business models at business schools
— in the age of networks. Journal of Management Development, 35(7), 889–900.
doi:10.1108/JMD-11-2014-0150
Markova, G., & Ford, C. (2011). Is money the panacea? Rewards for knowledge workers.
International Journal of Productivity and Performance Management, 60(8), 813–823.
Maxwell, J. A. (2013). Qualitative research design: An interactive approach. Thousand Oaks,
CA: Sage.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

172
Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson Education.
McBrien, J. L., Cheng, R., & Jones, P. (2009). Virtual spaces: Employing a synchronous online
classroom to facilitate student engagement in online learning. The International Review
of Research in Open and Distributed Learning, 10(3). doi:10.19173/irrodl.v10i3.605
McCrudden, M. T., Schraw, G., & Hartley, K. (2006). The effect of general relevance
instructions on shallow and deeper learning and reading time. The Journal of
Experimental Education, 74(4), 291–310.
McHenry, W. K. (2016). Online MBA programs and the threat of disruptive innovation. The
International Journal of Management Education, 14(3), 336–348.
doi:10.1016/j.ijme.2016.07.002
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation. San Francisco, CA: John Wiley & Sons.
Meyer, K. A. (2014). Student engagement in online learning: What works and why. ASHE
Higher Education Report, 40(6), 1–114.
Meyers, S. A. (2008). Using transformative pedagogy when teaching online. College Teaching,
56(4), 219–224.
Mezirow, J. (1991). Transformative dimensions of adult learning. San Francisco, CA: Jossey-
Bass.
Moore, J. L., Dickson-Deane, C., & Galyen, K. (2011). E-learning, online learning, and distance
learning environments: Are they the same? Internet and Higher Education, 14(2), 129–
135. doi:10.1016/j.iheduc.2010.10.001
Moore, M. G. (1993). Theory of transactional distance. Theoretical Principles of Distance
Education, 1, 22–38.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

173
National Survey of Student Engagement (NSSE). (n.d.). Construction of NSSE benchmarks.
Bloomington: Indiana University Center for Postsecondary Research. Retrieved from
http://nsse.indiana.edu/pdf/nsse_benchmarks.pdf
Pajares, F. (1997). Current directions in self-efficacy research. In M. Maehr & P. R. Pintrich
(Eds.), Advances in motivation and achievement (Vol. 10, pp. 1–49). Greenwich, CT: JAI
Press.
Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA:
Sage.
Pew, S. (2007). Andragogy and pedagogy as foundational theory for student motivation in higher
education. InSight: A Collection of Faculty Scholarship, 2, 14–25.
Pfeffer, J., & Fong, C. (2002). The end of business schools? Less success than meets the eye.
Academy of Management Learning & Education, 1(1), 78–95.
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95, 667–686.
doi:10.1037/0022-0663.95.4.667
Roobeek, A., & De Ritter, M. (2016). Rethinking business education for relevance in business
and society in times of disruptive change. Paper presented at the Academy of
Management (AoM) Annual Conference, Anaheim, CA, August 2016.
Rubin, H. J., & Rubin, I. S. (2012). Qualitative interviewing: The art of hearing data (3rd ed.).
Thousand Oaks, CA: Sage.
Rudestam, K. E., & Schoenholtz-Read, J. (2010). The flourishing of adult online education: An
overview. In K. E. Rudestam, & J. Schoenholtz-Read (Eds.), Handbook of online
learning (pp. 1–18). Thousand Oaks, CA: Sage.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

174
Rueda, R. (2011). The three dimensions of improving student performance. New York, NY:
Teachers College Press.
Rueda, L., Benitez, J., & Braojos, J. (2017). From traditional education technologies to student
satisfaction in management education: A theory of the role of social media applications.
Journal of Information and Management, 54(8), 1059–1071.
doi:10.1016/j.im.2017.06.002
Salkind, N. J. (2017). Statistics for people who (think they) hate statistics: Using Microsoft Excel
2016 (6th ed.). Thousand Oaks, CA: Sage.
Schein, E. H. (2017). Organizational culture and leadership. Hoboken, NJ: John Wiley & Sons.
Schneider, B., Brief, A. P., & Guzzo, R. A. (1996). Creating a climate and culture for sustainable
organizational change. Organizational Dynamics, 24(4), 7–19.
Schoemaker, P. J. H. (2008). The future challenges of business: Rethinking management
education. California Management Review, 50(3), 119–140.
Senge, P. (1990). The leader’s new work: Building learning organizations. Sloan Management
Review, 32(1), 7–23.
Starnes, B. J., Truhon, S. A., & McCarthy, V. (2010). Organizational trust: Employee-employer
relationships. A Primer on Organizational Trust. Milwaukee, WI: ASQ. Retrieved from
http://asqhdandl.org/uploads/3/4/6/3/34636479/trust.pdf
Suchman, M. C. (1995). Managing legitimacy: Strategic and institutional approaches. The
Academy of Management Review, 20(3), 571–610.
Thomas, H., Lee, M., Thomas, L., & Wilson, A. (2015). Securing the future of management
education. Bingley, UK: Emerald.
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

175
Thomas, H., Lorange, P., & Sheth, J. (2013). The business school in the twenty-first century.
Cambridge, UK: Cambridge University Press.
Turkay, S. (2014). Setting goals: Who, why, how? Cambridge, MA: Harvard University Office of
the Vice Provost for Advances in Learning.
Vaughan, N., & Garrison, D. R. (2006). How blended learning can support a faculty
development community of inquiry. Journal of Asynchronous Learning Networks, 10(4),
139–152.
Wheeler, D., & Sillanpää, M. (1998). Including the stakeholders: The business case. Long Range
Planning, 31(2), 201–210.
Wilson, D. C., & Thomas, H. (2012). The legitimacy of the business of business schools: What’s
the future? Journal of Management Development, 31(4), 368–376.
doi:10.1108/02621711211219040
Wisbauer, S. (2017). Shifts in learning: Will traditional universities survive? eLearning Industry.
Retrieved from https://elearningindustry.com/shifts-learning-will-traditional-universities-
survive
Wood, R., & Bandura, A. (1989). Social cognitive theory of organizational management.
Academy of Management Review, 14(3), 361–384.
World Economic Forum. (2016, January). The future of jobs: Employment, skills and workforce
strategy for the fourth industrial revolution. Geneva, Switzerland: Author. Retrieved
from http://reports.weforum.org/future-of-jobs-2016/
Yough, M., & Anderman, E. (2006). Goal orientation theory. Retrieved from
http://www.education.com/reference/article/goal-orientation-theory/
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

176
Yuan, J., & Kim, C. (2014). Guidelines for facilitating the development of learning communities
in online courses. Journal of Computer Assisted Learning, 30(3), 220–232.

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

177
APPENDIX A
INTERVIEW PROTOCOL — UFL ONLINE LEARNING TEAM
Beginning of meeting: “Thank you again for your time today. And, thank you for reviewing the
information sheet and signing the consent form. As a reminder, this interview is voluntary and
you have the right to decline to answer any question, or to end the interview, if you choose. In
addition, all of the information and responses will be consolidated and masked when I write my
dissertation so your identity will be protected. I’m hoping this will help you feel comfortable to
answer the questions openly and honestly. Lastly, I would like to record our discussion and take
just a few notes so I can listen closely to your responses and thoughts during our discussion, and
still capture all that you have to share. Once we finish our meeting, I will transcribe my notes
and the confidential recording and will then delete the recording. Do you mind if I record our
conversation? Thank you. Do you have any questions before we begin?”

“I will be focusing on several topics today around online learning and would like to start with the
end in mind – that is, the outcomes that business leaders are looking for in graduates today.”

1. (Knowledge: Conceptual) What do you think business leaders consider the 3-5 most
critical competencies when hiring or promoting students who complete the online
MBA?
2. (Knowledge: Procedural) What are you doing at the UFL to help develop these
competencies? Where do you know you need to make improvements?
3. (Organization: Cultural Setting) What recommendations do you have to help further
develop the online MBA to ensure students are graduating with the competencies they
need in their roles as future leaders in business?

“Thank you. Now, I’d like to talk about online learning and the Online Learning Team’s goal to
be ranked in the top five online MBA programs in the next year or two. I’ll focus specifically on
understanding your thoughts about how student engagement can help the department achieve
those rankings.”

4. (Knowledge: Procedural) What do you believe are the most effective methods in
online learning for building strong student engagement?
5. (Knowledge: Conceptual) How does this differ from the traditional on-campus
teaching?
6. (Organization: Cultural Model) When students are particularly engaged, what are the
conditions of engagement in those cases? When students are less engaged – what are
the conditions? What could be improved?
7. (Knowledge: Conceptual) Think about the most effective faculty. What makes them
particularly effective? What behaviors do they have that you wish all instructors had?
8. (Organization: Cultural Setting) What other recommendations do you have to help
improve overall student engagement in your program?

“Thank you. I’d like to ask you a few questions about your own personal experience in
developing and growing the online learning program at the UFL.”
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

178
9. (Knowledge: Conceptual) Do you know what the long and short-term goals are for
the UFL Online Learning Team and department?
10. (Motivation: Goal Orientation) Will you describe your experience in developing and
growing the online MBA program? What challenges have you faced so far? How
well do you think your organization has mitigated those challenges?
11. (Motivation: Self-Efficacy) Will you describe what has helped you in developing the
skills you need to achieve your goals? What skills did you need at the beginning?
Have those changed as the program has developed? Has anything hindered you?

“Thank you. Lastly, I’d like to discuss your experience in the organization and how it might
grow in the future.”

12. (Motivation: Goal Orientation) What is your vision for the future of this online
learning program and how will it serve the needs of business leaders? What would
you like to see it do next to serve the business community?
13. (Organization: Cultural Model) What organizational factors (Department and/or the
overall Business School) have helped your department achieve its goals? Which
factors have hindered your ability to achieve your goals?
14. (Organization: Cultural Setting) What do you think others in the university outside
the Online Learning Department think about or perceive online learning? What kind
of support do you receive?
15. (Organization: Cultural Model) What do you believe are the most important
opportunities to help grow the program? What are your recommendations for how the
Dean;s Cabinet and others in the organization can support you better?

“Thank you for your perspectives. Is there anything else you would like to add to our
discussion?”

“Thank you again and as a reminder, your responses will be kept confidential and your identity
will be protected. I very much appreciate your time.”

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

179
APPENDIX B
INTERVIEW PROTOCOL — DEAN’S CABINET
(SUPPLEMENTAL STAKEHOLDER GROUP)
Beginning of meeting: “Thank you again for your time today. And, thank you for reviewing the
information sheet and signing the consent form. As a reminder, this interview is voluntary and
you have the right to decline to answer any question, or to end the interview, if you choose. In
addition, all of the information and responses will be consolidated and masked when I write my
dissertation so your identity will be protected. I’m hoping this will help you feel comfortable to
answer the questions openly and honestly. Lastly, I would like to record our discussion and take
just a few notes so I can listen closely to your responses and thoughts during our discussion, and
still capture all that you have to share. Once we finish our meeting, I will transcribe my notes
and the confidential recording and will then delete the recording. Do you mind if I record our
conversation? Thank you. Do you have any questions before we begin?”

“I will be focusing on several topics today around online learning and would like to start with the
end in mind – that is, the outcomes that business leaders are looking for in graduates today.”

1. (Knowledge: Conceptual) What do you think business leaders consider the 3-5 most
critical competencies when hiring or promoting students who complete the online
MBA?
2. (Knowledge: Procedural) What is the UFL Online Learning Team (OLT) doing to
help develop these competencies? Where do you think the OLT needs to make
improvements, if any?
3. (Organization: Cultural Setting) What recommendations do you have to help further
develop the online MBA to ensure students are graduating with the competencies they
need in their roles as future leaders in business?

“Thank you. Now, I’d like to talk about online learning and the Online Learning Team’s goal to
be ranked in the top five online MBA programs in the next year or two. Since student
engagement is one of the most important factors in these rankings, I’ll focus specifically on
understanding your thoughts about how student engagement can help the department achieve
these rankings.”

4. (Knowledge: Procedural) What do you believe are the most effective methods in
online learning for building strong student engagement?
5. (Knowledge: Conceptual) How does this differ from the traditional on-campus
teaching?
6. (Organization: Cultural Model) When students are particularly engaged, what are the
conditions of engagement in those cases? When students are less engaged – what are
the conditions? What could be improved?
7. (Knowledge: Conceptual) Think about the most effective faculty. What makes them
particularly effective? What behaviors do they have that you wish all instructors had?
ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

180
8. (Organization: Cultural Setting) What other recommendations do you have to help
improve overall student engagement in the Online MBA program?

“Thank you. I’d like to ask you a few questions about your own personal experience and
perspectives about developing and growing the online learning programs at the UFL.”

9. (Knowledge: Conceptual) Do you know what the long and short-term goals are for
the online learning at the UFL School of Business?
10. (Motivation: Goal Orientation) Will you describe your experience in developing and
growing the online MBA program? What challenges has the organization faced so
far? How well do you think the organization has mitigated those challenges?
11. (Motivation: Self-Efficacy) Will you describe what skills are needed by the OLT to
achieve their goals? What skills were needed at the beginning? Have those changed
as the program has developed? Has anything hindered the team?

“Thank you. Lastly, I’d like to discuss the organization and how it might grow online learning in
the future.”

12. (Motivation: Goal Orientation) What is your vision for the future of this online
learning program and how will it serve the needs of business leaders? What would
you like to see the program do next to serve the business community?
13. (Organization: Cultural Model) What organizational factors (Department and/or the
overall Business School) have helped the department achieve its goals? Which
factors have hindered the organization’s ability to achieve its goals?
14. (Organization: Cultural Setting) What do you think others in the university outside
the Online Learning Department think about online learning? What kind of support
do you think they receive?
15. (Organization: Cultural Model) What do you believe are the most important
opportunities to help grow the program? What are your recommendations for how the
Dean;s Cabinet and others in the organization can support them better?

“Thank you for your perspectives. Is there anything else you would like to add to our
discussion?”

“Thank you again and as a reminder, your responses will be kept confidential and your identity
will be protected. I very much appreciate your time.”

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

181
APPENDIX C
MEMORANDUM OF UNDERSTANDING (SAMPLE)
UFL School of Business is committed to creating high-quality online courses. All Business
School online courses follow a “flipped” classroom model. Working in conjunction with our
instructional design (ID) team, faculty members act as subject matter experts (SME) in the
design, development, delivery, and evaluation of course content in the online space. This
document outlines the commitment and compensation for faculty who create online courses.

Faculty Commitment
Faculty must spend a minimum of six hours on average per week beginning six months prior to
course launch in order to design and develop a 3-unit online course. The following deliverables
are required of faculty members in order to receive full compensation for the design, development,
delivery, and evaluation of their online course:

Course design/development
 Work with our ID team to develop original online course content
 Work with our video production staff to pre-record all video content at our DTLA studio
 Create learning outcomes, activities, and assessments as well as scripts for videos
 Undergo faculty training in course platforms and online delivery methods
 Participate in Quality Matters (QM) quality review process

Course delivery/evaluation
 Conduct online course during the assigned semester
 Submit course evaluation/feedback narrative upon completion of the online course

Compensation for Online Course Creation
Compensation for a 3-unit course will be $X total, to be distributed as follows:
 50% to be paid upon completion of course design/development process
 50% to be paid upon submission of post-delivery evaluation narrative

The intellectual property of the course resides with UFL. If an instructor discontinues teaching the
online course, the University has the right to continue to use the course content and/or to re-develop
the course as needed. For more information regarding UFL’s policies on intellectual property
policy, refer to the “Courseware” and “Intellectual Property” sections at (website).

Upon your agreement to this commitment, an instructional designer will be assigned to your course
to assist you with the online course design and development process. If necessary, an overload
form will be created by the Online Learning office.

Name: ___________________________________ Date: ______________________________

Signature: ________________________________ Course: ____________________________

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

182
APPENDIX D
STRATEGIC PLANNING PROCESS EVALUATION AND ASSESSMENT — POST-
MEETING SURVEY
The purpose of this survey is to evaluate the strategic planning process that was implemented
with the Online Learning Team. We appreciate your feedback and guidance on the process so
that we may understand the quality of the planning process as we as make improvements for
future sessions. Thank you in advance for taking some time to share your thoughts and ideas on
how we might improve the participant experience in the future.

Level 1 — Engagement
Strongly
Disagree
Strongly
Agree
1) I took responsibility for being fully involved during this
strategic planning session.
❍ 1 ❍ 2 ❍ 3 ❍ 4
2) I was engaged with what was going on during the session.
❍ 1 ❍ 2 ❍ 3 ❍ 4
Level 1 — Relevance
❍ 1 ❍ 2 ❍ 3 ❍ 4
3) The discussion and materials from this session will be
helpful for my future success.
❍ 1 ❍ 2 ❍ 3 ❍ 4
4) I am clear about what is expected of me when I get back
to my job as a result of participating in this session.
❍ 1 ❍ 2 ❍ 3 ❍ 4
Level 1 — Customer Satisfaction
5) Participating in this planning session was worth my time
❍ 1 ❍ 2 ❍ 3 ❍ 4
Level 2 — Knowledge
6) I clearly understand the goals of the Online Learning
Organization.
❍ 1 ❍ 2 ❍ 3 ❍ 4
7) I clearly understand how achieving my goals help both
my organization and the overall business growth in the
future.
❍ 1 ❍ 2 ❍ 3 ❍ 4
Level 2 — Attitude
8) I believe that this strategic plan is important to
succeeding in the job.
❍ 1 ❍ 2 ❍ 3 ❍ 4
Level 2 — Confidence
9) I feel confident about applying what I learned back on my
job.
❍ 1 ❍ 2 ❍ 3 ❍ 4
Level 2 — Commitment
10) I am committed to applying what I learned to my work.
❍ 1 ❍ 2 ❍ 3 ❍ 4

ONLINE GRADUATE-LEVEL STUDENT LEARNING & ENGAGEMENT

183
APPENDIX E
STRATEGIC PLANNING PROCESS EVALUATION AND ASSESSMENT – FOLLOW-UP
SURVEY
The purpose of this survey is to evaluate the strategic planning process that was implemented
with the Online Learning Team. We appreciate your feedback and guidance on the process so
that we may understand the quality of the planning process as we as make improvements for
future sessions. Thank you in advance for taking some time to share your thoughts and ideas on
how we might improve the participant experience in the future.

L1: Reaction
Strongly
Disagree
Strongly
Agree
1) What I learned in the strategic planning session has
been useful in focusing my work.
❍ 1 ❍ 2 ❍ 3 ❍ 4
2) The timing of the planning session was appropriate
for me.
❍ 1 ❍ 2 ❍ 3 ❍ 4
L2: Learning
3) Looking back on the planning, it is clear why it was
important for me to attend.
❍ 1 ❍ 2 ❍ 3 ❍ 4
L3: Behavior
4) Since the session, I have successfully applied on the
job what I learned during the planning session.
❍ 1 ❍ 2 ❍ 3 ❍ 4
L4: Results
5) I have been successful in moving forward to achieve
my goals.
❍ 1 ❍ 2 ❍ 3 ❍ 4

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Leadership competencies in healthcare administration graduate degree programs: an evaluative study

PDF

Student engagement in online education: an evaluation study

PDF

Health care disparities and the influence of nurse leader cultural competency: an evaluation study

PDF

Developing socially intelligent leaders through field education: an evaluation study of behavioral competency education methods

PDF

Completion in online learning: graduate students' perspectives

PDF

Employment rates upon MBA graduation: An evaluation study

PDF

Creating effective online educational content: an evaluation study of the influences affecting course developers’ abilities to create online content with engaging learning strategies

PDF

The importance of rigor and engagement on student achievement and mastery: an evaluation study

PDF

Building leaders: the role of core faculty in student leadership development in an undergraduate business school

PDF

Leadership and the impact on organizational citizenship behaviors: an evaluation study

PDF

Cultivating culturally competent educators

PDF

Equitable schooling for African American students: an evaluation study

PDF

Efficacy of non-formal education programs in educational outcomes of marginalized Filipino children: an evaluation study

PDF

Modern corporate learning requires a modern design methodology: an innovation study

PDF

Line staff and their influence on youth in expanded learning programs: an evaluation model

PDF

Developing qualified workers to support individuals with intellectual and developmental disabilities in home and community settings: an evaluation study

PDF

Maximizing leadership development in the U.S. Army: an evaluation study

PDF

Sustaining faith-based organizations through leadership pipelines and programs: an evaluation study

PDF

Quality literacy instruction in juvenile court schools: an evaluation study

PDF

Facilitation of postsecondary opportunities for secondary students with special needs: an evaluation study

Asset Metadata

Creator McCloud, Cynthia J. (author)

Core Title Online graduate-level student learning and engagement: developing critical competencies for future leadership roles: an evaluation study

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Publication Date 01/14/2019

Defense Date 11/09/2018

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag critical competencies,OAI-PMH Harvest,online learning,student engagement

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Datta, Monique (committee chair), Bhambri, Arvind (committee member), Green, Alan (committee member)

Creator Email cmccloud@usc.edu,cynthia.mccloud@outlook.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-115520

Unique identifier UC11675617

Identifier etd-McCloudCyn-7022.pdf (filename),usctheses-c89-115520 (legacy record id)

Legacy Identifier etd-McCloudCyn-7022.pdf

Dmrecord 115520

Document Type Dissertation

Format application/pdf (imt)

Rights McCloud, Cynthia J.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA