University of Southern California Dissertations and Theses

Understanding student persistence in massive open online courses (MOOCs): an evaluation study

(USC Thesis Other)

Understanding student persistence in massive open online courses (MOOCs): an evaluation study

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 1
UNDERSTANDING STUDENT PERSISTENCE IN MASSIVE OPEN ONLINE COURSES
(MOOCS): AN EVALUATION STUDY

by
Estella Y. Chen

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

June 2017

Copyright 2017 Estella Y. Chen

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 2
Acknowledgments
Time passes very fast. I can’t believe that I graduated twice from USC Rossier! I started the
program simultaneously with my first-time entrepreneurial initiative nearly two years ago. It has
really been a life-changing decision. Many times, I had serious doubts about my path moving
forward. Whenever I thought I couldn’t go any further, my determination to see it through and
plain stubbornness fueled me again and again. I truly appreciate the Rossier faculty members for
their involvement and assistance with my dissertation; especially Dean Gallagher’s time and
feedback, and Professor Gallagher’s insight and editing. Special thanks to my chair, Dr. Mark
Robison, for his patience and clear guidance as always. This has been the absolutely perfect
combination of the right timing, program, and people to work with.
Grandma, I know you can still see me from Heaven and I know you are, and always be, proud of
me!

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 3
Table of Contents
Acknowledgments 2

List of Tables 6

List of Figures 8

Abstract 9

Chapter One: Introduction 10

Background of the Problem 11

Importance of Addressing the Problem 12

Organizational Context and Mission 14

Organizational Performance Status 14

Organizational Performance Goal 15

Description of Stakeholder Groups 15

Stakeholder Groups’ Performance Goals 17

Stakeholder Group for the Study 17

Purpose of the Project and Questions 18

Conceptual and Methodological Framework 18

Definitions 19

Organization of the Study 20

Chapter Two: Review of the Literature 21

MOOCs as a Prominent Trend in Higher Education 21

No Brick-and-Mortar 21

Technologies Used in MOOCs 22

Credit Transfer from MOOCs 23

Instructional Design for High-Quality MOOCs 25

Video Lectures for MOOCs 25

Pedagogical Strategies for Online Courses 26

Structured Course Content 27

Factors Impacting MOOCs’ Completion Rates 27

Student Characteristics 27

Student Sentiment 28

Teaching Presence 29

Interaction Through Social Networking 30

MOOC Participants’ Knowledge, Motivation, and Organization Influences 31

Knowledge and Skills 31

Motivation 34

Organization 36

Chapter Three: Methodology 38

Purpose of the Project and Questions 38

Methodological Framework 38

Assumed Performance-Based Influences 40

Preliminary Scanning Data 41

Participating Stakeholders 43

Data Collection 46

Surveys 47

Validation of the Performance Needs 47

Trustworthiness of Data 48

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 4
Role of Investigator 48

Data Analysis 49

Limitations and Delimitations 49

Limitations 49

Delimitations 50

Chapter Four: Results 51

Participating Stakeholders 51

Data Collection and Validity 52

Results and Findings for Knowledge Causes 54

Adequate Knowledge for Completing Courses 55

Unfamiliarity with Differences in Mandarin Chinese 58

Self-Awareness and Adaptability 59

Results and Findings for Motivation Causes 62

Participants Mastery of Learning Mandarin 63

Perceived Value of Taking Course 67

High Self-Efficacy Perceptions in High Completion Rates 73

Results and Findings for Organizational Causes 77

Pleased Participants 78

Role Models Within the Organization 80

Social Networking Helps Enhance Engagement 82

Flexibility in Online Learning Courses 86

Summary of Validated Causes 89

Chapter Five: Findings, Implementation, and Evaluation 91

Validated Causes Selection and Rationale 91

Findings for Knowledge Causes 93

Factual 94

Procedural 95

Metacognitive 96

Findings for Motivation Causes 96

Intrinsic Interest in the Subject 98

Extrinsic Benefit of the Course 99

Positive Attitudes Toward MOOCs 99

Findings for Organization Causes 99

Cultural Setting 100

Cultural Model 101

Implementation Plan 101

Stage 1: Preparation 103

Stage 2: Execution 103

Stage 3: Implementation 104

Organizational Environment and Features Relevant to Implementation 105

Related Solutions and Organizational Capacity to Implement 106

Evaluation Plan 110

Level 1: Reactions 110

Level 2: Learning 112

Level 3: Transfer 113

Level 4: Impact 114

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 5
Limitations 115

Conclusions 116

References 118

Appendix: Survey Protocol 145

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 6
List of Tables
Table 1: Organizational Mission, Goal, and Stakeholder Performance Goal 17

Table 2: Conceptual Framework for Addressing the Inquiry Questions 48

Table 3: Demographic Information of the Respondents 53

Table 4: Validated Assumed Knowledge Causes 55

Table 5: ANOVA Analysis for All of the Assumed Causes under Knowledge, Motivation, and
Organization Categories 58

Table 6: Validated Assumed Motivation Causes 62

Table 7: ANOVA Analysis for the Factor “Primary Goal and Expectation of this Course” in
Knowledge, Motivation, and Organization Assumed Causes 66

Table 8: ANOVA Analysis for the Factor “Well-structured with Adequate Exercises” in
Knowledge, Motivation, and Organization Assumed Causes 68

Table 9: The Crosstabulation Table (Intrinsic Motivation* Relevance to Individual Differences
Crosstabulation) 71

Table 10: Comparison of Higher Intrinsic Motivation and Lower Intrinsic Motivation
Participants’ Perceptions Choosing this MOOC 71

Table 11: The Chi-Square Tests Table 72

Table 12: Key Components Producing High-quality MOOCs and Relevance to Individual
Differences Core Value of a High-quality MOOCs Relevance to Individual Differences 73

Table 13: Validated Assumed Organization Causes 77

Table 14: Pearson Product-moment Correlation Analysis for the Factor “Overall Satisfaction
Level of the Course Design” in Knowledge, Motivation, and Organization Assumed Causes 80

Table 15: Three Reasons of MOOC Participants’ Perspectives Towards Social Networking and
Their Course Engagement 84

Table 16: Sample Descriptives Using T-test for the Factor “Teaching Presence” in “Study
Alone” Participants Group 85

Table 17: ANOVA Analysis for the Factor “Hours Spend on Average with this Course Every
Week” in Knowledge, Motivation and Organization Assumed Causes 88

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 7
Table 18: Validated Causes Summary 92

Table 19: Validated Causes under Knowledge Category and Findings Summary 94

Table 20: Validated Causes under Motivation Category and Findings Summary 98

Table 21: Validated Causes under Organizational Category and Findings Summary 100

Table 22: Summary Implementation Plan for Two Proposed Solutions: Create Strong MOOCs
with Instructional Design Strategies at MOOC Community and for University Coursework
Framework 108

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 8
List of Figures
Figure 1: The gap analysis process model. 39

Figure 2: Enrollments in MandarinX’s Current MOOCs 44

Figure 3: Age Distribution of the Enrollment in MandarinX’s Current MOOCs 45

Figure 4: Educational Backgrounds of the Enrollment in MandarinX’s Current MOOCs 45

Figure 5: Survey Participants Reasons of Failing to Submit an Assignment 57

Figure 6: Survey participants knowledge of differentiating language usages in mandarin-
speaking regions. 59

Figure 7: Survey participants confidence in seeking additional resources to help with learning
Mandarin. 61

Figure 8: Survey participants reasons of leaning Mandarin Chinese 64

Figure 9: Interviewed participants reasons for leaning Mandarin Chinese. 65

Figure 10: Survey participants perceptions towards solid course design with encourage online
learning environment and supportive feedback. 67

Figure 11: Survey participants reasons choosing this Mandarin MOOC 70

Figure 12: Survey participants perceptions, metacognition and self-efficacy. 74

Figure 13: Survey participants academic emotions are closely related to learning strategies and
academic performances. 76

Figure 14: Participants found the most engaging part while taking the course. 79

Figure 15: Survey teaching presence influences participants perceptions of the course. 81

Figure 16: Survey participants use of social networking events and opportunities to interact with
diverse groups of learners 83

Figure 17: Survey participants learning habits while taking this course. 84

Figure 18: Survey participants time spent on average with this course every week. 87

Figure 19: MOOC Scheme 103

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 9
Abstract
The use of Massive Open Online Courses (MOOCs) has been viewed as a transformational
approach to increase educational opportunities to a global audience. However, low persistence
rates have been highlighted as a major criticism. Evidence shows that only a small percentage of
MOOC participants complete their courses and little is understood about the specific MOOC
design and implementation factors that influence retention. This study reports a survey of 696
participants who were enrolled in Mandarin Chinese language MOOCs. Clark and Estes gap
analysis was utilized to investigate the knowledge, motivation, and organizational assumed
causes of persistence in MOOCs. Findings demonstrated that teaching presence has a significant
impact on MOOC participants’ perspectives and the success of the course itself. The design of
MOOC course content was also found to be a significant predictor of MOOC participants’
persistence. Understanding what institutional, course, and student characteristics are related to
student success in this relatively new educational modality is important to increasing student
retention and success.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 10
CHAPTER ONE: INTRODUCTION
Massive Open Online Courses (MOOCs) have drawn a great deal of attention since 2008
when the first MOOC appeared on the world scene (James, 2017; Loizzo & Ertmer, 2016;
Milligan & Littlejohn, 2017; Spector, 2017; Stich & Reeves, 2017). The rapid rise of MOOCs,
however, challenges the status quo regarding traditional face-to-face instructional models (Hone
& El Said, 2016). MOOCs have shown to be a disruptive innovation in that people who can
access the Internet can access education anytime, anywhere (Lee, 2017; Davis, Jivet, Kizilcec,
Chen, Hauff, & Houben, 2017; Simpson, 2017). MOOCs have quickly become popular with
more than 1.71 million course entries being provided through three dominant platforms: Udacity,
Coursera, and edX for 1.03 million participants within the first two years of launch (Deng,
Benckendorff, & Gannaway, 2017; Dennis, 2017). However, a 95% attrition rate suggests that
MOOC participants are generally more likely to enroll in courses without finishing them than to
pursue their completion (Ho, Chuang et al., 2015).
The focus of the study was to examine the factors contributing to a high rate of retention
in a MOOC, as the participants’ engagement is crucial to their success in online education. This
is due to participants being expected to be more self-disciplined, self-directed, and self-motivated
than students in traditional brick-and-mortar venues. Furthermore, the study attempted to
understand the circumstances that contribute to this MOOC’s success in Mandarin language
learning, and to evaluate the factors that contributed to the MOOCs high retention rate. In doing
so, this study examined the knowledge, motivation, and organizational influences that underpin
MandarinX’s high completion rate. It also identified ways to extend this success into future
MOOC designs so as to advance the MOOC community.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 11
Background of the Problem
There have been discussions among researchers, educators, and practitioners about
whether MOOCs may be effective and offer a high enough quality program of study to replace
some of the coursework offered at universities, or if MOOCs can be appropriately incorporated
into flipped classrooms or blended learning (Thai, De Wever, &Valcke, 2017; Huang, Lee,
&Dugan, 2017). There has been prolific research on factors that lead to an effective MOOC
from participants’ perspective (Alario-Hoyos, Estévez-Ayres, Pérez-Sanagustín, Kloos,
&Fernández-Panadero, 2017; Austin, 2016; Bonk & Lee, 2017; Cunningham, 2017; Davis et al.,
2017; Jiao, Yang, Zhong, & Ren, 2017; Nagrecha, Dillon, & Chawla, 2017; Salmon, Pechenkina,
Chase, & Ross, 2016; Spector, 2017). Studies have identified factors that have an impact on
students’ success in MOOCs. The factors include their technology, pedagogy, content, usability,
assessment, collaboration, and interactivity. The degree of networking opportunities was pointed
out to be the most important dimension among the factors identified above. Social networking
helps enhance the engagement of online courses. In addition, students have indicated that they
value relationships built during their online courses, in which they can practice and seek help
from peers (Gamage et al., 2015; Jiao et al., 2017). Although instructors’ roles within MOOCs
may differ in varied institutional contexts where they can perform communication skills face-to-
face, research suggested that MOOCs with more networking opportunities, where participants
can gain more engagement through participation, are most effective (Cruz-Benito, Borrás-Gené,
García-Peñalvo, Blanco, & Therón, 2017; Ross, Sinclair, Knox, Bayne, & Macleod, 2014).
Some of the weakest aspects of MOOCs shown by Bali (2014) in surveys and
observations of four MOOCs on Coursera were that participants tended to take a course as a
hobby, rather than as a learning experience equivalent to a face-to-face course. Other
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 12
weaknesses mentioned include participants feeling that they were lost or neglected if the
instructor or the course team did not provide timely feedback due to MOOCs’ large numbers of
students.
Furthermore, students sometimes have different interests than the instructor’s focus.
Students may also have other goals that conflict with those of instructors or even themselves
(Ambrose, Bridges, Lovett, DiPietro, & Norman, 2010). Given that some MOOC courses have
higher completion rates than others, there is a need for best practices that enhance learner
persistence in the MOOC learning experience. The identification of these factors may then drive
more MOOC participants to successful completion.
Importance of Addressing the Problem
MOOCs are the latest online learning initiative to attain widespread popularity in many
countries. Despite the critiques of low completion rates, MOOCs have yielded transformational
opportunities of teaching, learning, and researching. Due to the diversity in education, MOOCs
serve a unique opportunity to implement and explore new pedagogy (Daniel, 2012; Ko &
Rossen, 2017). MOOCs can also be a great environment not only to develop and test new online
teaching pedagogies, but also to understand diversity and better support diversity in education
(El-Hmoudova, 2014). It is imperative that MOOC pedagogy be explored by educators and
educational institutions, since video lectures are utilized as the main channel for delivering
knowledge and course content (Ko & Rossen, 2017). Studying MOOCs can help decipher how
millennials perceive education and learning (Dalipi, Yayilgan, Imran & Kastrati, 2016).
In addition, MOOCs bring benefits to society, and their massive database of learner
analytics helps understand the impact of this revolutionary online learning system on everyday
users, as well as on employee training in corporations (Breslow et al., 2013; Daniel, 2012;
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 13
Dellarocas & Van Alstyne, 2013; Martin, 2012). A recent survey revealed that 22% out of 525
business organizations use MOOCs as part of their learning and development program (Reiser,
2017). Although the enterprises have not yet developed their own MOOCs, they often encourage
employees to enroll in MOOCs to gain skills and knowledge that will enhance individual
performance (Head, 2017).
Alraimi, Zo, and Ciganek (2015) explained that two significant indicators influencing a
substantial percentage of the intention to continue using MOOCs are the reputation of an
institution and its openness. Participant intention is an important concern because most MOOC
completion rates are substantially lower than those found in traditional online education courses
(Hew & Cheung, 2014). Only learners’ own goals drive learning in MOOCs (Masters, 2011).
The primary goal of this study was to analyze and understand why a particular MOOC provider,
MandarinX, has high retention rates. The quality of MandarinX MOOCs education and an
assessment of student work upon completion of a MandarinX course have yet to be fully
discovered or resolved.
Understanding why students persist or not could also help optimize educational resources
and save on the cost of logistics (Engle, Mankoff, & Carbrey, 2015). Wang and Baker (2015)
indicated that many students soon, however, saw that this would not work if they took
department-based courses, because the workload in their major was heavy, and there were
conflicts in their schedules. MOOCs can provide time flexibility in a university course
framework. For example, students enrolled in MOOCs can manage their time by taking courses
which offer flexibility. Universities have also been encouraging their students to pursue other
fields of expertise besides their majors in order to promote interdisciplinary interests. Some
flagship Western universities such as Harvard and Stanford University have been advocating the
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 14
use of MOOCs as a means of enhancing teaching and learning efficiency in higher education
(Lombardi, 2013).
Organizational Context and Mission
MandarinX is an organization dedicated to providing a premier Chinese language
learning experience online through partnership with edX, a MOOC provider founded by Harvard
University and Massachusetts Institute of Technology. As of December 2016, MandarinX has
experienced high demand with an enrollment of 133,281 learners from 202 countries across four
in-house designed and developed MOOC courses. They are Basic Mandarin Series: Level One,
Two, Three, and Mandarin Chinese for Business. MandarinX’s mission is to do the following:
(1) provide an active, online language learning environment that is conducive to language
acquisition through strong learn-learner and learn-instructor interactions; and (2) bridge the gap
between cultures by introducing different facets of Chinese life, with corresponding materials.
To create MandarinX courses, an interdisciplinary team of professionals was formed
from fields of Chinese linguistics and teaching Chinese as a second language and instructional
design and media from top universities: Beijing University, Georgetown University, Indiana
University, National Taiwan Normal University, Stanford University, and Teachers College at
Columbia University. MandarinX courses are also reviewed with the assistance of faculty at the
Massachusetts Institute of Technology Chinese Language Program.
Organizational Performance Status
The course completion rate of MandarinX learners averages 86% for each of the six six-
week courses. The completion rate is significantly higher than the typical MOOC average
completion rate which ranges from 5% to 10% (Davis et al., 2017; Höfler, Zimmermann, &
Ebner, 2017). The high completion percentage phenomenon is even more notable considering
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 15
that 55.8% of MOOC participants access less than half of their course material (Nagrecha et al.,
2017). MandarinX aims to offer a six-level basic Mandarin MOOC series targeted at learners
with no Chinese learning background and a business-oriented Chinese MOOC series caters to
business travelers who need knowledge of culture and tradition to sharpen negotiation and
communication skills while being exposed to Chinese business markets. The latter course, with
its business slant, is a pioneering language MOOC focused on practical use, in contrast to
business-oriented classes at institutions such as the University of Pennsylvania and MIT which
tend to be conducted on campus rather than virtually.
Organizational Performance Goal
The organizational goal is that, by September 2018, MandarinX will have developed six
levels of basic Mandarin MOOCs and two business-oriented Chinese MOOCs, which ultimately
can be implemented to expand the scope of university course framework. These combined
efforts will triple the current enrollment to more than 500,000. The vision for MandarinX is to
offer a more complete Mandarin-learning system by cooperating with some of the most
renowned universities in North America, East Asia, and Europe. Aside from these objectives,
one of the organizational performance goals is to maintain and increase the completion rate
among MOOCs. In order to have a solid understanding of what has worked for the current
courses, this study examined students’ experiences further so the organization can repeat or
identify ways in which the completion rates can be improved and achieve the organizational
goal.
Description of Stakeholder Groups
There are several stakeholder groups that are critical to a MOOC’s success. Online
education revolves around convenience, which is what attracts people (Lee, 2017). However,
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 16
well-structured courses with well-produced videos supported by the instructor and teaching
assistants are fundamental elements of successful MOOCs (James, 2017; Ko & Rossen, 2017;
Simpson, 2017). The instructor who conducts the course encourages learners to discuss the
lessons in the discussion board and constantly interacts with participants (Garg & Paepcke, 2017;
Jiao et al., 2017). This instructor involvement has been demonstrated to be a key factor in
maintaining a high completion rate (Poquet, Dawson, & Dowell, 2017). Therefore, the four
stakeholder groups examined in this study were students, course development teams, teaching
assistants, and the instructor(s).
Students enrolled in a MandarinX six-week language course are expected to complete all
the online video courses and quizzes and successfully complete the final exam. The course and
its requirements are completed at each student’s individual pace. Students might arrange
themselves differently. For example, students may voluntarily form their own online study
groups across regions, countries, or time zones for practicing pronunciation of vocabulary.
Course development teams design the course by covering the fundamental knowledge needed to
reach course outcomes and produce high-quality instructional materials. A successful MOOC
experience requires a high degree of teamwork and collaboration among all individual
stakeholders.
Teaching assistants have the knowledge required for a given course and help facilitate
online forums and discussions. They take three shifts in a 24-hour period to attend to students in
different time zones. How they respond to student questions and inquiries can boost
participation.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 17
Stakeholder Groups’ Performance Goals
The following are the stakeholder groups’ performance goals (each group functions
differently when setting up a successful MOOC).
Table 1
Organizational Mission, Goal, and Stakeholder Performance Goal
Organizational Mission
MandarinX aims to offer a participatory online language learning environment that
is conducive to language acquisition.
Organizational Performance Goal
By September 2018, MandarinX will have endeavored to provide six levels of basic
Mandarin MOOC’s (accredited as university Chinese course credits) as well as two
business-oriented editions and will be collaborating with distinguished universities
from North America, East Asia, and Europe.
Students/
MOOC Participants
Course Development
Team

Teaching Assistants
By September 2018,
students will complete
every lesson, actively
respond to other students’
questions, and provide the
course development team
with feedback/information
about their individual
needs.
By September 2018,
course development team
will keep producing
highly rated videos with
scaffolded content and
online interactive
exercises/quizzes.
Courses will be
implemented to expand
the scope of university
framework.
By December 2018,
teaching assistants will
administer one-on-one
tutoring sessions while
integrating pronunciation
skills into the pre-recorded
video lessons.

Stakeholder Group for the Study
Generally, MOOCs have been criticized for low completion rates over the past few years
despite high enrollment (Austin, 2016; Spector, 2017). Considering the fact that enrollees are
from diverse cultural backgrounds, motivation may vary accordingly. Therefore, the key
stakeholder group were students/MOOC participants. Their persistence in completing a six-
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 18
week language MOOC is important to study different perspectives to discover the underlying
aspects of the MOOC that have worked in producing high completion rates.
Purpose of the Project and Questions
The purpose of this project was to evaluate the features that have enabled the six-week
MOOCs provided by MandarinX to achieve a completion rate of 86% compared to the 5% to
10% average for typical MOOCs. The analysis focuses on student knowledge, motivation, and
organizational influences related to achieving this organizational goal. For practical purposes,
the stakeholders which this analysis examines are participants.
The research questions for this study are
1. What are the knowledge, motivation, and organizational influences that underpin
MandarinX students’ high completion rate?
2. What knowledge, motivation, and organizational assets would extend this success into
future MOOC design?
Conceptual and Methodological Framework
Clark and Estes’ (2008) gap analysis utilized evidence-based research in the development
of organizational solutions by identifying performance gaps in organizations and was adapted for
evaluation analysis as this study’s conceptual framework. The methodological framework was a
quantitative case study with statistics and qualitative reported quotes. Assumed knowledge,
motivation, and organizational causes were generated based on personal knowledge and related
literature. These evaluations were validated using surveys, interviews, observations, and
document review. Research-based solutions were recommended for future implementation.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 19
Definitions
• MOOCs: Massive Open Online Courses. Fully online courses scaled to enable an
unlimited number of student registrants. Faculty members both design and lead courses.
This replaces the master design concept and leverages the natural scaling power of online
tools.
• cMOOCs: Principals of connectivist/networked-based learning. Knowledge creation,
content, context, connections (open learning and online network practices). Emphasis on
creation, creativity, autonomy, and social network based learning. Strong focus on online
discussion. Instructor-led.
• xMOOCs: Behaviorist pedagogy. Knowledge duplication, information transmission,
computer market assignments, and peer assessments. Emphasis on lecture video and
multiple-choice tests (video-taped lectures appear online). Instructor facilitated.
• Completion Rate: A percentage which reflects the total number of initial enrollees
divided by the number of students who complete the course.
• Retention: Students who persist in completion of their educational goals
• Blended/Hybrid/Flipped: Generally applied to the practice of using both content and
instruction via digital and online media and in-person learning experiences. Combines
face-to-face with online in a structured format. Objective is to maximize use of face-to-
face time. Students prepare for the class using online tools. The instructor then uses
class time to facilitate class participation and discussion.
• Coursera: Founded by Stanford computer science professors Andrew Ng and Daphne
Koller, with 62 university partners, including Brown, Duke, Princeton, Columbia, and
Stanford, and 2.8M enrollments. Courses are provided in four languages. One of the
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 20
main MOOC platform providers (Coursera, 2017). Course structure consists of fixed
terms, with automated assessment, lectures, and quizzes.
• edX: Founded by MIT and Harvard, added UC Berkeley and seeking additional partners;
122K students in pilot course, one of the main MOOC platform providers. Course
consists of fixed terms, automated assessment, Pearson testing centers. (edX Insights,
2016)
• Udacity: Founded by Sebastian Thrun from Stanford. Focuses on STEM and industry;
160K students in pilot course; one of the main MOOC platform providers (Udacity, Inc.,
2017). Course structure is self-paced, automated assessment, Pearson testing centers.
Organization of the Study
Five chapters are used to organize this study. This chapter provided the reader with key
concepts and terminology commonly found in a discussion about MOOC pedagogy and how to
retain learners’ attention. The organization’s mission, goals, and stakeholders, as well as the
initial concepts of gap analysis adapted to needs analysis, were introduced. Chapter Two
includes a review of current literature surrounding the scope of the study. Topics of completion
rate, MOOC pedagogy, social media, and preparation for faculty to teach MOOCs are addressed.
Chapter Three is a description of the assumed needs for this study as well as methodology,
comprising of participant choice, data collection, and analysis. In Chapter Four, the data and
results are assessed and analyzed. Chapter Five provides solutions based on data and literature
for addressing the needs and closing the performance gap as well as recommendations for an
implementation and evaluation plan.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 21
CHAPTER TWO: REVIEW OF THE LITERATURE
This chapter provides a general review of where MOOCs have been viewed as a
prominent trend in higher education with the rise of no brick-and-mortar, technologies used in
MOOCs, and credit transfer from MOOCs. Next is a discussion of instructional design for high-
quality MOOCs, including video lectures for MOOCs, pedagogical strategies for online courses,
and structured course content. Following that is a review of factors impacting MOOC
completion rates, such as student characteristics, student sentiment, teaching presence, and
interactions through social networking. This chapter concludes with specific knowledge,
motivation, and organizational factors affecting organizational management.
MOOCs as a Prominent Trend in Higher Education
No Brick-and-Mortar
MOOCs have been transformative, becoming a part of the educational delivery system at
the higher education level in the United States and around the world since 2012 (Kent & Bennett,
2017). In higher education, the data regarding the use of MOOCs is surprising, given the large
number of major institutions that have teamed up to help create MOOCs (Ingolfsdottir, 2016).
MOOCs have the capacity to host thousands of students simultaneously with just one instructor
while formal colleges can only host hundreds of students (Siegel & Carchidi, 2016). There are
no entry requirements, no admission applications, and no tuition fees. Everyone can attend the
class without being formally admitted to an educational program. MOOCs help promote the
globalization of higher education by lowering barriers to entry (Loeckx, 2016). As such,
MOOCs present no barriers to tertiary level study (Head, 2017; Whitehill, Mohan, Seaton,
Rosen, & Tingley, 2017). Although MOOCs do not have a long history, they have been viewed
as disruptive and threatening traditional face-to-face instructional formats (Dalipi et al., 2016).
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 22
MIT began posting course materials online with the launch of its Open Course Ware in 2001.
Stanford University took open courses to a new level when 60,000 students enrolled in its open
artificial intelligence course. Elite universities have since rushed to create MOOCs. These
MOOCs are dominated by the three major providers: Coursera, edX, and Udacity. Coursera and
Udacity grew out of Stanford University and are backed by venture capital. EdX is a joint
project of Harvard University and MIT.
Each year, MOOC providers become better at offering courses that appeal to students
who have neither the time nor resources to attend brick-and-mortar institutions. As of this year,
there were an estimated 4,200 MOOCs offered by over 500 universities, and the number of
students who have signed up for at least one course was estimated to be 35 million since 2015
(Dennis, 2017). About 8% of Coursera’s enrollment comes from India. For edX, although one-
third of the participants are from the United States, an estimated 2% of participants are from
developing countries (edX Blog, 2017). The U.K.-based MOOC platform FutureLearn has an
estimated three million users (Manathunga, Hernández-Leo, & Sharples, 2017). MOOCs are
social relationships and provide a large space where people can network with or without seeing
each other in very personal ways, but MOOCs are connected through physical computers and
networks, tied together by running codes. The new institutional form, no brick-and-mortar,
represents the software, highlighting and supporting certain types of interaction (Nakano, Padua,
& Jorente, 2017).
Technologies Used in MOOCs
Technology is changing rapidly in the area of online course delivery as seen with the rise
of MOOCs. All MOOCs operate mostly on the Internet. Although some MOOCs have been
designed to incorporate in-person meetings of students, all materials and interactivity within a
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 23
course generally are facilitated online, including virtual office hours with the instructor. MOOCs
rely on a combination of digital applications to achieve intended functions which serve to deliver
instructor’s lectures and course content to participants, interact with participants and the
instructor, and facilitate discussions and activities that promote learning (Ruipérez-Valiente,
Muñoz-Merino, Gascón-Pinedo, & Kloos, 2017). The digital applications are presented below.
Learning management systems. Learning management systems are web-based
applications that are designed to manage and deliver course content (e.g., background readings,
links to websites, images, relevant videos or audio, quizzes, exams, and even automatic
evaluations) to participants within an online course. MOOCs consist mainly of video-based
lectures and reading materials that can be accessed at a student’s own pace within the timeframe
of a course. It is important to regularly remind participants of new content uploads or certain
events through email, posts, or other means of communication.
Learning analytics. Learning analytics are typically employed in learning management
systems, including analyses on how long participants view materials, the paths they take to
complete a task, how frequently and for what purpose they communicate with other participants,
and which resources they find valuable (Ruipérez-Valiente et al., 2017). MOOC participants are
also provided with progress tracking, which are digital graphics that are awarded to participants
to indicate completion of course objectives or the degree to which they have participated.
Credit Transfer from MOOCs
The new remote education application of technology facilitates participation of students
from different geographical regions. Many universities use technology to offer alternate
instructional formats, providing students with more flexible learning options. The University of
Leeds and the Open University have agreed to accept FutureLearn MOOC courses for credit
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 24
toward a final degree or MBA (FutureLearn, 2017). A reported 440,000 students registered to
take the single-session MOOC “Understanding IELTS: Techniques for English Language Tests.”
which FutureLearn, in collaboration with the British Council, offered in 2016 (FutureLearn,
2017). In partnership with edX, Arizona State University’s global freshman academy allows
students to take their entire freshman year of courses online for credit. The program is the first
of its type offered by ASU (Stone, 2016; Byrne, 2015).
In Brazil, Unopar University offers low-cost degree courses by implementing online
materials and weekly seminars. Minerva University, based in San Francisco, utilizes technology
to lower tuition fees (around USD 10,000 a year) to compete with the best Ivy League colleges
(usually up to USD 60,000). Students spend some of their program doing online lectures while
living outside the United States, which also proves to future employers that they can manage
finances while earning a viable degree (The Economist, 2014).
The demands of university students and faculty for technology have been increasing
(Green & Gilbert, 2010). Some universities are already adding digital classes to their syllabi
(Baker, Nafukho, McCaleb, Becker, & Johnson, 2016). As digital courses have become more
entwined with existing curricula, over half of 4,500 students at MIT take a MOOC as part of
their degree requirements (Lazaroiu, Popescu, & Nica, 2016). The John F. Kennedy University
in California, which offers courses mainly for mature students, has started to accept edX MOOC
credits towards its degrees (The Economist, 2014). Some universities even collaborate with
MOOC providers and enterprises in order to improve the value of higher education. For
example, Georgia Tech offers an online master’s program in computer science in collaboration
with Udacity and AT&T (Hashmi, 2015).

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 25
Instructional Design for High-Quality MOOCs
Video Lectures for MOOCs
Researchers demonstrated that the length, composition, and production value of MOOC
videos are critical by using data from millions of MOOC video watching sessions (Herala,
Knutas, Vanhala, & Kasurinen, 2017; Malaga & Koppel, 2017; Ou, Goel, Joyner, & Haynes,
2016; Thornton, Riley, & Wiltrout, 2017). The peak length that any video is watched is around 6
minutes, regardless of full length of a particular video segment (Chew, Cheng, & Chen, 2017).
In addition, production value of a video, such as the quality of filmmaking, lighting, and
scripting, was related to achievement (Crook & Schofield, 2017; Rana, Besche, & Cockrill,
2017). However, large investments in film quality did not make a difference, suggesting a
middle ground in production quality (Guo et al., 2014).
A common practice of MOOC course content structure is segmenting the course into sub-
sections, referred to as weekly video lectures (Chew et al., 2017). There are several modules in
each section, similar to topics in each lecture session. Some MOOCs split the course structure
further into smaller parts within each section, yielding shorter video lectures in each lesson on
module. This provides learners with better navigation of the course flow. Two strategic
approaches to segmenting video lectures are part-by-part video segmenting and using timestamps
in the video lectures (Chew et al., 2017).
Part-by-part video segmenting. The complete video recording is segmented into
smaller portions according to the topics covered in the course content. A table of the course
content gives learners an overview of the course and allows instructors to evaluate the structural
frame of the course.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 26
Using timestamp. The video recordings are not segmented into smaller clips. Instead,
the whole recording is uploaded with additional timestamps embedded within the video,
indicated with labels or captions of the video partition.
Pedagogical Strategies for Online Courses
Pedagogy in designing online courses is critical as students do not have the benefit of
merely raising their hand and asking the teacher any time a question arises. In order to provide
more examples, resources, and guidance on how concepts can be incorporated into course
projects, Ou et al. (2016) proposed several pedagogical strategies to be integrated in designing
videos for MOOCs to create a coherent and dynamic instructional structure in the video lesson
and keep every video lesson short and concise. Students in an artificial intelligence course from
the Georgia Tech’s online master of science in computer science program agreed that such
dynamic and concise exercises kept them engaged and strongly believed that the video lessons
were valuable in helping them learn.
Clear guidance. Video lessons are organized based on the structure of the course topics,
and students are provided with a schedule articulating when they should finish studying each of
the lessons. All videos are available to the students and can be self-paced (Malaga & Koppel,
2017).
Learning by examples. Video lessons present real-world tasks and authentic scenarios.
Examples are tied with a particular practical problem. Each video lesson also includes several
interactive exercises, one for each main concept from the course (Crook & Schofield, 2017).
Personalized learning. Video lessons are equipped with teaching assistants to facilitate
discussions, monitor peer assessment, and provide timely feedback on student responses to
assignments or course content questions (Malaga & Koppel, 2017).
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 27
Structured Course Content
Although interactions with the instructor of MOOCs are a significant predictor of MOOC
retention, MOOC course content also has been a significant predictor of MOOC retention. Hone
and El Said (2016) noted that these constructs explained 79% of the variance in MOOC
retention. Instructional strategies for designing solid course content include opportunities for
students to learn from each other or adapting content to more simple activities and materials that
are still challenging to students (Macleod, Sinclair, Haywood, & Woodgate, 2016). Therefore,
fully-structured course content should be emphasized when creating MOOCs (Smith, Caldwell,
& Richards, 2017). The quality of course content should be used as the primary method of
encouraging long-term student involvement in the course and boosting student persistence
throughout the course (Jaggars & Xu, 2016).
Factors Impacting MOOCs’ Completion Rates
Student Characteristics
MOOCs require learners to have good learning habits, including being autonomous and
self-regulated. Greene, Oswald, and Pomerantz (2015) categorized student characteristics as one
of the major reasons for dropout rates in online education. They found that participants with
prior experience of MOOCs were less likely to drop out, as were older and more educated
participants. Self-rated commitment to completing the course was the most statistically
significant predictor (Hone & El Said, 2016).
Research found that students’ backgrounds predicted their performances in MOOCs
(Hew, 2016). Student characteristics refers to student background, previous academic
achievement, experience related to the course content, and skills dealing with tasks and time
management. Motivation, despite being a psychological characteristic, is also part of these
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 28
student characteristics (Greene, Oswald, & Pomerantz, 2015; Whitehill, Williams, Lopez,
Coleman, & Reich, 2015). Students have greatly varying characteristics due to open enrollment
and no prior student selection criteria (Agarwal, 2014). Thus, diagnosing student characteristics
before the course is impossible, especially considering that a MOOC is an environment that
anyone in the world can sign up for, with enrollment for some courses exceeding 50,000
students.
Student Sentiment
Robust evidence has shown that student emotions and achievements are reciprocal effects
(Pekrun, Lichtenfeld, Marsh, Murayama, & Goetz, 2017). Typically, positive emotions such as
enjoyment of learning show positive links with achievement, while negative emotions such as
test anxiety show negative links (Ben-Eliyahu, Linnenbrink-Garcia, & Putallaz, 2017;
Broadbent, 2017; Pekrun & Linnenbrink-Garcia, 2014; Zepke, 2017). Furthermore, positive
sentiment expressed in relation to the course instructor had the greatest positive effect on
likelihood of completion; sentiment expressed in relation to assignments and course material also
had a positive effect (Alexander & Grossnickle, 2016; Broadbent, 2017; Zepke, 2017). Positive
attitudes and feelings towards the learning process usually are triggered by learning motivation
(Richardson, Maeda, Lv, & Caskurlu, 2017; Topala & Tomozii, 2014). Students’ satisfaction is
strongly associated with students’ motivation.
Emotional support from faculty and friends and learners’ self-efficacy were also
important factors for students who persisted in remote learning (Kangas, Siklander, Randolph, &
Ruokamo, 2017; Ross et al., 2014; Shepherd, Bolliger, Dousay, & Persichitte, 2016).
Satisfaction in life or a program is also a construct that affects engagement (Wong, 2017). Lin
(2012) found that science and engineering graduate students showing dissatisfaction with their
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 29
graduate life had low motivation toward what they were studying. Other studies indicated that
learning satisfaction is influenced by factors such as content, location, facilities, teaching styles,
individual characteristics, and students’ participation (Kangas et al., 2017). Academic ambiance
also has positive effects on students’ level of satisfaction with learning.
Teaching Presence
Although Joy and Kolb (2009) agreed that learning preferences and styles were shaped
by the influence of individual cultural dimensions, Willingham (2008) argued that learning styles
do not exist. He proposed that students are more successful if teachers focus on teaching style
rather than about student learning styles. Specifically, teaching presence, or the instructor’s
personality, presentation skills, and delivery methods greatly influence student perception of the
course (Huang et al., 2017; Wang & Antonenko, 2017). MOOC pedagogy has drawn attention
among educators and researchers. On account of the massive enrollment in MOOCs,
instructional design should be structured in such a way as to ensure that there are interactions
with peers and with the instructor (Boettcher & Conrad, 2016; Shepherd et al., 2016; Ross et al.,
2014).
Udacity’s Mr. Thrun admits that “MOOCs’ pedagogy needs to improve very quickly,”
adding that MOOC participants “needed more personalized support to use a university-level
online course” (The Economist, 2014, para. 15). An analysis of instructional design quality
conducted by Margaryan, Bianco, and Littlejohn (2015) indicated that the majority of MOOCs
scored poorly on most instructional design principles. However, most MOOCs scored highly on
organization and presentation of course material. Thus, pedagogy tied with teaching presence
significantly affects student persistence (Bowers & Kumar, 2017; Wang & Antonenko, 2017);
even instructor enthusiasm or the university/institution are reasons that keep learners engaged
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 30
and persisting in their chosen courses. As a result, pedagogy and teaching styles need to be
explored when designing materials and activities in MOOCs in order to prevent a drastic
decrease in learners’ activities in distance courses (Boettcher & Conrad, 2016; Shepherd et al.,
2016).
Interaction Through Social Networking
Interaction through social networking has been considered an important means to
enhance communication in the collaborative process (Duval, Sharples, & Sutherland, 2017;
Zheng, Han, Rosson, & Carroll, 2016). Current technology familiar to today’s students such as
Facebook, Skype, Twitter, and other networking tools should be implemented in MOOCs to
promote learners’ engagement by encouraging them to collaborate in small groups based on
interests, and to support and learn from each other (Flavin, 2017; Veletsianos, 2017; Liu,
McKelroy, Kang, Harron, & Liu, 2016). Student interactions with peers and the instructor in the
course influenced persistence (Ostashewski, Howell, & Dron, 2016; Zheng et al., 2016). Online
discussion forums have shown to be effective methods for students to get feedback and
communicate among others in order to compensate for the frequency of usually one instructor to
many thousands of students in MOOCs (Smith et al., 2017; Manathunga et al., 2017; Reiser,
2017; Ostashewski et al., 2016). Although discussion forums can have low levels of student
participation and large amounts of low quality posts (Bouchet, Labarthe, Bachelet, & Yacef,
2017; Balakrishnan, Teoh, Pourshafie, & Liew, 2017), they are shown to be valuable in
enhancing learning and understanding for those who participated (Flavin, 2017; Ng, Lam, Ng, &
Lai, 2017; Ostashewski et al., 2016).
Students should be encouraged to actively participate in discussion forums or activities
due to the high volume of students enrolled in MOOCs (Ostashewski et al., 2016).
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 31
Communication through social networking tools or events becomes a necessary medium for
students to interact with peers and support each other (Flavin, 2017; Cruz-Benito et al., 2017;
Ostashewski et al., 2016). Further research showed that student likelihood of being at risk of
dropping the course or suffering from low motivation could be detected based on course
discussion forum posts via linguistic analysis (Ng et al., 2017; Wise, Cui, Jin, & Vytasek, 2017).
MOOC Participants’ Knowledge, Motivation, and Organization Influences
In this section, the knowledge, motivation, and organization (KMO) influences that might
impact the stakeholder’s goal of completing a six-week MOOC are examined from a theoretical
perspective and by reframing specific previously reviewed literature that is relevant to the
knowledge, motivation, and organization factors.
Knowledge and Skills
The framework evaluating factual, conceptual, procedural, and metacognitive knowledge
suggested by Anderson and Krathwohl (2001) were implemented in understanding MOOC
participant knowledge. Factual knowledge consists of the fundamental units that one must be
equipped with to solve problems, including terminology, definitions, and specific details.
Conceptual knowledge is the relationship among basic elements that enables them to function
together, including categories, generalizations, structures, and models. Procedural knowledge is
“how to do something,” such as criteria for using subject-specific skills or determination of when
to apply appropriate procedures. Lastly, metacognitive knowledge is the highest-level
knowledge. It is one’s own cognition: how one reflects, reviews retrospectively, and ruminates
on one’s progress toward achieving one’s goals. It is the knowledge of monitoring, controlling,
and regulating. The following sections examine the key knowledge influences.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 32
Factual knowledge. Participants may lack factual knowledge of the technology
proficiency or resources to access the course materials (Anderson & Krathwohl, 2001). This
factual knowledge could extend to participants’ geographical locations, whether they are
technology-friendly, and the features of the MOOC platform. Without this knowledge,
participants will not be able to navigate the course and use the features to complete their quizzes
and exercises.
Participants’ factual knowledge. Green and Gilbert (2010) found that the use of
technology in education facilitates the participation of the students from different geographical
regions; however, technology-unfriendly areas have not yet been included in educational
technology. As such, there are gaps between the registrants’ various regions, but mainly
between Western countries and Middle East regions. Participants from technology-unfriendly
areas tend to drop out of the course.
Conceptual knowledge. In addition to knowledge of accessing the online course
materials, participants also need knowledge of how the course will be conducted and what
learning outcomes might be expected from a required assignment. This conceptual knowledge is
assessed differently than other types of knowledge (Anderson & Krathwohl, 2001). Participants
may need to know that in order to earn a certificate, they must complete the segments involved in
each lesson using certain learning features on the MOOC platform. For example, students need
to practice with the audio vocabulary book assignments that are graded in the participation
section but are shown in the final grades section.
Participants’ conceptual knowledge. Learning with understanding is more likely to
occur when students are provided with categories of knowledge, or concepts, as opposed to an
independent body of facts (Schunk, 1996). However, there is a disconnect between the
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 33
participants and the instructors’ conceptual understanding of the course work. Head (2015)
indicated that most MOOC providers are from the US; professors and course designers often fail
to take consideration of different cultural backgrounds. For instance, certain assignments or
activities may illustrate an American concept such as Super Bowl Sunday, potentially impeding
the understanding of participants who are unfamiliar with American culture.
Procedural knowledge. Participants need to demonstrate that they have knowledge skill
to navigate the curriculum and course materials on the MOOC platform (Anderson & Krathwohl,
2001). With this procedural knowledge, participants will be able to explore the curriculum and
locate the course materials and assessment quizzes that they needed to complete in the course.
Participants’ procedural knowledge. Nearly all MOOCs failed to reflect the principle
of demonstration. Participants did not continue with the course due to the lack of explicit
demonstration and clear guidance throughout the course (Margaryan, et al., 2015).
Metacognitive knowledge. Anderson and Krathwohl (2001) indicated that
metacognitive knowledge is essential to building expertise at performing a task. Participants
learn how to reflect and monitor their learning and performance through opportunities to practice
metacognitive strategies, such as instructor approaches to tasks, scaffolded steps in the learning
process, peer coaching, and providing feedback.
Participants’ metacognitive knowledge. Several researchers have commented on the
importance of metacognitive knowledge for MOOC completion rates. Tomkin and Charleviox
(2014) agreed that aside from MOOC courses being fully-structured, many MOOC participants
are highly motivated with high academic achievement, stronger self-regulation, and goal-driven
attitudes. Whitmer et al. (2014) echoed that participants and university characteristics also affect
student retention. Since most of the MOOCs are provided by top-tier universities, the majority
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 34
of the participants are from those universities. As such, metacognitive knowledge is a critical
construct driving MOOC participants to assessment completion.
Motivation
Motivation has been considered a dominating factor that helps learners maintain
perseverance and achieve successful learning outcomes for many years (Ambrose et al., 2010).
Perseverance is to continue working until a goal is achieved with or without any obstacles.
Motivation-related constructs influence individual belief towards one’s ability to obtain a goal
and one’s reasons for doing an achievement activity (Wigfield & Cambria, 2010). When learner
motivation is inspired, learners show curiosity about the learning topic, immerse themselves into
the learning tasks, and seek strategies that enhance their learning (Hodges, 2004; Wlodkowski,
2008). Learner motivation and learning outcome was usually found to be positively correlated
(Busato, Prins, Elshout, & Hamaker, 2000; Liu, Bridgeman, & Adler, 2012; Sankaran & Bui,
2001; Waschull, 2005). It is also assumed that learners with higher achievement will be more
likely to have continued motivation in the future (Hodges, 2004). Thus, learner lack of
motivation can lead to disengagement in learning and cause them to perform poorly (Starcher &
Proffitt, 2011). Goal orientation theory and attribution theory, the two social-cognitive theories
considered to be the most important motivation influences, are discussed in the next section.
Goal orientation theory. A social-cognitive theory of achievement motivation examines
the reasons why students engage in their academic work. According to Pintrich (2000), the first
type is called a mastery goal. Students hold mastery goals (also referred to as being mastery-
oriented) when their goal is to truly understand or master the task at hand. Students who are
mastery-oriented are interested in self-improvement and tend to compare their current level of
achievement to prior achievements. The second type is called a performance goal. Students
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 35
hold performance goals (also referred to as being performance-oriented) when their goal is to
demonstrate their ability compared to others. Students who are performance-oriented are
interested in competition, demonstrating competence, and outperforming others; they tend to use
other students as points of comparison, rather than themselves.
Participants’ expectations. According to Pintrich’s (2000) goal orientation theory,
students learn Mandarin for specific reasons and expect to actually use what they learn in real-
life scenarios. Some courses on the MOOC platforms have catered to this in many ways, which
was found to be very helpful and effective for students’ learning (Gamage et al., 2015). An
example from the entrepreneurship course offered by the Massachusetts Institute of Technology
bridged the gap between course materials and industry needs by encouraging students to take
part in the industrial needs published on a platform (Coursolve.com). Students were also
directed and introduced to the industrial perceptions through live webinars with guest panelists
who were key people from the industry. Such course related activities are rarely implemented
for MOOCs. Nevertheless, students highly valued such activities and found it to be a very
important dimension for effective learning.
Attribution theory. Anderman and Anderman (2010) articulated that attribution theory
is an important method to examine and understand motivation in academic settings. They
examined the influence of certain events on individual perception of success and failure as well
as subsequent motivation in participating in the related learning activities. There were two main
ideas that Heider (1958) proposed that became influential: (1) internal attribution, the process of
assigning the cause of behavior to some internal characteristic, rather than to outside forces, and
(2) external attribution, the process of assigning the cause of behavior to some situation or event
outside a person’s control rather than to some internal characteristic, such as situational or
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 36
environment features. For example, under a theory of internal attribution, one attributes the
behavior of a person to their personality, motives, or beliefs.
Participants’ engagement. Several references examined in the previous sections of this
literature review shared a common finding: participants’ engagement is the key to completion
rate. Tomkin and Charleviox (2014) pointed out that whether a credit is tied with a MOOC
course or not affects student engagement although many students took Mandarin courses to
prepare for future career success. Travelling in Mandarin-speaking countries was the second
strongest motivating factor for respondents. However, participation levels decreased over time
and appeared to be related to the amount of effort required for activities (Whitmer et al., 2014).
Quizzes and the final examination had a higher level of participation than assignments.
Organization
Gallimore and Goldenberg (2001) suggested that cultural-related explanations of models
and settings have gained prominence in educational reform and the problem of change in
organizations. Cultural models consist of shared mental schema or normative understandings of
how the world works, or ought to work (Gallimore & Goldenberg, 2001). The concept
incorporates behavioral activities as well as cognitive and affective components. Cultural
settings are environments in which more than two people coming together and put time in to
accomplish something. Both cultural models and settings are considered the way things should
be, taken-for-granted assumptions that are unnoticed in an ecological niche. Clark and Estes’
(2008) gap analysis model also identified some organizational barriers which hinder the
completion of the original goals. In the next section, cultural models and settings are discussed
as an advanced organizational factor.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 37
Cultural models and settings. According to Gallimore and Goldenberg’s (2001)
definition of cultural models and settings, culture is thought to be composed of many cultural
models, internalized differently by culture members. Furthermore, the learning is social and
individual. With the belief that everyone should have the equal opportunity for high-quality
education through the Internet, MandarinX is committed to ensuring that students are fully and
timely supported whenever technical issues hinder their learning and networking.
Organization’s reputation and participants’ language barriers. One crucial factor
affecting MOOC participants’ completion rate is the reputation of the organization (HarvardX &
MITx, 2014). MandarinX, associated with Harvard and Massachusetts Institute of Technology,
has accreditation. As it was originally founded with a small team, every staff member has been
trained to cover one another’s job duty and is adequately acquainted with the platform and
course information. Weekly online meetings and social networking groups keep the staff
coordinated through near-instant communication. The task of translating and localizing content
has become a challenge for non-native English speakers to enhance the effectiveness of learning
because most MOOC courses have been taught in English with context based more on American
culture (Che et al., 2016).

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 38
CHAPTER THREE: METHODOLOGY
Purpose of the Project and Questions
The goal of this study was to evaluate the reasons why MOOCs provided by MandarinX
achieved an unusually high completion rate compared to the average for MOOCs, and to build
on this success with future MOOCs. The analysis focused on KMO influences related to
understanding this organizational achievement. For practical purposes, the stakeholders this
study focused on were MandarinX participants.
The research questions for this study were
1. What are the knowledge, motivation, and organizational influences that underpin
MandarinX’s high completion rate?
2. What knowledge, motivation, and organizational assets would extend this success into
future MOOC design?
In this chapter, a gap analysis methodological framework developed by Clark and Estes
(2008) was adapted as a needs analysis model to evaluate the assumed causes categorized by
knowledge, motivation, and organization. Preliminary scanning data were discussed based on
this KMO method. Following that, the focus shifted to participating stakeholders and
descriptions are provided. Surveys were utilized to obtain data in this study. Validation of the
performance issues/needs/assets, conceptual framework for addressing the inquiry questions,
trustworthiness of data, the role of investigator, data analysis, and limitations and delimitations
are discussed.
Methodological Framework
The gap analysis framework began with an overarching goal that was compared to the
current progress toward the goal. The model was then followed by an analytical procedure to
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 39
examine the gaps between the target achievement and the current achievement by looking at the
causes for any discrepancies between the two. These causes were then broken down into three
categories: knowledge, motivation, and organizational barriers (Figure 1).

Source: Clark and Estes (2008)

Figure 1. The gap analysis process model.

With further analysis and an understanding of the underlying causes for the gaps, steps
can be taken, and improvements can be made in order to attain a nearer completion of the
original goals. The eventual realization of the goal can be achieved with repeated
implementation of this methodological approach.
Three key elements, knowledge, motivation, and organization, were analyzed within this
gap analysis framework. Knowledge was further categorized into (1) factual knowledge, the
ability to understand the pieces of information related to any given concept; (2) conceptual
knowledge, the ability to associate factual knowledge with categories and classifications; (3)
procedural knowledge, the ability to use knowledge to accomplish certain tasks; and (4)
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 40
metacognitive knowledge, the ability to transfer knowledge to a new context or to solve a
problem (Clark & Estes, 2008).
Motivation was also categorized into (1) choice, the decision to begin the motion toward
the accomplishment of a settled goal; (2) persistence, the ability to be consistent with moving
forward regardless of obstacles; and (3) mental effort, the ability to maintain sustainability
toward the achievement (Clark & Estes, 2008).
Two categories of organization were (1) organization models, the culture and values
within any organization that are inherited rather than created, forming hidden assumptions that
are the basis of interactions and decision-making processes within the organization; and (2)
organizational settings, the organization’s structures and procedures. Both organizational models
and settings may be barriers to the good execution of performance goals (Clark & Estes, 2008).
Knowledge, motivation, and organization serve as the starting points to tackle the
problems raised in organizational performance. Clark and Estes (2008) defined knowledge (K),
motivation (M), and organization (O) as targets and fundamentals that must be internally aligned
with any successful organizational goal achievement.
Assumed Performance-Based Influences
The gap analysis model proposed by Clark and Estes (2008) was adapted as an evaluation
method for this study, which required an analysis of the organization’s goals and current
achievements that exhibit gaps. The causes for the gaps are classified as knowledge, motivation,
and organizational barriers. In order to examine the influences thoroughly, both qualitative and
quantitative studies were utilized in this study. Examples of factors related to each knowledge,
motivation, and organization at MandarinX were investigated based on the literature review and
data analysis. Components affecting student completion in the study were students’ dropout
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 41
rates, course design, and other technological or environmental barriers. This data was gathered
by surveying the stakeholders and analyzing the survey results. The motivation lens probes
individual’s motivation, participant’s persistence, and the mental effort of completing a six-week
online language course. The impact of instructor involvement was contrasted with students’
learning outcomes. Reasons were validated, and research-based solutions were sought in order
to address the dropout rate. Implementations and tools were then offered to the organization.
Preliminary Scanning Data
Knowledge and skills. With China’s economic growth, an increasing number of people
are starting to learn Mandarin Chinese. A survey conducted by a Mandarin-learning website
indicated the Chinese government recently held back Western expats for job opportunities or
promotions due to language barrier restrictions (Selmer, 2006). Although English ability is
improving in China, business conversations carried out in English might not be fully
comprehended. Ferraro and Briody (2017) pointed out that cultural differences and norms
sometimes hinder the efficiency of real understanding when two parties communicate in a
second language.
Based on geographic distribution, the top three countries by enrollment for MandarinX
are USA, India, and the United Kingdom. All three countries have economic transactions and
business partnerships with China. Thus, the learning goal of applying fundamental language
skills and cultural knowledge to real-world applications is valued highly (Mayer, 2010).
Chances are also higher that participants from these countries may remain enrolled to the end of
the course.
Motivation. High enrollment rates create challenging issues. Participant motivation
varies for a number of reasons: receiving certificates to improve resumes; learning new things in
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 42
a manner that allows for more flexible time management (e.g. businesspeople, retired professors,
double-majored college students); meeting with people from fields of similar interest, or just for
fun. Overall, students choose to continue their studies for personal betterment, self-improvement
by adding skills or credentials, or simply gaining knowledge of a topic that they had previously
been unable to for whatever reason. Therefore, the factors that helped to maintain perseverance
in studies has been the most critical of the assessments conducted within MOOC, given that the
average completion rate is often less than 10% (Alario-Hoyos et al.,2017; Aparicio, Bacao, &
Oliveira, 2017; Bonk & Lee, 2017; Cunningham, 2017; Milligan & Littlejohn, 2017).
Organization. Due to the large number of enrollees, time differences, and some
participants from Internet-unfriendly areas, obtaining support and reducing obstacles has become
a top priority (Sergis, Sampson, & Pelliccione, 2017). In order to encourage perseverance and
provide students with the resources that they need to keep learning, certain measures have been
implemented: (1) teaching assistants took shifts answering questions posted on the forum; (2) the
instructor offered periodic online office hours and tutoring; and (3) the technical support team
provided real-time solutions to problems caused by the Internet or platform which hindered
course viewing.
Support from the other staff members was critical as the team was small and there was no
hierarchy involved. The team comes from different professional backgrounds and different
Mandarin-speaking areas, which helped to enhance collaboration on cultural perspectives of
language usage. Often, MandarinX staff had to solve unprecedented problems, such as dealing
with time differences due to geographical location and language differences within the team
itself.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 43
Participating Stakeholders
The stakeholders of this study were the first (47,081 enrollments), second (39,849
enrollments), and the third ongoing cohort (24,084 enrollments as of August 2016) of three
MOOCs: Basic Mandarin Level One, Two and Business Chinese. Geographically, these
MandarinX participants come from around the world, for a total of 202 countries. The United
States was in the lead (25%), followed by India (8%), the United Kingdom (4%), Mexico (3.7%),
Spain (3.5%), Brazil (2.9%), undisclosed countries (2.6%), Australia (2.4%), Germany (1.8%),
France (1.8%), the Philippines (1.7%), Russia (1.6%), Colombia (1.6%), China (1.6%), Peru
(1.2%), Singapore (1.2%), Pakistan (1.2%), Netherlands (1.1%), Malaysia (1.1%), Indonesia
(1.0%), Italy (1.0%), and Poland (1.0%) with the remaining 1% consisting of various other
countries (edX Insights, 2016) as shown in Figure 2.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 44

Source: Learn Basic Chinese (edX Insights, 2016)

Figure 2. Enrollments in MandarinX’s Current MOOCs

Regarding the demographics, the median student age was 28; 35.4% of the students were
aged 25 and under; 45.1% of the students were aged 26 to 40; 19.5% of the students were 41 or
over (Figure 3). Also, 27.6% of the participants had a high school diploma or less; 41.5% of the
total possessed a college degree; 28.1% earned advanced degrees, including master’s or doctoral
degrees (Figure 4). In addition, 58.3% of the registrants were male, 41.1% were female, and
0.6% claimed the gender as other (edX Insights, 2016).
25%
8%
4%
3.70%
3.50%
2.90%
2.60%
2.40%
1.80%
1.80%
1.70%
1.60%
1.60%
1.60%
1.60%
1.20%
1.20%
1.10%
1.10%
1%
1%
1%
1%
Enrollments
from
202
countries
US India
UK Mexico
Spain Brazil
Undisclosed
Countries Australia
Gremany France
Philippines Russia
Colombia China
Peru Singapore
Pakistan Netherlands
Malaysia Indonesia
Italy Poland
Remaining
comprised
of
other
countries
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 45

Source: Learn Basic Chinese (edX Insights, 2016)

Figure 3. Age Distribution of the Enrollment in MandarinX’s Current MOOCs

Figure 4. Educational Backgrounds of the Enrollment in MandarinX’s Current MOOCs

A stratified random sampling approach was used in which the researcher identified the
relevant stratums and their actual representation in the population, namely completion and non-
completion. Random sampling was used to select a sufficient number of subjects from each
stratum. Too small a sample size would fail to claim the findings (Schabenberger & Gotway,
2017). Researchers determined the adequate sample size based on judgment and experience.
high

school

diploma
or

less
college

degree
advanced

degrees,

including

masters
or

doctoral
27.60%
41.50%
28.10%
Educational
Backgrounds
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 46
“Sufficient” referred to 1 to 2 percentage of the total enrollments of the three cohorts, which the
researcher considered significant enough to be reasonably confident that the stratum represented
the population. The margin of error that this study tolerated was 5%. The confidence level was
95%. Providing the population size was 111,014, and the response distribution was 50%, the
recommended sample size was 383 (Calculated by sample size calculator: Raosoft, Inc).
Data Collection
The sample data collection method for this study relied on enrollment information
tracked by MandarinX including participation in video lectures, quizzes, exercises, and activities
within the edX platform. All participants’ data, including demographic background, learning
process within the six weeks (responses, retention, participation, and performance) collected and
stored on the edX platform database was requested and assessed by edX partners after signing a
disclosure agreement on educational research use. The data included general student
demographics such as nationality, age, geographical location, level of engagement in the course,
and performance on tasks and quizzes. Permission from the University of Southern California’s
institutional review board was obtained. The Family Educational Rights and Privacy Acts was
observed as well due to the number of registrants from the United States (25% of the total). One
data collection method was used in this study. Data collection occurred by surveys only, since
surveys were the most efficient way to acquire desired information in a relatively short period
(Orcher, 2017). However, the utility of the data remains limited by the lack of insight into
learning analytics of survey respondents who complete or do not complete a MandarinX course.
There have also been concerns expressed over the potential research ethics of MOOC data
mining in cases where insufficient consideration has been given to issues such as informed
consent, privacy, anonymity and confidentiality (Rolfe, 2015; Marshall, 2014).
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 47
Surveys
Given the number of registrants from different countries, an exit course survey embedded
in a Google sheet was distributed in weekly newsletter emails. The survey was only provided in
English as the course was taught in English. Participants were encouraged to complete the
online survey. Muijs (2010) asserted that two things determine the size of the significance level
of a research: (1) the size of the relationship or difference found in the sample; (2) the sample
size. The latter is crucial to the significance level, which is of 0.051 (not significant) and 0.049
(significant). The arbitrary cut-off point is less than 0.05 (5%). Provided that the total
registrants exceeded more than 100,000, even with a low response rate, the number of responses,
which were collected anonymously, would still be considered effective. A 6-point Likert scale
questionnaire was used to prevent people from choosing the neutral option. Data from
individuals not responding completely was discarded. Statistical software, Statistical Package
for Social Sciences (SPSS), was utilized to run the data for quantitative analysis. Analysis
emphasized understanding phenomena as they exist, not following pre-determined hypotheses.
The survey protocol is included in the appendix.
Validation of the Performance Needs
Only surveys were used to validate the assumed needs necessary for performance goal
achievement of an unusually high completion rate as compared to the average for MOOCs. Each
of the critical knowledge, motivation, and organizational needs were validated through the
collection of quantitative data.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 48
Table 2
Conceptual Framework for Addressing the Inquiry Questions
Inquiry Question Survey
What are the knowledge, motivation, and
organizational influences that underpin to
MandarinX students’ high completion rate?

X
What knowledge, motivation, and
organizational assets would extend this success
into future MOOC design?
X

Trustworthiness of Data
This study utilized the statistical tool, SPSS, to ensure the trustworthiness of data.
Further, survey questions were based upon existing valid and reliable instruments. Additionally,
the anonymity and confidentiality of survey respondents were guarded with a disclosure
agreement on educational research use exclusively (Merriam, 2009).
Role of Investigator
The researcher is the founder of MandarinX, graduated from the University of Southern
California with a master’s degree in teaching English to speakers of other languages at Rossier
School of Education, and hold teaching credentials from Commission of Teacher Credentialing
California in both subjects of Mandarin and English. Currently, the researcher serves as a
professor in the College of Commerce at National Chengchi University in Taipei, Taiwan.
As the founder and instructor of MandarinX, the investigator oversaw all staff, including
the course development team, production team, and teaching assistants within the organization.
The founder’s passion is also the fuel and compass of the organization. As principal investigator
in this study, the founder conducted a gap analysis of the performance evaluation and proposed
recommendations to help MandarinX and other institutions achieve a better course completion
rate by designing MOOCs. Understanding MOOC participants’ persistence will help the
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 49
organization create a conductive online language environment and maintain the high completion
rate for the future courses. During this study, stakeholders were not made to be aware of the
investigator’s role as a researcher to prevent distraction and confusion.
Data Analysis
This study relied on the gap analysis model proposed by Clark and Estes (2008) to
determine if the presented information was relevant to the assumed influences. First of all, four
types of knowledge (factual, procedural, conceptual, and metacognitive) were employed to
identify the gaps caused by the lack of skills. Secondly, various types of motivational issues,
such as participants’ expectation towards goal setting and engagement in the course, were
identified. Lastly, the effects caused by cultural models and settings within the organization
were reviewed. A thorough framework based on the Clark and Estes model was developed after
probing the presented influences by scrutinizing the data, which affirmed what Rueda (2011)
asserted: gap analysis and solutions to the problems are in alignment.
Quantitative data are presented in a numerical format and were collected in a
standardized manner (surveys and open-ended interviews). The statistical analyses were
conducted with SPSS statistical software once all online surveys were completed in a period.
Statistical significance and factor analysis were reported. A unique identifier was assigned.
Completeness and accuracy was checked. Incomplete data was then removed.
Limitations and Delimitations
Limitations
Due to the fact that the researcher is the founder of the organization, several biases were
impossible to be avoided and thus need to be addressed. First, possible methodological
limitations could include self-reported gathered data from surveys which are not independently
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 50
verifiable in order to report a noteworthy outcome to achieve a high-performing organization’s
goal. Second, participants, the key stakeholder, may not interpret the survey questions and items
in the manner intended. The researcher had adequate explanations while conducting surveys.
Delimitations
The focus of this study was to conduct a gap analysis to understand the high completion
rate of the organizational designed MOOC. One of the delimitations of the study was that the
context was global and virtual which made it more connected to other institutions or
organizations. Moreover, the implementation of Clark and Estes’ (2008) gap analysis model of
this evaluation study served as a strategic tool to the reader. A second delimitation was the size
of the selected stakeholder group made the study most revealing for academic and practical uses.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 51
CHAPTER FOUR: RESULTS
The objective of this study was to explore the knowledge, motivation, and organizational
influences that underpin MandarinX students’ high completion rate in a six-week MOOC. A
systematic method adapted from Clark and Estes’ (2008) gap analysis framework was applied to
validate the assumed causes, examining organizational performance by clearly articulating
organizational performance goals, and evaluating the gap between actual performance and
desired performance goals. Validity of the assumed causes were then analyzed using data
collected from a survey. SPSS was employed to analyze all the quantitative data from the
questionnaire survey.
This chapter presents findings from the collected data organized according to the
categories of knowledge, motivation, and organization with key findings synthesized. The
objective is to determine which of the assumed causes listed in Chapter Three have been
validated through the data collection process. Causes which ensure the success of MOOCs were
validated in this chapter and can be summarized in three essential components: (1) clear
guidance is provided by the instructor and participants are prepared for reciprocal ability to
navigate course content on the MOOC platform; (2) online teaching pedagogy profoundly affects
participants’ perspectives towards MOOC and their emotions are significantly related to
motivations, learning strategies, and completion rates; and (3) interactions and prompt feedback
increase the satisfaction level of the overall MOOC design.
Participating Stakeholders
All cohorts of MandarinX participants (133,281 enrollments as of December 2016),
including participants who joined Basic Mandarin Level One, Two, and Business Chinese (both
instructor-led track and self-paced track), were invited to complete an online survey for this
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 52
study. The online survey questionnaire was distributed to the participants through weekly
newsletters, Facebook, and Twitter for one month, beginning on 6th December 2016. When this
survey was closed, a total of 696 members had provided valid responses. The stakeholders
surveyed as part of the data collection process were drawn from course participants in 202
countries. The demographic information of the respondents is provided in Table 3. All
respondents remained anonymous, and responses from participants under the age of 18 were
excluded.
Data Collection and Validity
This study, therefore, aimed to examine two research questions:
1. What are the knowledge, motivation, and organizational influences that underpin
MandarinX students’ high completion rate?
2. What knowledge, motivation, and organizational assets would extend this success into
future MOOC design?
To examine participants’ persistence in a six-week MOOC, a survey was conducted. All
the measures were developed from a gap analysis framework (Clark & Estes, 2008). This
method is more effective than other approaches with regard to obtaining KMO influences and
enhanced the broader application of the research findings. In all of the scales used in survey
questions, respondents were asked to assess the degree to which they agreed with each statement
(1 “Strongly Disagree” to 6 “Strongly Agree”). The appendix presents a complete copy of the
survey instrument.
To investigate the reliability of the survey used in this study, Cronbach’s alpha was used
as the statistical tool to calculate the internal consistency and reliability. According to Devellis
(1991), a value of Cronbach’s alpha coefficient of internal consistency between .65 and .70 is
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 53
acceptable, between .70 and .80 is quite good, and between .80 and .90 is excellent. Checking
the Cronbach’s alpha coefficients in this questionnaire, the alpha coefficients of the three main
KMO causes with 20 assumed causes all reached the acceptable degree. The alpha coefficients
were between .731 to .828 and total reliability reached .904. This means that the reliability and
internal consistency was acceptable.
There were more male (61.8%) than female (37.2%) respondents, and the majority
(76.1%) were above 26 years of age. A range of careers was represented, although nearly half
(49%) of the respondents were students, teachers, or retired professors, and almost all of the
respondents (96.9%) had a high school diploma or higher degree. ANOVA, t-test, Regression,
Pearson correlation, chi-square test and P value were used to find some significant differences
for statistical analysis. Involvement in a wide variety of factors in this language MOOC is also
represented in the sample. Table 3 presents the detailed demographic information of the
respondents.
Table 3
Demographic Information of the Respondents
Measure Item Frequency Percentage
Gender Male 430 61.8
Female 259 37.2
Decline to state 6 0.9

Age Under 18 31 4.5
18 – 25 years 135 19.4
26 – 35 years 158 22.7
36 – 45 years 124 17.8
Over 45 years 248 35.6

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 54
Table 3, continued
Measure Item Frequency Percentage
Ethnic Background Caucasian 258 37.1
African 50 7.2
Hispanic 112 16.1
Asian 204 29.3
Other 72 10.3

Education Level Primary 5 .7
Middle 16 2.3
Secondary 89 12.8
Associate 45 6.5
Bachelor’s 254 36.5
Masters 214 30.7
Doctorate 41 5.9
Other 31 4.5
Note: The sample size is 696.

An independent-samples t-test was conducted to compare participants’ gender
perceptions towards all of the assumed causes under knowledge, motivation, and organization.
There was no significant gender difference in the three domains of assumed causes. However,
there were significant differences by participants’ age, education level, and ethnic background
for the assumed causes under the motivation and organization categories based on one-way
ANOVAs.
Results and Findings for Knowledge Causes
Three assumed knowledge causes out of four outlined in Chapter Three were validated,
as summarized in Table 4. Following the table, there is a discussion of the knowledge causes in
the context of Krathwohl’s (2002) knowledge framework, dividing knowledge into four
dimensions: factual, conceptual, procedural, and metacognitive.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 55
Table 4
Validated Assumed Knowledge Causes
Category Assumed Cause Validated Not Validated
Factual Participants know reasons which
prevent them from
completing the course.

Ö

Conceptual Participants are aware that different
Mandarin-speaking countries have
different language usages.

Ö
Procedural Participants have the knowledge
necessary to navigate the curriculum
and the LMS.

Ö

Metacognitive Participants need to know where to
find supplementary materials to assess
their learning.

Ö

Three key findings associated with knowledge were identified: adequate knowledge for
completing courses, unfamiliarity with differences in Mandarin Chinese, and self-awareness and
adaptability. In the next section, each of these assumed causes are described in more detail; in
the later discussion, the relationships among several assumed causes under three main categories,
knowledge, motivation and organization, are summarized by tables and figures.
Adequate Knowledge for Completing Courses
Adequate knowledge and completion rates are assumed to be closely related in this study
and much of the research (Gütl et al., 2014; Jordan, 2014; Khalil & Ebner, 2014; Kizilcec, Piech,
& Schneider, 2013; Reiser, 2017). Participants have the requisite knowledge to diagnose reasons
which prevent them from completing the course. Factual knowledge encompasses only basic
facts related to a specific topic (Krathwohl, 2002). A learner must possess factual knowledge in
order to be familiar with a subject area and to solve problems in that subject area. Elements of
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 56
factual knowledge therefore include knowledge of awareness of facts that hinder course
completion.
Figure 5 shows that “busy schedule” was the reason 329 (47.3%) respondents failed to
submit an assignment. Conflict or busy schedule is the most commonly cited reason for not
completing an assignment or exam whether it is for online courses or school work. Also, 80
(11.5%) respondents admitted that they were “being lazy and procrastinated.” It came as no
surprise that these two reasons accounted for more than half of the total responses, which was
696. Surprising, however, was that 161 (23.1%) respondents, constituting almost a quarter of the
pie chart, selected “nothing impeded me,” which indicates that participants had the adequate
knowledge required to complete the course. Lastly, only 27 (3.9%) participants responded
“confusion about the instruction,” which also suggests that the course was well-structured with
explicit guidance and instructor support. MOOC instructor’s role, as a teacher and facilitator,
deeply affected participants’ learning experiences and the MOOC success, as discussed later in
the last section, the organizational category.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 57

Figure 5. Survey Participants Reasons of Failing to Submit an Assignment

A one-way ANOVA was conducted to compare all of the assumed causes under the
KMO categories to see if variables are related in different categories. That meant a survey
question was expected to identify mutually referencing factors under two or even the three
categories. The results showed that “reasons” played a significant part in impeding participants
from submitting assignments, completing video lectures or exams at the p<.000 level for the
three categories (Table 5).
What the study illustrated was the factual reasons for participants not completing the
courses; however, the data presented a significant effect on mutually reinforcing among the three
categories, which explained the assumed causes under the motivation and organization categories
affected participants’ viewpoints in perceiving every aspect of this MOOC. The motivation and
organization assumed causes are discussed individually in the next two sections.

11.5%
3.9%
47.3%
5.2%
1.1%
2.7%
23.1%
5.2% Laziness/procrastination
Confusion
about
the
instruction
Busy
schedule
Personal
emergency
Lack
of
interest
I
don’t
think
I
would
get
a
good

grade(or
helpful
feedback)
Nothing
impeded
me
Other
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 58
Table 5

ANOVA Analysis for All of the Assumed Causes under Knowledge, Motivation, and Organization
Categories

N Mean SD F Value
Knowledge 696 4.06 1.120 7.936 ***
Motivation 696 4.25 .972 8.735***
Organization 696 4.27 1.023 4.807***
Note: *P<0.05, **P<0.01, ***P<0.00

Unfamiliarity with Differences in Mandarin Chinese
Conceptual knowledge is whether leaners understand the different varieties of the
language. Mandarin Chinese is made up of an entire group of language varieties across most of
northern and southern China, including many local dialects and the basis of standard Mandarin
or standard Chinese. Apart from the main stream of the course design, additional learning of
related varieties of spoken Chinese was also conveyed through weekly cultural notes videos.
Nevertheless, the survey results did not validate the assumption that the participants were
aware of the differences among accents, dialects, and usages in various Mandarin-speaking
countries and regions after the course based on the conceptual knowledge that the participants
could access through additional weekly cultural notes videos. Slightly more than half of the
respondents 363 (52.2%) did not think that they could differentiate one from another. Still, 181
(26%) out of the total (696) regarded themselves as being capable of differentiating among the
varieties (Figure 6 shows more detail). Although the course materials covered two versions of
written Chinese: traditional and simplified Chinese characters, due to the geographic distribution
and various dialects, challenges of covering and delivering all are insurmountable.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 59

Figure 6. Survey participants knowledge of differentiating language usages in mandarin-
speaking regions.

Self-Awareness and Adaptability
Procedural knowledge for learners is to have the knowledge necessary to navigate the
curriculum and skills necessary to troubleshoot techniques to isolate the root cause of problems
and to adjust learning strategies (Krathwohl, 2002). Learning is increased when learners acquire
component skills, practice integrating them, and know when to apply what they have learned.
Metacognitive knowledge encompasses awareness of cognition; application of this knowledge
results in modification of one’s own thought processes and actions (Krathwohl, 2002). Based on
the data collected, the participants seemed to demonstrate a good understanding of awareness of
their own cognitive ability in general, validating both assumed procedural and metacognitive
causes. There were 544 (78.1%) participants who noted that they were “familiar with the
courseware for accessing all of the units for the lessons” while only 152 (21.8%) disagreed with
the statement. When asked, “how would you rate your ability to seek additional resources to
help with learning Mandarin?” there were 525 (75.4%) out of 696 participants who responded
that they were confident in their ability in seeking additional resources to help with learning
91
103
169
181
100
52
0
100
200
Very
Poor bad Below
Average Above
Average Good Excellent
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 60
Mandarin (Figure 7). A significant majority of the participants were confident that they had
reciprocal technical arrangement to access all of the course materials and activities by utilizing
the MOOC platform and network infrastructure, while some studies identified that lack of
knowledge is a major concern of drop-out rate, especially to those who have invested time and
effort and did not complete (Gütl et al., 2014). Further, they demonstrated self-reliance in
seeking additional sources for the purpose of learning Mandarin Chinese.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 61

Figure 7. Survey participants confidence in seeking additional resources to help with learning
Mandarin.

MOOCs reach a wide geographically dispersed group of participants. However,
participants face issues of isolation and disconnect, the same as those in most of the online
learning environments. Although MOOC participants are expected to learn autonomously, it is
also reported that the majority of them fail to self-organize. They are not prepared to control
their own learning and encounter problems in using the learning tools or have difficulty in
understanding the subject matter. With this in view, the results from the knowledge causes
demonstrate that MandarinX MOOC participants are equipped with sufficient knowledge to
navigate the curriculum and used the learning tools to access the materials and activities on the
MOOC platform. In addition, they have the ability to search for supplementary materials to
enhance their learning and detect various factors including academic and personal reasons which
stop them from submitting assignments and thus completing courses successfully.
Notwithstanding that Mandarin Chinese has the most variety of language usages and norms,
MandarinX MOOC participants could understand the basis of standard Mandarin Chinese.

31
38
102
182
210
133
0
100
200
Very
Poor Poor Below
AverageAbove
Average Good Excellent
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 62
Results and Findings for Motivation Causes
The 10 motivation assumed causes discussed previously in Chapter Three were, as with
the knowledge assumed causes discussed above, either validated or not validated based on the
same survey. The results of the data analysis are summarized in Table 6 and followed by a
discussion of the data.
Table 6
Validated Assumed Motivation Causes

Category

Assumed Cause

Validated
Not
Validated
Intrinsic
Motivation
The course has relevance to individual
differences.

Ö
Extrinsic
Motivation
Participants receive emotional support
from faculty members.
Ö
Participants have instant feedback from
the instructor, peers, and teaching staff.

Ö
Cost/Benefit Participants can audit the courses for
free unless they would like to have a
certificate.
Ö
Participants appreciate a certificate from
an organization which has
reputation/affiliation with top-tier
universities.

Ö
Metacognition Participants are aware of their own
learning.

Ö
Self-Efficacy
Perceptions
Participants are confident with their
academic abilities.

Ö

Affect Participants’ academic emotions are
closely related to their learning
strategies and academic performances.

Ö

Results of the findings associated with motivation were described into three categories:
participants mastery of learning Mandarin, perceived value of taking the course, and high self-
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 63
efficacy perceptions in high completion rates. In the following section, each of these assumed
causes is presented in tables and figures with further discussions. The relationships among some
of the assumed causes listed under three main categories, knowledge, motivation and
organization, are also summarized.
Participants Mastery of Learning Mandarin
The concept of mastery in goal orientation suggests that a person focuses on self-
improvement by mastering a particular skill or task, with the implication that the person has a
genuine interest in the topic (Linnenbrink & Pintrich, 2003). The assumed cause that the
participants actively seek to master Mandarin Chinese because of business reasons was not
validated here. Only 22.3% of the respondents selected “business opportunity” when asked
“why did you choose to learn Mandarin Chinese?” Surprisingly, 40.2% responses went to
“curiosity” for the same question (Figure 8), meaning that most participants enrolled in this
language MOOC mainly because they wanted to learn Mandarin Chinese out of their curiosity,
or to increase their knowledge, and/or to refresh what they had learned before. Only quite a few
of them are for the purpose of helping them in their work. Studies also presented that
participants enrolled because they were curious about MOOCs. They wanted to experience
taking an online course with thousands of people coming from diverse backgrounds and
interacting with the instructor (Shirvani Boroujeni, Hecking, Hoppe, & Dillenbourg, 2017;
Belanger & Thornton, 2013; Jacobs, 2013; Martin, 2012; Young, 2013).
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 64

Figure 8. Survey participants reasons of leaning Mandarin Chinese

Similarly, an even lower percentage (9.2%) of the total responses fell on the item “career
marketability” and (19.4%) on the item “improve my ability to communicate at work, or with
client” for the question “what was your primary goal/expectation of this course?” However, one-
third of the participants (33.3%) selected “personal hobby” for the same question; another one-
third (30.3%) chose “solid foundation for further Mandarin language learning” (Figure 9).
Analogously, the majority of the participants enrolled in this MOOC, particularly, for their
hobby and personal challenge, such as testing themselves to see if they could master one of the
most commonly spoken, but difficult languages in the world. Insightfully, another major group
of the participants were seeking fundamental bases for advanced personal development, either a
college degree or a professional credential/certificate.
22.3%
1.9%
40.2%
7.8%
14.5%
13.4%
Business
opportunity
School
requirement
Curiosity
Travel
Social
Other
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 65

Figure 9. Interviewed participants reasons for leaning Mandarin Chinese.

While the survey questions were designed to examine assumed causes listed respectively
under three main categories, they were expected to exist with mixed reinforcements among one
from another. Thus, when this survey question along with the assumed cause was analyzed
further by conducting a one-way ANOVA, the statistical data reported that all assumed causes
under the organization category were found to be significant (Table 7), meaning that participants
perceptions towards “primary goal and expectation of this MOOC” were relatively higher
associated with the factors under the organizational category. A significant level means that p
< .005. In the following section, all of the organization assumed causes are discussed separately.

19.4%
3.9%
30%
9.2%
33.3%
3.9%
Improve
my
ability
to
communicate

at
work,
or
with
clients
Marriage/familty
communication
Solid
foundation
for
further

Mandarin
language
learning
Career
marketability
Personal
hobby
Other
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 66
Table 7

ANOVA Analysis for the Factor “Primary Goal and Expectation of this Course” in Knowledge,
Motivation, and Organization Assumed Causes

N Mean SD F Value
Knowledge 696 4.06 1.120 1.545

Motivation 696 4.25 .972 1.896
Organization 696 4.27 1.023 3.687*
Note: *P<0.05

Figure 10, presented in orange bars, also shows a positive correlation with the “solid
foundation for further Mandarin language learning” item in Question 2, Part 2. It shows that
77.2% (538 responses out of 696) of the participants agreed that “they stay with this course
because it covers everything that they are looking for while learning Mandarin.” Yellow bars
present the responses for “I found the learning environment to be encouraging,” and 84.9% (591
out of 696) agreed with the statement. Likewise, the assumed cause “participants receive
emotional support and instant feedback from the instructor, peers, and teaching staff” as an
extrinsic motivation was validated by examining the responses for the survey question “I found
that the feedback helps me adapt to an online learning environment.” Shown in the blue bars,
76.9% (535 out of 696) agreed that the support and feedback motivated them to keep up with the
course and stay with this online learning environment. Therefore, all three assumed causes under
intrinsic and extrinsic motivation categories were validated.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 67

Figure 10. Survey participants perceptions towards solid course design with encourage online
learning environment and supportive feedback.

Perceived Value of Taking Course
Initially, this study assumed that the participants chose this MOOC because the certificate
is issued by edX. A significant amount of research indicated that MOOC participants appreciate
a certificate from an organization which has a good reputation or affiliation with top-tier
universities, e.g. Harvard University and Massachusetts Institute of Technology (Loizzo &
Ertmer, 2016). However, when asked “I enrolled in this MOOC because the certificate is issued
by edX,” more than half (51.7%) of the total responses (360 out of 696) disagreed with this
statement, with 172 participants (24.7%) choosing “strongly disagree.”
Validation of the assumed cause can be supported further by another survey question,
“what made you choose this Mandarin MOOC?” Only 7.5% of the responses selected “it has a
18
35
105
191
179
168
12
28
65
149
247
195
20
38
103
183
217
135
0
100
200
Disagree
Strongly Disagree Slightly
Disagree Slightly
Agree Agree Agree
Strongly
I
stayed
with
this
course
because
it
covers
everything
that
I
am
looking
for
while
learning
Mandarin.
I
found
the
learning
environment
to
be
encouraging.
I
found
that
the
feedback
helps
me
adapt
to
an
online
learning
environment.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 68
good reputation/affiliated with top-tier universities.” It was expected that more than one-third of
the participants, 37.9%, chose “it is free” which validates the MOOC mission. The learning
materials are open to everyone, everywhere. Another 11.6% thought “it works well with my
school/work schedule” which also validates the nature of online learning. However, 22.4% (156
out of 696) chose the reason “it is well-structured with adequate exercises.” A one-way ANOVA
was conducted to further analyze whether the assumed cause that the course has relevance to
individual differences by examining the survey item. The data are shown as Table 8.
Table 8

ANOVA Analysis for the Factor “Well-structured with Adequate Exercises” in Knowledge,
Motivation, and Organization Assumed Causes

Results of the Survey Item: What made you choose this Mandarin MOOC?

N

Mean

SD

F Value
Knowledge It is free 264 3.88 1.123
4.698**
It works well with my school/work schedule 81 4.32 1.032
It is well-structured with adequate exercises 156 4.36 1.024
It is taught in English 67 3.79 1.171
It has a good reputation/affiliated with top-tier
universities
52 4.18 .960
Someone referred it to me 32 3.70 1.428
Other 44 3.89 1.203
Total
696 4.06
1.120

Motivation It is free 264 4.11 .916
6.105**
It works well with my school/work schedule 81 4.44 .977
It is well-structured with adequate exercises 156 4.53 .786
It is taught in English 67 3.89 1.123
It has a good reputation/affiliated with top-tier
universities
52 4.51 .944
Someone referred it to me 32 3.91 1.041
Other 44 4.08 1.282
Total
696 4.25
.972

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 69
Table 8, continued
Results of the Survey Item: What made you choose this Mandarin MOOC?
Organization It is free 264 4.11 .961
4.803**
It works well with my school/work schedule 81 4.29 1.215
It is well-structured with adequate exercises 156 4.55 .819
It is taught in English 67 4.04 1.175
It has a good reputation/affiliated with top-tier
universities
52 4.58 1.033
Someone referred it to me 32 3.87 1.180
Other 44 4.29 1.063
Total 696 4.27 1.023
Note: *P<0.05, **P<0.01  

As can be seen in Table 8, to examine the assumed causes in knowledge, motivation, and
organization, the mean of “it is well-structured with adequate exercises” (M=4.36) was higher
than all the other factors in the knowledge part; the mean of “it is well-structured with adequate
exercises” (M=4.53) was also the highest among all the other factors in the motivation part; only
the mean of “it has a good reputation/affiliated with top-tier universities” (M=4.58) was slighter
higher than the mean of “it is well-structured with adequate exercises” (M=4.53) in the
organization part. In other words, “well-structured with adequate exercises” was the most
influential factor for undertaking this MOOC for the majority of participants. Figure 11 also
demonstrates the responses as percentages in a pie chart.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 70

Figure 11. Survey participants reasons choosing this Mandarin MOOC

A chi-square test (Table 9) was performed to examine the relation between those who
agreed and disagreed with Question 13, Part 2: “I stayed with this course because it covers
everything that I am looking for while learning Mandarin.” Participants who scored above 5
were divided into the “Score High” group; participants who scored below 5 were put into the
“Score Low” group because of the mean (M=4.4).

37.9%
11.6%
22.4%
9.6%
7.5%
4.6%
6.3%
It's
Free
It
works
well
with
my
school/work
schedule
It
is
well-‐structured
with
adequate
exercises
It
is
taught
in
English
It
has
a
good
reputation/affiliated
with
top-‐
tier
universities
Someone
referred
it
to
me
Other
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 71
Table 9

The Crosstabulation Table (Intrinsic Motivation* Relevance to Individual Differences
Crosstabulation)

Intrinsic Motivation* Relevance to Individual Differences Crosstabulation
Intrinsic Motivation Total
Score High Score Low
It is free 106 158 264
It works well with my school/work schedule 49 32 81
It is well-structured with adequate exercises 96 60 156
It is taught in English 30 37 67
It has a good reputation/affiliated with top-tier
universities
32 20 52
Someone referred it to me 11 21 32
Other 20 24 44
Total 345 351 696

Table 9 showed that those respondents whose intrinsic motivation was higher perceived
the value of this course for the following survey items: “It works well with my school/work
schedule,” “It is well-structured with adequate exercises,” and “It has a good reputation/is
affiliated with top-tier universities.” Among the three reasons, “well-structured with adequate
exercises” has the highest percentage. On the other hand, those who had lower intrinsic
motivation perceived this course in a more practical, less self-developed way based on the survey
items: “It is free,” “It is taught in English,” “Someone referred me to the course,” and “Other.”
Table 10 summarizes the ranking of the survey items chose by higher intrinsic motivation group
and lower intrinsic motivation group.
Table 10

Comparison of Higher Intrinsic Motivation and Lower Intrinsic Motivation Participants’
Perceptions Choosing this MOOC

Higher intrinsic motivation Lower intrinsic motivation
It is well-structured with adequate exercises

It is free
It works well with my school/work schedule

It is taught in English
It has a good reputation/is affiliated with
top-tier universities
Someone referred me to the course
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 72
Participants with higher intrinsic motivation weighed more on the course design than it is
offered free. They perceived that the course is solid, well-organized with skillful practices, and it
is also relevant to their own learning needs, personal growth, or self-fulfillment. Furthermore,
they appreciated the time flexibility for taking this course and it has the reputation. On the
contrary, participants who had lower intrinsic motivation were reported to value external and
more passive reasons. The demographic data presented that more than half (492 out of 696) of
the MOOC participants speak English as native or second language, nevertheless, nearly one-
third (29.3%) of the population was Asian, and mainly from Mainland China. They signed up
for this course to learn teaching skills by using English and to meet English speakers who want
to learn Mandarin, while participants in developing countries or who had less access to the
Internet or high-quality education took this course because it is free.
According to Table 11, the relation between these variables was significant, X2 (6, N =
696) = 24.254, p < .001. The results of the “Pearson Chi-Square” row showed that there is a
statistically significant association between the reasons recognized by the higher/lower intrinsic
motivation participants and its relevance to individual differences.
Table 11
The Chi-Square Tests Table
Value df
Pearson Chi-Square 24.254*** 6
N 696
Note: *P<0.05, **P<0.01, ***P<0.00

That is, those who exhibited higher intrinsic motivation respected more of the core
values: “It is well-structured with adequate exercises,” “It works well with my school/work
schedule,” and “It has a good reputation/is affiliated with top-tier universities” than external
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 73
reasons or side benefits: “It is free,” “It is taught in English,” “Someone referred me to the
course,” and “Other.”
Table 12

Key Components Producing High-quality MOOCs and Relevance to Individual Differences Core
Value of a High-quality MOOCs Relevance to Individual Differences

Core Value of a High-quality MOOCs Relevance to Individual Differences
It is well-structured with adequate exercises

Solid foundation for further learning
It works well with my school/work schedule No time and space limit in online
learning

It has a good reputation/is affiliated with
top-tier universities
High-quality course production when
designing a competitive MOOC

Table 12 described key components inherent in producing a high-quality MOOCs and the
relevance to participants’ individual differences. Importantly, it was found that the MOOC
design has to be well-structured with adequate exercises so that participants can have solid
foundation for further learning. No time and space limit in online learning fits participants
school/work schedule, and thus helps create completion rates. Meeting the criteria set by top-tier
universities or organizations maintains the reputation when designing a competitive MOOC.
High Self-Efficacy Perceptions in High Completion Rates
The assumed cause that participants were aware of their own learning and confident in
their academic abilities was validated by the data collected. First of all, 67.1% (467 out of 696)
of the participants stated that “they reflected on their learning progress and adapted their
strategies to assist with their learning.” Further, 71.2% (496 out of 696) respondents were
“confident that they can complete all of the assignments and pass the final exam” (Figure 12).
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 74

Figure 12. Survey participants perceptions, metacognition and self-efficacy.

Students’ motivation is a major factor that affects the attrition and completion rates in the
online courses (Frankola, 2012; Hone & El Said, 2016). MOOC participants were diagnosed to
be more self-motivated (Deng et al., 2017; Zheng, Rosson, Shih, & Carroll, 2015). The strength
of the learner’s self-motivation is influenced by self-regulatory attributes and self-regulatory
processes. The self-regulatory attributes are the learner’s personal learning characteristics
including self-efficacy, which is situation-specific self-confidence in one’s abilities (Bandura,
1977).
Thus, the results indicated that MOOC participants are more self-regulated and highly
motivated with self-discipline. Self-regulated learning requires changing roles of students from
passive learners to active learners. Learners must self-manage the learning process. The core of
self-regulated learning is self-motivation (Salmon et al., 2016; Smith, 2001). Participants with
strong motivation will be more successful and tend to learn the most in online courses than those
with less motivation. High self-efficacy also helps achieve in high completion rates. In addition
0
50
100
150
200
250
Never Very
Rarely Rarely Occassionally Very

Frequently
Always
36
59
134
207
185
75
45
62
93
131
170
166
I
reflected
on
my
learning
progress
and
adapted
my
strategies
to
assist
with
my
learning.
I
am
confident
that
I
can
complete
all
of
the
assignments
and
pass
the
final
exam.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 75
to metacognition and self-efficacy, the assumed cause in the category of affect was validated by
examining the survey question “I feel happy while taking this course, including watching videos,
working on exercises, and interacting with peers.” The data showed that 86.1% (600 out of 696)
of the participants agreed with the positive items (Figure 13).
Emotions are believed to be ever-present in academic settings and deeply affect students’
engagement and academic performance (Broadbent, 2017; Zepke, 2017; Pekrun & Linnenbrink-
Garcia, 2012). Results showed that academic emotions, (e.g. enjoyment of learning, pride of
success, or test-related anxiety) and social emotions are significantly related to learning
strategies, motivation, self-regulation, and academic achievement (Ben-Eliyahu et al., 2017;
Broadbent, 2017; Zepke, 2017; Pekrun, Goetz, Frenzel, Barchfeld, & Perry, 2011). Students’
perspective towards learning experience and classroom instruction intensely associated with
emotions which are directly linked to their academic achievement (Alexander & Grossnickle,
2016). Findings also show relatively high inter-correlations between emotion relations and
achievement outcomes in different academic domains. For example, most of the students’
academic enjoyment in language classes are higher than in mathematics classes (Benesch, 2017;
Goetz, Frenzel, Hall, & Pekrun, 2008). In line with this assumption, the result explains that the
academic emotions were closely related to their learning strategies and academic performances,
which is also indicative of a happy organization and a happy culture. Equally important,
employees are reported to be happier in an adhocracy cultural setting than in rigid bureaucracy
(Lund, 2003). This result is elaborated in the next organizational causes section.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 76

Figure 13. Survey participants academic emotions are closely related to learning strategies and
academic performances.

MandarinX language MOOCs attract a diverse audience from all age groups and 202
countries around the world. Motivations vary among tens of thousands of participants. How
participants perceive the learning experience in MOOCs that differs from that in physical
classroom settings and how they navigate through course content are major components related
to motivations (Çetinkaya-Rundel, & Canelas, 2017; Hew & Cheung, 2014; Milligan &
Littlejohn, 2017; Shapiro, Lee, Roth, Li, Stich & Reeves, 2017). There are intrinsic value and
extrinsic goal orientation motives for MandarinX participants to sign up for the course. They
have substantial interests to learn Mandarin and the course has relevance to individual
differences. Also, some of them learn Mandarin Chinese due to professional needs for work or
travel to Mandarin-speaking countries. Meanwhile, they receive emotional support and instant
feedback from the instructor, the teaching staff team through online discussion, and peers in the
community (Chew et al., 2017; Frank, 2012; Levy, 2011; Ko & Rossen, 2017).
One major group of the participants are retired professors and CEOs, who learn Mandarin
Chinese language out of curiosity (Jacobs, 2013; Kirschner, 2012; Martin, 2012; Young, 2013)
and their own fulfillment and personal growth (Breslow et al., 2013). Another major group of
10
24
62
127
234
239
0
100
200
Definitely
Not Probably
Not Possibly Probably Very
Probably Definitely
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 77
the participants join the course because it is free unless they would like to have a certificate.
Although some studies showed that participants want to get as many course certificates as they
can, especially issued from prestigious universities (Young, 2013), others pointed out that many
participants do not seek credits or credentials (Fini, 2009; Kolowich, 2013). MandarinX
participants did not appreciate a certificate from an organization which has reputation or
affiliation with top-tier universities; preferably, the quality of the MOOC that an organization
produced and conducted.
Findings also indicated that participants’ satisfaction levels taking MandarinX MOOCs
are mixed. Their academic emotions are closely related to learning strategies and academic
performances. Positive participants’ perceptions include metacognition and self-efficacy. They
are confident with their academic abilities and are aware of their own learning.
Results and Findings for Organizational Causes
Six organizational causes were identified in Chapter Three and have been validated or not
validated based on data collected from the survey. The results from this are articulated in Table
13 and discussed further.
Table 13
Validated Assumed Organization Causes

Category

Assumed Cause

Validated
Not
Validated
Cultural
Setting
Interactive language course design Ö
Social networking activities Ö
Opportunities to interact with diverse
groups of learners

Ö
Time flexibility and convenience Ö

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 78
Table 13, continued
Cultural
Models
Teaching presence influences learners’
perceptions of the course

Ö
Positive culture and happy organization Ö

Four central pieces of elements associated within a happy organization were: pleased
participants, role models within the organization, social networking helps enhance engagement,
and flexibility in online learning courses. In the last section, each of these assumed causes
categorized into the four elements are discussed individually with tables and figures. A
synthesized analysis among the three main categories, KMO, are also summarized.
Pleased Participants
In an online learning environment, specifically language courses, it is easy to neglect
interactive and engaging elements when it comes to course design. It is, therefore, unsurprising
that the first and last organizational assumed causes “interactive language course design” and
“positive culture and happy organization” were validated by survey questions distributed during
the data collection period. The results indicated that 86.5% of the participants found “the course
design on the platform to be interactive and engaging.” Another 92.1% of them were “satisfied
with the overall course design.”
Furthermore, when responding to “What did you find most engaging while taking this
course?” The data showed that 75% of the total responses noted “high-quality videos;” more than
half, 62.6%, of the respondents said, “practical exercises and peer assessment;” slightly less than
half, 41.8%, listed on the third, was “explicit guidance (including weekly newsletter and cultural
notes videos).” Participants could check all items that applied to this survey question (Figure
14).
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 79

Figure 14. Participants found the most engaging part while taking the course.

Again, a Pearson product-moment correlation coefficient was computed in order to assess
the relationship among assumed causes under the three categories: KMO and the overall
satisfaction level of this course. There were positive correlations among the four variables.
Table 14 presents the statistical results. Overall, there was a strong, positive correlation among
all assumed causes in all categories. Participants’ satisfaction level was correlated with all the
assumed causes in the KMO categories. Among the three, organization was highly correlated
with the satisfaction level of participants (r = 0.648, n = 696, p = 0.001). Also, assumed causes
under the organization category were highly connected to those under the motivation category (r
= 0.721, n = 696, p = 0.001).

0 100 200 300 400 500 600
Other
Family/friend
encouragement
Technical
support/course
content
assistance
Explicit
guidance
(including
weekly
…
Online
forum
discussions
Practical
exercises
and
peer
assessment
High
quality
videos
31
94
168
291
136
436
522
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 80
Table 14

Pearson Product-moment Correlation Analysis for the Factor “Overall Satisfaction Level of the
Course Design” in Knowledge, Motivation, and Organization Assumed Causes

Knowledge
Assumed
Causes
Motivation
Assumed
Causes
Organization
Assumed Causes
Knowledge
Assumed Causes
1 .574
**
.470
**

Motivation
Assumed Causes
.574
**
1 .721
**

Organization
Assumed Causes
.470
**
.721
**
1
Satisfaction
Level
.439
**
.621
**
.648
**

N 696 696 696
Note: ** Correlation is significant at the 0.01 level (2-tailed)

Role Models Within the Organization
According to Garrison, Anderson and Archer (2000), teaching presence is composed of
design and organization, facilitating discourse, and directing instruction (Huang et al., 2017;
Boettcher & Conrad, 2016; Swan, Shea et al., 2008). Instructional course design, including the
interaction and evaluation, is to plan and design the course structure and process. Facilitating
discourse refers to managing the discussion, namely, reviewing and commenting on student
discussion. Directing instruction is that the instructor as subject matter expert, provides
leadership through the sharing of expertise (Anderson, Rourke, Garrison & Archer, 2001;
Shepherdet al., 2016). Research has shown that teaching presence impacts participants’
perceptions of higher levels of learning (Evans, Ward & Reeves, 2017; Kanuka, 2011; Morris,
2011; Shea, Li, & Pickett, 2006) and their success in online courses (Arbaugh, 2010; Cleveland-
Innes & Fung, 2010; Garrison, Kupczynski, Ice, Wiesenmayer & McCluskey, 2010).
Reports collected in the process of data analysis showed that teaching presence
influenced learners’ perceptions of this course. This supports the assumed cause that there are
role models within the organization who have successfully underpinned MandarinX high
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 81
completion rates on the six-week long MOOCs. Pedagogy, teaching styles, and the most
important, the instructor’s presence and attitude are all indicated by the research as the top
criteria for online learning courses and environment (Boettcher & Conrad, 2016; Kanuka, 2011;
Morris, 2011; Wang & Huang, 2017; Shapiro et al., 2017; Shea, Li, & Pickett, 2006). Almost
half (43.1%) of the participants strongly agreed that “the instructor’s manner of speaking and
presentation skills are fluent, relaxed, and natural.” Likewise, nearly half (43.7%) of them
strongly agreed that “the instructor’s attitude kept them interested in watching the videos”
(Figure 15). The organizational assumed cause that there are role models within MandarinX
course design and teaching presence therefore was validated by the data collected.

Figure 15. Survey teaching presence influences participants perceptions of the course.

However, when responding to “the videos were very much like just being in the
classroom, where the teacher is talking and writing on the board,” not as many participants
strongly agreed with this statement as they had with the previous survey questions discussed
above. Only 23.4% of them strongly agreed; less than one-third of them, 28.4%, agreed; about
0
100
200
300
400
Strongly

Disagree
Disagree Disagree

Slightly
Agree

Slightly
Agree Strongly

Aagree
10 14
46
119
207
300
9 10
46
123
204
304
The
instructor’s
manner
of
speaking
and
presentation
skills
are
fluent,
relaxed
and
natural.
The
instructor’s
attitude
kept
me
interested
in
watching
the
videos.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 82
the same number of them, 26.4%, agreed slightly. It was expected that the percentage of
agreement would be similar to that of the previous two survey questions, because this survey
question was also designed to validate whether the instructor’s teaching presence would
influence participants’ perceptions towards the course or not. As discussed above, teaching
presence includes methods that an instructor utilizes to promote a quality online environment and
facilitate an effective online learning experience (Boettcher & Conrad, 2016; Evans et al., 2017;
Huang et al. 2017; Kangas et al., 2017; Shepherd et al., 2016; Wang & Huang, 2017).
Although the extent of agreement was not as high as the previous two survey questions,
the result was actually promising. The reason is because videos were shot in everyday scenarios
in real settings which aimed to let learners have an authentic learning experience, rather than
learning the language and memorizing words in a classroom dominated by a teacher who wrote
everything on the board. Nevertheless, the question item itself might be a bit confusing as
designed.
Social Networking Helps Enhance Engagement
It should not be surprising that the more that online learners can be exposed to social
networking activities and interact with diverse groups of learners, the higher the engagement and
completion rate. Nevertheless, the two assumed causes: social networking activities and
opportunities to interact with diverse groups of learners are validated and not validated
respectively by the data collected. The data showed that 71.3% of participants agreed that “the
broadcasting and online meetings led by the instructor were beneficial for them to learn
Mandarin.” The broadcasting was conducted through the MandarinX Facebook page and the
weekly online meeting was mainly held via Google Hangout or Skype. However, only 50.4% of
the participants thought “it was helpful for them to use the discussion forum to interact with
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 83
peers and support each other.” In other words, 49.6% of them did not regard the online
discussion forum, provided and designed by the edX platform, as a handy and timely way to
communicate with each other and the whole community (Figure 16).

Figure 16. Survey participants use of social networking events and opportunities to interact with
diverse groups of learners

Although Question 5 “How did you study while taking this course?” was examined
earlier in Part 2 to analyze the correlation between social networking and the high completion
rates of the course, it was still a surprise to see that 84.9% was checked “alone” (Figure 17).
0
50
100
150
200
Strongly

Disagree
Disagree Disagree

Slightly
Agree
Slightly Agree Strongly
Agree
47
40
113
187
176
133
85
98
162
167
105
66
It
was
beneficial
for
me
to
learn
Mandarin
through
live
broadcasting
and
online
meetings
led
by
the

instructor.
It
was
helpful
for
me
to
use
the
discussion
forum
to
interact
with
peers
and
support
each
other.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 84

Figure 17. Survey participants learning habits while taking this course.

By analyzing participants’ perceptions of social networking events and opportunities to
interact with diverse groups of learners and their learning habits in Figure 16 and Figure 17,
three reasons were also inferred from the findings in the previous paragraphs and knowledge and
motivation sections. Table 15 summarizes the three reasons of MOOC participants’ perspectives
towards social networking and their course engagement.
Table 15

Three Reasons of MOOC Participants’ Perspectives Towards Social Networking and Their
Course Engagement

Summary of MOOC Participants Perspectives of Social Networking and
Course Engagement
MOOC participants tended to be very self-directed

Busy schedules prevented them from interacting with other leaners in different
time zones

Guidance from the instructor and the interaction with the instructor were
valued more than online forum administered by the teaching assistants and
technical team

71
69
45
31
74
591
35
41
0 100 200 300 400 500 600
Online
study
group
members
Language
buddy
Family
members
School
classmates
Random
friends
Alone
I
did
not
study
Other
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 85
Initially, the study estimated to see that social presence was validated by the collected
data. Social presence is that students interact affectively with peers within an online learning
environment (Choy & Quek, 2016; Swan, Garrison, & Richardson, 2009). Given the massive
numbers of participants in a MOOC, it is more challenging to achieve social presence and to
connect at a personal level. Thus, participants’ perceived teaching presence had a positive
impact on their constructive and interactive engagement of the course. An independent-samples
t-test (Table 16) was conducted later to compare the “alone” (84.9% of the total) participants
who scored high and low in Question 23 “the instructor’s manner of speaking and presentation
skills are fluent, relaxed and natural” and Question 24 “the instructor’s attitude kept me
interested in watching the videos” in Part 2 conditions (M=5.02, SD=1.083).
Table 16

Sample Descriptives Using T-test for the Factor “Teaching Presence” in “Study Alone”
Participants Group

N M SD t-test
Score Low 188 3.57 .836 -13.260***
Score High 508 4.56 .866

There was a significant difference in the scores for those who perceived low in the
teaching presence (M=3.57, SD=0.863) and those who perceived high in the teaching presence
(M=4.56, SD=0.866) conditions; t (696) = -13.260, p = .000. The result suggested that the
instructor’s manner of speaking, presentation skills, and attitude really do have an effect on
MOOC participants. Findings presented in previous paragraphs and sections demonstrated that
instructor strategies in a MOOC design significantly impact MOOC participants’ perspectives,
attitudinal change, and the success of the course itself. Focusing on the establishment of a
collaborative learning community can help inform future instructional design and instruction of
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 86
MOOCs in general. Specifically, the results suggested that when participants are interested in
watching the videos, their completion rate for MOOCs increases.
Flexibility in Online Learning Courses
One stating feature of MOOCs is that they allow considerable flexibility in organizing
one’s own learning, particularly for people who do not have the time to learn a subject matter
(Thai, et al., 2017; Montoya & Hernández, 2016; Loya, Gopal, Shukla, Jermann, & Tormey,
2015). Coursera, a MOOC platform, described itself as “learn without limits” for those who are
busy with many demands on their time (Coursera, 2013). MOOCs allow much greater flexibility
than in traditional learning that provides school calendars, timetables, and deadlines. MOOCs,
on the other hand, allows participants to submit assignments and complete courses at a time and
place that is more convenient to their own schedule (Montoya & Hernández, 2016; Levi, 2013).
Although flexibility is viewed as a benefit for participants, it could be a challenge if participants
did not manage well the flexibility that MOOCs provide. Therefore, the assumed cause was that
the course provides flexibility and the participants benefit from that. More than half (68.6%) of
the participants agreed that the course allowed them to re-learn the concepts that they did not
understand previously in the other language programs or institutions.
Furthermore, video length is the key indicator of MOOC participants’ engagement.
Findings indicate that shorter videos are much more engaging, and median engagement time is at
most 6 minutes, regardless of total video length (edX Insight, 2016; Guo, Kim, & Rubin, 2014).
Participants seldom make it to the halfway point if videos are longer than 9 minutes. They also
engage less even if there are assessment exercises followed by longer videos. MandarinX course
design team and video producers take better advantage of the online video format. The six-week
long course consisted of 8 to 10 video clips (with a median time of 6 minutes) for the one hour
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 87
lecture, every week. Depending on the difficulty of the course level and learners’ proficiency of
the language, the required time for completing assignments and quizzes varied. On average,
participants could expect to spend around three hours or less on course related activities per
week. The survey showed that nearly half (48.9%) of the participants spent less than three hours
on average with this course every week (Figure 18). The time that participants devote
themselves into the weekly course content meet the criteria of ideal online video format in
MOOCs.

Figure 18. Survey participants time spent on average with this course every week.

As discussed previously in the knowledge, motivation, and organizational assumed
causes sections, a high-quality MOOC production involves various collaborations among many
teams. All assumed causes are mutually reinforcing. Thus, when examining hours that
participants spent while taking the course by conducting a One-way ANOVA, the results showed
that all the knowledge (M=4.05), motivation (M=4.25), and organization (M=4.26) assumed
causes are significantly related with one another (Table 17). That means no matter how many
48.9%
37.4%
9.5%
3.2%
0.1%
1%
Less
than
3
hours
3-‐5
hours
6-‐9
hours
10-‐12
hours
13-‐15
hours
More
than
15
hours
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 88
hours participants devoted in taking the course, they perceived the same towards this MOOC
production: quality delivery.
Table 17

ANOVA Analysis for the Factor “Hours Spend on Average with this Course Every Week” in
Knowledge, Motivation and Organization Assumed Causes

N Mean SD F Value
Knowledge 696 4.05 1.121 9.722***
Motivation 696 4.25 .973 19.612***
Organization 696 4.26 1.022 10.602***
Note: *P<0.05, **P<0.01, ***P<0.00

Much of the research identified that interaction is the key element in high-quality
MOOCs (Hood & Littlejohn, 2016; Fisher, 2012; Khalil & Ebner, 2013; Levy & Schrire; 2012;
McAuley, Stewart, Siemens, & Cormier, 2010; Waard, 2011). Milligan and Littlejohn (2017)
also stated that only the poor course design would make participants feel isolated and depressed.
The feeling of isolation among MOOC participants can be solved by focusing more on social
interactions when designing courses. MandarinX MOOC participants exhibited a high level of
satisfaction with the overall course design, as well as acknowledging that the course design on
the platform to be interactive and engaging.
Interactions in MOOCs helped students to develop their own ideas, express themselves,
establish a presence, and make thoughtful long-term relationships (Chew et al., 2017; Salmon et
al., 2016; Wang & Huang, 2017). Prompt and personalized communications with the instructor
particularly have a significant impact on MOOC participants’ satisfaction (Choy & Quek, 2016;
Kangas et al., 2017; Thai et al., 2017;). The MandarinX organization triumphed over most of
MOOCs’ failure by offering prompt, clear feedback from the instructor and the teaching
assistants, hosting social networking activities, and a virtual weekly office hour. Despite the fact
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 89
that interactions were time-consuming and difficult to arrange, especially with larger class sizes
in different time zones, the teaching staff team within the MandarinX organization managed a
way to sustain and thus create a positive culture and high-performing organization.
Furthermore, the complexities of instructor roles in MOOCs is critical, given the massive
numbers of students and their diverse background and goals. Role models within the MandarinX
organization succeeded in playing the roles as a distant lecturer with fluent, relaxed, and natural
presentation skills, a mentor whose attitude kept the participants interested in watching the
videos, and a facilitator within the network acting as a co-participant.
Summary of Validated Causes
The research confirmed several of the KMO influences identified in the literature review.
The MOOC participants in MandarinX online language environment are highly engaged with the
interactive courses and social networking activities led by the instructor. Participants’
perceptions towards the course are strongly influenced by the teaching presence, even though
opportunities to interact with diverse groups of learners seem less favorable. The organizational
culture also affects the learning ambiance and participants’ willingness to persist and thus
complete the course.
Participants were also found to be rich in factual, procedural, and metacognitive
knowledge and favor the time flexibility and convenience of learning Mandarin Chinese online.
However, participants lack conceptual knowledge in differentiating varying language usages in
Mandarin-speaking regions. Regarding motivation, MandarinX participants have a substantial
interest in learning Mandarin and are confident with their academic abilities and their own
learning. They feel the course has relevance to their individual goals. Thus, the KMO
influences for MOOC participants are likely best addressed by ensuring stakeholders are highly
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 90
motivated and engaged with an interactive learning environment and well-structured course
design. Chapter Five presents implementations for designing future MOOCs based on empirical
evidence.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 91
CHAPTER FIVE: FINDINGS, IMPLEMENTATION, AND EVALUATION
The purpose of the study was to understand the reasons why participants persisted in a
six-week long MOOC offered by MandarinX. Assumed causes within KMO dimensions were
adapted from a gap analysis model proposed by Clark and Estes (2008). The study validated the
assumed factors, which were based on a survey questionnaire completed by 696 participants. A
total of 17 assumed causes were validated after computing the data; this was done by use of the
SPSS statistical tool. This chapter is designed to identify and extend those factors leading to this
MOOC’s success. The next section of this chapter, validated causes selection and rationale,
discusses the assets, the recommended solutions, an implementation plan, and an evaluation plan
based on the Kirkpatrick (2006) model. The chapter then concludes with a discussion of the
limitations of the study and a discussion of possible future research in the field.
Validated Causes Selection and Rationale
In Chapter Four, a total of 17 assumed causes were validated. The validated causes stem
from KMO factors. In order to effectively answer the questions as to what MOOC participants
perceived about the quality of the courses, including participants being high performers and the
organization performing better than the average, the rest of the study presents the validated
causes and related recommended practices that could be adapted by other universities or
institutions to design courses well for the sake of improving the learning experience and
completion rates in the MOOC community. A list of validated causes is displayed in Table 18.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 92
Table 18
Validated Causes Summary
Gap Analysis Dimension Validated Causes
Knowledge Factual
Participants know reasons which prevent them from completing
the course.
Procedural
Participants have the knowledge necessary to navigate the
curriculum and the LMS.

Metacognitive
Participants need to know where to find supplementary materials
to assess their learning.

Motivation Intrinsic Value
Participants have substantial interests in learning Mandarin.
Goal Orientation
Participants learn Mandarin due to professional needs for work or
travel to Mandarin-speaking countries.
Intrinsic Motivation
The course has relevance to individual differences.
Extrinsic Motivation
Participants receive emotional support from faculty members.
Extrinsic Motivation
Participants have instant feedback from the instructor, peers, and
teaching staff.
Cost/Benefit
Participants can audit the courses for free unless they would like
to have a certificate.
Metacognition
Participants are aware of their own learning.
Self-Efficacy
Participants are confident in their academic abilities.
Affect
Participants’ academic emotions are closely related to their
learning strategies and academic performances.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 93
Table 18, continued
Organization Cultural Setting
Interactive language course design
Cultural Setting
Social networking activities
Cultural Setting
Time flexibility and convenience
Cultural Model
Teaching presence influences learners’ perceptions of the course
Cultural Model
Positive culture and happy organization

Findings for Knowledge Causes
The survey results which resulted from the use of the SPSS statistical tool validated the
assumed knowledge causes that contributed to the success of the MOOC performance and
completion rates. Participants were able to identify reasons which prevented them from
completing the course. Procedurally, participants had the knowledge necessary to navigate the
curriculum and access course materials on the MOOC platform. Lastly, participants had the
ability to find supplementary materials that helped enhance their learning.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 94
Table 19
Validated Causes under Knowledge Category and Findings Summary
Validated Causes Findings
Factual
Participants know reasons which prevent
them from completing the course.
• Participants possess knowledge which
allowed them to complete the course
successfully.
• It is fundamental that they be
provided with the factual knowledge
of reasons in successfully completing
courses in the MOOC learning
environment.
Procedural
Participants have the knowledge necessary to
navigate the curriculum and the LMS.
• It is a process of high performing that
is directly connected to academic
achievement.

• Knowledge of how to explore
required course materials and
assessments will ensure a higher
probability of course completion.

Metacognitive
Participants need to know where to find
supplementary materials to assess their
learning.
• Participants have the metacognitive
knowledge to anticipate and
potentially solve problems
encountered during the course.
• The acquisition of this metacognitive
knowledge will enhance participants’
ability to facilitate their use of
strategic methods and achieve the
goal of completing the course.
successfully.

Factual
MOOC participants in this study demonstrated the factual knowledge of knowing the
reasons which caused them to fall behind, and even prevented them from completing the course.
Factual knowledge refers to facts which are basic to specific disciplines and includes topics that
one must be familiar with in order to understand and function in a given field (Rueda, 2011). In
this regard, participants across various demographics knew that reasons which hinder the
progress of their learning would play a part in completing the course. This finding suggested
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 95
that participants possess the knowledge which allowed them to complete the course successfully
(Clark & Estes, 2008).
For other organizations and MOOC participants to achieve similar results, it is
fundamental that they be provided with the factual knowledge of reasons in successfully
completing courses in the MOOC learning environment (Clark & Estes, 2008). For an
instructor-led track, instructors and course development teams could lead course overview
sessions with clear guidance and deadlines for submitting exercises and the final examination at
the beginning of the six-week long course in order to ensure that participants have the factual
knowledge of possible reasons that could impede them from completing courses. Even if it is for
a self-paced track, the relative factual knowledge needed for successfully completing the course
can be embedded throughout the courseware on the MOOC platform (e.g. course syllabus and
pre-recorded course overview videos). Keeping participants informed of the knowledge they
need to know will help them achieve the goal of completing the course successfully (Clark &
Estes, 2008).
Procedural
In this study, participants demonstrated that they have adequate knowledge to navigate
the curriculum and course materials on the MOOC platform. The term knowledge skill refers to
procedural knowledge, simply meaning the ability to do something (Rueda, 2011). In the study,
participants knew where to locate the course materials and assessment quizzes that they needed
to complete in the course. This finding is crucial, because it is a process of high performing that
is directly connected to academic achievement (Clark & Estes, 2008). Being able to know how
to explore the curriculum requires knowledge of specific skills (Rueda, 2011).
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 96
Other instructors and the course development teams can employ this finding through hosting
trainings to help equip MOOC participants with the skills of utilizing the learning tools on the
MOOC platform which are needed to access the course materials and complete exercises (Clark
& Estes, 2008). Knowledge of how to explore required course materials and assessments will
ensure a higher probability of course completion. The training will also enable MOOC
participants to increase their efficiency of learning and to evaluate their own progress.
Metacognitive
Participants in this study demonstrated that they knew where to find supplementary
materials to assess their learning strategically. Metacognition refers to the ability to think about
thinking, rather becoming aware of one’s own thinking processes (Rueda, 2011). In the study,
the participants expressed confidence that they were capable of searching extensive learning
materials to augment their learning progress. They, therefore, had the metacognitive knowledge
to anticipate and potentially solve problems encountered during the course (Clark & Estes,
2008).
This finding is transferable to other instructors and the course development teams that
furnish the MOOC participants with factual, conceptual, theoretical, and strategic knowledge of
how to evaluate their own progress in the online learning environment (Clark & Estes, 2008).
The acquisition of this metacognitive knowledge will also enhance participants’ ability to
facilitate their use of strategic methods and achieve the goal of completing the course
successfully (Clark & Estes, 2008).
Findings for Motivation Causes
According to Clark and Estes (2008), validated factors should be prioritized depending
on the needs of the organization. Therefore, the MOOC participants’ intrinsic value, motivation,
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 97
and goal orientation are addressed first among motivational factors. The majority of the
participants have substantial interests in learning Mandarin due to professional needs for work or
travel to Mandarin-speaking countries and the course has relevance to individual differences
(Rueda, 2011). The survey results also validated the sources of extrinsic motivation among the
participants, such as the emotional support and instant feedback from the instructor, peers, and
teaching staff. The practice of allowing participants to audit the courses for free unless they
desire a certificate was also validated. Metacognition, which refers to the participants’
awareness of their own learning proved to be motivational causes. Self-efficacy, which relates to
participants’ confidence with their academic performances was also proved to be motivational
causes. Finally, participants’ academic emotions were found to be closely related to their
learning strategies and academic performances, which were defined as affect. These
motivational factors are discussed in sub-sections later on in this paper. Validated causes and
findings are grouped into three elements and shown in Table 20.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 98
Table 20
Validated Causes under Motivation Category and Findings Summary
Validated Causes Findings
Intrinsic Interest in the Subject
Participants have substantial interests to learn
Mandarin.
Participants learn Mandarin due to
professional needs for work or travel to
Mandarin-speaking countries.
The course has relevance to individual
differences.
• Participants demonstrated substantial
interests in learning Mandarin
Chinese and responded that the course
is relevant to individual differences.
Extrinsic Benefit of the Course
Participants receive emotional support from
faculty members.

Participants have instant feedback from the
instructor, peers, and teaching staff.

Participants can audit the courses for free
unless they would like to have a certificate.

• Receiving emotional support and
instant feedback from the instructor,
peers, and teaching staff acts as an
additional incentive for MOOC
participants to complete the course.
Positive Attitudes toward MOOCs
Participants are aware of their own learning.

Participants are confident with their
academic abilities.

Participants’ academic emotions are closely
related to their learning strategies and
academic performances
• Participants who were satisfied with
the degree of participation
demonstrated higher levels of learning
strategies (e.g. evaluating their own
learning) and academic performances
(e.g. showing confidence with their
abilities).

Intrinsic Interest in the Subject
The finding focuses on the intrinsic motivation in learning the subject of the MOOC
participants. It is believed that motivational factors that stem from the intrinsic value motivate
people to perform better in a particular task (Barba, Kennedy, & Ainley, 2016). MOOC
participants demonstrated substantial interests in learning Mandarin Chinese and responded that
the course is relevant to individual differences.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 99
Extrinsic Benefit of the Course
The fact that almost all MOOCs are free is one of the main motives for participants to
enroll (Instructure, 2013). Practical use, which has been viewed as utility value, is the belief that
people are more likely to increase commitment to a task because of the benefits associated once
completed (Clark & Estes, 2008). However, receiving emotional support and instant feedback
from the instructor, peers, and teaching staff acts as an additional incentive for MOOC
participants to complete the course.
Positive Attitudes Toward MOOCs
Participants’ attitudes result from the satisfaction derived from taking MOOCs. A senior
participant who completed ‘Basic Mandarin Series Level One to Three’ and ‘Business Chinese’
courses, commented that taking this MOOC was like taking a real class with weekly live
meetings, assignment deadlines, quizzes, a midterm, and a final. The importance of the
attachment to values can lead people to adopt the action and persist, regardless of distractors
(Clark & Estes, 2008). Participants who were satisfied with the degree of participation
demonstrated higher levels of learning strategies (e.g. evaluating their own learning) and
academic performances (e.g. showing confidence with their abilities).
Findings for Organization Causes
The assumed organizational causes were validated by the survey results, which was
computed through use of the SPSS statistical tool. The MandarinX organization has established
a positive cultural setting around happy models. Teaching presence, including the interactive
language course design and adequate social networking activities within the organization,
influenced participants’ perceptions towards this MOOC and motivated them to reach targeted
goals: completion of highly engaged and complete courses. Validated assets are discussed in the
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 100
following paragraphs by reviewing current relevant literature. Table 21 summarizes the
relationship between validated causes and findings.
Table 21
Validated Causes under Organizational Category and Findings Summary
Validated Causes Findings
Cultural Setting
Interactive language course design
Social networking activities
Time flexibility and convenience

• Participants in the study valued that
the courses were designed
interactively with social networking
activities.
• The organization offered the courses
with time flexibility and convenient
access so that participants could meet
their goals.
Cultural Model
Teaching presence influences learners’
perceptions of the course
Positive culture and happy organization
• The instructor’s attitude affected
participants’ learning outcomes of this
course.
• A positive attitude with a joyful
atmosphere within the organization
increases the likelihood for effective
learning.
• All teams have a clear vision and an
effective way to communicate, which
thus led to a culture with positive
attitude and happy working ambiance.

Cultural Setting
The finding is grounded in the organizational theory of cultural settings. Cultural settings
are visible and considered to be physical settings where the organization’s values and beliefs are
displayed (Gallimore & Goldenberg, 2001). The MOOC participants in the study valued that the
courses were designed interactively with social networking activities. Furthermore, the
organization offered the courses with time flexibility and convenient access so that participants
could meet their goals.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 101
Cultural Model
Cultural models refer to beliefs and values that are generally invisible, often involving
values that are relative (Gallimore & Goldenberg, 2001). MOOC participants in the study stated
that the instructor’s attitude affected their learning outcomes in this course. They believed that a
positive attitude with a joyful atmosphere within the organization increases the likelihood for
effective learning. Despite having different teams across various fields, all of the teams had a
clear shared vision and an effective way to communicate, which thus led to a culture with
positive attitude and happy working ambiance (Clark & Estes, 2008).
Implementation Plan
The gap analysis framework provides a manageable method of exploring the causes of
the MOOC participants’ knowledge, motivation, and organizational gaps. However, in the
development of an implementation plan, all of the findings addressing each validated gap should
work together as part of a whole. The completion rate (86%) within the organization is
significantly higher than the average completion rate, which ranges from 5% to 10% (Davis et
al., 2017; Höfler et al., 2017) for most MOOC courses. The phenomenon is even more notable
when taking into consideration that according to Ho, Reich et al. (2014), 55.8% of MOOC
participants access less than half of their course material. Granted, some MOOC courses have
higher completion rates than others, and with such a large gap in performance, one must ask
questions such as: what drives MOOC success? In particular, what promising practices are
needed to reduce the attrition rate of MOOC courses in general? Also, what promising practices
are needed in order to deliver effective online language teaching for foreign languages? Specific
criteria need to be considered when designing and implementing MOOCs in general, with
specialized approaches to teaching for language acquisition.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 102
Based on the research findings presented in Chapter Four, the policies to be implemented
within MandarinX organization are (1) identify instructional design suggestions to advance the
MOOC community by optimizing educational resources and saving on the cost of logistics
(Funieru & Lazaroiu, 2016), and, ultimately, (2) expand the scope of the coursework offered at
universities by appropriately incorporating MOOCs in flipped classrooms, blended learning, and
teacher training. MOOC teaching is more than the instructor, the course delivery platform, and
the participants. A successful MOOC consists of a number of “invisible” systems and actors. A
model (Figure 19) was created to illustrate the multiple collaborations and subsystems that are
necessary to support MOOC instructional design, and thus expand the scope of the coursework
offered at universities, noting that these systems are rarely implemented as well thought out
schemes. The MOOC scheme represents highly engaged learning and significant completion
rates that support the entire process of MOOC teaching. It is instructor-centered because MOOC
instructors need to complete three stages for each MOOC they teach: preparation, execution, and
implementation. In the following sub-sections, these stages and their associated tasks are
discussed in more detail.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 103

Figure 19. MOOC Scheme

Stage 1: Preparation
A MOOC instructor initiates a course by brainstorming with the course coordinator and
the course development team. A teaching proposal is then established to be aligned with
university requirements, strategic plans and students’ needs. After obtaining permission, the
instructor and the course development team design the MOOC curricula by preparing teaching
materials. The instructor and the course coordinator communicate with the video production
team to decide which format or layout can help teaching pedagogy for online courses. This stage
usually takes a long period to organize appropriate content into 4 to 6 weeks of lectures that are
clear and concise, especially since the MOOC audience comes from an extremely wide range.
Stage 2: Execution
In the second stage, an instructor’s primary duty is to launch the MOOC and ensure that
it proceeds as planned. In order to run the course more effectively, the project manager must
monitor the progress of the video production team to ensure the course is on time and on budget.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 104
Every week, the technology coordinator needs to upload the lectures, assignments, quizzes and
announcements each week created by the course development team. The technology coordinator
is also required to solve problems (e.g., correct broken links of course materials, deal with
students who encounter technical issues while viewing courses). During this stage, the instructor
and the course coordinator also need to collaborate with teaching assistants and the community
facilitator to answer questions raised by students in a timely and appropriate manner and host
online synchronous meetings. A community facilitator is usually a volunteer or a star student
from the previous cohorts that the instructor has nominated.
Stage 3: Implementation
Feedback and comments from forum discussions, social media posts, and even student
blogs received during the MOOC execution are collected and documented for the next MOOC’s
implementation, making this process iterative in nature and ensuring that MOOCs offered by
instructors align with university goals.
Collaboration across a larger team is an important and necessary part of a successful
MOOC administration. However, many of the current MOOC design systems only create
collaborations for student support and the platform due to the fact that researchers have been
emphasizing the concept of student-centered MOOC implementation and design. From this
model, instructors were highlighted as they needed to interact with a range of collaborators. This
includes participants’ engagement outside the MOOC platform (e.g. social networking events
and virtual meetings with the instructor and the course development team), as well as
interactions with other actors in the overall system. For instance, the instructor collaborates with
teaching assistants and the technical coordinator to run virtual office hours or make the social
networking events attractive. The instructor collaborates the most with the action item
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 105
individuals or heads (e.g. for curriculum design, lecture shooting, etc.) during the preparation and
execution phases. Collaborations take place often online (using tools like social media or virtual
meetings), but they sometimes occur in offline shooting studios or places with intensive
schedules to save on the cost of logistics.
Organizational Environment and Features Relevant to Implementation
The MandarinX organization consists of approximately 18 people, of which 90% are
based in different locations around the world and whose professional backgrounds are diverse.
Staff members come from different Mandarin-speaking areas, which helps enhance the
collaboration regarding the cultural perspectives on language usages. Learning how to support
each other as staff members is crucial, as the team is small and there is no hierarchy involved.
Because of the small team, everyone has been trained to cover one another’s job duties and is
adequately acquainted with all of the course information. Weekly Skype meetings and WeChat
groups are the tools that keep everyone on the same page with instant communication.
Oftentimes, staff members have to solve unprecedented problems. As a start-up organization,
the budget remains tight, with corresponding salary. Staff needs to be passionate about the
mission.
Despite the organization’s financial difficulties, the staff generally demonstrates extreme
enthusiasm and resilience, in that it makes attempts to provide findings to problems or target
problems efficiently and on time. A lack of capable project managers or engineers is the main
obstacle for the organization because in-house employees are a significant cost. A main issue for
MOOCs in general, and this organization in particular, is how they can be profitable in some
ways while the non-profit organization can be managed to be sustainable.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 106
With an individual founder and a very small team, decision-making is a combination of
top down and bottom up methods. Decisions are often made under time constraints without
prolonged analysis. It is rare that the organization can engage in a lengthy discussion of benefits
and drawbacks of a particular policy before making a decision. Thus, many decisions are made
based on intuition and a gut reaction. These environmental features of the organization must be
taken into account in implementation planning.
Related Solutions and Organizational Capacity to Implement
One must first examine the policy development process when it comes to evaluating the
relationship between decision-making and organizational behavior. Much like a garbage can’s
function is to receive trash as it is produced (Cohen, March, & Olsen, 1972), organizations may
treat the policy development process as a receptacle awaiting input in the forms of numerous
problems and solutions as the input is developed. Once the “receptacle” receives the input, the
organization then can make a decision based on the input. In general, no policy can be
developed without decision-making. Decisions made by the organization, however, cannot be
detached from the organization’s overall behavior or the organization’s culture. Intuition, such
as the “intuitive repulsion” felt in analysis of an image, can be a reaction based on prior
knowledge, experience, or culture (Gladwell, 2007). Therefore, policy development and
decision-making is heavily influenced by the organization’s overall behavior, or its culture.
Moving the suggested MOOC execution plan forward at university levels requires
specific solutions which are outlined below in Table 22. These include two primary
implementations. First, the instructional design within this MOOC execution plan focused on
establishing a strong collaboration among all related teams and a collaborative community of
learners. The instructor entered the MOOC execution plan with the idea of serving as a co-
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 107
producer and a participant was made to address the issue of multiple perspectives and
experiences. Although instructors’ roles within MOOC may differ in varied institutional
contexts, in which they can perform communication skills face-to-face, MOOCs with more
network opportunities, where participants can gain more engagement through participation, are
most effective (Ross et al., 2014; Veletsianos, 2017). The results in Chapter Four present a
detailed examination of instructional strategies and teaching presence in a MOOC designed to
focus on the establishment of a collaboration regarding MOOC production and a collaborative
learning community that can underpin the highly engaged MOOCs with high completion rates in
general and MOOCs for language learning specifically.
Secondly, carrying out these solutions rests on the development of working groups within
university MOOC strategy team members. There are two predicted oppositions that could hinder
the implementation: (1) some universities do not promote and accept course credits gained by
participants working online (Aydin, 2017) and (2) possible resistance from university faculty
members, who may view this alternative as unwelcome additional work (Zheng, Wisniewski,
Rosson, & Carroll, 2016). MOOCs would not have to consist of formal classes, as the hectic
schedule of a university would make it difficult to have students who wish to have double majors
to attend required course classes. Alternatives for instructors might consist of collaborating on
interdisciplinary coursework that could be used for reference by all instructors so that students
would receive consistent course framework regardless of which faculty member’s course they
are enrolled in. It was suggested that the courses’ scripts would need to be visual and written
since some of the participants are hearing impaired. However, a university needs to know how
to manage the MOOC platform, or it would be risky to invest a lot of time and money without
having specific expected outcomes.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 108
A university setting, which is more structured in nature, weighs strongly toward a more
methodical examination of policy and lengthy discussion before making any policy decision.
Oftentimes, the decision-making process seems overly rational and impeded by bureaucratic
decisions. The process here is closer to Etzioni’s (1967) mixed scanning approach, following a
systematic evaluation strategy before arriving at a decision. A series of working groups involves
hierarchy and a more rational process in which each policy decision is based on a careful and
thorough analysis of all options based on the facts available (Aydin, 2017). The decision is
formally decided upon after extended discussion among different levels within the university;
contrastingly, a sudden intuitive decision is often made at some point within the organization.
The process of making decisions among a variety of policy alternatives seems to depend on
specific organization’s cultures and should eliminate hasty initiatives of intuitive decision-
making, which may be the ultimate description of decision-making. Three validated assets are
discussed below through a review of the literature and summarized in Table 22.
Table 22

Summary Implementation Plan for Two Proposed Solutions: Create Strong MOOCs with
Instructional Design Strategies at MOOC Community and for University Coursework
Framework

Solutions
Human Resource Roles/
Responsibilities/Capacity

Timeframe
Develop specific training plan for
each group as well as broad training
on assessment. It will likely be
sequence of training as well as
repetition and practical development
of applicable tools for participants to
utilize.
Working group consisting of
instructor(s), course coordinator,
project manager, technology
coordinator (i.e., leading roles), and/or
outside consultant.
7/17 – 10/17
(3 months)
Develop clear mechanism of
communication among all of the
teams, including means, timeline,
and links to the knowledge capture.
Working group consisting of
instructor(s), course coordinator,
project manager, technology
coordinator (i.e., leading roles), and
(or) outside consultant.
7/17 – 10/17
(3 months)
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 109
Table 22, continued
Determine available funding and
source(s) of funding across the
organization or partnered with
sponsorship institutions.
Working group consisting of
instructor(s), course coordinator, project
manager, technology coordinator (i.e.,
leading roles), and university MOOC
strategy team.

7/17 – 12/17
(6 months) then
ongoing
Develop course content, including,
video lectures, textbook, workbook,
exercises, quizzes, final exam,
feedback loop and ongoing
assessment, edition. and revisions.
Working group consisting of
instructor(s), course coordinator,
course development team, teaching
assistants, video production team,
project manager and technology
coordinator.

10/17 – 12/17
(3 months)
Launch course on MOOC platform,
including: having virtual office
hours, managing forum discussions,
hosting social networking events
and providing technical support.
Communications among all teams are
continually encouraged to achieve
efforts of a high-performing MOOC
with high completion rates.
Working group consisting of
instructor(s), course coordinator,
course development team, teaching
assistants, community facilitator, video
production team, project manager,
technology coordinator, and MOOC
participants.
1/18 – 3/18
(3 months)
Evaluate and assess MOOC
execution plan and activities, make
necessary adjustments. Develop
plan for coming school year focused
on university framework and
stakeholders (e.g. faculty from
online division) within the
institution who have been trained
and had experiences teaching online
courses. Provide teacher training to
faculty who have interests in
teaching online courses.

Working group consisting of
instructor(s), course coordinator,
project manager, technology
coordinator, and university MOOC
strategy team.
2/18 – 4/18
(3 months)
Identify organization(s),
universities, and/or individual
instructor to provide MOOC
execution plan holistically, as well
as within and across each area
(curricular and co-curricular). This
will include both on-campus flipped
classroom implementation for larger
groups, as well as 100% online
degree programs and other hybrid
programs for targeted stakeholders
(MOOC participants).

Working group consisting of
instructor(s), course coordinator,
project manager, technology
coordinator, and university MOOC
strategy team.
2/18 – 4/18
(3 months)

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 110
Table 22, continued
Take redrafted implementation
plans and policies to governing
bodies (e.g. University Council) for
approval.
Working group consisting of university
MOOC strategy team (e.g. members of
University Council, Deans, Chairs of
programs, Director of Academic
Administration, and Director of Online
Education).
2/18 – 7/18
(6 months)
Implement university course
framework Fall 2018 in alignment
with MOOCs incorporations and
accreditation.
Working group consisting of university
MOOC strategy team (e.g. members of
University Council, Deans, Chairs of
programs, Director of Academic
Administration, and Director of Online
Education) and accreditation
institutions or agencies.
8/18 – 1/19
(6 months)

Evaluation Plan
Kirkpatrick and Kirkpatrick (2006) describe the four levels of evaluation: reaction,
learning, behavior, and impact. The objective of the levels of evaluation is to determine whether
policies implemented by the organization are effective and how these policies can be improved
or otherwise made more efficient (Kirkpatrick & Kirkpatrick, 2006). Assessment can be formal
or informal, depending on the existing culture of the organization.
Moving from one level to the next, they represent a sequence of complexity and the
evaluation process becomes more difficult and time-consuming with each step, but it also
provides increasingly more valuable information. (Kirkpatrick & Kirkpatrick, 2006).
Kirkpatrick and Kirkpatrick’s (2006) evaluation model will be applied, and the four levels of
evaluation needed to fully determine the effectiveness of the implementation plan will also be
discussed in the following sections.
Level 1: Reactions
At Level 1, the focus is on the participants’ reactions to the implementation plan. The
measurement instruments request surveys or comments about the course content, training
materials, instructors, facilities, delivery methods, venues, and so on (Kirkpatrick & Kirkpatrick,
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 111
2006). Positive reactions to an implementation plan may encourage participants to get involved
in the future project. However, negative reactions towards the implementation plan may
discourage participants from attending or joining in the project. Both the positive and negative
reactions can be used to modify the proposed solutions and to ensure organizational support for
the implementation plan. Kirkpatrick highlighted the importance of Level 1 evaluation because
favorable reactions to the implementation plan do not guarantee that learning (Level 2), or
transfer (Level 3) would occur (Kirkpatrick & Kirkpatrick, 2006).
It is imperative to devote time for extensive analysis of data in lieu of participants’
reactions towards the MOOC scheme for university coursework framework in the first level of
the evaluation plan. Assessment and data collection are in the form of formal surveys, as
recommended by Kirkpatrick and Kirkpatrick (2006), rather than an informal process based on
staff feedback, observations, or participants’ testimonials. A set of Level 1 reaction questions
will be administered through a survey, which will include Likert-scale and open-ended questions.
The survey questionnaire is used by the instructors and the course development team in the
beginning of the implementation plan to better understand their motivation and perceptions
towards the MOOC scheme. One sample Likert-scale question would ask participants to rate the
following statement on an agreeability scale from 1 “strongly disagree” to 6 “strongly agree:”
“Do you think that MOOC would enhance students’ learning experiences and positive
outcomes?” One sample open-ended question would be “How does teaching a MOOC differ
from non-MOOC online courses?” Altogether, the results of the survey will indicate the
motivational impact of the MOOC scheme implementation plan.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 112
Level 2: Learning
Kirkpatrick’s Level 2 is learning. As Kirkpatrick and Kirkpatrick (2006) underlined, no
behavioral change will happen without learning. Thus, evaluating learning is important.
Kirkpatrick contended that learning to some extent changes participants’ attitudes, improves
knowledge, and/or increases skills (Kirkpatrick & Kirkpatrick, 2006.) Although research does
not support that acquired knowledge and skills associate with behavioral changes (Payne &
Isaacs, 2017), it is evident in the literature that the examination of what participants learned
during the training used the most in evaluations (Giacumo & Bremen, 2016). Evidence needs to
be provided to demonstrate the merit of the implementation plan.
The second level of the evaluation plan will check how learning, motivation, and
organizational change impact the implementation plan of the MOOC scheme for university
coursework framework. The best way to measure learning is to utilize a direct assessment to
determine what type of learning has occurred. It is essential to truly understand if the solution
resulted in any changes at this level (Kirkpatrick & Kirkpatrick, 2006).
Participants will be provided with clear guidance and explicit demonstration throughout
the training, and the training professionals will provide timely feedback if and when questions
are raised. There are formal assessments of participants’ performance and their perceptions
towards the training. Assessment plans will be required among various teams (i.e., the
stakeholder groups), including instructors, course development team, video production team,
project manager, and technology coordinator, to determine the acquired learning and training
while implementing the MOOC scheme at the universities (Kirkpatrick & Kirkpatrick, 2006).
This assessment will be demonstrated and the engagement of the stakeholder groups will also be
determined to measure if they learned the appropriate skills in the training.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 113
Level 3: Transfer
Level 3 addresses the issue of learning transfer and measures participants’ performance
by determining the extent to which participants transfer their newly acquired knowledge and
skills on the task/project (Kirkpatrick, 2006). The training effort cannot have an impact on the
organizational results (Level 4) if participants do not apply what they learned to their new
tasks/projects. Only positive transfer can predict progressive results. Kirkpatrick and
Kirkpatrick (2006) noted that the evaluation in this level is more complicated, demanding, and
time-consuming than the reaction and learning evaluations in Level 1 and 2. Nevertheless,
executives often neglect Level 3, jumping straight to Level 4, as it should be since most of the
time, energy, and expense are invested in Level 1 and 2 by training professionals (Kirkpatrick &
Kirkpatrick, 2006).
The third level of the evaluation plan focuses on transfer, which measures the
effectiveness of the implementation in an actual work setting (Kirkpatrick & Kirkpatrick, 2006),
namely, if the MOOC scheme collaborations were transferred over into the university setting
where the scope of the coursework was appropriately expanded. Ongoing evaluation and
observation will be required to measure the transfer of the skills from the MOOC scheme
collaborations (Kirkpatrick & Kirkpatrick, 2006). It is imperative to observe and survey the
stakeholder groups after the implementation plan has been executed for a few months to gauge
the transfer of the required skills. The triangulation of surveys and observations will be
conducted to determine if the stakeholder groups effectively transferred the skills learned from
the training and the collaboration into the university setting where they are employed.
Positively, the results will indicate the success of the transferred skills and learning from the
MOOC scheme into the university coursework framework.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 114
Level 4: Impact
Level 4 is the most essential and challenging level to assess (Kirkpatrick & Kirkpatrick,
2006). Typically, at Level 4, organizations seek business results for their training efforts.
Organizations attempt to measure actual organizational changes due to training, and place a
numerical value on those changes. The four levels are positively inter-correlated. It is important
to evaluate both reactions (Level 1) and learning (Level 2) to ensure transferals (Level 3) occur.
Participants become more accountable for their own performance and achievement by evaluating
both Level 1 and 2. Furthermore, time constraints, complexity of analysis, lack of support for
the process, cost, ineffectiveness, and not being familiar with the previous level processes are all
barriers to organizational impact (Level 4).
The fourth level measures the impact of the MOOC scheme on expanding the scope of
the coursework offered at universities. The measurement will also include if the identified
instructional design suggestions advanced the MOOC community, meanwhile, optimizing
educational resources and saving on the cost of logistics. The evaluation plan will take place at
the end of the school year to determine how MOOCs were incorporated appropriately to
contribute to expand the scope of the coursework offered at universities.
To measure the overall impact, the university MOOC strategy team (e.g. members of the
university council, deans, chairs of programs, director of academic administration, and director
of online education) and accreditation institutions or agencies should look at course evaluations
to determine the impact of the effectiveness on implementing the MOOC scheme to expand the
university course framework throughout the school year. The university MOOC strategy team
should also examine students’ academic performance in their subjects to determine overall
effectiveness of their learning. As for the MOOC scheme collaborations, the university MOOC
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 115
strategy team can survey the stakeholder groups (e.g. instructors, course development team,
video production team, etc.) and analyze students’ feedback to determine the impact of the
implementation plan throughout the school year. Accordingly, the evaluation will indicate the
success of the implementation plan and areas for improvement (Kirkpatrick & Kirkpatrick,
2006).
The ultimate determination of whether the organization’s actions are effective is simple:
if the organization continues to build up high-quality language learning MOOCs and maintains
high completion rates, then whatever actions the organization is implementing are effective.
Thus, assessments need to be formal and extensive for evaluating the organization’s actions. A
systematic examination of performance data, including participants’ continued participation and
the collaboration of various teams, is absolutely vital for the organization and the implementation
plan.
Limitations
Several biases are impossible to avoid and thus need to be addressed owing to the fact
that the researcher is the founder of the organization. First, possible methodological limitations
could include self-reported gathered data from surveys which are not independently verifiable in
order to report a noteworthy outcome to achieve a high-performing organization goal. Second,
participants, who are the key stakeholders, may not interpret the survey items in the manner
intended. The qualitative data could have revealed additional information beyond the
perceptions of the MOOC participants taking the courses. The qualitative data could also have
led to assumed causes that were validated by quantitative data and generated new assets in the
knowledge, motivation, and organization dimension.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 116
Furthermore, the contexts in which these incorporated solutions transfer to other
universities or institutions could vary because of the infrastructures and available capital of those
universities or institutions. Other universities or institutions may not have the inclination or the
ability to replicate the practices at the MandarinX organization for many reasons.
Implications for Future Research
Future studies with larger sample sizes and more varied populations should validate the
findings empirically through more confirmatory methods, such as surveys, interviews, or
experimental designs. The study is not intended to advocate for MOOC instructors over
participants or other MOOC stakeholders. Instead, the focus has been shifted on the instructors,
instructional design, and teaching pedagogy when evaluating benefits, challenges, and design
implications associated with the rapidly emerging and evolving MOOC paradigm. Lastly, the
applied recommendations are based solely on the experiences and data collected within the
organization, which require additional exploration to determine if they are sustainable solutions
for overall MOOC success.
Conclusions
The evaluation study focused on the reasons that participants persist in their enrollments
in six-week long MOOCs. Through the gap analysis framework, the study explored KMO
factors that impact MOOC participants’ completion rates (Clark & Estes, 2008). SPSS was
employed to analyze all the quantitative data from the questionnaire survey to validate all of the
assumed causes. Evidence which ensures the success of MOOCs was integrated into solutions
and a MOOC scheme in order to maximize and ensure effective implementation of all of the
validated assets.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 117
Additionally, the study enables instructors and the course development team to
implement identified instructional designs for a successful MOOC. Although specific subjects
might be different, the MOOC scheme is designed for practical application and effective
collaboration towards high-performing learning outcomes. With the aim of having an impact on
the completion rates, ongoing engagement throughout interaction among all participants and
autonomous learning improvement in learning communities, the study decomposed the complex
and iterative process of administering a successful MOOC as a proposed model: preparation,
execution, and implementation.
A main finding is that successful MOOC administration requires a team effort (e.g.,
instructors, curriculum design, video production, technical support, etc.), and the mandatory
collaboration is rare to be seen. By boosting support for collaboration, teaching and learning
outcomes will be elevated for all MOOC stakeholders. The study provides actionable guidance
for organizing, delivering, and managing MOOCs as part of the MOOC community and the
university course framework. In addition to existing research that examines MOOCs as a novel
learning pedagogy, the study advocates for a view of MOOCs as complex and collaborative
bionetwork.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 118
References
Adamopoulos, P. (2013). What makes a great MOOC? An interdisciplinary analysis of student
retention in online courses. New York, NY: New York University. Retrieved from
http://pages.stern.nyu.edu/~padamopo/What%20makes%20a%20great%20MOOC.pdf
Agarwal, A. (2014). TED. (2014). Why massive open online courses (still) matter. Retrieved from
https://www.ted.com/talks/anant_agarwal_why_massively_open_online_courses_still_matter
/transcript?language=en
Alario-Hoyos, C., Estévez-Ayres, I., Pérez-Sanagustín, M., Kloos, C. D., & Fernández-Panadero, C.
(2017). Understanding Learners’ Motivation and Learning Strategies in MOOCs. The
International Review of Research in Open and Distributed Learning, 18(3).
Alexander, P. A., & Grossnickle, E. M. (2016). Positioning interest and curiosity within a model of
academic development. Handbook of Motivation at School, 10, 188-208.
Alraimi, K. M., Zo, H., Ciganek, A. P. (2015). Understanding the MOOCs continuance: The role of
openness and reputation. Computers & Education, 80, 28-38.
Ambrose, S., Bridges, M., Lovett, M., DiPietro, M., & Norman, M. (2010). How learning works: 7
research-based principles for smart teaching. San Francisco, CA: Jossey-Bass.
Anderman, E., & Anderman, L. (2010). Attribution theory. Retrieved from
http://www.education.com/reference/article/attribution-theory/
Anderson, L. W., & Krathwohl, D. R. (Eds.), (2001). A Taxonomy for learning, teaching, and
assessing: A revision of Bloom’s taxonomy of educational objectives. New York, NY:
Longman.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 119
Anderson, T., Rourke, L., Garrison, R., & Archer, W. (2001). Assessing teaching presence in a
computer conferencing context. Journal of Asynchronous Learning Networks, 5(2).
Retrieved from http://auspace.athabascau.ca/bitstream/2149/725/1/assessing_teaching_
presence.pdf
Aparicio, M., Bacao, F., & Oliveira, T. (2017). Grit in the path to e-learning success. Computers in
Human Behavior, 66, 388-399.
Arbaugh, J. B. (2010). Sage, guide, both, or even more? An examination of instructor activity in
online MBA courses. Computers & Education, 55(3), 1234-1244.
Austin, K. (2016). [Review of the book MOOCs, high technology, and higher learning. by
R. Rhoads] Education Review/Reseñas Educativas, 23. Retrieved from
http://edrev.asu.edu/index.php/ER/article/viewFile/2081/617
Aydin, C. H. (2017). Current status of the MOOC movement in the world and reaction of the
Turkish higher education institutions. Open Praxis, 9(1), 59-78.
Baker, C., Nafukho, F. M., McCaleb, K., Becker, M., & Johnson, M. (2016). The tangible and
intangible benefits of offering massive open online courses: Faculty perspectives. Internet
Learning, 4(2), 6.
Balakrishnan, V., Teoh, K. K., Pourshafie, T., & Liew, T. K. (2017). Social media and their use in
learning: A comparative analysis between Australia and Malaysia from the learners’
perspectives. Australasian Journal of Educational Technology, 33(1).
Bali, M. (2014). MOOC pedagogy: Gleaning good practice from existing MOOCs. MERLOT
Journal of Online Learning and Teaching, 10(1), 44-56.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 120
Bandura, A. (1977). Self-efficacy: Toward a unified theory of behavioral change. Psychological
Review, 84(2), 191-215.
Barba, P. D., Kennedy, G. E., & Ainley, M. D. (2016). The role of students’ motivation and
participation in predicting performance in a MOOC motivation and participation in
MOOCs. Journal of Computer Assisted Learning 32(3), 218-231.
Belanger, Y. & Thornton, J. (2013). Bioelectricity: A quantitative approach, Duke University’s First
MOOC, Duke Center for Instructional Technology. Retrieved from
https://dukespace.lib.duke.edu/dspace/handle/10161/6216
Ben-Eliyahu, A., Linnenbrink-Garcia, L., & Putallaz, M. (2017). The intertwined nature of
adolescents’ social and academic lives: Social and academic goal orientations. Journal of
Advanced Academics, 28(1), 66-93.
Boettcher, J. V., & Conrad, R. M. (2016). The online teaching survival guide: Simple and practical
pedagogical tips. San Francisco, CA: John Wiley & Sons.
Bonk, C. J., & Lee, M. M. (2017). Motivations, achievements, and challenges of self-directed
informal learners in open educational environments and MOOCs. Journal of Learning for
Development-JL4D, 4(1).
Bouchet, F., Labarthe, H., Bachelet, R., & Yacef, K. (2017, May). Who wants to chat on a MOOC?
Lessons from a peer recommender system. In C. D. Kloos, P. Jermann, M. Pérez-
Sanagustín, D. T. Seaton, & S. White (Eds.), European Conference on Massive Open
Online Courses (pp. 150-159). Cham, Switzerland: Springer.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 121
Bowers, J., & Kumar, P. (2017). Students’ perceptions of teaching and social presence: A
comparative analysis of face-to-face and online learning environments. In I. Management
Association (Ed.), Blended Learning: Concepts, Methodologies, Tools, and Applications (pp.
1532-1550). Hershey, PA: IGI Global.
Bradley, P. (2014). Move over, MOOCs, competency-based education takes center stage as ‘next
big thing’. Community College Week, 6-7.
Breslow, L. B., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013).
Studying learning in the worldwide classroom: Research into edX’s first MOOC. Research
& Practice in Assessment, 8, 13-25.
Broadbent, J. (2017). Comparing online and blended learner’s self-regulated learning strategies and
academic performance. The Internet and Higher Education, 33, 24-32.
Busato, V. V., Prins, F. J., Elshout, J. J., & Hamaker, C. (2000). Intellectual ability, learning style,
personality, achievement motivation and academic success of psychology students in higher
education. Personality and Individual Differences, 29(6), 1057-1068.
Byrne, J. A. (2015). Arizona State, edX to offer entire freshman year of college online. Retrieved
from http://fortune.com/2015/04/22/arizona-state-edx-moocs-online-education/
Carr, N. (2012, 09 27). The crisis in higher education. (M. T. Review, Ed.) Retrieved from
http://www.technologyreview.com/featuredstory/429376/the-crisis-in-higher-education/
Che, X., Luo, S., Wang, C., & Meinel, C. (2016). An attempt at MOOC localization for Chinese-
speaking users. International Journal of Information and Education Technology, 6(2), 90-96.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 122
Chew, S. W., Cheng, I. L., & Chen, N. S. (2017). Yet another perspectives about designing and
implementing a MOOC. In M. Jemni, Kinshu, & M. K. Khribi, (Eds.), Open Education:
from OERs to MOOCs (pp. 117-133). New York, NY: Springer Berlin Heidelberg.
Choy, J. L. F., & Quek, C. L. (2016). Modelling relationships between students’ academic
achievement and community of inquiry in an online learning environment for a blended
course. Australasian Journal of Educational Technology, 32(4).
Christensen, C. M. (2012). Dr. Clayton Christensen discusses disruption in higher education.
Retrieved from https://www.youtube.com/watch?v=yUGn5ZdrDoU
Christensen, C. M., & Weise, M. R. (May 09, 2014). MOOCs’ disruption is only beginning. The
Boston Globe. Retrieved from https://www.bostonglobe.com/opinion/2014/05/09/moocs-
disruption-only-beginning/S2VlsXpK6rzRx4DMrS4ADM/story.html
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Atlanta, GA: CEP Press.
Cohen, M. D., March, J. G., Olsen, J. P. (1972). A garbage can model of organizational choice.
Administrative Science Quarterly, 17, 1-25.
Coursera. (2017). About Coursera. Retrieved from https://coursera.org
Crook, C., & Schofield, L. (2017). The video lecture. The Internet and Higher Education, 34, 56-64.
Cruz-Benito, J., Borrás-Gené, O., García-Peñalvo, F. J., Blanco, Á. F., & Therón, R. (2017).
Learning communities in social networks and their relationship with the MOOCs. IEEE
Revista Iberoamericana de Tecnologias del Aprendizaje, 12(1), 24-36.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 123
Cunningham, J. A. (2017). Using traces of self-regulated learning in a self-paced mathematics
MOOC to predict student success (Master’s thesis). Retrieved from
https://www.researchgate.net/publication/316554776_Using_Traces_of_Self-
Regulated_Learning_in_a_Self-Paced_Mathematics_MOOC_to_Predict_Student_Success
Dalipi, F., Yayilgan, S. Y., Imran, A. S., & Kastrati, Z. (2016, July). Towards understanding the
MOOC trend: Pedagogical challenges and business opportunities. In P. Zaphiris & A.
Ioannou (Eds.), International Conference on Learning and Collaboration Technologies (pp.
281-291). New York, NY: Springer International Publishing.
Daniel, J. (2012). Making sense of MOOCs: Musings in a maze of myth, paradox and possibility.
Journal of Interactive Media in Education, 3(0). Retrieved from http://www-
jime.open.ac.uk/jime/article/view/2012-18
Davis, D., Jivet, I., Kizilcec, R. F., Chen, G., Hauff, C., & Houben, G. J. (2017, March). Follow the
successful crowd: Raising MOOC completion rates through social comparison at scale.
Paper presented at the proceedings of the 7th International Conference on Learning
Analytics and Knowledge, Vancouver, Columbia, Canada.
Deng, R., Benckendorff, P., & Gannaway, D. (2017, May). Understanding learning and teaching in
MOOCs from the perspectives of students and instructors: A review of literature from 2014
to 2016. In C. Delgado Kloos, P. Jermann, M. Pérez-Sanagustín, D. Seaton, S. White.
(Eds.), Digital Education: Out to the World and Back to the Campus. EMOOCs 2017.
Lecture Notes in Computer Science, 10254 (pp. 176-181). New York, NY: Springer.
Retrieved from: 10.1007/978-3-319-59044-8_20
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 124
Dennis, M. J. (2017). Review changes in higher education. Enrollment Management Report, 20(11),
1-3.
Dey, I. (2003). Qualitative data analysis: A user friendly guide for social scientists. New York, NY:
Routledge.
Dellarocas, C., & Van Alstyne, M. (2013). Money models for MOOCs. Communications of the
ACM, 56(8), 25-28.
DeVellis, R. F. (1991). Scale development: Theory and applications. Newbury Park, CA: SAGE.
Duval, E., Sharples, M., & Sutherland, R. (Eds.). (2017). Technology Enhanced Learning. New
York, NY: Springer International Publishing. Retrieved from: 10.1007/978-3-319-02600-8
edX Blog. (2017, June). Celebrating 10 Million edX Learners Worldwide!
Retrieved from http://blog.edx.org/celebrating-10-million-edx-learners-
worldwide?track=blog
edX Insights. (2016). Instructor Dashboard [Enrollment Data].
El-Hmoudova, D. (2014). MOOCs motivation and communication in the cyber learning
environment. Procedia-Social and Behavioral Sciences, 131, 29-34.
Engle, D., Mankoff, C., Carbrey, J. (2015). Coursera’s introductory human physiology course:
Factors that characterize successful completion of a MOOC. International Review of
Research in Open and Distributed Learning, 16(2). 46-68.
Etzioni, A. (1967). Mixed scanning: A “third” approach to decision-making. Public Administration
Review, 27, 385-392.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 125
Evans, S. M., Ward, C., & Reeves, S. (2017). An exploration of teaching presence in online
interprofessional education facilitation. Medical Teacher, 1-7.
Ferraro, G. P., & Briody, E. K. (2017). The cultural dimension of global business (7th ed.). New
York, NY: Routledge.
Fini, A. (2009). The technological dimension of a massive open online course: The case of the
CCK08 course tools. International Review of Research in Open and Distance Learning,
10(5), 1-26.
Fisher, D. H. (2012). Regional sections of massively open online courses, Presentation to the 18th
Annual Conference of the Coalition of Urban and Metropolitan Universities. Retrieved from
http://www.vuse.vanderbilt.edu/~dfisher/CUMU-2012-MOOC-Abstract.pdf
Flavin, M. (2017). Free, simple and easy to use: Disruptive technologies, disruptive innovation and
technology enhanced learning. In Disruptive Technology Enhanced Learning (pp. 19-52).
London, UK: Palgrave Macmillan.
Frank, S. (2012). Review: MITx’s online circuit and analysis course. IEEE Spectrum. Retrieved
from http://spectrum.ieee.org/at-work/ education/review-mitxs-online-circuit-design-and-
analysis-course.
Frankola, K. (2012). Why online learners drop out high dropout rates are e-learning’s embarrassing
secret. Here’s what you can do about it. Retrieved from http://www.c3l.uni-
oldenburg.de/cde/support/readings/frankola.htm
Frymier, A. B., & Shulman, G. M. (1995). “What’s in it for me?”: Increasing content relevance to
enhance students’ motivation. Communication Education, 44(1), 40-50.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 126
Funieru, L. M., & Lazaroiu, F. (2016). Massive open online courses (MOOCs): A comparative
analysis of the main platforms. Informatica Economica, 20(2), 35.
FutureLearn Blog. (2017, June). Introducing degrees: you can now take a postgraduate degree
on FutureLearn. Retrieved from https://about.futurelearn.com/blog/postgraduate-degrees-on-
futurelearn/
Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect minority
achievement and school improvement research. Educational Psychologist, 31(1), 45-56.
Gamage, D., Fernando, S., & Perera, I. (2015, August). Factors leading to an effective MOOC from
participants perspective. Paper presented at the meeting of International Conference on Ubi-
Media Computing (UMEDIA), Colombo, Sri Lanka.
Garg, A., & Paepcke, A. (2017). Supporting the encouragement of forum participation. Retrieved
from http://ilpubs.stanford.edu:8090/1153/1/forumPrompts.pdf
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment:
Computer conferencing in higher education model. The Internet and Higher Education, 2(2–
3), 87-105.
Garrison, D. R., Anderson, T., & Archer, W. (2003) A theory of critical inquiry in online distance
education. M. G. Moore and W. G. Anderson (Eds.). Handbook of Distance Education.
Mahwah, NJ: L. Erlbaum Associates.
Garrison, D. R., Cleveland-Innes, M., & Fung, T. (2010). Exploring causal relationships among
cognitive, social and teaching presence: Student perceptions of the Community of Inquiry
framework. The Internet and Higher Education, 13(1-2), 31-36.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 127
Giacumo, L. A., & Bremen, J. (2016). Emerging evidence on the use of big data and analytics in
workplace learning: A systematic literature review. Quarterly Review of Distance
Education, 17(4), 21.
Gladwell, M. (2007). Blink: The power of thinking without thinking summary. New York, NY:
Hachette Digital, Inc.
Goetz, T., Frenzel, A. C., Hall, N. C., & Pekrun, R. (2008). Antecedents of academic emotions:
Testing the internal/external frame of reference model for academic enjoyment.
Contemporary Educational Psychology, 33(1), 9-33.
Green, K. C., & Gilbert, S. W. (2010) Great expectations: Content, communications, productivity,
and the role of information technology in higher education. Change: The Magazine of
Higher Learning, 27(2), 8-18.
Greene, J. A., Oswald, C. A., & Pomerantz, J. (2015). Predictors of retention and achievement in a
massive open online course. American Educational Research Journal, 52(5), 925-955.
Retrieved from:10.3102/0002831215584621.
Guàrdia, L., Maina, M., & Sangrà, A. (2013). MOOC design principles. A pedagogical approach
from the learner’s perspective. Journal of eLearning Papers, 33, 1-6.
Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An
empirical study of MOOC videos. Conference on Learning@ Scale Conference: Proceedings
of the first Association for Computing Machinery (pp. 41-50). New York, NY: Association
for Computing Machinery.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 128
Gütl, C., Rizzardini, R. H., Chang, V., & Morales, M. (2014). Attrition in MOOC: Lessons learned
from drop-out students. Learning Technology for Education in Cloud. MOOC and Big Data,
(446), 37-48.
HarvardX and MITx. (2014). Two years of open online courses fall 2012-summer 2014. Retrieved
from:10.2139/ssrn.2586847
Hashmi, A. R. (2015). Rapid growth of massive open online courses (MOOCs) and the market for
university graduates. Asian Journal of the Scholarship of Teaching and Learning, 5(1), 24-
39.
Head, K. (2015). The single canon: MOOCs and academic colonization. MOOCs and open
education around the world, 62(6), 12-20.
Head, K. J. (2017). Disrupt this!: MOOCs and the promises of technology. University Press of New
England.
Herala, A., Knutas, A., Vanhala, E., & Kasurinen, J. (2017). Experiences from video lectures in
software engineering education. International Journal of Modern Education and Computer
Science (IJMECS), 9(5), 17.
Hew, K. F. (2016). Promoting engagement in online courses: What strategies can we learn from
three highly rated MOOCS. British Journal of Educational Technology, 47(2), 320-341.
Hew, K. F. & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses
(MOOCs): Motivations and challenges. Educational Research Review, 12, 45-58.
Hill, P. (2012). Online educational delivery models: A descriptive view. In T. D. Diggs, (Ed.),
Educause Review - Why IT Matters in Higher Education, 47(6), 84-97.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 129
Ho, A. D., Chuang, I., Reich, J., Coleman, C., Whitehill, J., Northcutt, C., Williams, J. J., Hansen, J.,
Lopez, G., & Petersen, R. (2015). HarvardX and MITx: Two years of open online courses
(HarvardX Working Paper No. 10).
Ho, A. D., Reich, J., Nesterko, S. O., Seaton, D. T., Mullaney, T., Waldo, J., & Chuang, I. (2014).
HarvardX and MITx: The first year of open online courses (HarvardX and MITx Working
Paper No. 1).
Hodges, C. B. (2004). Designing to motivate: Motivational techniques to incorporate in e-learning
experiences. The Journal of Interactive Online Learning, 2(3), 1-7.
Höfler, E., Zimmermann, C., & Ebner, M. (2017). A case study on narrative structures in
instructional MOOC designs. Journal of Research in Innovative Teaching &
Learning, 10(1), 48-62.
Hone, K. S., & El Said, G. R. (2016). Exploring the factors affecting MOOC retention: A survey
study. Computers & Education, 98, 157-168.
Hood, N., & Littlejohn, A. (2016). MOOC Quality: The need for new measures. Journal of Learning
for Development-JL4D, 3(3).
Huang, K., Lee, S. J., & Dugan, A. (2017). Leveraging teaching presence in online courses:
strategies, technology, and student perspectives. In Blended learning: Concepts,
methodologies, tools, and applications (pp. 1687-1711). Hershey, PA: IGI Global.
Ingolfsdottir, K. (2016). Winds of change in higher education. Trends in Pharmacological
Sciences, 37(12), 990-992.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 130
Instructure. (2013). Qualtrics and instructure partner to reveal top motivations for MOOC students.
Instructure Inc. Retrieved on March 28, 2017 from http://www.instructure.com/press-
releases/qualtrics-and-instructure-reveal-mooc-students-top-motivations.
Jacobs, A. J. (2013). Two cheers for Web U! New York Times, 162(56113), 1-7.
James, M. (2017). The story of ‘doing a MOOC’ or knowing ‘the beast’ from within. In R. Bennett
& M. Kent (Eds.), Massive Open Online Courses and Higher Education: What Went Right,
What Went Wrong and Where to Next? (Part 2; Chapter 6) Abingdon, UK: Routledge.
Jiao, J., Yang, Y., Zhong, H., & Ren, G. (2017). Improving learning in MOOCs through peer
feedback: How is learning improved by providing and receiving feedback? In Lai, Feng-Qi
& J. D. Lehman (Eds.), Learning and Knowledge Analytics in Open Education (pp. 69-87).
Springer International. Retrieved from:10.1007/978-3-319-38956-1_6
Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses. The
International Review of Research in Open and Distributed Learning, 15(1), 133-160.
Joy, S., & Kolb, D. A. (2009). Are there cultural differences in learning style? International Journal
of Intercultural Relations, 33(1), 69-85.
Kangas, M., Siklander, P., Randolph, J., & Ruokamo, H. (2017). Teachers’ engagement and
students’ satisfaction with a playful learning environment. Teaching and Teacher
Education, 63, 274-284.
Kanuka, H. (2011). Interaction and the online distance classroom: Do instructional methods effect
the quality of interaction? Journal of Computing in Higher Education, 23, 143-156.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 131
Kent, M., & Bennett, R. (2017). What was all that about? Peak MOOC hype and post-MOOC
legacies. In R. Bennett & M. Kent (Eds.), Massive Open Online Courses and Higher
Education: What Went Right, What Went Wrong and Where to Next? (pp 1-8). Thames,
Oxfordshire, UK: Taylor & Francis.
Khalil, H., & Ebner, M. (2013). “How satisfied are you with your MOOC?” – A research Study on
Interaction in Huge Online Courses. In proceedings of the World Conference on Educational
Multimedia, Hypermedia and Telecommunications, 2013, 830-839.
Khalil, H., & Ebner, M. (2014). MOOCs completion rates and possible methods to improve
retention: A literature review. In World Conference on Educational Multimedia, Hypermedia
and Telecommunications, 2014, (1), 1305-1313.
Kirkpatrick, D., & Kirkpatrick, J. (2006). Evaluating training programs: The four levels. Oakland,
CA: Berrett-Koehler Publishers.
Kirschner, A. (2012). A pioneer in online education tries a MOOC. Chronicle of Higher Education,
59(6), B21-22.
Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: analyzing learner
subpopulations in massive open online courses. Conference on Learning@ Scale
Conference: Proceedings of the third international conference on learning analytics and
knowledge (pp. 170-179). New York, NY: Association for Computing Machinery.
Ko, S., & Rossen, S. (2017). Teaching online: A practical guide (3rd ed.). New York, NY:
Routledge.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 132
Kolowich, S. (2013). The professors who make the MOOCs. Chronicle of Higher Education,
59(28), A20-A23.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice,
41(4), 212-218.
Kupczynski, L., Ice, P., Weisenmayer, R., & McCluskey, F. (2010). Student perceptions of the
relationship between indicators of teaching presence and success in online courses. Journal
of Interactive Online Learning, 9(1), 23–43. Retrieved from:
http://eric.ed.gov/?redir=http%3a%2f%2fwww.ncolr.org%2fjiol%2fissues%2fpdf%2f9.1.2.p
df
Lazaroiu, G., Popescu, G. H., & Nica, E. (2016, July). Democratizing education: The potential of
edX in revolutionizing learning. In The International Scientific Conference eLearning and
Software for Education (Vol. 3, p. 34). “Carol I” National Defence University Publishing
House.
Lee, K. (2017). Rethinking the accessibility of online higher education: A historical review. The
Internet and Higher Education, 33, 15-23. Retrieved from: 10.1016/j.iheduc.2017.01.001.
Levi, B. (2013). MOOCs change academic playing field, The Leader (Elmhurst College), Retrieved
on March 28, 2017 from http://ecleader.org/2013/11/08/moocs-change-academic-playing-
field
Levy, D. (2011). Lessons learned from participating in a connectivist massive online open course
(MOOC). Proceedings of the Chais conference on instructional technologies research 2011:
Learning in the technological era. Eshet-Alkalai, Y., Caspi, A., Eden, S., Geri, N. & Yair, Y.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 133
(Eds.), The Open University of Israel, Raanana, 31-36. Retrieved on March 28, from
http://www.openu.ac.il/research_center/chais2011/download/f-levyd-94_eng.pdf
Levy, D., & Schrire, S. (2012) The case of a massive open online course at a college of education.
Retrieved 30th August 2013, from http://conference.nmc.org/files/smkbMOOC.pdf
Lin, Y. N. (2012). Life experiences of dissatisfied science and engineering graduate students in
Taiwan. College Student Journal, 46(1), 51-66.
Linnenbrink, E. A., & Pintrich, P. R. (2003). The role of self-efficacy beliefs in student engagement
and learning in the classroom. Reading &Writing Quarterly, 19(2), 119-137.
Liu, O. L., Bridgeman, B., & Adler, R. M. (2012). Measuring learning outcomes in higher
education: Motivation matters. Educational Researcher, 41(9), 352-362. Retrieved from
http://doi.org/10.3102/0013189X12459679
Liu, M., McKelroy, E., Kang, J., Harron, J., & Liu, S. (2016). Examining the use of Facebook and
Twitter as an Additional Social space in a MOOC. American Journal of Distance
Education, 30(1), 14-26.
Loeckx, J. (2016). Blurring boundaries in education: Context and impact of MOOCs. The
International Review of Research in Open and Distributed Learning, 17(3), 92-121.
Loizzo, J., & Ertmer, P. A. (2016). MOOCocracy: The learning culture of massive open online
courses. Educational Technology Research and Development, 64(6), 1013-1032.
Lombardi, M. M. (2013). The inside story: Campus decision-making in the wake of the latest
MOOC tsunami. MERLOT Journal of Online Learning and Teaching, 9(2), 239-248.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 134
Loya, A., Gopal, A., Shukla, I., Jermann, P., & Tormey, R. (2015). Conscientious behaviour,
flexibility and learning in massive open on-line courses. Procedia-Social and Behavioral
Sciences, 191, 519-525.
Lund, D. B. (2003). Organizational culture and job satisfaction. Journal of Business & Industrial
Marketing, 18(3), 219-236.
McNeely, J. H. (1938). College student mortality. Bulletin 1937, No. 11. US Government Printing
Office. Retrieved from http://files.eric.ed.gov/fulltext/ED542540.pdf
Macleod, H., Sinclair, C., Haywood, J., & Woodgate, A. (2016). Massive open online courses:
Designing for the unknown learner. Teaching in Higher Education, 21(1), 13-24.
Maeroff, G. I. (2004). A classroom of one: How online learning is changing our schools and
colleges. England: Macmillan.
Malaga, R. A., & Koppel, N. B. (2017). A comparison of video formats for online
teaching. Contemporary Issues in Education Research (Online), 10(1), 7.
Manathunga, K., Hernández-Leo, D., & Sharples, M. (2017, May). A social learning space grid for
MOOCs: Exploring a FutureLearn Case. In European Conference on Massive Open Online
Courses (pp. 243-253). Cham, Switzerland: Springer.
Margaryan, A., Bianco, M., Littlejohn, A. (2015). Instructional quality of massive open online
courses (MOOCs). Computers & Education. 80, 77-83.
Marshall, S. (2014). Exploring the ethical implications of MOOCs. Distance Education, 35(2), 250-
262.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 135
Martin, F. G. (2012). Will massive open online courses change how we teach? Communications of
the ACM, 55(8), 26-28.
Masters, K. (2011). A brief guide to understanding MOOCs. The Internet Journal of Medical
Education, 1(2). Retrieved from http://ispub.com/IJME/1/2/10995
McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for digital
practice. Retrieved from http://www.elearnspace.org/Articles/MOOC_Final.pdf
Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco
CA: Jossey-Bass.
Meyer, D. K., & Turner, J. C. (2006). Re-conceptualizing emotion and motivation to learn in
classroom contexts. Educational Psychology Review, 18(4), 377-390. Retrieved from
http://doi.org/10.1007/s10648-006-9032-1
Milligan, C., & Littlejohn, A. (2017). Why study on a MOOC? The motives of students and
professionals. The International Review of Research in Open and Distributed
Learning, 18(2), 92-102.
Montoya, M. S. R., & Hernández, D. D. C. R. (2016). Inverted learning environments with
technology, innovation and flexibility: Student experiences and meanings. Journal of
Information Technology Research (JITR), 9(1), 18-33.
Morris, T. A. (2011). Exploring community college student perceptions of online learning.
International Journal of Instructional Technology & Distance Learning, 8(6).
Muijs, D. (2010). Doing quantitative research in education with SPSS. Thousand Oaks, CA: SAGE.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 136
Nagrecha, S., Dillon, J. Z., & Chawla, N. V. (2017, April). MOOC dropout prediction: Lessons
learned from making pipelines interpretable. In Proceedings of the 26th International
Conference on World Wide Web Companion, (pp. 351-359). Retrieved from:
10.1145/3041021.3054162
Nakano, N., Padua, M. C., & Jorente, M. J. V. (2017). MOOC as a complex system. In First
Complex Systems Digital Campus World E-Conference 2015 (pp. 125-131). Cham,
Switzerland: Springer.
Ng, R. Y. K., Lam, R. Y. S., Ng, K. K., & Lai, I. K. W. (2017). Identifying the needs of flexible and
technology enhanced learning in vocational and professional education and training’s
(VPET) workplaces. In New Ecology for Education—Communication X Learning (pp. 107-
117). Singapore: Springer.
Orcher, L. T. (2017). Conducting a survey: Techniques for a term project. New York, NY:
Routledge.
Ostashewski, N., Howell, J., & Dron, J. (2016). Crowdsourcing MOOC interactions: Using a social
media site cMOOC to engage students in university course activities. Retrieved from
http://oasis.col.org/bitstream/handle/11599/2528/PDF?sequence=4
Ou, C., Goel, A. K., Joyner, D. A., & Haynes, D. F. (2016). Designing videos with pedagogical
strategies: Online students’ perceptions of their effectiveness. Conference on Learning@
Scale Conference: Proceedings of the Third Association for Computing Machinery (pp. 141-
144). New York, NY: Association for Computing Machinery.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 137
Payne, V. G., & Isaacs, L. D. (2017). Human motor development: A lifespan approach. New York,
NY: Routledge.
Pekrun, R., & Linnenbrink-Garcia, L. (2012). Academic emotions and student engagement. In S. L.
Christension et al. (Eds.), Handbook of research on student engagement (pp. 259-282).
Retrieved from:10.1007/978-1-4614, 2018-7_12
Pekrun, R., & Linnenbrink-Garcia, L. (Eds.). (2014). Inter-national handbook of emotions in
education. New York, NY: Taylor & Francis.
Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring emotions in
students’ learning and performance: The Achievement Emotions Questionnaire (AEQ).
Contemporary educational psychology, 36(1), 36-48.
Pekrun, R., Lichtenfeld, S., Marsh, H., Murayama, K., & Goetz, T. (2017). Achievement emotions
and academic performance: Longitudinal models of reciprocal effects. Child Development.
Retrieved from:10.1111/cdev.12704
Pintrich, P. R. (2000). An achievement goal theory perspective on issues in motivation terminology,
theory, and research. Contemporary Educational Psychology, 25(1), 92-104.
Poquet, O., Dawson, S., & Dowell, N. (2017). How effective is your facilitation? Group-level
analytics of MOOC forums. Proceedings of the Seventh International Learning Analytics &
Knowledge Conference (pp. 208-217). New York, NY: Association for Computing
Machinery.
Rana, J., Besche, H., & Cockrill, B. (2017). Twelve tips for the production of digital chalk-talk
videos. Medical Teacher, 1-7.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 138
Reiser, R. A. (2017). Eight trends affecting the field of instructional design and technology:
opportunities and challenges. In Learning and Knowledge Analytics in Open Education (pp.
139-147). New York, NY: Springer International.
Richardson, J. C., Maeda, Y., Lv, J., & Caskurlu, S. (2017). Social presence in relation to
students’ satisfaction and learning in the online environment: A meta-
analysis. Computers in Human Behavior, 71, 402-417.
Rolfe, V. (2015). A systematic review of the socio-ethical aspects of massive online open
courses. European Journal of Open, Distance and E-Learning, 18(1), 52-71.
Ross, J., Sinclair, C., Knox, J., Bayne, S., Macleod, H. (2014). Teacher experiences and academic
identity: The missing components of MOOC pedagogy. MERLOT Journal of Online
Learning and Teaching, 10(1), 57-69.
Rueda, R. (2011). The 3 dimensions of improving student performance: Matching the right solutions
to the right problems. New York, NY: Teachers College Press.
Ruipérez-Valiente, J. A., Muñoz-Merino, P. J., Gascón-Pinedo, J. A., & Kloos, C. D. (2017).
Scaling to massiveness with ANALYSE: A learning analytics tool for Open edX. IEEE
Transactions on Human-Machine Systems, PP(99), 1-6.
Salmon, G., Pechenkina, E., Chase, A. M., & Ross, B. (2016). Designing massive open online
courses to take account of participant motivations and expectations. British Journal of
Educational Technology, 1-11. Retrieved from: 10.1111/bjet.12497
Sankaran, S. R., & Bui, T. (2001). Impact of learning strategies and motivation on performance: A
study in web-based instruction. Journal of Instructional Psychology, 28(3), 191-198.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 139
Schunk, D. H. (1996). Learning theories. Englewood Cliffs, NJ: Prentice-Hall, Inc.
Selmer, J. (2006). Language ability and adjustment: Western expatriates in China. Thunderbird
International Business Review, 48(3), 347-368.
Sergis, S., Sampson, D. G., & Pelliccione, L. (2017). Educational design for MOOCs: Design
considerations for technology-supported learning at large scale. In Open Education: from
OERs to MOOCs (pp. 39-71). New York, NY: Springer, Berlin Heidelberg.
Shapiro, H. B., Lee, C. H., Roth, N. E. W., Li, K., Çetinkaya-Rundel, M., & Canelas, D. A. (2017).
Understanding the massive open online course (MOOC) student experience: An examination
of attitudes, motivations, and barriers. Computers & Education, 110, 35-50.
Shea, P., Li, C. S., & Pickett, A. (2006). A study of teaching presence and student sense of learning
community in fully online and web-enhanced college courses. The Internet and Higher
Education, 9(3), 175−190.
Shepherd, C. E., Bolliger, D. U., Dousay, T. A., & Persichitte, K. (2016). Preparing teachers for
online instruction with a graduate certificate program. TechTrends, 60(1), 41-47.
Boroujeni, M. S., Hecking, T., Hoppe, H. U., & Dillenbourg, P. (2017, March). Dynamics of MOOC
discussion forums. Proceedings of the 7th International Learning Analytics and Knowledge
Conference (pp. 128-137).
Siegel, D. J., & Carchidi, D. M. (2016). The meaning of MOOC-topia. Academe, 102(3), 28.
Simpson, O. (2017). Innovations in distance education student support: What are the chances? The
Envisioning Report for Empowering Universities, 56-58.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 140
Smith, N., Caldwell, H., Richards, M., & Bandara, A. (2017). A comparison of MOOC development
and delivery approaches. In Greg Anderson (Ed.), The International Journal of Information
and Learning Technology, 34(2), 152-164. England, UK: Emerald Publishing Limited.
Smith, P. A. (2001). Understanding self-regulated learning and its implications for accounting
educators and researchers. Issues in Accounting Education, 16(4), 663-700.
Song, S. H. (2000). Research issues of motivation in web-based instruction. Quarterly Review of
Distance Education, 1(3), 225-29.
Spector, J. M. (2017). A critical look at MOOCs. Open Education: from OERs to MOOCs, (pp. 135-
147). Retrieved from:10.1007/978-3-662-52925-6_7
Starcher, K., & Proffitt, D. (2011). Encouraging students to read: What professors are (and aren’t)
doing about it. International Journal of Teaching & Learning in Higher Education, 23(3),
396-407.
Stich, A. E., & Reeves, T. D. (2017). Massive open online courses and underserved students in the
United States. The Internet and Higher Education, 32, 58-71.
Stone, J. (2016). Awarding college credit for MOOCSs: The role of the American council on
education. Education Policy Analysis Archives, 24(38). Retrieved from:
http://dx.doi.org/10.14507/ epaa.24.1765
Swan, K., Garrison, D. R., & Richardson, J. (2009). A constructivist approach to online learning:
The community of inquiry framework. In C. R. Payne (Ed.), Information technology and
constructivism in higher education: Progressive learning frameworks (pp. 43-57). Hershey,
PA: IGI Global.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 141
Swan, K., Shea, P., Richardson, J. C., Ice, P., Garrison, D. R., Cleveland-Innes, M., & Arbaugh, J.
B. (2008). Validating a measurement tool of presence in online communities of inquiry. E-
mentor, 2(24), 1-12. Retrieved from: http://www.e-mentor.edu.pl/artykul/index/numer/
24/id/543
Thai, T. N., De Wever, B., & Valcke, M. (2017). The impact of a flipped classroom design on
learning performance in higher education: Looking for the best “blend” of lectures and
guiding questions with feedback. Computers & Education, 107, 113-126.
The Economist. (2014). The digital degree: The staied higher-education business is about to
experience a welcome earthquake. Retrieved from http://www.economist.com/news/briefing/
21605899-staid-higher-education-business-about-experience-welcome-earthquake-digital
Thornton, S., Riley, C., & Wiltrout, M. E. (2017). Criteria for video engagement in a biology
MOOC. Conference on Learning@ Scale Conference: Proceedings of the Fourth Association
for Computing Machinery (pp. 291-294). New York, NY: Association for Computing
Machinery.
Tomkin, J. H., & Charlevoix, D. (2014). Do professors matter?: Using an a/b test to evaluate the
impact of instructor involvement on MOOC student outcomes. Conference on Learning@
Scale Conference: Proceedings of the first Association for Computing Machinery (pp. 71-
78). New York, NY: Association for Computing Machinery.
Topala, I., & Tomozii, S. (2014). Learning satisfaction: Validity and reliability testing for students’
learning satisfaction questionnaire (SLSQ). Procedia - Social and Behavioral Sciences, 128,
380e386. Retrieved from http://dx.doi.org/10.1016/ j.sbspro.2014.03.175.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 142
Udacity, Inc. (2017). Jobs of tomorrow start here. Retrieved from
https://www.udacity.com/?utm_source=google&utm_medium=cpc&utm_campaign=google_
search_alpha_brand&gclid=CNmCidXynNMCFROUfgodobcOSA
Useem, J. (May 31, 2014) Business school, disrupted. The New York Times. Retrieved from
http://www.nytimes.com/2014/06/01/business/business-school-disrupted.html?_r=0
Veletsianos, G. (2017). Toward a generalizable understanding of Twitter and social media use
across MOOCs: who participates on MOOC hashtags and in what ways? Journal of
Computing in Higher Education, 29(1), 65-80.
Waard, I. (2011). Explore a new learning frontier: MOOCs. Learning Solutions Magazine, 25.
Wang, J., & Antonenko, P. D. (2017). Instructor presence in instructional video: Effects on visual
attention, recall, and perceived learning. Computers in Human Behavior, 71,
79-89.
Wang, Y., & Baker, R. (2015). Content or platform: Why do students complete MOOCs? MERLOT
Journal of Online Learning and Teaching, 11(1). 17-30.
Wang, Q., & Huang, C. (2017). Pedagogical, social and technical designs of a blended synchronous
learning environment. British Journal of Educational Technology. Retrieved from:
10.1111/bjet.12558
Waschull, S. B. (2005). Predicting success in online psychology courses: Self-discipline and
motivation. Teaching of Psychology, 32(3), 190-192.
Whitehill, J., Mohan, K., Seaton, D., Rosen, Y., & Tingley, D. (2017). Delving deeper into MOOC
student dropout prediction. Retrieved from: arXiv preprint arXiv:1702.06404.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 143
Whitehill, J., Williams, J. J., Lopez, G., Coleman, C. A., & Reich, J. (2015). Beyond prediction:
First steps toward automatic intervention in MOOC student stopout. Retrieved from
http://www.educationaldatamining.org/EDM2015/uploads/papers/paper_112.pdf
Whitmer, J., Schiorring, E., & James, P. (2014). Patterns of persistence: What engages students in a
remedial English writing MOOC? In Proceedings of the Fourth International Conference on
Learning Analytics and Knowledge, 279-280.
Wigfield, A., & Cambria, J. (2010). Students’ achievement values, goal orientations, and interest:
Definitions, development, and relations to achievement outcomes. Developmental Review,
30, 1-35.
Willingham, D. (2008). Learning styles don’t exist. [YouTube; viewed 27 March 2016]
http://www.youtube.com/watch?v=sIv9rz2NTUk
Wise, A. F., Cui, Y., Jin, W., & Vytasek, J. (2017). Mining for gold: Identifying content-related
MOOC discussion threads across domains through linguistic modeling. The Internet and
Higher Education, 32, 11-28.
Wlodkowski, R. J. (2008). Enhancing adult motivation to learn: A comprehensive guide for
teaching all adults (3rd ed). San Francisco, CA: Jossey-Bass.
Young, J. R. (2013). What professors can learn from ‘hard core’ MOOC students. Chronicle of
Higher Education, 59(37).
Yuan, L., & Powell, S. (2014). MOOCs and disruptive innovation: Implications for higher
education. In Proceedings of the e-learning paper.
PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 144
Zepke, N. (2017). A critique of mainstream student engagement. In Student Engagement in
Neoliberal Times (pp. 95-112). Singapore: Springer.
Zheng, S., Han, K., Rosson, M. B., & Carroll, J. M. (2016). The role of social media in MOOCs:
How to use social media to enhance student retention. Conference on Learning@ Scale
Conference: Proceedings of the Third Association for Computing Machinery (pp. 419-428).
New York, NY: Association for Computing Machinery.
Zheng, S., Rosson, M. B., Shih, P. C. & Carroll, J. M. (2015). Designing massive open online
courses as interactive places for collaborative learning. Conference on Learning@ Scale
Conference: Proceedings of the Second Association for Computing Machinery (pp. 343-
346). New York, NY: Association for Computing Machinery.
Zheng, S., Wisniewski, P., Rosson, M. B., & Carroll, J. M. (2016). Ask the Instructors: Motivations
and Challenges of Teaching Massive Open Online Courses. Conference on Learning@ Scale
Conference: Proceedings of the 19th Association for Computer-Supported Cooperative Work
& Social Computing (pp. 206-221). New York, NY: Association for Computing Machinery.

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 145
Appendix
Survey Protocol
The purpose of this survey is to analyze student engagement and retention in MandarinX’s
MOOC: Basic Mandarin Chinese (Level One)
Part 1: DEMOGRAPHICS
1. What is your gender?
__ Female
__ Male
__ Other
__ Decline to state
2. What is your age?
__ under 18
__ 18–25
__ 26–35
__ 36–45
__ Above 45
3. What is your predominant ethnic background?
__ Caucasian
__ African
__ Hispanic
__ Asian
__ Other
4. What languages do you speak fluently?
______________________
5. What is your occupation?
______________________
6. What education level have you completed?
__ None
__ Primary
__ Middle
__ Secondary
__ Associate
__ Bachelor’s
__ Masters
__ Doctorate
__ Other

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 146
Part 2: SELF EVALUATION

1. Why did you choose to learn Mandarin Chinese?
__ Business opportunity
__ School requirement
__ Curiosity
__ Travel
__ Social
__ Other (Please specify ______________)

2. What was your primary goal/expectation of this course?
__ Improve my ability to communicate at work, or with clients
__ Marriage/family communication
__ Solid foundation for further Mandarin language learning
__ Career marketability
__ Personal hobby
__ Other (Please specify ______________)

3. How many hours did you spend on average with this course every week?
__ Less than 3 hours
__ 3-5 hours
__ 6-9 hours
__ 10-12 hours
__ 13-15 hours
__ More than 15 hours

4. How did you study while taking this course? (Check as many as apply)
__ Online study group members
__ Language buddy
__ Family members
__ School classmates
__ Random friends
__ Alone
__ I did not study
__ Other

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 147

5. What made you choose this Mandarin MOOC?
__ It is free
__ It works well with my school/work schedule
__ It is well-structured with adequate exercises
__ It is taught in English
__ It has a good reputation/affiliated with top-tier universities
__ Someone referred it to me
__ Other (Please specify ______________)

6. If you had any trouble meeting course deadlines for submitting work, what impeded you from
submitting assignments, completing video lectures, or exams?
__ Laziness/procrastination
__ Confusion about the instruction
__ Busy schedule
__ Personal emergency
__ Lack of interest
__ I don’t think I would get a good grade (or helpful feedback)
__ Nothing impeded me
__ Other (Please specify ______________)

7. How would you rate yourself in terms of being familiar with the courseware for accessing all
of the units for the lessons?
Poor 1 2 3 4 5 6 Excellent

8. How would you rate your knowledge regarding the differences among accents/dialects/usages
in various Mandarin-speaking countries and regions after the course?
Poor 1 2 3 4 5 6 Excellent

9. How would you rate your ability to seek additional resources to help with learning Mandarin?
Poor 1 2 3 4 5 6 Excellent

10. I reflected on my learning progress and adapted my strategies to assist with my learning.
Disagree 1 2 3 4 5 6 Agree

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 148
11. I am confident that I can complete all of the assignments and pass the final exam.
Disagree 1 2 3 4 5 6 Agree

12. I feel happy while taking this course, including watching videos, working on exercises, and
interacting with peers.
Disagree 1 2 3 4 5 6 Agree

Part 3: COURSE EVALUATION

13. I stayed with this course because it covers everything that I am looking for while leaning
Mandarin.
Disagree 1 2 3 4 5 6 Agree

14. I found the learning environment to be encouraging.
Disagree 1 2 3 4 5 6 Agree

15. I found that the feedback helps me adapt to an online learning environment.
Disagree 1 2 3 4 5 6 Agree

16. Cost-savings (travel, materials) were the main reason for me to register for this language
MOOC.
Disagree 1 2 3 4 5 6 Agree

17. I enrolled in this MOOC because the certificate is issued by edX.
Disagree 1 2 3 4 5 6 Agree

18. I found the course design on the platform to be interactive and engaging.
Disagree 1 2 3 4 5 6 Agree

19. It was beneficial for me to learn Mandarin through live broadcasting and online meetings led
by the instructor.
Disagree 1 2 3 4 5 6 Agree

20. It was helpful for me to use the discussion forum to interact with peers and support each
other.
Disagree 1 2 3 4 5 6 Agree

21. The course allowed me to re-learn the concepts that I did not understand previously in other
language programs/institutions.
Disagree 1 2 3 4 5 6 Agree

22. The videos were very much like just being in the classroom, where the teacher is talking and
writing on the board.
Disagree 1 2 3 4 5 6 Agree

PERSISTENCE IN MASSIVE OPEN ONLINE COURSES 149
23. The instructor’s manner of speaking and presentation skills are fluent, relaxed and natural.
Disagree 1 2 3 4 5 6 Agree

24. The instructor’s attitude kept me interested in watching the videos.
Disagree 1 2 3 4 5 6 Agree

25. What did you find most engaging while taking this course? (Check all that apply)
__ High quality videos
__ Practical exercises and peer assessment
__ Online forum discussions
__ Explicit guidance (including weekly newsletter and cultural notes videos)
__ Technical support/course content assistance
__ Family/friend encouragement
__ Other (Please specify ______________)

26. Please rate your satisfaction with the overall course design.
Poor 1 2 3 4 5 6 Excellent

Abstract (if available)

Abstract The use of Massive Open Online Courses (MOOCs) has been viewed as a transformational approach to increase educational opportunities to a global audience. However, low persistence rates have been highlighted as a major criticism. Evidence shows that only a small percentage of MOOC participants complete their courses and little is understood about the specific MOOC design and implementation factors that influence retention. This study reports a survey of 696 participants who were enrolled in Mandarin Chinese language MOOCs. Clark and Estes gap analysis was utilized to investigate the knowledge, motivation, and organizational assumed causes of persistence in MOOCs. Findings demonstrated that teaching presence has a significant impact on MOOC participants’ perspectives and the success of the course itself. The design of MOOC course content was also found to be a significant predictor of MOOC participants’ persistence. Understanding what institutional, course, and student characteristics are related to student success in this relatively new educational modality is important to increasing student retention and success.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Validation matters - student experiences in online courses: a mixed method study

PDF

Creating effective online educational content: an evaluation study of the influences affecting course developers’ abilities to create online content with engaging learning strategies

PDF

Student engagement in online education: an evaluation study

PDF

Coaching to increase retention in online degree completion programs

PDF

An exploration of student experiences in a preparation program for online classes in the California community college system

PDF

Retention rate of online students in the associate's degree program in addiction education counseling: a gap analysis

PDF

A comparative study of motivational predictors and differences of student satisfaction between online learning and on-campus courses

PDF

The development and implementation of global partnerships in student affairs

PDF

Effective course design for improving student learning: a case study in application

PDF

Changing course, creating opportunity: a study on the implementation of an early college model for African American ninth grade students

PDF

Creating and implementing an offshore graduate program: a case study of leadership and development of the global executive MBA program

PDF

Increasing the number of petroleum engineering students in the United Arab Emirates: an improvement model

PDF

The role of divisional principals in teacher retention in East African international schools

PDF

Improved student success at California State University, Dominguez Hills: a promising practice study

PDF

Relationships between a community college student’s sense of belonging and student services engagement with completion of transfer gateway courses and persistence

PDF

Evaluation of the career development counseling in public high school in Qatar

PDF

The TOEFL exam gateway: understanding performance factors among community college students

PDF

Measuring student persistence at an alternative charter high school: internal school evaluation and external policy implications

PDF

Preparing international students for management school through pathway programs

PDF

Building national capacity in student affairs at a local university in Qatar: A gap analysis

Asset Metadata

Creator Chen, Estella Youmin (author)

Core Title Understanding student persistence in massive open online courses (MOOCs): an evaluation study

School Rossier School of Education

Degree Doctor of Education

Degree Program Global Executive

Publication Date 07/27/2019

Defense Date 04/19/2017

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag Massive Open Online Courses (MOOCs),materials design,MOOC scheme,OAI-PMH Harvest,online pedagogy,organizational performance,student emotions,teaching presence,video quality

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Robison, Mark Power (committee chair), Gallagher, Karen Symms (committee member), Gallagher, Raymond John (committee member)

Creator Email estella@mandarinx.com,goodestella@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-417880

Unique identifier UC11213907

Identifier etd-ChenEstell-5648.pdf (filename),usctheses-c40-417880 (legacy record id)

Legacy Identifier etd-ChenEstell-5648.pdf

Dmrecord 417880

Document Type Dissertation

Format application/pdf (imt)

Rights Chen, Estella Youmin

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA