Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Preparing university faculty for distance education: an evaluation study
(USC Thesis Other)
Preparing university faculty for distance education: an evaluation study
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Running head: PREPARING FACULTY FOR DISTANCE EDUCATION 1
Preparing University Faculty for Distance Education: An Evaluation Study
by
Justin Fowler
A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2020
Copyright 2020 Justin Fowler
PREPARING FACULTY FOR DISTANCE EDUCATION 2
DEDICATION
To my father, Mark Daniel Fowler (24 June 1960 – 30 August 2017), who raised me as a single
parent and always encouraged me to go to college even though he had never gone himself.
I miss you Dad, and I’m sorry you won’t get to read this. Everything I have ever achieved can
be traced back to your encouragement and guidance. Thank you for always asking me “what do
you want to go to college for” instead of “do you want to go to college?” Thank you for buying
me a computer when I was young because you knew that it would help prepare me for the future.
Thank you for teaching me the value of open, honest, and direct communication.
Thank you for everything. I will always love you.
PREPARING FACULTY FOR DISTANCE EDUCATION 3
ACKNOWLEDGEMENTS
Thank you to my partner Jenn for supporting me throughout this program. Thank you for
listening when I needed someone to talk to. Thank you for understanding when I was absent
during late night classes and writing sessions. I look forward to supporting you throughout the
remainder of your own doctoral studies. I love you.
Thank you to my dissertation committee, Dr. Kathy Stowe, Dr. Helena Seli, and Dr. Monique
Datta, for all of your invaluable guidance and feedback. I will be forever grateful to you for your
assistance throughout this process. Thank you for pushing me to think critically and to become a
better researcher and writer. I would like to especially thank Dr. Seli for guiding me throughout
my entire experience at USC, from the very beginning when I enrolled in the first cohort of the
Learning Design and Technology program in September of 2014.
Thank you to all of the faculty at USC who helped shape this experience. Each of you
contributed something that helped with the construction of this dissertation.
Thank you to all of my classmates for your support and encouragement. I am inspired by the
work that you are all doing and I am humbled to be included among you. I believe that we have
both the power and the responsibility to make the world a better place through our leadership.
PREPARING FACULTY FOR DISTANCE EDUCATION 4
TABLE OF CONTENTS
Dedication ....................................................................................................................................... 2
Acknowledgements ......................................................................................................................... 3
List of Tables .................................................................................................................................. 7
List of Figures ................................................................................................................................. 8
Abstract ........................................................................................................................................... 9
Introduction to Problem of Practice .............................................................................................. 10
Organizational Context and Mission ............................................................................................ 11
Importance of Addressing the Problem ........................................................................................ 11
Organizational Performance Goal ................................................................................................. 13
Stakeholder Group of Focus ......................................................................................................... 13
Purpose of the Project and Questions ........................................................................................... 14
Background of the Learning Management System ....................................................................... 15
Review of the Literature ............................................................................................................... 16
History of Distance Education .................................................................................................. 16
Research on the Benefits of Faculty Training........................................................................... 17
TPACK Framework for Measuring Faculty Proficiency .......................................................... 18
The Clark and Estes Gap Analytic Framework ............................................................................ 20
Knowledge, Motivation and Organizational Influences ............................................................... 21
Knowledge and Skills ............................................................................................................... 22
Knowledge influences. .......................................................................................................... 22
Understanding of how to use basic LMS features. ........................................................... 23
Understanding of how to adapt content to LMS. .............................................................. 24
Understanding of how to apply pedagogical strategies using technology. ....................... 25
Motivation Influences ............................................................................................................... 26
Self-efficacy theory. .............................................................................................................. 26
Expectancy value theory. ...................................................................................................... 27
Organizational Influences ......................................................................................................... 29
General theory. ...................................................................................................................... 29
SMD faculty specific factors. ............................................................................................... 30
The organization needs to provide faculty members with training opportunities. ........... 30
The organization needs to have a system of monitoring faculty performance. ................ 30
Interactive Conceptual Framework ............................................................................................... 32
PREPARING FACULTY FOR DISTANCE EDUCATION 5
Qualitative Data Collection and Instrumentation ......................................................................... 34
Interviews .................................................................................................................................. 35
Artifact Analysis ....................................................................................................................... 36
Findings and Results ..................................................................................................................... 37
Procedural and Conceptual Knowledge Findings ..................................................................... 39
Most faculty members in the department only have experience with performing basic
tasks in the LMS such as uploading files .............................................................................. 39
Faculty members are able to describe how they would share instructional content in an
online course in varying levels of detail ............................................................................... 43
Most faculty members can describe pedagogical rationales for utilizing technology
in online courses. .................................................................................................................. 45
Summary of knowledge influence findings .......................................................................... 48
Self-Efficacy and Expectancy Value Motivation Influence Findings ...................................... 49
Faculty members are moderately confident in their ability to utilize the LMS .................... 49
Faculty members believe that distance education is valuable for different reasons ............. 50
Faculty members are willing to teach online courses if the department does not
attempt to launch a fully online degree program .................................................................. 52
Summary of motivation influence findings .......................................................................... 53
Training and Performance Evaluation Organizational Influence Findings .............................. 54
The faculty members have been provided with few opportunities for LMS training. .......... 54
There is no performance evaluation system in place for faculty which takes into
account faculty utilization of the LMS. ................................................................................ 56
Summary of organizational influence findings. .................................................................... 57
Summary of Results and Findings ............................................................................................ 58
Knowledge Recommendations for Practice to Address KMO Influences .................................... 60
Procedural Knowledge of Utilizing LMS Tools ....................................................................... 62
Conceptual Knowledge of Online Course Design .................................................................... 63
Conceptual Knowledge of Pedagogical Strategies for Distance Education ............................. 64
Motivation Recommendations for Practice to Address KMO Influences .................................... 65
Provide Modeling, Practice, and Feedback ............................................................................... 66
Communicate the Importance and Value .................................................................................. 66
Organization Recommendations for Practice to Address KMO Influences ................................. 67
Faculty Performance Evaluation ............................................................................................... 69
Faculty LMS Training............................................................................................................... 70
Integrated Implementation and Evaluation Plan Framework ....................................................... 71
PREPARING FACULTY FOR DISTANCE EDUCATION 6
Organizational Purpose, Need and Expectations ...................................................................... 72
Level 4: Results and Leading Indicators ................................................................................... 72
Level 3: Behavior ...................................................................................................................... 73
Required drivers .................................................................................................................... 74
Organizational support .......................................................................................................... 76
Level 2: Learning ...................................................................................................................... 76
Program ................................................................................................................................. 77
Evaluation of the components of learning ............................................................................ 79
Level 1: Reaction ...................................................................................................................... 81
Evaluation Tools ....................................................................................................................... 82
Delayed for a period after the program implementation ....................................................... 82
Data Analysis and Reporting .................................................................................................... 83
Summary ................................................................................................................................... 83
Limitations of the Research .......................................................................................................... 84
Recommendations for Future Research ........................................................................................ 85
Conclusion .................................................................................................................................... 85
References ..................................................................................................................................... 88
Appendix A: Participating Stakeholders with Sampling Criteria for Interview,
Survey and Observation .............................................................................................................. 100
Appendix B: Interview Protocol ................................................................................................. 102
Appendix C: Artifact Review Protocol ....................................................................................... 105
Appendix D: Credibility and Trustworthiness ............................................................................ 106
Appendix E: Ethics ..................................................................................................................... 108
Appendix F: Immediate Feedback After Training Sessions ....................................................... 111
Appendix G: Feedback Survey Two Weeks after Training Program ......................................... 112
Appendix H: Sample Figure for Level 4 Goal Reporting ........................................................... 113
Appendix I: Sample Figures for Level 3 Goal Reporting ........................................................... 114
PREPARING FACULTY FOR DISTANCE EDUCATION 7
LIST OF TABLES
Table 1: Organizational Mission, Goal, and Stakeholder Goals ................................................... 14
Table 2: Summary of Knowledge Influences ............................................................................... 25
Table 3: Summary of Motivation Influences ................................................................................ 29
Table 4: Summary of Organization Influences ............................................................................. 31
Table 5: Summary of Results and Findings .................................................................................. 59
Table 6: Summary of Knowledge Influences and Recommendations .......................................... 60
Table 7: Summary of Mortivation Influences and Recommendations ......................................... 65
Table 8: Summary of Organization Influences and Recommendations ....................................... 69
Table 9: Outcomes, Metrics, and Methods for External and Internal Outcomes ......................... 73
Table 10: Critical Behaviors, Metrics, Methods, and Timing for Evaluation .............................. 74
Table 11: Required Drivers to Support Critical Behaviors ........................................................... 75
Table 12: Evaluation of the Components of Learning for the Program ....................................... 80
Table 13: Components to Measure Reactions to the Program ...................................................... 81
PREPARING FACULTY FOR DISTANCE EDUCATION 8
LIST OF FIGURES
Figure 1. Venn diagram outlining the TPACK framework ......................................................... 19
Figure 2. Graphical representation of conceptual framework ..................................................... 33
Figure 3. LMS features used by SMD faculty in Moodle courses............................................... 40
Figure 4. Example of hypothetical data for Level 4 goal reporting ........................................... 113
Figure 5. Example of hypothetical data for Level 3 goal reporting. .......................................... 114
Figure 6. Example of hypothetical data for Level 3 goal reporting. .......................................... 114
Figure 7. Example of hypothetical data for Level 3 goal reporting. .......................................... 115
Figure 8. Example of hypothetical data for Level 3 goal reporting. .......................................... 115
PREPARING FACULTY FOR DISTANCE EDUCATION 9
ABSTRACT
Distance education is becoming a common practice in the field of higher education and presents
new challenges for faculty members who teach in online environments. It is especially important
that faculty members who teach online are familiar with their institution’s learning management
system (LMS). Using the gap analysis framework created by Clark and Estes (2008), the
purpose of this study was to evaluate the knowledge, motivation, and organizational barriers or
influences to the development of LMS competency among faculty members in a specific
department at a small medical school. Using a census sample approach, all 10 full-time faculty
members in the department were invited to participate in semi-structured qualitative interviews.
Additionally, artifact analysis of faculty activity in the LMS was utilized in order to observe
which features faculty members are currently utilizing. The findings of this study suggest that
several of the assumed barriers will present challenges for the faculty if the department should
attempt to create distance education course offerings. A faculty training program for addressing
these barriers is proposed.
PREPARING FACULTY FOR DISTANCE EDUCATION 10
Introduction to Problem of Practice
Distance education programs in the United States are becoming more ubiquitous every
year. Overall enrollments in distance education have increased every year from 2002 to 2016,
with 31.6% of all higher education students enrolled in 2016 (Seamen, Allen & Seamen, 2018).
A total of over 6.3 million students, took at least one online course, and approximately three
million of those students took their entire degree program online (Seamen, Allen & Seamen,
2018). The increasing number of students participating in these courses is driving an increase in
the demand for distance education instructors but teaching in online courses requires faculty
members to learn new technology skills which may not have been acquired from their previous
experiences in conventional classrooms (Brinkley-Etzkorn, 2018; Lee & Hirumi, 2004; Moore,
Sheffield, Robinson & Phillips, 2014; Wilson & Stacey, 2004).
Despite the need for faculty members to learn these new skills, a survey of 225 higher
education institutions in North America found that only 58% of those institutions reported
having any form of mandatory technology training for faculty members teaching an online class
for the first time (Poulin, 2013). Furthermore, faculty members without prior experience
teaching distance education courses are more likely to believe that distance education is
ineffective and that it contributes to a decline of educational quality at their institution (Ulmer,
Watson & Derby, 2007). Allen and Seamen (2013) found that only 30.2% of chief academic
officers believed that their faculty members accepted the value and legitimacy of online
education, but that 69.1% of these officers also reported that online learning is critical to their
long-term strategy. These findings suggest that, while interest in distance education is increasing
among both students and academic leaders, some institutions may be experiencing challenges
PREPARING FACULTY FOR DISTANCE EDUCATION 11
with adequately training and motivating their faculty members to teach in these distance
education programs.
Organizational Context and Mission
Small University of Medicine (SUM), a pseudonym for a private institution of higher
education located in the Pacific Northwest, is a medical school offering a variety of
undergraduate and graduate degrees. The mission of SUM is to train future medical
professionals in the art and science of practicing medicine. SUM has fewer than 750 full-time
students, all of whom are enrolled in residential programs. The institution does not currently
offer any distance education programs. The institution recently received approval from its
regional accreditor to launch a distance education Master’s degree program, with the first cohort
of students anticipated to begin taking online courses in the fall term of 2019.
With the anticipated arrival of SUM’s first-ever distance education program now being
promoted internally within the institution, other academic departments at SUM are beginning to
express interest in developing their own distance education offerings. One such department, the
Specialized Medicine Department (SMD), a pseudonym, has started to investigate the process
and requirements for receiving approval from its programmatic accreditor to begin offering
distance education courses. The SMD academic leaders will therefore need to identify faculty
members with the necessary knowledge and motivation to be effective distance education
instructors and ensure that they are given technology training opportunities.
Importance of Addressing the Problem
The problem of ensuring that faculty members teaching distance education courses are
provided with adequate training is important to address for several reasons. When faculty
members participate in technology training, this will typically increase their self-efficacy for
PREPARING FACULTY FOR DISTANCE EDUCATION 12
teaching online (Abbitt, 2011; Dunbar & Melton, 2018; Gonzales, Pickett, Hupert, & Martin,
2002; Watson, 2006). Adequate technology training has also been identified as a factor which
can improve the retention and satisfaction of faculty in online distance education programs
(Bolliger & Wasilik, 2009; Bolliger, Inan & Wasilik, 2014; Green, Alejandro & Brown, 2009;
Scott, Lemus, Knotts & Oh, 2016; Wiesenmayer, Kupczynski & Ice, 2008). In turn, higher
faculty retention rates promote student retention in these programs (Kane, Shaw, Pang, Salley &
Snider, 2015), and high levels of faculty satisfaction are associated with higher levels of student
satisfaction (Bolliger, 2004; Hartman, Dziuban & Moskal, 2000; Tompkins, Brecht, Tucker,
Neander & Swift, 2016). Additionally, when faculty members perceive that adequate training
for distance education is not provided by the institution, they may be unwilling to participate in
the institution’s distance education offerings at all, which can create challenges for faculty
recruitment (Betts, 2014; Lee & Busch, 2005).
Collectively, the relationship between adequate training for faculty members and faculty
retention, satisfaction, and recruitment suggests that institutions which seek to deliver effective
distance education programs should ensure that faculty members are provided with adequate
opportunities for technology training in order to promote faculty retention and participation.
Furthermore, these findings also suggest that higher levels of faculty retention and satisfaction
will be associated with better student retention, which is often lower than the retention rates in
conventional on-campus programs (Bawa, 2016; Carr, 2000). It is therefore imperative that any
institution which seeks to launch a successful distance education program must also ensure that
there is an effective organizational strategy in place for the training of distance education faculty.
PREPARING FACULTY FOR DISTANCE EDUCATION 13
Organizational Performance Goal
An organizational goal has been set in which the SMD will begin delivering distance
education courses that are approved by the programmatic accreditor by January 2020. To
facilitate achieving this organizational goal, SMD has set an additional stakeholder goal that
faculty members teaching these courses will achieve competency with the software tools
provided in SUM’s learning management system (LMS) by January 2020. Faculty competency
with the LMS will be evaluated based on their ability to utilize the LMS gradebook for providing
student feedback, to utilize the video creation tools within the LMS for uploading video content,
and to other instructional tools within the LMS such as discussion forums. Additionally, SMD
will seek to ensure that faculty members who teach these courses are motivated to engage in
regular and substantive interaction with distance education students and that their participation in
distance education courses is voluntary.
Stakeholder Group of Focus
The stakeholder group of focus for this study was the 10 full-time faculty members who
currently teach classes in SMD. This stakeholder group was selected because, while there are
many factors which may affect the success or failure of a distance education program, the
performance of faculty members is one of the most important (Cole, Shelley & Swartz, 2014).
Without faculty members who are sufficiently trained or motivated, any distance education
program started within SMD is unlikely to succeed. This study was intended to identify any
training gaps that may need to be addressed in order for SMD faculty to prepare for the delivery
of distance education courses. The study also sought to identify whether or not SMD currently
employs enough faculty members who would be willing to participate in distance education, or if
the department will need to prioritize addressing issues of motivation before addressing any
PREPARING FACULTY FOR DISTANCE EDUCATION 14
training gaps. Clark and Estes (2008) would recommend addressing any motivational gaps
before expending institutional resources on addressing knowledge gaps. Therefore, the results of
this study will assist institutional leaders in the process of both identifying obstacles and
determining the order in which those obstacles should be addressed.
Table 1
Organizational Mission, Goal, and Stakeholder Goals
Organizational Mission
The mission of SUM is to train future medical professionals in the art and science of medical
practice.
Organizational Global Goal
By January 2020, the SMD will begin delivering distance education courses that are
approved by the programmatic accreditor.
Stakeholder Goal
By January 2020, the SMD faculty will achieve competency with the software tools provided
in SUM’s LMS
Purpose of the Project and Questions
The purpose of this project was to conduct an assessment of the knowledge, motivational,
and organizational needs required to achieve the organizational performance goal of SUM to
begin offering distance education courses within the SMD by January 2020. While the collective
performance of all organizational stakeholders will contribute to the achievement of this goal, the
focus for this study was on the 10 full-time faculty members currently employed by SMD.
These faculty members within SMD are the primary candidates for potentially teaching distance
education courses within the department. The results of this study can be used to help determine
what needs the faculty members might have if they were to teach these courses at SUM. As
such, the questions that guided this study were the following:
PREPARING FACULTY FOR DISTANCE EDUCATION 15
1. To what extent is SUM currently on track to begin offering distance education courses
within the SMD by January 2020 that are fully compliant with the standards on distance
education that are established by the programmatic accreditor of the SMD?
2. What are the knowledge, motivation and organizational elements related to the ability of
the full-time faculty within the SMD to achieve competency with the software tools
provided in SUM’s LMS by January 2020?
Background of the Learning Management System
The LMS platform utilized at SUM is called Moodle. Moodle is a freely available open-
source LMS with over 90 million users worldwide (“About Moodle,” 2018). Moodle is the third
most commonly used LMS in the United States, after Blackboard and Canvas, with a market
share of approximately 23% (Feldstein, 2018). SUM currently utilizes a Moodle site that is
supported by a certified Moodle Partner, which is a third party company offering paid hosting
and management for Moodle. The Moodle Partner provides assistance with the administration of
Moodle, including management of the hosting server, implementation of version updates, review
of user permissions and security practices, installation of plug-ins, and tier three technical
support. Moodle provides key features that are common to many LMS platforms, including
systems for assignment submissions, student feedback, discussion forums, and more.
SUM has integrated additional third-party software plug-ins with Moodle for enhanced
functionality, such as Kaltura, a video management platform which allows faculty members to
record and publish instructional videos for students. SUM has also contracted with a different
third party company in addition to the Moodle Partner for the provision of 24/7 tier one technical
support for students and faculty. Technical support issues that are not resolved by the tier one
support team are escalated to SUM staff who handle tier two technical support, and technical
PREPARING FACULTY FOR DISTANCE EDUCATION 16
issues that cannot be resolved by SUM staff are further escalated to the tier three technical
support team at the Moodle Partner.
Review of the Literature
There is an extensive body of literature on the field of distance education and on the
subject of faculty development within this context. This review of the literature will summarize
the history of distance education in the United States and some research on the benefits of faculty
training and development. Finally, the review will explore the background literature which
forms the basis of the conceptual framework for this research.
History of Distance Education
Training faculty in the delivery of distance education has been a concern within the field
of higher education for over a century. Since as early as 1873, some institutions have engaged in
a form of distance education known as correspondence courses, where students and faculty
members shared assignments and feedback via written letters (Caruth & Caruth, 2013).
However, distance education as it is known today is largely synonymous with education
delivered via the internet. In 1984, the “Electronic University Network” or EUN was established
in order to encourage the development of online courses, EUN offered its first online course in
1986. The EUN marked the first major foray into online education by established institutions of
higher education and these initial courses offered by the EUN were likely the first computer-
based courses to ever be offered on anything resembling the internet as it is known today
(Osgood, 1986).
It was not long after these first online courses that institutions began to take an increasing
interest in the opportunities presented by this new medium. With the increased interest came an
acknowledgement of the critical role that faculty would play in these developments. Several
PREPARING FACULTY FOR DISTANCE EDUCATION 17
authors acknowledged that faculty resistance to instructional technology was a key barrier that
would prevent the growth of distance education programs (McNeil, 1990; Stinehart, 1988).
Beaudoin (1990) wrote a widely-cited article in the American Journal of Distance Education
which discussed the changing role of faculty members in online learning programs,
acknowledging that teaching in this new medium would require new skills for faculty members
who were accustomed to teaching in more conventional classroom settings. Similarly, Dillon
and Walsh (1992) wrote a review of the research related to faculty training and other issues
related to faculty participation in distance education. They found at the time that the majority of
the research to date had been primarily focused on student outcomes, with little attention paid to
the faculty themselves.
Since those early years in the field of online education, the medium has increased
exponentially in popularity (Seamen et al., 2018) and a great deal of research has been conducted
on faculty training for distance education. This review of the literature will provide an overview
of the research on the benefits of faculty training, summarize the technological pedagogical
content knowledge (TPACK) framework for measuring faculty competence, and present a
summary of the Clark and Estes Gap Analytic conceptual framework which served as the basis
for examining the knowledge, motivational, and organizational challenges that were discovered
through the course of this research study.
Research on the Benefits of Faculty Training
The importance of providing training for faculty members in distance education programs
cannot be understated. There is ample evidence that faculty who participate in technology
training express higher levels of self-efficacy for online teaching (Abbitt, 2011; Dunbar &
Melton, 2018; Gonzales, Pickett, Hupert, & Martin, 2002; Horvitz, Beach, Anderson & Xia,
PREPARING FACULTY FOR DISTANCE EDUCATION 18
2015; Watson, 2006). Clark and Estes (2008) discuss the importance of self-efficacy as a
motivational construct. Faculty members with higher levels of confidence are more likely to
participate in distance education and invest effort into their participation.
It is particularly important that faculty members who teach online invest this effort
because a key predictor of student outcomes in online programs is the frequency of interaction
with faculty members (Abrami, Bernard, Bures, Borokhovski & Tamim, 2011; Kuo, Walker,
Schroder & Belland, 2014; Zhao, Lei, Yan, Lai & Tan, 2005). The importance of frequent
interaction with students is another aspect of distance education that faculty members should
receive training for. It is important for the institution to provide faculty members with training
that will emphasize the effect of frequent and substantive interaction on student success (Baran
& Correia, 2014; Richardson et al., 2015).
TPACK Framework for Measuring Faculty Proficiency
The idea that faculty members must be trained to interact with students when teaching
online speaks to the need for faculty members to receive pedagogical training in addition to
practical technology training. It is not sufficient to simply ensure that faculty members know
how to use the technology tools available to them; it is also necessary to ensure that faculty
understand the best pedagogical practices for using those tools to promote student learning
outcomes. However, faculty members cannot implement good pedagogical ideas without
understanding how to use the tools. An effective faculty training program must address both
needs. This idea is perhaps best expressed by the TPACK framework, or “technological
pedagogical content knowledge” (Herring, Koehler, & Mishra, 2016; Koehler & Mishra, 2009;
Mishra & Koehler, 2006). TPACK codifies the idea that in order to be effective instructors,
faculty members must have expertise in three distinct areas: technological knowledge,
PREPARING FACULTY FOR DISTANCE EDUCATION 19
pedagogical knowledge, and content knowledge. It also distinguishes between various
combinations of these knowledge areas. For example, technological pedagogical knowledge
represents the knowledge of how to implement different pedagogies using technology. These
various combinations of knowledge are summarized in Figure 1 (Koehler, 2011).
Figure 1. Venn diagram outlining the TPACK framework developed by Koehler and Mishra
(2009). Each circle represents a different element of faculty knowledge. Overlapping areas
between circles represent the intersections of these knowledge elements. Reproduced by
permission of the publisher, copyright 2012 by tpack.org.
The TPACK framework is an extension of the PCK framework originally popularized by
Shulman (1986) and it acknowledges the increasingly prevalent significance of computer
technology for the modern faculty member. The TPACK framework has been used by many
researchers in practical evaluations of faculty knowledge at specific institutions (Archambault &
Crippen, 2009; Chai, Koh, & Tsai, 2010; Graham et al., 2009; Harris & Hofer, 2011; Read,
PREPARING FACULTY FOR DISTANCE EDUCATION 20
Morel, Butcher, Jensen & Lang, 2019), and there are a wide variety of different instruments that
have been used to measure TPACK among faculty populations (Koehler, Shin & Mishra, 2012).
The usefulness of the TPACK framework is most easily communicated by way of
example. Take, for example, the case of a faculty member with strong technology knowledge
and strong content knowledge, but weak pedagogical knowledge (Rodgers, 2018). This
hypothetical faculty member might use a variety of tools to create course content in a series of
PDF files with very detailed and complex content, and then upload these files to the LMS for
students to browse. Creating these resources and uploading them to the LMS would certainly be
a strong demonstration of both technological and content knowledge, but it would also
demonstrate a lack of understanding that students should also be provided with lesson plans and
activities that encourage interaction and active learning.
The Clark and Estes Gap Analytic Framework
Clark and Estes (2008) provide an analytic framework that examines organizational and
stakeholder performance goals and provides organizational leaders with tools that can be utilized
to identify any gap between the organization’s actual performance level and the performance
goal. If a gap is identified, the framework can be used to examine the stakeholder knowledge,
motivation, and organizational influences that may impact the gap. There are four knowledge
and skill types, as outlined by Krathwohl (2002) and Mayer (2011): factual, conceptual,
procedural, and metacognitive. Factual knowledge refers to knowledge of basic, specific details
or terminology that are related to different contexts or professional fields (Krathwohl, 2002).
Conceptual knowledge is somewhat more complex than factual knowledge, and refers to the
ability to apply principles, theories, or models to new and unique situations or problems
(Krathwohl, 2002; Mayer, 2011). Procedural knowledge refers to the process or the series of
PREPARING FACULTY FOR DISTANCE EDUCATION 21
steps that are required for an individual to complete a particular task (Krathwohl, 2002; Mayer,
2011). Metacognitive knowledge refers to an individual’s knowledge of their own knowledge; in
other words, an individual’s ability to analyze and think about his or her own thought processes
(Krathwohl, 2002; Mayer, 2011). Motivational principles such as values, attributions, and self-
efficacy can be taken into consideration when identifying the cause of a performance gap
(Rueda, 2011). Organizational influences may include other factors, such as workplace culture,
financial resources, or work processes (Clark & Estes, 2008).
Each of these factors in Clark and Estes’ (2008) gap analysis are addressed in terms of
SMD faculty’s knowledge, motivation, and organizational needs to meet their performance goal
of achieving competency with the software tools provided in SUM’s LMS by January 2020. The
first section will discuss assumed influences on stakeholder performance in the context of
knowledge and skills. The following section will examine motivational influences on
stakeholder performance. Finally, the last section will examine organizational influences on
stakeholder performance. Each of these assumed influences on achievement of the stakeholder
goal are then examined through the methodology discussed in the Data Collection and
Instrumentation section.
Knowledge, Motivation and Organizational Influences
This review of scholarly research on knowledge and motivation is intended to outline
what is required for SMD faculty to achieve their stakeholder goal of achieving by January 2020
competency with the software tools provided in SUM’s LMS. The study first examined the
assumed knowledge influences, followed by the motivational influences as they relate to the
stakeholder goal. The final section examines the assumed organizational influences.
PREPARING FACULTY FOR DISTANCE EDUCATION 22
Knowledge and Skills
Clark and Estes (2008) describe how an organization must invest in the development of
its employees’ knowledge and skills in order for the organization to succeed. The authors assert
that this investment in employee development is critical because it improves the ability of the
employees to solve problems that are related to their own performance and to the organization’s
mission. When employee performance is enhanced, employees are better equipped to achieve
organizational goals and close gaps in organizational performance.
In order for SUM to help SMD faculty close performance gaps related to effective
utilization of the LMS, an assessment of the knowledge influences and types that affect faculty
ability to utilize the LMS is useful in guiding further recommendations and analysis
In the next section, three knowledge influences of the SMD faculty are discussed and
then categorized according to one of the four knowledge types. This categorization assisted with
determining the best methods or recommendations for closing the identified knowledge gaps
after stakeholder knowledge was fully assessed. An assessment recommendation for each
knowledge influence is also outlined.
Knowledge influences. While a comprehensive examination of SMD faculty’s online
teaching abilities would ideally examine every aspect of the TPACK framework discussed in the
literature review (Herring et al., 2016; Koehler & Mishra, 2009; Mishra & Koehler, 2006), this
study was primarily concerned with examining the faculty’s ability to utilize the learning
management system. Therefore, instead of examining every intersection of TPACK described in
Figure 1, this study evaluated faculty knowledge of the three TPACK intersections that are
related to the use of technology; technological knowledge (TK), technological content
knowledge (TCK), and technological pedagogical knowledge (TPK).
PREPARING FACULTY FOR DISTANCE EDUCATION 23
Understanding of how to use basic LMS features. Koehler and Mishra (2009) define
TK as the ability to “understand information technology broadly enough to apply it productively
at work and in their everyday lives, to recognize when information technology can assist or
impede the achievement of a goal, and to continually adapt to changes in information
technology” (p. 64). The authors explain that TK is related to the technologies that faculty use
on a day-to-day basis in their practice. In accordance with the TPACK model, the researcher
identified three key tools within the LMS that will be essential for SMD faculty to become
familiar with in order to prepare for teaching online classes. These tools are Kaltura, a video
content management system; Moodle discussion forums, an LMS tool for facilitating student
interaction; and the Moodle gradebook, an LMS tool for providing students with summative
feedback. The skills required to utilize the basic functions of each of these tools can be classified
as both factual and procedural knowledge.
Kaltura allows faculty members to record instructional videos from essentially any
computer and upload these videos to the LMS. The platform is designed to allow faculty
members to create and share content with minimal assistance from technical support staff
(Kaltura, 2018). The platform also enables faculty members to monitor student activity by
tracking which videos have been viewed by specific students. Kaltura also allows faculty
members to insert either graded or ungraded quizzes into videos in order to make them more
interactive.
Discussion forums in Moodle can be used to provide students with an opportunity to
actively engage with the course content by interacting with their peers and instructors. The
frequency and quality of student interaction is a key predictor of student success in distance
education programs (Bolliger & Martin, 2018; Gray & DiLoreto, 2016; Jaggars & Xu, 2016;
PREPARING FACULTY FOR DISTANCE EDUCATION 24
Trolian, Jach, Hanson & Pascarella, 2016). Discussion forums are a commonly-used tool in
distance education teaching (Manning & Smith, 2018).
The Moodle gradebook (referred to hereafter simply as the gradebook) is important for
providing students with prompt feedback on their assignments and their overall course progress.
It is important for students to have access to feedback on their performance throughout their
courses as this feedback is key for student self-regulation and a predictor of their academic
success (Bandura, 2005; Denler, Wolters, & Benzon, 2006; Oakes, Lane, Menzies & Buckman,
2018). Use of the gradebook also ensures that students receive performance feedback as
promptly as possible, because Moodle will automatically notify them when they have received a
grade on an assignment; providing prompt feedback is important for students as it is more likely
to influence their behavior than delayed feedback (Daly, 2006; Lantz & Stawiski, 2014).
Understanding of how to adapt content to LMS. Koehler and Mishra (2009) define
TCK as the understanding of “which specific technologies are best suited for addressing subject-
matter learning in their domains and how the content dictates or perhaps even changes the
technology” (p. 65). Faculty members with high levels of TCK should be able to identify which
technology solutions are most appropriate for teaching specific kinds of content and then adapt
accordingly. A faculty member might, for example, recognize that full-motion video would be a
better solution for demonstrating a medical procedure than a series of still images (Dong & Goh,
2015; Kelly, Lyng, McGrath & Cannon, 2009), or that an interactive computer simulation would
be a better solution for demonstrating the anatomical function of an organ than a simple written
description (Brenton et al., 2007; Wilson et al., 2018). TCK is a key skill for faculty members
adapting their content for distance education because they will face more technology constraints
than they face in a conventional classroom. TCK can be defined as conceptual knowledge
PREPARING FACULTY FOR DISTANCE EDUCATION 25
because it requires faculty members to have a strong understanding of the content within their
field of expertise and the teaching technologies available to them, as well as an understanding of
how the content can interact with the technology and vice-versa.
Understanding of how to apply pedagogical strategies using technology. Koehler and
Mishra (2009) define TPK as “an understanding of how teaching and learning can change when
particular technologies are used in particular ways” (p. 65). Faculty members with high levels
of TPK are able to identify ways in which technology can be used to enhance student learning.
For example, a faculty member with high TPK might recognize the value of the previously-
discussed in-video quizzing feature of Kaltura, because it can be used to give students
customized feedback when they provide the wrong answer to a question. There is evidence that
feedback with an explanation is more effective at promoting student learning than feedback
simply on the correctness of the answer (Van der Kleij, Feskens, & Eggen, 2015). Like TCK,
TPK can be defined as conceptual knowledge because it requires an understanding of
pedagogical principles in addition to technology skills, and the ability to adapt different
principles to different technology platforms as needed.
Table 2
Summary of Knowledge Influences
Assumed Knowledge Influence Knowledge Influence Assessment
Procedural: SMD faculty need to be able to
use basic software functions of LMS tools
including Kaltura, discussion forums, and the
Gradebook (technological knowledge).
Use interviews to ask faculty about their
experience with these tools; use artifact
analysis to examine the current use of these
tools in the LMS.
Conceptual: SMD faculty need to be able to
conceptualize ways of sharing content
knowledge with students using the tools
available in the LMS (technological content
knowledge).
Use interviews to ask faculty about their
ideas for creating content for distance
education students using the LMS.
PREPARING FACULTY FOR DISTANCE EDUCATION 26
Conceptual: SMD faculty need to be able to
conceptualize ways of using the tools
available in the LMS to improve student
learning and employ pedagogical strategies
(technological pedagogical knowledge).
Use interviews to ask faculty about their
ideas for engaging students and applying
different pedagogical strategies using the
LMS.
Motivation Influences
In addition to the technological knowledge of how to perform various tasks using the
tools provided within the LMS, the SMD faculty will also require motivation in order to achieve
the organizational goal. According to Clark and Estes (2008), an individual’s motivation is
based on their beliefs about themselves and their coworkers. The authors suggest that there are
three process areas required for an individual to be motivated. First is active choice, when an
individual chooses to work toward a goal. Second is persistence, when an individual continues
choosing to work toward a goal in the face of obstacles or distractions. Third is mental effort,
when an individual engages in the work with enough cognitive effort in order to succeed.
Assessing motivational challenges in any of these process areas can help improve employee
performance.
The SMD faculty will be more likely to achieve their stakeholder goal if any motivational
challenges are assessed and resolved. While there are many motivational influences that attempt
to explain motivation, there are two key motivational influences that were analyzed with respect
to their influence on faculty members: self-efficacy and value. These motivational influences are
derived from self-efficacy theory and expectancy value theory. The application of these
motivational theories is discussed in the following section and an assessment strategy for each
motivational influence is recommended.
Self-efficacy theory. Self-efficacy is defined as an individual’s belief or confidence in
their ability to perform a task (Pajares, 2006; Rueda, 2011; Schunk & DiBenedetto, 2016). Self-
PREPARING FACULTY FOR DISTANCE EDUCATION 27
efficacy is influenced by a variety of factors, including previous failures or successes, prior
knowledge about how to perform a task, and positive or negative feedback from other people
regarding previous performance on the task. When self-efficacy is high, an individual will be
more motivated to actively choose and persist in performing a task. Therefore, SMD faculty will
be more likely to engage with the task of becoming proficient at utilizing the LMS if they have a
high level of technological self-efficacy (Abbitt, 2011; Dunbar & Melton, 2018; Robinia &
Anderson, 2010; Schunk & DiBenedetto, 2016).
Expectancy value theory. Value is defined as the perceived importance of completing a
task (Eccles, 2006; Wentzel & Miele, 2016). When an individual believes that a task is
important, he or she is more likely to be motivated to do the work necessary for completing the
task. The other component of expectancy value theory is expectation; in addition to believing
that the task is important or valuable, it is also necessary for an individual to expect that they can
succeed at the task. Eccles (2006) explains that expectancy of success at performing a task is
related to an individual’s self-efficacy for that task.
There are four key value constructs of expectancy value theory (Eccles, 2006; Mayer,
2011; Rueda, 2011; Wentzel & Miele, 2016). These four constructs are attainment value, utility
value, intrinsic value, and perceived cost. All four of these constructs must be considered when
assessing an individual’s motivation to complete a given task, although a specific dimension may
be a primary focus in an assessment if it is identified as a key factor which contributes to an
employees’ unwillingness to perform a task. Eccles (2006) defines attainment value as the
connection between an individual’s self-image and the given task. If a task is related to a
person’s self-image, they are more likely to place value on the task. For example, if SMD
faculty perceive themselves as technologically savvy, or as effective online instructors, they are
PREPARING FACULTY FOR DISTANCE EDUCATION 28
more likely to place a high attainment value on developing proficiency with the various features
of the LMS.
Utility value is related to how an individual believes that completion of a task is related to
their goals (Eccles, 2006; Wentzel & Miele, 2016). If an individual believes that completing a
task is important for achieving their goals, he or she is more likely to engage with that task. If
faculty members believe that learning how to effectively utilize the LMS is important for their
goal of achieving competency with the software tools provided in the LMS, they will be more
likely to invest mental effort in learning how to utilize those tools.
Intrinsic value is related to the enjoyment that an individual derives from completing a
task (Eccles, 2006; Wentzel & Miele, 2016). If an individual enjoys working on a particular
task, he or she is more likely to engage with that task. Therefore, if faculty members derive
enjoyment from utilizing the LMS, they will be more motivated to work on improving their LMS
skillset.
Finally, perceived cost is related to the loss of time or energy spent on a task that could
have been dedicated to other tasks (Eccles, 2006; Wentzel & Miele, 2016). If faculty members
perceive that learning how to use the LMS will require a low cost of time and energy, they will
be more likely to engage with the task of learning how to utilize the LMS features. It is therefore
important that the time and energy required by faculty in order to learn about the LMS is
minimized whenever possible.
PREPARING FACULTY FOR DISTANCE EDUCATION 29
Table 3
Summary of Motivation Influences
Assumed Motivation Influence Motivation Influence Assessment
Self-Efficacy: SMD faculty need to believe
they are capable of effectively learning how
to utilize the LMS.
Use interviews to ask faculty about their self-
confidence with utilizing the LMS tools.
Expectancy-Value: SMD faculty need to
believe that the LMS can provide them with
tools that will improve the quality of their
courses and that there is value in providing
distance education courses in the SMD.
Use interviews to ask faculty about whether
or not they believe the LMS tools are useful
for distance education and whether or not
they believe that creating distance education
courses is a worthwhile pursuit.
Organizational Influences
General theory. The knowledge and motivation influences discussed previously may be
used to explain the causes of performance gaps with individual faculty members. However, it is
also important to examine the organizational context in which faculty members participate in.
Organizations such as SUM are complex, with policies and cultures that may influence
performance. Clark and Estes (2008) describe the importance of examining any organizational
barriers that may impact stakeholder performance, because such an examination can help
organizational leaders understand the factors that impede or facilitate stakeholder performance.
Schein (2004) describes culture as the shared and often unspoken learning experiences that cause
members of an organization to hold similar assumptions. Therefore, organizational climate is the
combined routines, practices, procedures, and policies that stakeholders are expected to follow
based on the shared assumptions within the organizational culture (Gallimore & Goldenberg,
2001; Schneider, Brief, & Guzzo, 1996; Schneider, González-Romá, Ostroff & West, 2017). An
understanding of these shared assumptions can help determine whether organizational culture is
promoting or impeding success. Climate and culture are connected because the beliefs and
PREPARING FACULTY FOR DISTANCE EDUCATION 30
values that form the organizational culture in turn influence the procedures and policies that form
the organizational climate (Schneider et al., 1996; Schneider et al., 2017).
SMD faculty specific factors. Research shows that organizational culture can have a
strong influence on faculty performance (Anra & Yamin, 2017; Velcoff & Ferrari, 2006).
Additional research has also shown that culture is especially important to consider for faculty
teaching in distance education programs, because of the unique challenges related to teaching in
an environment that heavily utilizes technology with which some faculty may be unfamiliar
(Blakely, 2017; Georgina & Hosford, 2009; Marek, 2009). These cultural influences that affect
faculty performance manifest themselves through the development of organizational culture and
the processes that determine daily activity (Gallimore & Goldenberg, 2001; Schneider et al.,
2017).
The organization needs to provide faculty members with training opportunities. The
availability of training resources and opportunities is an important organizational influence on
stakeholder performance (Clark & Estes, 2008). If faculty members have knowledge gaps with
regard to their knowledge of how to utilize the LMS, it will be difficult even for highly-
motivated faculty members to resolve these gaps if they are not given adequate access to training
resources.
The organization needs to have a system of monitoring faculty performance.
Performance monitoring is an essential component of any organizational strategic plan (Clark &
Estes, 2008). Without a method for monitoring faculty performance, organizational leaders will
not be able to determine whether or not performance gaps are present, nor will they be able to
determine whether or not performance is improving over time (Alexander, 2000; Collan,
Stoklasa & Talasova, 2014; Harris, Ingle & Rutledge, 2014). If faculty members perceive that
PREPARING FACULTY FOR DISTANCE EDUCATION 31
their performance is not being monitored, many faculty members will not perform at their
highest level (McInnis, 2002; Miller & Seldin, 2014). It is essential that any performance
monitoring system has clearly defined and measurable benchmarks, so that organizational
stakeholders and leaders alike are able to objectively determine whether or not performance
standards are being met (Clark & Estes, 2008). It is also important for the organization to
consider a performance monitoring system that measures faculty performance beyond simply
measuring student grades and student feedback, as these measurements may not be a true
reflection of faculty performance. Measuring faculty performance using simplistic metrics such
as these may encourage faculty members to engage in grade inflation (Craig, Amernic, &
Tourish, 2014; Langbein, 2008; Rutherford & Rabovsky, 2014). A more robust performance
monitoring system would evaluate the frequency of faculty interaction with students, the extent
to which the faculty’s instructional methods promote active learning, and the course alignment
between learning outcomes, assessments, and activities (Piña & Bohn, 2014). The Quality
Matters Rubric for evaluating the quality of online courses is one such system that could be
incorporated in a performance evaluation model for distance education faculty (Legon, 2015;
Varonis, 2014).
Table 4
Summary of Organization Influences
Assumed Organization Influence Organization Influence Assessment
Cultural setting influence 1: SMD needs to
have a system of monitoring faculty
performance in order to determine their
ongoing progress with learning how to utilize
the LMS
Use interviews to ask faculty about how their
performance is currently monitored.
Cultural setting influence 2: SMD needs to
provide faculty members with adequate
training opportunities for learning how to
utilize the LMS
Use interviews to ask faculty about the
training opportunities that they have been
provided with.
PREPARING FACULTY FOR DISTANCE EDUCATION 32
Interactive Conceptual Framework
The conceptual framework that guided this research study was the gap analysis
framework as described by Clark and Estes (2008), which can be used to identify and analyze the
causes of any organizational performance gaps. A conceptual framework is a model that is used
to plan and make meaning of a study (Maxwell, 2013). A conceptual framework is used to
inform the research questions, data collection, and data analysis techniques that are used in a
research study (Merriam & Tisdell, 2016).
While each of the individual influences on stakeholder knowledge, stakeholder
motivation, and organizational culture have been presented independently of each other, these
three categories of influences are interdependent (Clark & Estes, 2008). Each category of
influence is capable of improving or diminishing either of the other categories. Figure 2
provides a graphical representation of these interdependent relationships.
PREPARING FACULTY FOR DISTANCE EDUCATION 33
Figure 2. Graphical representation of conceptual framework. This diagram provides an outline
of the framework for this research. SMD faculty are influenced by the organizational culture
while also contributing to the formation of the culture. The faculty knowledge and motivation
influences interact with the organizational culture and all of these elements contribute to the
outcome of the stakeholder goal.
Figure 2 is intended to show how knowledge, motivation, and organizational influences
are closely related. Knowledge and motivation influences on stakeholders are represented within
the same circle because they are closely related to each other; when an organizational
stakeholder has strong knowledge of a subject, that stakeholder is more likely to be motivated to
perform work related to that subject, and vice versa, because the stakeholder will have higher
self-efficacy (Bandura, 1993) and an expectation of success in the endeavor (Schunk &
PREPARING FACULTY FOR DISTANCE EDUCATION 34
DiBenedetto, 2016; Wigfield & Eccles, 2000). Similarly, if stakeholders value a particular task,
they are more likely to improve their knowledge of how to perform that task, and conversely a
high degree of knowledge about how to perform a task may lead to a deeper understanding of the
task’s importance, thereby increasing the perceived value of the task (Schunk & DiBenedetto,
2016; Wigfield & Eccles, 2000). This relationship between self-efficacy or expectancy and
motivation has been supported by research on faculty members in higher education settings,
which has demonstrated that faculty members with more experience utilizing different forms of
technology are in turn more likely to incorporate that technology into their teaching practices
(Abbitt, 2011; Buchanan, Sainter, & Saunders, 2013; Dunbar & Melton, 2018; Georgina &
Olson, 2008; Robinia & Anderson, 2010).
The stakeholder knowledge and motivation influences are represented within a circle that
is fully encompassed by the circle that represents organizational culture and climate. These
circles are nested because within any organization, stakeholders are both influenced by the
organizational culture while also simultaneously having influence on the culture, because they
form a part of the larger organization itself (Gallimore & Goldenberg, 2011; Schein, 2004;
Schneider et al., 1996; Schneider et al., 2017). This dual relationship wherein faculty members
are both influenced by culture while also influencing culture has been noted by many academic
scholars studying this interaction within a higher education context (Johnson, 2015; Kezar &
Eckel, 2002; Tierney, 1997).
Qualitative Data Collection and Instrumentation
This research utilized a qualitative approach to data collection and instrumentation. The
data for this was collected through interviews and artifact analysis. Merriam and Tisdell (2016)
note that interviews are the most common method of data collection in qualitative research. The
PREPARING FACULTY FOR DISTANCE EDUCATION 35
authors also describe how interviews are useful for understanding the underlying cause of a
phenomenon. Interviews were identified as more useful than surveys in this research, because
the format of an interview allows the researcher to explore issues in greater depth than a survey
would permit. The small size of the target population permitted a census sample. In this
research, interviews were useful not only for measuring the self-efficacy of faculty members, but
also for understanding what underlying explanations or causes are influencing their self-efficacy.
The interviews conducted were primarily intended to collect information on the motivational and
organizational influences affecting the faculty members, although self-reported knowledge
influences was explored in the interviews as well.
Artifact analysis was utilized as the primary method of data collection for assessing
knowledge influences on the faculty members. Specifically, the faculty members’ courses in the
learning management system (LMS) were examined in order to look for evidence that faculty
members know how to utilize key LMS features; this is possible because the LMS that faculty
members currently utilize for their on-campus courses is the same LMS that will be utilized for
any future distance education courses. Merriam and Tisdell (2016) describe how artifact analysis
is less prone to desirability bias than interviews for assessing knowledge, which is a key reason
why artifact analysis was utilized in this study. The artifact analysis in this research also served
to triangulate data on self-reported faculty member knowledge that is collected in the interviews.
Interviews
The interviews conducted with faculty members employed a semi-structured design.
This design was ideal for this research because it allowed the researcher to follow an outline of
basic questions while still allowing for some deviation from the outline in order to ask clarifying
PREPARING FACULTY FOR DISTANCE EDUCATION 36
questions and to explore unexpected avenues of inquiry that were brought up by the faculty
members during the course of the interviews (Maxwell, 2013).
Interviews were conducted with faculty members prior to artifact analysis. This decision
was made due to the possibility that evidence found during artifact analysis might bias the
researcher during the interview. For example, if artifact analysis suggests that a faculty member
has never utilized a particular feature of the LMS before, but the faculty member describes
themselves as being highly experienced with utilizing that feature in the interview, it may cause
the researcher to view the faculty member as less credible in the interview and thereby affect the
researcher’s affect or demeanor within the interview. Instead, the artifact analysis was
conducted afterward, so that any misalignment of evidence discovered was only discovered by
the researcher after the interviews are fully concluded.
A single interview was conducted with each participant. Each interview required
approximately forty-five minutes to complete. The semi-structured interviews were conducted
formally in a private conference room or in another private location of the participant’s choosing.
Maxwell (2013) describes the importance of an interview setting which protects participant
anonymity and is also free from visual or auditory distractions. Audio recordings of each
interview were produced with a digital audio recorder. The audio recordings were then utilized
to produce detailed transcripts after each interview was concluded. In order to protect the
participants’ anonymity, the audio recordings were deleted once the transcripts were finalized.
Artifact Analysis
Artifact analysis in this research consisted of examination of the faculty members’
courses in the LMS that have been developed for their current classes in existing residential
programs. The researcher observed a copy of each course in the Moodle environment. The
PREPARING FACULTY FOR DISTANCE EDUCATION 37
recorded observations did not include any student data such as assignment submissions or
student grades, thereby preventing any potential FERPA violations related to the mishandling of
student information. The observed courses were then examined for evidence that the faculty
members had utilized key LMS features, such as videos produced with the integrated Kaltura
software, or gradebook entries for providing student feedback.
This analysis is especially useful for generating evidence regarding the knowledge
influences on faculty members; while asking the faculty members knowledge questions during
the interview process would present a variety of issues that were previously discussed, artifact
analysis can provide an opportunity to directly observe faculty behavior, eliminating many
potential forms of bias in the data (Maxwell, 2013; Merriam & Tisdell, 2016). The researcher
currently acts as the primary administrator of the LMS at the institution, which means that the
researcher already had access to this information without seeking any special permission or
approval. The researcher received permission from the institutional review board to access it for
the purposes of this study.
Findings and Results
The findings of the study were based on interviews with full-time faculty members from
the Specialized Medicine Department (SMD) and artifact analysis of the course pages developed
by the faculty members within the LMS. The SMD currently employs a total of 10 full-time
faculty members. A census sample was utilized in which all 10 of the faculty members were
invited to participate in the interviews. Five of the faculty members participated in an interview,
the remaining five members either declined the invitation or did not respond to the recruitment
inquiry. Artifact analysis of the course Moodle pages was conducted for the courses created by
all 10 faculty members from the four most recent academic terms with 96 courses reviewed. In
PREPARING FACULTY FOR DISTANCE EDUCATION 38
order to protect the confidentiality of the participants, no demographic information is included in
the findings. All of the participants were full-time faculty members within the department.
Further information on stakeholder and sampling criteria is included in Appendix A.
While demographic information on the participants would typically be provided in
research of this nature, the small size of the target population would put the anonymity of the
participants at risk if additional demographic information was provided. Additionally, no
pseudonyms have been used to identify the participants, due to the risk that a participant might
become identifiable to certain individuals within SUM if multiple interview quotes were
connected with a specific individual through the use of a pseudonym.
Through semi-structured interviews, the faculty members shared their experience and
knowledge on using the LMS and their opinions on the value of distance education in general.
Artifact analysis of the faculty’s activity in the LMS contributed to determining their knowledge
of how to perform different tasks in the LMS and helped to support the findings from the
interviews.
Identifying performance gaps through the analysis of knowledge, motivation, and
organizational factors provides information on the type of support that a stakeholder group may
need to better achieve their goals (Clark & Estes, 2008). The SMD faculty members need to
have the appropriate influences in order to become effective distance education instructors and
assist the SMD with its goal of providing online courses in the future; if the faculty members are
not effective at teaching in a distance education environment then the SMD may not be able to
provide online courses that are effective or successful. The next three sections will address each
research question by evaluating the knowledge, motivation and organization (KMO) influences
identified through the interviews and artifact analysis.
PREPARING FACULTY FOR DISTANCE EDUCATION 39
Procedural and Conceptual Knowledge Findings
Procedural knowledge is defined as the knowledge of how to perform a task and the
criteria needed for completing the task (Krathwohl, 2002; Rueda, 2011). Utilizing the LMS in
order to create distance education course content requires the performance of many procedural
tasks, such as creating assignment activities or delivering feedback to students. This study
utilized both interviews and artifact analysis to examine the faculty’s procedural knowledge of
how to use basic features within the LMS, such as recording and publishing a video or creating a
discussion forum. The study also examined the faculty’s conceptual knowledge of how these
features might be used to create online course content and support pedagogical strategies
designed to enhance student learning. Conceptual knowledge is defined as the understanding of
principles or theories that are related to a topic (Krathwohl, 2002; Rueda, 2011).
Most faculty members in the department only have experience with performing
basic tasks in the LMS such as uploading files. Both interviews and artifact analysis
corroborated this finding. Artifact analysis showed that in 81% of the courses, the LMS was
only used for posting the course syllabus and occasionally used for posting other files. Analysis
also found that 99% of the courses did not utilize the LMS for assignment submissions, nor did
they provide students with feedback on their grades or performance using the gradebook, or use
any advanced features such as discussion forums or Kaltura videos. These findings are
summarized in figure 3.
PREPARING FACULTY FOR DISTANCE EDUCATION 40
Figure 3. LMS features used by SMD faculty in Moodle courses.
The LMS was most frequently used for posting the course syllabus, with 94% of the
examined courses having an uploaded syllabus. However, examination of the course activity
logs revealed that a staff member providing administrative support for the department posted the
syllabus in each course. This task is likely performed by the administrative staff member due to
an institutional policy which requires that course syllabi are posted to the LMS. The next most
frequent use of the LMS, posting other files for students to download, was utilized in only 64%
of the observed courses.
In the conceptual framework for this study, three specific procedural tasks were identified
as especially relevant to the delivery of distance education, including the use of discussion
0%
0%
0%
1%
3%
3%
12%
19%
64%
94%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Quizzes
In-video quizzes
Announcements forum
Assignment submissions
Gradebook
Kaltura videos
Discussion forums
External links
Handouts
Syllabus
Artifact Analysis: Frequency of LMS Features Used in Courses
PREPARING FACULTY FOR DISTANCE EDUCATION 41
forums to facilitate student interaction, the use of the LMS gradebook for providing feedback to
students, and the use of Kaltura to create and publish instructional videos. The artifact analysis
found that only 12% of the courses used discussion forums, only 3% of the courses provided
students with feedback in the gradebook, and only 3% of the courses utilized Kaltura for
publishing videos.
These findings from the artifact analysis are consistent with the interview findings.
Interview participants were asked in separate questions if they had utilized each of these three
LMS features in their courses. Two out of five participants indicated that they had used
discussion forums; one participant described using discussion forums “to allow students to
submit their case reports so that other students can download them and look at them.” The other
participant who reported using discussion forums described using them “for students to post their
group assignments.” Participants who had not utilized the discussion forums in the LMS self-
reported that there was little need for such a feature in their existing classes, because the face-to-
face nature of their existing classes means that discussion takes place naturally in the classroom.
One participant remarked that “I don’t have any classes right now where I could see integrating
them, but if I did an online class I would definitely want to use them.” Another participant said
that “I haven’t felt the need to add discussion forums because we already have weekly
discussions in class.” These interview comments corroborated the finding in artifact analysis
that only 12% of the courses utilized discussion forums. It is likely based on these findings that
the majority of the faculty members do not have experience using discussion forums in Moodle.
Only one of the participants indicated using Kaltura videos. This participant stated that
“I’ve done a lot of recording. Hours and hours of Kaltura videos now ... I think it’s a great way
to have a flipped classroom.” This participant was enthusiastic about the use of Kaltura, stating
PREPARING FACULTY FOR DISTANCE EDUCATION 42
that “I like it better than what I did before we had Kaltura ... Kaltura is much quicker for the
upload than what I did before.” However, the other four participants indicated that they had
never used Kaltura. One participant seemed indifferent, stating that “Kaltura seems interesting, I
just haven’t found a need for it.” Another participant expressed some very strong reservations
about Kaltura and described some concerns about the intellectual property implications of
uploading recorded lectures to the LMS. This participant expressed some concern that recorded
videos of lectures might be used to replace the faculty members, explaining that “I have really
big hesitations about this ... nobody can reassure us that the material is not going to be used to
replace us and our teaching.” The participant went on to say that “I just don’t trust it” and
indicated that there are “a number of faculty” who feel the same way.
Only one of the participants indicated using the gradebook for providing feedback to
students. This participant stated that “I just started using the gradebook this term and it works
great for one of my classes, but I think I set up another one wrong, it’s not tallying right.” The
other four participants said that they had never used the gradebook before. One faculty member
said “honestly I haven’t even looked at the gradebook” and then described the other ways in
which feedback is provided in his or her courses, stating that “there are several ways of doing
feedback, one of which could be e-mail, or another could be direct feedback in person; another
could be writing comments on some kind of assignment and handing it back.” The researcher
clarified that this statement about “writing comments” and “handing it back” was in reference to
assignments which are printed and submitted by students on paper. The participant went on to
say that he or she only recently began recording grades electronically, remarking that “I was
really proud of myself this term, I got an excel spreadsheet that did weighted grades and I was
able to go and do that. I used to do it by hand, and I could do it, but I mean it took time.” Other
PREPARING FACULTY FOR DISTANCE EDUCATION 43
faculty members indicated that they did not see the importance of utilizing the gradebook,
providing explanations such as “our department does not require us to use it” or that “I’ve been
recording my grades on paper for years.” Overall, both the artifact analysis and the interviews
indicated that faculty members are only familiar with performing very basic tasks in the LMS,
such as uploading files, and that faculty members have limited experience with utilizing more
advanced features such as discussion forums, gradebooks, or Kaltura videos. These findings
suggest that there is a knowledge gap in the faculty’s ability to carry out procedures in the LMS
that are essential to teaching an online course.
Faculty members are able to describe how they would share instructional content in
an online course in varying levels of detail. In order to assess the faculty’s conceptual
knowledge of how they might create and share instructional content with students in a distance
education course, faculty members were asked to describe how they might hypothetically
convert one of their existing classes into an online course. These responses were then sorted into
three different categories based on the level of detail provided in the description: highly detailed,
moderately detailed, and minimally detailed.
When asked to describe how the features in the LMS might be used to create an online
class, one participant gave a very detailed description:
Well, I think I would take all those clinical medicine videos and I'd use the video with
some quizzes within the video and then I'd use a discussion forum to discuss the video
and different ways of approaching it ... So I'd have them going through cases and then
having discussion forums based off the case, trying to simulate different ways of
approaching that patient and thinking about it. I'd also want some kind of like once a
PREPARING FACULTY FOR DISTANCE EDUCATION 44
week type thing where they actually have to log in and they all log on together and
there’s some kind of discussion that is facilitated by me.
This response was categorized as “highly detailed” because it incorporates multiple
methods of content delivery and multiple forms of assessment. There are three methods of
content delivery, including videos, written cases, and live video conferences. There are three
methods of assessment, including quizzes embedded in videos, asynchronous discussion forums,
and synchronous class discussions facilitated by live video conferences.
Another interview participant gave a detailed response when describing how certain
courses would be difficult to teach in an online format because they are focused on how to teach
hands-on medical procedures, explaining that “you would have to have video close-ups, and you
would have to have a great awareness of the common mistakes.” This participant then went on
to describe this difficulty in more detail: “for instance, many techniques, you have to have a
completely relaxed hand to do it ... so, you would have to be using the visuals, like the Kaltura
stuff, and your language, to make sure people are getting it.” The participant then reiterated the
importance of using well designed videos to teach these hands-on techniques, remarking that
“you would have to have really good work with the videos, you would have to make up for them
not being there by continually going back to the rudiments.” Finally, to summarize, the
participant said that “if they can get these rudiments and practice them for six months at home
before they show up for an on-site intensive where you can observe them, then you can get
somewhere with them.” This participant’s description included two forms of content delivery
and one form of assessment. This response was categorized as “moderately descriptive.”
The remaining three responses were categorized as “minimally detailed.” One participant
expressed a preference for live video conferencing: “I really liked the program that was used by
PREPARING FACULTY FOR DISTANCE EDUCATION 45
another institution ... they can raise a hand and actually use a mic to actually ask a question.”
This response described only one method of content delivery, live video conferencing, and did
not describe any forms of assessment. Another participant said, “I would record my lectures and
post discussion forums about them,” a description that included one form of content delivery and
one form of assessment. Finally, the last participant gave a similar response: “I would use videos
and discussions.” When asked if they could provide more detail, the participants with responses
in this category were unsure about what else to say, and offered explanations such as “I’m not
sure, I would want to see what other people are doing first” or “I would need to learn more about
what is possible to do in Moodle.”
These findings suggest that there is a knowledge gap in the faculty’s ability to
conceptualize an online course because three out of five faculty members provided only minimal
detail when describing their hypothetical courses. This finding is partially corroborated by the
artifact analysis, which demonstrated that 88% of the faculty’s courses did not utilize any
interactive components such as discussion forums or quizzes, although this corroboration is
somewhat limited by the fact that the observed courses were constructed for a conventional
classroom setting and do not necessarily represent how the faculty members might utilize the
LMS for an online course.
Most faculty members can describe pedagogical rationales for utilizing technology
in online courses. In order to assess the faculty’s understanding of how technology can be used
to facilitate student learning, faculty members were asked to describe the benefits that students
might receive from the inclusion of key LMS features in an online course, including in-video
quizzes, discussion forums, and gradebook feedback. These responses were analyzed in order to
identify evidence that faculty members were able to consider the pedagogical implications of
PREPARING FACULTY FOR DISTANCE EDUCATION 46
utilizing these tools. Five out of five participants described pedagogical benefits of in-video
quizzes and discussion forums, while three out of five described pedagogical benefits of
gradebook feedback. This finding suggests that there is no gap in this knowledge influence.
When asked how students might benefit from taking a quiz embedded in a video using
Kaltura, all five of the faculty members used informal language to describe how this might
support learning through immediate reinforcement. When describing the possible pedagogical
benefits of using a quiz embedded in a video, one faculty member said “well it's checking for
comprehension ... you know, right then and there, because the thing is attention span only lasts a
certain amount of time.” Another faculty member made a similar comment about embedded
quizzes, stating that “I think it can help them review and think about what they were just
watching, because there is more a tendency online to space out and not give the 100% presence
as when you're doing your seat time.” Yet another faculty member remarked that “it has real
value ... for people trying to learn something at home, even if they're meditators and have good
focus, they're gonna drift away.” All five of the faculty members responding to this question
made similar comments about how quiz questions during a video might promote student
engagement or check for comprehension.
All five faculty members were also able to identify potential benefits associated with
using discussion forums. One participant stated that discussion forums could help students with
“staying engaged to the material through the week,” and went on to state that forums could help
create “a collegial learning environment where different ideas get presented and explored and
they learn from each other as well as from me.” Another participant explained how students in
his classes came from different professional backgrounds, “maybe someone comes into this
program having been a sports massage person, or a chiropractor, and they've been thinking that
PREPARING FACULTY FOR DISTANCE EDUCATION 47
way for years” and then described how discussion forums might help those students from
different backgrounds exchange different ideas from their professions, “to kind of add tools to
their toolkit,” and that “that kind of discussion could be a good tool to promote the reflective
education.” All five of the interview participants identified similar benefits of discussion
forums, describing how these forums could promote an “exchange of ideas” and encourage
“collaborative learning” among students.
There was less consistency in the participants’ responses regarding potential benefits of
using the Moodle gradebook. Three out of five faculty members were able to describe how they
thought the gradebook might help students self-regulate by keeping track of their performance in
the course. One participant said that “the reason why they might want to see their grade in any
one moment is that they can see their current grade going into a midterm or a final ... I think that
would be something very beneficial.” Another participant said that “using the gradebook would
let them see their grades right away.” Finally, the third participant described how using the
gradebook would be more effective in his existing on-campus courses than his current practice of
returning grades on individual assignments in class:
I was just going through like my term papers yesterday when I was grading them and
there's papers from beginning of the term I put out every week that some students never
picked up ... they weren’t getting the feedback. So that's the advantage of having it on
the computer ... it's more accessible than me having the paper and recycling it at the end
of the term.
The other two participants were unsure about whether or not using the Moodle gradebook
would provide any benefit for their students. One participant, speaking about the gradebook,
said “well, never having done it, I'm not sure I have a good answer there.” The other participant
PREPARING FACULTY FOR DISTANCE EDUCATION 48
explained that “they receive their final grades in SONIS, so putting the grades in Moodle seems
redundant.” This last participant did not seem to consider the possible benefit of students
monitoring their grades throughout the course, and only remarked on students receiving their
final grades.
These interview results suggest that there is no gap in this conceptual knowledge
influence. When compared to the findings from artifact analysis, the interview findings for this
knowledge influence are inconsistent with the LMS course observations. The faculty members
are able to describe the pedagogical benefits of these features, but artifact analysis shows that
they are not utilizing the features in their existing courses. However, the fact that faculty
members are not utilizing these tools does not necessarily mean that they are incapable of
articulated the pedagogical benefits of the tools, as demonstrated from the interview findings.
Ultimately, the findings still do not indicate a gap in this conceptual knowledge influence.
Summary of knowledge influence findings. Overall, these findings suggest that faculty
members in the SMD are able to describe how technology can provide pedagogical benefits for
students, as evidenced by the fact that all of the faculty members who participated in interviews
were able to describe ways in which student learning might be enhanced through the use of
features such as discussion forums or videos with embedded quizzes. However, the faculty
members have limited experience actually utilizing these technology features in their classes, as
evidenced by both their self-reported utilization of these features in the interviews and by the
findings from the artifact analysis. Most of the faculty members are also unable to describe how
they might design an online course with more than minimal detail. These findings suggest that
the faculty has a procedural knowledge gap in their ability to use technology features of the
LMS, and a conceptual knowledge gap in their ability to describe or create an online course. The
PREPARING FACULTY FOR DISTANCE EDUCATION 49
findings also suggest that there is not a gap in the faculty’s ability to identify pedagogical
benefits of utilizing interactive technology features in an online course.
Self-Efficacy and Expectancy Value Motivation Influence Findings
Motivational principles such as self-efficacy and expectancy value can be taken into
consideration when identifying the cause(s) of a performance gap (Rueda, 2011). This study
utilized interview responses to assess the SMD faculty’s self-efficacy and expectancy value with
regard to distance education. To assess self-efficacy, interview participants were asked to
describe how confident they felt in their ability to utilize the LMS (Moodle), and how confident
they would feel if they received additional training. To assess expectancy value, participants
were also asked whether or not they thought that providing distance education courses in their
department would be a good idea. To further assess expectancy value, participants were also
asked whether or not they would volunteer to teach online classes if these classes were being
taught in their department.
Faculty members are moderately confident in their ability to utilize the LMS. When
asked to describe how confident they were at using the LMS, four out of five faculty members
indicated that they were at least somewhat confident in their ability. However, all four included
caveats that they were only confident at using the basic features they are familiar with, which
suggests that there may be a motivation gap in the faculty’s self-efficacy for utilizing the LMS.
One participant was self-described as “modestly confident” but “not a techie” and could “do the
simple things I want to do.” Another participant made similar statements, saying that “for what I
need, I feel very confident ... for what I don’t need, I haven’t taken a look at it.” This sentiment
was a common theme; a third participant felt “okay with the basics” and that “for what I do
utilize, I feel confident in,” but also remarked that “it’s kind of like I’m not utilizing it to the
PREPARING FACULTY FOR DISTANCE EDUCATION 50
fullest potential.” Only one participant expressed a lack of confidence, saying that “I would need
some training in how to use it before I felt confident” and that “I really just haven’t used Moodle
for much of anything at all.” These comments suggest that the faculty are confident in their
ability to use the LMS for “simple things” or “the basics,” which indicates that there may be a
self-efficacy gap regarding their confidence in utilizing LMS features that they perceive as more
advanced.
Faculty members were more confident when asked if they would feel confident learning
how to perform new tasks in the LMS if they received additional training. Five out of five
participants indicated a high level of confidence in response to this question. One participant
would be “very confident.” Another participant would be “quite confident” with additional
training. A third participant would be “fairly confident” and does not “find Moodle to be that
difficult.”
Faculty members believe that distance education is valuable for different reasons.
When asked whether or not they believed that it would be a good idea for their department to
offer distance education courses, five out of five participants stated that this would be a good
idea, indicating that there is not a gap in this expectancy-value motivation influence. Three out
of five participants described intrinsic motivations for offering distance education courses. One
participant suggested that having some classes online might provide better work-life balance by
allowing him to teach from home, stating that “it’d be nice not to have to get here all the time ...
the commute is the real issue.” Another participant suggested that distance education might
work better for some of the classes that are taught in the department, remarking that “I think it
would be good to offer them ... that might actually be preferable in some ways.” When asked to
explain why this would be preferable, the participant explained that some courses teach students
PREPARING FACULTY FOR DISTANCE EDUCATION 51
to recognize certain types of characters that are drawn on a chalkboard, and that in an online
course, “you can see the change right in front, and then also the references are right there ...
they’re right there in front of you on the screen.” When asked for clarification, the participant
confirmed that the reason this was preferable is that it can sometimes be hard for students in the
back of a conventional classroom to see what is being drawn on the chalkboard. This participant,
like the first participant, also identified that distance education courses might provide better
work-life balance for faculty and student. The participant suggested a specific class in the
program which would be ideal for an online setting, and then said that “it would be nice if some
people could literally eat breakfast or sip some tea in a more relaxed manner and still not come at
8:00 in the morning on Monday, which is when it's always scheduled.” The fact that three out of
five participants described intrinsic motivations for their belief in the value of offering distance
education courses is positive for the department because the literature suggests that stakeholders
with intrinsic motivation are more likely to perform at a high level when compared with
stakeholders who are extrinsically motivated (Clark & Estes, 2008; Rueda, 2011).
The remaining two participants gave responses which were also in support of distance
education courses being offered, but the reasons provided suggested extrinsic motivation rather
than intrinsic motivation. One participant remarked that “I think it's good, because we're being
forced to,” and then went on to describe a competing institution which offers a one-year doctoral
completion program that is fully online. “Price-wise, none of the on-site programs can compete
with that, and people are just doing that ... and I've heard they do a reasonably good job of it, that
it's not Mickey Mouse.” The participant then described a different competitor which has a
master’s program wherein “a lot of their content is delivered online now.” This participant then
reiterated that distance education courses were important to offer in order to compete with these
PREPARING FACULTY FOR DISTANCE EDUCATION 52
other institutions: “the fact that in the master’s realm, someone is doing it, in the doctoral realm,
someone's doing it, we have to do it. It's just on the level of economic viability.” The other
participant who described extrinsic reasons regarding the value of offering distance education
courses simply stated that “we have to do it because that’s the way education is going these days
... it’s already happening at other schools.” These findings suggest that faculty members see
value in the idea of offering distance education courses in their department and that there is not a
gap in this expectancy-value motivation influence.
Faculty members are willing to teach online courses if the department does not
attempt to launch a fully online degree program. Five out of five participants indicated that
they would be willing to volunteer to teach online courses if they were being offered in the
department, indicating that they value the idea of teaching online and that there is not a
motivation gap in this expectancy-value influence. However, it should be noted that all five
participants said they did not feel that a fully online degree program would be appropriate for the
department at this time; they are only willing to teach in online courses if those courses are
offered in the context of the department’s pre-existing clinical degree programs which are
offered on campus. This reluctance of the faculty to participate in a fully online degree program
in their department is not a motivation gap as currently the department’s goal is to begin offering
online classes and not a fully online program. “We couldn’t offer a fully online degree, because
this is a clinical based medicine program, but switching probably 30 to 40 percent of our classes
to online would be good for the students, good for faculty” one participant remarked. Another
participant expressed a similar thought, stating that “I mean I did my doctoral program online,
and it was ... I really appreciated it. I really liked it. But you wouldn't want to do all of it online.
No, you just couldn't.” When asked to clarify if this was because it was a program which
PREPARING FACULTY FOR DISTANCE EDUCATION 53
required students to have hands-on clinical experiences, the participant said “yeah, we had to be
there. But there were a number of classes online, which is very beneficial. It's very good.”
Another participant volunteered to teach online “because I guess that’s where we’re
going” and that “the sooner we start, the more experience we'll gather, so that we do a good job
of it.” This participant also suggested that the department should start by offering specific
classes online, “where there's no hands-on component whatsoever, those would be the ones to
start with.” These descriptions of hands-on clinical experiences which are essential to the
existing programs reinforced the faculty member’s assertions that it would not be feasible for the
department to offer fully online programs at this time. However, the faculty’s willingness to
potentially volunteer for teaching online courses indicates that there is no motivation gap for this
influence.
Summary of motivation influence findings. The findings suggest that there are some
motivation gaps in faculty self-efficacy and no significant motivation gaps in faculty expectancy-
value. Faculty members are moderately confident in their own ability to utilize the LMS when
using features that they describe as “simple” or “basic,” but expressed that they are less confident
at using advanced features in the LMS. Five out of five faculty members stated that they would
be more confident if they were provided with additional training. All five faculty members
agreed that offering distance education courses would be a good idea; three out of five
participants had intrinsic motivations such as improved work-life balance while the other two
participants had extrinsic motivations such as improved competitiveness with other institutions.
Faculty members indicated that they would be willing to volunteer as instructors for online
courses, provided that these courses were provided in the context of an existing on-campus
PREPARING FACULTY FOR DISTANCE EDUCATION 54
degree program and not part of a fully-online program. Overall, these findings suggest that self-
efficacy is currently the motivation influence with the largest gap.
Training and Performance Evaluation Organizational Influence Findings
This study identified the availability of LMS training and the existence of a performance
evaluation framework for online courses as important factors which might contribute to the
success or failure of any distance education initiatives. Interviews were utilized in order to
assess whether or not the organization had provided LMS training for the faculty members and
whether or not there was a performance evaluation framework for online courses in place. The
interview findings suggested that there are gaps in both of these influences. The previously
discussed artifact analysis of faculty member LMS activity is consistent with the interview
findings as it suggests that faculty members are not utilizing advanced features in the LMS that
are essential for the delivery of distance education and provides supporting evidence of the
cultural setting gaps.
The faculty members have been provided with few opportunities for LMS training.
Four out of five faculty members indicated that they had not received any training when they
were initially hired. This finding indicates that there is a gap in the cultural setting of the
organization which would impede the faculty’s ability to fulfill their stakeholder goal. One
participant said, “I never had any formal training. I learned along the way.” This participant
indicated that a faculty support staff member was available to help when faculty had specific
questions and that the staff member “showed me how to use it as needed.” Another participant
also stated that “there was none,” but was “familiar with it” from using the same platform
(Moodle) at another institution. This participant described the institution’s approach to LMS
training as a “sink or swim” experience.
PREPARING FACULTY FOR DISTANCE EDUCATION 55
Only one faculty member did recall receiving some initial training on Moodle,
approximately ten years ago, when Moodle was first introduced. This participant explained that
“whenever it was that Moodle came on board, they introduced it in our department’s faculty
meetings.” This participant explained that in this training, they showed “how to put assignments
up, how to create links to things you wanted the students to watch, and how to do discussions.”
It does not seem that this training was repeated for faculty members who were hired afterwards
as none of the other participants could recall having received such training. There are no records
or documents cataloging any LMS training offerings prior to January 2019, so it is not possible
to definitively confirm or support this observation.
When asked if the participants had received any additional LMS training since being
hired initially, two out of five indicated that they had. One participant had “attended the Moodle
training in January when the new site was launched.” Another participant talked about “training
at the faculty meeting” where there was a presentation on how to “track attendance in Moodle.”
Both of these participants could only recall a single anecdote where additional training had been
provided; neither indicated that there had been any kind of regularly scheduled training
opportunities offered. Review of the meeting agendas for the January Moodle training and the
faculty meeting both suggest that attendance at the trainings was not required and that faculty
members who attended did so voluntarily.
The remaining three participants did not recall receiving any additional training at all;
one participant remarked that “actually the training has been through students telling me what
they do in other classes.” This participant then described how he or she had learned how to use
new features in Moodle by independently researching online: “I just investigated what I was
prompted by the students to do and I just went online and checked it out on the forums. They
PREPARING FACULTY FOR DISTANCE EDUCATION 56
told me what to do.” The artifact analysis presented in the knowledge influences section, shown
in Figure 3, corroborates these interview findings, as 88% of the department’s courses observed
in the analysis did not utilize any interactive LMS features. These findings indicate that there is
a cultural setting gap in the availability of LMS training for faculty members as well as a gap in
the promotion of existing asynchronous training courses.
There is no performance evaluation system in place for faculty which takes into
account faculty utilization of the LMS. Interview participants were asked to describe how
faculty performance is evaluated in their department and whether or not any evaluation of their
LMS utilization was included in any existing performance evaluation framework. All five
participants said that the existing performance evaluation system does not take into account their
use of the LMS in any way. When asked if the performance evaluation process would evaluate
their use of Moodle, the participant responses were short and definitive. One faculty member
stated, “It never came up, no. Probably, you can certainly make the argument that it should have,
but it didn't.” Another faculty member simply said “no.”
All five participants indicated that in the current performance evaluation system, the
primary method of evaluation is the analysis of student responses on course evaluation surveys.
One participant described the process like this:
Once a year I have a half hour kind of, evaluation with the dean for academics, and they
pull together my course feedback from the students, they have me do a self-evaluation,
and then we just meet and talk about it and then also come up with goals of what I want
to work on into the future.
Another participant provided a similar description of the process and seemed to view it as
a positive experience:
PREPARING FACULTY FOR DISTANCE EDUCATION 57
Well, of course, they go through the student evaluations from all your classes and they,
highlight some points that students are making ... you sit down with the dean and the
associate dean, and they go through all the documents with you, and they try to put
together a plan with you to address any deficiency in the coming year, it's really a
friendly atmosphere in the evaluations here, the way we do it, and it's a learning
experience.
The performance evaluation system is partially applicable to distance education as
student course evaluations may still be utilized in online courses to evaluate faculty performance.
However, there is still a gap in the cultural setting because evaluating the quality of course
materials in the LMS is essential to the evaluation of distance education performance and the
current system lacks any standards or metrics for assessing these materials.
Summary of organizational influence findings. Gaps are present in the cultural setting
for both LMS training opportunities and performance evaluation methods. LMS training
opportunities have been limited; interview findings suggest that there was an initial training
when the LMS was first introduced approximately ten years prior, and a training when the LMS
was migrated to a new hosting company in January 2019. There have also been occasional
presentations on specific features at some faculty meetings. Most faculty members could only
recall having attended one training, and LMS training is not given to new faculty members when
they are hired. Asynchronous LMS training courses exist in Moodle, but these courses are not
frequently accessed by faculty members and they have not been adequately promoted to new
faculty members during the onboarding process.
Faculty performance evaluation is primarily based on feedback from students given via
course evaluation surveys. Faculty members also perform a self-evaluation and meet with the
PREPARING FACULTY FOR DISTANCE EDUCATION 58
dean to set goals for professional development. Faculty members are also observed in the
classroom and notes from these observations are also included in their performance evaluation.
All of the faculty members confirmed that their performance is in no way evaluated based on
their use of the LMS. The only required use of the LMS in the department is ensuring that the
course syllabus is posted prior to the beginning of the course, and this one task is handled by a
staff member and not the faculty themselves. None of the interview participants could recall this
being discussed in their performance evaluations.
Summary of Results and Findings
Overall, two out of three knowledge influence gaps were validated, with indications that
faculty members in the SMD have gaps in their procedural skills for using LMS technology tools
and their conceptual knowledge of how to construct lesson plans for online courses. One out of
two motivation influence gaps were validated, showing that faculty members self-report low
self-efficacy for utilizing advanced LMS features and that advanced LMS features are not
utilized in their existing LMS courses. Finally, two out of two organizational influence gaps
were validated, demonstrating that faculty members have limited opportunities for LMS training
and that they have not engaged with the few training resources that are available. It was also
demonstrated that the evaluation system for faculty performance does not evaluate their use of
the LMS in any way, which is a key limitation that would prevent the evaluation system from
providing an adequate overview of their performance in a distance education environment.
These findings are summarized in Table 5.
PREPARING FACULTY FOR DISTANCE EDUCATION 59
Table 5
Summary of Results and Findings
Knowledge, Motivation, and Organizational Influences Type Validated
as a Gap?
SMD faculty need to be able to use basic software functions of
LMS tools including Kaltura, discussion forums, and the
Gradebook (technological knowledge).
Knowledge:
Procedural
Yes
SMD faculty need to be able to conceptualize ways of sharing
content knowledge with students using the tools available in the
LMS (technological content knowledge).
Knowledge:
Conceptual
Yes
SMD faculty need to be able to conceptualize ways of using the
tools available in the LMS to improve student learning and
employ pedagogical strategies (technological pedagogical
knowledge).
Knowledge:
Conceptual
No
SMD faculty need to believe they are capable of effectively
learning how to utilize the LMS.
Motivation: Self-
Efficacy
Yes
SMD faculty need to believe that the LMS can provide them
with tools that will improve the quality of their courses and that
there is value in providing distance education courses in the
SMD.
Motivation:
Expectancy-
Value
No
SMD needs to have a system of monitoring faculty performance
in order to determine their ongoing progress with learning how
to utilize the LMS
Organization:
Cultural Setting
Yes
SMD needs to provide faculty members with adequate training
opportunities for learning how to utilize the LMS
Organization:
Cultural Setting
Yes
PREPARING FACULTY FOR DISTANCE EDUCATION 60
Knowledge Recommendations for Practice to Address KMO Influences
While a comprehensive examination of SMD faculty’s online teaching abilities would
ideally examine every aspect of the TPACK framework discussed in the literature review
(Herring, et al., 2016; Koehler & Mishra, 2009; Mishra & Koehler, 2006), this study was
primarily concerned with examining the faculty’s ability to utilize the learning management
system and therefore focused on the three TPACK intersections that are directly related to the
use of technology: technological knowledge (TK), technological content knowledge (TCK), and
technological pedagogical knowledge (TPK). The previously presented findings on these
knowledge influences are described in Table 6, along with information on research-based
principles that are used to inform the recommendations which will be discussed in the following
sections.
Table 6
Summary of Knowledge Influences and Recommendations
Knowledge
Influence
Validated as a
Gap?
Principle and
Citation
Context-Specific
Recommendation
SMD faculty need
steps to utilize
advanced LMS
features including
Kaltura, discussion
forums, and the
Gradebook (P).
Yes Managing
intrinsic load by
segmenting
complex
material into
simpler parts
and pre-training,
among other
strategies,
enables learning
to be enhanced
(Kirshner,
Kirshner, &
Paas, 2006)
Information
learned
meaningfully
Provide SMD faculty
training opportunities
with job aids in order
to learn the
procedural steps
required to utilize
these LMS features
PREPARING FACULTY FOR DISTANCE EDUCATION 61
and connected
with prior
knowledge is
stored more
quickly and
remembered
more accurately
because it is
elaborated with
prior learning
(Schraw &
McCrudden,
2006)
SMD faculty need to
know which tools in
the LMS are most
appropriate for
sharing different
kinds of learning
activities or content
(P).
Yes Social
interaction,
cooperative
learning, and
cognitive
apprenticeships
(such as
reciprocal
teaching)
facilitate
construction of
new knowledge
(Scott &
Palincsar, 2006).
Provide SMD faculty
designing online
courses with a
collaborative
instructional design
team including other
faculty members and
an instructional
designer.
SMD faculty need
education that
supports their ability
to create learning
activities and course
curricula which
employs pedagogical
strategies and
principles to enhance
student learning in
distance education
environments (M).
No Social
interaction,
cooperative
learning, and
cognitive
apprenticeships
(such as
reciprocal
teaching)
facilitate
construction of
new knowledge
(Scott &
Palincsar, 2006).
Provide SMD faculty
designing online
courses with a
collaborative
instructional design
team including other
faculty members and
an instructional
designer.
PREPARING FACULTY FOR DISTANCE EDUCATION 62
Procedural Knowledge of Utilizing LMS Tools
The results and findings of this study indicated that the SMD faculty are unfamiliar with
all three of the advanced LMS features included in the study: creating Kaltura videos, creating
discussion forums, and utilizing the gradebook. Three out of five faculty members who
participated in an interview self-reported that they had not used any of these features, and 88% of
the courses observed in artifact analysis also did not utilize any of these features. A
recommendation rooted in cognitive load theory has been identified to close this procedural
knowledge gap. Kirshner, Kirshner and Paas (2006) found that learning can be enhanced by
segmenting complex material into simpler parts. This evidence from the literature suggests that
providing faculty members with a job aid would support their learning. Therefore, the
recommendation is to provide faculty members with a job aid which details the steps for utilizing
each of these features. In the form of step-by-step procedural outlines, the job aid would
illustrate each step of the process for utilizing the features, breaking the process down into a
series of simpler components.
There is research which suggests that providing job aids for faculty members can be an
effective strategy for enhancing faculty members’ ability to utilize technology. Brill and Park
(2011) found that a collection of online self-paced technology tutorials was an effective tool for
enhancing faculty effectiveness with technology at a large multi-disciplinary university. The
authors examined faculty feedback on the tutorials through surveys with a combination of Likert
scale responses and open-ended qualitative comments. Faculty members reported a high level of
satisfaction with the tutorials. Faculty members also reported a higher level of engagement with
the self-paced online tutorials than with live in-person training due to the 24/7 on-demand
availability of the tutorials. These findings suggest that technology job aids would be an
PREPARING FACULTY FOR DISTANCE EDUCATION 63
effective solution to close the knowledge gap in this area, and potentially more effective than live
trainings.
Schraw and McCrudden (2006) identified that information connected with prior
knowledge is remembered more accurately when it is elaborated with prior learning. This
research suggests that the job aid may be more effective if it can be described in the context of
the faculty’s prior knowledge, such as providing context for the purpose of various LMS features
by drawing parallels to activities in conventional classrooms that the LMS feature might be used
to simulate. Jimoyiannis (2010) also found that a scenario-based training program for
developing technological knowledge in science teachers yielded positive results, which suggests
that a job aid might be further enhanced by including these scenario-based examples.
Conceptual Knowledge of Online Course Design
The results and findings of this study also indicated that there is a gap in the ability of the
SMD faculty to describe or create a lesson plan for an online course. Three out of five faculty
members who participated in an interview were able to describe a hypothetical online course
outline in only minimal detail. A recommendation rooted in sociocultural theory has been
identified to close this knowledge gap. Scott and Palincsar (2006) identified that social
interaction and cooperative learning can facilitate the construction of new knowledge. This
relationship between social interaction and learning suggests that providing faculty members
who are designing online courses with a collaborative course design team may be an effective
strategy for closing this knowledge gap. A collaborative course design team with multiple
faculty members and an instructional designer will provide faculty members with an opportunity
to support and reinforce each other’s learning through the co-construction of knowledge.
PREPARING FACULTY FOR DISTANCE EDUCATION 64
Koehler, Mishra, Hershey, and Peruski (2004) studied the creation of online courses at
Michigan State University utilizing a “learn by design” approach in which six faculty members
were paired with other faculty members and master’s degree students in order to co-create online
courses. The faculty members all had extensive experience designing courses for traditional
classroom environments, but little experience adapting these learning experiences to an online
format. The authors’ results suggested that, throughout the course of the collaborative course
design experience, the faculty members broadened their knowledge of technology in general, in
addition to learning new pedagogical strategies that were relevant to developing effective
distance education. These findings suggest that a similarly designed collaborative design process
might be effective at improving a faculty member’s ability to identify technology solutions for
incorporating pedagogical strategies within their online courses.
Conceptual Knowledge of Pedagogical Strategies for Distance Education
The results of the study identified that the SMD faculty members are able to describe
how technology can be used to facilitate pedagogical strategies and principles in their online
courses. Five out of five faculty members were able to describe pedagogical benefits of
discussion forums and in-video quizzes, while three out of five faculty members were able to
describe benefits of utilizing the gradebook. As a gap in this influence was not identified, no
recommendation is provided. If a gap in this area is identified in the future, the
recommendations rooted in sociocultural theory previously discussed for training faculty in
online course design could also be utilized for training faculty in the development of pedagogical
strategies.
PREPARING FACULTY FOR DISTANCE EDUCATION 65
Motivation Recommendations for Practice to Address KMO Influences
The SMD faculty will be more likely to achieve their stakeholder goal if any motivational
challenges are assessed and resolved. While there are many motivational influences that attempt
to explain motivation, there are two key motivational influences that were analyzed with respect
to their influence on faculty members: self-efficacy and expectancy-value. These motivational
influences are derived from self-efficacy theory and expectancy value theory. The study findings
with regard to these motivation influences are described in Table 7.
Table 7
Summary of Motivation Influences and Recommendations
Motivation Influence
Validated
as a Gap?
Principle and
Citation
Context-Specific
Recommendation
SMD faculty are not
confident in their ability to
utilize advanced features of
the LMS (Self-Efficacy)
Yes Feedback and
modeling
increases self-
efficacy
(Pajares, 2006).
Provide SMD faculty with
examples of positive
modeling via case studies of
faculty members who have
created successful online
classes in the LMS,
combined with guided
practice and immediate
feedback
SMD faculty do believe that
offering distance education
courses in their department
is useful or will contribute
to the achievement of the
department’s goals
(Expectancy-value)
No Rationales that
include a
discussion of the
importance
and utility value
of the work or
learning can help
learners develop
positive values
(Eccles, 2006;
Pintrich, 2003)
Provide SMD faculty with
information about the
usefulness of distance
education within their own
field.
PREPARING FACULTY FOR DISTANCE EDUCATION 66
Provide Modeling, Practice, and Feedback
Five out of five faculty members indicated that they were only confident in their ability to
use basic features in the LMS. A recommendation rooted in self-efficacy theory has been
identified to close this motivation gap. Pajares (2006) found that modeling and feedback
increases self-efficacy. This relationship suggests that providing the faculty with examples of
how to utilize the LMS and then giving them feedback on their performance will increase their
confidence or self-efficacy. The recommendation is therefore to provide the faculty with
demonstrations of how to utilize the LMS features, followed by opportunities to practice using
these features with immediate feedback. This combination of modeling, practice, and feedback
has been shown to increase self-efficacy.
Clark and Estes (2008) have stated that when organizational stakeholders have higher
levels of self-efficacy, they are more likely to demonstrate higher levels of performance. Abbitt
(2011) found that higher levels of self-efficacy among university faculty leads to higher levels of
technology adoption and utilization in teaching practices. Dunbar and Melton (2018) also found
that self-efficacy is an important predictor of performance for faculty members who teach online
classes. From a theoretical perspective, these findings suggest that increasing faculty self-
efficacy for utilizing the LMS will result in higher performance and more engagement when
teaching in distance education courses.
Communicate the Importance and Value
Five out of five faculty members indicated in interviews that they believed distance
education courses would provide value for their department. Therefore, a gap in this motivation
influence was not identified. However, if a gap in this area should ever emerge in the future, a
recommendation rooted in expectancy-value theory has been identified for addressing such a
PREPARING FACULTY FOR DISTANCE EDUCATION 67
gap. Eccles (2006) and Pintrich (2003) both found that providing a rationale that includes a
discussion of the importance and utility value of the work can help learners develop positive
values. These findings suggest that providing faculty members with rationales for providing
distance education offerings within their department that communicate the importance and utility
of distance education may increase their perceived value of distance education. Therefore, the
recommendation is to develop a communication plan or a set of training materials which explains
the value of distance education within the department’s field of study, and the value of the
various LMS features in facilitating distance education.
Lin and Ha (2009) conducted a study on the relationship between utility value and LMS
utilization at a Midwestern state university in the United States with about 2,500 full-time faculty
and staff employees. The researchers distributed a survey to all 2,500 stakeholders in both
written and online formats. The purpose of the survey was to evaluate the faculty and staff
perceptions of the LMS utility value and their usage of the LMS. A total of 1,022 surveys were
completed out of 2,484 participants surveyed. In addition, the researchers conducted 20 in-depth
qualitative interviews with various administrators, staff members, and faculty members. The
researchers found significant correlation between the reported utility value and the reported
usage of the LMS. This research suggests that increasing the level of utility value among SMD
faculty will increase their usage of the LMS and would thereby increase their willingness to
utilize the LMS in the delivery of distance education.
Organization Recommendations for Practice to Address KMO Influences
Organizations such as SUM are complex, with policies and cultures that may influence
performance. Clark and Estes (2008) describe the importance of examining any organizational
barriers that may impact stakeholder performance, because such an examination can help
PREPARING FACULTY FOR DISTANCE EDUCATION 68
organizational leaders understand the factors that impede or facilitate stakeholder performance.
The availability of training and the existence of a performance monitoring system were both
identified as organizational influences that are likely to have an impact on achievement of the
organizational goal. The availability of training resources and opportunities is an important
organizational influence on stakeholder performance (Clark & Estes, 2008). If faculty members
have knowledge gaps with regard to their knowledge of how to utilize the LMS, it will be
difficult even for highly-motivated faculty members to resolve these gaps if they are not given
adequate access to training resources. Similarly, performance monitoring is an essential
component of any organizational strategic plan (Clark & Estes, 2008). Without a method for
monitoring faculty performance, organizational leaders will not be able to determine whether or
not performance gaps are present, nor will they be able to determine whether or not performance
is improving over time (Alexander, 2000; Collan, Stoklasa & Talasova, 2014; Harris, Ingle &
Rutledge, 2014). If faculty members perceive that their performance is not being monitored,
many faculty members will not perform at their highest level (McInnis, 2002; Miller & Seldin,
2014). These two organizational influences and the study findings are described in Table 8.
PREPARING FACULTY FOR DISTANCE EDUCATION 69
Table 8
Summary of Organization Influences and Recommendations
Assumed
Organization
Influence
Validated
as a Gap?
Principle and
Citation
Context-Specific
Recommendation
SMD does not have
any system of
evaluating faculty
performance in
online courses
(cultural setting)
Y Performance
monitoring
systems must
have clearly
defined and
measurable
benchmarks
(Clark & Estes,
2008)
Develop a performance evaluation
system for SMD faculty teaching
online courses
SMD does not
provide ongoing
LMS training
opportunities for
faculty (cultural
setting)
Y The availability
of training
resources and
opportunities is
an important
organizational
influence on
stakeholder
performance
(Clark & Estes,
2008)
Provide LMS training and distance
education training opportunities for
SMD faculty teaching online
courses
Faculty Performance Evaluation
Five out of five SMD faculty reported that there is no performance evaluation system in
place which would assess their utilization of the LMS. This finding represents a gap because
evaluating the quality of LMS course content is a key requirement of an effective system for
evaluating faculty performance in a distance education setting. A recommendation rooted in
performance feedback theory has been identified to close this gap. Clark and Estes (2008)
indicate that performance monitoring systems must have clearly defined and measurable
benchmarks in order to be effective. Therefore, the recommendation is to create or adopt an
evaluation system for online courses with defined benchmarks. For example, the institution may
PREPARING FACULTY FOR DISTANCE EDUCATION 70
wish to adopt a set of standards for evaluating distance education courses that has been utilized
by other organizations, such as the Quality Matters rubric developed by the Maryland Online
college consortium, which is used by more than 1,300 colleges and universities throughout the
world (Legon, 2015; Varonis, 2014).
Clark and Estes (2008) indicate that it is important to have clearly defined performance
monitoring systems in order for organizational stakeholders and leaders to determine if
organizational goals are being met. Evidence suggests that faculty members will not perform to
the best of their ability if they perceive that their performance is not being monitored (McInnis,
2002; Miller & Seldin, 2014). It is important that the selected performance monitoring system
should evaluate faculty on a variety of criteria beyond simply measuring student performance
and feedback; there is evidence which suggests that faculty members are more likely to engage
in grade inflation when these are the only criteria utilized for performance evaluation (Craig,
Amernic, & Tourish, 2014; Langbein, 2008; Rutherford & Rabovsky, 2014). As such, the
literature supports the recommendation to implement additional performance evaluation criteria
beyond student performance and student feedback which are specific to the online learning
environment. Comprehensive evaluation systems such as the Quality Matters rubric can provide
this, providing standards for evaluating faculty on the presentation of their online materials and
the frequency and quality of their interactions with students (Legon, 2015; Varonis, 2014).
Faculty LMS Training
Five out of five SMD faculty members reported that LMS training was not provided for
them when they were initially hired. This lack of training opportunities represents an
organizational culture gap because of the previously described knowledge gaps. A
recommendation rooted in organizational theory has been identified to close this gap. Clark and
PREPARING FACULTY FOR DISTANCE EDUCATION 71
Estes (2008) indicate that the availability of training resources is an important factor in
stakeholder performance. Therefore, the recommendation is to provide ongoing and regular
LMS training opportunities for SMD faculty teaching distance education courses. For example,
providing a quarterly LMS training workshop for SMD faculty which teaches specific LMS
features or concepts might help to close this gap.
Clark and Estes (2008) noted that even highly motivated stakeholders may have difficulty
meeting performance goals if they are not provided with adequate training. Several authors have
found that training for faculty is especially important in distance education programs, as faculty
members may be required to heavily utilize technology that they are unfamiliar with (Blakely,
2017; Georgina & Hosford, 2009; Marek, 2009). Distance education courses often require
faculty members to develop skills with technology that they may not have developed during their
prior experiences teaching in conventional classroom settings (Brinkley-Etzkorn, 2018; Lee &
Hirumi, 2004; Moore, Sheffield, Robinson & Phillips, 2014; Wilson & Stacey, 2004). As such,
the literature supports the importance of providing distance education faculty with regular and
ongoing opportunities to receive technology training.
Integrated Implementation and Evaluation Plan Framework
This implementation and evaluation plan was created using the New World Kirkpatrick
Model (Kirkpatrick & Kirkpatrick, 2016). This framework is an updated version of the original
Four Level Model of Evaluation (Kirkpatrick & Kirkpatrick, 2006). The New World model
includes updated frameworks for the modern workplace and clarifies some misconceptions about
the original model. The updated model recommends that evaluation plans are drafted by starting
with the organization’s end goals and working backwards. This method of designing
implementation and evaluation plans in reverse allows organizational leaders to more easily
PREPARING FACULTY FOR DISTANCE EDUCATION 72
develop solutions for closing organizational performance gaps that are aligned with the
organization’s mission and that include performance assessments which are aligned with the
intended goals. Ensuring that goals and solutions are closely connected using backwards design
leads to a higher rate of success for the proposed solutions (Kirkpatrick & Kirkpatrick, 2016).
Organizational Purpose, Need and Expectations
SUM’s mission is to train future medical professionals in the art and science of practicing
medicine. SUM has only recently begun to offer distance education courses and programs.
Offering distance education courses is important for SUM’s mission as it will increase the
number of potential students that might receive medical training from SUM. The SMD, an
academic department within SUM, has a goal of beginning to offer distance education courses in
the year 2020. Because the SMD has not previously offered distance education courses, the
SMD faculty are mostly unfamiliar with the technology skills and pedagogical principles
associated with distance education. This study examined the KMO influences that facilitated or
impeded the SMD faculty from being prepared to teach in distance education settings. The
proposed solution is to create various training programs which are suited for preparing the SMD
faculty to teach distance education courses, and to create methods of performance evaluation that
can evaluate the SMD faculty’s effectiveness in distance education settings. The goal of these
programs is to ensure that all SMD faculty who teach in future distance education courses are
competent at utilizing the learning management system for teaching in an online setting.
Level 4: Results and Leading Indicators
Table 9 shows the outcomes, metrics, and methods for both internal and external
outcomes that are proposed as Level 4 indicators within the New World Kirkpatrick Model
(Kirkpatrick & Kirkpatrick, 2016). If the methods in Table 9 are utilized, then the metrics
PREPARING FACULTY FOR DISTANCE EDUCATION 73
should indicate whether the SMD faculty are achieving or not achieving their intended outcomes.
When the internal outcomes are being achieved, the external outcomes should also be achieved.
Table 9
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
External Outcomes
Increased number of
distance education courses
offered to students within
the SMD
Number of online or distance
education courses offered by
SMD in the course catalog each
academic term
Record data from the course
catalog and track changes in
course offerings over time
Internal Outcomes
Increased number of
training opportunities for
LMS technology provided
to SMD faculty
Number of asynchronous LMS
training resources provided for
SMD faculty and number of
synchronous LMS training
sessions or activities offered
each term
Record data each academic term
regarding (1) the number of
LMS training videos or self-
paced courses and (2) the
number of LMS training
sessions or activities offered
Increased SMD faculty
confidence in ability to
teach distance education
courses
Percentage increase on quarterly
SMD faculty survey related to
self-reported self-efficacy for
teaching distance education
courses
Administer quarterly survey to
SMD faculty and record results
Increased SMD faculty
utilization of LMS
technology tools in
conventional courses
Percentage increase of quarterly
LMS artifact analysis related to
SMD faculty utilization of LMS
features
Conduct quarterly examination
of SMD faculty LMS courses
and record usage rates of
various LMS features such as
the gradebook, Kaltura videos,
or discussion forums
Level 3: Behavior
The primary stakeholders in this study are the full-time faculty members of the SMD.
Key critical behaviors are needed in order to ensure stakeholder success. The first critical
behavior that the SMD faculty must engage in is participation in the LMS training opportunities
that are provided. Training participation is important in order for the SMD faculty to increase
PREPARING FACULTY FOR DISTANCE EDUCATION 74
their competence and confidence with utilizing LMS features that are necessary for distance
education. The second critical behavior is that an adequate number of SMD faculty must
volunteer to teach in distance education courses.
Table 10
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s)
Method(s)
Timing
1. Increased rate of
SMD faculty who
participate in LMS
training opportunities
Increased rate of SMD
faculty who have
completed
asynchronous training
materials and who
attend synchronous
training each
academic term
Record SMD faculty
participation in self-
paced training courses
or videos and record
SMD faculty attendance
at live training sessions
or activities
New training
opportunities for
SMD faculty must
be offered each
academic term
(quarterly)
2. Increased number
of SMD faculty who
are willing to
volunteer as
instructors for
distance education
courses
Increased rate of
positive responses
from SMD faculty on
quarterly survey
related to motivation
for teaching distance
education courses
Administer quarterly
survey to SMD faculty
and record results
SMD faculty must
volunteer to teach
distance education
courses at least 3
months before the
launch of a new
course
Required drivers. SMD faculty must meet these goals in order to become more
effective distance education instructors. For SMD faculty to meet these goals, certain drivers
will be implemented to reinforce defined behaviors, reward positive outcomes, and monitor
overall faculty performance. Table 11 shows the drivers that are recommended to support the
faculty’s critical behaviors.
PREPARING FACULTY FOR DISTANCE EDUCATION 75
Table 11
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical Behaviors Supported
1, 2, 3 Etc.
Reinforcing
Academic leaders provide
SMD faculty with a list of
upcoming LMS training
opportunities at faculty
meetings
Quarterly 1
Academic leaders provide
incoming SMD faculty
members with a set of job aids
demonstrating how to use
various LMS features
Quarterly 2
Encouraging
Technology training providers
offer ongoing mentorship to
SMD faculty who participate
in training or seek assistance
for technology issues
Ongoing 1
Academic leaders offer
ongoing mentorship to SMD
faculty who volunteer to teach
in distance education courses
Ongoing 2
Rewarding
Credits toward promotion in
faculty incentive system for
participating in training
opportunities
Annually 1
Credit toward promotion in
faculty incentive system for
volunteering to teach distance
education courses
Annually 2
Public acknowledgement for
faculty members who
volunteer to teach distance
education courses, such as
congratulations and thanks at
faculty meetings
Quarterly 2
Monitoring
Academic leaders will
communicate with faculty
who participate in LMS
Ongoing 1
PREPARING FACULTY FOR DISTANCE EDUCATION 76
training to make sure that they
are satisfied with the training
provided
Academic leaders will
communicate with faculty
teaching distance education to
find areas of concern or issues
that might negatively impact
their performance
Ongoing 2
Organizational support. To ensure that SMD faculty meet their goal of achieving
competency with the software tools provided in SUM’s LMS, academic leaders will need to
implement the drivers noted in Table 11. This study recommends that new opportunities for
LMS training are provided to all SMD faculty on a quarterly basis. Further, to ensure that
enough SMD faculty will be available to teach ongoing distance education courses, it is
recommended that academic leaders provide faculty with credits toward promotion in the faculty
incentive structure for participating in LMS training and volunteering to teach distance education
courses. It is also recommended that academic leaders regularly communicate with SMD faculty
on an ongoing basis in order to receive feedback about their experiences and concerns with
regard to distance education. This ongoing communication should provide academic leaders
with opportunities to identify and resolve challenges which may be preventing SMD faculty
from participating in distance education initiatives.
Level 2: Learning
The solutions recommended above are intended to improve the performance of SMD
faculty members while also achieving organizational goals. If correctly implemented, Level 2
learning should allow the faculty to achieve the skills, knowledge, and confidence (Kirkpatrick
& Kirkpatrick, 2016) needed to become effective distance education instructors. After SUM
implements the recommended solutions, the SMD faculty should be able to:
PREPARING FACULTY FOR DISTANCE EDUCATION 77
1. Develop curriculum for distance education courses that incorporates pedagogical
strategies which have been shown to improve student learning outcomes (Metacognitive)
2. Identify common learning and motivational challenges for students in distance education
courses and describe basic instructional principles for overcoming those challenges
(Declarative)
3. Utilize the Kaltura video content management system in Moodle to create and publish
instructional videos in distance education courses (Procedural)
4. Utilize the Moodle gradebook to deliver grades and assignment feedback to students
through the LMS (Procedural)
5. Utilize the discussion forum feature in Moodle to create online discussion activities
which encourage student interaction in distance education courses (Procedural)
6. Feel a greater level of confidence in their ability to teach distance education courses
(Self-efficacy)
7. Feel a greater level of confidence in their ability to use key features in Moodle (Self-
efficacy)
Program. To achieve the goals in the previous section, SUM should implement the
following training program for SMD faculty. This program will provide conceptual instruction
on distance education pedagogy, as well as hands-on instruction on how to use various features
in Moodle which are essential to facilitating distance education. This training can be facilitated
by an internal member of the organization. A staff member with long-term experience at the
organization and widespread respect among the SMD faculty would be an ideal facilitator. This
staff member could be a member of the administration with distance education and instructional
design experience, or the staff member could be a member of the faculty with proven experience
PREPARING FACULTY FOR DISTANCE EDUCATION 78
teaching in distance education courses. The training program to be implemented includes four
two-hour sessions for a total of eight training hours.
Following the delivery of this initial training series, faculty members who volunteer to
teach a distance education course are invited to participate in an ongoing collaborative course
design team, facilitated by an instructional designer, which meets for one hour each week over a
12-week term. During these 12 collaborative meetings, faculty members will discuss how they
are applying the skills learned in the initial training and share ideas about activities or
assessments in their upcoming courses. The meetings will also act as an open forum for faculty
members to describe technical issues they may be experiencing with the LMS and receive advice
or assistance from their peers or from the instructional designer. If possible, a senior faculty
member with prior experience teaching online courses should also be invited to assist the
instructional designer with facilitating the discussions at these weekly meetings.
During the initial four training sessions, the SMD faculty will meet at SUM’s main
campus. This facility will provide access to an instructional computer lab which faculty
members can use for hands-on practice during the training, which can be used for faculty
members to practice performing different tasks in Moodle and receiving immediate feedback
from the trainer. The first 2-hour session will take place in a conventional classroom and will
consist of learning objectives related to distance education challenges and pedagogy, as well as
provide participants with an outline of the upcoming series of sessions.
The remaining three two-hour sessions will take place in the instructional computer lab
and consist of learning objectives related to hands-on practice with using Moodle features. The
first of these sessions will provide training on using the Moodle gradebook to create assignments
and give feedback to students. Instructors will practice taking the syllabus for an example course
PREPARING FACULTY FOR DISTANCE EDUCATION 79
and creating all of the gradebook entries for that course. The second of these sessions will
provide training on creating videos in Kaltura. Instructors will practice using a webcam to
record a video and then post that video in a practice Moodle course. The third of these sessions
will provide training on creating discussion forums. Instructors will practice creating a
discussion forum topic and grading student responses within it.
The 12 weekly meetings to follow the initial trainings will also be hosted in the
instructional computer lab, as this will allow faculty members to work directly on their course
content during the meetings. Hosting the discussions in the computer lab will allow faculty
members to immediately practice new skills or ideas that are presented by their peers, and to
receive technical assistance with course content during the meeting. Overall, completion of the
eight initial training hours and the additional 12 hours of weekly discussion meetings will require
a total time commitment of 20 hours per participating faculty member.
Evaluation of the components of learning. To apply the knowledge necessary to
become effective distance education instructors, SMD faculty should be able to demonstrate their
declarative knowledge of distance education principles and then demonstrate procedural
knowledge of how to perform the tasks that are necessary for creating distance education
content. For SUM, this means that the SMD faculty should be able to demonstrate mastery of
the declarative and procedural knowledge influences that were validated in the study. Therefore,
it is essential that mastery of these knowledge influences is confirmed by assessment of the SMD
faculty both during and after the recommended training program. Table 12 lists the suggested
evaluation methods in order to assess the achievement of this knowledge.
PREPARING FACULTY FOR DISTANCE EDUCATION 80
Table 12
Evaluation of the Components of Learning for the Program
Method or Activity Timing
Declarative Knowledge “I know it.”
Guided conversations regarding general
pedagogical principles related to distance
education
During the first training session
Post-test assessment survey asking faculty
members to describe how each Moodle feature
can be used to facilitate student learning
Within a two-week period following the fourth
training session
Procedural Skills “I can do it right now.”
Observation of faculty member ability to create
an example assignment that incorporates
pedagogical principles related to distance
education
During the first training session
Observation of faculty member ability to create
a graded assignment in Moodle and utilize the
gradebook to deliver grades and feedback to
students
During the second training session
Observation of faculty member ability to
record a Kaltura video and publish it within a
Moodle course
During the third training session
Observation of faculty member ability to create
a discussion forum and grade student posts
within that discussion forum
During the fourth training session
Attitude “I believe this is worthwhile.”
Observation of faculty engagement throughout
the training session and discussion meetings
During each of the four training sessions and
the 12 meetings
Conversations regarding the value of distance
education within the organization
During the first training session
Follow-up survey regarding faculty member
feelings about the value of distance education
Within two weeks after the fourth training
session and again two weeks after the 12
th
discussion meeting
Confidence “I think I can do it on the job.”
Guided conversation regarding faculty
members’ beliefs on their ability to become
effective distance education instructors
During the first training session and
periodically throughout the 12 post-training
discussion meetings
Guided conversations regarding faculty
members’ beliefs on their ability to utilize the
various features in Moodle to create distance
education course content and assignments
Following instruction on Moodle features
during the second, third, and fourth training
sessions
Follow-up survey regarding faculty member
self-efficacy for teaching distance education
Within two weeks after the fourth training
session and again two weeks after the 12
th
discussion meeting
PREPARING FACULTY FOR DISTANCE EDUCATION 81
Commitment “I will do it on the job.”
Review of distance education curriculum
developed by faculty members to ensure that
proposed course curriculum incorporates
pedagogical principles
Within three months after the fourth training
session and again after the 12
th
discussion
meeting
Review of Moodle course materials developed
by faculty members to observe whether or not
Moodle features are utilized by faculty
Within three months after the fourth training
session and again after the 12
th
discussion
meeting
Level 1: Reaction
A series of assessment methods will be used during and after the program in order to
ascertain if faculty members are fully engaged in the training. For example, the training
facilitator will be prepared to observe the faculty members’ engagement as they are being trained
on how to perform various tasks in Moodle. The facilitator will also be prepared to observe the
faculty members’ engagement and responses in guided conversations on the value of distance
education within the organization. The facilitator will check-in with the faculty members during
guided hands-on practice that occurs during the training, and the faculty members will be sent a
feedback survey on the training after the conclusion of the fourth training session. Table 13
indicates all of the methods that will be used to assess faculty reactions to the training program.
Table 13
Components to Measure Reactions to the Program
Method(s) or Tool(s) Timing
Engagement
Observation by the training facilitator Ongoing during each of the four training
sessions and each of the 12 discussion
meetings
Attendance During each of the four training sessions and
each of the 12 discussion meetings
Training feedback survey Within two weeks after the fourth training
session and again two weeks after the 12
th
discussion meeting
Relevance
PREPARING FACULTY FOR DISTANCE EDUCATION 82
Recurring check-ins with faculty members
through guided discussions and questions
Ongoing during each of the four training
sessions and the 12 discussion meetings
Training feedback survey Within two weeks after the fourth training
session and again two weeks after the 12
th
discussion meeting
Customer Satisfaction
Recurring check-ins with faculty members
through guided discussions and questions
Ongoing during each of the four training
sessions and the 12 discussion meetings
Training feedback survey Within two weeks after the fourth training
session and again two weeks after the 12
th
discussion meeting
Evaluation Tools
When each of the four training sessions end, faculty members will be asked to complete a
short evaluation of the session. Appendix F shows the full evaluation tool and questions that will
be asked of the faculty members after each session. The questions include Level 1 assessments
for engagement, relevance, and customer satisfaction, as well as Level 2 assessments for
procedural knowledge and confidence.
During each of the training sessions, the facilitator will make observations regarding
faculty engagement. The facilitator will check for relevance and customer satisfaction by
checking in with the faculty members through guided discussions and providing time for
questions and answers during each session. Recurring check-ins and opportunities for hands-on
practice will be provided in order to verify that faculty members are meeting the learning
objectives of each session.
Delayed for a period after the program implementation. Within two weeks after the
fourth training session, faculty members will be asked to complete a feedback survey about the
training program. Appendix G shows the survey questions that will be included in the evaluation
tool. The feedback survey will assess the faculty members’ perspectives on the relevance of
material presented throughout the program (Level 1), their ability to apply the training in their
PREPARING FACULTY FOR DISTANCE EDUCATION 83
work on developing distance education courses (Level 2), their success at immediately utilizing
knowledge from the training on their existing courses in Moodle (Level 3), and their willingness
to volunteer for teaching future distance education courses (Level 4). This survey will be repeated
again for the collaborative discussion groups after the 12
th
weekly meeting.
Data Analysis and Reporting
The Level 4 goal of offering more distance education courses within the SMD is measured
by reviewing the course catalog offerings each term. On a quarterly basis, academic leaders in the
SMD will report on the number of distance education courses that are listed in the course catalog.
The recommended approach to presenting this data is through a written report that is delivered to
the provost’s office. An example figure with hypothetical data is shown in Appendix H.
Similarly to the Level 4 goal, the Level 3 goals of increased faculty participation in LMS
training and increased numbers of faculty volunteering to teach distance education courses will be
monitored and reported on a quarterly basis. Data will be collected by tracking faculty attendance
at training opportunities and by tracking faculty responses to academic leaders’ inquiries regarding
distance education opportunities. An example figure with hypothetical data for the Level 3 goals
is shown in Appendix I.
Summary
This implementation and evaluation plan was developed using the New World
Kirkpatrick Model. This model is designed to begin with the organizational goals and then guide
the change process through the four levels to achieve the desired results. The model is flexible
enough for many different types of goals and training needs. The model’s emphasis on
integrating implementation and evaluation of training in a single process helps to ensure that the
training can deliver high returns on expectations for the organization (Kirkpatrick & Kirkpatrick,
PREPARING FACULTY FOR DISTANCE EDUCATION 84
2016). Through the implementation of the training and monitoring outlined in this recommended
program, SUM should be able to achieve its goal of launching distance education courses within
the SMD and training the SMD faculty to become competent distance education instructors.
Limitations of the Research
A key limitation of this research is that the results cannot be generalized to other
institutions, or even to other departments within SUM. This research is only relevant to the
academic department which was directly studied. Another important limitation is that the
research findings may not be representative of the entire faculty population within the SMD.
Because the target population consisted of 10 full-time faculty members and only five out of 10
faculty members participated in the research, it is possible that those additional faculty members
may have different beliefs, attitudes, or challenges than the faculty members who did choose to
participate.
This limitation is especially important to note due to the likelihood of non-response bias
(Creswell, 2014), which could mean that the faculty members who chose to participate might
differ in some significant way from the faculty members who chose not to participate. For
example, the recruitment protocol for this study specified that potential participants be informed
of the subject matter and goals of the study. Because prospective participants were aware that
this research was related to the development of distance education, it is possible that faculty
members who chose not to participate might have different opinions regarding distance
education. It is therefore important to note that the results and findings of this study may not be
applicable to all of the faculty members within the SMD. It should also be noted that the SMD
employs a number of part-time adjunct faculty members who may also be considered as potential
PREPARING FACULTY FOR DISTANCE EDUCATION 85
distance education instructors, and the findings of this research cannot be generalized to that
population either.
Recommendations for Future Research
There are several potential avenues for future research in this area. Within the SMD,
future research might be conducted to examine the KMO influences on the full-time faculty
members who did not participate in the study. A new study with different methods might
produce a higher response rate among the target population. For example, a study utilizing short
online surveys might achieve a higher response rate as faculty members might perceive a survey
as less of a demand on their time than an interview. Additionally, if the SMD leaders wished to
consider the possibility of launching distance education courses utilizing adjunct faculty
members, the KMO influences that affect the adjunct faculty may be different and a study
examining those influences could be considered.
Additionally, within SUM, departments other than the SMD may also be considered for
the development of new distance education offerings. As discussed previously, a limitation of
this research is that it is not generalizable to other departments within SUM. Therefore, research
examining the goals and KMO influences within those departments should be considered in
order to assess the challenges that might be faced within their unique organizational and cultural
contexts.
Conclusion
Distance education is already a common practice within the field of higher education.
Between 2002 and 2016, the percentage of higher education students taking online courses in the
U.S. has increased every year. In 2016, 31.6% of all higher education students in the U.S. were
enrolled in at least one distance education course, and 14.9% of all students were enrolled in
PREPARING FACULTY FOR DISTANCE EDUCATION 86
fully online programs (Seamen, et al., 2018). If these trends continue, distance education will
become even more widespread in the future. It is even possible that the majority of higher
education students may eventually complete their programs in an online format.
The findings of this study suggest that the Specialized Medicine Department at SUM is
well-positioned to begin work on creating distance education course offerings. Despite the
study’s limitations from the likelihood of non-response bias, the research does suggest that at
least half of the full-time faculty members in the SMD are willing to volunteer as online
instructors. This willingness to teach online means that one of the major motivational obstacles
to creating distance education offerings has already been at least partially overcome. If the
recommended training program is implemented by SUM, it is likely that the faculty members
will be able to learn the skills necessary to achieve competence with the LMS and thereby
become more effective distance education instructors.
The original organizational goal of offering distance education courses in the department
by January 2020 is no longer feasible at this time. The department will need to revise the target
date of this goal in order to provide the faculty members with adequate time to complete the
recommended training program. However, it is likely that the department could meet this goal
with a new timeline. Even if 100% of the full-time faculty members are not yet ready to teach
online, the department may be able to proceed with online courses taught by a small number of
willing faculty volunteers. If these courses are then successful, more faculty members may
become motivated to participate after witnessing the success of their colleagues. The gap
analysis framework developed by Clark and Estes (2008) can then be used to facilitate a cycle of
continuous improvement in which the SMD, and by extension, SUM as a whole, becomes
PREPARING FACULTY FOR DISTANCE EDUCATION 87
gradually more prepared over time to offer a wider variety of distance education courses and
degree programs.
PREPARING FACULTY FOR DISTANCE EDUCATION 88
References
Abbitt, J. T. (2011). An investigation of the relationship between self-efficacy beliefs about
technology integration and technological pedagogical content knowledge. Journal of
Digital Learning in Teacher Education, 27(4), 134.
About Moodle. (2018, December 4). Retrieved from:
https://docs.moodle.org/37/en/About_Moodle
Abrami, P. C., Bernard, R. M., Bures, E. M., Borokhovski, E., & Tamim, R. M. (2011).
Interaction in distance education and online learning: Using evidence and theory to
improve practice. Journal of Computing in Higher Education, 23(2-3), 82-103.
Alexander, F. K. (2000). The changing face of accountability: Monitoring and assessing
institutional performance in higher education. The Journal of Higher Education, 71(4),
411-431.
Allen, I. E., & Seaman, J. (2013). Changing course: Ten years of tracking online education in
the United States. Sloan Consortium. PO Box 1238, Newburyport, MA 01950.
Anra, Y., & Yamin, M. (2017). Relationships between lecturer performance, organizational
culture, leadership, and achievement motivation. Ф орс айт, 11(2 (eng)).
Archambault, L., & Crippen, K. (2009). Examining TPACK among K-12 online distance
educators in the United States. Contemporary issues in technology and teacher
education, 9(1), 71-88.
Bandura, A. (1993). Perceived self-efficacy in cognitive development and
functioning. Educational psychologist, 28(2), 117-148.
Bandura, A. (2005). The evolution of social cognitive theory. In K. G. Smith & M. A. Hitt
(Eds.), Great minds in management (pp. 9–35). Oxford: Oxford University Press.
PREPARING FACULTY FOR DISTANCE EDUCATION 89
Baran, E., & Correia, A. P. (2014). A professional development framework for online
teaching. TechTrends, 58(5), 95-101.
Bawa, P. (2016). Retention in online courses: Exploring issues and solutions—a literature
review. Sage Open, 6(1), https://doi.org/10.1177/2158244015621777
Betts, K. (2014). Factors influencing faculty participation & retention in online & blended
education. Online journal of distance learning administration, 17(1), 1.
Blakely, B. J. (2015). Pedagogical technology experiences of successful late-career
faculty. College Teaching, 63(4), 146-152.
Bolliger, D. U. (2004). Key factors for determining student satisfaction in online
courses. International Journal on E-learning, 3(1), 61-67.
Bolliger, D. U., Inan, F. A., & Wasilik, O. (2014). Development and validation of the online
instructor satisfaction measure (OISM). Journal of Educational Technology &
Society, 17(2).
Bolliger, D. U., & Martin, F. (2018). Instructor and student perceptions of online student
engagement strategies. Distance Education, 39(4), 568-583.
Bolliger, D. U., & Wasilik, O. (2009). Factors influencing faculty satisfaction with online
teaching and learning in higher education. Distance Education, 30(1), 103-116.
Brenton, H., Hernandez, J., Bello, F., Strutton, P., Purkayastha, S., Firth, T., & Darzi, A. (2007).
Using multimedia and Web3D to enhance anatomy teaching. Computers &
Education, 49(1), 32-53.
Brill, J., & Park, Y. (2011). Evaluating online tutorials for university faculty, staff, and
students: The contribution of just-in-time online resources to learning and performance.
International Journal on E-learning, 10(1), 5-26.
PREPARING FACULTY FOR DISTANCE EDUCATION 90
Brinkley-Etzkorn, K. E. (2018). Learning to teach online: Measuring the influence of faculty
development training on teaching effectiveness through a TPACK lens. The Internet and
Higher Education, 38, 28-35.
Burke, G., Mac and Bhaird, C., & OʼShea, A. (2012). The impact of a monitoring scheme on
engagement in an online course. Teaching Mathematics and Its Applications: An
International Journal of the IMA, 31(4), 191-198.
Carr, S. (2000). As distance education comes of age, the challenge is keeping the
students. Chronicle of higher education, 46(23).
Caruth, G. D., & Caruth, D. L. (2013). Distance education in the United States: From
correspondence courses to the Internet. Turkish Online Journal of Distance
Education, 14(2).
Chai, C. S., Koh, J. H. L., & Tsai, C. C. (2010). Facilitating preservice teachers' development of
technological, pedagogical, and content knowledge (TPACK). Educational Technology
& Society, 13(4), 63-73.
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Charlotte, NC: Information Age Publishing, Inc.
Cole, M. T., Shelley, D. J., & Swartz, L. B. (2014). Online instruction, e-learning, and student
satisfaction: A three year study. The International Review of Research in Open and
Distributed Learning, 15(6).
Craig, R., Amernic, J., & Tourish, D. (2014). Perverse audit culture and accountability of the
modern public university. Financial Accountability & Management, 30(1), 1-24.
Creswell, J. W. (2014). A concise introduction to mixed methods research. Sage Publications.
PREPARING FACULTY FOR DISTANCE EDUCATION 91
Daly, E. (2006) Behaviorism. Retrieved
from: http://www.education.com/reference/article/behaviorism/.
Denler, H., Wolters, C., & Benzon, M. (2006). Social cognitive theory. Retrieved
from: http://www.education.com/reference/article/social-cognitive-theory/.
Dillon, C. L., & Walsh, S. M. (1992). Faculty: The neglected resource in distance
education. American Journal of Distance Education, 6(3), 5-21.
Dong, C., & Goh, P. S. (2015). Twelve tips for the effective use of videos in medical
education. Medical teacher, 37(2), 140-145.
Dunbar, M., & Melton, T. D. (2018). Self-Efficacy and Training of Faculty Who Teach Online.
In Self-Efficacy in Instructional Technology Contexts (pp. 15-33). Springer, Cham.
Eccles, J. (2006). Expectancy value motivational theory. Retrieved from:
http://www.education.com/reference/article/expectancy-value-motivational-theory/.
Feldstein, M. (2018). Canvas surpasses Blackboard Learn in US Market Share. E-Literate, July
8, 2018. Retrieved from https://mfeldstein.com/canvas-surpasses-blackboard-learn-in-us-
market-share/.
Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist,
36(1), 45–56.
Georgina, D. A., & Hosford, C. C. (2009). Higher education faculty perceptions on technology
integration and training. Teaching and Teacher Education, 25(5), 690-696.
Gonzales, C., Pickett, L., Hupert, N., & Martin, W. (2002). The regional educational technology
assistance program: Its effects on teaching practices. Journal of Research on Technology
in Education, 35(1), 1.
PREPARING FACULTY FOR DISTANCE EDUCATION 92
Glesne, C. (2011). Chapter 6: But is it ethical? Considering what is “right.” In Becoming
qualitative researchers: An introduction (4th Ed.) (pp. 162-183). Boston, MA: Pearson.
Graham, R. C., Burgoyne, N., Cantrell, P., Smith, L., St Clair, L., & Harris, R. (2009).
Measuring the TPACK confidence of inservice science teachers. TechTrends, 53(5), 70-
79.
Gray, J. A., & DiLoreto, M. (2016). The Effects of Student Engagement, Student Satisfaction,
and Perceived Learning in Online Learning Environments. International Journal of
Educational Leadership Preparation, 11(1), n1.
Green, T., Alejandro, J., & Brown, A. H. (2009). The retention of experienced faculty in online
distance education programs: Understanding factors that impact their involvement. The
International Review of Research in Open and Distributed Learning, 10(3).
Harris, D. N., Ingle, W. K., & Rutledge, S. A. (2014). How teacher evaluation methods matter
for accountability: A comparative analysis of teacher effectiveness ratings by principals
and teacher value-added measures. American Educational Research Journal, 51(1), 73-
112.
Harris, J. B., & Hofer, M. J. (2011). Technological pedagogical content knowledge (TPACK) in
action: A descriptive study of secondary teachers’ curriculum-based, technology-related
instructional planning. Journal of Research on Technology in Education, 43(3), 211-229.
Hartman, J., Dziuban, C., & Moskal, P. (2000). Faculty satisfaction in ALNs: A dependent or
independent variable. Journal of Asynchronous Learning Networks, 4(3), 155-179.
Herring, M. C., Koehler, M. J., & Mishra, P. (Eds.). (2016). Handbook of technological
pedagogical content knowledge (TPACK) for educators. Routledge.
PREPARING FACULTY FOR DISTANCE EDUCATION 93
Horvitz, B. S., Beach, A. L., Anderson, M. L., & Xia, J. (2015). Examination of faculty self-
efficacy related to online teaching. Innovative Higher Education, 40(4), 305-316.
Jaggars, S. S., & Xu, D. (2016). How do online course design features influence student
performance? Computers & Education, 95, 270-284.
Jimoyiannis, A. (2010). Developing a technological pedagogical content knowledge framework
for science education: Implications of a teacher trainers’ preparation program. In
Proceedings of Informing Science & IT Education Conference (InSITE) (Vol. 597, p.
607). FL: InSITE.
Johnson, W. B. (2015). On being a mentor: A guide for higher education faculty. Routledge.
Kane, R. T., Shaw, M., Pang, S., Salley, W., & Snider, J. B. (2015). Relationships among faculty
training, faculty degree, faculty longevity, and student satisfaction in online higher
education. Online Journal of Distance Learning Administration, 18(4), n4.
Kaltura. (2018). Kaltura Video Package for Moodle. Retrieved from:
https://corp.kaltura.com/Products/Video-Applications/Kaltura-Video-Package-for-
Moodle/.
Kelly, M., Lyng, C., McGrath, M., & Cannon, G. (2009). A multi-method study to determine the
effectiveness of, and student attitudes to, online instructional videos for teaching clinical
nursing skills. Nurse education today, 29(3), 292-300.
Kezar, A., & Eckel, P. D. (2002). The effect of institutional culture on change strategies in
higher education: Universal principles or culturally responsive concepts? The Journal of
Higher Education, 73(4), 435-460.
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick's four levels of training evaluation.
Association for Talent Development.
PREPARING FACULTY FOR DISTANCE EDUCATION 94
Koehler, M. (2011, May 11). Using the TPACK Image. Retrieved: from http://tpack.org/
Koehler, M., & Mishra, P. (2009). What is technological pedagogical content knowledge
(TPACK)? Contemporary issues in technology and teacher education, 9(1), 60-70.
Koehler, M. J., Shin, T. S., & Mishra, P. (2012). How do we measure TPACK? Let me count the
ways. In Educational technology, teacher knowledge, and classroom impact: A research
handbook on frameworks and approaches (pp. 16-31). IGI Global.
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into
practice, 41(4), 212-218.
Krueger, R. A., & Casey, M. A. (2009). Chapter 3: Developing a questioning route. In Focus
groups: A practical guide for applied research (4th Ed.) (pp. 35-61). Thousand Oaks,
CA: SAGE Publications.
Kuo, Y. C., Walker, A. E., Schroder, K. E., & Belland, B. R. (2014). Interaction, Internet self-
efficacy, and self-regulated learning as predictors of student satisfaction in online
education courses. The Internet and Higher Education, 20, 35-50.
Langbein, L. (2008). Management by results: Student evaluation of faculty teaching and the mis-
measurement of performance. Economics of Education Review, 27(4), 417-428.
Lantz, M. E., & Stawiski, A. (2014). Effectiveness of clickers: Effect of feedback and the timing
of questions on learning. Computers in Human Behavior, 31, 280-286.
Lee, J. A., & Busch, P. E. (2005). Factors related to instructors' willingness to participate in
distance education. The Journal of Educational Research, 99(2), 109-115.
Lee, J. L., & Hirumi, A. (2004). Analysis of Essential Skills and Knowledge for Teaching
Online. Association for Educational Communications and Technology.
PREPARING FACULTY FOR DISTANCE EDUCATION 95
Legon, R. (2015). Measuring the impact of the Quality Matters Rubric™: A discussion of
possibilities. American Journal of Distance Education, 29(3), 166-173.
Lin, C., & Ha, L. (2009). Subcultures and use of communication information technology in
higher education institutions. The Journal of Higher Education, 80(5), 564-590.
Manning, R., & Smith, D. (2018). Creating Spaces for Learning: Online Forums. In Creativity
and Critique in Online Learning (pp. 31-57). Palgrave Macmillan, Cham.
Marek, K. (2009). Learning to teach online: Creating a culture of support for faculty. Journal of
Education for Library and Information Science, 275-292.Maxwell, J. A.
(2013). Qualitative research design: An interactive approach (Vol. 41). Sage
publications.
Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson/Allyn & Bacon.
McInnis, C. (2002). The impact of technology on faculty performance and its evaluation. New
directions for institutional research, 2002(114), 53-62.
McNeil, D. R. (1990). Wiring the Ivory Tower: A Round Table on Technology in Higher
Education.
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation. John Wiley & Sons.
Miller, J. E., & Seldin, P. (2014). Changing practices in faculty evaluation. Self, 58, 67-6.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A
framework for teacher knowledge. Teachers college record, 108(6), 1017.
Moore, M., Sheffield, A., Robinson, H., & Phillips, A. (2014). Designing a course for teachers:
Introduction to teaching online. Society for Information Technology & Teacher
PREPARING FACULTY FOR DISTANCE EDUCATION 96
Education International Conference (pp. 583-588). Association for the Advancement of
Computing in Education (AACE).
Oakes, W. P., Lane, K. L., Menzies, H. M., & Buckman, M. M. (2018). Instructional Feedback:
An Effective, Efficient, Low-Intensity Strategy to Support Student Success. Beyond
Behavior, 27(3), 168-174.
Osgood, D. (1986). The electronic university network. Byte, 11(3), 171-176.
Pajares, F. (2006). Self-efficacy theory. Retrieved from:
http://www.education.com/reference/article/self-efficacy-theory/.
Patton, M. Q. (2002). Two decades of developments in qualitative inquiry: A personal,
experiential perspective. Qualitative social work, 1(3), 261-283.
Piña, A. A., & Bohn, L. (2014). Assessing online faculty: More than student surveys and design
rubrics. Quarterly Review of Distance Education, 15(3), 25.
Poulin, R. (2013). Managing online education 2013: Practices in ensuring quality. WICHE
Cooperative for Educational Technologies.
Read, M. F., Morel, G. M., Butcher, T., Jensen, A. E., & Lang, J. M. (2019). Developing
TPACK Understanding Through Experiential Faculty Development. In Handbook of
Research on TPACK in the Digital Age (pp. 224-256). IGI Global.
Richardson, J. C., Koehler, A. A., Besser, E. D., Caskurlu, S., Lim, J., & Mueller, C. M. (2015).
Conceptualizing and investigating instructor presence in online learning
environments. The International Review of Research in Open and Distributed
Learning, 16(3).
Robinia, K. A., & Anderson, M. L. (2010). Online teaching efficacy of nurse faculty. Journal of
Professional Nursing, 26(3), 168-175.
PREPARING FACULTY FOR DISTANCE EDUCATION 97
Rodgers, D. (2018, January 19). The TPACK Framework Explained (With Classroom
Examples). Retrieved from https://www.schoology.com/blog/tpack-framework-explained
Rubin, H. J., & Rubin, I. S. (2012). Chapter 6: Conversational partnerships. In Qualitative
interviewing: The art of hearing data (3rd Ed.) (pp. 85-92). Thousand Oaks, CA: SAGE
Publications.
Rueda, R. (2011). The 3 Dimensions of Improving Student Performance: Finding the Right
Solutions to the Right Problems. Teachers College Press. 1234 Amsterdam Avenue, New
York, NY 10027.
Schein, E. H. (2004). Organizational culture and leadership. San Francisco: Jossey-Bass.
Schneider, B., Brief, A., & Guzzo, R. (1996). Creating a culture and climate for sustainable
organizational change. Organizational Dynamics, 24(4), 7–19.
Schneider, B., González-Romá, V., Ostroff, C., & West, M. A. (2017). Organizational climate
and culture: Reflections on the history of the constructs in the Journal of Applied
Psychology. Journal of Applied Psychology, 102(3), 468.
Schunk, D. H., & DiBenedetto, M. K. (2016). Self-efficacy theory in education. Handbook of
motivation at school, 2, 34-54.
Scott, W., Lemus, D., Knotts, G., & Oh, J. (2016). Why learner-centered new faculty orientations
matter: Organizational culture and faculty retention. The Journal of Faculty
Development, 30(1), 15-22.
Seaman, J. E., Allen, I. E., & Seaman, J. (2018). Grade Increase: Tracking Distance Education in
the United States. Babson Survey Research Group.
Sevier, R. A. (2003). Fine-tuning your mission: Your mission statement can put you right on top
of your market. Marketing University Business, 6(2), 292-295.
PREPARING FACULTY FOR DISTANCE EDUCATION 98
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational
researcher, 15(2), 4-14.
Stinehart, K. (1988). Increasing faculty involvement in distance teaching. Developing distance
education, 412-15.
Tierney, W. G. (1997). Organizational socialization in higher education. The Journal of Higher
Education, 68(1), 1-16.
Tompkins, K. A., Brecht, K., Tucker, B., Neander, L. L., & Swift, J. K. (2016). Who matters
most? The contribution of faculty, student-peers, and outside support in predicting
graduate student satisfaction. Training and Education in Professional Psychology, 10(2),
102.
Trolian, T. L., Jach, E. A., Hanson, J. M. & Pascarella, E. T. (2016). Influencing Academic
Motivation: The Effects of Student–Faculty Interaction. Journal of College Student
Development 57(7), 810-826.
Ulmer, L. W., Watson, L. W., & Derby, D. (2007). Perceptions of higher education faculty
members on the value of distance education. Quarterly Review of Distance
Education, 8(1).
Van der Kleij, F. M., Feskens, R. C., & Eggen, T. J. (2015). Effects of feedback in a computer-
based learning environment on students’ learning outcomes: A meta-analysis. Review of
educational research, 85(4), 475-511.
Varonis, E. M. (2014). Most courses are not born digital: An overview of the Quality Matters
peer review process for online course design. Campus-Wide Information Systems, 31(4),
217-229.
PREPARING FACULTY FOR DISTANCE EDUCATION 99
Velcoff, J., & Ferrari, J. R. (2006). Perceptions of university mission statement by senior
administrators: Relating to faculty engagement. Christian Higher Education, 5(4), 329-
339.
Watson, G. (2006). Technology professional development: Long-term effects on teacher self-
efficacy. Journal of Technology and Teacher Education, 14(1), 151-165.
Wentzel, K. R., & Miele, D. B. (2016). Expectancy-Value Theory. In Handbook of Motivation at
School (pp. 67-86). Routledge.
Wiesenmayer, R., Kupczynski, L., & Ice, P. (2008). The role of technical support and
pedagogical guidance provided to faculty in online programs: Considerations for higher
education administrators. Online Journal of Distance Learning Administration, 11(4).
Wigfield, A., & Eccles, J. S. (2000). Expectancy–value theory of achievement
motivation. Contemporary educational psychology, 25(1), 68-81.
Wilson, A. B., Miller, C. H., Klein, B. A., Taylor, M. A., Goodwin, M., Boyle, E. K., & Lazarus,
M. (2018). A meta-analysis of anatomy laboratory pedagogies. Clinical Anatomy, 31(1),
122-133.
Wilson, G., & Stacey, E. (2004). Online interaction impacts on learning: Teaching the teachers to
teach online. Australasian journal of educational technology, 20(1).
Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, H. S. (2005). What makes the difference? A practical
analysis of research on the effectiveness of distance education. Teachers College
Record, 107(8), 1836.
PREPARING FACULTY FOR DISTANCE EDUCATION 100
Appendix A: Participating Stakeholders with Sampling Criteria for Interview, Survey and
Observation
Participating Stakeholders
The stakeholder of focus for this study was the full-time faculty members of the
Specialized Medicine Department (SMD). The SMD full-time faculty consists of 10 faculty
members. The goal for these stakeholders is to become competent at utilizing the learning
management system (LMS) by January 2020. Due to the small size of the study population, the
research utilized a census sample in which all 10 faculty members are invited to participate in the
study. These full-time faculty members will be directly involved with the planning and
implementation of distance education courses if SMD is able to realize its goal of introducing
online courses or an online program within the department.
Interview Sampling Criteria and Rationale
Criterion 1. The faculty members selected must teach classes within SMD, as this
department is the department under examination by this research.
Criterion 2. The faculty members selected must be employed by SMD on a full-time
basis. These faculty members have the longest tenure within SMD and would be selected by the
dean to perform a variety of key roles in any distance education program that the department
might develop in the future, including content design, learning outcome development, and
program oversight.
Interview Sampling Strategy and Rationale
A census sampling strategy in which all members of the targeted population are sampled
was utilized for this research. A census sample is ideal for extremely small study populations
(Merriam & Tisdell, 2016), and the SMD employs just 10 full-time faculty members. Interviews
were selected as the data collection method as conducting interviews allowed the researcher to
PREPARING FACULTY FOR DISTANCE EDUCATION 101
collect data through open-ended questions, as well as follow up with additional questions to
probe for more details or clarifications (Creswell, 2014).
Review of Artifacts Attainment Strategy and Rationale
Artifact review was selected as the second data collection strategy for this research.
Artifacts are a strong source of data in qualitative research (Merriam & Tisdell, 2016). In this
research, the artifacts sampled consisted of the LMS course materials currently utilized by the
faculty members. Examining the faculty’s activity within the LMS provided concrete evidence
regarding the faculty’s experience with utilizing various LMS features. For example, the
researcher was able to examine the LMS in order to determine whether or not the faculty
members currently utilize the LMS gradebook for providing students with assignment feedback
or information on the students’ overall progress within the course. Utilizing artifact review in
this manner allowed the researcher an opportunity to see direct and unbiased evidence of the
faculty’s competence with utilizing the LMS features. Because the artifacts in question were
freely available to the researcher via the LMS, some limitations that are traditionally associated
with artifacts did not present a challenge, as the artifacts were complete and easily accessible
(Creswell, 2014).
PREPARING FACULTY FOR DISTANCE EDUCATION 102
Appendix B: Interview Protocol
This research utilized a semi-structured open-ended interview guide on the topic of
SUM’s LMS and general distance education concepts. This guide allowed the researcher to ask
each interviewee a consistent set of questions (Patton, 2002) while also giving the researcher
flexibility to explore and probe as needed (Merriam & Tisdell, 2016). A copy of the interview
guide is included below.
Interview Guide
Interviews took place in the faculty member’s office or another private location of the
faculty member’s choosing located on the campus of the organization being studied. An
informed consent information form was provided for the participant and an audio recording
device was placed on the table.
“Welcome and thank you again for agreeing to participate in this interview. I would like
to remind you that your participation in this interview is voluntary; you may withdraw from the
interview at any time and you may decide not to answer any question that you don’t wish to
answer. The information you share in this interview will be confidential and we will use a
pseudonym during the interview instead of your real name. With your permission, the interview
will be recorded with this audio recorder which I will turn on when we are ready to begin.
Please take a moment to review the information form which is identical to the form I e-mailed to
you previously. Do you have any questions for me that you’d like to ask before we begin?”
Give the participant time to review the form. Establish the pseudonym that will be used
during the interview. Once the participant is ready to begin, confirm permission to record, turn
on the audio recorder, and ask the interview questions in the following order.
Interview Questions
PREPARING FACULTY FOR DISTANCE EDUCATION 103
1. How long have you been working as a faculty member at the institution? (D)
(get specific year of hire)
2. Can you describe the Moodle training that you received when you were initially hired?
(O-T)
3. Have you received any additional Moodle training since then? If so, please describe the
training you received. (O-T)
4. How confident do you feel in your ability to use Moodle today? Why do you feel this
way? (M-SE)
5. How confident would you feel in your ability to learn new tasks in Moodle if you
received additional training? Why do you feel this way? (M-SE)
6. I’d like to ask you some questions about your experience with using some specific
features in Moodle. Have you ever used the Moodle gradebook to give students feedback
on their assignments in your courses? Why or why not? (K-TK)
7. Have you ever used Moodle to administer a quiz or exam? If so, can you please describe
how you utilized it? (K-TK)
8. What benefits, if any, do you think students might derive from receiving their grades in
Moodle? (K-TPK)
9. As you have probably heard, we’ve recently integrated the Kaltura video platform with
Moodle. Can you please describe what kind of training you have received on how to use
Kaltura? (O-T)
10. Have you used Kaltura to produce videos for any of your classes? If so, please describe
how you have utilized these videos. (K-TK)
11. Are you familiar with Kaltura’s ability to include quizzes in the middle of a video? If
you have used this feature, please describe how. (K-TK)
12. What value, if any, do you think there would be in adding a quiz question to a video? (K-
TPK)
13. Have you ever used a discussion forum in Moodle? If so, please describe how you have
utilized this feature in your classes. (K-TK)
14. What benefit, if any, do you believe students might derive from participating in a
discussion forum? (K-TPK)
15. Imagine that, hypothetically, you were teaching an online version of a class that you
currently teach on campus. How do you think you might use all of these tools in Moodle
PREPARING FACULTY FOR DISTANCE EDUCATION 104
to adapt your course content for online students? (K-TCK)
16. What elements of your courses do you think would be most challenging to adapt to an
online environment? Why do you feel this way? (K-TCK)
17. Do you think that it would be a good idea for your department to offer online classes or
degree programs? Why do you feel this way? (M-V)
18. If, hypothetically, your department did offer online classes or degree programs, would
you be willing to volunteer as an online instructor? Why do you feel this way? (M-V)
19. I’d like to wrap up by asking you a few questions about how faculty performance is
evaluated in your department. If, hypothetically, I were teaching as a faculty member in
your department, describe how my overall performance would be evaluated? (O-P)
20. How would my use of Moodle would be evaluated? (O-P)
21. Here’s my final question for you. If your department started to offer online classes, and
you did volunteer to teach in these online classes, what kind of tools, resources, or
support do you think you would need to be successful that the institution does not
currently provide? (O-T)
Turn off the audio recorder after the interviewee concludes their response to the final
question.
“Thank you so much for your time today. Please remember that everything you’ve
shared with me today is strictly confidential and that your anonymity will be protected.
Everything you’ve shared will be extremely helpful for my research and I really appreciate
your participation. Do you have any questions for me before we conclude? If you do have
any questions that come up later, feel free to contact me. Once I have concluded all of my
interviews and written an initial draft of my findings, I will contact you to ask you for your
feedback on my conclusions.”
PREPARING FACULTY FOR DISTANCE EDUCATION 105
Appendix C: Artifact Review Protocol
The artifacts selected for review included the Moodle course materials from each course
within the current term and the previous three terms that were taught by each of the interviewed
faculty members. The researcher examined each course in Moodle and recorded the following
data:
1. A pseudonym of the course code (e.g., SMD101)
2. Whether or not the course included any of the following elements:
a. Copy of the course syllabus
b. Informational files of any kind such as handouts, rubrics, or assignment
instructions
c. Hyperlinks to any resources on external websites (e.g., YouTube videos)
d. Individual assignment grades provided for students in the Moodle gradebook
e. Final course grades provided for students in the Moodle gradebook
f. Online assignment submissions
g. If assignment submissions are present, whether or not written feedback on the
assignments was provided directly in Moodle
h. Moodle quizzes
i. Kaltura videos
j. If any Kaltura videos are present, whether or not the videos included any in-video
quizzes
k. Discussion forums for course announcements
l. Discussion forums specifically intended for discussion assignments or other forms
of student-to-student interaction
PREPARING FACULTY FOR DISTANCE EDUCATION 106
Appendix D: Credibility and Trustworthiness
Credibility is an important consideration in this research. Due to the researcher’s position
at the institution being studied, the researcher has prior relationships with the study participants,
which could bias the researcher’s interpretation of the interviews. Additionally, due to the
researcher’s role as a provider of technology training at the institution, the researcher may have
been tempted to misrepresent the efficacy or competency of the faculty members who have
received training at the institution. Despite this role, the researcher does not supervise any of the
institution’s faculty members. The researcher utilized three key strategies to support the
credibility of the research conclusions.
The first strategy that was used to enhance credibility is the production of “rich” data
from the interview process. Maxwell (2013) explains that verbatim transcripts of interviews are
more credible than notes taken by the researcher. The use of audio recordings in the interview
process ensured that verbatim transcripts were produced. Additionally, the interview audio
recordings were transcribed by a professional third-party service, eliminating the possibility that
the researcher might misrepresent something in the transcription. In order to protect the
anonymity of the participants, the third-party transcriber was required to sign a non-disclosure
agreement with regard to the interview details. Pseudonyms for the interview participants were
also used during the interviews, preventing the transcriber from knowing their actual names.
Finally, the files were encrypted when they were electronically transmitted to the transcriber,
minimizing the possibility that an unauthorized party could decode them if they were intercepted.
The second strategy was the use of data triangulation. The use of multiple data sources
can make qualitative research more credible (Maxwell, 2013). Instead of relying on interviews
alone, the use of artifact analysis as described above allowed the researcher to assess the
PREPARING FACULTY FOR DISTANCE EDUCATION 107
knowledge of faculty members through analysis of LMS artifacts. The aggregated artifact
analysis were then compared to the interview findings.
Finally, the third strategy utilized was respondent validation. Soliciting feedback from
study participants about the research findings and conclusions can help verify that the data was
not misinterpreted or that the participants’ views were not misrepresented in the findings
(Maxwell, 2013). After the research was concluded and the findings were summarized, the
researcher sought input from the participants to see whether or not they generally agreed on the
conclusions generated by the researcher.
PREPARING FACULTY FOR DISTANCE EDUCATION 108
Appendix E: Ethics
The researcher did not conduct any research until approval was received from the
Institutional Review Board (IRB) at the University of Southern California (USC). The research
was also reviewed by the IRB at the research site, the Small University of Medicine (SUM), a
pseudonym. IRB review is an essential process that helps to ensure that human participants are
treated ethically in the research process (Merriam & Tisdell, 2016). Specifically, the research
must do no harm to the participants, they must give their fully informed consent to participate in
the research, they must not be coerced to participate, and their right to privacy must be respected.
Once the research was approved by the IRB at USC, a formal recruitment e-mail was sent
to the online teaching faculty who were included in the targeted stakeholder group. This e-mail
provided information about the purpose of the research, the criteria for participation, and the
anticipated length of the interviews or surveys. Once a participant confirmed their interest, the
researcher also provided them with a more detailed description of the project, a list of the
interview questions, and an opportunity to decline participation after reviewing the questions.
Allowing the participants to review the interview questions prior to the interview helped to
ensure that they had a high level of comfort with the topic and that their participation was fully
informed and voluntary (Krueger & Casey, 2009).
The researcher also provided the research participants with an informational form at the
beginning of any surveys or interviews, in order to ensure as thoroughly as possible that the
participants recognized their involvement as voluntary, confidential, and low risk (Glesne, 2011).
The participants were allowed to retract their participation at any point before or during the
surveys or interviews and the researcher did not express any disappointment at their lack of
participation or pressure them to participate in any way at any point in the process. The
PREPARING FACULTY FOR DISTANCE EDUCATION 109
researcher informed participants that no incentive would be provided for their participation, as
well as informed them that no punishment or disincentive would be administered if they chose
not to participate. These assurances were important in order to prevent the participants from
feeling coerced to participate in the research.
There was some potential for bias in this research due to the researcher’s position within
the institution being studied. The research is related to the faculty’s ability to utilize the learning
management system (LMS), and the researcher is also responsible for providing training
opportunities on this platform for all faculty members at the institution. The researcher is new to
this position at the institution and most of the training that faculty members have received was
not developed or coordinated by the researcher. The researcher therefore does not feel that the
readiness of the faculty members is a reflection of the researcher’s effectiveness as a training
provider. Nevertheless, due to the nature of the researcher’s position, the potential for any
possible bias to influence the research was minimized through careful selection of the target
population; this research was originally intended to target the online teaching faculty in SUM’s
upcoming online degree program, but it was determined that the potential for bias or other
conflicts of interest would be too high, as the researcher is currently working very closely with
those faculty members on a day-to-day basis in order to prepare the degree program for launch.
The SMD was selected as an alternative target population for this research because SMD is not
actively developing any online program at this time, which means that the researcher has
minimal day-to-day contact with SMD faculty members.
The researcher further diminished the potential for bias through the development of a
thorough audit trail in which a detailed description of the study procedures is provided, as well as
a description of how the results were derived from the data (Merriam & Tisdell, 2016). An audit
PREPARING FACULTY FOR DISTANCE EDUCATION 110
trail helps to ensure that the researcher is unable to modify or misrepresent the results of the
research.
The researcher also recorded the participant interviews, and the existence of these
interview recordings further strengthened the research audit trail and helped to ensure that the
researcher did not misrepresent any of the interview findings. The researcher asked participants
during the interviews to clarify the meaning of any potentially ambiguous statements, in order to
help ensure that their statements were not unintentionally misinterpreted due to any inherent bias.
The participants were only recorded with their fully informed consent, and the recorded
interviews were stored on an encrypted and password-protected digital storage device in order to
ensure privacy. Because the recording device used was unobtrusive, the participants were
occasionally reminded during the interview that they are being recorded; this was important in
order to ensure that the participants were not unintentionally deceived by forgetting that the
recording device is turned on (Rubin & Rubin, 2012).
PREPARING FACULTY FOR DISTANCE EDUCATION 111
Appendix F: Immediate Feedback After Training Sessions
To be administered immediately after each of the four training sessions is complete.
Scale 1 - 10 (strongly disagree to strongly agree)
1. The training session held my interest. (Level 1 - Engagement)
2. The training session was relevant to the tasks that I will be performing as an instructor of
a distance education course. (Level 1 - Relevance)
3. I am satisfied with the training session. (Level 1 - Customer Satisfaction)
4. I would recommend this training to another faculty member. (Level 1 - Customer
Satisfaction)
5. I am confident in my ability to be an effective distance education instructor. (Level 2 -
Confidence)
6. I am confident in my ability to use the Moodle gradebook to deliver grades and
assignment feedback to my students. (Level 2 - Confidence)
7. I am confident in my ability to create instructional content and activities in Moodle that
will be engaging for distance education students. (Level 2 - Confidence)
Open-Ended Question
1. How will you use what you learned during today’s session to create distance education
activities that will be engaging for distance education students? (Level 2 - Procedural)
2. Please add any comments on the relevance or effectiveness of this training. How can we
improve this training in the future? (Level 1 - Relevance, Customer Satisfaction)
PREPARING FACULTY FOR DISTANCE EDUCATION 112
Appendix G: Feedback Survey Two Weeks after Training Program
To be administered within two weeks of the fourth training session
Scale 1 - 10 (strongly disagree to strongly agree)
1. The information provided during the training program was valuable for me as a future
instructor of distance education courses (Level 1)
2. I have been able to apply the knowledge I learned during the training program when
creating lesson plans for distance education courses (Level 2)
3. I have used information from the training program in order to better utilize Moodle in my
existing courses, such as by utilizing the Moodle gradebook to provide feedback for my
students (Level 3)
4. After completing the training program, I am interested in volunteering to teach distance
education courses at SUM (Level 4)
PREPARING FACULTY FOR DISTANCE EDUCATION 113
Appendix H: Sample Figure for Level 4 Goal Reporting
Figure 4. Example of hypothetical data for Level 4 goal reporting
PREPARING FACULTY FOR DISTANCE EDUCATION 114
Appendix I: Sample Figures for Level 3 Goal Reporting
Figure 5. Example of hypothetical data for Level 3 goal reporting.
Figure 6. Example of hypothetical data for Level 3 goal reporting.
PREPARING FACULTY FOR DISTANCE EDUCATION 115
Figure 7. Example of hypothetical data for Level 3 goal reporting.
Figure 8. Example of hypothetical data for Level 3 goal reporting.
Abstract (if available)
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Full-time distance education faculty perspectives on web accessibility in online instructional content in a California community college context: an evaluation study
PDF
Inclusion of adjunct faculty in the community college culture
PDF
Affluent teens and school stress: an evaluation study
PDF
Social work faculty practices in writing instruction: an exploratory study
PDF
Satisfactory academic progress for doctoral students: an improvement study
PDF
Graduation rates in college of students with disabilities: an innovation study
PDF
Line staff and their influence on youth in expanded learning programs: an evaluation model
PDF
Dual-enrollment program implementation to address the problem of college affordability as a barrier to student access and a contributing factor toward student debt
PDF
The impact of faculty interactions on online student sense of belonging: an evaluation study
PDF
Sustaining faith-based organizations through leadership pipelines and programs: an evaluation study
PDF
Student engagement in online education: an evaluation study
PDF
Job placement outcomes for graduates of a southwestern university school of business: an evaluation study
PDF
Scaling online graduate programs: an exploratory study of insourcing versus outsourcing
PDF
Supporting faculty in preparing entrepreneurship: an exploration in the context of active learning
PDF
An examination of factors that affect online higher education faculty preparedness
PDF
Developmental education pathway success: a study on the intersection of adjunct faculty and teaching metacognition
PDF
Mandatory reporting of sexual violence by faculty and staff at Hometown University: an evaluation study
PDF
Analyzing the implementation of a learning management system into a post-merger & acquisition organization and its effects on sales performance (improvement model)
PDF
Measuring the impact of short-term campus exchange programs: an evaluation study
PDF
Effective practices for managing staff performance in higher education: an exploratory study
Asset Metadata
Creator
Fowler, Justin Lee
(author)
Core Title
Preparing university faculty for distance education: an evaluation study
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Organizational Change and Leadership (On Line)
Publication Date
01/10/2020
Defense Date
10/24/2019
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
Distance education,e-learning,faculty,Higher education,Internet,learning management system,Moodle,OAI-PMH Harvest,online,Technology,Training
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Stowe, Kathy (
committee chair
), Datta, Monique (
committee member
), Seli, Helena (
committee member
)
Creator Email
fowlerpdx@gmail.com,justinlf@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-261854
Unique identifier
UC11674240
Identifier
etd-FowlerJust-8116.pdf (filename),usctheses-c89-261854 (legacy record id)
Legacy Identifier
etd-FowlerJust-8116.pdf
Dmrecord
261854
Document Type
Dissertation
Rights
Fowler, Justin Lee
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
e-learning
faculty
Internet
learning management system
Moodle
online
Training