Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
The effect of guidance on learning in a web-based, media-enriched learning environment
(USC Thesis Other)
The effect of guidance on learning in a web-based, media-enriched learning environment
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
THE EFFECT OF GUIDANCE ON LEARNING IN A WEB-BASED,
MEDIA-ENRICHED LEARNING ENVIRONMENT
by
Sean Francis Early
A Dissertation Presented to the
FACULTY OF THE GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(EDUCATION)
December 2008
Copyright 2008 Sean Francis Early
ii
ACKNOWLEDGMENTS
Although by their nature dissertations are single-author documents, the work
that they represent would not be possible without the support of many. In a
dissertation project examining the role of instructional supports on learning, I would
be remiss were I not to briefly acknowledge the guidance and support that has made
my work possible.
First, and closest to home, I would like to express my love and gratitude for my
partner in all things, Scott Lewis. Without his encouragement, belief in my ability and
love, this journey would never have begun and most certainly would not have ended in
success. Scott, more than anyone else I know, will also understand my desire to thank
the ladies of our house for their companionship and love. Pearblossom, Lola and even
Miss Dolores, thank you for reminding me that stacks of paper are sometimes meant to
fall over, that computer printers are also sources of warming comfort and not just work
and that short naps are occasionally just what is needed.
Second, but no less crucially, I owe many thanks to my program advisor, Dr.
Richard Clark. Dick has been an unwavering source of intellectual, academic and
emotional support. He has been more than just an advisor; he has been a mentor, a
colleague and most importantly a friend. In conversations with students from other
programs and other universities, I have heard my share of apocryphal stories of
advisors who gave more criticism than support and gave more commands than
opportunities. I am very happy to report that Dick has been the opposite of the
stereotypical ivory-tower advisor. There was never a procedural hoop that needed
iii
jumping through that he didn’t have a way around, never an obstacle to my progress
and growth that he didn’t somehow diminish or eliminate. Those who know me also
know that I am not given to flowery praise, but I happily tell all that Dick has been the
perfect advisor for me.
Dr. Adrianna Kezar and Dr. John J. McArdle, the remaining members of my
dissertation committee are also owed many thanks. Adrianna and Jack have been
flexible, responsive, and generous with their time and advice. Few committees, to the
best of my knowledge, feature a scholar known for advanced quantitative techniques
as well one known for her exceptional work in phenomenology. The fact that they
represent two approaches to understanding that some would place in opposition to
each other never gave me a moment’s pause. Having the diversity of their perspectives
has been a true delight; one that I believe has improved the dissertation itself
whenever I took their advice. Dr.’s McArdle, Kezar and Clark represent the best that
universities have to offer to the communities in which they are found. They are
rigorous in their practice, creative in their thinking, and open to the possibility of the
new. I draw great inspiration from their example and feel honored to receive their
support.
Finally, I owe a debt of deepest thanks to the group of remarkable colleagues I
know as the cohortians. Lindsey, Margaret, Hyo, Jarrett and Vicki are my academic
touchstone, my support group in times of need and my friends in times of success.
iv
TABLE OF CONTENTS
Page
Acknowledgements ii
List of Tables vii
List of Figures viii
Abstract ix
Chapter I: Introduction
Purpose of the Study
The Problem
Significance
Limitations
Hypotheses
1
1
1
4
6
6
Chapter II: Review of the Literature
Cognitive Architecture
Information Processing Theory
Models of Working Memory
Modular Cognitive Architecture
Instructional Design
Guidance During Instruction
Online Learning
Leadership Theory
Self-Efficacy
Hypothesis 1
Hypothesis 2
Hypothesis 3
10
14
18
22
30
38
46
71
79
82
84
85
85
Chapter III: Methodology
Participants
Design
Apparatus
Variables
Procedure
86
86
86
87
91
94
Chapter IV: Results
Descriptive Statistics
Tests of Between-Group Differences:
Hypothesis 1a
96
96
98
98
v
Hypothesis 1b
Regression Analysis:
Hypothesis 1
Hypothesis 2
Hypothesis 3
Self-Efficacy Scale
99
102
105
105
111
111
Chapter V: Discussion
Discussion of Research Hypotheses
Hypothesis 1a
Hypothesis 1b
Hypothesis 2a
Hypothesis 2b
Hypothesis 2c
Hypothesis 2d
Hypothesis 2e
Hypothesis 3a
Discussion of Implications
Theoretical
Practical
Future Research
Limitations
Conclusions
113
114
114
115
117
118
118
118
118
118
119
119
121
122
125
129
References
132
Appendix A:
Diagram of ACT-R
147
147
Appendix B:
Participant recruitment flyer
148
148
Appendix C:
Summative assessment
149
149
Appendix D:
Self-efficacy scale
154
154
Appendix E:
Informed consent document
155
155
Appendix F:
Questions from online course
157
157
vi
Appendix G:
Regression equation scatterplot figures
160
160
Appendix H:
Pearson’s Correlation Matrix
165
165
vii
LIST OF TABLES
Table 1: Demographic Information 86
Table 2: Summary of Demographic Variables and Codes 92
Table 3: Summary of Learning Outcome Variables 94
Table 4: Summary of Efficacy Belief Variable 94
Table 5: Summary Statistics 98
Table 6: Summary Table of t-tests 102
viii
LIST OF FIGURES
Figure 1: Model of human information processing system 18
Figure 2: Taxonomy of leadership styles and prescriptive matches of SLT 80
Figure 3: Diagram of research design 87
Figure G-1: Total score by educational level 160
Figure G-2: Total score by condition 160
Figure G-3: Verbal recognition by condition 161
Figure G-4: Verbal recognition by educational level 161
Figure G-5: Verbal recall by condition 162
Figure G-6: Verbal recall by educational level 162
Figure G-7: Classification of leadership styles by condition 163
Figure G-8: Classification of leadership styles by education level 163
Figure G-9: Classification of developmental level by condition 164
Figure G-10: Classification of developmental level by education level 164
ix
ABSTRACT
The appropriate role for explicit, directive guidance during instruction has been
debated in the literature for several decades (Sweller, Kirschner, & Clark, 2006;
Mayer, 2004; Savery & Duffy, 2001). The two dominant positions are that explicit
guidance should for the core of procedural knowledge instruction for novices (Sweller,
Kirschner & Clark, 2006) and conversely that explicit guidance should be avoided
during procedural knowledge instruction so that students can construct their own
pathway to success (Duffy & Jonassen, 1992). This project examined the role of
guidance, in the form of corrective feedback, on web-based learning with high quality
media. This study used a randomized experimental model to assess differences
between three learning conditions: (a) text-only, (b) text plus video and practice, and
(c) text, video, practice and feedback. Hypotheses predicted that increased practice
and guidance would improve performance for procedural knowledge but would have
no significant effect on declarative knowledge, in keeping with Anderson’s model of
cognitive processing (Anderson, 1996). Performance on a summative test of
declarative and procedural knowledge was analyzed for 60 participants. The central
hypothesis of the study was supported. Participants in the condition with the most
practice and guidance (in the form of corrective feedback) performed significantly
better than participants on the other treatment conditions. No significant differences
for declarative (e.g. facts, concepts) knowledge were found, in support of research
hypotheses. Ordinary Least Squares regression analysis demonstrated that treatment
condition and educational level were the significant predictors of performance.
x
Implications discussed in this study include the use of corrective feedback for
procedural knowledge training, the use of high-quality video demonstrations in online
learning environments, and the effectiveness of online content delivery platforms for
completing randomized experimental studies of learning.
1
CHAPTER I: INTRODUCTION
This chapter will provide a description of the purpose of the study, clarification
of the problem and its significance and outline the research objectives and limitations.
The chapter concludes with the main research questions.
Purpose of the study
The purpose of the study is to examine the effect of guidance on performance
of a procedure related to leadership skills in an online learning environment. Using an
experimental design, the study compares three treatment conditions with differing
levels of instructional guidance to determine which is most effective at increasing
performance on a procedural task. The study also explores the differences between
scores of participants on assessments of fact-based (knowing what) and procedural
(knowing how) knowledge to determine if these differences can be attributed to
participation in specific treatment conditions, differences of educational level, gender,
chronological age, and/or prior leadership training experiences.
The problem
This study examines the effect of guidance on performance in an online
learning environment. Additional factors such as education level and prior leadership
training are included in the analysis to account for the effects of potentially
confounding variables. The central problem, however, addresses the use of guidance
during instructional activities. In this project, guidance takes the form of response-
contingent, corrective, strategy-focused feedback during practice exercises. By
examining a number of subscales of performance that separate out participants’
2
performance according to knowledge type, a deeper understanding of when guidance
can be used most effectively is developed. Finally, the relationship between guidance,
assessment performance and the participants’ belief in their own ability to apply the
knowledge they have gained is examined.
To date, studies in this area generally manipulate many variables
simultaneously, complicating interpretation of results. For example, a typical study
might assess the effectiveness of a novel instructional practice used in concert with a
new curriculum by comparing learner results in the experimental group with a group
who received neither the instructional intervention nor the novel curriculum (Mayer,
2004). The generally cross-sectional data collection techniques favored in this area
make it difficult if not impossible to ascertain the unique effects of each element of the
new instructional practice from those of the new curriculum. This practice of
manipulating multiple variables has been defended as a more authentic form of
assessing educational practices but often leads to statistically significant differences
between groups that have no utility for answering questions about instructional
efficiency or quality (Feuer, Towne, & Shavelson, 2002).
A second issue in this area concerns epistemological differences within the
research community. Scholars of instructional design who adopt a constructivist
worldview often reject elements of traditional pedagogy and experimentation
(Burbules, 2000) making side-by-side comparison between instructional systems
difficult and developing a clear understanding of design-informed practice
functionally impossible. Studies from this epistemological perspective generally take
3
one of two forms – classroom based case studies and quasi-experiments. Setting aside
specific methodological issues, the overarching difficulties in interpreting these
studies are the lack of a well-matched reference or control group and lack of control of
instructional delivery. This camp within the conversation about guidance generally
advocates a generalized minimal-guidance position (Cobb, 1994; Palincsar, 1998).
Because the methodologies used do not support drawing causal inferences, the
constructivist push for generalization of a minimal-guidance approach finds itself on
insubstantial footing.
Within a more traditional post-positivist experimental tradition, the evidence
and advocacy is generally for increased guidance during instruction (Ayres, 2006a;
Carlson, Lundy, & Schneider, 1996; Carroll, 1994; Gagné & Dick, 1983; Kluger &
Dinisi, 1996; Merrill, 2002). One of the subtleties of this position that is addressed in
this project is that guidance is often used or implemented in ways that are inconsistent
with the evidence on effective guidance. Another is that guidance during instruction is
sometimes interpreted as a panacea for all instructional deficits, an issue compounded
by the historical lack of focus on the importance of content and context of guidance on
outcomes.
Quite apart from these differences of worldview, practice, and interpretation is
a debate about the role of media in learning that is also addressed in this study (Clark,
1994; Mayer, 2001). In spite of decades of research demonstrating that the delivery
media have little or no lasting effect on performance, scholars continue to assert a
hypothetical, unique role for the importance of multimedia for improving learning.
4
This study seeks to address problems stemming from both sides of the
guidance as well as media debates by allowing for a minimally guided but content-
identical control condition and two treatment conditions with varying forms of
guidance. This will allow the researcher to tease out the difference, if any, between the
effect of media on learning and the effect of media-based practice with guidance.
Further, by disaggregating the full assessment measure into subscales of knowledge
type and content type, differences along these axes between treatment conditions and
between-individual differences will be illuminated.
Significance
This study addresses a central question in the field of instructional design, the
relationship between guidance in the form of corrective feedback and performance of
a procedure. There are three categories of justification for this project, the first relating
to questions of instructional design, the second to questions of media-based learning
and the third to the learning of leadership skills. Each of these categories is explored in
turn in the section that follows.
Instructional design and guidance
As noted above, an issue that continues to plague the field of instructional
design is that of guidance. By completing an experiment that explores the relationship
between the most widely acknowledged form of effective guidance and learning
outcomes, this study contributes meaningfully to this scholarly discussion. Insofar as
this project provides comparison conditions that are reasonable and meaningful
5
control conditions to the experimental condition, it also lends insights to the subtleties
of the interaction between guidance, instructional content, and instructional context.
Media-based learning
This study, as will be described in greater detail in Chapter 3, uses a web-based
instructional platform and two of the three conditions feature studio-quality video
vignettes developed by professional screenwriters, featuring professional actors and
based in authentic military mission contexts in which leadership decisions were
critical to mission outcomes. As such, this study of guidance also serves as an
opportunity to compare three levels of media-based content delivery, the first with a
graphical display of text, the second with the same text and the addition of video and
practice, and the third with text, video, and guided practice. Were there a unique
contribution of the media itself, it could be expected to operate equivalently across at
least two of the three conditions. The assessment data for the online instruction is
parsed by knowledge and task type and can therefore lend valuable insights to the
ongoing debate over the influence of media on learning
Leadership skills
Finally, this study may contribute meaningfully to the literature on leadership
training. The course content with respect to leadership skills contains factual
information about leader and subordinate behavior classifications as well as process
knowledge about how to identify subordinate readiness accurately and match these
needs with the appropriate leader behaviors. By including both the knowing-what
information and the knowing-how information in both the course content and the
6
assessment, this study may yield valuable information about the connection between
these two forms of knowledge with respect to leadership. Further, the study may yield
information about the use of media-based examples as well as the role of guidance for
the learning of leadership tasks such as matching leader and subordinate behaviors.
Limitations
This study examines several broad areas of interest in the instructional
literature, but does so in a way that is limited by its design. The implications of this
study are generally limited to application in web-based learning environments. The
relatively small sample size, in spite of random assignment, may also limit the
application of the findings in light of the potential for future samples to respond to the
treatment in different ways. Third, the reliance on one theory of leadership dynamics
may have limited application to contexts in which other theories of leader-subordinate
interaction are dominant. Finally, when examining the relationships between predictor
and outcome variables, there is always the risk of specification error in the model –
omitting variables that may have important effects on the outcome or including
variables that in fact only serve as mediators for other, unmeasured predictors.
Hypotheses
The set of research questions for this project addresses two broad areas of
interest: the possible value-added effects of high-quality media demonstrations during
instruction, and the possible value-added effects of corrective feedback and guidance
during instruction. The following questions and hypotheses relating to these topics
were developed:
7
1. Does the use of high-quality video vignettes during instruction improve
performance? Research hypotheses:
a. Performance scores for media-use groups will be significantly greater
than those for non-media-use group.
i. Performance scores for declarative knowledge subscales for
media-use groups will NOT be significantly greater than those
for non-media-use groups.
ii. Performance scores for procedural knowledge subscale for
media-use group will be significantly greater than that of the
non-media-use group.
iii. Total test performance scores for the media-use groups will be
significantly greater than that of the non-media-use group.
iv. Efficacy scale scores for the media-use groups will be
significantly greater than that of the non-media-use group.
2. Does the provision of guidance, in the form of structured practice and
corrective feedback, affect performance on assessments of declarative and
procedural knowledge? Research hypotheses:
a. Total test performance for groups receiving opportunities for structured
practice will be significantly greater than for the group that does not
receive practice opportunities.
8
b. Declarative knowledge subscale performance for groups receiving
opportunities for structured practice will be significantly greater than
for the group that does not receive practice opportunities.
c. Procedural knowledge subscale performance for groups receiving
opportunities for structured practice will be significantly greater than
for the group that does not receive practice opportunities.
d. Efficacy scale results for the group receiving opportunities for
structured practice will be significantly greater than for groups not
receiving corrective feedback.
e. Total test performance for the group receiving corrective feedback will
be significantly greater than for groups not receiving corrective
feedback.
f. Declarative knowledge subscale performance for the group receiving
corrective feedback will be significantly greater than for groups not
receiving corrective feedback.
g. Procedural knowledge subscale performance for the group receiving
corrective feedback will be significantly greater than for groups not
receiving corrective feedback.
h. Efficacy scale results for the group receiving corrective feedback will
be significantly greater than for groups not receiving corrective
feedback.
9
Summary
This chapter has presented a brief introduction to the study, including a
statement of the purpose of the project, the central research problem and its
significance, and the research hypotheses. This discussion next turns to a review of the
relevant literature. The literature review is followed by a description of the
methodology used in the study, which in turn is followed by a chapter describing the
results. The dissertation concludes with a chapter discussing the results of the study,
noting relevant limitations to this work, and exploring the implications of this work for
both scholars and educational practitioners.
10
CHAPTER II: REVIEW OF THE LITERATURE
Introduction
At one time or another, for one reason or another, everyone has felt the need for
help. This need-for-help state has been described and examined throughout the history
of psychology, and particularly the psychology of learning. The fathers of educational
psychology, James (1842-1910) and Dewey (1859-1952) studied this phenomenon and
identified its importance in the pathway of learning. Vygotsky (1992) and Piaget
(1969) deepened our understanding of what has become known as the teachable
moment (Stanford & Roark, 1972). This state, called the zone of proximal
development by Vygotsky and an assimilation process by Piaget, represents an
underlying condition of cognitive readiness to store information for future retrieval. In
short, it represents a readiness to learn. Regardless of the labels applied after the fact,
the process of learning is one in which new information is connected to old
information (i.e. memories of many types) in a way that enables the individual to
quickly and effectively access the new information when it becomes needed. On this
matter there is widespread agreement, bolstered by recent developments in the
cognitive sciences that have enabled a more accurate picture of learning than was
available in the past (cf. Thompson & Madigan, 2005). The scholarly community has
attained some measure of agreement about how people learn but a maelstrom of
disagreement about how best to teach the learner continues to rage.
The recent debate centers on the subject of instructional guidance. Should we
“teach” learners as the cognitive sciences would suggest, or should we instead
11
facilitate their progress toward mastery of curriculum content by providing a context
and allowing them to “make sense” of the material without imposing structural
constraints, as is suggested by those who adopt a more classically humanistic
perspective on the matter? Strong claims are made on both sides of the argument, with
the discussion particularly vigorous in the area of computer-based instruction. On the
one hand researchers advocating a strong-guidance paradigm (Clark, 1994; Mayer,
2004; Kirschner, Sweller, & Clark, 2006; Merrill, 2002; Van Merriënboer, Clark, & de
Croock, 2002) claim that a minimally-guided or ‘discovery’ learning approach is
inconsistent with human cognitive architecture, has no support in the peer-reviewed
research literature, and at best represents a negligent inefficiency in the application of
instructional resources both human and economic. On the other hand, researchers
advocating minimal guidance in instruction (discovery, constructivist, problem-based,
etc.) claim that guided approaches have an insubstantial philosophical foundation
(Jonassen, 1994), ask the wrong question of instructional-design studies (Kozma,
1994a, 1994b; Duffy & Kirkley, 2004), lead to learning outcomes that are rigid,
context-bound or resistant to farther transfer (Duffy & Jonassen 1992) and provide a
false sense of classroom practices related to minimally-guided instruction (Levine &
Resnick, 1993).
One of the dilemmas faced by witnesses to this debate is the difficulty in finding
research studies that conform to the expectations of both sets of advocates, as the
epistemologies of the groups are inherently inconsistent. Many studies in the
constructivist, minimal guidance camp (Kozma, 1994a; Dochy, Segers, Van den
12
Bossche, & Gijbels, 2003) make comparisons between a discovery or problem based
model with robust materials and instructional expertise/tutoring against a control
condition in which the students receives a “natural” control condition that provides no
explicit supports and potentially inferior instructional materials (Sweller, Clark,
Kirschner, 2007). Studies with poorly matched control conditions are predisposed to
finding between-group differences. Differences have been found that favor the
minimal guidance paradigm (Savery & Duffy, 2001), but one can reasonably wonder
whether these differences are the result of the inherent superiority of the treatment or
whether the study simply established that some instruction is most often better than no
instruction.
Many studies in the strong-guidance camp conform more closely to classical
experimental practices in the social sciences and may share some of classical
experimentations underlying weaknesses. Random assignment to condition provides
the best path to identifying causal effects (Feuer, Towne & Shavelson, 2002; Slavin,
2002) but may be less meaningful in light of cohort effects (e.g. college students,
participants who have the time and interest in completing such studies). Some
experimental procedures have also been interrogated on the grounds of inauthenticity
to real-world task demands (Burbules, 2000). Finally, and most frequently, advocates
of minimal guidance question the integrity of the experimental procedure itself;
suggesting that because there is no external, objective truth that lies outside our
perception awaiting discovery, any methodology that presumes to capture such extant
truth is, a priori, fatally flawed (Jonassen, 2000; Schwartz, 2008) In this way, each
13
side of the debate questions the findings of the contrasting position as well as its
epistemology, methodologies, and methods themselves. While this long-standing
debate is unlikely to be settled by any single piece of evidence, this study seeks to
address some of the issues that stand between the two positions as well as providing
contextualized information about the role guidance may play during instruction.
Understanding Instructional Theory and Research
Understanding instructional theory requires a review of the instructional
literature as well as a review of related literatures. Contemporary instructional theory
and the debates in play depend on evidence from the cognitive sciences – cognitive
architecture and cognitive load theory- as well as media research, which has
established many of the characteristics of media-based learning environments
important for learning. What follows is a review of these literatures, beginning with
the fundamentals of cognitive architecture and processing. Next, instructional design
is examined, both broadly and in some detail with respect to cognitive load theory and
the role of guidance. The third large section of the review contains a discussion of
online learning environments and relevant design principles. Finally, the review
concludes with brief descriptions of the leadership theory used in the project and the
nature of efficacy beliefs that are measured in the experiment. These bodies of
literature are examined in order to explore the relationship between learning and
various forms of instruction. This review concludes with a set of research hypotheses
developed to collect and analyze data relevant to the role of guidance on performance
of procedural tasks in a web-based learning environment.
14
Cognitive Architecture
This section presents an overview of cognitive architecture, the manner in
which the human brain is organized with respect to thinking. The review focuses on
those features of the process of thinking that are most relevant to the research project
at hand: working memory, long-term memory and cognitive load theory. Each of these
subtopics has been studied intensively in multiple bodies of literature ranging from the
relatively pure abstractions of the philosophy of the mind to the relatively concrete
measurement techniques used in the neurosciences. Researchers whose work bears
most directly on instructional design are favored in this discussion. Although one
might reasonably trace all of this thinking about thinking back to humanity’s most
ancient texts, this treatment begins with a brief portrait of the anatomy of the brain, as
it is understood today.
The human brain
The human brain is a complex organ housed in the bony structures of the
cranium, and is composed of many types of neural tissues and brain-specific fluids. It
is richly supplied with highly oxygenated blood via the carotid arteries. It is separated
into two hemispheres with deeply wrinkled or convoluted surfaces that are sheathed
by two primary layers of protective tissues. In addition to the two hemispheres, there
are a number of important structures that can be seen with the naked eye during
dissection. The cerebellum is a relatively small structure located underneath and
toward the rear of the brain and acts as a master coordinator of movement. The corpus
callosum is a dense neural structure that ties the brain hemispheres at their bases.
15
Superior to the corpus callosum are the thalamus and hypothalamus, structures that
coordinate the movement of information about the body to and from the cortex. The
auditory and visual nerves are visible to inspection as they insert into thalamic tissues
as well as in their transit from the sense organs. Several fluid-filled potential spaces
called ventricles are found on both sides of the brain and serve as storehouses for
important brain fluids including cerebrospinal fluid.
The visible, superficial surfaces of the brain are composed of grayish cells,
cortical neurons, that are densely packed and number in the billions. This so-called
gray matter varies in thickness within the brain, and between individuals but is always
composed of multiple layers of neurons that are connected to each other through the
most common form of neural connection, called a synapse. Where the central
projection from the cell nucleus, known as an axon, connects to another neuron it does
this through the partner cell’s dendrites (Koch, 2004). Each cell has both axon and
dendrites and cells are generally connected to many, many other neurons with which
they communicate. This communication occurs in the language of electro-chemistry,
with neural inhibitor and disinhibitor chemicals triggering an electrical potentiation
and discharge in the neuron which is tuned so as to convey the message in specific
directions and with specific intensities such that some messages are conveyed more
slowly and closely while others are sent very rapidly over relatively vast distances
(Koch, 2004). As similar messages are sent repeatedly through the same channels,
these pathways become capable of transmitting the information much more rapidly
through a complex process of dendritic protein deposition. At the most basic level
16
relevant to this discussion, when these protein channels are built, learning has
occurred (Ericcson, Chase, & Falloon, 1980; Feldman, 2006). More broadly, evidence
from fMRI studies of training in humans for visual tasks, such as memory and
inhibiting of irrelevant information (i.e. Stroop test) shows that regionally specific
activation in dorsolateral prefrontal cortex (DLPFC) first increases (Oleson,
Westerberg & Klingberg, 2004) and then decreases across time as performance
improves. Although fMRI studies do not allow the cell-level analysis that primate
studies offer, there is little reason to assume that the physiological mechanisms are
fundamentally different between higher primates and humans (Jonides, 2004).
Understanding the basics of neurophysiology improves the clarity of the picture of
learning being developed here.
A highly simplified model of the brain reveals the connection between
neuronal protein deposits and learning but fails to address the ways in which
information is perceived, processed and stored for future use. The neuronal network
does not simply function to communicate with itself; it receives and interprets
information, processes, plans and executes all possible human actions. Additional
tissues and structures now come into play. Single neuron to single neuron
communication forms the core of cognition but is relatively inefficient when
transmitting large amounts of information between diffuse brain structures and
between brain structures and the body (Feldman, 2006). Evolution has favored a
special form of tissue known as white matter for this function. Two pertinent examples
are the auditory and visual nerves. These ‘nerves’ are densely bundled collections of
17
individual neurons designed to carry vast volumes of information to regions of the
brain dedicated to auditory and visual processing. There are many such transmission
pathways in the brain and accessing these pathways is crucial to rapid recall of stored
information. The fact that auditory and visual processing of information occurs in
specific regions of the brain gave early brain researchers insights into a modular
organization of the brain. In early conceptualizations, specific regions such as the
central-inferior portion of the left hemisphere were thought to exert solitary control
over auditory reception while others, such as the anterior inferior portion of the left
hemisphere were thought to exert solitary control over verbal expression (Feldman,
2006; Thompson & Madigan, 2005).
While these areas of the brain are deeply involved in these processes,
contemporary imaging studies of the working, healthy brain reveal a radically more
complex picture of information processing with diffuse regions often activated for
seemingly specific tasks (Duncan& Owen, 2000). A processing system with multiple,
diffuse activations in response to stimuli, with highly automated patterns of
unconscious storage of information has at least two important implications for
learning. First, the system consumes energy at prodigious rates, which has led to
adaptations for automation of many mental processes (Sweller & Sweller, 2006).
Automate mental processes govern many aspects of learning (Anderson, 1996b) and
the instructional plan that takes greatest advantage of this fact is more likely to lead to
positive learning outcomes than a plan that discounts the importance of automated
processing. Extending this line of reasoning to the second notable implication, the
18
highly diffuse system of cerebral activations calls for organization. In the absence of
external organizers, the mind relies on past performance history and/or complex
automated salience judgments to determine the best way to store information for
future retrieval. These two implications are accounted for by instructional design
systems that seek to maximize the utility of automation and present stimuli (i.e.
instruction) in ways that are well organized and appropriate to the learner’s past
experiences (i.e. prior knowledge). Such designs generally feature significant explicit
guidance, one of the topics explored in this study. The raw structures and processes of
the brain, however, give at best an incomplete and opaque picture of how learning
occurs. There is a need for some sort of model for how the brain functions that reflects
the underlying physical structures described above but goes beyond them to describe
the ways in which the brain uses information more generally. To such a model the
discussion now turns.
Information-Processing Theory
Shiffrin and Schneider’s (1977) model of information processing has influenced the
majority of widely accepted subsequent models. Its key features are a modular system
of cognitive processing, discrete working memory and long-term memory stores, and
sensory buffers. In this model input is received via the five senses, passing first
through a set of filters known as the sensory registers (see Figure 1., below).
19
Forgetting
Initial Processing
Retrieval
Encoding
Elaboration
Figure 1. Model of Human Information Processing, after Atkinson & Shiffrin (1963)
The two most frequently described of these registers, first conceptualized by Baddeley
(1986) are the visual sketchpad and the phonological loop. These registers are the
processing step at which salience determinations are made. For example, the noise of
traffic passing on a busy street is less salient to the content of a phone conversation
than the sounds coming through the telephone handset and is therefore less likely to be
passed on from the sensory register for further processing. Similarly, the buildings
alongside a familiar route are less salient to the safe conduct of the vehicle than the
moving vehicles on the road. For this reason, the buildings are less likely than the
other vehicles to continue past the sensory registers, which are continuously updated,
to the next step of processing.
Information that passes the sensory registers is held briefly in working
memory
1
1
Baddeley’s (1986) definition of working memory suffices for this preliminary model;
working memory is a system for, “the temporary storage of information that is being
processed in any range of cognitive tasks.” (p.34)
, a primary processing module with severe limits on the amount of
information held at any one time. Any information held in working memory but not
External Stimuli
Sensory
Register/Memory
Long-Term Memory
Short-Term
(Working)
Memory
Response
20
acted upon, that is to say prepared for storage, combined with other information to
form some greater chunk, subjected to higher-order processes such as inspection or
calculation, etc., soon decays and is forgotten in a number of seconds (Schneider &
Shiffrin, 1977). For the purposes of building this first model of information
processing, the key features of working memory
2
The final element in this model is output. Information from LTM is activated
or retrieved via working memory, which then forwards the information after relevant
processing to the appropriate action center. These centers are far too numerous to
describe in any detail; the important point is that many (but not all) actions have their
are that it is severely limited in
scope, severely limited in time, and severely burdened by the ongoing demands of
continuous input from the sensory registers. The functional partner to working
memory is long-term memory. Long-term memory (LTM) is functionally limitless in
its capacity to store information. Many models (e.g. Anderson, 1996b; Anderson, Qin,
Jung, & carter, 2007; Baddeley, 1986; Koch, 2004; Thompson & Madigan, 2005)
assume that this storage occurs in an associative network in which association are
made between pieces of information which are referred to as “propositions”. Because
the brain is an organic and dynamic system, in the healthy brain new information is
constantly being recorded into long-term memory stores. The speed and accuracy of
retrieval deteriorate as we age (Small, 2001), but the process of depositing new
memories into LTM is continuous.
2
Shiffrin labeled the form of working memory under discussion here “short-term
working memory”, an important distinction that will be addressed in the discussion of
Ericsson’s model of long-term working memory and Anderson’s ACT-R model of
cognitive processing.
21
origin in working memory. In sum, the core components of the system are a set of
sensory registers, a capacity-limited working memory, functionally limitless long-term
memory stores, and systems for output/action.
With these basic parameters of the information processing system, several
important complications to the basic model need mentioning before the discussion
proceeds. Paramount among these is automated knowledge, or automated mental
processes. By some estimates (Wegner, 2003) more than ninety percent of everything
that we do, think, or say is automated, functioning outside our conscious control. The
Atkinson and Shiffrin (1968) model seeks to describe the ten percent of cognition that
is conscious and deliberative. The vast majority of cognition, however, is neither
conscious nor deliberative. Walking or driving a car over smooth pavement, using a
toothbrush, typing by touch and eating are all examples of actions that were first
learned through effortful practice with working memory actively engaged. As
experience accumulates however, nearly all repeated actions become automated so
that precious working memory resources can be devoted to actions that require
conscious thought (Sweller & Sweller 2006). If one visualizes the previously
described model of information processing with separate modules for the sensory
registers, output, working and long-term memory, automated knowledge is a distinct
module that overlaps portions of all other modules. A more complete picture will be
conveyed in the discussion of Anderson’s (Anderson, 1996a, 1996b; Anderson,
Bothell, Byrne, Douglass, Lebiere & Qin, 2004; Anderson, Rede, & Simon, 1996))
ACT-R model of cognitive architecture. Anderson’s model suggests that there are, in
22
fact, dual systems of storage and retrieval – automated and non-automated (Anderson,
1996a). Before describing these more advanced, contemporary models, some of the
details of working memory need unpacking. The functions of WM are critical to
learning, and the manner in which WM functions are conceptualized have important
implications for implications for instruction. Some of these implications are presented
in subsequent sections on Cognitive Load Theory (Sweller, 1988). The following
section focuses on several important frameworks for understanding WM functions
themselves.
Models of Working Memory: Miller, Baddely, Cowan
If a unit of information – a concept, a process, a principle – is to be learned, it
must be held in working memory and processed for storage in a way that allows for
future retrieval. With the exception of a very small number of primary reflexive
responses such as the plantar Babinski reflex
3
3
Frequently used in assessment of brain status for unconscious individuals, the
Babinski reflex is involuntary plantar flexion in response to scraping stimulation
, skills that are demonstrated in healthy
adults are learned. Reading, writing, cooking, playing sports, driving cars, interacting
with one’s life partner – all of these are learned behaviors that rely on working
memory. Ebbinghaus (1850-1909), one of the founders of experimental psychology,
was deeply interested in memory and performed a wide range of experiments on
himself that sought to control for individual-difference factors such as life experience
by focusing on content-deprived memory tasks such as memorization of nonsense
syllables. The pattern of researchers using content-deprived, lab-based
experimentation to describe memory systems has been vital to the development of a
23
detailed understanding of memory. Unfortunately, it also served for many years to
bind the disciplines’ conceptualizations in such a way as to limit their utility in
predicting a range of real-world performance issues such as expert content mastery.
Miller (1956) sought to deepen the understanding of memory functions and
published a foundational piece, “The Magical Number 7, plus or minus 2” (1956).
Miller found, through a thorough review of the extant literature in the area as well as
his own line of experimentation that human working memory was limited to holding
approximately seven different chunks of information (5 to 9 being the average range).
For nearly fifty years, this allocation was accepted as dogma. Many hundreds of
experiments were performed examining working memory from a multitude of angles,
most of which appeared to confirm Miller’s thesis regarding capacity. In the 1980’s,
however, scholars such as Cowan (1988) began to interrogate the normative
methodologies of examining memory, drawing into question earlier findings.
Eventually, this line of inquiry led to a revised assessment of working memory
capacity such that many now believe that the capacity is even more severely limited.
Cowan (2000; Cowan, Elliott, Saults, et al., 2005) asserts that the magical number is
not 7 ±2 but rather 4±1 chunks held in working memory. If one examines this issue on
a macro scale, both estimates are in some sense of the word “right”. For many
cognitive tasks, particularly those that are highly novel, our working memory appears
to be limited to about 4 chunks of information. There are, however, certain tasks
(serial number memory, for example) that give every indication of approaching
Miller’s magic 7 chunks. The much more important point for the purposes of this
24
discussion is that ample evidence exists demonstrating the severe limits on working
memory (Anderson, 1990).
Baddeley (1986) made meaningful refinements to the Schiffrin model by
unpacking the role of working memory, describing its duration and function in greater
detail. Baddeley’s work demonstrated that working memory resources are best
described in terms of “chunks” of information. The concept of chunking in working
memory provides a point of entry into a revised model of information processing.
Chunking described the process through which the amount of information that an
individual can process changes over time and across contexts. For example, chess
experts have larger chunks of information for chess board configurations than chess
novices (Ericcson & Kintsch, 1995). Working memory, then, has a fixed capacity for
handling novel information, but readily adapts to handling increasing amounts of
familiar information. This model of working memory helps to define some of the
research questions for this study. For example, individuals with some prior experience
with the target leadership task would be expected to outperform their fellow
participants in each treatment condition because of their greater ability to handle the
(somewhat) novel information presented in the course. Similarly, individuals with
greater levels of experience at extracting information from text and media-based
instruction (as measured by weekly hours of computer use and education level
variables, for example) would be expected to perform better on a summative
assessment because of their previously developed schema for managing the cognitive
demands of such instructional events. This theoretical model of working memory is
25
well supported by recent findings from the neurosciences, reviewed next in this
discussion.
Neuroanatomical correlates of working memory
Returning briefly to a neuroanatomic perspective, Goldman-Rakic (1995;
Williams & Goldman-Rakic, 1995) demonstrated the localization of working memory
to prefrontal cortex in primate models. Because single-neuron measures are not
possible in humans, as they require the implantation of electrodes into the cortex,
trained or “behaving” primates are used to model human memory systems. The classic
studies of working memory in behaving monkeys are delayed response tasks in which
the monkey must hold a memory/image of the stimulus (e.g. food) for a duration on
the order of seconds before making a choice between several possible locations for the
stimulus without re-presentation. More recently, the delayed-selection tasks have been
supplanted by visual fixation studies that eliminate physical responses (i.e. reaching)
that create noise in the neuronal measures.
A number of interesting findings about working memory have followed. First,
that working memory has physical correlates in prefrontal cortex, most specifically in
dorsolateral prefrontal cortex, a region near the lower, side portion of the front of the
brain (Koch, 2004). Second, that specific neurons in this region perform specific
functions with respect to stimuli (Feldman, 2006). For example, the same neurons
were activated when a visual stimulus was presented in a 3 o’clock position and
inhibited when a visual stimulus was presented in the opponent, or 9 o’clock position
on the display. Moreover, these ‘location’ neurons became immediately inactive when
26
action was initiated. Finally, the neuronal activity in PFC functions in a temporally
bound manner through which, via feed-forward loops, neurons remain tonically active
on the order of seconds between the moment of stimulus presentation and action
initiation (Goldman-Rakic, 1995).
Although the usefulness of monkey models to the study of human
neuroanatomy can be questioned, many cortical functions from lower-order species
have been preserved in humans including much of the visual cortex, the general
structures and cell types of the brain, and the regionalized activation of cortical
activities in response to stimuli. In fact, scholars have argued that the evidence for
distinct working and LTM networks of regional cortical activity is far from
compelling (Ruchkin, Grafman, Cameron & Berndt, 2003). The argument made by
this opposing camp is, in short, that the most parsimonious and least complex model
should be favored when alternative are under consideration. Because a system with
sensory buffers or modules that directly activate relevant LTM stores is simpler than a
buffers + working memory + LTM model, it should be favored until falsified. They
argue that working memory, as demonstrated during empirical examination, is not
represented cortically by unique STM systems. Although not without merit, the details
of this argument are somewhat beside the point being made in this paper. If the control
systems in prefrontal cortical regions are purely intentional, as Ruchkin et al. and
others (Awh, Jonides, & Reuter-Lorenz, 1998) would argue or, conversely, that these
systems are in fact evidence of unique working memory module matters only insofar
as the final model helps further the understanding of learning, and more particularly
27
learning in media-based environments. Cowan et al. (2005) suggests that the
distinction between working memory and attentional systems may be a false
classification. Cowan et al. frames the capacity of working memory as, “the capacity
of the focus of attention, i.e. the scope of attention.”(p. 49, emphasis in original)
Moreover, fMRI studies of executive functions in humans, which are hypothesized to
require the integration of working and LTM, demonstrate routine activation of both
prefrontal cortex and hippocampal regions of the brain (Williamson & Goldman-
Rakic, 1995).
Goldman-Rakic and others working with behaving primates demonstrated
nothing about the duration or specific localization of working memory in humans, but
did successfully demonstrate that working memory is a valid construct with respect to
the neurophysiology of the brain (e.g. Sawaguchi & Goldman-Rakic, 1991).
Conceptual models of memory are useful for designing experiments and designing
instruction, but they offer no proof per se of their validity. By studying working
memory processes in primates, these scholars have demonstrated that the concept of
working memory could be verified as a localized, i.e. modular, cerebral function. By
extension, if working memory exists, it must operate in concert with LTM as a master
archive. This assumption is borne out by the numerous feed-forward and feedback
loops from numerous cortical regions to DLPFC during task completion in primates
(single neuron studies) and humans (e.g. Gray, Chabris & Braver, 2003).
28
Long-term Working Memory
Ericsson and Kintsch (1995) provide an extensive review of the final
complication to the base model of information processing under discussion in this
paper. Early models of working memory claimed that all processing occurred through
fixed resources (either 7±2 or 4±1 chunks). Baddeley (1986) demonstrated that the
sensory systems themselves have discrete representation in working memory,
particularly for visual and auditory information. These are sometimes referred to as
memory slave systems – task-specific processing that can be thought of as robust
filters for salience, as described briefly above. Even this slightly more nuanced model,
however, fails to address a fundamental principle observed in both laboratory and
field-based studies of working memory: expert task performers can remember
somewhat novel information within their area of expertise more quickly and
accurately than novices. If working memory resources were immutable, the
performance improvements seen in experts would be derived solely from faster
processing or automation of familiar task components. While these are relevant to
performance improvement, particularly automation, Ericsson and his colleagues have
asked a slightly subtler question – how do experts handle novel information? Over a
number of years and many experiments they have found that when operating within
one’s domain of expertise, chess or taking orders in a restaurant for example, there is a
supplementary system that they have labeled long-term working memory (LTWM,
Ericsson & Kintsch, 1995).
29
In this model, LTWM is capable of holding task-relevant information from
long-term memory in an active state. The information in LTWM is brought forward
for use based on cues received from short-term working memory (working memory as
heretofore described). This bridging between working memory and long-term memory
is a crucial addition to the information processing model and related architecture under
consideration here. With experience, increasingly robust and stable connections can be
made between domain-specific information in long-term memory and working
memory, allowing for many routine (though sometimes extremely complex) task
components to be performed as “wholes” by experts. Further, this connection system
allows for important savings in energy outlay as information is sought and retrieved
from long–term memory.
It is important to note that these efficiencies are bound by domain. Ericsson
does not claim to have, nor has he found the Holy Grail of the psychology of learning
– far transfer. Mastery of chess does not improve memory for strategic or tactical
decisions in a military context, nor does mastery of mathematics improve performance
in music. The importance of this distinction for the general debate over the importance
of guidance during instruction becomes more apparent in the contemporary models of
cognitive architecture, to which the review now turns. Critics of guidance (Jonassen,
2000) suggest that because of domain specificity, guidance is of limited use as its
benefits are unlikely to transfer to ill-defined tasks or domains. Although explored in
greater detail in the section below that specifically addresses the guidance debate
itself, ill-defined tasks and domains are, by logic, functionally immeasurable. The first
30
step toward understanding whether or not a task has been completed successfully is
some stable definition of a desired outcome state. Poorly defined tasks and domains
have desired outcome states that are highly subjective and therefore resistant to the
identification of the influence of particular prior experiences on performance. Put
simply, if one cannot reach broad agreement on whether or not a performance is
successful, then it is well nigh impossible to identify whether (or not) some prior
training had any influence, for better or for worse. The models of cognition to which
this discussion turns next provide further delineation of the ways in which these
knowledge acquisition and application processes function. There is a wide range of
alternative models of cognition, modular models have been chosen for this application
because they are broadly influential in the learning sciences and because of their
relative transparency in comparison to computational models.
Modular Models of Cognitive Architecture
John Anderson et al. –
Anderson (1996) and his colleagues (Anderson & Lebiere, 1998; Anderson, et
al., 2004) have developed a model for cognition that is consistent with contemporary
understandings of neurophysiology and provides a clear structure for understanding
cognitive architecture:
All that there is to intelligence is the simple accrual and tuning of many small
units of knowledge that in total produce complex cognition. The whole is no
more than the sum of its parts, but it has a lot of parts. (Anderson, 1996b, p.
356)
This model, known as ACT-R, has important implications for instructional design
including modeling the effects of excess memory load on learning, the process by
31
which task performance becomes automated, and the functions of working memory.
ACT-R is a software application that simulates cognition. Insights gained from ACT-
R experiments have been supported by later examination of the same processing
elements with fMRI studies. ACT-R describes a modular processing system for human
cognition with a notably constrained working memory, sensory and long term memory
buffers that contribute to working memory facilities, and a retrieval system that relies
on LTM buffers to rapidly retrieve relevant information from LTM.
Anderson’s model focuses first on the two general forms of knowledge –
declarative (knowing what) and procedural (knowing how and when). Declarative
knowledge is encoded in chunks (Miller, 1956) that contain the essential factual
information of any proposition, 11 + 25 = 36 for example, would be represented as the
following chunk of information:
Fact 11+25
Is addition-fact
Addend1 eleven
Addend2 twenty-five
Sum thirty-six
Procedural knowledge, manifest in such propositions, is represented by production
rules in the ACT-R model. These production rules can be represented by IF-THEN
statements (e.g. IF the goal is to add n1 and n2 when n1 + n2 = n3 THEN write n3).
There is no neurophysiological imperative requirement that declarative chunks cannot
flow from production rules, but in the ACT-R model production rules always follow
from chunk encodings. The implication of this for education, stated clearly by
Anderson (1996) is that, “acquiring competence is very much a labor-intensive
32
business in which one must acquire one-by-one all the knowledge components.” (p.
359)
In ACT-R, and other models of modular processing (e.g. EPIC; Kieras, Meyer,
Mueller & Seymour, 1999), the act of learning is essentially the act of developing,
storing, and automating mental processes for goal attainment. This is completed
through the four major systems within the ACT-R architecture: the perceptual motor
modules, the goal module, the declarative module and the procedural system
(Anderson et al., 2004, p. 1037). The perceptual modules receive information and
pass it, after preliminary processing to perceptual buffers. These sensory buffers
(which serve virtually identical functions to Baddeley’s “slave systems” (1986) and
Ericsson’s (1995) LTWM) forward the information to a production module that
performs matching, selecting, and performing tasks. Simultaneously, a goal buffer
retains the overall gestalt of the task in working memory while a retrieval buffer
performs parallel searches in concert with the declarative module. For action to be
realized, the outcomes of the search and retrieval systems feed forward through the
production module to manual (and other physical action) buffers and modules to
trigger bodily action.
This model contains the same basic elements as the Atkinson & Shiffrin (1968)
information processing system, but with significant refinements (see diagram in
Appendix A). Novel information for procedural tasks is developed based on chunks of
declarative knowledge. For example, someone learning to change a tire on their car
might first develop a mental list of the tools and steps needed to complete the
33
procedure. This knowledge, without practice, is quite fragile and shallow (Atkinson &
Shiffrin). It is likely that some item, a lug wrench for example, might be forgotten and
only recognized as missing when the wheel needed to be removed from the axle. With
practice, however, the declarative bits of information gradually become stored more
and more deeply (both metaphorically and concretely) until some degree of
automation occurs (Ericsson & Kintsch, 1995) For this discussion, automation has
occurred when some external cue leads to an appropriate action without conscious
deliberation (Kalyuga, Chandler, Sweller, 1998). At this point in the process, task
efficiency is increased with respect to speed and accuracy, assuming that the task has
been practiced correctly (Paas, Renkl, & Sweller, 2004; Paas, Renkl, & Sweller, 2003;
Nadolski, Kirschner, & van Merriënboer, 2005). This efficiency increase occurs
because working memory resources are not relied upon as heavily as when expertise is
being developed (Sweller, 2004). With continued practice the degree of automation
develops until the point at which the procedure can be completed without conscious
deliberation or awareness of the discrete actions or decisions required by the task. This
absence of conscious awareness, again given the accuracy of practice, is a great
benefit of the architecture – allowing precious WM resources to be devoted to unusual
or otherwise challenging aspects of the procedure that may vary across instantiations
(Gerjets, Scheiter, & Schorr, 2003; Sweller et al., 1998). Returning to the example at
hand, a new driver might first develop rudimentary competency at changing a tire. The
novice would “talk” their way through the procedure, referring to a printed job aid or
recollection of instruction to retrieve the necessary action and decision steps from
34
memory stores. The expert, an auto-racing pit-crew member tasked with tire changing
for example, has such a high degree of automation that the task is performed with
lightning speed and perfect accuracy as part of a team.
Automation and Cognitive Load
Automated mental processes are ubiquitous in human cognition. Current
estimates (Wegner, 2003; Sweller & Sweller, 2006) are that automated mental
processes of one type or another control more than 90% of everything we do. Even in
moments of apparent intentionality, evidence from the neurosciences examining
neuronal activity spikes and action potentiations
4
The contemporary models for cognitive architecture have important
implications for the instructional design of complex tasks. These implications are best
captured in Sweller’s Cognitive Load Theory (CLT, Sweller, 1988). Sweller points out
that the severe limits on working memory described by Cowan (2000) and the modular
(Koch, 2004) demonstrates that our
apparently willful decisions are sometimes preceded by initiation of decision-related
actions such as grasping or reaching. These cognitive efficiencies would not be
possible were it not for automated connections between working and long-term
memory stores, most probably through some form of buffer or slave system as
described above (Anderson et al., 2004). Sweller’s (1988) Cognitive Load Theory
provides a compelling example of how important these efficiencies can be to learning.
Sweller’s cognitive load theory
4
An action potentiation is a state transition at the neuronal level during which a
neuron passes from an inactive state to a ready-to-act state based on cues from
adjacent neurons.
35
system of cognitive processing described by Anderson (Anderson et al., 2004) and
others (Atkinson & Shiffrin, 1968) constrain the ways in which we learn to perform
new tasks. At the most basic level of the system, Sweller recognizes the generally
incremental pathways from coding of bits of declarative information (concepts, i.e.
facts, names, sight and sound memories) to the automation of complex procedures.
CLT presumes that mastery of complex tasks occurs most efficiently when the load of
task-relevant information on memory resources is optimized. Evidence demonstrates
that learning of complex tasks occurs inefficiently, if at all, when load is too high and
when load is too low (e.g. Teigen, 1994). Learning is therefore bound by three forms
of cognitive load, intrinsic, extraneous and germane load, or the aspects of the target
learning content that make it easier or harder to master (De Leeuw & Mayer, 2008).
The first of these forms, intrinsic cognitive load, is the inherent difficulty of the
task for the learner cause by the number of novel task elements that must be processed
in working memory for learning to occur. Like all forms of cognitive load, intrinsic
load is a function of learner characteristics such as fluid intelligence and even more
importantly, prior experience with the task or domain in question. The opposite of
intrinsic load is extraneous cognitive load – those elements in the presentation of the
task that are irrelevant to its learning and must be filtered out in order to focus on the
task itself. Examples of extraneous load include physical context variables such as
classroom temperature and noise as well as design variables such as the amount of
material presented simultaneously, the relevance of examples to the target task, aurally
distracting elements such as background music, or visually distracting elements such
36
as animated characters, etc. Extraneous load is not a fixed entity – task presentation
characteristics that are highly distracting for some learners (e.g. extreme heat in
classroom) are less so for those who have grown accustomed to the same conditions.
That said, there are aspects of task presentation that are universally likely to serve as
cognitive noise and occupy precious working memory capacity when they are present.
In multimedia-based instruction, colorful, frequently changing graphics and
background musical soundtracks are examples of sources of extraneous load. Finally,
there is germane cognitive load. Germane load is caused by elements of task
presentation or instruction that increase the likelihood of successful task completion.
The goal, then, of any instructional design effort is to optimize cognitive load. The
three component tasks of this effort are to minimize extraneous cognitive load, to
understand the intrinsic load (or likely difficulty level for the target group of learners),
and integrate germane load during instruction (in this case in the form of corrective
feedback). This discussion will return to the theme of optimization of cognitive load
throughout the remainder of the paper.
Summary of cognitive theory
The human mind has a modular organization for processing. Early descriptions
of the information processing system proposed a series of modules for receiving and
interpreting stimuli, performing mental work, storing and retrieving multiple forms of
memories to perform actions. This modular cognitive system has a critical bottleneck,
limited working memory capacity, which constrains many other aspects of cognition
during learning from instruction. One of the responses to limited working memory
37
resources is the overriding process of automation of thought and action. The majority
of human thoughts and actions are controlled in whole or part by automated mental
processes. This adaptation frees precious working memory capacity for conscious,
deliberative thought. The generally modular organization of processing is represented
in the brain by regionalized activations under task demands. For example, activity
increases in the lower front, side portions of the cortex during tasks that require
working memory. This physical representation of cognitive activity occurs at gross,
regional levels in the brain as well as at the level of the neuron. Visual cortical
neurons, for example, have been identified that respond to specific orientations of
patterns (e.g. vertical versus horizontal). These modules for processing and automation
function within an overlaid system of knowledge types. Facts of nearly every type are
forms of declarative knowledge and are easily brought forward into consciousness for
processing or output. Actions, particularly actions that require multiple steps or
decisions are forms of procedural knowledge and are extremely challenging to bring
into conscious awareness.
The contemporary understanding of human cognitive architecture that is
grounded in both robust theories of processing such as Anderson’s ACT-R (1996) and
findings from the neurosciences (e.g. Koch, 2004; Feldman, 2006; Thompson &
Madigan, 2005) has two broad implications for learning. First, that the human mind is
prone to automation of routine action. Because the mind rapidly automates actions that
are repeated, the most efficient learning occurs when successful actions are ensured
from the outset of the instructional experience. To encourage learner explorations that
38
inconsistently lead to successful results is to reduce efficiency in the long run by
increasing the likelihood of the need for un-learning. Learner explorations, to be
discussed in greater detail below, may have important motivational benefits but are by
definition less efficient paths to success. Secondly, the resource-constrained system
for processing novel information requires that the amount of information delivered to
learners at any one time during instruction must be carefully modulated to match their
unique needs and constraints. Too much or too little load on these resources leads to
less useful storage and reduced ability to complete target actions successfully. These
implications act upon the learner at the level of the individual and must be considered
by the designer of instruction. With these characteristics of cognition in hand, this
discussion now moves to a discussion of their implications for instructional design.
Instructional Design
Instructional design is the process of planning procedures for efficient delivery
of content that can be expected to lead to learning. There seem to be as many
instructional theories, and related instructional design principles, as there are epistemic
positions in relation to learning. This review intentionally focuses on those design
principles that are guided by cognitive psychology. This choice has been made for
three reasons. First, learning is an inherently cognitive process and to ignore the
literatures of cognitive psychology when designing instruction would be to risk
irrelevance. Second, it will be established that design systems based in cognitive
psychology have the weight of evidence on their side. Finally, the fields of cognitive
psychology informs the design and practice of media-based learning, the general
39
research field of this presentation, to a greater degree than other learning practices
investigated elsewhere such as apprenticeship or distributed cognition (e.g., Cole &
Engeström, 1993). The two models that are most influential on the design of
instruction for this project are those developed by Gagné and Briggs (1979) and by
David Merrill (2002a). Both are based in the science of learning and seek to clarify
which elements of instruction are critical to learning.
Gagné-Briggs
Bloom (1956) postulated three broad forms of learning outcomes as types of
knowledge – declarative (knowing that), procedural (knowing how) and cognitive
strategies (e.g., metacognition). While these types are consistent with the information
processing models discussed above, they lack specificity. Gagné and Briggs (1979)
provided a more highly specified taxonomy of knowledge types, with five broad
categories: (i) verbal information, (ii.) intellectual skills, (iii.) cognitive strategies, (iv.)
motor skills, and (v.) attitudes. Gagné also provides a durable definition of instruction,
defined as, “a set of events external to the learner which are designed to support the
internal processes of learning” (Gagné & Dick, 1983, p. 265). Again, he provides a
highly specified taxonomy of these events that are thought to move gradually from
purely external provision of instructional events (e.g. “Look at me while I
demonstrate” as an attention-getter) to a commonly internal provision of instructional
events (e.g. a learner thinking, “That looks important…”). The elements in this model
are:
40
a. Gaining the learner’s attention
b. Stating the objective of the lesson
c. Activating the learner’s relevant prior knowledge
d. Presenting the material to be learned
e. Providing guidance to successful performance
f. Eliciting performance
g. Providing feedback
h. Assessing performance
i. Enhancing retention and transfer
These elements are consistent with predictions for learning that can be inferred from
the model of cognitive architecture described above. For example, the suggestion to
activate relevant prior knowledge stems from the relationship between working and
long-term memory stores. By foregrounding the elements of the learners’ prior
experiences or memories, the portions (i.e. schemata) within long-term memory that
will facilitate successful retention and/or task completion are brought into working
memory buffer systems (Anderson et al., 2004). Additionally, the strategies for
enhancing retention and transfer explicitly reference the need for moving information
that has been presented from working memory stores to long-term schemata. In this
application, transfer can be understood in a relatively narrow sense – successful
completion of the target task (e.g. long division, changing a tire) outside of the
instructional context and without additional formal guidance.
41
The Gagné-Dick model has two additional advantages over other, more highly
specific models. First, it has an intentionally comprehensive scope. The elements of
the design model are structured so that the same design expectations can be brought to
bear on declarative as well as procedural knowledge mastery. Earlier models such as
Bloom’s original learning model (1956) suggested distinct instructional strategies for
particular tasks; drill and practice was tied to memorization of facts, for example. The
Gagné-Dick model focuses instead on the more general theories of learning drawn
from cognitive psychology to develop a foundation for designing all types of
instruction. Finally, this model attends to the necessity for the consideration of
automation in instructional design. Although the authors reference external v. internal
task control mechanisms, a more contemporary framework would cast this dichotomy
in terms of automation, with internal control mechanisms being those that have been
automated during the course of instruction. Other authors (Bloom, 1976) have
emphasized that the feedback stage of instruction should be ‘corrective’, which is to
say that the feedback provides explicit information about the accuracy of the learner’s
performance. When embedded in these early models, the instructional design system
provides great clarity for the learner about what to do when confronted with a
particular problem, but not how to accomplish these actions. Based on the model of
cognitive architecture Cognitive Load Theory reviewed above, emphasizing what
instead of what and how (and when) leads to problems with performance in the real
world. Mastering the declarative knowledge component of a task is not the same as
mastering the procedure itself. By overloading working memory stores, and building
42
storage and retrieval patterns that draw on declarative rather than procedural
knowledge, the design requirement for corrective feedback in this model can have the
effect of hindering performance. Merrill (Merrill, 2002a, 2002b; Merrill & Tennyson,
1977) provides additional refinements of the basic Gagné model described above that
address the shortcoming by focusing on practice, as well as feedback. Although
undeniably important, corrective feedback without the opportunity for practice has a
less powerful effect on learning than does practice with feedback, an assertion tested
in this study.
Merrill
Merrill’s work is consistent with Gagné’s suggested techniques, as well as with
the cognitive architecture assumed for this discussion. As with many contemporary
models, Merrill divides all knowledge into declarative and procedural categories, with
specific treatments developed for concepts (declarative), principles (declarative &/or
procedural) and procedures (procedural) (Merrill & Tennyson, 1977). These categories
provide a useful framework for thinking about instruction; the categories themselves
are deeply intertwined. Concepts can lead to classification procedures for identifying
examples and non-examples. Procedures can also be learned step-by-step as a series of
discrete concepts rather than as a performance, as anyone who has committed a recipe
to memory and then prepared the dish can attest. More recently, Merrill’s work has
focused on so-called “first principles” of instruction, all of which conform to a four-
phase instructional cycle (Merrill, 2002a, 2002b). These phases are activation (of
attention and prior knowledge), demonstration (of procedure with an application of
43
knowledge), application (i.e. practice and feedback) and integration (into memory
archives and daily life/real world) (Merrill, 2007, p.3). Merrill has extracted from the
literatures on instructional design and cognitive science a set of principles to guide the
design of instruction. These principles, which form the core of the Guided Experiential
Learning model (Clark, 2004b) that will be described in greater detail below, relate to
the four phases listed above as well as an overarching emphasis on problem-centered
design. To be clear, in this instance problem-centered design is a form of guided
instruction that focuses on a specific problem or real-world task as a whole. This is not
equivalent with techniques of problem-based learning, which is most commonly
(Kirschner, Sweller, & Clark, 2006) a minimally guided program of discovery
learning in which a case or problem is presented to learners without explicit
instruction on how to solve the problem or resolve the case.
Problem-centered
Merrill points out (2002a, 2002b, 2007) that the instruction should, “involve
authentic real-world problems or task[s]” (2007, p. 7) and that, as in the Gagné model,
the objective must be made explicit at the outset. Further, the instruction should
present a series of problems that are increasingly complex or difficult rather than a
single exemplar with task practice first coming in small chunks before integrated
wholes (2007, p.8). Problem-centered design might best be thought of as a pre-design
consideration, for it is realized throughout the design rather than at one stage or
another alone.
44
Activation
This step in design focuses the designer’s attention to the importance of prior
knowledge. Ideally, the learner is encouraged to make this knowledge explicit through
recounting or diagramming past experiences (or providing experiences) with which
the new information may be elaborated (Merrill, 2002b). Additionally, the designer
should provide, or elicit recall of, some structure or organizer so that the new
information is quickly and deeply connected to previously stored information (Merrill,
2007, p.8).
Demonstration
Put most simply, this amounts to showing rather than only telling the learner
what to do when targeting procedural knowledge instruction. For declarative
knowledge, the demonstrations focus on providing concrete definitions and examples
and non-examples of the concepts themselves. Demonstrations (procedural
knowledge) or examples (declarative knowledge) must be also consistent with the
content, rather than using multiple unrelated example/demonstrations contexts (2002;
2007, p.9). Explicit guidance plays an important role in the demonstration phase of
Merrill’s model for procedural knowledge and is particularly relevant for this study,
which also targets procedural knowledge.
Application
Practice, or application, should occur throughout instruction, with practice
occurring first with small parts of the procedure (part-task practice) and progressing
toward some form of whole-task practice with feedback. Practice must be consistent
45
with content and feedback, as in Bloom (1976), and should also be corrective in
nature. Scaffolding, in the Vygotskyian (1992) sense should be provided throughout
practice, with support diminished or faded as skill levels improve across trials or
practice sessions.
Integration/Implementation
At the final stage of instruction, the focus moves to the learner’s performance.
Instruction should provide ample opportunities for demonstration of and reflection on
their new knowledge or skill(s) (Merrill, 2007, p. 12). The designer should consider
the way in which instruction is actually delivered, or implemented. Navigation through
tasks should be uncomplicated, learner control should be appropriate (see discussion
of expertise reversal effect, below), instruction should be personalized and
collaboration between learners, if used, should be implemented effectively, that is to
say with shared responsibility for outcomes, collaborative groups of no more than 4
persons and with ability-heterogeneous grouping (p. 13).
Summary of instructional design theory
In sum, Merrill’s presents a design framework that is centered on authentic,
that is to say real-world, problems. As with many models for design that are grounded
in cognitive psychology, Merrill’s emphasizes attention and activation of prior
knowledge, providing explicit guidance and other forms of instructional support
during content-matched demonstrations, application with corrective feedback and
practice that begins with parts of the task before proceeding to whole-task practice.
While Merrill’s principles share surface characteristics with other design philosophies
46
(e.g., an emphasis on authentic problems is shared by proponents of constructivist
pedagogy such as Savery & Duffy, 2001), Merrill’s model emphasizes the role of
explicit instruction and guidance with respect to the successful solution of the
problem, as well as the development or activation of relevant prior knowledge prior to
explicit instruction. Merrill’s model has informed much of the work that has occurred
in the field of instructional design in the last decade, particularly the application of his
instructional design principles to media-based learning (Carroll, 1994; Clark, 1994;
Kalyuga, Chandler, & Sweller, 1998, 2000; Mayer, 1999b, 2001; Moreno & Mayer,
1999; Nadolski, Kirschner & van Merriënboer, 2005; Rogers, Maurer, Salas & Fisk,
1997). It is this applied work to which the discussion now proceeds.
Guidance During Instruction
Few would suggest that students be given access to materials without any
direct instruction whatsoever, but many have suggested that the amount of direct
instruction be held to a strict minimum so that students ‘make sense’ of the material on
their own and “take ownership” of the material (Jonassen, 1991; Klahr & Nigam,
2004; Norman & Schmidt, 1992; Savery & Duffy, 2001). Assumed benefits from this
minimal-guidance or constructivist pedagogy are improvements in the underlying
cognitive skills that support future learning. For example authors (e.g., Hmelo-Silver,
2004) have suggested that by allowing learners to generate and test their own
hypotheses in a science curriculum without any instructor interference develops a
more robust set of scientific reasoning skills that can be applied to future science tasks.
Those who recommend a minimally guided form of instruction typically acknowledge
47
the need for some structure or scaffolding for novice learners to enable success and
minimize fruitless efforts. Making the argument that those who design instruction -
researchers, authors, and classroom teachers – should not provide some form of
support for learners has generally been left to the fringes in the field.
Some of those advocating a no-guidance pedagogy appear to seek political
change as much as they do learner growth toward content mastery (e.g. Jonassen,
2000; Levine & Resnick, 1993). When dropout rates, standardized test performance,
college-going, college completion, school funding, school quality and teacher
preparation strongly favor white students over historically marginalized ethnic
minorities, political change seems more important now than at any other time in our
nation’s history of public schooling. That said, the literature on political change in
school contexts is important but not relevant to this review. The relevant instructional
design issues for this discussion arise from Dewey’s (2004) interest in instruction
rather than Freire’s (1970) equally important emphasis on the use of teaching, (if such
a term can be applied to Freirean pedagogies), to create political and structural change
within a society from the grassroots to leaf-tips. The issues here relate more directly to
the design of instruction that intends to develop content mastery in a learner. With
respect to instructional design and guidance, there are two camps that, borrowing from
the Bauhaus, where one camp believes that less is more and conversely, the other
believes that that more is more. In keeping with recent work in this area, these two
positions will be referred to as “minimally guided” and “strong guidance”. Advocates
for minimally guided instruction rely upon constructivist theory while those who
48
advocate strong guidance rely on a post-positivist epistemology grounded in the
history of experimental science. Unpacking some of the important differences between
these two positions makes understanding the differences in the respective approaches
to instruction more clear.
Constructivism
Constructivism is in some ways a philosophy without a center. Unlike other
philosophical traditions such as Cartesian dualism, constructivism does not have one
set of precepts or logical conditions at its center. Instead, constructivism, for the
relatively simple purposes of this discussion, can be understood for what it rejects
more easily than for what it may support. In short, it rejects the notion that there is a
quantifiable, unitary reality that can be known (Burbules, 2000; Confrey, 1990; Fox,
2001; Lebow, 1993). Authors who support this position point out that the human mind
is an organ of vast complexity, one that varies over time for each individual and
dramatically between individuals. At a fundamental level, each person maps their
knowledge onto their extant neural structures in ways that are unique to that
individual. “Shared” knowledge, in a constructivist paradigm, can only be
superficially similar because of the underlying idiosyncrasies of the storage and
retrieval processes. On this point there is no useful refutation. Knowledge is indeed
“constructed” at the level of the individual and is never the same for any two
individuals (Koch, 2004; Anderson, 1996b). In the words of one of its primary
proponents, Lebow (1993), “the seven primary constructivist values [are]
collaboration, personal autonomy, generativity, reflectivity, active engagement,
49
personal relevance, and pluralism.” (p. 5). These values reflect the extension of the
constructivist philosophical position into the description of knowledge accumulation
as a constructive process.
Constructivism applied to instruction extends these philosophical descriptions
into prescriptions for instruction that are less firmly grounded in cognitive science but
may nevertheless provide useful insights. Savery and Duffy (2001) describe three
instructional propositions of constructivism. First, they write that, “understanding is in
our interaction with the environment.” (p. 1). In this, the authors focus the reader’s
attentions on the importance of contextualizing learning; without a context, there can
be no learning. As far as this goes, the logic is sound – it is, after all, inconceivable
that disembodied learning can take place. Our bodies, in which our minds reside,
provide a base context from which learning is inextricable. This must not be conflated,
as do Savery and Duffy, with casting learning as equivalent to distributed cognition.
By confusing the importance of context to cognition with the relationship between
context and cognition, constructivists downplay the importance of conveying the most
accurate understanding of content available in favor of their claim of superior
development of analytic skills. Second, Savery and Duffy propose that puzzlement
provides stimulus for learning. Again, there is much theory (e.g. Piaget, 1969, 2000)
that suggests that cognitive actions such as accommodation of novel information into
extant schema leads to learning and forms the basis of ongoing cognitive organization
processes. These theories stand in contrast the a robust body of evidence that suggests,
conversely, that clarity in instruction which ties novel information to prior knowledge
50
(and/or develops such prior knowledge before presenting novel information) is the
most effective means for knowledge transmission (Anderson, Reder, & Simon, 1996).
Finally, Savery and Duffy propose that all knowledge is negotiated within social
interactions through viability evaluations or testing of learner understandings. In this
sense, any knowledge that results in successful goal achievement is viable and
therefore equally valid to all other paths to successful goal achievement. The authors
claim Rorty’s (1991) and Dewey’s (1938, 1997) emphasis on pragmatism in support
of this proposition. This claim is unwarranted in this case as only those forms of
knowledge that are most widely shared, i.e. least context-bound, are likely to be useful
across multiple instantiations of goal pursuit. For example, many heuristics are
available for solving quadratic equations, but only those solutions that are consistent
with basic mathematical operator laws are likely to be applicable to future tasks in the
realms of matrix algebra or calculus. The immediate, negotiated utility of knowledge
must not be its only index of its propriety as a learning goal. In the constructivist view,
these viability searches are the sine qua non index of learning attainment (Savery &
Duffy, 2001; Resnick, 1987). Some scholars have advocated a less rigidly
constructivist approach (Wise & O’Neill, in press), but for the sake of this discussion,
the more clearly framed argument against guidance and in favor of constructivist
pedagogy is more instructive.
Jonassen (1994), for example, describes the ways in which a traditional
approach to instruction is always a series of control processes between the instructor
and the learner. Constructivism provides a point of entry to understand this
51
relationship as one not simply of knowledge exchange and development but of power
relationships, hierarchies, and positions of privilege. Because of the ways in which
these relationships have provided advantages to those in positions of power at the
expense of those on the margins, a constructivist pedagogy suggests learning goals
that are not content-driven but focus instead on “learning to learn” skills such as
metacognition, self-regulation and so on. Many authors (Colliver, 2000; Confrey,
1990; Duffy & Kirkley, 2004; Halloun, 2006; Hmelo-Silver, 2004; Kozma, 1994;
Jonassen, 1994) recommend that the instructional environment be re-structured, re-
purposed to suit this alternative agenda for learning. The movement in this literature
from description to prescription may have, thus far, proceeded in an excessively
piecemeal or anarchic manner as there are precious few indications that constructivist
pedagogies work for any but the most skilled segment of any learner population
(Kirschner, Sweller, & Clark, 2006; Sweller, Kirschner, & Clark, 2007).
Postpositivism
With respect to educational achievement and instructional design, a
postpositivist framework is one that presumes most fundamentally that learning can be
quantified. Further, it assumes that the design of instruction according to principles of
learning that are grounded in cognitive psychology can provide useful efficiencies
with respect to maximizing learner content mastery and time spent acquiring new
knowledge. Postpositivism itself grows out of a positivist or objectivist epistemology
(Rorty, 1991). The objectivist position, in the contemporary context of the social
sciences, is inherently flawed because of its central assumption that “truth” is
52
inherently knowable and immutable in the face of changing context(s). Postpositivism
makes no such assumptions about the immutability of truth or “facts” as such. Instead,
as Jonassen (1994) points out, postpositivist researchers in the field of instruction
generally take a Deweyan pragmatic approach – focusing on providing instruction that
leads to learning, when given a particular context. It is this contextualization of the
instruction that characterizes much of what contemporary instructional designers
attend to with respect to the design plan itself (e.g. Clark 2004b; Kalyuga, Chandler, &
Sweller, 1999; Merrill, 2007; van Merriënboer, 1997). In this theoretical framework
instruction becomes a system for controlling the content, its delivery and the context
in which this delivery is made manifest. The central argument of content delivery is a
question of guidance; that is to say, how much do we tell the learner about what it is
we want them to learn? The overwhelming preponderance of evidence suggests that in
many cases explicit guidance that focuses on teaching declarative information before
proceeding to step-by-step instruction for procedural information with frequent
practice and corrective feedback is much more effective for learning than forms of
instruction featuring minimal guidance or guidance that fails to focus the learner’s
attention on the solution steps at hand. This project examines the relationship between
guidance and achievement in a variety of ways, but the arguments against and in
support of guidance require their own examination before turning in subsequent
chapters to the study itself.
53
Minimal guidance during instruction
As presented in the section on constructivism, there is a group of researchers
who favor a pedagogy that rejects much of the most important findings in the
cognitive sciences and favors an emphasis on problem-based learning, social learning
models, and other minimally guided techniques. Burbules (2000) has suggested that
some of the enmity between constructivist and instructivist pedagogy advocates would
be reduced if everyone could accept that,
A skilled teacher needs many resources in her bag of tricks, and that different
situations, different students and different subject matters require the ability to
adopt and adapt multiple approaches if they are going to succeed as a teacher
in the face of many learning styles and degrees of motivation found among
students. (p. 314)
Burbules goes on to remind us that theories of learning and theories of teaching need
not necessarily be bound together. These points seem beyond dispute. Certainly, good
teachers have many capacities and abilities through which they help children of
different motivational and learning ‘styles’ succeed. Without doubt, a theory of
learning does not require a theory of teaching. That said, at some point in the progress
of education, particularly public education in an environment of deep and shameful
inequities and fixed often inadequate resources, there must be a recognition that the
most effective way to teach students is to efficiently and effectively provide them with
the skills that they need to complete the cycle of education and join society as
productive members.
54
Duffy – constructivist learning principles
Duffy suggests a number of instructional principles that flow from a
constructivist epistemology that are relevant to a discussion of guidance.
1. Connect learning activities to a larger problem.
2. Support learner ownership for the task (ensure shared goal orientation)
3. Design authentic tasks
4. Design tasks and learning environments to reflect the complexity of the
authentic task environment
5. Give ownership of solution development to the learner
6. Design the learning environment to support and challenge the learner’s
thinking
7. Facilitate testing ideas against alternative perspectives and contexts
8. Support reflection on content learned and the learning process
None of these principles makes mention of guidance specifically, but the call for
challenges to the learner’s thinking (principle 6) may reflect a call for some form of
guidance. Further, principle eight suggests designs that include reflection on the
learning process, albeit after the conclusion of the lesson (if the use of the word
reflection is used in its most common sense). In this constructivist framework for
instruction, the learner stands at the center of all activity. With the target task or skill
(be it procedural or some chunk of declarative information) thus marginalized,
guidance become superfluous when it relates specifically to task performance. The
guidance, or more accurately directives, in this model for instruction focuses on the
55
process of engaging abstractly with content. It may be that the difference in emphasis
on guidance is revealed most clearly when Duffy writes, in reference to the task
authenticity principle, “…we do not want the learner to study science – memorizing a
text on science or executing a scientific procedure as dictated – but rather to engage in
scientific discourse and problem solving.” (Savery & Duffy, 2001, p. 4). Other
researchers (Anthony, 1973; Hmelo-Silver, 2004) share this interest in emphasizing
the learner’s engagement with discourse and scaffolded problem solving over mastery
of content.
Minimal guidance & problem-based learning
Problem-based learning (PBL) is based on the notion that if the goal is to
develop deep knowledge that can be flexibly applied in multiple contexts, then the
instructional environment should mirror the task environment in both the degree of
complexity as well as the lack of supports. Hmelo-Silver (2004) describes the PBL
environment in this way:
A PBL tutorial session begins by presenting a group of students with minimal
information about a complex problem. From the outset, students must question
the facilitator to obtain additional problem information; they may also gather
facts by doing experiments or other research. (p. 242)
This model provides as little guidance as possible with respect to problem solutions,
focusing instead on guidance toward developing problem-solving skills, including
metacognitive skills such as self-identification of knowledge deficits. Although no
information is given to the learner that explicitly connects the problem to their prior
knowledge, the PBL model assumes that the “naïve discussion of the problem” (p.
244) in which the PBL group engages will activate relevant prior knowledge. Whether
56
or not the knowledge thus activated is relevant to the most efficient problem solution
cannot be known until the solution is delivered and explicated by the learner. If the
solution fails the social verification test described by Savery and Duffy (2001), the
process begins anew. Indeed, authors who support a PBL, minimally guided approach
to instruction acknowledge (Chi, M. T. H., De Leeuw, N., Chiu, M. -H., & Lavancher,
C., 1994; Hmelo-Silver, 2004; Klahr & Nigam, 2004) that PBL learners are more
likely to make errors than students receiving guided instruction. Because PBL is
focused on the development of problem solving skills themselves, as well as self-
regulated learning skills, the error patterns are, “a necessary step in learning to apply
new knowledge” (Hmelo-Silver, 2004, p. 250). It is a paradox that routine errors
function to improve learning efficiency or task mastery. However, PBL learning
models predicts significant error during initial knowledge application (Norman &
Schmidt, 1992; Palincsar, 1998; Tan, 2004). This would suggest that the target
learning has not, in fact, taken place during the PBL phase of instruction but is arrived
at through subsequent trial and error.
What is overlooked in PBL and other minimally guided approaches to
instruction are the underlying mechanisms of knowledge acquisition that precede the
long-term storage of information. Knowledge indeed appears to be constructed
(Geary, 2002; Sweller & Sweller, 2006; Koch, 2005) but this construction is a process
that relates directly to storage and retrieval systems, which in turn are dependent on
working memory, as described above. Kirschner, Sweller and Clark (2006) point out
that many, if not all, of the techniques in PBL or other minimally guided learning
57
environments may act as sources of extraneous cognitive load. If WM is the pathway
through which all novel information must pass in order to be acted upon and/or stored,
as is widely accepted to be the case, it seems illogical to provide extraneous sources of
load on the limited WM resources available. Although a detailed analysis of the PBL
literature is outside the scope of this discussion, it bears noting that many of the most
prominent authors in the field (Hmelo-Silver, 2004, 2006; Burbules, 2000; Duffy &
Savery, 2001; Edelson, Gordin, & Pea, 1999; Schwartz, Lindgren, & Lewis, 2008)
support the use of some forms of guidance. If this scaffolding (Duffy & Jonassen,
1992) or support (Duffy & Kirkely, 2004), however minimal, is a component of PBL,
the model leaves the following question unanswered: If learners are to be supported,
why not maximize the support and minimize extraneous cognitive load? Models of
instruction that advocate a form of this strong support are presented next.
Strong guidance during instruction
Cognitive load theory (CLT, Sweller, 1988) provides a framework for
understanding learning that has clear implications for instruction. As is true for
constructivist pedagogies, there is no requirement that a theory of learning be linked to
a system of instruction without evidence of the instructional system’s effectiveness.
Evidence of instructional effectiveness would be some measured improvement in
knowledge – declarative, procedural and/or application – following instruction
(Kalyuga, Chandler, & Sweller, 2001). The clearest examples of this sort of
effectiveness would be demonstrated by randomized experiments with pretests and
posttests, random assignment to treatment condition, and authentic tasks (as well as
58
reasonable comparisons between conditions so that the comparison isn’t simply one
between something and nothing) (Slavin, 2002). Such evidence of instructional
effectiveness based on design systems that follow from the tenets of CLT exists, and
will be described below. First, however, this discussion takes a broad look at the
instructional implications of CLT before revisiting Merrill’s principles of instruction
and several explicit design systems that advocate guidance that have been influence by
both Merrill and CLT.
Instructional Implications of CLT
There are many implications for instruction that follow logically from CLT.
Most generally, CLT suggests that by eliminating or minimizing extraneous cognitive
load, restructuring the onset of intrinsic cognitive load and controlling germane load,
working memory resources can be focused most intensively on the requirements of
learning (Sweller, van Merriënboer, & Paas, 1998). In short, working memory, and by
extension learning, are maximized by providing learner-appropriate forms of
instructional support (guidance) (Sweller & Cooper, 1985). Scaffolding, a generic
term for instructional support, may have as many definitions as there are instructional
designers, or so it would seem from a reading of the literature. For this discussion,
scaffolding is, “a combination of performance support and fading.” (van Merriënboer,
Kirschner, & Kester, 2003, p.5). These supports, and guidelines for fading, take
multiple forms including embedding the support in authentic tasks, providing feedback
on performance, proceeding from simplicity toward complexity of tasks demands,
providing demonstrations and examples, fading supports in keeping with performance
59
improvements and providing procedural and other support resources in a just-in-time
manner.
Embedding support in authentic tasks
One of the elements of instructional design systems upon which constructivists
and instructivists alike agree is the utility of authentic tasks for instruction. By
providing tasks that are authentic, in that they mirror or validly represent the real-
world tasks environment, transfer (sometimes described as near transfer for cases such
as this) between the learning and performance environments is facilitated. (Gagné &
White, 1978; Merrill, 2002) For example, including a variety of examples of types of
cells while teaching content related to underlying biology concepts related to cell
growth such a DNA replication or division. Within this authentic task environment,
extraneous cognitive load is generated when a learner switches their attention from
one knowledge object (i.e. content) to another (i.e. supportive information). Sweller
and colleagues have described this phenomenon as the split-attention effect (Kalyuga,
Chandler, & Sweller, 1999); a phenomenon that often leads to performance
decremements by activating a cognitive overload mechanism in which the extraneous,
albeit supportive, information is unconsciously disregarded. Ironically, when
supportive information is not embedded adequately into the task environment, the
learners who are in greatest need of support, that is to say those who require the most
working memory resources to address the task due to insufficiently developed prior
knowledge for example, are the least likely to access or benefit from such supports.
60
Because of the split-attention effect, it is imperative that instructional supports of all
types be embedded in an authentic task environment (Kirschner, Sweller, & Clark,
2006).
Feedback improves germane load
To review, germane cognitive load is that form of cognitive load that is not a
function of the task itself that provides working memory resource allocation
efficiencies that lead to improved performance (Ayres, 2006a, 2006b; Paas & van
Gog, 2006). When a learner performs a task but receives no feedback about the
accuracy of their performance, a comparison search for accuracy occurs. This search
of LTM stores for results from matching tasks that have been performed using similar
heuristics, which occurs automatically and unconsciously, serves as a form of
extraneous cognitive load. While processing external feedback requires working
memory resources as well, the message must be processed, schemata may require
modification, etc., these processing demands are a form of germane load because this
processing provides future performance benefits by developing solution heuristics or
schema modifications that are effective within the natural task environments.
Simplicity before complexity
The condition of cognitive overload occurs when working memory resources
are taxed beyond their capacity for successful processing, leading to default to an
unconsciously selected automated mental process that may or may not lead to
successful task completion (Clark, 1999). Presentation of overly complex (defined at
the level of the individual) learning materials creates a condition in which the intrinsic
61
cognitive load of the task exceeds resources, which leads to default to unconscious
processes that may be at crossed purposes with the learning objectives. The central
implication of this tenet of CLT is that information should be presented in a manner
proceeding from simple toward more complex information. By beginning with simple
information, WM resources are preserved for processing, activation and modification
of schema, and storage of new information in LTM.
Use of Examples
Using examples that relate directly to the authentic task environment reduces
extraneous load in the form of information that does not pertain to the task itself. For
example, a showing a successful negotiation when teaching negotiation skills reflects
a mirroring between the training and application contexts. This demands fewer
cognitive resources than an example for the same content in which some underlying
principle, such as equity of outcomes, is demonstrated absent the negotiation context
itself. The work in this area is unequivocally in support of the use of example in
training, but the form taken by these examples varies. Some advocate worked-out
examples (Mayer, 2001, 2004; Clark, 2004, in press), which are correlated with
improved performance. Others (van Merriënboer, Kirschner, & Kester, 2003) advocate
completion tasks, in which the learner completes more and more of the task with
decreasing amounts of support as their skills improve across training sessions. In
either of these, or many other cases of example use during instruction, the benefits
conferred on task performance are a function of the degree to which extraneous load is
reduced (Anderson, Reder, & Simon, 1996; Merrill, 2002). The greater the degree of
62
match between exemplar tasks and the authentic task, the lower the related extraneous
load and the greater the WM resources available to devote to the act of learning
(Anderson & Lebiere, 1998).
Fading
Providing support when none is needed is sometimes worse than providing no
support at all. This principle of CLT, known as the expertise reversal effect (Kalyuga,
Chandler, & Sweller, 2003), states that when instructional supports are given to a
learner who already performs the target task successfully, the likelihood is that task
performance will deteriorate. Put simply, support given when not required acts as a
form of extraneous load, taxing working memory resources and interfering with what
would otherwise be useful practice. Because this effect is not binary – no uniform
tipping point after which time additional support is always detrimental has been
identified – support must be faded when success is achieved, as further support may
interfere with the previously learned procedure (Bell & Kozlowski, 2002; van
Merriënboer, Kirschner, & Kester, 2003). Although unrelated to CLT, it bears noting
that fading of support as success increases has important effects that support task
motivation including improved self-efficacy and task commitment (Keller & Suzuki,
2004). Vygotskyian theory of scaffolding also suggests (1992) that instructional
supports be removed as rapidly as possible, based on the learner’s performance.
Closely related to the importance of fading is the last CLT instructional implication
described here – just-in-time presentation of information.
63
Just-in-time information
As has been described, there are multiple forms of cognitive load, extraneous
load being generally deleterious, intrinsic load essentially neutral and germane load
generally helpful to task success. When more information than can be efficiently
processed is provided to learners, this overload acts as a form of extraneous load, with
the previously discussed adverse effects. In contrast, when information relevant to task
completion is provided just in time, or on as as-and-when needed basis, this
information acts as an important form of germane load. Like all forms of novel
information, it taxes working memory resources. However, in this case the working
memory resources required for processing are resources well spent, leading to
performance improvement (Ayres, 2006a, 2006b).
Summary of CLT implications
In sum, the parameters of human cognitive architecture – particularly limited
working memory stores and automation across instantiations – have been described
with respect to learning in a set of principles known as Cognitive Load Theory (CLT,
Sweller, 1988; Sweller & Cooper, 1985). CLT, in turn, has important implications for
the design of instruction itself (Paas, Renkl, & Sweller, 2003). Among these, there are
several that have strong support in the empirical literature reviewed above: the use of
scaffolding, embedding instructional supports in authentic tasks, providing corrective
feedback, proceeding from simple toward more complex task elements, using
examples and demonstrations, fading support as expertise develops, and providing
information on a just-in-time basis. With these principles of CLT in hand, this
64
discussion turns next to examples of instructional design systems that apply these
principles to practice.
CLT-based Instructional Design Models
Instructional design systems based on the principles of CLT generally follow
one of two models – task-specific or comprehensive (Clark, 2004; van Merriënboer,
Clark, & de Croock, 2002; van Merriënboer & Sweller, 2005). Each of these types can
be applied across type with more specific models being applied to complete course
designs and comprehensive models used to design single-task instructional units. The
first model provided as an example of CLT-informed design is a task-specific model
developed by van Merriënboer and colleagues (van Merriënboer, 1997; van
Merriënboer, Jelsma & Paas, 1992; van Merriënboer, Clark & de Croock, 2002). The
second, a comprehensive model, was developed by Clark (2004b) and integrates
elements of several other design systems into a start-to-finish platform for design.
Four-component instructional design (4C/ID)
The 4C/ID system (van Merriënboer, 1997; van Merriënboer, Jelsma, Paas,
1997; van Merriënboer, Clark & de Croock, 2002) focuses on management of
cognitive load for the learning of complex tasks. The four components of this model
are the (i.) learning task, (ii.) supportive information, (iii.) procedural information, and
(iv.) part-task practice (van Merriënboer, Kirschner, & Kester 2003). The first
component suggests that a task be presented to the learner with sufficient scaffolding
to allow successful task completion. This is in direct contrast to constructivist models
described above in that it calls for explicit guidance as one of the forms of
65
instructional support in the initial phase of learning complex tasks. The underlying
goal of the task component is to, “confront [the learners] with all constituent skills that
make up the whole complex skill.” (p. 11) The forms of support can take many forms
including worked-examples and completion tasks but always fades as the learner gains
competence in completing the task so that by the end of instruction the learner is able
to perform the whole task in a realistic environment without further support. The
second component, supportive information, is centered on providing information
during instruction on a just-in-time basis, i.e. no sooner than it is needed, so that
extraneous load is minimized. This attempt at control of extraneous load frees
cognitive resources, as predicted by CLT, but does not require the learner to devote
these resources to mental effort relevant to the task. The just-in-time information often
takes the form of procedural information in the form of step-by-step, how-to
instructions other forms of direct guidance toward successful task completion. Finally,
part-task practice is provided through skill development to allow for the necessary
elaborative rehearsal required for skill automation. Ideally, part-task practice focusing
on unique, i.e. non-recurrent, task elements follows instruction and practice for task
elements that recur throughout the whole task.
This system directs the designer’s attention to developing instruction for a
single, albeit complex, task within a course of instruction. It does not make specific
recommendation to address issues of cognitive resource allocation, motivation or other
important elements of instructional design. For a model that is also based on CLT but
66
includes these and other elements, this discussion turns next to Clark’s model for
Guided Experiential Learning (GEL, Clark, 2004b)
Guided Experiential Learning
The GEL model is consistent with the principles of CLT and integrates
research findings regarding motivation and media-based learning environments into a
comprehensive design system for teaching procedural knowledge. This sort of
comprehensive strategy provides a template onto which content can be placed for live
or media-based instruction. GEL is not the right system for teaching learners with
advanced skill, ill-defined content (e.g. aesthetics), or purely conceptual knowledge
(Clark, 2004b). In addition to guidelines for course features, GEL presumes that the
information being provided to the learner is based on a rigorous Cognitive Task
Analysis
5
1. Lesson introductions and course goals
. CTA has been shown to provide important advantages of accuracy of
procedural information (Clark, Feldon, van Merriënboer, Yates, & Early, 2008). While
these advantages are worthy of consideration on their own merit, a treatment of CTA
lays outside the scope of this discussion. With procedural information in hand there
are specific elements to the GEL design system that will be discussed following their
introduction. These elements are:
2. Reason(s) for the course
5
For a review of Cognitive Task Analysis techniques, the reader is referred to: Clark,
R.E., Feldon, D., Van Merriënboer, J.J.G., Yates, K., and Early, S. (2008). Cognitive
task analysis. In J.M. Spector, M.D. Merrill, J.J.G. van Merriënboer, & M.P. Driscoll
(Eds.). Handbook of research on educational communications and technology (3rd
ed.). Mahwah, NJ: Lawrence Erlbaum Associate
67
3. Course overview
4. Lesson structure
a. Learning objective
b. Reason
c. Overview
d. Knowledge development
e. Demonstration of procedures (if applicable)
f. Practice of procedure (if applicable)
g. Review practice and provide feedback
5. Media selection for content delivery
6. Evaluation
a. Tests of conceptual knowledge
b. Tests of procedural knowledge
c. Transfer task evaluation
Lesson introductions, overview and reasons:
Providing a clear introduction to the lesson’s content, and stating the reasons
(in terms of risks and benefits) for learning that content, several tasks are
accomplished. First, an introduction that specifies what is to be learned and how this
knowledge relates to the learners’ prior experiences activates prior knowledge –
making subsequent storage and/or schema modification more efficient (Anderson,
1996a; Sweller, 2004). This support is augmented by providing an overview, ideally
visual, of the course so that incoming information is received within the framework of
68
the course as a whole. Clarifying the reasons for the course is one way of supporting
the task-specific motivation of the learner. By providing a description of both risks of
not learning the material and benefits of mastery, the learner is more likely to value
the content and apply mental effort to learning (Wigfield & Eccles, 1999). Further,
providing an introduction that connects to prior experiences as well as the reasons for
completing the course, task-specific positive efficacy beliefs are supported (Bandura,
1977; Zimmerman, 2000).
Structure of lessons
Lessons designed according to GEL principles share a common pedagogy,
which emphasizes direct instruction and corrective feedback in keeping with CLT.
Lessons are presented in the order of task completion in authentic task setting, or when
such an order is not necessary in the target setting from easier subtasks to more
difficult subtasks. As is true for the course as a whole, GEL lessons call for an
introduction in which the objective of the lesson, the reasons for the lesson and an
overview of the lesson components are presented. If for successful completion the
targeted procedural knowledge requires prior knowledge of, or experience with
concepts, processes or principles (declarative knowledge) then this is taught before
proceeding to instruction regarding the procedure itself. This stands in direct contrast
to constructivist or problem-based pedagogical models in which the whole task is
presented without explicitly addressing issues of prior knowledge (Anthony, 1973;
Confrey, 1990; Craig, 1956). The next phase of each lesson is demonstration-practice-
feedback cycle. Depending on the procedure, a set of no more than four novel steps
69
are demonstrated to the learner. The learner then completes supported practice that is
followed immediately by corrective feedback (Clark, in press). This feedback focuses
on the portions of practice that were correctly completed and what adjustments in their
strategy for completion should be changed for success. For all lessons, determination
must be made in advance of the delivery system, or media (see discussion of media-
based learning issues below). Finally, each lesson should be reflected in assessments
that include practical or application tasks, conceptual knowledge (when applicable)
and some form of within-domain transfer tasks.
Summary of GEL
The GEL model takes a comprehensive view of instructional design for
complex procedural knowledge that is based on the principles of CLT and
incorporates motivational supports absent from lesson-specific models. Generally, the
GEL system calls for introductions to courses and lessons that activate prior task-
relevant knowledge. By providing a rationale for successfully completing the lesson in
terms of risks and rewards, the learner’s expectation of success and valuation of the
end-state are supported pedagogically. The format of lessons centers on the
development of relevant conceptual knowledge followed by cycles of demonstration,
practice and corrective, strategy-focused feedback. Many elements of the GEL system
that describe the selection of experts for CTA, the CTA process, and course
development have not been examined here as they pertain more directly to
instructional development than design per se.
Summary of Guidance Issues in Instructional Design
70
The preceding section reviewed the two predominant positions on the issue of
guidance during instruction. These positions are informed by constructivism and its
emphasis on maintaining complexity of the task throughout instruction and by CLT-
influenced designs that seek to moderate extraneous load by presenting tasks and
support in smaller, easier-to-manage units. Constructivist pedagogies, including
problem-based learning, inquiry-learning, minimally-guided discovery, etc., suggest
that learning outcomes including transfer tasks are best facilitated by first allowing
learners to attempt solutions to an authentically complex problem with a minimum of
external supports such as step-by-step instruction (Duffy & Jonassen, 1992). Although
there may be few who currently advocate a sink-or-swim approach to instruction,
constructivist teaching is focused on long-term, albeit rarely measured, learning
outcomes and so rarely provides the level or depth of instructional support called for
by CLT-based models. Among the benefits of a constructivist approach to instruction
are its emphasis on contextualized learning, addressing issues of learner motivation
(i.e. self-regulation, application of effort), and the awareness of the importance of
future learning outcomes when considering instructional effectiveness.
In contrast, CLT-based models approach instructional design from the
perspective of the cognitive sciences, sharing many of this discipline’s strengths and
some of its weaknesses. CLT-based models operate in the tradition of scientific
experimentation and generally perform more transparently when assessed for their
merits. Additionally, CLT-based models seek to integrate what is currently accepted as
known with respect to the processes of learning and conditions under which learning
71
occurs most efficiently. CLT-based models also have the advantage of being based on
a single set of converging theories about learning that have been demonstrated to have
broad validity (Kirschner, Sweller, & Clark, 2006; Sweller, Kirschner, & Clark, 2007).
This theoretical grounding, combined with a set of measurement traditions that
emphasize serial control of individual variables in experiments or quasi-experiments,
allows researchers to interpret findings with considerably less ambiguity than
constructivist techniques which are plagued with confounds to interpretation
(Kirschner, Sweller, & Clark, 2006). While there is no essential, immutable benefit to
the CLT model for instruction when compared to constructivism, the clarity of
interpretation surely provides a more efficient avenue toward pedagogical
improvement over time than does a model in which multiple variables are in play
simultaneously. Historically, design models that emphasize explicit instruction over
discovery learning have sometimes been divorced from real-world contexts and
authentic tasks (Anderson, Reder, & Simon, 1996). That said, the best of
contemporary models, with 4C/ID and the GEL models as two exemplars emphasize
real-world tasks, authentic performance, and instructional design for complex
learning. Because of these design and measurement advantages, a CLT-based
approach is adopted for this project.
Online learning
Setting aside for the moment the ongoing controversy over guidance in
instructional design, this discussion now turns to the role of media in learning. While
some, Hastings and Tracey (2005) for example, continue to believe that computers can
72
provide some unique value-added effect on learning; the majority of research in this
area (Mayer, 2001) suggests that instructional design, not media-characteristics, is
responsible for learning. The section that follows outlines the differences between
these positions, assesses the merits of the relevant arguments, and reviews principles
of learning in media-based environments of relevance to this project. As with the prior
discussion of guidance, differences of perspective in this area of the literature grow
from divergent epistemologies.
Influence of media on learning
Those who advocate a situated cognition perspective on learning (Cobb, 1994;
Duffy & Jonassen, 1992) apply the emphasis on context of situated cognition to an
analysis of the effects of media on learning. The argument, in short, states that because
media are a context, the experiences of the learner in a media-based lesson are at a
unique remove from reality. This filtering of experience “is biased by the creator of
the media and its implementation in instruction.” (Jonassen, Campbell, Davidson,
1994, p. 34) Moreover, this camp suggests that, “It is difficult or impossible to isolate
which components of the learning system, the medium, the attributes, the activities,
the learner or the environment affect learning and in what ways.” (p. 35) From this
somewhat pessimistic view of the cognitive sciences, those who argue in favor of a
media effect point to the concept of affordances. Affordances (Gibson, 1988) are
those things that are offered or furnished by an object or entity. Brooms afford
sweeping; windows afford ventilation of rooms and views from within. Computers
and other media-based learning environments in this view afford different things than
73
are afforded by more traditional media such as books. Because the affordances
(described elsewhere as attributes; Clark, 1994) of media are inherently complex and
interact dynamically with the learner, the role of any single affordance of the media
cannot be determined with certainty (Jonassen, Campbell, & Davidson, 1994, p. 38).
Among the affordances, hereafter referred to as attributes in the interest of
clarity, of media are its ability to rapidly respond to the input of many users
simultaneously, providing multiple representations of knowledge objects, and
mediating (for better or worse) the learner’s ability to interact with instructional
materials (Kozma, 1994). Claims for the benefits of televised over live instruction
(Saloman, 1984), computer-aided instruction over classroom settings (Kozma, 1994;
Hastings & Tracey, 2005), and more recently the use of virtual-reality gaming
environments for instruction (Stapleton, 2004) are all part of an historical pattern of
emphasizing media over method. For those who reject a postpositivist empiricism, the
media v. method question is unanswerable because of the nested relationships between
media attributes, learners, and instructional design or methods. A classically empirical
model seeks to assign the variation in scores between treatment and control groups to
the treatment itself. If and when the relationships between contextual attributes (in this
case, media attributed), the learners and the instructional methods (in this case,
purportedly constrained by the medium) are inextricably nested, ascribing outcomes to
any particular element of the treatment condition is simply a guessing game. Some
researchers in this area believe that this presents an opportunity for enriched, novel
models of assessment that are supported by constructivist conceptualizations of
74
learning (Kozma, 1994; Schwartz et al., in press). That said, such a measurement
model is yet to be developed and, as proposed by Schwartz and others (Wise et al., in
press), would necessarily involve longitudinal data collection of multiple forms.
Media do not influence learning
Clark (1994) and Mayer (2001) formulate the relationship between media and
learning quite differently. These formulations state that to test the unique contribution
of a media attribute to learning outcomes, several conditions must be in place. First,
the comparison condition must be an authentic match to the treatment condition;
studies that compare a media-based treatment using a novel instructional method to a
traditional instructional practice absent the novel instructional method must be
disallowed as they are more likely to find a difference between conditions (Feuer,
Towne, & Shavelson, 2002). This difference cannot be credited to the media because a
difference between instructional methods was also in play (Clark, 1994). Second, the
study must seek to control one variable at a time (Feuer et al., 2002; Hargreaves, 1997;
Slavin, 2002). If the researcher seeks to study some media attribute, this should be the
targeted variable. Finally, the study should mirror as closely as possible the conditions
of a randomized experiment, including random assignment to groups (Coalition for
Evidence-Based Policy, n.d.). When these conditions are applied (which, to reiterate,
researchers operating from a constructivist or situated cognition epistemology reject),
the question is allowed to turn from a choice between media and method as causal
factors in learning to an investigation of the attributes of media most likely to lead to
learning.
75
The debate over the whether or not media make a unique contribution to
learning outcomes has reached something of a stalemate. Those who continue to seek
a unique causal factor attributable to media do so with vigor (Edelson, Gordin, & Pea
1999; Kozma, 1994), and seem to have made constructivist theory (Duffy & Jonassen,
1992; Savery, Duffy, 2001) into a pathway toward conceptualizations of these
contributions. This pathway rejects contemporary paradigms of experimentation in the
social sciences in favor of an approach that seeks to capture media contributions in
natural learning environments using a multiplicity of methods (Schwartz et al., in
press). Those who accept the decades-long bodies of experimental and observational
(Mayer, 2004) evidence about the absence of a unique media contribution to learning
nevertheless explore the ways in which different aspects of the media environment
lead to better or worse learning outcomes given the same instructional method. The
distinction being made bears reiteration: while rejecting a unique contribution of
media (e.g., computer-based, online courses, etc.) a group of researchers continue to
examine the ways in which elements of media-based design relate to learning. Many
such principles, or explanations of relationships, have been identified. Several of the
more relevant principles to this project are described below.
Learning in media environments
In a series of projects Mayer and other media researchers (e.g., Mayer, 1999a,
2001; Moreno & Mayer, 1999, 2000; Quilici & Mayer, 1996) have identified a set of
principles of media-based learning that, when implemented, may lead to improved
outcomes. Not all of these principles are relevant to all instructional designs but all
76
describe the ways in which learners interact with media-based learning environments
and are generally consistent with CLT
6
1. Multimedia principle: people learn more from pictures and words than pictures
or words alone
. Several of these principles relate to the project
currently under discussion and can serve as examples of the ways in which CLT
informs instructional design in media-based, online learning environments. These are:
2. Principles to manage germane and intrinsic cognitive load: task segmenting,
pretraining, presentation modality
3. Principles to manage extraneous cognitive load: media coherence, redundancy
4. The guided discovery principle
5. Navigation principles in media-based learning
These principles are not ways in which media makes a unique, irreplaceable
contribution to learning. Instead, they are characteristics of the ways in which the
human processing system interacts with media and thereby facilitate instructional
planning and design.
Multimedia principle
The multimedia principles states that individuals learn more from words
(ideally via narration) combined with relevant pictures than from words alone. This
principle is supported by CLT in that auditory and visual information are processed by
different systems in the brain (Anderson et al., 2004) and these processing systems
each have their own capacity limits. By activating both auditory and visual channels
6
For a complete treatment of the principles, see Cambridge Handbook of Multimedia
Learning R.E. Mayer, ed., 2004)
77
simultaneously, germane load levels are improved and storage and/or schema
modification procedures proceed more efficiently (Fletcher & Tobias, 2005, p. 121).
Although there are important subtleties to the multimedia principle, some of which are
described below, a convincing line of empirical research supports the benefits of
multi-modal presentation of information with respect to learning and within-domain
transfer (Mayer, 2004)
Managing cognitive load
Several specific principles that emerge from the CLT framework apply to
learning in multimedia environments. Several of these have been discussed in the
context of instructional design writ large, and apply here as well. For example,
segmenting instructional information into manageable chunks (generally no more than
four novel steps or pieces of information) (Ayres, 2006a, 2006b) providing pretraining
of information needed to complete procedural tasks have been shown to provide
performance benefits in multimedia learning environments. Other principles such as
coherence, modality and redundancy principles apply more specifically to media
learning. The coherence principle states that the images and words presented as part of
multimedia instruction should relate to one another very closely. Words should
describe the picture with which they are shown rather than referring to something not
shown (Mayer, 2004). For example, a description of the procedure for changing a tire
should be paired with images showing the steps described. The redundancy principle
states information presented should not duplicate any other information (Mayer,
2004). For example, a narration of the steps of changing a tire should not be paired
78
with matching text, as these are duplicate forms of the same information, presented in
two modalities. Finally, the modality principle states that whenever possible,
information should be provided auditorally (Moreno & Mayer, 1999). A single
cognitive processor processes visual information, such as printed text while auditory
information such as spoken words is processed by the auditory and phonological
cognitive processing systems. This dual processing leads to more robust encoding and
storage of the information, a learning benefit that has been documented for more than
thirty years (Moreno, 2006).
Summary of media-based learning
These principles of media learning are tied to CLT and inform instructional
design systems such as GEL and the 4C/ID when they are used to plan for media-
based learning environments. Contemporary understandings of the human cognitive
processing system inform these principles and suggest ways in which instruction can
be designed for maximum instructional efficiency. Although not a principle of
learning in media-based environments in the same way as those described above, it
bears noting that instructional efficiencies are among the most powerful
recommendations for computer-based instruction. With robust design grounded in the
cognitive science, computer-based instruction allows for the delivery of instruction of
the highest quality to a vastly broadened population of learners. While the story of
learning in media-based environments is far from complete, dynamic motivational
supports during instruction (Song & Keller, 2001) continue to present design
challenges for example, a large majority of the available evidence suggests that CLT-
79
informed design provides a range of benefits including improved learning outcomes,
and improved self-efficacy.
Leadership Theory
Leadership, the practice of wielding authority to guide the action of others, has
been examined and described for millennia in Asian texts (e.g. The Art of War, Sun
Tzu) as well as the Western cannon (Machiavelli’s The Prince, 1513; the Christian
Bible; the Q’Uran). A treatment of the prevailing theories of leadership over the
millennia is well beyond the scope of this project. Instead, the specific leadership
theory used for the content of the course used in this study will be briefly described.
The leadership theory in this study is called Situational Leadership Theory
(SLT) and dates originally from the late 1960’s (Northouse, 2004). SLT is a theory
that focuses on the interactions between superiors and subordinates in organizations.
Developed originally for business settings, SLT has been applied widely as it focuses
the leaders’ energies on identifying the readiness level of the subordinates in the
organization to complete a task or tasks and prescribes particular leadership styles to
match readiness levels. The leadership styles described in SLT are based on two types
of behaviors – directing behaviors and supporting behaviors. Directing behaviors
include one-way, top down communication, giving directions, rejecting subordinate
input and providing explicit guidance for task completion. Supporting behaviors
include two-way communication between leaders and subordinates, delegation of
authority, integration of input from subordinates, and providing social supports in the
form of praise or acknowledgement. These directing and supporting behaviors can be
80
placed on a 2×2 matrix (see figure 2) with the combinations of supporting and
directing behaviors leading to four leadership styles: directing, coaching, supporting,
and delegating.
Figure 2: Taxonomy of leadership styles and prescriptive matches of SLT
The four leadership styles are designed as matches for four subordinate readiness
levels. The subordinate readiness levels are in turn combinations of levels of
commitment (a motivational variable) and competence or ability to perform the target
task. Individuals high in commitment but low in competence are matched with a
directing leadership style in order to facilitate the development of competence and
ensure clarity of message content from leader to subordinate. Individuals moderately
high in commitment and with moderate competence require a coaching style. These
individuals continue to require clear direction but are ready for some social support
81
and acknowledgement of their relevant prior experiences in order to foster increased
commitment. Individuals with moderately high commitment and competence require
the leader to take a supporting role, in which the individual’s task-relevant competence
is acknowledged and minimal directing behaviors are called for in light of the
subordinate’s skills. Finally, individuals who are high in competence and commitment
are best served by a delegating leadership style in which the leader relinquishes task
control to the subordinate in consideration of their competence to perform the task and
determination to perform it well. The prescriptive matches are shown in figure 2.
Situational leadership theory has strengths and weaknesses. Among its
strengths are the emphasis on interaction, the descriptions of leader and subordinate
behavior categories, and a set of prescriptive matches that guide future leader
behavior. Among its weaknesses are the ways in which SLT categorizes variables such
as competence and commitment that are more often continuous by nature. Making the
determination regarding which subordinate behaviors can be categorized as
moderately high in competence or simply high in competence can be difficult because
of the nuances of such behaviors in practice. Similarly, the leadership styles capture
only two dimensions of the range of possible leader behaviors and fail to address other
important leader characteristics such as frequency of interaction, social tone of
feedback (authoritative vs. authoritarian, for example) and the context of the leader-
subordinate interaction (e.g. public vs. one-to-one). In spite of these weaknesses SLT
can be used as a rough guide for leadership interactions.
82
Learner Motivation: Self-efficacy
The last section of this review of the literatures related to this project presents a
brief discussion on one aspect of learner motivation: self-efficacy. Self-efficacy is the
belief that one is capable of completing a particular action successfully (Bandura,
1977). There is great controversy surrounding the nature of self-efficacy, particularly
in attempts to settle the question of whether efficacy are general in nature or are
related more specifically to domains and tasks (Zimmerman, 2000; Clark, 1998). For
the purposes of this project, task-specific self-efficacy is the more relevant
conceptualization and will therefore take the center of this description.
Self-efficacy
Bandura (1977, 2000) described self-efficacy as one of the determinants of
motivation, which in turn was seen as a component of his social-cognitive theory of
human development. Clark (1998), Zimmerman (2000), and Pintrich and Schunk
(2002) have written extensively on the topic of motivation, and meaningfully on the
construct of self-efficacy. Self-efficacy is the belief one holds about the likelihood for
success at some task (Clark, 1998). Efficacy beliefs are sometimes inaccurate, with
both underestimates and overestimates of one’s true ability quite common
(Zimmerman, 2000). Self-efficacy is an important construct to understand because it
helps to predict the level of effort an individual is likely to expend on future attempts
at the same task. For example, if efficacy beliefs for taking tests of French language
are very low or inappropriately high (i.e. based on overconfidence) then a student is
less likely to expend effort preparing for a test than would a student whose efficacy
83
beliefs were generally positive or appropriately high. In the first case, the individual
with low efficacy would assume that their efforts at study would not amount to
performance gains because of an underlying inability. The overconfident individual
would assume that they had no need to prepare for the test and so would also see no
purpose in devoting effort to study. Efficacy is a continuous variable, changes
dynamically in response to perceived task difficulty and performance contexts
(Bandura, 1977; Pintrich & Schunk, 2002).
Efficacy and future performance
Task-specific efficacy beliefs have been shown to meaningfully predict future
task performance. This effect can be manipulated in a number of ways, with stereotype
threat serving as one example of the possibly deleterious effects of such manipulations
(Steele & Aronson, 1995). When working to the individual’s benefit in terms of
performance, the belief that he or she is capable of successfully completing a task
increases the likelihood of successful completion (Zimmerman, 2000). A number of
models that unpack this phenomenon have been developed but one of the clearest is
Clark’s (Clark, 1999) CaNE model. The CaNE model suggests that three indexes of
motivation, goal choice, persistence in the face of obstacles and the application of
necessary mental effort can reveal the functioning of motivation on performance. For
example, an individual with higher efficacy beliefs for a task would be more likely
than those holding less positive beliefs to choose that task as a suitable goal, more
likely to persist until completion and more likely to apply sufficient effort to ensure
success. The scope of this discussion does not permit a detailed description of the
84
efficacy literature, which has developed over the last thirty years into a robust set of
theories and multiple lines of research (Zimmerman, 2000). For this project, the
important consideration is that higher efficacy beliefs have been shown to relate to
future performance in a positive manner. Instruction that includes strategy-based
corrective feedback has in turn been shown to increase task-specific efficacy beliefs
(Kluger & Dinisi, 1996). A subgoal of this project is to verify this relationship in the
context of online instruction for procedural knowledge.
Hypotheses
There are three central hypotheses that are explored in this study, the first
relating to the effect of the treatment condition on performance, the second to the
influence of predictor variables of interest, and the last to the relationship between
performance and task-specific efficacy beliefs. Each of these central hypotheses has a
set of research hypotheses, as found below.
H1: Study participants in the treatment condition with the most instructional guidance
will achieve higher scores on the summative assessment than those in either other
treatment condition.
H1a: Participants scores on the summative assessment will be predicted by
their assignment to treatment condition.
i. Participants in the maximum guidance condition will achieve higher
scores than participants in the practice-without-feedback condition.
ii. Participants in the maximum guidance condition will achieve higher
scores than participants in the minimal-guidance condition.
85
iii. Participants in the practice-without-feedback condition will achieve
higher scores than participants in the minimal-guidance condition.
H1b: Treatment condition will predict participant scores on subscales.
i. Participants in the maximum guidance condition will achieve higher
scores than participants in the practice-without-feedback condition.
ii. Participants in the maximum guidance condition will achieve higher
scores than participants in the minimal-guidance condition.
H2: Individual-level variables will predict participant scores on the summative
assessment.
H2a: Participants with higher education levels will achieve higher scores on
the summative assessment when compared with participants having lower
levels of education.
H2b: Participants with prior leadership experience will achieve higher scores
on the summative assessment than those without prior leadership training.
H2c: Participants with more weekly hours of computer use will achieve higher
scores on the summative assessment than those with fewer hours.
H2d: Chronological age will not predict performance on the summative
assessment.
H2e: Gender will not predict performance on the summative assessment.
H3: Treatment condition will predict self-efficacy scale scores.
H3a: Participants in the maximum-guidance condition will report great task-
specific self-efficacy than participants in the minimal-guidance condition.
86
CHAPTER III: METHODOLOGY
Participants
Participants included 63 adults from military and civilian populations.
Participants were recruited through a study flyer (Appendix B), referral through prior
participants and through direct verbal recruitment at two large, urban universities and
one small, rural university. Demographic information was obtained for 90 percent of
all participants and is summarized in Table 1, below.
Table 1: Demographic Information
Age Gender Prior
leadership
training
Highest level of
education completed
Weekly hours of
computer use
Median /
Frequency
31.5 Female = 38
Male = 20
No = 30
Yes = 28
Course
AA
BA
MA
Postgrad
= 17
= 8
= 5
= 20
= 6
0-5
5-10
10-15
15-20
>20
= 5
= 11
= 8
= 6
= 26
Mode 21 Female No MA >20
N 57 58 58 56 56
Design
An experimental design (Figure 3) was used to examine the effects of differing
levels of guidance, in the form of corrective feedback, on performance on various
measures of performance and self-efficacy. Participants were randomly assigned to
one of the three study conditions, a control condition, a practice-without-feedback and
a practice-with-feedback condition. Following completion of the instructional portion
of the study, participants completed a summative assessment and a scale of task-
specific self-efficacy.
87
Random
Assignment
Figure 3: Diagram of research design
Apparatus
The study used of two assessment instruments and a media-delivery platform.
The assessment tools were a measure of declarative and procedural knowledge related
to Situational Leadership Theory (SLT, Northouse, 2004) that was the instructional
content for this project and a measure of self-efficacy for leadership tasks developed
for this project (Bandura, 2001). Because the study explores the effects of instructional
strategies on learning in a web-based environment, the media-delivery system was a
crucial component of the project. The web-based instructional delivery platform,
Army Excellence in Leadership, will be described first, followed by a brief description
of the assessment instruments.
Army Excellence in Leadership
USC’s Institute for Creative Technologies (ICT), through a research
collaboration with the U.S. Army Research Institute, developed a web-based
Informed
Consent
Procedure
Summative
Assessment
Self-
Efficacy
Scale
Debrief
UnGuide
ModGuide
FullGuide
Participant
Entry
Treatment
Conditions
Assessments
Participant
Exit
88
instructional system known as Army Excellence in Leadership (AXL). The goal of
AXL was to provide tools for developing the leadership and interpersonal skills of
junior leaders (Hill, Kim, Zbylut, Gordon, Traum, Gandhe, et al., 2006; Zbylut,
Metcalf, Kim, Hill, & Rocher, 2007). The AXL system allows for the authoring of
learning courses or modules that include embedded video content. Information can be
provided to the learner through static text alone, text plus graphics, text plus media
(such as sound or video clips), all with or without the use of questions. For example,
pages were created that contained only textual information while subsequent pages in
the same course contained text with graphics. A wide variety of question types can be
used in the AXL system including true-false and other forced-choice questions,
multiple-choice, open-ended (short-answer), and Likert-type rating scales. Responses
to these questions are automatically recorded as a string of responses tied to an
alphanumeric identifier string and stored in a central server for later analysis. This
information about user performance can be extracted from the AXL server remotely in
a variety of data file formats. In this project, files were exported as comma-delimited
files.
The AXL system also allows for response-dependent feedback to the learner,
developer control of learner navigation (either free navigation, controlled navigation
or some combination thereof), and seamless linking between multiple courses or
modules. Response-dependent feedback is embedded in the question response choices
so that when a particular response is given, the feedback assigned to that response
choice appears on the user’s computer display. Short-answer questions cannot be
89
analyzed at the time of entry, but are captured for subsequent analysis. Nonetheless,
when short answer questions are submitted to the AXL system, text that the course
developer embeds in the question appears on the display as in forced-choice question
types. For this project, practice question types used during portions of the instruction
included: multiple-choice, true-false and short answer. For the summative assessment
and self-efficacy scale, these question types were supplemented with several rating-
scale questions. Although the AXL system allows the developer to compel users to
provide answers to all questions in order to progress through the course, this project
allowed for questions to be skipped so as to avoid participant coercion and comply
with human subjects protection requirements of the University.
A number of short films, also created by ICT, demonstrating leader-
subordinate interactions were made available for this study. These films featured three
sets of characters and storylines, all of which were based in a military context
generically situated in Middle Eastern, tribally organized nations. The characters
demonstrate superior-subordinate relationships and interactions over the course of
three complex and dangerous military missions that were based on events drawn from
actual military maneuvers. Short segments from longer films were selected for
demonstration and practice purposes based on their match to the concepts described in
SLT. For example, a clip of a superior officer using a harsh tone of voice to give
orders and refusing to accept questions or feedback from his subordinates was used to
demonstrate the concept of directive behaviors.
90
Summative assessment
A summative assessment with twenty-three questions was developed to
measure the learning of declarative (facts, concepts) and procedural (how-to)
knowledge related to SLT. The assessment was designed so as to contain several
subscales - verbal recognition, recall, and video recognition. True-false questions (9)
captured the participants’ ability to identify directive and supportive behaviors as well
as leadership styles. Multiple-choice questions (13) captured the participants’ ability to
identify leadership styles and developmental levels, both in written vignettes and
video-based scenarios. Four video segments that had not been seen previously by
participants in any of the three conditions were followed by questions requiring the
participant to correctly perform the procedure of identifying subordinate readiness
levels and selecting appropriate leadership styles based on the prescriptive matches
recommended by SLT. Test items can be found in Appendix C.
Self-efficacy Scale
Following the procedure outlined by Bandura (2001), a Likert-type (Likert,
1932) rating scale for efficacy beliefs was developed for this study. The scale asked
participants to rate their beliefs about their ability to perform a number of leadership-
related skills in future interactions with subordinates. For example, one item asked
participants to rate their ability to give supportive feedback to subordinates in a work
setting. Self-efficacy scale items can be found in Appendix D.
91
Variables
Variables were measured for participants in three broad categories:
demographic characteristics of participants, learning outcomes, and efficacy beliefs.
Each of these categories of variables is described below.
Demographic variables
Demographic variables collected from participants included: chronological
age, gender, prior leadership training, attained education level, hours spent using
computers weekly and current educational status. These variables and their codes are
found in Table 2, below. Age and gender information were collected to assess their
relationship to learning in online environments across conditions as well as to
understand possible influences of age and gender on efficacy for leadership following
instruction. Prior leadership training information was collected to account for the
influence of prior experience on the summative assessment and efficacy scales.
Similarly, the highest level of education attained and hours spent using computers
weekly were collected from participants to account for the possible influence of years
of education on and familiarity with computers on performance.
92
Table 2: Summary of Demographic Variables and Codes
Variable Name Code
Age in years (AGE) Integer values
Gender (GNDR) Male = 1; Female = 0
Prior leadership training (PRLDR) Yes = 1; No = 0
Highest level of education attained
(EDLVL)
< 12 years of education
Postsecondary courses or A.A.
B.A. or B.S.
M.A. or M.S.
J.D., PhD., M.D., etc
= 0
= 1
= 2
= 3
= 4
Hours spent weekly using computers
(CMPUS)
0-5 hours
5-10 hours
10-15 hours
15-20 hours
>20 hours
= 0
= 1
= 2
= 3
= 4
Current educational status (EDST) Undergraduate
Graduate student
Military (plebe, ROTC)
Other
= UG
= GS
= ML
= OR
Learning outcome variables
The summative assessment contains several subscales, based on task
requirements and knowledge types, each of which was coded as a separate variable. In
all six learning outcome variables were measured for participants: whole test score,
verbal recognition score, verbal recall score, video recognition score, classification of
leadership styles and classification of developmental levels. These variables and their
codes are found below in Table 3. The whole test variable provides a gross measure of
learning related to the course content, SLT. The recognition and recall variables were
developed to capture declarative knowledge related to course content. Learning of
declarative knowledge has been hypothesized by Anderson (1996a) to be less sensitive
to the principles of learning in media-based environments when compared to
93
procedural knowledge. The classification variables were developed to capture the
procedural knowledge targeted by SLT, namely the classification of subordinate
developmental levels and leader interaction, or leadership styles.
Table 3: Summary of Learning Outcome Variables
Variable Name Code
Whole test score (SCORE) = Σ (item
1…22
)
Verbal recognition (VRBRCG) = Σ (item
1…11, 21, 22
)
Verbal recall (VRBRCL) = Σ (item
12…20
)
Classification of leadership styles (CLSLDR) = Σ (item
15…17, 19…21)
Classification of developmental levels (CLSDEV) = Σ (item
12, 13, 14, 18, 20)
Efficacy belief variables
Efficacy beliefs were measured with a scale of self-efficacy for leadership
activities. Questions asked participants to rate their ability to perform tasks such as
determining the level of confidence demonstrated by a subordinate, helping others to
enjoy challenging tasks, getting teams to work together and determining a
subordinate’s willingness to perform a challenging task. These items were all rated on
a one-to-five scale with one indicating low levels of confidence or predicted skill and
five indicating a high level of confidence or predicted skill. To calculate the efficacy
variable, ratings for the ten items in the scale were averaged to give a general picture
of the individual’s efficacy beliefs pertaining to leadership tasks. This is summarized
in Table 4, below.
Table 4: Summary of Efficacy Belief Variable
Variable Name Code
Efficacy for leadership (EFBEL) = Σ (item
1…10
)/10
94
Procedure
An online experiment was designed to examine the relationship between
practice with corrective feedback and learning of content related to leadership theory.
After logging on to the study website and completing the informed consent procedures
(Appendix E), participants were randomly assigned to one of three groups using an
automated routing function in the AXL system. An Unguided (UnGuide) control
group received textbook reading on Situational Leadership Theory (SLT) with
instructions to read the chapter carefully and complete an assessment of declarative
and procedural knowledge. The second, or moderately guided group (ModGuide)
received the same textual information as well as segments of leadership vignettes from
the videos described above. The ModGuide group also received study questions
(Appendix F) that asked them to apply tenets of SLT as they watched the videos but
did not provide any feedback about their responses to the questions. In short, the
ModGuide group received an opportunity for structured practice, but without
feedback. Finally, a Fully Guided (FullGuide) group received all content delivered to
the first two groups (i.e., text, video, and questions) as well as corrective and
supportive feedback as they answered the practice questions (Appendix F). The
FullGuide group also completed a whole-task practice for the procedure of diagnosing
subordinate readiness and prescribing appropriate leadership styles.
Following instruction, all groups completed identical assessments of
declarative (fact-based) and procedural (how-to) knowledge. As noted above in the
description of study apparatus, the assessment included true-false questions that
95
tapped into declarative knowledge related to course content (e.g., characteristics of
subordinate readiness levels), multiple-choice questions of declarative and procedural
knowledge (e.g., judgment tasks of leader diagnoses) and application questions based
on a novel set of videos. The assessment videos were consistent with those used for
demonstration and practice with respect to general context, quality, and complexity of
interactions but were otherwise novel; the characters, events, and relationships had not
previously been presented to study participants. After completing the summative
assessment, participants completed a self-efficacy scale, described above.
Data collection occurred automatically, with results available for download by
the course designer through the AXL server. Data recorded included responses to all
demographic questions, responses to practice questions for the Mod Guide and
FullGuide groups, responses to summative assessment questions, and responses to the
self-efficacy rating scale. Because individuals in all treatment conditions completed
the same assessment instrument, the anonymous participant identification code was
recorded three times: for the condition, the summative assessment, and the efficacy
scale. This enabled the researcher to create one data file, organized by case number,
for use in subsequent analyses. Following completion of the efficacy scale,
participants were thanked for their participation, were reminded of the study’s
purpose, and directed to exit the online study.
96
CHAPTER IV: RESULTS
Analyses performed on the data collected included calculation of descriptive
statistics for all variables, completion of a series of pairwise t-tests to assess between-
group differences in performance, and completion of Ordinary Least Squares (OLS)
regression analysis to account for variance in scores and to calculate observed power
and effect sizes. The results of this analysis were then compared against the research
hypotheses for the study. Descriptive statistics for the data are presented first,
followed by t-test results, grouped by hypothesis. The final section presents the results
of the regression analyses, also grouped by hypothesis.
Descriptive statistics
Demographic variables
With random assignment to condition, there were nearly twice as many
participants in the UnGuide group as in each of the other groups. The UnGuide group
had twenty-eight (28) participants. The ModGuide and FullGuide group each had
sixteen (16) participants. The median age of participants was 31.5 years with a
minimum of 19 and a maximum of 53 years. More females than males participated in
the study (female = 38, male = 17). Participants having no prior leadership training
experience (n = 30) narrowly outnumbered those with prior leadership training
experience (n = 25). With respect to educational level the majority of participants
reported earned four-year degrees (69.6% B.A., M.A., post-grad). Seventeen
participants reported completing some college coursework (28.3%). Eight participants
(13.3%) reported receiving an Associate’s degree. Five participants (8.3%) completed
97
a Bachelor’s degree and twenty completed a Master’s degree (33.3%). Of the fifty-six
participants reporting educational attainment, six (10.0%) completed a post-graduate
degree (Ed.D., Ph.D., M.D., J.D.). Four participants (6.7%) did not report their
educational attainment level. Nearly half (n = 26; 43.3%) of participants reported
using a computer more than twenty hours per week. Six participants used a computer
between fifteen and twenty hours weekly (10%). Eight participants reported using a
computer between ten and fifteen hours weekly (13.3%) and eleven reported five to
ten hours of weekly use (18.3%). Finally, five participants reported using a computer
five or fewer hours per week (8.3%).
Assessment instruments summary statistics and reliability indexes
The total score for the knowledge assessment, with n = 60, had a mean of 15.6,
a median of 16 and an sd = 3.3. The standardized Cronbach’s alpha for this scale with
22 items was .65. This value is within the acceptable limits (cf. Schmidt, 1996) given
the fact that the scale addresses declarative and well as procedural knowledge.
Cronbach’s alpha assesses the degree to which test items are correlated. High values
are generally assumed to indicate that the scale’s items are highly predictive of or
intercorrelated with each other, which is generally interpreted as a sign that the scale
measures some construct reliably. This index assumes that scales are developed to
measure single constructs. In this study, the assessment instrument contains several
theoretically minimally related constructs, which would predict a relatively low value,
as has been observed (Schmidt, 1996, p. 351).
98
All subscales described below have n = 60. The verbal recall subscale, with 13
items had a mean of 10.1, with sd = 1.8. The standardized Cronbach’s alpha for this
subscale was relatively low, with α = .48. Deletion of items from this subscale would
not improve the alpha. The verbal recognition subscale with 8 items had a mean of 5
and sd = 1.7. This scale had a standardized Cronbach’s α = .44. The classification of
developmental levels subscale, with 5 items had a mean of 3.3 and sd = 1.2. This
subscale also had a relatively low standardized Cronbach’s alpha, with α = .37. The
low number of items in this subscale may have influenced the alpha. Deletion of items
would not improve the alpha. The classification of leadership styles subscale with 6
items had a mean of 3.2 and sd = 1.5. This subscale also had an α = .37.
Table 5: Summary Statistics
Instrument: Sample =
n
Items = n Cronbach’s α Mean (SD)
Summative assessment 60 22 0.65 15.6 (3.3)
Verbal recall 60 13 0.48 10.1 (1.8)
Verbal recognition 60 8 0.44 5.0 (1.7)
Classification - levels 60 5 0.37 3.3 (1.2)
Classification - styles 60 6 0.37 3.2 (1.5)
Tests of between-group differences: t-tests
Hypothesis 1a: Participants scores on the summative assessment will be predicted by
their assignment to treatment condition.
This hypothesis cannot be assessed directly because t-tests are not predictive.
They do, however, reveal whether or not the observed differences (if any) between
groups are statistically significant. Based on visual inspection of the correlation matrix
99
between independent and dependent variables (Appendix H), a series of t-tests were
run to assess whether the observed performance differences between groups on the
assessment measure as a whole were statistically significant with alpha = 0.05.
UnGuide’s mean score of 14.5 (sd = 3.25) was found to be significantly different (t =
3.62, two-tailed p < .01) than FullGuide’s mean score of 17.9 (sd = 2.1). Additionally,
participants in FullGuide significantly outperformed those in ModGuide (mean = 15,
sd = 3.5) with t = 2.8, p < .01). Although ModGuide participants marginally
outperformed those in UnGuide as shown by their mean scores, this difference was not
statistically significant with t = 0.48, p = 0.63. In sum, FullGuide participants
performed significantly better than either of the other two treatment conditions. The
mean score for FullGuide was higher, with scores less widely distributed than for
either other group.
Hypothesis 1b: Treatment condition will predict participant score on subscales
As above, t-tests cannot provide predictive information. However, t-test results
provide information about observed differences and can provide a general measure of
whether or not further analysis is warranted. For example, in the absence of significant
differences between any groups, there is little reason to proceed with further analysis
if the central research question is a function of assignment to condition. In this case,
the summative test is comprised of three classification subscales – verbal recognition,
verbal recall, and video recognition. For the verbal recognition (a declarative
knowledge task) subscale, no significant difference was found between the UnGuide
group (mean = 8.64, sd = 1.8) and FullGuide (mean = 9.33, sd = 1.3) with t = 1.3, p =
100
.191, ModGuide (mean = 9.25, sd = 2) and FullGuide (t = .136, p = .893) nor between
UnGuide and ModGuide (t = 1.0, p = .3). This result is consistent with research
hypotheses (H1b.iii) that predicted no significant difference between groups on
measures of declarative knowledge. For the recall subscale, no significant differences
were found between groups. UnGuide’s mean of 3.75 (sd = 1.6) was not significantly
different from either FullGuide (mean = 4.66, sd = 1.3) or ModGuide (mean 3.94, sd =
1.5), with all p-values greater than 0.10.
For the video recognition subscale, a significant difference was found between
the mean performance of participants in FullGuide when compared to either UnGuide
or ModGuide groups. The difference between FullGuide (mean 4.1, sd = .96) and
UnGuide (mean 2.1, sd = 1.4) yielded a t-test value of 4.8, with p < .000. The
performance difference between FullGuide and ModGuide (mean = 2.1, sd = .96) was
also significant, with t = 5.63, p < .000. No significant difference was found between
the performance of UnGuide and ModGuide 2, with t = .045, p = .965. Effect sizes
were also large, with Cohen’s d for the FullGuide v. ModGuide comparison = 2.01
and the FullGuide v. UnGuide comparison yielding a d = 1.6. The smaller effect size
coefficient for the latter comparison is a result of the larger standard deviation in
UnGuide compared to ModGuide. This finding is inconsistent with the predicted
outcome, and may be the result of the video recognition items calling upon procedural
knowledge (a classification procedure) as well as declarative knowledge (example
identification).
101
Finally, to assess hypothesis H1b.i and H1b.ii classification tasks were
grouped into two subscales reflecting the underlying course content: classification of
leadership styles or classification of developmental levels. For classification of
developmental levels, a significant difference in performance was found between
UnGuide (mean 2.5, sd = 1.1) and FullGuide (mean = 3.3, sd = .81) with t = 2.63 and
p < .012. This yielded an effect size of d = 0.83. Differences in performance between
UnGuide and ModGuide and between ModGuide and FullGuide were not statistically
significant for this subscale. For the classification of leadership styles, FullGuide
(mean = 5.2, sd = 1.1) participants performed significantly better than UnGuide (mean
= 3.4, sd = 1.5) and ModGuide (mean = 3.1, sd 1.4) participants. The difference
between UnGuide and FullGuide yielded a t = 4.1, with p < .000 and d = 1.4. The
difference between ModGuide and FullGuide yielded a t = 4.6, with p < .000 and d =
1.7. No significant difference was found between UnGuide and ModGuide for this
subscale. Although for the leadership classification subscale UnGuide perfomed better
than ModGuide, this difference was not statistically significant and may be related to
the unequal sample size in the two groups.
102
Table 6: Summary Table of t-tests
UnGuide mean
(sd)
ModGuide mean
(sd)
FullGuide mean
(sd)
t-test (p = n)
Total test 14.5 (3.3) - 17.9 (2.1) 3.6 (p < .01)
Total test 14.5 (3.3) 15 (3.5) - Ns
Total Test - 15 (3.5) 17.9 (2.1) 2.8 (p < .01)
Verbal
recognition
8.64 (1.8) - 9.33 (1.3) Ns
Verbal
recognition
8.64 (1.8) 9.25 (2) - Ns
Verbal
recognition
- 9.25 (2) 9.33 (1.3) Ns
Verbal recall 3.75 (1.6) 4.66 (1.3) Ns
Verbal recall 3.75 (1.6) 3.94 (1.5) Ns
Verbal recall 3.94 (1.5) 4.66 (1.3) Ns
Classification –
levels
2.5 (1.1) - 3.3 (0.8) 2.6 (p < .01)
Classification –
levels
2.5 (1.1) 3.1 (1.4) - Ns
Classification -
levels
- 3.1 (1.4) 3.3 (0.8) Ns
Classification -
styles
3.4 (1.5) - 5.2 (1.1) 4.1 (p < .00)
Classification –
styles
- 3.1 (1.4) 5.2 (1.1) 4.6 (p < .00)
Classification -
styles
3.4 (1.5) 3.1 (1.4) - Ns
Ns = no statistically significant result
Regression analyses
Background
Regression analysis generates equations that seek to predict future outcomes
for some variable of interest, the dependent variable, based on observed data for the
same variable and some set of predictor variables, the independent variables. This
equation takes the form:
Υ
i
= β
o
+ β
1
x
i
+ ε
i
, i = 1, N
When Υ
i
is the predicted value of the dependent variable for some future sample, β
o
is
the intercept term, or beginning value of the dependent variable, β
1
x
i
is the slope
103
coefficient that is the measure of the effect of an independent variable on the slope of
the regression line, and ε
i
is a term representing measurement error. Ordinary Least
Squares (OLS) regression is a common form of regression analysis that is grounded in
critical assumptions about the data. First, that the relationship between dependent
(DV) and independent variables (IV) in the equation is linear in nature. This is to say
that the quality of the predictions made by the OLS model deteriorates when the
relationship between DV and IVs is flat, or curvilinear, etc. In this study, these
relationships were assessed both through visual inspection of scatterplots with the DV
on the Y-axis and the IV on the X-axis and through correlation matrix inspection to
identify statistically significant correlations between DVs and IVs. Second, that the
DV is prone to measurement and other forms of error, and that this error term is a
random variable, with a mean of zero. In a related assumption, the opposite holds true
for the IVs, which are assumed to be without measurement error. Fourth, each IV must
not be able to be expressed as some linear combination of the other IVs in the model,
also known as collinearity. Next, the manner in which the error term varies across the
sample is assumed to be constant, a characteristic known as homoscedasticity. Finally,
the sample, in this case the study participants, must be meaningfully representative of
the population of interest. Direct assessment of the assumptions for each model was
completed for this study.
Preliminary analyses
To assess hypothesis H2, OLS stepwise regression analysis was completed.
Visual inspection of scatterplots (see Appendix G) revealed generally linear
104
relationships between the treatment condition variable, education level and the
outcome variables total test score, verbal recognition, verbal recall, classification of
leadership styles and classification of developmental levels.
A preliminary step in regression analysis is to inspect the correlation matrix for
all variables present in the hypothesized models. In the case of hypothesis H2, which
predicts performance differences as a function of individual-level variables, this is
particularly important as it allows for the identifications of significant correlations
between variables. These correlations form the backbone of regression models for
both predictive purposes and in the accounting for possible confounds to
interpretation. Analysis of the Pearson’s correlation matrix revealed significant (1-
tailed) correlations between the total test score (SCORE) and treatment condition (.41,
p = .001), SCORE and prior leadership experience (.299, p = .01), SCORE and
education level (.31, p = .008) and SCORE and computer use (.277, p = .02). As
would be predicted with random assignment to condition, the treatment condition
variable was not significantly correlated with any demographic variable. Prior
leadership experience, age, computer use and education level were all significantly
correlated with each other, which presents confounds to interpretation for forward, and
backward regression analyses. In each of these forms of regression analysis, the
variance would be assigned on a first-come, first-served basis, privileging those
variables entered first (and kept) in the equation. Stepwise regression presents a
reasonable alternative to this dilemma by setting criteria for entry into and removal
from the regression equation. This ensures that the variables remaining in the equation
105
account for significant variance over and above that accounted for by all other
variables in the equation, thereby reducing specification error and ensuring that those
variables retained not only account for variance but do so in a way that is meaningful
(Pedhazur, 1997, p. 225). In stepwise regression, over multiple iterations, each IV is
entered into the equation last so that candidates for removal (those not contributing
variance over and above that of all other IVs) can be identified. Variables not meeting
criteria are removed, leaving only those that explain the most variance in the clearest
way. This form of regression analysis was used to analyze data from the summative
assessment and its subscales as well as the self-efficacy scale. Analysis of the
summative assessment and its components will be described first, followed by the
efficacy scale.
Regression analyses
Hypothesis H1: Assignment to treatment condition will predict participant
scores on the summative assessment.
Hypothesis H2: Individual level variable will predict participant scores on the
summative assessment.
With multiple significant correlations between variables, a blockwise
regression model has an increased risk of assigning variance to variables that are in
fact not meaningful predictors of future outcomes. For this reason, a stepwise
regression analysis was completed with an entry criterion for y of F <= .05 and a
removal criterion for y of F >= .100. In short, this means that a predictor variable had
to pass a significance test to be kept, and to fail another to be removed. With this set of
106
constraints, the variables kept in the regression equation were the treatment condition
variable and the educational level variable. The final regression equation with these
variables accounted for approximately 35% of the observed variance in scores (R-
square = 0.35), a significant finding (F = 13.3; df = 2, 49; p < 0.000). This regression
equation is:
Y’
SCORE
= 12.8 + 1.5
COND
+ 0.78
EDLVL
+ ε
For this equation, t-values for the intercept and slope terms were calculated. For the
intercept term, t = 14.7, with p = .000. The slope coefficient for treatment condition
and for educational level each accounted for significant variance in the prediction
model ((β
0
t = 3.7, with p = .001; β
1
t = 3.3, with p = .002. The Durbin-Watson test
for autocorrelation of the residual term yielded a value of 2.02, indicating an absence
of autocorrelation. Collinearity diagnostic tests revealed an eigenvalue of 2.62,
indicating a lack of collinearity that would pose a threat to interpretation. Finally a
scatterplot of the standardized residual against the standardized predicted value
revealed a heteroscedastic distribution. Based on the entry criterion, variables
excluded from this model were age (t = 1.1; p = .26), gender (t = 1.1; p = .27), prior
leadership experience (t = .2; p = .85) and hours of weekly computer use (t = 1.3; p =
.2). Post-hoc power analysis with a p-value of .05 yielded an observed power of 0.999,
or a less than .01% chance of falsely identifying an effect that was not present. These
findings support hypothesis H1a, in that treatment group was a significant predictor of
outcomes for the total test as well as hypothesis H2a, in that education level was also a
significant predictor of outcomes. Hypotheses H2b and H2c were not supported, as
107
neither prior leadership training nor weekly hours of computer use accounted for
significant score variance. Hypotheses H2d and H2e were supported, as chronological
age and gender did not account for significant score variance for the summative
assessment.
Declarative knowledge subscale regression analysis
Results from a stepwise regression analysis of the declarative knowledge
subscale indicate that prior leadership training alone accounted for a significant
amount of variance in the observed data across all treatment conditions. This is
consistent with the absence of significant differences observed in the pair-wise t-tests
described above. Although statistically significant, prior leadership training accounted
for only 8% of the observed variance in subscale scores (R-square = .085; F = 4.6; p =
.04). As in the total test regression analysis, the intercept term was significantly
different than zero (β
0
= 8.7; t = 30.4; p < .000). Prior leadership experience improved
scores for those with such experience by β
1
= .9, with t = 2.2 and p = .04. The Durbin-
Watson value for this model of 2.4 indicates an absence of autocorrelation and the
eigenvalue of 1.7 indicates no collinearity, which is to be expected given the single
predictor variable for this model. Last, a scatterplot of the residual against the
predicted value indicates a heteroscedastic distribution of the residual. Variables
excluded from this model were: treatment condition, age, gender, education level and
hours of weekly computer use. The resulting regression equation is:
Y’
DECL
= 8.7 + .9
PRLDR
+ ε
108
Power analysis for this model with a p-value of .05 yielded an observed power of 0.66,
or a relatively large risk of falsely identifying an effect when none exists.
Procedural knowledge regression analysis
Results from a stepwise regression analysis of the procedural knowledge subscale
revealed that the treatment condition and educational level variables accounted for
significant variance in observed scores. The variables accounted for approximately
32% of the observed variance in procedural knowledge subscale scores (R-square =
.32; F = 11.8; p <.000). The coefficients for the intercept term and the variables were
statistically significant (β
0
= 4.4, t = 7, p < .000; β
COND
= 1.2, t = 4, p < .000; β
EDLVL
=
.43, t = 2.4, p < .02). For this equation, the Durbin-Watson value of 1.6 indicates some
risk of autocorrelation. Although no rule of thumb should be used for the assessment
of threats to regression assumptions regarding collinearity, the very small correlations
between independent variables in concert with collinearity diagnostic s indicate little
or no collinearity (eigenvalue = 2.6) (Pedhazur, 1997 p. 309). A scatterplot of the
residual against the predicted value reveals a heteroscedastic distribution of the
residual term. Age, gender, prior leadership training and hours of weekly computer
use were excluded from this model. Post-hoc power analysis with a p-value of .05
yielded an observed power of 0.998, or a less than .01% chance of falsely identifying
an effect that was not present (Cohen, 1988). The resulting regression equation is:
Y’
PROC
= 4.4 + 1.2
COND
+ .43
EDLVL
+ ε
109
Verbal recognition subscale regression analysis
Inspection of the correlation matrix for this subscale and all predictor variables
revealed significant correlations between verbal recognition and treatment condition (p
= .001), (p = .01) and hours of weekly computer use (p = .02). In a stepwise regression
analysis with criteria as above, the final regression model included only treatment
condition and education level variables accounting for significant variance in the
observed data (F = 8.2; p = .001; R-square = .25). The eigenvalue of 2.6 for this model
revealed no meaningful collinearity between predictor variables. The Durbin-Watson
value of 1.6 indicates no severe threats to interpretation from intercorrelation. The
resulting regression equation is:
Y’
VRCGN
= 4.4 + .93
COND
+ .41
EDLVL
+ ε
The regression coefficients in this model were all significant (COND t = 3.0, p = .003;
EDLVL t = 2.4, p = .02). Visual inspection of regression against predicted residual
plot revealed a heteroscedastic distribution. Post-hoc power analysis with a p-value of
.05 yielded an observed power of 0.98, or a less than 0.2% chance of falsely
identifying an effect that was not present.
Verbal recall subscale regression analysis
Inspection of the correlation matrix for this subscale revealed significant
correlations between verbal recognition and treatment condition (p = .006), (p = .004)
and hours of weekly computer use (p = .04). A stepwise regression analysis with all
predictor variables assessed using the criteria described above yielded a regression
equation with education level and treatment condition alone accounting for significant
110
amounts of variance in the observed data (F = 7.6, p = .001; R-square = .24). The
Durbin-Watson value for this model is 2.4, an indication of no threats to the regression
assumption of no intercorrelation. Collinearity diagnostics indicated that the
assumption of no collinearity was not violated in the data (eigenvalue = 2.6). Post-hoc
power analysis with a p-value of .05 yielded an observed power of 0.98, or a less than
0.2% chance of falsely identifying an effect that was not present. The resulting
regression equation is:
Y’
VREC
= 8.3 + .4
EDLVL
+ .65
COND
+ ε
The regression coefficients for this equation were statistically significant (EDLVL t =
2.7, p = .01; COND t = 2.6, p = .01). Visual inspection of regression against predicted
residual plot revealed a heteroscedastic distribution.
Classification subscales regression analysis
Finally, stepwise regression analyses were completed for the two classification
subscales – classification of leadership styles and classification of developmental
levels. For the developmental level classification subscale, the treatment condition
alone accounted for a significant amount of observed variance (R-square = .14, F =
8.9, p = .004). The parameter coefficients for this model were statistically significant
(β
COND
= .51, t = 2.9, p = .004), resulting in the following regression equation:
Y’
CLASSDEV
= 3.5 + .51
COND
+ ε
The Durbin-Watson value of 1.8 for this model indicates no threat to interpretation
from autocorrelation. Collinearity diagnostics revealed no collinearity (eigenvalue =
1.9), again to be expected with single predictor variables. Finally, visual inspection of
111
the scatterplot of residual against predicted values indicates a heteroscedastic
distribution of the residual term. Post-hoc power analysis with a p-value of .05 yielded
an observed power of 0.87, or approximately equivalent risks of falsely identifying an
effect that was not present and failing to detect a true effect.
For the leadership styles classification subscale, the treatment condition and
hours of weekly computer use variable accounted for a significant amount of observed
variance in scores (R-square = .27, F = 9.9, p < .000). The parameter coefficients for
this model were statistically significant (β
COND
= .82, t = 3.6, p = .001; β
COMPUS
= .35, t
= 2.7, p = .01), resulting in the following regression equation:
Y’
CLASSVID
= 1.4 + .82
COND
+ .35
COMPUS
+ ε
The Durbin-Watson value for this model of 1.7 indicates a very low risk of threats to
interpretation from autocorrelation. The eigenvalue of 2.7 indicates no threatening
collinearity. Finally, inspection of the scatterplot of the residual term against predicted
values indicates a heteroscedastic distribution of the residual term. Post-hoc power
analysis with a p-value of .05 yielded an observed power of 0.99, or a less than 0.1%
chance of falsely identifying an effect that was not present.
Self-efficacy scale
Hypothesis H3: treatment condition will predict self-efficacy scale scores.
The efficacy scale for this project was not completed by all participants,
yielding a sample with n = 30. The mean value for this measure was 34.5 (sd = 5.4),
with a possible range of 0-50, with higher values indicating more confidence in
abilities to apply SLT principles. The distribution of scores for this variable was
112
approximately normal in shape, with a strong peak at the modal value of 35.
Inspection of scatterplots of the efficacy scale scores against predictor variables did
not clearly reveal linear relationships between DV and IVs. Focusing on the variables
that served as predictors for the total score, treatment condition, prior leadership
training and education level, inspection of the correlation matrix revealed no
significant correlations between efficacy scale results and hypothesized predictor
variables. Entering all variables into the regression equation did not yield a significant
accounting of variance in scores (R-square = .04; F = .38; p = .77). With all variables
entered as predictors, significant collinearity between prior leadership experience and
educational level was revealed (eigenvalue = 3.2). This result does not appear to have
been affected by homoscedasticity, as inspection of the scatterplot of the residual
terms against predicted values revealed a heteroscedastic distribution.
113
CHAPTER V: DISCUSSION
The act of instruction can take many forms and is always bound by its content
and context. The same instructional method might be more or less effective for a
particular content area, or across instructional contexts. This study examined the effect
of guidance on mastery of declarative and procedural knowledge related to leadership
in an online instructional environment. Within this instructional context, a long-
standing debate centers on the question of guidance. Are learners likely to master the
content when left to puzzle their way through the content as some constructivist
scholars would suggest, or are they in fact more likely to master content when
provided with deliberate, structured practice with corrective feedback? By establishing
three learning conditions with varying degrees of such guidance, this experiment
provides clear evidence that for the instructional content in question and in the context
of web-based learning, more guidance has the effect of improving performance. While
the effects of guidance on performance are powerful and theoretically meaningful, the
picture of these effects is complex. To present this relatively complex picture, the
three general sections of this chapter will first give a general interpretation of the
results of the experiment, treating each research hypothesis in turn. Next, the
implications of these findings will be discussed in three sections. Implications related
to theory, implications related to practice and implications of this study for future
research. Finally, the chapter concludes with a discussion of the ways in which the
results support a particular framework for instructional design and a reiteration of the
conclusions that can be drawn from this project.
114
Discussion of research hypotheses
Three general research hypotheses were developed for this study. The first set,
pertaining to the effect of guidance on summative assessment achievement, presents a
mixed picture of results. To review, the first hypothesis was:
H1: Study participants in the treatment condition with the most instructional
guidance will achieve higher scores on the summative assessment than those in
either other treatment condition.
H1a: Participants scores on the summative assessment will be predicted
by their assignment to treatment condition.
i. Participants in the maximum guidance condition will achieve
higher scores than participants in the practice-without-feedback
condition (supported).
ii. Participants in the maximum guidance condition will achieve
higher scores than participants in the minimal-guidance
condition (supported).
iii. Participants in the practice-without-feedback condition will
achieve higher scores than participants in the minimal-guidance
condition (supported).
This set of research hypotheses was, in broad terms, supported by the data. In spite of
the differences in size between the minimal guidance and other conditions, a test of the
difference between the mean scores of the UnGuide and FullGuide groups revealed a
statistically significant difference (t = 3.62, p < .01) between the groups. The
115
FullGuide group, on average, performed better than the UnGuide group on the
summative assessment as a whole, and the observed difference between the means
could be expected to happen by chance only one time in a hundred. Similarly, the
FullGuide group’s mean performance was significantly better than the ModGuide
group’s mean (t = 2.8, p < .01). These results suggest that on average, those
participants who received the most instructional guidance performed better than other
groups in a way that would not be expected to occur by chance alone.
Upon closer examination, this broad picture is complicated by results that
differ based on the specific subscale. This can be seen in hypotheses H1b, below:
H1b: Treatment condition will predict participant scores on subscales.
i. Participants in the maximum guidance condition will achieve higher
scores than participants in the practice-without-feedback condition.
ii. Participants in the maximum guidance condition will achieve higher
scores than participants in the minimal-guidance condition.
For H1b.i., with respect to the declarative knowledge subscales – verbal recall and
verbal recognition – the data do not support the conclusion that treatment condition
has a significant effect on the outcome measure. No significant differences were
detected between groups for these declarative knowledge tasks. This may be, as
suggested by Anderson (1996a), due to the nature of declarative knowledge
acquisition. Guidance during practice may be less helpful when committing facts and
concepts to memory than it is when learning how to complete a procedure. With
respect to the procedural knowledge subscales, classification of developmental levels
116
and leadership styles, the group that received the most guidance did in fact outperform
the groups that received less guidance. For the classification of developmental levels,
the FullGuide group’s performance mean was significantly higher than that of the
UnGuide group (t = 2.63, p < .012). There may have been a meaningful benefit from
practice, however, as the difference in performance between the ModGuide and
FullGuide group means was not statistically significant. For the classification of
leadership styles, hypothesis H1b.i. was supported by the data. The difference between
the FullGuide and UnGuide groups was statistically significant (t = 4.1, p < .000) as
was that between FullGuide and ModGuide (t = 4.6, p < .000). No significant
difference was found between the ModGuide and UnGuide groups, which suggests
that for the classification of leadership styles, simply being given the opportunity to
practice (the ModGuide condition) does not confer the same performance benefits.
The comparisons between groups described above suggest that the observed
mean score differences between groups are meaningfully different when one examines
the assessment as a whole and the procedural tasks contained within the assessment.
Declarative knowledge, in the form of recall and recognition tasks, was not sensitive
to the guidance treatment when we compare results by group. Between-group
comparisons leave the more important issue of meaningfulness unaddressed.
Regression analysis can be helpful in addressing this question as it provides estimates
of the amount of observed differences in scores that can be accounted for by including
particular variables in a regression equation that predicts future performance
outcomes. In this study, the stepwise regression analysis revealed multiple statistically
117
significant and meaningful predictor variables, and will therefore be discussed in the
context of hypothesis H2, which addresses the effects of individual-level variables.
Effects of individual-level variables
Several individual-level variables accounted for significant amounts of
variance in observed scores, in support of research hypothesis H2. As with hypothesis
H1, the picture is complicated, with some predictor variables having effects that are
stable across subscales, others that have effects only on particular subscales and yet
others that were predicted to have stable effects but were observed to have no effects
on observed scores. Again, the broadest level of the research hypothesis was supported
with education level, prior leadership experience and hours of weekly computer use
accounting for significant amounts of observed variance for one or more subscales.
Predictor variables hypothesized to have no relationship to observed scores – age and
gender, for example – were found to not account for significant variance in observed
scores as described above. Generally, the predictors with the most stable effects were
the treatment condition to which an individual had been assigned and the participants’
educational level.
H2: Individual-level variables will predict participant scores on the summative
assessment.
H2a: Participants with higher education levels will achieve higher
scores on the summative assessment when compared with participants
having lower levels of education (supported).
118
H2b: Participants with prior leadership experience will achieve higher
scores on the summative assessment than those without prior leadership
training (not supported).
H2c: Participants with more weekly hours of computer use will achieve
higher scores on the summative assessment than those with fewer hours
(not supported).
H2d: Chronological age will not predict performance on the summative
assessment (supported).
H2e: Gender will not predict performance on the summative
assessment (supported).
H3: Treatment condition will predict self-efficacy scale scores.
H3a: Participants in the maximum-guidance condition will report great task-
specific self-efficacy than participants in the minimal-guidance condition.
As described in Chapter 4, the self-efficacy scale scores were not meaningfully
predicted by any of the independent variables included in data collection procedures. It
is apparent that the severe incompleteness of the data set with respect to the self-
efficacy scale may have contributed to this null finding. It may also be the case that
the research hypothesis itself had an authentic null result for this population and that
the missingness in the data did not affect the results. Interpretation of this null finding
must therefore go beyond the expected absence of support for the hypothesis to an
absence of support for any single interpretation of the data due to the lack of
119
significant correlations between IVs and the DV, the lack of meaningful predictors of
the DV, and the severity of the missingness of the data itself.
Discussion of implications
Theoretical implications
The results of this study have three key implications for theory related to
instructional design. First, in the ongoing discussion of the role of guidance during
instruction, the findings of this study strongly supports guided instruction. Second,
with respect to theories of learning from media, the results of this study suggest that it
is not the media that matters, but the implementation of structured practice with
feedback that improves performance. Finally, the results suggest that completing more
years of postsecondary education confers performance benefits during online
instruction.
Many researchers (Anderson, 1993; Merrill, 2002a, 2002b; Sweller & Sweller,
2006) make a distinction between declarative and procedural knowledge types in
human learning. This study measured these different forms of knowledge in an
attempt to understand the relationship between knowledge type and guidance in an
online learning environment. The results provide compelling evidence that for
procedural knowledge, in this case accurately completing a classification procedure,
guidance provides strong instructional benefits to learners. All participants in the study
performed at a level better than chance on the summative assessment, with the
strongest performance coming from those in the treatment condition receiving the
most practice and feedback during instruction. A closer examination of the results
120
reveals that for the declarative knowledge content – the concepts of Situational
Leadership Theory, making judgments of whether or not an action was an example of
these concepts, etc. – there were no meaningful differences between the three
treatment conditions. This suggests that video-based practice, with or without
corrective feedback, does little to improve performance on the declarative knowledge
storage-and-recall patterns. In a sample of adults, this result is not surprising, as
normally functioning adults can be expected to have well-developed automated
systems for storage and retrieval of novel information that are highly resistant to
modification. More interesting is the result with respect to procedural knowledge. For
the novel classification procedure, for which participants had no direct prior
experience, practice with corrective feedback conferred important performance
benefits. Practice without feedback was not meaningfully superior to the text-only
condition, providing further support for the theoretical stance that guidance is crucial
for the development of procedural knowledge, particularly when successful solutions
to the problem (in this case, selecting an appropriate leadership style) have been
identified.
The results of the study also contribute to theories of learning from media. As
described in Chapter 2, some scholars continue to assert that the use of media for
content delivery confers benefits over and above instructional design. This assertion
was contradicted by the findings of this project. In this study, the ModGuide treatment
group saw the same video vignettes as the FullGuide group and was given the same
practice questions. The stronger performance for procedural knowledge exhibited by
121
the FullGuide group suggests that the instructional practice of corrective feedback,
rather than viewing video vignettes, was the source of the performance improvement.
This is consistent with findings from Clark (in press), Mayer (2004), and Moreno and
Mayer (1999).
Third, in keeping with a long line of research linking years of education with a
range of performance benefits (Bowen, 1977), this study suggests that years of
education also confer performance benefits for online instructional environments.
Results indicated that completing years of postsecondary education was correlated
with improved performance.
Practical implications
This study of learning with a web-delivered instructional module also has
several practical implications for the instructional designer and researcher. First, the
results suggest that for procedural knowledge development, practice with corrective
feedback is a superior strategy to both practice without corrective feedback and to
text-based instruction without practice. Second, the results also suggest that using high
quality video vignettes to illustrate concepts and provide practice opportunities is more
effective than using text alone during instruction.
As described above, the performance benefits from practice with corrective
feedback contribute meaningfully to instructional design theory. This result also
contributes to instructional practice in online learning. The three experimental
conditions reflect three types of online learning structures frequently encountered in
distance learning programs (Bernard et al., 2004). The first condition (UnGuide)
122
mirrors a read-and-click design in which learners read information displayed on a web
page before clicking through to the next page of text. The second (ModGuide)
condition includes high-quality video to supplement the text as well as presenting
practice questions. This method is less common (Bernard et al., 2004) but some
combination of text and video can be found with increasing frequency in web-based
instruction. Finally, the third condition (FullGuide) reflects the current state of the art,
in which learners receive corrective, response-contingent feedback during practice.
The significantly stronger performance of participants in the FullGuide condition
suggests that, whenever development funds are available, response-contingent
corrective feedback should be implemented in online instruction.
Closely related to this implication is that text-based instruction supplemented
by high-quality video demonstrations is superior to text-only instruction in online
courses. For instructional designers, the practice of presenting information in a
multimedia, or audio-visual, display has a long record of empirical support (Mayer,
2004). This study’s results are consistent with this empirical record and provide
further impetus for the use of multimedia instructional designs, particularly in web-
based delivery contexts.
Implications for future research
This project has a number of implications for future research in this area. The
most important of these is that randomized, web-based experiments in the area of adult
learning are feasible. Although there are challenges inherent in web-based data
collection, which will be discussed below, the clarity of the results of experiments
123
with random assignment to treatment condition far outweigh these challenges. In the
literature examining the psychology of education, quasi-experiments continue to be
the standard of practice. After many years of calls for wider (though not exclusive) use
of ‘true’ experiments, web-based content delivery platforms present a reasonable
option for conducting such studies. The platform used in this experiment is capable of
presenting a wide range of question types (including open-ended questions), all of
which may be tied to course content, whether text-based, video-enriched or some
combination thereof. Feedback in this platform can be generic or response contingent.
These features allow for the development of an extremely wide range of experiments
that examine the relationship between content delivery, supportive feedback, structure
and guidance, and aspects of media and learning outcomes. Additionally, the
population from which web-based experiments can draw is also much broader than the
pool of individuals who are able to complete experiments in a lab or campus-based
study using computers (Saba, 2000). Samples of participants in web-based
experiments are theoretically more representative of the general population than those
samples that draw from traditional campus-based experimental participant pools.
Samples that are more representative of the population of interest are desirable as they
increase generalizability and validity of findings.
With respect to the results themselves, this study indicates that unpacking the
differences between declarative and procedural knowledge in studies of learning may
bear fruit in future experiments. When declarative and procedural knowledge tasks are
combined into a single measure, the net result may be a null finding when an
124
underlying performance difference for procedural knowledge is present. In this study,
the between-group performance differences were significant for the full assessment as
well as for the procedural subtasks. Differences were not meaningful for declarative
tasks, which points to the necessity for teasing apart knowledge types when analyzing
and interpreting results of such studies. For scholars who examine similar tasks in the
future, such differentiation of knowledge types appears to be essential to avoid the
possible confound of unique skill acquisition patterns based on type of knowledge.
This study provides evidence that such distinctions are critical to the design of
experiments in the area of learning.
Finally, although the differences were not part of the research hypotheses for
this study, an interesting pattern of performance for the classification procedure was
observed that has bearing on future research on leadership. Across conditions,
participants were much more accurate in identifying leadership styles and behaviors
than they were at categorizing the developmental level of subordinates. This
difference would be likely to adversely affect compliance with the taxonomy of SLT
that requires accurate assessment of subordinate readiness levels in order to determine
the appropriate leadership style. More generally, further study on the leadership
classification task is warranted because of this unexpected finding. If learners are
unable to correctly classify subordinate readiness levels using the principles of SLT,
then the theory itself may be flawed or the instruction that is based on SLT may be
incomplete.
125
Limitations
All research projects have limitations that must be held in mind when
interpreting the results of the study. For this project, there are several categories of
limitations that will be described in this section. First, there are limitations related to
web-based experimentation and sampling for this study. Second, there are limitations
that are based on the dataset itself. Third, there are limitations related to course content
and instructional materials. Finally, there are limitations related to the assessment
tasks.
Web-based experimentation and sampling limitations
Web-based studies must be interpreted with caution because there is no ready
means for identifying malingering participants who seek to confound the study or
those who enroll as study participants on multiple occasions. For this study, the
recruitment flyer clearly states the nature and importance of the research, but in order
to protect the anonymity of participants, no identifying information linking them to a
stable Internet service provider (which would enable reverse tracking of individuals)
was collected. Because no such information was included in the dataset, at the most
basic level there is a possibility, hopefully quite remote, that participants with devious
intentions affected the results. In this project, the only individual-level variable that
was shown to have a significant predictive relationship with performance was attained
education level and it is possible that participants misrepresented their attainment
either through a desire to appear more educated than they were or through a
misinterpretation of the categories. For example, an individual who was working
126
toward a Master of Arts degree might indicate that they had already received such a
degree. That said, the primary effect was that of treatment condition, which was the
result of random assignment and therefore not subject to misinterpretation or
misrepresentation.
The sampling procedures for this study may have affected the results. A variety
of sampling techniques were implemented to accrue the eventual sample with n = 60.
As stated in Chapter 3, these sampling procedures included direct recruitment through
personal solicitation, indirect recruitment through distribution to lists of email
addresses, and indirect referral through “snowball sampling” in which the participants
themselves recruit additional participants. All of these sampling strategies are prone to
cohort effects, which may distort the results of a study. In this project, random
assignment to condition can be expected to mitigate the risk of cohort effects.
However, random assignment is not a panacea and the possibility of a distorted or
otherwise poorly representative sample remains a possibility.
Limitations of the dataset
The most important limitation to this dataset is the severe pattern of
missingness for the self-efficacy scale, which was to be completed after the summative
assessment. Fewer than half of the participants completing the summative assessment
are represented in the efficacy data, making interpretation of the null result of the
efficacy-assessment hypothesized relationship impossible. Second, the data for this
project is cross-sectional in nature. Each record of performance is a snapshot of that
individual’s skills on the day and time of sampling and may inaccurately represent that
127
same individual’s true score. In an ideal experiment, each participant would have a
unique login identification and password. In contrast to the anonymous login used for
this experiment, a participant-specific login procedure would allow the experiment to
be designed in such a way as to collect performance data over a number of occasions
of measurement. Multiple measurement designs would allow for additional task
performance data to be collected. This data set would enable the researcher to analyze
the data using more complex statistical models that may more accurately represent
real-world performance.
Additionally, the dataset did not record the amount of time spent in each
course, only the time at which the participant was assigned to a condition and the time
at which the same individual began the assessment. Because no time codes are
available for the individual web pages of content within each course, the possible
effects of study time on performance are absent from the data. In future studies, this
could be easily corrected by including time stamps for each page of the instructional
content as well as measurement instruments.
Finally, a small number of demographic variables were collected. Although
only one of these, attained educational level, was significantly related to performance
across assessment subscales, the possibility of specification error in the model
remains. If the model used for this study was in fact incorrectly specified, there may
be other unmeasured variables to which the performance effects should be attributed.
For example, other variables shown to have relationships to performance that were not
collected include fluid intellectual ability (G-f) and parental education level. Fluid
128
ability is a complex characteristic to capture empirically, and participants may be
unaware of their fluid ability or reluctant to report such socially sensitive information.
Similarly, parental educational attainment levels were not collected in order to avoid
placing the participant in an uncomfortable or emotionally sensitive situation.
Limitations related to course content and instructional materials
The content for this course was drawn from a textbook used in a course taught
at an institution of higher learning and shares any weaknesses that the textbook might
have
7
7
For a discussion of the limitations of Situational Leadership theory, the reader is
directed to: Vecchio, Bullis, & Brazil (2006).
. The explanations of concepts and procedures may have been more effective for
some participants than others for reasons unrelated to participant ability. The
instructional videos in the course were developed to demonstrate leadership principles
and intercultural interactions, but were not designed as demonstrations of the specific
concepts and procedures of Situational Leadership Theory. For this reason, some of
the video vignettes chosen may have functioned more or less well for individual
participants – acting as clear demonstrations for some and obfuscations of concepts for
others. The military context of the videos may also have functioned as a distraction for
some learners who are unfamiliar with the sights and sounds of military campaigns.
These distractions may have been a source of extraneous cognitive load and decreased
performance on the summative assessment. With the results of this study, if such
negative effects of the video were present, the logical interpretation of similar but
more effective videos would be to increase the performance differences for procedural
knowledge tasks between the UnGuide and the ModGuide and FullGuide groups, both
129
of which outperformed UnGuide and received video demonstrations. In future studies,
starting first with the instructional design and then developing the demonstration video
content can address this possible weakness.
Limitations of assessment tasks
Although carefully developed based on the assessment tasks from the live
course from which the content for this project was drawn, the assessment tasks may
have poorly or unreliably measured participant performance. The summative
assessment and its subscales were found to have generally acceptable indexes of
reliability, but because of the relatively small number of items and relatively small
sample, these reliability estimates may themselves be less than completely reliable.
Future samples from the same population might yield different reliability estimates,
which would confound interpretation of the results of this study. The possibility that
such an event might occur should be considered when interpreting the results of this
study. Further, apart from issues of reliability, the assessment tasks may also be less
than completely valid. This is to say that the measures may ineffectively capture the
participants’ knowledge of the course material and ability to complete the leadership
tasks required in the assessment. Because the data for this study are cross-sectional
and anonymous, requesting participants complete additional leadership tasks at a later
date (which would address the validity of the tasks used) was not possible. This
limitation should be considered when interpreting results.
With a larger sample, an improved understanding of the measurement
instrument’s characteristics could be developed using the principles of item response
130
theory (Wilson, 2005). With a sample of sixty participants, the item and test
characteristic estimates are very poor due to instability (Wilson). Future samples could
be combined with the data from this study to develop a sufficiently large population of
respondents.
Conclusions
The purpose of this dissertation project was to answer a straightforward
question using an experimental design: What are the effects of guidance on
performance of a procedure related to leadership skills in an online learning
environment? The study also explored differences between participants’ scores on
assessments of fact-based or declarative and task-based or procedural knowledge to
determine if these differences can be attributed to participation in specific treatment
conditions, differences of educational level, gender, chronological age, and/or prior
leadership training experiences. With respect to the first of these purposes, the effect
of structured practice and corrective feedback (the maximum guidance condition) on
performance of the classification task was to improve performance overall. The
differences between treatment conditions for the summative assessment were
statistically significant and favored participants assigned to the maximum guidance
treatment. These differences were identified through a series of pairwise t-tests that
revealed meaningful benefits in terms of performance for the maximum guidance
(FullGuide) condition and no meaningful differences between the other treatments that
had no practice (UnGuide) and/or no corrective feedback (ModGuide). Regression
analysis revealed that the best predictor of performance was the treatment condition,
131
with educational level also serving as a statistically significant predictor. Other
demographic variables, such as prior leadership training experience, gender, age, and
hours of weekly computer use did not meaningfully predict performance across
measures. The hypothesized relationships between performance on the summative
assessment, treatment condition and efficacy beliefs regarding future leadership
performance could not be evaluated because of severe data loss for the efficacy scale.
132
REFERENCES
Albanese, M., & Mitchell, S. (1993). Problem-based learning: A review of the
literature on its outcomes and implementation issues. Academic Medicine, 68,
52-81.
Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Lawrence
Erlbaum.
Anderson, J. R. (1996a). The architecture of cognition. Mahwah, NJ: Lawrence
Erlbaum.
Anderson, J. R. (1996b). ACT: A simple theory of complex cognition. American
Psychologist, 51, 355–365.
Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y.
(2004). An integrated theory of the mind. Psychological Review, 111(4), 1036-
1060.
Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Mahwah,
NJ: Lawrence Erlbaum.
Anderson, J. R., Qin, Y., Jung, K.-J., & Carter, C. S. (2007). Information-processing
modules and their relative modality specificity. Cognitive Psychology, 54, 185-
217.
Anderson, J. R., Reder, L. M., & Simon, H. A. (1996). Situated learning and
education. Educational Researcher, 25(4), 5-11.
Anthony, W. S. (1973). Learning to discover rules by discovery. Journal of
Educational Psychology, 64, 325–328.
Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its
control processes. In K. W. Spence and J. T. Spence (Eds.), The psychology of
learning and motivation: Advances in research and theory (vol. 2). New York:
Academic Press.
Awh, E., Jonides, J., & Reuter-Lorenz, P. A. (1998). Rehearsal in spatial working
memory. Journal of Experimental Psychology: Human perception and
performance, 24(3), 780-790.
Ayres, P. (2006a). Impact of reducing intrinsic cognitive load on learning in a
mathematical domain. Applied Cognitive Psychology, 20(3), 287-298.
133
Ayres, P. (2006b). Using subjective measures to detect variations of intrinsic cognitive
load within problems. Learning and Instruction, 16(5), 389-400.
Baddeley, A. D. (1986). Working memory. Oxford: Oxford University Press.
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change.
Psychological Review, 84, 191-215.
Bandura, A. (2001). Guide for constructing self-efficacy scales. In G. V . Caprara
(Ed.), La valutazione dell 'autoeffcacia [The assessment of self-efficacy], pp.
15-37. Trento, Italy: Erickson
Bell, B. S., & Kozlowski, S. W. J. (2002). Adaptive guidance: Enhancing self-
regulation, knowledge and performance in technology-based training.
Personnel Psychology, 55(2), 267-306.
Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L.,
Wallet, P. A., Fiset, M., & Huan, B. (2004). How does distance education
compare with classroom instruction? A meta-analysis of the empirical
literature. Review of Educational Research, 74(3), 379-439.
Bloom, B. S. (1956). Taxonomy of educational objectives: Book 1, cognitive domain.
New York: Longman.
Bloom, B. S. (1976). Human characteristics and school learning. New York:
McGraw-Hill.
Bowen, H. R. (1977). Investment in learning: The individual and social value of
American higher education. San Francisco: Josey-Bass.
Bruner, J. S. (1961). The art of discovery. Harvard Educational Review, 31, 21–32.
Burbules, N. C. (2000). Moving beyond the impasse. In D. C. Phillips (Ed.),
Constructivism in education: Opinions and second opinions on controversial
issues: Ninety-ninth yearbook of the National Society for the Study of
Education, Part 1 (pp. 308-330). Chicago: National Society for the Study of
Education.
Carlson, R. A., Lundy, D. H., & Schneider, W. (1992). Strategy guidance and memory
aiding in learning a problem-solving skill. Human Factors, 34, 129–145.
Carroll, W. (1994). Using worked examples as an instructional support in the algebra
classroom. Journal of Educational Psychology, 86, 360–367.
134
Chi, M. T., De Leeuw, N., Chiu, M.-H., & Lavancher, C. (1994). Cognitive Science,
18(3), 439-477.
Clark, R. E. (1994). Media will never influence learning. Educational Technology
Research and Development, 42(2), 21-29.
Clark, R. E. (1996). The CANE model of motivation to learn and work: A two-stage
process of goal commitment and effort. In J. Lowyck (Ed.), Trends in
Corporate Training. Leuven, Belgium: University of Leuven Press.
Clark, R. E. (1998). Motivating performance: Part 1 – diagnosing and solving
motivation problems. Performance Improvement, 37(8), 39-47.
Clark, R. E. (1999). Yin and yang cognitive motivational processes operating in
multimedia learning environments. In J. J.G. van Merriënboer (Ed.) Cognition
and multimedia design, pp. 73-107. Herleen, Netherlands: Open University
Press.
Clark, R. E. (2004a). What works in distance learning: Instructional strategies. In H.
O’Neil (Ed.) What works in distance learning: Guidelines. Greenwich, CT:
Information Age.
Clark, R. E. (2004b). Design document for a guided experiential learning course. Final
report on contract DAAD 19-99-D-0046-0004 from TRADOC to the Institute
for Creative Technology and the Rossier School of Education.
Clark, R. E. (in press). How much and what type of guidance is optimal for learning
from instruction? In S. Tobias and T. M. Duffy (Eds.), Constructivist Theory
Applied to Instruction: Success or Failure? New York: Routledge, Taylor &
Francis.
Clark, R. E., & Estes, F. (1999). The development of authentic educational
technologies. Educational Technology, 37(2), 5–16.
Clark, R.E., Feldon, D., Van Merriënboer, J.J.G., Yates, K., and Early, S. (2008)
Cognitive task analysis. In J. M. Spector, M. D. Merrill, J. J. G. van
Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational
communications and technology (3rd ed.). Mahwah, NJ: Lawrence Erlbaum
Associates.
Coalition for Evidence-Based Policy (n.d.). Identifying and implementing educational
practices supported by rigorous evidence: A user-friendly guide. Washington,
D.C.
135
Cobb, P. (1994). Where is the mind? Constructivist and sociocultural perspectives on
mathematical development. Educational Researcher, 23(7), 13-20.
Cole, M. & Engeström, Y. (1993). A cultural-historical approach to distributed
cognition. In G. Salomon (Ed.), Distributed cognitions: Psychological and
educational considerations, pp. 1-43. Cambridge: Cambridge University Press.
Colliver, J. A. (2000). Effectiveness of problem-based learning curricula: Research
and theory. Academic Medicine, 75, 259–266.
Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in
web- or internet-based surveys. Educational and Psychological Measurement,
60(6), 821-836.
Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of
training motivation: A meta-analytic path analysis of 20 years of research.
Journal of Applied Psychology, 85(5), 678-707.
Confrey, J. (1990). What constructivism implies for teaching. Journal for Research in
Mathematics Education. Monograph, Vol. 4, Constructivist views on the
teaching and learning of mathematics, pp. 107-210. Washington, DC: National
Council of Teachers of Mathematics.
Cowan, N. (1988). Evolving conceptions of memory storage, selective attention, and
their mutual constraints within the human information-processing system.
Psychological Bulletin, 104(2), 163-191.
Cowan, N. (2000). The magical number 4 in short-term memory: A reconsideration of
mental storage capacity. Behavioral and Brain Sciences, 24, 87–114.
Cowan, N., Elliott, E. M., Saults, J. S., Morey, C. C., Mattox, S., Hismjatullina, A., &
Conway, A. R. A. (2005). On the capacity of attention: Its estimation and its
role in working memory and cognitive aptitudes. Cognitive Psychology, 51,
42-100.
Craig, R. (1956). Directed versus independent discovery of established relations.
Journal of Educational Psychology, 47, 223–235.
De Leeuw, K. E., & Mayer, R. E. (2008). A comparison of three measures of cognitive
load: Evidence for separable measures of intrinsic, extraneous and germane
load. Journal of Educational Psychology, 100(1), 223-234.
Dewey, J. (2004). Democracy and education. Whitefish, MT: Kessinger.
136
Dewey, J. (1938, 1997). Experience and education. New York: Touchstone.
Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem-
based learning: A meta-analysis. Learning and Instruction, 13(5), 533-568.
Duffy, T. M., & Jonassen, D. H. (1992). Constructivism and the technology of
instruction. Mahwah, NJ: Lawrence Erlbaum.
Duffy, T. M., & Kirkley, J. (2004). Designing environments for distributed learning:
Learning theory and practice. Mahwah, NJ: Lawrence Erlbaum.
Duncan, J., & Owen, A. M. (2000). Common regions of the human frontal lobe
recruited by diverse cognitive demands. Trends in Neuroscience, 23, 475-483.
Edelson, D. C., Gordin, D. N., & Pea, R. D. (1999). Addressing the challenges of
inquiry-based learning through technology and curriculum design. The Journal
of the Learning Sciences, 8(3,4), 391-450.
Ericcson, K. A., Chase, W. G., & Falloon, S. (1980). Acquisition of a memory skill.
Science, 208, 1181-1182.
Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological
Review, 102(2), 211–245.
Feldman, J. A. (2006). From molecule to metaphor: A neural theory of language.
Cambridge, MA: MIT Press.
Feldon, D. F. (2007). Implications of research on expertise for curriculum and
pedagogy. Educational Psychology Review, 19(2), 91-110.
Feuer, M. J., Towne, L. Shavelson, R. J. (2002). Scientific culture and educational
research. Educational Researcher, 31(8), 4-14.
Fletcher, J. D., & Tobias, S. (2005). The multimedia principle. In R. E. Mayer (Ed.),
The Cambridge handbook of multimedia learning, pp. 117-134. Cambridge:
Cambridge University Press.
Fox, R. (2001). Constructivism examined. Oxford Review of Education, 27(1), 23-35.
Freire, P. (1970). Pedagogy of the oppressed. New York: Continuum.
Gagné, R. B., & Briggs, L. J. (1979). Principles of instructional design. New York:
Holt, Rinehart & Winston.
137
Gagné, R. B., & Dick, W. (1983). Instructional Psychology. Annual Review of
Psychology, 34, 261-295.
Gagné, R. B., & White, R. T. (1978). Memory structures and learning outcomes.
Review of Educational Research, 48(2), 187-222.
Geary, D. C. (2002). Principles of evolutionary educational psychology. Learning and
Individual Differences, 12, 317-345.
Gerjets, P., Scheiter, K., & Schorr, T. (2003). Modeling processes of volitional action
control in multiple-task performance: How to explain the effects of goal
competition and task difficulty on processing strategies and performance
within ACT-R. Cognitive Science Quarterly, 3, 355-400.
Gibson, E. J. (1988). Exploratory behavior in the development of perceiving, acting
and the acquiring of knowledge. Annual Review of Psychology, 39, 1-42.
Goldman-Rakic, P. S. (1995). Cellular basis of working memory. Neuron, 14, 477-
485.
Graham, S. & Weiner, B. (1996). Theories and principles of motivation. In D. C.
Berliner & R. C. Calfee (Eds.), Handbook of Educational Psychology, pp. 63-
83. New York: MacMillan Library References.
Gray, J. R., Chabris, C. F., Braver, T. S. (2003). Neural mechanisms of general fluid
intelligence. Nature Neuroscience, 6(3), 316-322.
Halloun, I. A. (2006). Mediated modeling in science education. Science and
Education,16, 653-657.
Hammersley, M. (1997). Educational research and teaching: A response to David
Hargreaves’ TTA lecture. British Educational Research Journal, 23(2), 141-
161.
Hargreaves, D. H. (1997). In defence of research for evidence-based teaching: A
rejoinder to Martyn Hammersley. British Educational Research Journal, 23(4),
405-419.
Hastings, N. B. & Tracey, M. W. (2005). Does media affect learning: Where are we
now? Tech Trends Linking Research and Practice to Improve Learning, 49(2),
28-30.
138
Hill, R., Kim, J., Zbylut, M., Gordon, A., Traum, D., Gandhe, S., et al. (2006).
AXL.Net: Web-Enabled Case Method Instruction for Accelerating Tacit
Knowledge Acquisition in Leaders. Proceedings of the 25th Army Science
Conference, Nov 2006, Orlando, FL.
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn?
Educational Psychology Review, 16, 235–266.
Jonassen, D. (1991). Objectivism versus constructivism: Do we need a new
philosophical paradigm? Educational Technology Research and Development,
39(3), 5–14.
Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational
Technology Research and Development, 48(4), 63-85.
Jonassen, D. H., Campbell, J. P., Davidson, M. E. (1994). Learning with media:
Restructuring the debate. Educational Technology Research and Development,
42(2), 31-39.
Jonides, J. (2004). How does practice make perfect? Nature Neuroscience, 7(1), 10-
11.
Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal
effect. Educational Psychologist, 38(1), 23–31.
Kalyuga, S., Chandler, P., & Sweller, J. (1998). Levels of expertise and instructional
design. Human Factors, 40, 1-17.
Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and
redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351-
372.
Kalyuga, S., Chandler, P., & Sweller, J. (2000). Incorporating learner experience into
the design of multimedia instruction. Journal of Educational Psychology, 92,
126-136.
Kalyuga, S., Chandler, P., & Sweller, J. (2001). Learner experience and efficiency of
instructional guidance. Educational Psychology, 21(1), 5-19.
Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving
is superior to studying worked examples. Journal of Educational Psychology,
93,579–588.
139
Keller, J. M., & Suzuki, K. (2004). Learner motivation and e-learning design: A
multinationally validated process. Journal of Educational Media, 29(3), 229-
239.
Kennedy, M. M. (1997). The connection between research and practice. Educational
Researcher, 26(7), 4-12.
Kieras, D. E., Meyer, D. E., Mueller, S., & Seymour, T. (1999). Insights into working
memory from the perspective of the EPIC architecture for modeling skilled
perceptual-motor performance. In P. Shah & A. Miyake (Eds.), Models of
Working Memory: Mechanisms of Active Maintenance and Executive Control.
Cambridge: Cambridge University Press.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during
instruction does not work: An analysis of the failure of constructivist,
discovery, problem-based, experiential, and inquiry-based teaching.
Educational Psychologist, 41(2), 75-86.
Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early science
instruction: Effects of direct instruction and discovery learning. Psychological
Science, 15, 661–667.
Kluger, A. N., & Dinisi, A. (1996). The effects of feedback interventions on
performance: A historical review, a meta-analysis, and a preliminary
intervention theory. Psychological Bulletin, 119(2), 254-284.
Koch, C. (2004). The quest for consciousness: A neurobiological approach.
Englewood, CO: Roberts and Company.
Kozma, R. B. (1994a). Will media influence learning? Reframing the debate.
Educational Technology Research and Development, 42(2), 7-19.
Kozma, R. B. (1994b). A reply: Media and methods. Educational Technology
Research and Development, 42(3), 11-14.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory Into
Practice, 41(4), 212-218.
Lebow, D. (1993). Constructivist Values for Systems Design: Five Principles Toward
a New Mindset. Educational Technology Research and Development, 41, 4-16.
Levine, J. M., & Resnick, L. B. (1993). Social foundations of cognition. Annual
Review of Psychology, 44, 585-612.
140
Mathieu, J. E., Tannenbaum, S. I., & Salas, E. (1992). Influences of individual and
situational characteristics on measures of training effectiveness. Academy of
Management Journal, 35(4), 828-847.
Mayer, R. E. (1999a). Multimedia aids to problem-solving transfer. International
Journal of Educational Research, 31, 611-623.
Mayer, R. E. (1999b). Research-based principles for the design of instructional
messages: The case of multimedia explanations. Document Design, 1, 7-20.
Mayer, R. E. (2001). Multimedia learning. Cambridge; Cambridge University Press.
Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery
learning? The case for guided methods of instruction. American Psychologist,
59, 14–19.
Merrill, M. D. (2002). First principles of instruction. Educational Technology
Research and Development, 50(3), 43-59.
Merrill, M. D. (2002b). A pebble-in-the-pond model for instructional design.
Performance Improvement, 41(7), 39-44.
Merrill, M. D. (2007). A task-centered instructional strategy. Journal of Research on
Technology in Education, 40(1), 5-22.
Merrill, M. D., & Tennyson, R. D. (1977). Teaching concepts: An instructional design
guide. Englewood Cliffs, NJ: Educational Technology Publications.
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on
our capacity for processing information. Psychological Review, 63, 81-97.
Moreno, R. (2006). Does the modality principle hold for different media? A test of the
method-affects-learning hypothesis. Journal of Computer Assisted Learning,
22(3), 149-158.
Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The
role of modality and contiguity. Journal of Educational Psychology, 91, 358-
368.
Moreno, R., & Mayer, R. E. (2000). A coherence effect in multimedia learning: The
case for minimizing irrelevant sounds in the design of multimedia messages.
Journal of Educational Psychology, 92, 117-126.
141
Nadolski, R. J., Kirschner, P. A., & van Merriënboer, J. J. G. (2005). Optimizing the
number of steps in learning tasks for complex skills. British Journal of
Educational Psychology, 75, 223–237.
Norman, G. R., & Schmidt, H. G. (1992). The psychological basis of problem-based
learning: A review of the evidence. Academic Medicine, 67(9), 557-565.
Northouse, P. G. (2004). Leadership: Theory and practice (3d Ed.). Thousand Oaks,
CA: Sage.
Oleson, P. J., Westerberg, H., & Klingberg, T. (2004). Increased prefrontal and
parietal activity after training of working memory. Nature Neuroscience, 7, 75-
79.
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional
design: Recent developments. Educational Psychologist, 38, 1-4.
Paas, F., Renkl, A., & Sweller, J. (2004). Cognitive load theory: Instructional
implications of the interaction between information structures and cognitive
architecture. Instructional Science, 32(1), 1-8.
Paas, F., Tuovinen, J. E., van Merriënboer, J. J. G., & Darabi, A. A. (2005). A
motivational perspective on the relation between mental effort and
performance: Optimizing learner involvement in instruction. Educational
Technology Research and Development, 53(3), 25-34.
Paas, F., & van Gog, T. (2006). Optimising worked example instruction: Different
ways to increase germane cognitive load. Learning and Instruction, 16(2), 87-
91.
Paas, F., & van Merriënboer, J. (1994). Variability of worked examples and transfer of
geometrical problem solving skills: A cognitive-load approach. Journal of
Educational Psychology, 86, 122–133.
Palincsar, A. S. (1998). Social constructivist perspectives on teaching and learning.
Annual Review of Psychology, 49, 345-375.
Pedhazur, E. J. (1997). Multiple regression in behavioral research: Explanation and
prediction (3d Ed.). Toronto, Canada: Wadsworth-Thomson Learning.
Piaget, J., & Inhelder, B. (1969, 2000). The psychology of the child. New York: Basic
Books.
Pintrich, P. R., & Schunk, D. H. (2002). Motivation in education. Upper Saddle River,
NJ: Merrill.
142
Quilici, J. L., & Mayer, R. E. (1996). Role of examples in how students learn to
categorize statistics word problems. Journal of Educational Psychology, 88,
144–161.
Reeves, B., & Anderson, D. R. (1991). Media studies and psychology.
Communication Research, 18(5), 597-600.
Reeves, B. & Nass, C. (1996). The media equation. New York: Cambridge University
Press.
Resnick, L. B. (1987). The 1987 presidential address: Learning in school and out.
Educational Researcher, 16(9), 13-20, 54.
Roblyer, M. D., Edwards, J., & Havriluk, M. A. (1997). Integrating educational
technology into teaching (2nd ed.). Upper Saddle River, NJ: Prentice-Hall.
Rogers, W. A., Maurer, T. J., Salas, E., & Fisk, A. D. (1997). Training design,
cognitive theory and automaticity: Principles and a methodology. In J. K. Ford
(Ed.), Improving training effectiveness in work organizations, pp. 19-45.
Mahwah, NJ: Lawrence Erlbaum.
Rorty, R. (1991). Objectivism, relativism, and truth. Cambridge: Cambridge
University Press.
Ruchkin, D. S., Grafman, J., Cameron, K., & Berndt, R. S. (2003). Working memory
retention systems: A state of activated long-term memory. Behavioral and
Brain Sciences, 26, 709-728.
Saba, F. (2000). Research in distance education: A status report. International Review
of Research in Open and Distance Learning, 1(1), 1-9.
Salomon, G. (1984). Television is easy and print is tough: The differences in
investment of mental effort in learning as a function of perceptions and
attributions. Journal of Educational Psychology, 76(4), 647-658.
Savery, J. R., & Duffy, T. M. (2001). Problem based learning: An instructional model
and its constructivist framework. Manuscript published by the Center for
Research on Learning and Technology, Indiana University, Bloomington, IN.
Sawaguchi, T., & Goldman-Rakic, P. S. (1991). D1 dopamine receptors in prefrontal
cortex: Involvement in working memory. Science, 947-950.
Schmidt, H. G. (2000). Assumptions underlying self-directed learning may be false.
Medical Education, 34, 243–245.
143
Schmidt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment,
8(4), 350-353.
Schwartz, D. L., Lindgren, R., & Lewis, S. (in press). Constructivism in an age of non-
constructivist assessments. In S. Tobias and T. M. Duffy (Eds.), Constructivist
Theory Applied to Instruction: Success or Failure? New York: Routledge,
Taylor & Francis.
Shiffrin, R. M., & Schneider, W. (1977). Controlled and automatic human information
processing. Psychological Review, 84(1), 1-66.
Shulman, L. S., & Hutchings, P. (1999, September–October). The scholarship of
teaching: New elaborations, new developments. Change, 11–15.
Slavin, R. E. (2002). Evidence-based education policies: Transforming educational
practice and research. Educational Researcher, 31(7), 15-21.
Small, S. A. (2001). Age-related memory decline: Current concepts and future
directions. Archives of Neurology, 58(3), 360-364.
Song, S. H., & Keller, J. M. (2001). Effectiveness of motivationally adaptive
computer-assisted instruction on the dynamic aspects of motivation.
Educational Technology Research and Development, 49(2), 5-22.
Stanford, G., & Roark, A. E. (1972). Seizing the teachable moment: Social learning in
the classroom. People Watching, 2(1), 14-18.
Stapleton, A. J. (2004). Serious games: Serious opportunities. Paper presented at the
Australian Game Developers’ Conference, Academic Summit, Melbourne,
AUS.
Steele, C. M. & Aronson, J. (1995). Stereotype threat and the intellectual performance
of African Americans. Journal of Personality and Social Psychology, 69(5),
797-811.
Sun, R., Merrill, E., & Peterson, T. (2001). From implicit skills to explicit knowledge:
A bottom-up model of skill learning. Cognitive Science, 25, 203-244.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning.
Cognitive Science, 12, 257–285.
Sweller, J. (2004). Instructional design consequences of an analogy between evolution
by natural selection and human cognitive architecture. Instructional Science,
32, 9–31.
144
Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for
problem solving in learning algebra. Cognition and Instruction, 2, 59–89.
Sweller, J., Kirschner, P. A., & Clark, R. E. (2007). Why minimally guided teaching
techniques do not work: A reply to commentaries. Educational Psychologist,
42(2), 115-121.
Sweller, J., & Sweller, S. (2006). Natural information processing systems.
Evolutionary Psychology, 4, 434-458.
Sweller, J., van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and
instructional design. Educational Psychology Review, 10, 251–296.
Tan, O. S. (2004). Students’ experiences in problem-based learning: three blind mice
episode or educational innovation? Innovations in Education and Teaching
International, 41(2), 169-184.
Teigen, K. H. (1994). Yerkes-Dodson: A law for all seasons. Theory and Psychology,
4, 525-547.
Thompson, R. F., & Madigan, S. A. (2005). Memory: Key to consciousness.
Washington, DC: Joseph Henry Press.
Tuovinen, J. E., & Sweller, J. (1999). A comparison of cognitive load associated with
discovery learning and worked examples. Journal of Educational Psychology,
91, 334–341.
van Merriënboer, J. J. G. (1997). Training complex cognitive skills: A four-component
instructional design model for technical training. Englewood Cliffs, NJ:
Educational Technology Publications.
van Merriënboer, J. J. G., Clark, R. E., & de Croock, M. B. M. (2002). Blueprints for
complex learning: The 4C/ID-Model. Educational Technology Research &
Development, 50(2), 39-64.
van Merriënboer, J. J. G., Jelsma, O., & Paas, F. G. W. C. (1992). Training for
reflective expertise: A four-component instructional design model for complex
skills. Educational Technology Research and Development, 40(2), 23-43.
van Merriënboer, J. J. G., Kirschner, P. A., & Kester, L. (2003). Taking the load of a
learner’s mind: Instructional design for complex learning. Educational
Psychologist, 38(1), 5-13.
145
van Merriënboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and complex
learning: Recent developments and future directions. Educational Psychology
Review, 17(2), 147-177.
Vecchio, R. P., Bullis, R. C., & Brazil, D. M. (2006). The utility of situational
leadership theory: A replication in a military setting. Small Group Research,
37(5), 407-424,
Vygotsky, L. S. (1992). Educational Psychology. Miami, FL: St. Lucie Press.
Ward, M., & Sweller, J. (1990). Structuring effective worked examples. Cognition and
Instruction, 7, 1–39.
Wegner, D. (2003). The illusion of conscious will. Cambridge: Harvard University
Press.
Weiner, B. (1990). History of motivational research in education. Journal of
Educational Psychology, 82(4), 616-622.
Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement.
Contemporary Educational Psychology, 25, 68-81.
Wilhite, S. C. (1990). Self-efficacy, locus of control, self-assessment of memory
ability and study activities as predictors of college course achievement.
Journal of Educational Psychology, 82(4), 696-700.
Williamson, G. V., & Goldman-Rakic, P. S. (1995). Modulation of memory fields by
dopamine D1 receptors in prefrontal cortex. Nature, 376, 572-575.
Wise, A., & O’Neill, K. (in press). Beyond more versus less: A re-framing of the
debate on instructional guidance. In S. Tobias and T. M. Duffy (Eds.),
Constructivist Theory Applied to Instruction: Success or Failure? New York:
Routledge, Taylor & Francis.
Woltz, D. J. (2003). Implicit cognitive processes as aptitudes for learning. Educational
Psychologist, 38, 95–104.
Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn. Contemporary
Educational Psychology, 25, 82-91.
146
Zbylut, M. L., Metcalf, K. A., Kim, J.M., Hill, R. W., & Rocher, S. (2007). Army
Excellence in Leadership (AXL): A multimedia approach to building tacit
knowledge and cultural reasoning. (ARI Technical Report 1194).
Alexandria,VA: U.S. Army Research Institute for the Behavioral and Social
Sciences.
147
Appendix A:
A-1: Diagram of Anderson’s ACT-R model of cognitive architecture
Environment
Visual
Module
Motor
Module
Speech
Module
Audition
Module
ACT-R
Buffers
Declarative
Memory
Production
Memory
Pattern
Matching
Production
Execution
Cognition Layer
Perceptual-Motor Layer
video
writing
audio
Raw audio
148
Appendix B:
B-1: Recruitment flyer
Online Study Opportunity
USC’s Institutes for Creative Technologies and the Rossier School of Education’s
Center for Cognitive Technology are currently conducting a computer-based study
that examines the role of media in learning.
All adults with access to the Internet are encouraged to participate.
Participation in this important study is completely voluntary. The average time for
completion is 25 minutes, although this varies from 15 to 40 minutes.
The study can be completed from any computer with Internet access.
To access to study site, direct your Internet browser to:
http://axlserver.ict.usc.edu/axl/anon/?USCSLT
You may also copy and paste this link into a new browser window.
If you have questions feel free to contact Richard Clark (clark@usc.edu) or Sean Early
(searly@usc.edu) at:
The Center for Cognitive Technology
250 N. Harbor Dr., Suite 309
Redondo Beach, CA 90277
Tel: (310) 379-0844
149
Appendix C:
C-1: Summative Assessment Items
You will now begin a short assessment of your understanding of Situational
Leadership Theory in a military context.
Read each question carefully and consider the learning materials before responding.
Answer the following true-false questions.
1. If a leader says the following, it would be an example of a directive behavior: "The
tire is flat. Go and get the jack out of the trunk of the car and change the tire." (T)
2. If a leader says the following, it would be an example of a directive behavior: "SGT
Jones, call up the MCS computer system and prepare a map overlay that represents the
location of friendly and enemy positions." (T)
3. If a leader says the following, it would be an example of a directive behavior: "We
have a flat tire and are on a steep hill. You've changed tires on steep hills, do you think
we should roll the truck to a more level spot before we change the tire?" (F)
4. If a leader says the following, it would be an example of a directive behavior: "We
need to conduct a house-to-house search in this neighborhood. You know the
individuals in the neighborhood, what is your advice on a how to proceed?" (F)
5. If a leader says the following, it would be an example of a directive behavior:
"Janet, you do a good job at prioritizing meetings like this, will you handle that?" (F)
Now answer these true-false questions about supportive behaviors.
6. If a leader says the following, it would be an example of a supportive behavior:
"John, will you prepare a news release to describe our project? Follow the approach I
150
took in our news release and come show it to me when you are done." (F)
7. If a leader says the following, it would be an example of a supportive behavior: "As
a team, you are struggling with a very difficult problem. You have succeeded in
solving similar problems in the past and I know you are going to do a great job with
this one." (T)
8. If a leader says the following, it would be an example of a supportive behavior:
Drill SGT: "Report for PT at 0500 tomorrow for a 6-mile run!" (F)
9. If a leader says the following, it would be an example of a supportive behavior: "I
am new to this unit. Tell me how you would accomplish this mission." (T)
10. If a leader says the following, it would be an example of a supportive behavior:
"Take your platoon across this open country and position them in this location on the
map. I am going to be watching you through binoculars with a field manual at my side
to make certain you follow the correct step-by-step procedure." (F)
11. If a leader says the following, it would be an example of a supportive behavior: "In
the past, I've been very clear about how I want you to do things. Now, I want you to
decide how you are going to attack this position and then describe and defend your
choice to me." (F)
Answer the following multiple-choice questions.
12. Choose the most accurate developmental level: A successful team has recently
experienced some project setbacks and has become somewhat discouraged. Their
moral has dropped as well as their performance. (Supporting)
13. Choose the most accurate developmental level: You have worked with a particular
151
team member in the past and you notice that he has the skills and experience to do the
job well. You notice, however, that he does not seem to share your confidence in his
ability. (Coaching)
14. Choose the most accurate developmental level: You are thinking of asking a
highly capable member of your team to supervise a consolidation project within the
organization. She has the trust of team members and has expressed her willingness to
help. (Delegating)
Now answer these multiple-choice questions about leadership styles.
15. Choose the most appropriate leadership style: Because of some project setbacks,
your team has become discouraged. Their morale has dropped as well as their
performance. (Supporting)
16. Choose the most appropriate leadership style: After transferring to a new
department, you notice that an inexperienced worker fails at assigned tasks. She shows
great enthusiasm, and wants to get ahead in the organization. (Directing)
17. Choose the most appropriate leadership style: You are thinking of asking a highly
capable member of your team to supervise a consolidation project within the
organization. She has the trust of team members and has expressed her willingness to
help. (Delegating)
There are several segments of video that follow.
Watch the videos carefully, as you will be asked questions based on the information
contained in the clips.
Watch each of these videos before proceeding, keeping in mind the principles of
152
Situational Leadership.
You will be asked question about what you see, and you can watch each video as
many times as you need to before responding.
[Media: uploads/8300674101_SLT-RedTight2.mov]
[Media: uploads/8300674129_SLT-RedTight3.mov]
[Media: uploads/8300674253_SLT-RedTight4.mov]
[Media: uploads/8300674351_SLT-RedTight5.mov]
18. Review the video before answering the question. [Media:
uploads/8300674101_SLT-RedTight2.mov]
What is the developmental level of LT Turner? (Moderate competence, low
commitment)
19. What leadership style is CPT Stone demonstrating with LT Turner? (Directing)
20. CPT Stone's leadership style is a good match for LT Turner's developmental level,
based on the prescriptive matches of Situational Leadership Theory. (F)
21. Again, review the video clip if you choose to before answering the questions.
[Media: uploads/8300674253_SLT-RedTight4.mov]
What is CPT Stone's leadership style in this sequence? (Coaching)
153
22. CPT Stone's leadership style in this clip demonstrates some elements of supportive
behaviors. (T)
END
154
Appendix D:
D-1: Self-efficacy scale
1. How able are you to determine someone's confidence about their ability to
solve a problem?
2. How successfully could you apply Situational Leadership Theory in a real-life
context?
3. How successful are you when you try to determine the individual ability level
of each member of a team?
4. How much can you do to help people enjoy tackling a challenging task?
5. How able are you now to list or describe all or most of the elements of
Situational Leadership Theory described by Hirsch and Blanchard?
6. How successfully could you apply Situational Leadership Theory if you were
leading a team of people who were attempting to solve a problem in your
neighborhood?
7. How much can you do to get team members to work together smoothly on a
task?
8. How able are you to apply leadership theory to leading people in real
situations?
9. How able are you to determine a person's willingness to perform a challenging
task?
10. How able are you to successfully solve a textbook leadership problem by
applying the key elements of Situational Leadership Theory?
155
Appendix E:
E-1: Informed Consent Form
Effects of Robust Media on Learning: Situational Leadership Theory in the AXL
environment
RESEARCH PROCEDURES
This research is being conducted to investigate the role of media in learning. If you
agree to participate, you will be assigned to one of lessons that vary in terms of media
but are otherwise identical. Following the lesson, you will complete a short test about
the lesson content. Last, we will ask you to provide common demographic
information to help us understand the data generated during the study.
RISKS
There are no foreseeable risks for participating in this research.
BENEFITS
By participating in this project, you may gain knowledge that may improve your
future job performance. General benefits of this project to society include clarifying
the relationship between media and learning outcomes.
CONFIDENTIALITY
The data in this study will be confidential. Your answers will not contain any
information that can trace the responses to an individual participant.
156
PARTICIPATION
Your participation in this study is completely voluntary. You may withdraw from the
study at any time and for any reason without any negative repercussions. If you decide
not to participate or if you withdraw from the study, there is no penalty or loss of
benefits to which you are otherwise entitled. There are no costs to you or any other
party.
CONTACT
This research is being conducted by Richard Clark, the Center for Cognitive
Technology, Rossier School of Education at the University of Southern California. He
may be reached at (310) 379-0844; Fax (310) 372-7788 for questions or to report a
research-related problem. You may also contact the University of Southern
California’s Institutional Review Board at (213) 821-5272 if you have questions or
comments regarding your rights as a participant in the research.
This research has been reviewed according to the University of Southern California’s
procedures governing your participation in this research.
CONSENT
I have read this form and agree to participate in this study.
__________________________
Name
157
Appendix F:
F-1: Questions from online course
1. When you have finished watching the clip, answer the questions that you find
below.
[Media: uploads/8300674633_SLT-Tripwire1.mov]
Was the CPT being more directive or more supportive in his interactions with his
subordinates?
2. Review the same clip again if you would like before proceeding to answer the
question found below.
[Media: uploads/8300674633_SLT-Tripwire1.mov]
Was the XO being more directive or more supportive in his interactions with his
subordinates?
3. Now watch this section of video.
While watching, think about how the leader interacts with his subordinates. When you
have finished watching the clip, answer the questions below.
[Media: uploads/8300674708_SLT-Tripwire2.mov]
What leadership style is demonstrated in this clip, and how does it match the situation
and readiness level of the subordinate?
4. This page begins a sequence of pages that contain segments of video portraying
mission situations.
Watch the video segments for examples of the prescriptive matches between
leadership styles and developmental levels developmental levels. Answer the
158
questions that you will find below each clip.
[Media: uploads/8300673537_SLT-PowerHungry2.mov]
In this clip, what behaviors for CPT Young and 1LT Perez demonstrate that are
consistent with a "Directing" style, and why might this be a good match for the
subordinate's development level?
5. Watch this interview clip with CSM Pullman and think about the leadership style he
perceives on the part of CPT Young and what developmental level it might best
match.
[Media: uploads/influencing-young.mov]
What behaviors does CSM Pullman describe in his interview that are consistent with a
"coaching" style, and what developmental level does this match?
6. Watch this third clip, and think about the implications of what CPT Young says in
terms of leadership style.
[Media: uploads/get-options.mov]
How is what CPT Young describes consistent with a "delegating" style of leadership,
and whom might this be appropriate for?
7. In this final clip, watch the video and think about the ways in which CPT Young
interacts with the SGT Major.
[Media: uploads/51807120101_clip_for_Sean_Early.mov]
What supporting behaviors does CPT Young demonstrate in this clip, and how do they
match the SGT Major's developmental level?
159
8. Watch this clip and think about the communication behaviors displayed.
[Media: uploads/51807120101_clip_for_Sean_Early.mov]
What readiness level does the SGT demonstrate, and how does CPT Young's behavior
match this level?
160
Appendix G: Scatterplots
Figure G-1: Total score by education level
Scatterplot Total Score by
Education Level
0
5
10
15
20
25
0 1 2 3 4 5
Education Level
Total Score Linear (Total Score)
Figure G-2: Total score by condition
Scatterplot Total Score by Condition
0
5
10
15
20
25
0 1 2 3 4
Condition
Total_Score Linear (Total_Score)
161
Figure G-3: Verbal recognition by condition
Scatterplot Verbal Recognition by Condition
0
2
4
6
8
10
12
0 1 2 3 4
Condition
VRBRCG Linear (VRBRCG)
Figure G-4: Verbal recognition by education level
Scatterplot Verbal Recognition
by Education Level
0
2
4
6
8
10
12
0 1 2 3 4 5
Education Level
VRBRCG Linear (VRBRCG)
162
Figure G-5: Verbal recall by condition
Scatterplot Verbal Recall by Condition
0
2
4
6
8
10
12
14
0 1 2 3 4
Condition
VERBRCL Linear (VERBRCL)
Figure G-6: Verbal recall by education level
Scatterplot Verbal Recall by Education
0
2
4
6
8
10
12
14
0 1 2 3 4 5
Education Level
VERBRCL Linear (VERBRCL)
163
Figure G-7: Classification of leadership styles by condition
Scatterplot Classification Leadership
Styles by Condition
0
1
2
3
4
5
6
7
8
0 1 2 3 4
Condition
CLASSLDR Linear (CLASSLDR)
Figure G-8: Classification of leadership styles by education level
Scatterplot Classification Leadership
Styles by Education Level
0
1
2
3
4
5
6
7
8
0 1 2 3 4 5
Educational Level
CLASSLDR Linear (CLASSLDR)
164
Figure G-9: Classification of developmental levels by condition
Scatterplot Classification
Development Levels by Condition
0
1
2
3
4
5
6
0 1 2 3 4
Condition
CLASSDEV Linear (CLASSDEV)
Figure G-10: Classification of developmental levels by education level
Scatterplot Classification Developmental
Levels by Education Level
0
1
2
3
4
5
6
0 1 2 3 4 5
Education Level
CLASSDEV Linear (CLASSDEV)
165
Appendix H:
Table H-1: Pearson Correlation Coefficients for Summative Assessment, Subscales and Demographic Variables
SCORE VRBRCG VRBCL CLSLDR CLSDEV COND AGE GNDR PRLDR EDUC CMPUS
SCORE 1.00
VRBRC
G 0.88 1.00
VRBCL 0.89 0.59 1.00
CLSLDR 0.78 0.85 0.53 1.00
CLSDEV 0.72 0.66 0.58 0.54 1.00
COND 0.41 0.41 0.30 0.44 0.37 1.00
AGE -0.07 0.00 -0.15 0.09 -0.16 -0.01 1.00
GNDR -0.27 -0.18 -0.33 -0.07 -0.19 0.14 0.65 1.00
PRLDR -0.15 -0.12 -0.21 0.01 -0.14 0.15 0.61 0.78 1.00
EDUC -0.23 -0.12 -0.32 0.01 -0.24 0.15 0.73 0.88 0.88 1.00
CMPUS -0.23 -0.12 -0.32 0.01 -0.24 0.14 0.74 0.88 0.88 1.00 1.00
Legend: SCORE: Total Score; VRBRCG: Verbal Recognition; VRBRCL: Verbal Recall; CLSDR: Classification Leadership Styles;
CLSDEV: Classification Developmental Levels; COND: Condition; AGE: Age in Years; GNDR: Gender; PRLDR: Prior Leadership
Training; EDUC: Educational Level; CMPUS: Hours of Weekly Computer Use
Abstract (if available)
Abstract
The appropriate role for explicit, directive guidance during instruction has been debated in the literature for several decades (Sweller, Kirschner, & Clark, 2006
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
The effects of culturally responsive standards based instruction on African American student achievement
PDF
The effectiveness of computer supported collaborative learning on helping tasks in a mathematics course
PDF
Equitable choices in higher education: investigation of an equity-based faculty search committee development program
PDF
Negotiating racial conflict: the leadership role of the dean of students
PDF
Supporting the professional development of early childhood educators: a case study of an emergent literacy intervention project
PDF
Reframing school reform with effective tactics: institutionalizing culturally responsive pedagogy in alignment with policy-based accountability to ensure the achievement of African aAmerican stud...
PDF
How peer leadership impacts hazing in the arena of non-revenue generating college athletics
PDF
The impact and sustainability of programs, practices and norms on student academic performance: a case study
PDF
Achieving equity in educational outcomes through organizational learning: enhancing the institutional effectiveness of community colleges
PDF
The effect of a language arts intervention on the academic achievement of English language learners
PDF
The effects of cognitive readiness in a surface warfare simulation
PDF
Factors that promote or inhibit the involvement of African American parents in a community college early childhood education program
PDF
The impact of resource allocation on professional development for the improvement of teaching and student learning within a site-based managed elementary school: a case study
PDF
Enacting competing ideologies: how classroom dynamics influence the education of disaffected early adolescents at a community day school
PDF
The role of music in the English language development of Latino prekindergarten English learners
PDF
Effects of the prompts of depth and complexity on gifted and non-gifted students
PDF
Balancing the pros and cons of a new community college "grow-your-own" leadership program
PDF
Effects of help-seeking in a blended high school biology class
PDF
The search for transformative agents: the counter-institutional positioning of faculty and staff at an elite university
PDF
The effect of active and passive listening condition on young children's musical understanding of familiar and unfamiliar melody through researcher-invented notation
Asset Metadata
Creator
Early, Sean Francis
(author)
Core Title
The effect of guidance on learning in a web-based, media-enriched learning environment
School
Rossier School of Education
Degree
Doctor of Philosophy
Degree Program
Education
Publication Date
12/12/2008
Defense Date
07/01/2008
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
effect size,Guidance,leadership,Learning and Instruction,multiple regression,OAI-PMH Harvest,Training,web-based
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Clark, Richard E. (
committee chair
), Kezar, Adrianna (
committee member
), McArdle, John J. (
committee member
)
Creator Email
early.sean@gmail.com,searlyusc@yahoo.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-m1921
Unique identifier
UC1122771
Identifier
etd-Early-2343 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-148951 (legacy record id),usctheses-m1921 (legacy record id)
Legacy Identifier
etd-Early-2343.pdf
Dmrecord
148951
Document Type
Dissertation
Rights
Early, Sean Francis
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Repository Name
Libraries, University of Southern California
Repository Location
Los Angeles, California
Repository Email
cisadmin@lib.usc.edu
Tags
effect size
multiple regression
Training
web-based