University of Southern California Dissertations and Theses

Does problem -based learning in medical education lead to more self -directed learning by physicians? A review of comparative studies, theory and analysis

(USC Thesis Other)

Does problem -based learning in medical education lead to more self -directed learning by physicians? A review of comparative studies, theory and analysis

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content DOES PROBLEM-BASED LEARNING IN MEDICAL EDUCATION LEAD TO
MORE SELF-DIRECTED LEARNING BY PHYSICIANS?
A REVIEW OF COMPARATIVE STUDIES, THEORY AND ANALYSIS
by
Karen D. Tsoulas
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
December 2003
Copyright 2003 Karen D. Tsoulas
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
UMI Number: 3133345
Copyright 2003 by
Tsoulas, Karen D.
All rights reserved.
INFORMATION TO USERS
The quality of this reproduction is dependent upon the quality of the copy
submitted. Broken or indistinct print, colored or poor quality illustrations and
photographs, print bleed-through, substandard margins, and improper
alignment can adversely affect reproduction.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
®
UMI
UMI Microform 3133345
Copyright 2004 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
University of Southern California
Rossier School of Education
Los Angeles, California 90089-0031
This dissertation written by
fe f* R CiO 1 6 c, a .S
under the discretion of h e f Dissertation Committee,
and approved by all members of the C om m ittee, has
been presented to and accepted by the Faculty of the
Rossier School of Education in partial fulfillment of the
requirements for the degree of
D octor o f Education
December 17, 2003
Date
AW - D b ,
*jMU)
'ean
Dissertation C om m ift^
Chairperson
su, U-^4
A
/ S
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
ii
ACKNOWLEDGMENTS
I would like to express sincere appreciation to my dissertation committee
members, Dr. Richard Clark, Dr. Kaaren Hoffman, and Dr. Allan Abbott for their
continued advice and support throughout the writing of this dissertation. I am
especially grateful to Dr. Clark in his role as faculty advisor and dissertation chair
whose extended patience, unwavering encouragement and genuine care and
communication contributed immensely not only to the writing of this paper, but also
to the completion of my doctoral program.
The professional opportunity granted to me by Dr. Pamela Schaff during my
doctoral program deserves special acknowledgment as her extended support was a
significant factor in making the pursuit of this degree possible.
I also express my thanks to the very kind and accommodating resource
librarians at the Norris Medical Library for their frequent help with literature
identification.
And I would like to thank my husband, Jim Dixon, and my close friend, Clare
Hannan, for their steadfast love, support and understanding throughout my doctoral
studies.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
TABLE OF CONTENTS
iii
Acknowledgements ii
List of Tables and Figures iv
Abbreviations v
Abstract vii
CHAPTER 1: The Definition of Self-directed Learning (SDL) 1
CHAPTER 2: The Use of Problem-based Learning (PBL) in Medical 17
Education to Promote SDL
CHAPTER 3: The Definition, Assessment and Research of SDL 54
in PBL Undergraduate Medical Education
CHAPTER 4: SDL in PBL Research in Postgraduate Medicine 126
CHAPTER 5: Physician SDL in Clinical Practice 172
CHAPTER 6: A Closing Digest on SDL 209
BIBLIOGRAPHY 232
APPENDIX: Rating and Summary Form Used for Non-statistical 251
Meta-analysis on PBL and SDL Research in Postgraduate Medical
Training
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
LIST OF TABLES AND FIGURES
IV
Table 1: Theoretical and Quantitative Definitional Comparisons 12
of SDL
Table 2: General SDL Descriptions Reported in the Undergraduate 58
Medical Education Literature
Figure 1: SDL Process/Product Model in Medical Practice 132
Table 3: Summary of Research Threats to Validity 136
Table 4: Research Populations and Institutional Programs Utilized 146
by Studies Reviewed for Non-statistical Meta-analysis
Table 5: Non-statistical Meta-analysis Results 147
Table 6: Complete Listing of SDL Operationalized Measures Used 150
in Studies Reviewed for Non-statistical Meta-analysis
Table 7: Findings Reported by Studies Reviewed for Non-statistical 153
Meta-analysis
Table 8: SDL Data Reporting in Studies Reviewed for Non-statistical 165
Meta-analysis
Table 9: Possible SDL Examples/Activities for Undergraduate Medical 228
Education Based on Study Findings From the CME
Literature
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
V
ABBREVIATIONS
APA American Psychological Association
CC Conventional Curriculum
CL/AAL Commitment to Learning versus Apathy or Aversion to Learning
CME Continuing Medical Education
CO/D Cognitive Openness versus Defensiveness
DM Didactic Method
FM Family Medicine
GPEP General Professional Education of the Physician
LCME Licensing Committee for Medical Education
MAC Multidi sciplinary Ambulatory Clerkship
MCAT Medical College Admission Test
MCM Modified Case Method
NP Harvard’s New Pathway
OCLI Oddi’s Continuing Learning Inventory
PASS Performance Assessment of Self-directed Study
PBL Problem-based Learning
PD/RD Proactive Drive versus Reactive Drive
SDL Self-directed Learning
SDLRS Self-directed Learning Readiness Scale
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
ABBREVIATIONS (CONTINUED)
SRL Self-regulated Learning
SUI Southern Illinois University
TIE Triple Jump Examination
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
ABSTRACT
This dissertation reviews available literature on self-directed learning (SDL)
in problem-based learning (PBL) medical education curricula to assess whether PBL
trained medical students and physicians develop more effective SDL behavior than
their conventionally trained counterparts. Process/product dimensions of SDL in
PBL are described and an analysis by Everson & Hmelo (2000) is provided to
support the impact of PBL on SDL as defined by Candy (1991). Similarities and
differences between SDL and self-regulated learning are also described.
Undergraduate research examining SDL development in PBL and
conventional curriculum (CC) students appears to possess substantial methodological
limitations, including undeveloped conceptual SDL definition and inadequate
operational measures. Analysis of different study subsections from this research
compendium is provided to support the conclusion that PBL has not yet been found
to be superior to traditional medical education in developing SDL and that
undergraduate SDL in PBL research does not offer strong or clear evidence to
support the lasting impact of PBL on SDL.
Similarly, post-graduate research also is found to possess variable quality in
SDL conceptual and operational definitions. Definitions of SDL and/or PBL were
absent in many studies. The poor design of this research prohibited conducting a
meaningful, quantitative meta-analysis. Rather, a qualitative, or non-statistical meta
analysis, was performed to determine the extent of the relationship between the two
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
viii
variables. Three of nine studies reviewed found statistically significant results
favoring PBL graduates over CC graduates on SDL measures.
Recommendations for advancing SDL development in medical education and
research are proposed. Additionally, a supplemental review of the continuing
medical education (CME) literature reveals well-developed models of physician
learning. Analysis of SDL research in CME suggests that physicians may report
learning needs discrepant from actual observed learning needs, answer only a small
percentage of questions generated following patient encounters, and misinterpret
clinical research findings. Implications these finding may have for undergraduate
medical education are discussed and SDL-enhancing curricular strategies are
proposed.
In summary, a review of SDL in PBL undergraduate and post-graduate
medical education research does not support the hypothesis that problem-based
learners become more effective self-directed learners than those in conventional
medical curricula.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
1
CHAPTER 1
THE DEFINITION OF SELF-DIRECTED LEARNING
For hundreds of thousands of years, evolutionary development has depended
on the concept of self-directed learning (SDL). By necessity, modem man’s Homo
ancestors depended on self-directed learning for survival. There was no teacher to
instruct Homo habilis in his quest to pound a rock into a tool to enhance hunting and
building. From the harnessing of fire, to the construction of the wheel, and building
of civilizations, prehistoric man demonstrated tremendous capacity to successfully
direct his own learning.
In early modem history, precursors of self-directed learning in more
organized education can be identified in ancient Greek scholastic teachings, with
their encouragement of open forum reflective thinking, rhetoric and debate extolling
the attainment of virtue and morality. Nineteenth century America saw the growth
not only of privileged university education, but also greater permeation of
noninstitutionalized learning within the common ranks. With the expansion of
literacy via reading societies, circulating libraries, and adult Sunday schools, a new
culture practice emphasizing self-improvement and self-help for social advancement
and intellectual fulfillment was promulgated (Candy, 1991).
Advancement in the formal study of self-directed learning in adult education
has blossomed in the last thirty years. In the 1970s, educators began their quest to
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
2
more folly understand SDL by researching the formulated components of adult
learning projects (Tough, 1971; Tough, 1978). Quantitative measurement scales of
SDL began to emerge in the 1980s (Guglielmino, 1978; Guglielmino, 1989; Oddi,
1986; Oddi, 1987), also a time in which the field’s theoretical underpinnings began a
re-evaluation (Brookfield, 1984; Brookfield, 1985) that would continue into the 90s
(Candy, 1991; Garrison, 1997).
The Process and Product of SDL
During this evolution, SDL has undergone a periodic remodeling of its
definition that can be exemplified by the variety of paradigms circulated throughout
the literature. Knowles (1975) described SDL as “a process in which individuals
take the initiative, with or without the help of others, in diagnosing their learning
needs, formulating learning goals, identifying human and material resources for
learning, choosing and implementing appropriate learning strategies, and evaluating
learning outcomes” (p. 18). Brockett & Hiemstra (1991) characterize SDL as two
distinct but related dimensions, the first of which is “a process in which a learner
assumes primary responsibility for planning, implementing, and evaluating the
learning process” (p. 24). This definition, the authors suggest, is the notion of self
directed learning largely promoted in the professional literature. The second
dimension of SDL, referred to as learner self-direction, “centers on a learner’s desire
or preference for assuming responsibility for learning” (p. 24). In proposing this
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
3
two-dimensional definition, the authors suggest that self-direction in learning
comprises both the external characteristics of the instructional process and the
internal characteristics of the individual learner.
Brookfield (1985) supports this critical distinction between the techniques of
the SDL process and the autonomy or, ‘internal change in consciousness’ necessary
for its authentic achievement. SDL techniques are said to be concerned with
objective setting, appropriate resource selection, learning strategy design and
evaluation procedures. The autonomy necessary for the complete attainment of
SDL, Brookfield (1985) argues, is realized only when individuals exercise control
over their learning goals by making a conscious and informed choice with full
knowledge of alternative learning formats and possible learning activities. The
author further states, “When the techniques of self-directed learning are allied to the
adult’s quest for critical reflection and the creation of personal meaning after due
consideration of a full range of alternative value frameworks and action possibilities,
then the most complete form of self-directed learning is exemplified” (p. 30).
Candy (1991) also views SDL as the integration of both ‘process’ (method),
and ‘product’ (goal/outcome), but extends this conceptualization further when he
describes the construct as four distinct although related phenomena. The ‘process’ of
SDL includes both leamer-control - the mode of organizing instruction in formal
settings, and autodidaxy - the non-institutional pursuit of learning opportunities in a
natural setting. The ‘product’ or goal of SDL includes self-management - the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
4
willingness and capacity to conduct one’s own education, and self-determination or
personal autonomy, which Candy (1991) compositely defines as the extent to which
a person: 1) conceives of goals and plans; 2) exercises freedom of choice; 3) uses
the capacity for rational reflection; 4) has will power to follow through; 5) exercises
self-restraint and self-discipline; and 6) views himself or herself as autonomous (p.
125). Throughout his discussion, Candy (1991) also emphasizes that SDL does not
occur in isolation and must be regarded as both subject and context specific.
In a more recent discourse, Garrison (1997) proposes a model of SDL that
integrates the dimensions of self-management (task control), self-monitoring
(cognitive responsibility), and motivation (entering and task). In his analysis, the
author suggests that SDL is best viewed from a “collaborative constructivist”
approach, in which an individual takes responsibility for “constructing meaning
while including the participation of others in confirming worthwhile knowledge” (p.
19). Key concepts included in Garrison’s (1997) tripartite SDL model include
metacognitive awareness, the need for shared control between student and teacher in
the learning process, and a postulated link between SDL and critical thinking. This
model may have direct implications for medical education and will be examined
further in a later chapter.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
5
The Quantitative Measurement of SDL
Comparable to a Venn diagram, there is both overlap and separation among
these author’s conceptual definitions of SDL. A further dimension is superimposed
when one considers the intersection between theory and quantitative measurement,
or, in other words, the translation of SDL theory into operational research
definitions. The genesis of SDL quantitative measurement was largely provided by
Allen Tough (1971), who sought to investigate the nature of adult learning projects,
originally described as “a series of related episodes, adding up to at least seven
hours” where “more than half of the person’s total motivation is to gain and retain
certain fairly clear knowledge and skill, or to produce some other lasting change in
him self’ (p. 6). Tough (971) reported that adults commonly spent about 700 per
year at learning projects, while participating in a median of eight projects. Tough’s
(1971) learning project research, which constituted one of the first efforts to research
self-direction in learning, succeeded in reaching populations both inside and outside
of formal educational settings, and thereby helped to redefine the meaning of adult
education participation. Many limitations were seen in this form of research,
however, including researcher and memory biases inherent in the definitional use of
Teaming projects’, bias related to level of formal educational attainment, and a high
degree of variation within the interview process itself, the questions asked and the
data analysis procedures (Brockett & Hiemstra, 1991). Subsequent development of
SDL quantitative research occurred with the creation of two more widely used
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
6
measures - Guglielmino’s (1978) Self-Directed Learning Readiness Scale (SDLRS),
and Oddi’s (1986) Continuing Learning Inventory (OCLI). The SDLRS has been
used to explore the relationships between self-directed readiness and other
psychosocial variables, as well as a diagnostic tool for assessing learners’ perception
of readiness for self-directed learning (Brockett & Hiemstra, 1991). Factor analysis
of the instrument by Guglielmino (1978) indicated the presence of eight factors in
self-direction in learning: 1) openness to learning opportunities; 2) self-concept as
an effective learner; 3) initiative and independence In learning; 4) informed
acceptance of responsibility for one’s own learning; 5) love of learning; 6) creativity;
7) future orientation; and 8) ability to use basic study skills and problem solving
skills. Criticism has been directed at the SDLRS (Field, 1989) for, among other
reasons, the lack of definition of the term “self-directed learner.” In response to this
criticism, Guglielmino (1989) restated that the basis for construction of the scale was
determined through consensus by a Delphi panel (a three-round survey utilizing
fourteen authorities on self-direction in learning). The purpose of the construction of
the SDLRS was to “name and rate characteristics they (Delphi panel) considered
important for self-direction in learning, including such things as abilities, attitudes,
and personality characteristics” (Guglielmino, 1978). In this way, the conceptual
‘product’ or goal of SDL becomes quantitatively defined by the extent with which it
is thought to be associated with a given set of characteristics. As Candy (1991)
notes, “In view of the absence of a clear definition of major terms, there appears to
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
7
be some confusion as to precisely what is being measured. The attribute being
assessed is variously referred to, within the instrument itself, as an “attitude,” a
“preference,” a “learning style,” “readiness,” “skills,” “abilities,” and
“characteristics” (p. 153). Use of the SDLRS instrument will be further discussed in
chapter 3.
The Oddi Continuing Learning Inventory (OCLI) also views SDL from a
personality perspective and is based on “personality characteristics of individuals
whose learning behavior is characterized by initiative and persistence in learning
over time through a variety of modes” (Oddi, 1986, p. 98). Oddi (1986)
hypothesized that three clusters of personality dimensions were essential features of
self-directed learners: 1) proactive drive versus reactive drive; 2) cognitive openness
versus defensiveness; and 3) commitment to learning versus apathy or aversion to
learning. Factor analysis from Oddi’s (1986) original study data (n= 271) revealed
three principal components which accounted for 45.7% of the total variance: 1) a
general factor related to the author’s concept of proactive drive, which included high
self-esteem, self-confidence, and self-initiated and sustained learning directed toward
higher goals, in addition to two new characteristics reflecting the ability to work
independently and to learn through involvement with others; 2) a factor labeled
Ability to be Self-Regulating; and 3) a factor labeled Avidity for Reading. Oddi
(1986) also reports that scores were found to be significantly and positively
correlated with separate measures of educational participation (r = .363, p = .004),
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
8
self-confidence (r = .551, p < .0001) and affiliation (r = .265, p = .04). Scores did
not correlate with measures of adult intelligence, a finding Oddi (1986) states has
also been previously supported in other studies (Chickering, 1964; Gunzburger,
1980). Although Oddi’s (1986) three factors were shown to remain stable across
studies (Six, 1989), Brockett & Hiemstra (1991) relate concern that other evidence
was found (Landers, 1990) that suggested the internal reliability of the scale was
weak.
SDL Definition Limitations
The mixed reliability and validity findings from studies using either of these
two instruments as reported by Brockett & Hiemstra (1991, pp. 58-80) and Candy
(1991, pp. 150-155) emphasizes the reservation surrounding the operationalization of
SDL in the educational literature. While the authors are quick to suggest that these
instruments have indeed made a contribution to the knowledge base within the field
of SDL, they acknowledge their own as well as other authors’ substantive and
methodological concerns, noting that these concerns to a large extent can be linked
to questions over how SDL is defined and the theoretical foundation of the concept.
Mixed reliability and validity results from quantitative measurement scales fuels the
debate of how far an acceptable operational definition of SDL can be defined
through a collection of personality traits. In the midst of this debate, however, these
authors do not offer substantial suggestions for remedying this difficulty, except to
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
9
support further refinement of existing measures In response to identified limitations
of the previous Instruments, or to construct new ones (Brockett & Hiemstra, 1991).
Perhaps one minor problem with operationalizing SDL via personality measures can
be viewed according to the necessary and sufficient logic argument. Which variables
are simply necessary for the internalization of SDL as a product to occur, and which
are both necessary and sufficient? Candy (1991) notes that researchers have
identified well over 100 competencies that they have linked with successful
independent learning. A more major problem with the issue of mixed validity results
in this area, however, may be that investigators have attempted to measure and
generalize SDL personality traits that may well exist as contextual and situation-
specific characteristics. As argued by Candy (1991), while there may be generic
self-directed learning readiness components, it is more probable that this readiness is
shaped by the construction of the particular situation and circumstances. Candy
(1991) points to research from the Goteborg Group in Sweden and the Institute for
Research in Post Compulsory Education at Lancaster University in England
(Entwistle & Hounsell, 1979; Marton, Hounsell, & Entwistle, 1984) as confirmatory
findings of the situation-specific or context-bound nature of learning competence,
and argues that this specificity likely extends to autodidactic learning as well. This
debate may be comparable to a former one in educational research concerning the
generalizability versus context-specific nature of problem solving.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
10
A final illustration over the confusion in defining SDL in quantitative
research can be found in Brockett & Hiemstra’s (1991) discussion of findings from a
meta-analysis of quantitative studies on self-direction conducted by McCune
(1989a). Using individual studies reported between 1977 and 1987, the authors
report that in the 67 studies included by McCune (1989a), 18 different approaches
were used to measure self-direction. The most frequent measures were
Guglielmino’s SDLRS, the Tough interview schedule (or modified version of the
schedule), reported number of hours devoted to self-directed learning activity, and
course participation/persistence/ completion. Although review of the McCune
(1989a) citation (dissertation abstract) did not reveal the extended findings noted by
Brockett & Hiemstra (1991), one common finding reported in these two citations
concerned an association found by McCune (1989a) between SDL and seven
demographic or psychosocial/behavioral variables. These variables included:
degree of SDL activity (r = .242); positive self-concept (r = .230); educational
attainment level (r = .200); self-development (r = .194); autonomy (r = .165); ability
to master the environment in work, school, play, or social relations (r = .147); and
factors related to longevity on the job (r = -.138).
The research on SDL in medical education will be discussed in chapter 3;
however, it is significant to note at this point that the continuing difficulty with SDL
operationalization has extended into quantitative studies within this field as w ell It
is perhaps this discrepancy within SDL quantitative research that prompted Candy
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
11
(1991) to encourage the use of research approaches within a predominantly
interpretive framework (“If ever there were a topic that lent itself to, or even
demanded, an idiographic or case study approach, it must be the phenomenon of self
directed learning,” p. 452), or for Brookfield (1984) to express the concern that the
adoption of quantitative instruments places an overemphasis on the quantity of self-
directed learning, largely to the exclusion of assessing its quality or effectiveness.
Table 1 lists several theoretical and quantitative definitional comparisons of
SDL as discussed by different authors in the SDL general education literature.
While similarities can be noted, also evident are conceptual and quantitative
distinctions among authors in their Individual construction of SDL definitions.
Numerous authors discuss the confusion of SDL in the educational literature
(Brookfield, 1984; Oddi, 1987; Candy, 1991; Brockett & Hiemstra, 1991) and as
Candy (1991) recounts, “.. .the literature on self-direction is extensive, but it is also
confusing. The lack of internal consistency precludes the possibility of developing a
coherent theory of self-direction, or even of self-directed learning, from within the
literature itself (p. 411).”
Review of the adult educational literature has illustrated the progression of
SDL conceptualization, from Tough’s (1971) adult learning projects and Knowles’
(1975) process-based evaluation, to Brookfield’s (1984, 1985) and Candy’s (1991)
theoretical distinction between procedural skill and attitudinal consciousness, to
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
12
TABLE 1 - THEORETICAL AND QUANTITATIVE SDL DEFINITIONAL
COMPARISONS
Author SDL Definition
Skager, Dave &
Robinson (1977)
Guglielmino (1978)
Theoretical
SDL defined as “individualization of the learning
experience toward the goal of developing the learner’s
own skills and competencies in the planning, execution
and evaluation of learning activities both as an individ
ual and as a member of a cooperative learning group.”
Quantitative
Self-directed Learning Readiness Scale; self-report
questionnaire; “Factor analysis indicated the presence
of eight factors in self-direction in learning: openness
to learning opportunities, self-concept as an effective
learner, initiative and independence in learning,
informed acceptance of responsibility for one’s own
learning, love of learning, creativity, future orientation,
and the ability to use basic study skills and problem
solving skills.”
Della-Dora &
Blanchard (1979)
Theoretical
SDL “refers to characteristics of schooling which
should distinguish education in a democratic society
from schooling in autocratic societies”
Knowles (1985)
Brookfield (1985)
Theoretical
SDL described as “a process in which individuals take
the initiative, with or without the help of others, in
diagnosing their learning needs, formulating learning
goals, identifying human and material resources for
learning, choosing and implementing appropriate
learning strategies, and evaluating learning outcomes”
On SDL: .we should distinguish between the
techniques of self-direction and the internal change in
consciousness we might label as self-directed learning.
The technique of self-directed learning is., .discernible
in terms of setting realistic and achievable objectives,
locating and choosing appropriate resources, designing
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
TABLE 1 - THEORETICAL AND QUANTITATIVE DEFINITIONAL
COMPARISONS OF SDL (CONTINUED)
Author
Brookfield (1985)
(con’t)
Oddi (1986)
SDL Definition
learning strategies, and generating evaluative
procedures.. .Self-directed learning as the mode of
learning characteristic of an adult who is in the process
of realising his or her adulthood is concerned much
more with an internal change of consciousness.. .This
consciousness involves an appreciation of the context-
uality of knowledge and an awareness of the culturally
constructed form of value frameworks, belief systems
and moral codes influencing behavior and the creation
of social structures. The most complete form of self
directed learning occurs when process and reflection
are married in the adult’s pursuit of meaning.”
Theoretical
“The perspective in the present study...focused
on the personality characteristics of individuals whose
learning behavior is characterized by initiative and
persistence in learning over time through a variety
of learning modes, such as the modes of inquiry,
instruction, and performance proposed by Houle
(1980). The term “self-directed continuing learning”
was adopted to differentiate this broader view from
“self-directed learning,” a term which is generally
used in reference to a self-instructional process.”
Quantitative
Oddi Continuing Learning Inventory; 24-item self-
report questionnaire; includes 3 dimensions:
Proactive Drive versus Reactive drive (PD/RD);
Cognitive Openness versus Defensiveness (CO/D);
Commitment to Learning versus Apathy or Aversion to
Learning (CL/AAL); “Factor I contained 15 salient
items and accounted for 30.9% of the variance; it
represented elements from the PD/RD dimension of
the theoretical formulations.. .and gained additional
items reflecting the ability to work independently and
to learn through involvement with others.”
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
TABLE 1 - THEORETICAL AND QUANTITATIVE DEFINITIONAL
COMPARISONS OF SDL (CONTINUED)
Author
Caffarella &
O ’Donnell (1989)
Brockett &
Hiemstra (1991)
Candy (1991)
SDL Definition
Theoretical
SDL - a construct with 2 themes: “(1) self-directed
learning as a phenomena [sic], it is a form of learning,
and (2) self-directed learning as a personal attribute,
individuals are self-directed in their learning”
Theoretical
. .self-direction in learning refers to two distinct but
related dimensions. The first of these dimensions is a
process in which a learner assumes primary respons
ibility for planning, implementing and evaluating the
learning process.. .The second dimension, which we
refer to as learner self-direction, centers on a learner’s
desire or preference for assuming responsibility for
learning...Thus, self-direction in learning refers to
both the external characteristics of an instructional
process and the internal characteristics of the learner,
where the individual assumes primary responsibility
for a learning experience.”
Theoretical
“ .. .the term self-direction actually embraces
dimensions of process and product, and that it
refers to four distinct (but related) phenomena:
“self-direction” as a personal attribute (personal
autonomy); “self-direction” as the willingness and
capacity to conduct one’s own education (self
management); “self-direction” as a mode of
organizing instruction in formal settings (leamer-
control); and “self-direction” as the individual,
noninstitutional pursuit of learning opportunities
in the “natural society setting” (autodidaxy).”
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
15
TABLE 1 - THEORETICAL AND QUANTITATIVE DEFINITIONAL
COMPARISONS OF SDL (CONTINUED)
Author SDL Definition
Garrison (1997) Theoretical
“Self-directed learning is defined here as an approach
where learners are motivated to assume personal
responsibility and collaborative control of the
cognitive (self-monitoring) and contextual (self
management) processes in constructing and confirming
meaningful and worthwhile learning outcomes.”
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
16
Garrison’s (1997) model featuring the integration of contextual, cognitive and
motivational dimensions of SDL. Chapter 2 introduces the concept of SDL within
medical education, and discusses the extent to which problem-based learning - the
major curriculum strategy associated with SDL - is thought to promote its
development. The procedural and outcome dimensions of SDL also will be
considered in the context of medical education. To help guide this discussion, a
preliminary definition of SDL is offered here. The procedural component of SDL is
defined as the set of skills necessary to fulfill a designated learning task, including
the: 1) accurate self-assessment of current knowledge; 2) articulation o f learning
needs and objectives; 3) appropriate utilization of learning resources; 4) use of
effective learning strategies; 5) application of new knowledge; and 6) objective self-
evaluation of learning outcomes. The goal of SDL is defined as persistent learning
behavior associated with self-initiated, disciplined and sustained motivation required
to critically identify and pursue one’s learning needs to create intentional, purposeful
and permanent change in learning throughout one’s lifetime.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
17
CHAPTER 2
THE USE OF PROBLEM-BASED LEARNING (PEL) IN MEDICAL
EDUCATON TO PROMOTE SDL
In chapter 2, analysis of the specific steps involved in PEL instruction is
presented and the extent to which seven SDL-fostering strategies discussed in the
adult education literature transfer to a PBL context is considered. Examples of PBL
methodology are described that lend theoretical support for potential SDL process
and product development. Uncertainty regarding classification of SDL-promoting
strategies as either process or product dimensional components is considered and
proposal for dual classification representing a process/product continuum is made.
The comparison and contrast of SDL with self-regulated learning (SRL) reveals
similarities in the areas of learning objective identification, strategy use/resource
selection, and self-evaluation. Four proposed differences are identified in the areas
of learning task initiation, learning environments/research populations, critical
thinking/knowledge application, and study of learning motivation. Research
supporting a positive association between SLR and academic achievement is
presented.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
18
SDL in Medical Education
Medicine is arguably the leading profession where one must keep pace with
information from an already unmasterable body of knowledge, only to see that same
information change as rapidly as it was established. The four-year medical education
curriculum can present only the veritable tip of the proverbial iceberg of information
and skills that a physician will need throughout his or her career. The medical
education literature addresses the importance of medical school graduates to be
effective self-directed learners, not only to keep up with changing information and
patient management practices (Collins & Hammond, 1987; Jennett, 1992; Spencer &
Jordan, 1999; Towle & Cottrell, 1996), but to keep abreast of diagnostic and
technological advances, changing expectations of patients, society, and the
government, and shifting professional standards as well (jennett & Swanson, 1994).
Toward that goal, medical education has put an increasing emphasis on
incorporating SDL into the undergraduate curriculum over the past two decades. In
their influential 1984 report, the General Professional Education of the Physician
Panel (GPEP) (Panel on the general professional education of the physician and
college preparation for medicine, 1984) concluded,
.. .a general professional education should prepare medical
students to leam throughout their professional lives rather than
simply to master current information and techniques. Active,
independent, self-directed learning requires among other qualities
the ability to identify, formulate, and solve problems; to grasp
and use basic concepts and principles; and to gather and assess
data rigorously and critically (p. 9).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
19
The GPEP panel recognized the instructional hazards inherent to the highly
rote, passive form of medical education that pervaded most U.S. medical schools and
advocated the reduction of lecture hours and expansion of independent learning and
problem solving, specifically suggesting that, “problem solving through learning
independently will better prepare medical students to use new knowledge and new
technologies throughout their careers” (p. 12). Concurrently, the GPEP panel also
addressed the importance of independent, lifelong learning in continuing medical
education to help physicians adapt their medical practice to new knowledge and new
techniques.
The continuing importance placed on SDL development since the
establishment of the GPEP Report (1984) is now reflected in the accreditation
standards for medical education. In their latest guidelines, the Liaison Committee on
Medical Education (2003) states in a section on curricular design:
The medical faculty must design a curriculum that provides
a general professional education, and fosters in students
the ability to learn through self-directed, independent study
throughout their professional lives, (p. 2)
SDL Promotion Through PBL
What has transpired from this clarion call to promote SDL in medical
education? Although the GPEP and LCME dicta do not specify particular
educational strategies for promoting SDL, one popular educational reform that has
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
20
Incorporated SDL development is PBL. While SDL may be alternatively
incorporated into medical education, PBL is most consistently discussed in the
literature as a formal Instructional technique utilized to promote SDL. As this
association of SDL within PBL curricula appears well established and widely
utilized in medical education, SDL within PBL education will be the primary focus
of this paper In an effort to best characterize the discussion and research in this area.
To characterize SDL in actual medical practice, an abbreviated review o f general
SDL behavior in established physicians, without consideration of undergraduate
curriculum, Is presented in chapter 5.
As an alternative educational approach, PBL was fust introduced into
medical education in the 1960s, largely pioneered at The School of Medicine at
McMaster University in Ontario, Canada in 1969. Dr. Howard Barrows (1986) was
the first person to establish a formal taxonomy of PBL, and is often cited in
definitions or descriptions of PBL (Albanese & Mitchell, 1993; Foley, Poison, &
Vance, 1997; Lloyd-Jones, Margetson, & Bligh, 1998; Papa & Harasym, 1999).
PBL, as originally conceived, is a small group (5-6 students), student-centered,
faculty-facilitated teaching method centered on the presentation of an authentic
patient problem without the benefit of prior study. Presentation of a patient problem
is given much as an actual patient presents to a physician in reality (e.g., “I’ve come
to the clinic because I’ve had this cough...”). The case is presented in a series of
steps (e.g., history, physical examination, laboratory information, etc.), and at each
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
21
step, students collaboratively identify problems, generate and refine a list of possible
hypotheses, discuss their knowledge of basic science as it is thought to relate to
clinical symptomology, and identify learning issues to be researched. In the pre-
clinical years, greater emphasis is directed toward determining the underlying
mechanisms responsible for the patient’s symptoms and signs in terms of altered
biochemistry, physiology, anatomy, etc. rather than on achieving a correct diagnosis.
According to Barrows (1983), “this process is characterized by early generation of
multiple hypotheses, the use of a problem-oriented inquiry strategy to decide among
these hypotheses to resolve the problem, and the ability to make diagnostic and
therapeutic decisions in the face of sometimes ambiguous data” (p. 3078). Students
are given a period o f time to research their individual learning issues and the group
later reconvenes for a second session in order to discuss and integrate their new
information into the case while continuing the problem-solving process.
Barrows (1986) asserts that four objectives are of primary importance in
PBL: 1) Structuring of knowledge for use in clinical contexts; 2) developing an
effective clinical reasoning process; 3) the development of effective self-directed
learning skills; and 4) increased motivation for learning. These objectives
correspond with the PBL goals cited throughout the literature (Hmelo, 1998; Norman
& Schmidt, 1992; Thomas, 1997). An additional cited PBL goal is to make
knowledge more relevant and retrievable by teaching basic science in the context of
a clinical problem (Walton & Matthews, 1989). Some 100 medical schools in the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
22
United States have utilized PBL (Finucane, Johnson, & Prideaux, 1998) in a variety
of applications, and it is now an entrenched component of medical school programs
in Canada, the United Kingdom, The Netherlands, Sweden, Australia, the Middle
East and Asia.
Strategies Used to Promote SDL
To examine the extent to which PBL methodology incorporates learning
practices and strategies believed to encourage SDL, those SDL-promoting strategies
professed in the educational literature will be reviewed and compared to PBL
practices. Candy (1991) adheres to Ms caveat that SDL skills are best viewed as
situation and context-specific when he suggests that these skills are best taught and
learned in the context of real learning tasks. The author asserts that in addition to
general learning skills (reading, questioning, information seeking, etc.), other skills
such as time management, critical thinking, goal setting, and problem solving are
particularly central to self-directed learning efforts. Candy (1991) specifically cites
the following educational interventions as potentially encouraging self-direction: 1)
making use of learners’ existing knowledge; 2) encouraging deep-level learning; 3)
increasing question-asking by learners; 4) developing critical thinking; 5) enhancing
reading skills; 6) improving comprehension monitoring; and 7) creating a supportive
climate for learning (p. 322). In discussing the promotion of self-direction in
learning, Candy (1991) does not clearly differentiate these abilities as characteristics
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
23
belonging to either his ‘process’ or ‘product’ SDL model. Their initial discussion
appears under the subheading ‘Approaches to Increasing Independence in Learning’
(p. 322), and thus may lead the reader to presume that these abilities fit more closely
with SDL as an instructional process. However, Candy (1991) also suggests that
critical thinking can be encouraged as both a skill and an attitude or predisposition.
He also later implies that the seven aforementioned attributes are intellectual or
cognitive aspects of self-direction or autonomy (half of the author’s earlier
mentioned ‘product’ conceptualization). This point is mentioned here for two
reasons. First, other authors (Brockett & Hiemstra, 1991) describe three similar SDL
promotional strategies - facilitating critical reflection, promoting rational thinking,
and using helping skills in the facilitation process (p. 133) - as enhancing the
learner’s internal motivation or personal responsibility (goal) dimension of SDL.
This obscuring of SDL in a discussion of its promotion as either an instructional
process or an attitudinal goal augments the definitional confusion outlined in chapter
1. Secondly, this argued partition of SDL as an instructional strategy and a learner
attitude or goal may represent a further obstacle if authors conceive and
operationalize the construct as distinctly dichotomous rather than holistic. As SDL
definitions lack agreement, so too may authors’ categorizations of defining not only
what variables are significant for the promotion of SDL as a goal or a process, but
furthermore, which variables constitute instructional strategies, which constitute
attitudes, and which may be elements of both. For the purposes of the present
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
24
analysis, the identified ‘strategies’ listed above will be considered potential elements
of both SDL instructional process as well as product and this point will be further
discussed after reviewing these strategies.
Association of SDL-enhancing Strategies with PBL Methodology
Each of the proposed SDL strategies can be expounded on and considered
within a PBL context. The first strategy, making use of learners’ existing
knowledge, is important for integrating new information into previous knowledge
structures, or schemata, that are familiar to learners in order to construct meaning
from an established point of reference. The assimilation and comprehension of new
knowledge are known to be weak when prior schemata are not activated during
learning (Bruning, Schraw, & Ronning, 1999; Norman & Schmidt, 1992). Two
methods Candy (1991) supports for activating prior knowledge are concept maps,
which can link ideas in multidimensional ways, and ‘repertory grids’, which simply
described, present qualitatively similar elements to learners for the purpose of
making comparisons and contrasts. Small-group discussion of a problem is noted by
Norman & Schmidt (1992) to be another method utilized in activating prior
knowledge. This feature is employed in PBL by encouraging students to review and
discuss information they have learned that may in any way relate to the patient’s
presenting problem. Activating prior knowledge is also encouraged in PBL to
potentially stimulate information that may otherwise be beyond conscious recall
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
25
(Barrows, 1986), or to reveal incorrect knowledge that, if not addressed, allows
instruction to provide a student “with new terminology for expressing his erroneous
beliefs” (McCloskey, Caramazza, & Green, 1980, p. 1141). Interestingly, an
approach similar to a concept map, referred to as mechanistic case diagramming, has
been used at one medical school to facilitate clinical reasoning in both the initial and
reformulative, or second, phase of PBL (Guerrero, 2001).
The next two SDL strategies identified concern the encouragement of inquiry
and deep-level learning. Deep-level learning attempts to facilitate qualitative change
in learners’ understanding by enhancing comprehension of underlying concepts,
principles and potential connections between related principles within a given area of
study. In contrast, superficial or surface learning encourages the rote memorization
of learning material. Increasing questioning behavior is seen as one strategy used to
encourage deep-level learning (Candy, 1991). Deep-level learning and inquiry are
promoted in PBL by consistently encouraging students to ask questions regarding the
patient’s presentation and to generate multiple hypotheses concerning the clinical
problem. With the generation of these hypotheses, students are challenged to
determine associations between possible underlying biological mechanisms and
manifestations of patient symptoms, thereby making connections between principles
and integrating basic and clinical science knowledge. In theory, PBL facilitators are
not meant to act as knowledge experts. Thus, rather than supplying answers to
students, facilitators encourage students to identify what they clearly know and do
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
26
not know, and help them prioritize any information they may need to learn in order
to understand the interrelationship of case elements.
Critical thinking is a proposed SDL enhancing behavior endorsed by both
Candy (1991) and Brockett & Hiemstra (1991). Candy (1991) notes that this is
another educational topic where no clear definitional consensus exists; the author
suggests that debate on the subject has focused on whether this skill can be
developed as a general reasoning capacity or whether critical thinking exists as a
discipline-specific skill. Perkins & Salomon (1989) suggest that cognitive skills of
this nature can exist as both: . .there is something disturbing about
casting.. .general and contextualized as though they were exclusive of one
another.. .There are general cognitive [and critical thinking] skills; but they always
function in contextualized ways” (p. 19). Both Candy (1991) and Brockett &
Hiemstra (1991) refer to Brookfield (1987) in their discussions of critical thinking.
Brookfield (1987, pp. 7-9) identifies four components of critical thinking:
identifying and challenging assumptions; recognizing the influence of context on
thoughts and actions; having the capacity to imagine and explore alternatives to
existing ways of thinking; and developing reflective skepticism toward universal
truths or ultimate explanations simply because they have existed over time. Candy
(1991) considers critical thinking to additionally include components such as making
learners aware of the conventions and principles of the subject being learned,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
27
teaching what sorts of questions to ask and when, and showing how much and what
Mud of evidence is enough to determine the validity of a statement (p. 331).
Brookfield (1987) additionally identifies five phases of critical thinking: a
triggering event; appraisal; exploration; developing alternative perspectives and
integration. The triggering event exists as some sense of inner discomfort or
perplexity; self-scrutiny and appraisal of the situation then follows. Exploration
progresses by searching for new ways to explain pertinent discrepancies. Option
testing, being open to new information and searching for meaning occurs in this
phase. Alternative ways of thinking or acting then arise from exploration. After
deciding on the worth, accuracy and validity of the new ways of thinking, they then
become integrated within our cognitive structure. The application of critical thinking
within a PBL instructional context is proposed to occur as a result of working
through and problem solving the case. The procedural steps involved can be roughly
compared with Brookfield’s (1987) five phases of critical thinking: a student
encounters a new clinical situation for which he does not have experience (triggering
event - perplexity); he is asked to recount all relevant pertinent knowledge and asked
how it may relate to the case (appraisal); he proceeds to ask questions and generate
hypotheses about the case (exploration); he identifies personal learning issues that
may help Mm to better understand the case, researches these issues and is asked to
explain their application to the case (developing alternative perspectives;
integration). Barrows and Tambiyn (1980) offer the following description:
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
28
By working with an unknown problem, the student is forced
to develop problem-solving, diagnostic, or clinical reasoning skills.
He must get information, look for cues, analyze and synthesize the
data available, develop hypotheses, and apply strong deductive
reasoning to the problem at hand (p. 15).
Barrows (1983) also cites as an educational objective of PBL “to encourage
independent, critical thinking skills (p. 3078). Walton and Matthew (1989) contend
that PBL “is a way of enabling students to develop the required reasoning and
critical thinking more effectively than they can do in .. .traditional methods” (p. 543).
Elements of critical thinking listed above can be associated with a PBL case
discussion through a variety of facilitation means, including: encouraging
questioning by students to obtain thorough patient data; challenging students to make
connections between pieces of information and between patient symptoms and
scientific principles; asking students to explain how different patient variables (e.g.,
demographics, habits) may change their thinking in the case; challenging students to
discriminate between levels of data significance, while defending their progression
of thinking and rationale for hypotheses generation; seeking logical associations
from students between history, physical examination and laboratory data; inquiring
how students will validate hypotheses or deal with ambiguous data; and encouraging
students to self-assess their own individual level of knowledge, understanding and
reasoning as being distinct from assumption or conjecture. Through this process,
rudiments of critical thinking such as challenging assumptions and hypothesis
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
29
assertions, seeking alternative explanations, discussing contextual significance, and
providing opportunity for analytical reflection and self-evaluation are stimulated.
The next SDL-promoting interventions of enhancing reading skills and
improving comprehension monitoring may be identified briefly in the initial PBL
session when students are encouraged to clarify any unfamiliar terms and concepts
using notes or texts; however, these skills become more evident in the second PBL
session. Between the two sessions, students have spent time studying different
learning issues they are responsible for explaining to the group, and analyzing how
this information now impacts the patient case (e.g., help to rule in or rule out certain
hypotheses, clarify associations between symptoms and laboratory data, etc.). As
Barrows (1983) expounds:
After a period of time sufficient for self-directed learning.. .they
return to start over with the patient problem.. .and together as a group
reason their way through the problem and decide on the underlying
mechanisms responsible. In doing this, they critique their prior
thinking: hypotheses, choice of questions and examinations
for inquiry, analysis and synthesis of the patient data, and their
decisions...When students are finished with a problem, they do a
“post-mortem” critiquing their inquiry, decisions, treatment, and
self-directed study to determine how such a problem might be better
evaluated and managed and how they might solve problems better in
the future. In a very important last step in the sequence, the group
attempts to summarize what they learned and to integrate it with
their existing knowledge (p. 3079).
Candy (1991) borrows a four-step model from Morris & Stewart-Dore (1984,
p. 30) in which the authors promote the following stages to enhance reading skills:
1) preparing for reading; 2) thinking through the reading; 3) extracting and
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
30
organizing information; and 4) translating information. These stages may be viewed
as correlating with the reading requirements in PBL. First, students prepare for
reading through the identification of their learning issues. They then perform their
reading while thinking about the information as it relates to the variables presented
within the patient case. During this time, they extract and organize the new
information in order to discuss the areas that are directly relevant to the PBL case.
Lastly, students translate the new information for the group in order to clarify how it
impacts their construction of diagnostic hypotheses or understanding o f basic and
clinical science integration. With the anticipation of discussing and analyzing their
previously studied learning issues within the group, students are being asked to read
for understanding and comprehension as they apply their new knowledge within the
PBL case.
The final SDL-promoting strategy endorsed by Candy (1991), the creation of
a supportive climate for learning, is similar to what Brockett & Hiemstra (1991) refer
to as ‘helping skills,’ which include respect and genuineness and basic
communication skills (listening, attending, empathizing and probing) in the
facilitator-leamer relationship. Candy (1991) identifies such supportive learning
elements as low threat, unconditional positive regard, honest and open feedback,
respect for the ideas and opinions of others, approval of self-improvement as a goal,
and collaboration rather than competition (p. 337). O f the environment envisioned
for PBL, Barrows (1983) states:
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
31
To be successful, a climate of openness between students and
teachers and an ability to offer and accept constructive criticism is
established. The students should feel free to offer their knowledge
and opinions openly, even though they may be naive or wrong.
Learning is most effective when students are able to freely reveal
what they know and believe (p. 3078).
Literature Appraisal of the Enhancement of SDL in PBL
In an infrequent reference found in the literature that purposely addresses the
association between SDL and PBL in medical education, Hmelo & Lin (2000)
discuss several features of PBL that they assert specifically support the development
of SDL skills. Initially they suggest a definition of SDL that states:
In order to go beyond what is known and to create new knowledge
for novel situations, they [physicians] need to know what they need
to learn about a particular situation, what particular medical procedures
and resources are appropriate for the situation, how to formulate
questions to seek the specific information needed, and how the
knowledge (new or old) can be applied in practice. Collectively, these
abilities define what has been referred to as expertise in self-directed
learning (SDL) skills or metacognitive learning abilities (p. 227).
The authors suggest that PBL features capable of supporting SDL
development include: 1) the nature of PBL as a student-centered learning method; 2)
students’ use of existing knowledge in their attempts to identify and solve problems;
3) the identification of knowledge deficits and generation of appropriate learning
issues; 4) students’ independent research efforts; 5) critique of the resources used for
research; 6) the application of new knowledge to the problem; and 7) the
collaborative reflection on SDL (p. 229). Hmelo & Lin (2000) then offer their
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
32
assessment of how these features promote SDL. First, by interacting with facilitators
who ask students to reflect and elaborate on their thinking (e.g., “Why do you want
to know that?”, “What more do you need to know?”, “What exactly are you
thinking?”), the authors contend that PBL supports the gradual transfer of assessment
to students, and that students will begin to internalize these questions and pose them
to themselves in a metagcognitive fashion as they continue to leam. Furthermore, by
recognizing the deficiencies in their knowledge base as they attempt to solve patient
problems, students continue to develop self-assessment ability and also leam how to
generate their own personal learning needs and to plan for further learning. By
activating prior knowledge, knowledge deficits can be articulated as learning goals
relative to a targeted problem, and new information can become more easily
assimilated. Hmelo & Lin (2000) assert that inquiry and the process of creating
individual learning opportunities serve to motivate students in their learning.
By being presented with the task of independent research learning, Hmelo &
Lin (2000) argue that students become more proficient in not only locating
appropriate information resources, but also in posing questions to experts when
necessary. With this opportunity, students are able to construct conceptual
knowledge as well as procedural knowledge for solving problems, which prepares
them to become more adaptive in their learning throughout a range of novel
situations. Hmelo & Lin (2000) also suggest that the critical evaluation of their
resources is an important feature of PBL that promotes SDL:
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
33
They [students] need to consider the reliability of the resources
and how those resources contribute to knowledge construction.
Students might consider the date of a publication, whether it was
from a respected source that underwent an appropriate review process
and how useful that resource might be for a future problem (pp. 230-1).
Two other SDL enhancing features of PBL methodology addressed by the
authors are the application of students’ new knowledge and the reflection of the SDL
experience. By applying their research to solving a patient problem, Hmelo & Lin
(2000) state that students leam to distinguish relevancy of information as well as
how knowledge can be used as a tool. The authors refer to studies conducted by
Bransford and colleagues (Bransford, Franks, Vye, & Sherwood, 1989; Bransford &
Nitsch, 1978; Bransford, Sherwood, Vye, & Rieser, 1986; Bransford & Stein, 1993)
that support the proposition that students “who leam in ways that facilitate an
understanding of the relevance of information are more likely to develop
contextualized knowledge structures that connect isolated pieces of
information.. .Furthermore, contextualized knowledge is important in problem
recognition and in monitoring problem solving” (p. 231). Reflection of the SDL
process is suggested to occur when students consider the usefulness of their
knowledge construction, the processes in which they engage to achieve their learning
goals, and the effectiveness of their strategies and how these might be improved in
the future. Hmelo & Lin (2000) assert that reflection is critical to SDL in
transferring strategies and knowledge to new situations. Process-oriented reflection,
in which students monitor and evaluate the quality of PBL learning processes, is said
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
34
to engage students in metacognitive monitoring and regulation of their own learning.
This enhances students’ ability to “identify what they need to understand, build
connections between the procedures and concepts, and identify the causal
mechanisms underlying a wide range of problems” (p. 231).
Lastly, Hmelo & Lin (2000) contend that theoretical support for the
effectiveness of PBL in the development of SDL can be drawn from information-
processing theories of transfer and sociocultural theories such as cognitive
apprenticeship:
Information-processing theories of transfer-appropriate processing
suggest that, in order for knowledge and strategies to be used in
problem-solving situations, they must be learned in such contexts
(Adams et ah, 1988; Needham & Begg, 1991; Perfetto, Bransford, &
Franks, 1983). In PBL, students continually apply their knowledge and
practice their SDL strategies in problem-solving contexts, so this theory
predicts that there should be transfer to novel problems. Because the
students in a problem-based curriculum have opportunities to use their
knowledge in a large variety of cases, they should be able to flexibly
apply their SDL strategies to new situations. Moreover, the reflective
activities in the tutorial groups enhance the likelihood that the students
will be able to apply their SDL strategies in a range of situations
(Salomon & Perkins, 1989).
Sociocultural theories offer additional support for the development
of SDL skills in PBL. Cognitive apprenticeship models of instruction
note the importance of situating learning in authentic tasks such as
the medical cases used in PBL. Because the problems are complex,
the students need various supports, also called scaffolding, to help them
deal with the complexity of the problems. Scaffolding may take the
form of modeling of expert thinking and coaching provided to learners
as they try to apply new strategies (Collins, Brown, & Newman, 1989).
Sociocultural theorists suggest that through participation in the tutorial
discourse, students internalize the thinking processes that are initially
made external through the group’s discussion (Wertsch & Bivens, 1993)
(p. 231).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
A Closer Look at the Process versus Product View of SDL
35
The earlier issue regarding the obscuring between the process versus outcome
characterization of SDL strategies is now reconsidered. As noted, Candy (1991)
does not clearly delineate the strategies he refers to as supporting either aspect of
SDL, and loosely refers to critical thinking as capable of being both. Brockett &
Hiemstra (1991) distinguish those strategies they believe educators can use to
support the internal, personal orientation of SDL, but limit their discussion to three
main approaches - critical reflection, which also encompasses reading and writing,
promoting rational thinking, which involves controlling irrational beliefs and taking
personal responsibility, and the use of helping skills (e.g., communication, respect).
Hmelo & Lin (2000), however, did not specifically discuss SDL development in
terms of a process/product division, but rather addressed the concept holistically.
The lack of agreement in this area is comparable to the absence of SDL
definitional agreement explicated in chapter 1. How to define SDL and how best to
develop it appears to vary depending on which author one happens to be reading.
Perhaps the further the definition of SDL is deconstructed, the greater the difficulty
becomes in articulating the specific nature of the presumptive cognitive changes
thought to be occurring during its development. This may represent one reason why
some authors prefer to view SDL as a generalized concept.
One may contend that many, if not all, of the previously discussed techniques
could exist as both a process and an outcome. A learner can be taught how, or the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
36
steps one uses to engage in critical thinking, or a self-evaluation process. A learner
may just as well be encouraged to adopt these skills as SDL attitudes, or goals. For
example, a person who internalizes critical thinking or self-assessment as an attitude
or consciousness may routinely analyze and evaluate new information for its quality
and reflect upon its assimilation within existing cognitive frameworks. A critical
thinker assesses his understanding of the information, considers its application, and
is able to identify when further learning is required. This view is similar to Wertsch
& Bivens’ (1993) premise previously presented by Hmelo & Lin (2000) that students
internalize thinking processes that are originally externally developed in the PBL
discussion process.
SDL Process and Product Dimensions in PBL
PBL instruction theoretically provides an opportunity to practice each of the
abilities previously defined as SDL processes (skills associated with self-assessment,
learning needs identification, learning strategies and resource utilization, knowledge
application and learning outcome self-evaluation) within the learning environment of
a patient case presentation. However, critical thinking, self-assessment, deep-level
learning, reading and comprehension monitoring, etc. can also be viewed as
components of a supporting framework necessary to achieve the goal of SDL
(persistent learning behavior associated with self-initiated, disciplined and sustained
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
37
motivation required to critically identify and pursue one’s learning needs to create
intentional, purposeful and permanent change in learning throughout one’s lifetime).
PBL potentially fosters the development of SDL as a goal by modeling a
process that attempts to instill within students the appreciation of a learning attitude
and associated behavior considered to be a valuable and necessary asset to
postgraduate, lifelong learning. Training students in PBL to adopt a calculated,
disciplined and reflective approach to learning is an attempt to infuse learners with a
motivation and commitment to learning by using a process that simulates a realistic
approach to patient management; it is an initial step to incorporating a method of
thinking and behaving that signifies medical professionalism. In SDL, it appears that
those promotional skills previously discussed are taught initially to students as
conscious, deliberate learning strategies that with practice may develop into
unconscious, internal constituents that comprise the attitudinal foundation o f a
disciplined physician’s self-directed learning behavior. This premise raises a pivotal
question when hypothesizing over the theoretical association between PBL and SDL
- do students from a PBL curriculum superficially fulfill a specific set of learning
requirements that decay after they leave that environment, or do they purposefully
adopt a self-directed learning attitude that is sustained long into the future? A review
of the research addressing this question will be presented in chapters 3 and 4.
Is it important to reach agreement on which strategies promote SDL or to
differentiate characteristics that may be components of SDL process, product,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
38
perhaps both, or does this quest represent pedantic hair-splitting? Authors within the
SDL field may not agree on which strategies are considered optimal in producing
SDL behavior, much as they continue to disagree over the very definition of SDL,
but one may argue that any direction chosen to enhance SDL development should be
purposefully associated with a clear understanding and articulation of what
behavior(s) and/or attitudes an educator is trying to effect. Vital to the promotion of
SDL behavior in medical education and postgraduate education should be concern
both for the quality and the consequences of students’ and physicians’ learning. This
issue will be further explored throughout this paper.
The Association of SDL with Self-regulated Learning
Before ending the examination of SDL development within a PBL
curriculum, it is important to address a closely related construct of SDL originating
from educational psychology known as self-regulated learning. Self-regulated
learning (SRL) is concerned with identifying and understanding the assimilation of
metacognitive aspects of learning, such as planning, goal setting, strategy use, and
self-evaluation, with such affective variables as motivation and self-efficacy, or the
belief in one’s capability to fulfill a learning task. Many theories of SRL comprise
the three major categories of metacognition, strategy utilization, and motivational
control. Zimmerman (1986) describes effective self-regulated learners as those who
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
39
are metacognitively, motivationally, and behavi orally active in their personal
learning process. The author specifically notes:
Metacognitively, self-regulated learners are persons who plan,
organize, self-instmct, self-monitor, and self-evaluate at various
stages during the learning process. Motivationally, self-regulated
learners perceive themselves as competent, self-efficacious, and
autonomous. Behaviorally, self-regulated learners select, structure
and create environments that optimize learning, (p. 308)
Comparable to the evolution of SDL, Butler (2002) states that SRL
definitions have become increasingly encompassing over the years, and within the
last two decades the understanding of SRL has evolved to incorporate the
interactions of students’ knowledge, metacognitive ability, motivation and cognition.
Butler (2002) suggests that more recent emerging definitions now consider the social
context-specificity of SRL and the shaping influence of the interplay between
individual, teacher, and peers. In a similar manner, Candy (1991) also argued that
SDL is largely domain- and context-specific and socially interdependent as, “the
self-directed learner in all but the most intimate of personal learning has to rely on
and collaborate with those who are practitioners in the field or area of his or her
learning” (p. 307).
There are two significant questions to be addressed when discussing the
constructs of SRL and SDL: 1) How are these two ideologies alike and different?;
and 2) can research from educational psychology on the relationship between SRL
skill development and student achievement outcomes validate medical education’s
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
40
attempts to develop comparable SDL skills in medical students? In considering the
answer to the first question, it is noteworthy to mention that upon reviewing the
literature in these two fields, one finds very little overlapping discussion, initially
giving any reader the inaccurate impression that these are two distinct and separate
fields. Three references were located in the medical education literature that
discussed aspects of SRL in this context. Two of these references are offered by
Mann (1994; 1999) in which the author campaigns for the development of self-
efficacy In medical students both to build performance confidence (1994), and to
enhance their intrinsic motivation for learning and practicing medicine (1999). In
the former article, Mann (1994) briefly discusses Bandura’s social learning theory,
noting that the Importance of the self-regulatory capability of setting, monitoring and
achieving personal goals incorporated within this theory was borne out in study
findings (Fox, Mazmanian, & Putnam, 1989a) indicating that physicians’ motivation
to change was powerfully related to personal goals. The author’s call for self-
efficacy instruction In medical students is consistent with her presentation of
significant study findings (Attarian, Fleming, Barron, & Strecher, 1987; Mann &
Putnam, 1989) in which physicians’ perceptions of self-efficacy regarding preventive
medical practices (e.g., smoking cessation, exercise) were found to correlate with the
proportion of time and persistence devoted to these specific practices. Mann (1994)
also notes that self-efficacy can be developed in SDL skills and that the two abilities
of self-regulation and self-reflection make SDL a natural process.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
41
In the third reference, Zimmerman & Lebeau (2000) offer a comparative
commentary between SDL within a PBL curriculum and SRL, The authors suggest
that definitions of SDL are highly similar to the definitions of SRL found in the
educational psychology literature. They note that three major classes of SDL
processes derived from the theoretical accounts put forth within the work by Evensen
& Hmelo (2000) parallel those used in research in SRL, including the identification
of learning objectives, strategy use and resource selection in the pursuit of learning
issues, and self-evaluation of learning. Zimmerman & Lebeau (2000) postulate that
these three SDL processes correspond directly to the components within SRL that
include forethought (e.g., goal setting, task analysis strategies, self-motivational
beliefs such as self-efficacy), performance or volitional control (e.g., strategy use,
self-observational processes), and self-reflection (e.g., self-judgments, self
reactions). The authors state that within this SRL model, more successful SRL
efforts have been revealed to depend particularly on the quality of students’
forethought processes, which tend to encourage proactive self-regulatory control
rather than reactive control (Zimmerman & Lebeau, 2000).
In their commentary, Zimmerman & Lebeau (2000) conclude that greater
cross-fertilization between the SDL and the SRL literature bases would be mutually
beneficial in future research. Given the authors’ discussion of the procedural
similarities between the two fields, this follows as a logical suggestion. However,
what Zimmerman & Lebeau (2000) do not make entirely apparent is how SDL and
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
42
SRL differ. In fact, no literature was located that discussed this point. What must be
assumed, however, is that the two constructs are indeed somehow different as
authors do not completely equate the two, and moreover, some authors imply a
separation of the two concepts that is not made altogether clear. Consider the
following passage from an article discussing qualitative approaches to SRL:
Indeed, most definitions of SRL have at their core the goal
o f students’ self-directing learning independently. For example,
Paris and Paris (2001) link self-regulation to “autonomy and
control by the individual who monitors, directs, and regulates
action toward goals of information acquisition, expanding expertise,
and self-improvement” (Paris & Paris, p. 89). Even Meyer and
Turner (2002) who explicitly describe SRL as a social process,
identify students’ development of autonomy and responsibility as
important instructional goals. If SRL is a social process, where
do constructs such as independence, control, autonomy, and
responsibility fit in? (Butler, 2002, p. 60)
Two implications are striking about this reference. By inferring that SDL
occurs as an extension or goal of SRL if students are capable of learning
“independently,” the author first implies that the two concepts are somehow
distinctive. Additionally, by referring to the social context parameters o f SRL,
Butler (2002) seems to be implying that ‘autonomy’ and ‘independence’ are not
strictly part of the SRL process, but rather better distinguish the concept of SDL.
This allows the reader to presume that Butler’s (2002) use of the phrase “self
directing learning independently” may refer to learning in ‘isolation’, rather than
learning ‘initiated on one’s own’. Butler (2002) seems to be implying that SRL is
distinguished from SDL by the parameters of its social context. However, this
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
43
feature cannot accurately represent the element distinguishing SDL from SRL, as
SDL has also been noted (Candy, 1991; Garrison, 1997; Brockett & Hiemstra, 1991)
to also be more of a social learning process than an isolated one.
Further illustration of implied distinction between SRL and SDL is offered by
Bandura (1996) who stated, “Knowledge gained from a broadened conception of
self-regulation is now being fruitfully applied to the cultivation of academic self-
directedness” (p. 1220). Additionally, Mann (1994) previously suggested a
distinction between the two concepts when she noted that “the two abilities of self
regulation and self-reflection make SDL a natural or inherent process” (p. 42).
It is difficult to clearly interpret how authors are viewing the distinction
between SDL and SRL. From the literature reviewed in these two fields, four
potentially significant differences are being proposed here. The first difference
concerns learning task initiation. In SDL, students independently identify their
learning issues, and although in the specific environment of PBL students
collectively are defining learning issues, these issues are still generated from students
themselves in relationship to their own knowledge bases. The literature on SRL,
however, appears to be more concerned with students’ learning goals and strategies
based on classroom assignments as given to them by their instructors. While
students may be free to select their personal learning strategies, the overall learning
task or goal is initially generated by the teacher, and not by the student. Perhaps it is
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
44
this distinction that SRL authors are implying by separating SRL from the notions of
‘autonomy5 and ‘self-directedness’.
The second suggested difference concerns learning context, and perhaps
research population. Other than a few references related to SRL within other fields
such as athletics or music, the greater part of the SRL literature reviewed for this
paper concerned the development of SRL strictly as it occurred within a classroom,
or formal educational setting, with the majority of studies utilizing grade school
through high school students, and a minority of studies employing college students.
In SDL research, however, there is emphasis on understanding the dimensions of
SDL in both a fixed, formal educational setting as well as over time in everyday,
non-academic life, hence the oft-mentioned and somewhat confusing, alternate SDL
definition as ‘lifelong’ or ‘independent’ learning. Research in SDL initially began
outside of a formal educational context and subsequently expanded to encompass
SDL within this educational arena. Within the formal educational setting, most SDL
research employs college and postgraduate students, while no SDL general education
literature reviewed for this paper referred to research using grade school through
high school students.
The next perceived difference between SDL and SRL concerns critical
thinking/application of new knowledge, and as these two fields seem to deal with
different populations, this may be partially age-related. In SDL, particularly as it
applies to medical education, but also as discussed in a general adult learning
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
45
context, there is an emphasis on how the newly learned information will either be
used to conceptually solve a learning problem or to change one’s course of action.
In PBL, for example, students are asked to apply their new knowledge to a patient
problem and describe how this application changes their thinking or potential
management of a patient case; they are also asked to critique the quality of their
learning resources. Although SDL is also concerned with learning strategies, there is
a greater emphasis on how the information is to be used (i.e., critical thinking) rather
than on the learning strategy the student used to gain the information. In SRL,
however, conscious consideration of the interplay between the quality of
informational resources with critical analysis of how the information is to be used
would appear, in the case of younger children, to be above their cognitive capability,
and in the case of older students, is not discussed in the literature as being a critical
component of SRL. Rather, SRL appears to be more concerned with students’
metacognitive skill development and instruction in the planning, organizing,
monitoring and evaluation of their own learning, and not with the critical application
of learning.
The fourth, and perhaps most significant difference between SDL and SRL,
involves the issue of motivation and represents an area of study where SDL is quite
underdeveloped. Motivation is intimately connected with self-efficacy, as Bandura
(1989) notes:
Perceived self-efficacy also plays an influential role in the exercise
of personal control over motivation. It Is partly on the basis of self-
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
46
beliefs of efficacy that people choose what challenges to undertake,
how much effort to expend in the endeavor, and how long to persevere
in the face of difficulties (p. 730).
Evidence of the importance of physicians’ self-efficacy in practicing
medicine was previously offered by Mann (1994), who strongly advocated that self-
efficacy be incorporated into medical education in general and within SDL
development specifically. The product or goal dimension of SDL centers on one’s
motivation to initiate learning change, as reflected in the SDL goal definition given
in chapter 1. However, both the general and the research literature in the field of
SDL have not sufficiently addressed this critical component of SDL behavior,
whereas the SRL literature has contributed a great deal toward the understanding of
how motivation and self-efficacy impact learning. The subject of motivation in SDL
will be featured more fully later in this paper after discussing SDL in the
postgraduate setting.
In summary, similarities between SDL and SRL respectively are noted by
Zimmerman & Lebeau (2000) to include the areas of learning objective
identification/forethought, strategy use and resource selection/performance or
volitional control, and self-evaluation of learning/self-reflection. Four proposed
differences between SDL and SRL can be found within the areas of learning task
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
47
initiation, learning environments and research populations employed, emphasis on
critical thinking or application of new knowledge, and study of the role of motivation
in learning.
SRL Research
The second question to be considered in the comparison of SDL with SRL
concerns research findings on the association between SRL instruction and student
achievement. As SDL has many similar procedural components in common with
SRL, if the SRL literature can substantiate better learning outcomes in students with
greater SRL skill, then these findings would justify medical education’s efforts to
develop similar SDL skills in its students.
As it stands, the SRL literature is replete with examples describing the
association of SRL development with improved learning outcomes. Such improved
outcomes are described in the literature in areas such as mathematics, reading, and
writing as well as others. Research has now extended to include very young
children, once considered too cognitively immature to engage in SRL. For example,
Perry, VandeKamp, Mercer & Nordby (2002) developed a framework with
kindergarten through grade 3 teachers that structured tasks and interactions with their
respective students to promote SRL in reading and writing activities. The authors
describe that over time the proportion of students who reported negative affect as a
response to errors decreased (64% to 37%) while the proportion of students who
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
48
preferred easy tasks to more difficult tasks decreased (50% to 26%). Perry et al.
(2002) concluded that:
.. .young children can and do engage in SRL in classrooms where
they have opportunities to engage in complex open-ended activities,
make choices that have an impact on their learning, control challenge,
and evaluate themselves and others. In addition, our observations
revealed the ways in which teachers provide instrumental support to
students (e.g., through questioning, clarifying, correcting, elaborating,
modeling) and create opportunities for students to support one
another (e.g., through collaborating, sharing ideas, and brainstorming
problem-solving strategies), (p. 14)
What is particularly striking about this conclusion is that it describes equally
well the process by which medical students engage in PBL/SDL as it does the
process by which 5-8 year-old children develop SRL.
Paris and Paris (2001) describe two studies (Graham, 1997; Page-Voth &
Graham, 1999) in which executive control or goal-setting strategies were examined
in sixth- to eighth-grade students who struggled with writing. In the first study
(Graham, 1997), researchers found that providing students with support in managing
and coordinating their plans and decisions demonstrated positive effect on their
revising behavior and in the quality of their final text. In the second study (Page-
Voth & Graham, 1999), students learned strategies that facilitated goal attainment by
helping them develop processes that involved the generation, evaluation, and
incorporation of target elements into essays. Students who learned these goal-setting
strategies wrote longer papers, included more supporting reasons, and produced
qualitatively better essays than students in the control condition.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
49
Zimmerman (1990, 1998) discusses a series of studies (Zimmerman &
Martinez-Pons, 1986; Zimmerman & Martinez-Pons, 1988; Zimmerman &
Martinez-Pons, 1990) examining the role of self-regulatory processes in academic
performance. In an initial study by Zimmerman & Martinez-Pons (1986), the
authors developed a structured interview in which high school students from
advanced and lower academic tracks were presented with a series of common
learning problems or contexts and asked to specifically respond to each situation.
Responses were coded and scored (86% intercoder agreement) for the presence of
one or more of fourteen SRL strategies (e.g., self-evaluation, goal setting and
planning, self-monitoring, etc.). The quality and quantity of SRL strategies reported
were found to be highly predictive of academic achievement. High achievers
reported significantly greater use of 13 of 14 processes and used them at a frequency
of more than twice that of low achievers. Students’ achievement track in school was
predicted with 93% accuracy using these self-reports, while self-reports were highly
correlated with standardized test performance (r = .61).
Zimmerman (1990) describes a second study (Zimmerman & Martinez-Pons,
1988) that attempted to further establish the validity of student reports of SRL use.
The authors asked high school teachers to rate their students for use of strategies,
where items focused on learning strategies that were either observable directly in
school (e.g., asking for further information or being self-evaluative about test results)
or deducible from their observable effects (e.g., completing assignments on time,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
50
being prepared for class). Students’ mathematics and verbal standardized
achievement test scores were submitted to multivariate analyses with teacher ratings
in order to separate students’ achievement outcomes associated with their use of SRL
strategies from their general ability. Factor analyses revealed a single, ‘Self-
Regulated Learning factor’ that accounted for nearly 80% of the variance in scores.
Students’ reports of using SRL strategies as assessed by the structured interview
procedure described above correlated .70 with the derived SRL factor, suggesting
that students’ use of SRL strategies made a distinctive contribution to academic
achievement apart from their general ability. Students’ self-reports were also found
to correlate highly (R = .70) with teachers’ ratings of their observed self-regulation
in class and completion of homework assignments.
Zimmerman & Martinez-Pons (1990) investigated students’ underlying
motivation in using these self-regulatory methods to guide their study, particularly as
related to their perceptions of efficacy. Using 5th-, 8th-, and 1 l th -grade students from
regular or gifted schools, the authors found that developmental increases in SRL
strategy use were reported by both regular and gifted students, but gifted students
surpassed the regular students at each grade level. Zimmerman & Martinez-Pons
(1990) developed a corresponding self-efficacy scale using verbal and mathematical
problems ranging in difficulty (elementary through high school level) and asked
students to rate their confidence about answering each item correctly. The authors
found that corresponding to increases in SRL strategy use were increases in verbal
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
51
and mathematical self-efficacy. The correlation between strategy use and self-
efficacy reports was r = .42 for verbal functioning and r = .41 for mathematical
functioning, a finding the authors state indicates a relationship between self-
regulatory strategic competence and this key form of motivation.
Zimmerman (1998) describes a subsequent study (Zimmerman, Bandura, &
Martinez-Pons, 1992) that investigated the causal role of self-efficacy to self-regulate
studying in high school students. It was hypothesized that self-efficacy to regulate
learning would be linked causally to self-efficacy for academic achievement, and
that self-efficacy for academic achievement would predict the grade goals that
students set for themselves. Using path analysis, the authors’ results supported their
hypothesis - self-efficacy for SRL was linked to self-efficacy for academic
achievement, which in turn was predictive of the students’ grade goals as well as
their final grades (all path coefficients significant at p < .05 level). Zimmerman
(1998) also notes that similar findings were obtained in a subsequent study
(Zimmerman & Bandura, 1994) with college students enrolled in a writing course,
where self-efficacy for writing was predictive of self-efficacy for academic
achievement, which was in turn predictive of the students’ writing goals as well as
their final grade in the course (all path coefficients significant at p < .05 level).
The preceding description of SRL research suggests that fostering SRL
development can indeed lead to enhancement of learning outcomes. As self
regulation theories have developed in educational psychology, underlying processes
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
52
of SRL have been distinguished qualitatively and quantitatively and found to be
highly predictive of academic motivation and achievement (Zimmerman, 1998), As
such, an argument can be made that a similar investment in developing SDL in
medical students is justified.
Chapter Summary
The preceding discussion presented a theoretical overview of how the
instructional method of PBL is thought to support the development of SDL in
medical education. Strategies from the literature in adult education believed to
encourage SDL (Candy, 1991; Brocket! & Hiemstra, 1991) were discussed and
compared with those educational processes utilized within a PBL curriculum.
Literature from medical education was presented to help analyze how different
instructional elements within a patient case presentation are believed to contribute to
SDL development, and a recent work on PBL and SDL (Hmelo & Lin, 2000) was
offered as support. The process and product dimensions of SDL were further
discussed in the context of SDL strategy development, and this was noted to be
another area of disagreement in the overall understanding of SDL. An argument was
proposed suggesting that SDL strategies may exist on a process/product continuum.
Lastly, SDL was compared and contrasted with SRL, a closely related construct from
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
53
the field of educational psychology. SRL research found to be successful in
improving learning outcomes was presented in support of similar attempts to develop
SDL in medical education.
Findings presented in chapters 1 and 2 will be summarized to open chapter 3.
Subsequently, the definition of SDL specific to medical education will be reviewed.
Methods used to evaluate SDL within PBL will be discussed to further consider the
specific SDL skills targeted in assessment. Attention will then focus on presenting
an overview of research regarding PBL and SDL in undergraduate medical education
and an ensuing analysis of research strengths and limitations will be presented.
Recommendations will be proposed for improving the development of SDL in
medical education as well as the quality of future research in the field.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
54
C H A PTER 3
THE DEFINITION, ASSESSMENT AND RESEARCH OF SDL IN PBL
UNDERGRADUATE M ED IC A L E D U C A T IO N
Summary of Chapters 1 and 2
Chapter 1 presented various conceptual and quantitative definitions of SDL
disseminated throughout the aduit education literature. A review of SDL described
the evolution of the construct over the last half century, and highlighted the growing
confusion and debate that has arisen during this time over both its precise theoretical
meaning and the most accurate means of translating its theoretical basis into
operationalized measures. The major rubric for conceptualizing SDL featured the
view of SDL as a separation of two distinct dimensions, one concerned with the
instructional methods or processes of SDL, and one focused on SDL as a learner
outcome goal or attitude. Quantitative measurement of SDL was reviewed and three
scales extensively used in the research of SDL over the past thirty years were
discussed and their potential design weaknesses were addressed. Various SDL
conceptual and quantitative definitions were also presented in table form. Novel
definitions of SDL process and product/goal were proposed to help guide further
discussion of SDL in medical education.
Chapter 2 opened with a discussion of the rational for incorporating SDL into
medical education, followed by a description of PBL, the curricular method most
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
55
widely utilized to promote SDL behavior in medical students. Strategies designated
by authors in adult education to enhance SDL were compared with those
instructional processes employed in PBL, and support was offered that endorsed
PBL as a curricular method theoretically capable of developing SDL behavior.
Inconsistency in the process/product delineation of SDL-enhancing strategies was
noted and an argument was presented for considering a flexible interpretation of the
designated strategies as belonging to a process/product continuum, rather than
forcefully partitioning them into one or the other of these definitional dimensions.
SDL was compared and contrasted with SRL, an analogous construct from
the field of educational psychology. Similarities between the two concepts were
noted to be in the domains of learning objective identification, strategy and learning
resource use and self-evaluation. Four proposed differences were noted to be within
the areas of learning task initiation, learning environment and preferred research
population, critical thinking and knowledge application, and most importantly, in the
area of learning motivation, where SDL research is underdeveloped in comparison to
SRL. Research findings reporting a positive association between self-regulation and
academic achievement were presented in support of an analogous development of
SDL in medical education.
Chapter 3 reviews the conceptual definition of SDL in the medical education
literature and discusses the poor development and discrepancy in the area. Two
models of SDL assessment are also described, and the use of the ‘triple jump
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
56
examination’ as an acceptable SDL evaluation tool is discussed. Considerable
attention then focuses on SDL in PBL research and several empirical and review
articles are reported. Analysis of this research reveals limitations in the areas of SDL
definition, SDL operationalization, sampling, study design, instrumentation and
interpretation of results. Research strengths are noted to include use of behavioral
measures to supplement self-reports, data triangulation, and inclusion of
comparability data for non-randomized PBL and conventional curriculum subject
groups. Drawing conclusions on differential effectiveness of SDL development
between curricula is discussed and six recommendations for advancing SDL
development in medical education and research are proposed.
SDL in PBL Medical Education
While the appraisal of a PBL curriculum reveals a methodology capable of
supporting SDL, the PBL literature does not extensively elaborate on its
development and application. Significant description can be found on the clinical
reasoning process of PBL and its association with the development of medical
expertise (Hmelo, 1998; Norman & Schmidt, 1992; Patel, Groen, & Norman, 1991;
Schmidt et ah, 1996), as well as the information-processing theory thought to
underlie the PBL process (Norman & Schmidt, 1992; Schmidt, 1983). However, the
literature contains considerably less reference to a similar analysis on SDL
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
57
development within PBL, including an examination of those particular SDL
behaviors, attitudes or skills PBL is attempting to enhance.
Definitional Inconsistency of SDL
While Hmelo & Lin (2000) attempt to provide some theoretical examination
of SDL within PBL, this is more the exception than the rale upon review of the
medical education literature. As the third of the four PBL educational objectives
originally proposed by Barrows (1986) (see chapter 2, p. 20), SDL is not as well
documented as the first two. A review of the medical education literature reveals an
SDL definitional inconsistency parallel to that besetting the field of adult education.
When considering discussions of PBL and SDL in the undergraduate literature,
arguably the use of the term SDL has either not been adequately established or
perhaps has been diluted to the point of questionable significance. This point is
illustrated in Table 2, which lists several definitions of SDL found in the medical
education literature. O f the descriptions generally classified according to a
process/product division, most of these are focused principally on the SDL
procedural skill aspect. Additionally, articles reviewing general PBL curricular
effects were located that directly or implicitly referred to SDL without further
definition, or did not consider SDL as a potential PBL educational objective (Bligh,
1995; Finucane et ah, 1998; Martenson, Myklebust, & Stalsberg, 1992; Schmidt,
1989; Schmidt, Dauphinee, & Patel, 1987; van Luijk & van der Vleuten, 2001).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
58
TABLE 2 - GENERAL SDL DESCRIPTIONS REPORTED IN THE
UNDERGRADUATE MEDICAL EDUCATION LITERATURE
Study SDL Description
Barrows (1983), (1986) Students’ use of learning/information resources
Dolmans & Schmidt (1995)
Dwinnell & Adams (1998)
David et al. (1998)
Lloyd-Jones & Bligh (1998)
Mann & Kaufman (1995) Identification of learning issues or objectives
David, Dolmans, Patel &
van der Vleuten. (1998)
Dwinnell & Adams (1998)
Lloyd-Jones & Bligh (1998)
Rankin (1992)
Norman & Schmidt (1992)
David et al. (1998)
Library usage and/or information-seeking skills
Norman & Schmidt (1992)
Colliver (2000)
SDL as associated with keeping up with
the literature/to date
Mann & Kaufman (1995) Measurement of SDL by Guglielmino’s (1978)
Shokar & et al. (2002) Self-directed Learning Readiness Scale
Peng (1989) Replacement of didactic lecture hours with
specific discipline self-learning guidelines,
including chapter requirements, reference lists,
review questions and self-evaluation exercises
Walton & Matthews (1989) SDL as associated with independent behavior
necessary for becoming responsible for one’s
own education and the techniques necessary
for its Implementation as an end in itself;
includes self-assessment and self-criticism
ability, and acquisition o f the skills of
reflection and deliberation
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
59
TABLE 2 - GENERAL SDL DESCRIPTIONS REPORTED IN THE
UNDERGRADUATE MEDICAL EDUCATION LITERATURE
(CONTINUED)
Study SDL Description
Blumberg & Michael (1992) SDL skills include the ability to recognize
the need for new learning, set one’s own
learning objectives, define relevant questions
for study, access relevant information, and test
the depth of understanding of what one has
learned
Dolmans & Schmidt (1995)
Williams, Saarinen-
Rahikka & Norman (1995)
SDL skills as associated with learning to reflect
upon and control learning and develop self-
regulatory skills
Amount of time spent in non-scheduled
educational activities
Thomas (1997)
Shokar, Shokar, Romero
& Bulik (2002)
Assessment of SDL skills via documentation
of the range, depth and appropriateness of
the databases accessed by students, the
assessment of contribution of tutors to self-
learning skills, and evaluation of how
students’ self-learning skills translate into
specific decisions about patient care
SDL skills as associated with lifelong learning
Schmidt (2000) Preparedness of a student to engage in learning
activities defined by himself rather than by a
teacher. ‘Preparedness’ includes both
motivation and skilled behavior. An
accomplished self-directed learner experiences
an intrinsic need to acquire knowledge, not
dominated by teacher-set requirements. He has
mastered the appropriate information seeking
skills; he knows where and how to find infor
mation resources that would fulfill his need.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
60
Two fairly comprehensive definitions of SDL also deserve mention. In the
first, Blumberg (2000) borrows from Candy (1991) in formulating an analysis of
SDL considered from a three dimensional model: the process of learning itself,
learning strategies, and performance outcomes of SDL. These three components are
also conceptualized as either being discrete components or holistic skills. Learning
processes were said to include “abilities to define what to leam; plan and
operationalize learning, specifically through time management; seek, use, and
evaluate the effectiveness of resources; and evaluate SDL skills” (p. 199), Learning
strategies “refer to the methods students use to study material or process
information” and performance outcomes “are classified as either short term
(immediate) or long term (years after student completed the.. .program)” (p. 200).
Elsewhere in her discussion, Blumberg (2000) reports library usage as data
indicative of short-term component outcomes, and again includes this behavior in her
discussion of long-term component outcomes. Other long-term outcomes discussed
included research on behavior related to reading unassigned literature for the purpose
of staying aware of medical advances, familiarity with new management regimens,
and participation in continuing medical education activities.
The second developed SDL definition was offered by Miflin, Campbell &
Price (2000), who described the SDL conceptual framework developed for a nascent
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
61
PBL curriculum instituted at the University of Queensland Graduate School of
Medicine. The authors provide a thoughtful analysis of the goal of SDL in their
program:
Self-directed, lifelong learning is defined in the Graduate Medical
Course as the development of graduates who:
• are conscious of the need and accept responsibility for evaluation
of practice in the light of changing understanding;
• are able to identify deficiencies in their own knowledge, skills,
and attitudes;
• are motivated to generate a learning programme to address
deficiencies, including finding and using the best evidence;
• have the skills to identify, access and use resources wisely
and efficiently;
• are able to evaluate learning efforts, including resources used,
and the effects on practice, and
® are committed to repeating the cycle with each patient and
clinical situation (p. 300)
A review of the medical education literature thus reveals both
underdeveloped definitions, as well as a wide spectrum of SDL conceptual
definitions. This variation and often limited description of SDL, coupled with in
some cases the absence of a definition at all, may imply that the need to explicitly
state a definition of SDL in the literature does not exist because there is a common,
passive understanding o f the concept in the medical education community. If this is
the case, it clearly represents a hazardous assumption, perhaps implying that medical
educators are free to fill in any definitional gap with their own opinion of SDL, since
the apparent understanding of the concept is so diffuse that, in creating one’s own
definition, a person is bound to “hit the side of the bam.” Equally concerning is that
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
62
in skimming over the description of SDL, only a handful of educators seem
interested in understanding what specific skills and behaviors are even important
SDL developmental components. When a governing body on medical education
deems SDL important enough to recommend its development in a seminal report on
medical education improvement, and authors repeatedly speak of its necessity in
shaping future physicians, why is there not more fervent discussion in the literature
on gainer a broader comprehension of what SDL is and how to develop it in medical
students?
One medical school newly implementing PBL found conflicting definitions
of SDL to be so counterproductive that instead of developing self-direction, students
became increasingly frustrated and overly dependent on faculty direction (Miflin et
al., 2000). When it comes to deconstructing the definitional components of SDL, it
appears that medical educators are undergoing the same growing pains that originally
plagued scholars in the adult education community. The ambiguity and variance
found in SDL definitions pose a difficult obstacle in the examination and
advancement of SDL in medical education. To determine if the understanding of
this concept can be further elucidated, attention will now turn to the evaluation of
SDL in PBL and whether any specifically targeted skills or behaviors are reported in
its assessment.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
63
Assessment of SDL in PBL
Much of the general PBL literature describes various aspects of the
educational theory and methodology behind the process, or focuses on the
comparison/contrast of PBL versus conventional curricula in the areas of basic
science knowledge acquisition or clinical performance. As such, literature on the
evaluation of PBL is also skewed toward a discussion on general, global assessment
of the process or is predominantly concerned with different techniques for assessing
clinical performance. Two references that were more elaborative on the specific
assessment of SDL within PBL are presented here.
Barrows (1980) devotes a chapter to the overall evaluation of PBL in which
he details several alternative evaluation techniques. The author proposes that “since
the ultimate goal of self-directed learning is to make it a part of the student’s
professional life, the evaluation tools should be designed so that the student may use
them himse lf’ (p. 112). As one aspect of SDL, Barrows (1980) advocates for regular
self-assessment by students through various methods, including the use of
computerized patient management problems and simulated patient encounters, to
evaluate a range of competencies (e.g., diagnostic decision making). In his
discussion of individual competencies or behaviors requiring evaluation in PBL,
Barrows (1980) identifies a category of ‘self-study skills’, which include self-
evaluation, study-question design, use of resources, and new information. These
skills respectively refer to self-assessment of strengths and weaknesses in clinical
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
64
reasoning, knowledge, etc.; formulation of a relevant study plan with focused
questions; effective use of reference books, computer searches, etc.; and assessment
of the scope, depth, and accuracy of information acquired in self-directed study.
Barrows (1980) lists several questions (pp. 146-148) that an instructor may ask in
evaluating each of these specific areas and suggests that a portion of these questions
may be used to evaluate students’ self-directed study, the most crucial step of which
is considered, “the student’s ability to generalize and synthesize the information and
skills learned in his work with a problem” (p. 147). The author suggests that the best
way to evaluate students’ self-directed study is through informal discussions, either
with an individual student, or with a group if the students are involved in small-
group learning.
Barrows (1980) also offers a more formal method for evaluating self-directed
study, in which he suggests that students proceed through the following steps: 1)
work through a self-directed study patient problem as completely as possible; 2)
write up the case using all data essential to understanding the problem, and include a
problem formulation, hypotheses for the problem written in terms of either diagnosis
or basic mechanisms, and treatment or management plan; 3) take a pretest for
assessment of information base (optional); 4) carry out freely chosen self-directed
study over a prescribed time period; 5) revise the case write-up upon completion of
this study, indicating how new information is applied to the patient problem; 6) take
a posttest (optional); 7) write a self-evaluation answering certain questions, as well
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
65
as listing any resources used and describing any educational gains made; and 8) write
down recommendations for future study based on this experience. An alternative to
this sequence is to give a student a second patient problem to work through after
completing the course of self-directed study.
The most elaborate discussion on this topic was offered by Kelson (2000),
who describes in detail the assessment of SDL in the PBL curriculum at the Southern
Illinois University of Medicine (SIU). The assessment model, called Performance
Assessment of Self-Directed Study, or PASS, is a performance-based examination
given to students at the end of every PBL unit (8-10 weeks). Students encounter
three cases in which they interview and perform a physical examination on a
standardized patient. They complete a formal case write-up on one case and write
“Impressions” and “Plans” for the other two. Students also compose a list of issues
significant to the case, and from these they identify learning issues. Upon
completion of the write-up and learning issues, students are given a written case
description that incorporates all pertinent information they should have gathered in
the patient history and physical examination, thereby correcting for discrepant levels
of clinical skills between students. Students are then allowed to revise their list of
learning issues, and are given 2-3 days for SDL of their identified learning issues.
Matrices are constructed for each student on which individual learning issues are
recorded and then matched with significant case issues and corresponding question
probes that were initially formulated by the exam design team prior to case testing.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
66
This examination design team initially dedicates considerable attention to
formulating appropriate and wide-ranging probes and provides extensive training to
case examiners to increase inter-rater reliability. At the end of the designated self-
study period, students are evaluated by an individual examiner who tracks the
student’s case discussion on the matrix, and is granted free license to probe for
broader or deeper explanations from corresponding openings in the student’s
discussion (using the agreed upon probes). Students’ correlation of the case
discussion with their individually generated learning issues provides insight into how
they approached those concepts considered critical to understanding and managing
the case. Students are evaluated on the accuracy and depth of knowledge
demonstrated with respect to key issues in the case, and the extent to which that
knowledge is spontaneously integrated into the case explanation. The PASS
examination, which strives to preserve student-centeredness and to present authentic
modeling of clinical reasoning, is part of an SIU PBL curriculum committed to
developing in students a mental model known as the ‘proactive lifelong learner’.
The previous SDL assessment descriptions are modified examples of a triple
jump examination (TIE), an evaluation technique that basically consists of three
parts - a problem definition with student-generated learning issues; a designated
period of self-study; and problem revision and analysis with an application of one’s
newly learned information. SDL assessment using a TIE or modified form thereof is
also described for PBL programs at the University of Hawaii (Smith, 1993),
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
67
University of New Mexico (Kaufman et al., 1989) and McMaster University
(Neufeld, Woodward, & MacLeod, 1989), whose occupational and physiotherapy
PBL program also adopted this SDL assessment technique (Chapman, Westmorland,
Norman, Durrell, & Hall, 1993). Many of these articles also address the difficulty
associated with achieving high measures of inter-rater reliability and predictive
validity, issues inherent to all performance-based examinations. Potentially partially
due to these methodological concerns, many of these exams were noted to be graded
as pass-fail.
Discussion of SDL Assessment
Two illustrative points appear from this discussion of SDL assessment using
a TIE. First, this method of SDL assessment closely reflects many of the skills
defined in the procedural component of SDL. Within a TJE, students are asked to
assess their learning needs by composing a list of learning issues, plan a study
strategy in which they demonstrate effective use of learning resources, integrate
newly learned information into a case, and evaluate their learning outcomes. All of
these skills were proposed in the SDL process definition given in chapter 1. The SIU
PBL program also recognized SDL as a potential goal, by specifically suggesting
that SDL and its assessment were repeatedly utilized to encourage and develop
students as lifelong, proactive learners. As presented in chapter 2, the use of TJE
assessment also features many of the interventions posited to encourage SDL -
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
68
making use of students’ existing knowledge, encouraging deep-level learning,
developing critical thinking, and enhancing reading skills and comprehension
monitoring. Based on this analysis, the TJE would theoretically appear to be one
evaluation technique capable of assessing SDL.
The second point that emerges from this discussion is this - if there is an
established theoretically supported basis for effectively assessing SDL, why do
definitions of SDL in PBL continue to be so discrepant? There may be two reasons
for this. The first reason may encompass the more general problem of defining PBL
itself. Although PBL is well described in the literature, several authors note that the
mere agreement of what constitutes PBL is an ongoing problem (Albanese &
Mitchell, 1993; Barrows, 1986; Foley et al., 1997; Lloyd-Jones et ah, 1998), as is the
lack of a “gold standard” to serve as an outcome measure (Albanese & Mitchell,
1993). One would predict that lack of agreement would be a bigger problem in
medical schools that sparsely implement PBL, or perhaps those that use PBL in
clinical instruction, as PBL is more widely described as a basic science curricular
intervention. If the overarching definition of PBL educational strategy itself is not
definitive, then it follows that the educational objectives of PBL, particularly lesser-
researched objectives such as SDL, would by extension suffer the same fate. It is
noteworthy that the schools reporting the use of the TJE for SDL assessment were all
schools with long established, predominantly PBL curricula.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
69
The second reason discrepant SDL definitions may continue to exist is triple-
fold: 1) SDL is still a relatively new concept in medical education: although
proposed as one of the original educational objectives of PBL, the examination of
SDL has remained quiescent until the last 15-20 years, and has only been widely
recognized and specifically promoted as a discrete, educational goal within the past
decade; one could compare this with the timeline for PBL itself, an educational
concept still searching for definitional accord after existing for approximately 30-35
years; 2) due to its emerging status, minimal research on SDL appears in the
literature to assist in the promotion or guidance of definitional clarity; and 3) there
may not be extensive dissemination or networking on the subject at the national
level, thereby impeding medical schools with more established incorporation of SDL
within their curricula from becoming well-known throughout the medical education
community. In the adult education literature on SDL, there are scholars such as
Tough, Knowles, Candy, Brookfield, and Brockett who are recognized experts on the
subject and provide lengthy discourse on SDL; there do not appear, as yet, to be any
respective counterparts to these authors on SDL within the field of undergraduate
medical education. These academics in adult education provided essential direction
to the topic of SDL as it developed into a recognized area of educational research.
Perhaps the lack of recognized expertise on SDL in medical education perpetuates
inconsistent definition and understanding of the concept.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
70
SDL in PBL Research in Undergraduate Medical Education
The discussion now turns to a review of the research that has been conducted
on SDL in PBL within the undergraduate curriculum. Examination of SDL research
may help to further elucidate how the term has been defined, operationalized and
studied in the research arena. Reviewing SDL outcomes at the undergraduate level
may also demonstrate strengths and limitations of research design that can serve as a
bridge in the examination of SDL research at the postgraduate level, which is the
subject of chapter 4.
One literature search was conducted to capture all potential SDL research at
both the medical undergraduate and graduate levels; a reference librarian was
consulted to help maximize the search radius. MEDLINE 1966-October 2002 was
the primary database searched, and the following terms were entered and combined
in consideration of a variety of possible permutations: ‘self-directed learning’;
‘lifelong learning’; ‘problem-based learning’; ‘independent learning’; ‘education
continuing or medical’; ‘curriculum’; ‘medical students’; and ‘internship and
residency’. Citations were limited to the English language and included 1469
matches. Secondary databases searched included ERIC, Education Abstracts,
Education Index and PsycINFO and permutations of ‘self-directed learning’;
‘medicine’; ‘physician or doctor’; and ‘problem-based learning’ were used. Total
records found for these databases included 171 matches. All citations from the five
databases were evaluated and relevant abstracts were reviewed for information
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
71
appropriate to medical under/post-graduate education research concerning SDL in
PBL. Bibliographies of relevant articles or books were also reviewed for potential
references.
Three categories of articles concerning SDL in undergraduate PBL curricula
will be presented here: those that empirically compare PBL and non-PBL students
on measures of SDL; PBL review articles that reference other studies on SDL within
PBL curricula; and studies on SDL outside of a defined PBL curriculum. The last
category is included for possible further exemplification of the understanding of SDL
in medical education. Studies and review articles were evaluated with a focus on
SDL conceptual and operational definitions, research design, presentation of study
results and interpretation of results. Discussion of these findings and analysis of
potential strengths and limitations of SDL in PBL undergraduate research will be
addressed after presenting this literature.
Empirical Studies of SDL Within PBL Curricula
Six articles were located that measured SDL in a PBL curriculum, four of
which directly compared PBL students and conventional curriculum (CC) students
on measures of SDL, and two that looked at changes in SDL over time in PBL
students. The first of these studies, an oft-cited article by Blumberg & Michael
(1992), compared two recent graduating classes from a PBL or a CC curriculum
concerning their use of various study and library resources during their preclinical
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
72
years; data sources included student self-reports of learning resource use, library
circulation data, and both student-reported and faculty-reported perceptions of
specific SDL skill proficiency. PBL students (n = 31) reported significantly lesser
use of cooperative lecture notes, course syllabi (both p < .0001) and personal lecture
notes (p < .01) than the CC students (n = 112), and greater use of textbooks, journals
(both p < .001), and informal discussions with faculty or peers (p < .01). PBL
students also reported significantly more frequent library use to access information
than did CC students, as well as a significantly greater number of bibliographic
searches, use of reference books, recommended and reserve reading, and unassigned
research literature (all p-values < .0001). Circulation data for all materials from the
library confirmed the students’ self-report data. The PBL students were also found
to borrow more material from the library than their CC counterparts during the
clinical years when both groups of students were in the same curriculum.
In evaluating advantages and disadvantages of their medical school
curriculum, the CC graduates made no mention of SDL skills, whereas the PBL
students reported SDL skills to be a benefit of the curriculum, specifically that they
possessed an increased ability to: define what they needed to leam on their own
prior to initiating the study of the material, find relevant material on their own, and
see gaps in their knowledge after their study (internal unpublished report). Faculty
were also reported to note a consistent difference between students from the two
curricula in their SDL skills, with PBL students being more willing to admit and
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
73
pursue self-perceived lack of understanding, find relevant references and obtain
necessary information, and to frequently ask questions, where these perceived
differences persisted into the clinical years. The authors suggest that, “Students in
the mixed PBL curriculum, in which content is defined by both student-generated
learning issues and by faculty-generated objectives, seem to develop or m anifest
self-directed learning behaviors to a greater extent than do students in the teacher-
centered [CC]” (p. 7).
A second empirical study (Mann & Kaufman, 1995) reported on an
evaluation of second year PBL and CC students’ self-reported SDL skills and
attitudes, as well as their perceptions of the extent to which their courses in the first
and second years facilitated the acquisition of SDL behaviors. Measures included
responses on the Self-directed Learning Readiness Scale (SDLRS, see chapter 1) and
on an evaluation entitled ‘Students’ Opinions about Courses’, a 12-item evaluation
used for both PBL and CC curricula, of which 7 items were believed to reflect course
facilitation of SDL opportunities (these variables included “stating learning
objectives”, “making decisions”, “independent thinking”, “problem solving”,
“gathering and analyzing information”, “stimulated to learn more”, and “stimulated
to read the medical literature”). PBL students (n = 60) who completed the SDLRS at
both the end of year one and year two showed no significant differences between the
two scores. The year two overall SDLRS scores and attitude and behavior subscales
for the PBL students (n = 71) and CC students also showed no statistically
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
74
significant differences (n = 51). PBL students (n = 73) scored higher on all 12
features of the ‘Students’ Opinions about Courses’ evaluation form than the CC
students (n = 73), including the 7 believed to reflect SDL behavior (all p-values =
.001).
The authors posit three potential reasons for the lack of significant
differences in the PBL and CC students’ SDLRS scores: 1) the SDLRS is a measure
of self-reported rather than actual behavior, and considerable debate exists in the
literature concerning the definition of the construct of SDL and the separation of its
attitudinal and behavioral components; 2) even in a highly structured curriculum,
students choose which instructional activities to attend (e.g., selective attendance of
lectures), and control their own learning more than is apparent; and 3) as the school’s
admission criteria had not at that time changed with the implementation of the new
PBL curriculum, it is possible that the school has “always admitted highly
sophisticated learners and they continue this way, regardless of the curriculum” (p.
609). In support of a potential association between level of learning and SDL, the
authors note that construct validity studies of the SDLRS (Sabbaghian, 1980) have
reported a positive relationship between self-concept and SDLRS score, and that the
higher the level of learner, the higher the self-concept. Review of this reference
indicated that higher SDLRS scores were associated with higher levels of self-esteem
and self-acceptance (as measured by the Tennessee Self-Concept Scale).
Additionally, women and older learners were found to have higher SDLRS scores.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
75
A third study (Rankin, 1992) examined library usage by surveying second
year students from four different medical schools, including two that offered both a
PBL and a conventional track (response rate = 34%). Students were surveyed on
questions measuring information-seeking competencies, preferences among
resources, perceived barriers to information use, and actual behavior patterns related
to library and information use. No differences in range and variety of resources
chosen were found between PBL (n = 30) and CC students (n = 67) when asked to
select information resources that would be useful in addressing a specific clinical
problem. The preferred resource selected by both groups was a textbook, while
significant differences were found in students’ second and third choices (p < .05);
PBL students ranked journal and Index Medicus/MEDLINE second and third
respectively while CC students ranked faculty and journal second and third
respectively, a difference the author suggests demonstrates a more independent
approach to problem solving in the PBL students. No significant difference was
found between the groups in information-seeking competencies when posed with a
particular informational resource problem. PBL students reported being significantly
more frequent library users than CC students (as measured by range and intensity of
use of fourteen identified library activities; no p-value given); PBL students were
also found to have a greater use of self-selected information resources, as opposed to
faculty-recommended, than CC students (p < .05). Curriculum group was found to
be a significant predictor of library use, accounting for 18.6% of the variability,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
76
while frequency of library use as an undergraduate was the second most important
predictive variable. In addressing possible limitations of this study, Rankin (1992)
refers to a discussion in the literature concerning whether greater use of the library
and information resources is necessarily better for the user. She also states, “the
ultimate question.. .is whether efficient and effective use of information contributes
to maintaining physicians’ competencies and, most importantly, to improved patient
care” (p. 42).
The fourth empirical study (Marshall, Fitzgerald, Busby, & Heaton, 1993)
similarly compared library usage between PBL and CC students. Three different
surveys (response rates 9% to 54%) were administered to students from three
different medical schools concerning number of library visits per month, length of
visits, proportion of study materials obtained from the library, and MEDLINE
searches. Based on survey results, the authors concluded that a greater proportion of
PBL students use the library than CC students, and when they use it, they do so more
frequently, for longer periods of times, and as a source for a greater proportion of
their study materials (no p-values given).
The fifth study in this category (Shokar, Shokar, Romero & Bulik, 2002)
measured SDL readiness in third-year medical students from a PBL curriculum (n =
182) via the SDLRS. The purpose of the study was to establish whether third-year
medical students were more self-directed than average in their approach to learning,
as compared with the national mean score of general adult learners, and to see if
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
77
correlations existed between students’ scores on the SDLRS and measures of
performance on two required third-year clerkship rotations (family medicine [FM]
and a multidisciplinary ambulatory clerkship [MAC]). Students’ mean scores of 235
were found to be significantly higher (p < .001) than the general adult learner mean
of 214. Clinical preceptor scores for both the FM clerkship and the MAC
demonstrated significant positive Pearson correlations with SDLRS scores of .26 and
.24 respectively (p < .01). Shokar et al. (2002) suggest that correlation of SDLRS
scores with the US Medical Licensure Examination steps 1 and 2 scores, as well as
with other clerkship evaluations, is necessary before definitive major conclusions can
be drawn; they also suggest that studying learners’ performance through residency
training and future clinical practice is necessary to determine whether predictive
value of the SDLRS can be established.
The last study (Dolmans & Schmidt, 1994) in this group did not compare
PBL and CC students, but rather focused on the extent to which various elements of
a PBL curriculum influenced students’ decisions on what to study during SDL. A
questionnaire was administered to students across the first four curriculum years (n =
407; average response rate for all years = 68%) at the University of Limburg,
Maastricht, The Netherlands. For the six areas surveyed, the factors found to be
most influential to SDL activities in descending order were reported to be: course
objectives; discussion in tutorial group; reference literature; PBL tutor; lectures; and
content tested. Average scores were found to significantly differ (p = 0.000 - 0.001)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
78
between curriculum years 1-4 for the factors content tested, lectures and reference
literature. The authors concluded that first-year students tended to rely more on the
literature cited in the reference list and content covered in tests and lectures than
students in the other three curriculum years and that, in general, influence of these
elements diminished over the four years. They also noted that although not
statistically significant, the influence of the discussion in the tutorial group seemed to
increase over the four years. Overall, the authors concluded that, “These findings
suggest that students in a problem-based curriculum indeed become better self-
directed learners as a result of being in the curriculum for a longer time and, hence,
becoming more experienced” (p. 379).
PBL Review Articles Referencing SDL Within PBL Curricula
Numerous general summary articles on PBL were reviewed that either did
not incorporate a discussion on SDL or provided descriptive sections only. Twelve
articles (including one book chapter) were located that referenced SDL data,
including five that featured a more comprehensive PBL review, three of which were
meta-analyses.
Five Comprehensive PBL Reviews
In the first of these articles, a seminal PBL review cited by at least 22 other
articles, Albanese & Mitchell (1993) examined the effects of PBL in a ‘meta
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
79
analysis-type’ review of the literature from 1972 to 1992. Their reference to SDL
was limited to two articles, the previously cited Blumberg & Michael study (1992)
on library and learning resource usage, and an additional study (Saunders, Northup,
& Mennin, 1985) concerning study time and location, and types of resources used for
PBL and CC students. For the first study, Albanese & Mitchell (1993) report effect
sizes in favor of the PBL students when compared to the CC students for greater
frequency o f use of textbooks (ES = 1.77), journals and other books (ES = 1.74),
informal discussions with faculty or peers (ES = .61), and less use of cooperative
lecture notes (ES = -2.75), course syllabi (ES = -2.74) and personal lecture notes (ES
= -.51). The authors note that effect size could not be computed for the PBL
students’ greater library utilization (p < .001). For the second study (Saunders et ah,
1985), Albanese & Mitchell (1993) also report effect sizes in favor of first and
second-year PBL students (n = 10) over CC students (n = 10) in areas of hours
studied per day (ES = .95 for first-year students, .82 for second-year students), the
library as a more frequent study location than home (ES - .39 for both years), and
likelihood of texts read to be student-selected rather than faculty-selected (ES = 1.40
for first-year students; 1.48 for second-year students). The authors conclude that
PBL students control a substantially greater degree of their learning efforts than do
CC students, and are substantially more likely to use the library and library resources
to study, while noting “these self-directed learning tendencies seem to accrue even in
the more structured forms of PBL” (p. 62).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
80
In a second meta-analysis, Vemon & Blake (1993) also evaluated the
effectiveness of PBL by reviewing research from 1970 to 1992. Similar to the
analysis by Albanese & Mitchell (1993), these authors also confine their discussion
of SDL to students’ use of learning resources, citing four studies in this area. Vemon
& Blake (1993) specifically cite the Rankin study (1992), the third empirical study
previously mentioned that surveyed second year students from four different medical
schools on questions measuring information-seeking competencies, preferences
among resources, etc. The authors report effect sizes in favor of the PBL students
for greater library use (d = +.41), greater use of self-selected reading materials (d =
+.41), and more frequent reporting of feeling competent in information-seeking skills
(d = +.32). Vemon & Blake (1993) note that similar findings have been reported by
three other small-sample studies (Nolte, 1986; Nolle, 1989; Saunders et ah, 1985),
including the Saunders et al. study (1985) cited by Albanese & Mitchell (1993).
Following the discussion of these studies, Vemon & Blake (1993) suggest, “In
general, the findings of these reports suggest a greater degree of independent study in
the PBL programs than in traditional programs” (p. 557). The authors first use the
term SDL in the discussion section when referring to the aforementioned study
findings, as they state, .the pattern of resource use suggested more self-directed
learning in PBL programs,.. (p. 557), thus presumably equating SDL with the
previous mention of ‘independent study’. In expressing limitations of this research,
Vemon & Blake (1993) note that studies related to SDL have employed static-group
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
81
research designs and have relied on questionnaires rather than behavioral measures
for outcome measures.
In a third (non-meta-analytic) review from the same year, Berkson (1993)
devotes substantial attention to the issue of SDL development in PBL versus
traditional curricula. In this section, Berkson (1993) contrasts studies that found no
difference between PBL and CC students’ use of faculty and reference materials
(Olson, 1987) against studies reporting a wider variety of resource use for PBL
students (Anderson, Camp, & Philp, 1990; Blumberg & Michael, 1992; Saunders et
ah, 1985). In noting limitations of these studies, Berkson (1993) suggests:
And none of these latter three studies explores the content of the
consulted material, or allows us to judge whether curiosity or anxiety
fueled library utilization. But these design details should not detract
from an underlying assumption present in the three papers: that the
exhibition of self-directed learning behavior is equivalent to the
demonstration that self-directed learning skills are being developed
(P- S84).
Berkson (1993) continues her argument by suggesting that proponents of
SDL believe that the practice of SDL in a PBL context enhances SDL skills, which
will then maximize the probability and quality of postgraduate learning throughout
one’s career. The author responds to this belief by arguing, “These convictions echo
the problem-solving literature of the 1970s and early 1980s. If experience in the
search for teachable problem-solving skills repeats itself in the currently poorly
explored self-directed learning domain, these same convictions risk eventual
challenge” (p. S84). In suggesting that “the exhibition of self-directed learning
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
82
behavior may be highly strategic,” with students “responding to the demands of the
curriculum, and eventually, the patient” (p. S84), Berkson (1993) may be echoing the
argument previously espoused by Candy (1991) when he suggested that SDL was
best viewed as both subject and context specific. Berkson (1993) additionally
suggests that SDL skills may be well-entrenched in students successful in gaining
entry into medical school, and expresses the following skepticism in the ability of
any curriculum to develop these skills:
The postgraduate practice of self-directed learning strategies may
prove more dependent on the proximity of available resources,
peer expectations, role models, the physician’s practice profile,
and time constraints than on “putative” skills previously acquired
or refined in a PBL or traditional curriculum, (p. S84)
In the fourth comprehensive PBL review article, and the last of the meta
analyses, Colliver (2000) offers a more recent examination of the effectiveness of
PBL. Beginning with a discussion of the three previously cited PBL reviews from
1993 (Albanese & Mitchell, 1993; Vemon & Blake, 1993; Berkson, 1993), Colliver
(2000) then reviews PBL research published from 1992 to 1998, including three
randomized studies (Mennin, Friedman, Skipper, Kalishman, & Synder, 1993;
Moore, Block, Style, & Mitchell, 1994; Schmidt et ah, 1996) and five non
randomized studies (Cariaga-Lo, Richards, Hollingsworth, & Camp, 1996;
Distlehorst & Robbs, 1998; Hmelo, 1998; Kaufman & Mann, 1998; Ripkey,
Swanson, & Case, 1998) from this time period. Although the focus of Colliver’s
(2000) meta-analysis was on the effectiveness of PBL for knowledge acquisition and
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
83
clinical performance, these eight studies were reviewed for potential data on SDL;
two of the eight studies contained specific references to SDL. One study (Distlehorst
& Robbs, 1998) reported performance data for PBL versus CC students on six third-
year clinical clerkship evaluations that included the category of SDL as one of 11
competencies measured. However, as the focus of this study was to compare the two
groups of students to ensure that the PBL students were not disadvantaged, only
overall performance ratings were reported and compared. Thus, individual data for
SDL were not reported, nor was SDL defined.
The second of Colliver’s (2000) studies that specifically addressed SDL
(Moore et ah, 1994) concerned Harvard’s ‘New Pathway’ (NP) or PBL-equivalent
curriculum, which included as one of its three major objectives, “.. .to have students
acquire skills in and positive attitudes toward self-directed learning” (p. 983). This
study directly reports findings on the school’s other two major objectives, yet leaves
the reader to presume that data on the objective of SDL skill development is included
in the authors’ discussion of ‘Learning styles and preferences’ and ‘Student
experiences’. Within these subcategories, Moore et al. (1994) discuss several
findings. Using a self-report instrument called the Preferred Learning Style Index,
PBL students manifested a significant preference for discovery-style, or student-
directed, learning compared to the CC students who preferred learning in a
traditional, more receptive, or teacher-directed, fashion. PBL students scored higher
than CC students on measures of reflection and lower on a scale measuring use of
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
84
memorization; PBL students scored higher than CC students in reported preferences
for faculty support and educational innovation, significantly lower in their need for
clarity, and perceived significantly greater autonomy and more innovation in their
work. In their discussion, the authors conclude, “Putting these evaluation findings
together, we believe the NP curriculum.. .fostered students’ development of self
directed learning skills” (p. 988). As SDL is not explicitly defined, the reader is thus
invited to make an educated assumption on what measures constitute the authors’
interpretation of SDL.
After Colliver (2000) discusses his findings regarding PBL effectiveness In
other targeted areas, the author concludes his discussion by lamenting the lack of
research in SDL:
One important, but neglected, area of research .. .getjting] directly
at a valuable skill that is central to the PBL approach is self-directed
learning. PBL is said to teach the practice of clinical medicine by
requiring students to teach themselves, in order to firmly establish
life-long habits of self-directed learning. Rapid advances in medical
science make it imperative for practitioners to keep up to date. Thus,
the development of strong habits of life-long, self-directed learning
has become a critical challenge for medical education, and the PBL
approach seems to directly addresses this challenge... (p. 265)
More research on self-directed learning is sorely needed, (p. 266)
The fifth and last of the more comprehensive PBL reviews included a chapter
by Blumberg (2000), who presents a literature review evaluating the evidence that
PBL students are self-directed learners. In her discussion, the author analyzes SDL
research from a three-dimensional framework, as previously noted in this chapter, in
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
85
which she divides SDL into the categories of learning processes, learning strategies,
and performance outcomes. Learning processes were described as: the ability to
define what to leam; ability to plan and operationalize learning; and the ability to
seek out, use, and evaluate resources. In addressing the ability to define what to
leam, Blumberg (2000) recounts an earlier descriptive study (Blumberg, Michael, &
Zeitz, 1990) comparing seven PBL schools on the degree to which learning
objectives were student- versus teacher-generated. This study found that students
from PBL programs which de-emphasized student-generated objectives were found
to rarely utilize additional, self-selected materials, and had little contact with
librarians, whereas librarians from PBL schools heavily emphasizing student
generated learning issues reported extensive contact with students. Additionally,
faculty from two schools that did not evaluate their students on their ability to
develop student-generated learning issues reported that their students had decreased
motivation to become self-directed learners.
Blumberg (2000) discusses self-directed learners’ ability to plan and
operationalize learning in terms of time management, suggesting that self-reported
time spent in independent study is generally considered an appropriate measure of
effort (Schmidt, van der Arend, Moust, Kokx, & Boon, 1993). Although she
references four studies addressing this topic, only one clearly involved medical
students, a study (Blumberg & Michael, 1992) in which PBL and CC students self
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
86
estimated spending equivalent time in educational activities, although PBL students
spent more time “in nonscheduled, or SDL activities” (p. 205).
On the issue of resources sought, Blumberg (2000) presents evidence for
PBL students’ greater use of library resources and student- over faculty-directed
resources compared with CC students by citing several previously discussed articles
(Blumberg & Michael, 1992; Rankin, 1992; Saunders, Northup, & Mennin, 1985;
Marshall, Fitzgerald, Busby, & Heaton, 1993). For evaluation of resource
effectiveness, Blumberg (2000) suggests that the ability to evaluate evidence in order
to influence decision-making is associated with the type of SDL that PBL attempts to
foster, but does not present evidence pertaining to medical students.
Blumberg (2000) defines learning strategies as the methods students use to
study and process information. Comparing memorization techniques against
learning by conceptualization, defined as constructing a knowledge base in an active
way, the author reports the results from a study (Camp, Mitchell, Blumberg,
Kalishman, & Zeitz, 1992) in which preclinical medical students from five schools
were surveyed regarding their learning behaviors. PBL students reported using
conceptualization as the most frequent learning process while CC students reported
using memorization the most. Blumberg (2000) also cites studies from PBL
programs in Australia (Newbie & Clark, 1986), Europe (Coles, 1985) and North
America (Camp et ah, 1992) that suggest that PBL supports the development of
deep-level processing in students.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
87
On the last dimension of SDL, Blumberg (2000) divides performance
measures into short-term and long-term component outcomes. The author re-cites
the Blumberg & Michael (1992) study that measured library resource usage as short
term outcome evidence of PBL-generated SDL behavior. Regarding long-term
component outcomes, Blumberg (2000) initially acknowledges that few research
studies have looked at the long-term effects of PBL in terms of SDL, and then cites
as long-term outcome evidence the finding from the previous study (Blumberg &
Michael, 1992) which found that the significant differences in library borrowing
practices favoring the PBL students continued into the clerkship years. The author
also cites two postgraduate studies as additional evidence of long-term outcomes,
and these and other postgraduate studies will be further discussed in the next chapter.
Other PBL Review Articles Referencing SDL Within PBL Curricula
The previously cited five comprehensive meta-analyses/reviews on PBL were
a subset of twelve total articles located that referenced SDL data within a discussion
of PBL. O f the other seven, five (David, Dolmans, Patel, & van der Vleuten, 1998;
Dolmans & Schmidt, 1996; Finucane et ah, 1998; Norman & Schmidt, 1992;
Schmidt, 2000) directly reference the Blumberg & Michael (1992) study regarding
library/resource usage as evidence of SDL development in PBL. The sixth article
(Thomas, 1997) indirectly references this article by citing Albanese & Mitchell’s
(1993) effect size findings for the study, while the seventh article (Bligh, 1995)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
88
referenced one of the above five articles (Norman & Schmidt, 1992) who themselves
cited Blumberg & Michael’s (1992) work. In four of these studies (Norman &
Schmidt, 1992; David, Dolmans, Patel & Vleuten, 1998; Dolmans & Schmidt, 1995;
Bligh, 1995), the Blumberg & Michael (1992) study was the only study data noted
for SDL development
in PBL undergraduate medical education. Other previously discussed SDL studies
were also cited in two of the three remaining articles (Dolmans & Schmidt, 1994 in
Finucane et ah, 1998; Rankin, 1992 in Thomas, 1997).
In one of these seven reviews of SDL, Schmidt (2000) states that his
MEDLINE literature search uncovered 269 articles on SDL in the last 10 years,
noting the majority of these to be descriptive only. Other than the Blumberg &
Michael (1992) study, Schmidt (2000) references two further studies (Lawrence,
Grosenick, Simpson, & Susteren, 1992; Peng, 1989) that he claims support the
assumption that problem-based instruction encourages SDL, noting: “Students in
these programmes tend to spend more self-study time using a greater variety of
learning resources, and describe themselves as more intrinsically interested in
subject-matter” (p. 243). A review of the Lawrence et al. (1992) article reveals a
study comparison of fourth-year medical students on an ambulatory medicine
rotation, half (N = 45) assigned to didactic method instruction (DM) and half (N =
43) assigned to modified case-method instruction (MCM). It is important to note
that ‘modified case-method instruction’ does not represent authentic PBL; however
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
89
Schmidt (2000) suggested that problem-based instruction, not problem-based
learning, encouraged SDL, It will be emphatically argued in the ‘Discussion5 section
that this terminology is neither interchangeable nor inconsequential to PBL and SDL
research.
In the Lawrence et al. (1992) study, no significant differences between the
groups were found on measures of objective examinations or measurements of
student satisfaction. The MCM group’s statistically greater number of hours (p <
.001) spent in preparing for topic sessions and significantly greater number of
outside resources (p < .001) used for class preparation led the authors to conclude
that, “The increased amount of time devoted to self-directed learning (independent
reading and study) by the MCM students has important implications for lifelong
learning” (p. 224). The second study (Peng, 1989) cited by Schmidt (2000) is
reviewed below.
The last article in this category that cites SDL data (Thomas, 1997), discusses
‘self-learning skills’, rather than SDL specifically. Other than the aforementioned
referenced articles, Thomas (1997) reports findings from other studies concerning
differences in PBL and CC students’ study approaches (Coles, 1985; Newbie &
Clark, 1986) as well as the influence of PBL tutor expertise on students’ study
patterns and satisfaction level (Davis, Naim, & Paine, 1992; Eagle, Harasym, &
Mandin, 1992).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
90
Studies on SDL Outside of a Defined PBL Curriculum
Two studies were located on SDL outside of a PBL curriculum that are
briefly presented here for further reference concerning SDL research. Peng (1989)
was the second study cited by Schmidt (2000) as “supporting the assumption that
problem-based instruction encourages SDL” (p. 244). Conducted at Sun Yat-sen
University of Medical Sciences in China, this study used stratified random sampling
in comparing two groups of medical students (recent high school graduates) assigned
to either a CC group or a pilot SDL group. The latter group had lecture hours
reduced an average of 50% and replaced with self-learning guidelines, including
reading requirements, key points for SDL, reference lists, review questions, self-
evaluation exercises and discussion and summary sessions. Outcome measures used
were results of 15 of 20 essay and multiple-choice examinations given over a three-
year time period that met a (Cronbach) minimal reliability criterion of .50 (range =
.50 to .71). Questions were further divided between basic knowledge and applied
knowledge evaluation. Results indicated that the SDL students outperformed the CC
students on three basic knowledge tests (inorganic chemistry and biochemistry II, p <
.05; microbiology, p < .01) and one applied knowledge test (human anatomy II, p <
.05). Comparison of total examination results between the two groups showed no
significant differences.
A similar non-PBL study in SDL conducted in London (Graham, Seabrook,
& Woodfield, 1999) compared third-year medical students assigned to randomized
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
91
and paired-group general practice and community-care clinical rotations. Students
were assigned to either an SDL group (i.e., received structured independent learning
packs; N = 135) or a conventional teaching group (i.e., taught by tutors based on
patient presentations; N = 91). No significant differences were found between the
groups in learning outcomes as measured by an examination and a writing task.
Discussion of SDL Research in PEL Curricula in Undergraduate Medical Education
A multitude of articles concerning PBL and/or SDL instruction were
reviewed for this analysis with roughly forty articles, either empirically or review-
based, referenced in an attempt to characterize research in SDL in the undergraduate
medical education arena. The literature in this area, particularly research-based
studies, was not plentiful, yet trends did emerge from this analysis and several points
can be synthesized from the results. Areas opined as significant to research of SDL
in PBL will be featured in this discussion, relevant strengths and/or limitations of
each area will be discussed, and, where applicable, individual articles will be
referenced to illustrate particular points. Significant areas of discussion include:
SDL definition; SDL operationalization; sampling; study design; instrumentation;
and interpretation of results.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
92
SDL Definition
The difficulty in establishing consensus on the meaning of SDL applies to the
research arena of SDL as it did within the general medical education literature.
Similar lack of agreement and underdevelopment of the overall construct was found
in this review. O f the articles referenced in the preceding discussion, only three
contained relatively developed definitions of SDL (Blumberg & Michael, 1992;
Schmidt, 2000; Blumberg, 2000). Blumberg’s (2000) tri-dimensional framework of
SDL was well described; the other two SDL definitions were listed in Table 2 (p.
55). Four articles offered limited definitions (Distlehorst & Robbs, 1998; Dolmans
& Schmidt, 1994; Graham et ah, 1999; Mann & Kaufman, 1995) and the remaining
articles did not define SDL. One may infer that in those articles where SDL was not
conceptually defined that its total or partial definition was implied by the outcome
measure in the case of research studies (e.g., library utilization), or through citations
of these research studies in SDL reviews. Although defining SDL through a general
outcome measure such as library/resource utilization is open to criticism, these
studies did define the subcomponents of such variables (e.g., bibliographic searches,
journals read, etc.) in an attempt to measure specific activities. In non-PBL studies
(Peng, 1989; Graham et a l, 1999) where CC students were compared with students
who received less formalized or structured teaching (i.e., the SDL group), exam
results were used as outcome measures; as SDL was not explicitly defined, one is led
to assume that SDL is somehow implicitly being defined as the global process by
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
93
which students independently leam the set of information to be examined. In these
two studies, perhaps what is being implied as SDL may more appropriately be
defined as ‘self-taught learning’. In these cases, learning objectives had been pre-
established for the students, and many of the procedural elements of SDL, such as
articulating learning needs, critically applying knowledge to a problem, etc. were
absent from the learning task. However, it is notable that these were both foreign
studies (China, England), and it is very possible that foreign medical universities
view SDL differently.
O f substantial concern in this area was the identification of authors’ use of
unclear terminology or the interchanging of assumedly synonymous terms that are
not regarded here as equivalent. Effective analysis of SDL in PBL research findings
depends on clearly understanding authors’ interpretations of both of these concepts.
The above discussion exemplified one aspect of this problem, which involves
leaving the reader of a study to adopt a conceptual definition of SDL solely through
the identification or interpretation of the dependent measure. Another problem
involves authors’ usage of terms that are similar to, but are not exactly stated as
either PBL or SDL. The reference to Schmidt’s (2000) use of the term ‘problem-
based instruction’ is one representative case. There is nothing inherently wrong with
using this specific term; the concern is that it is not entirely clear what Schmidt
(2000) was referring to in his assertion that problem-based instruction encouraged
SDL. A careful review of this study reveals the use of several terms: ‘problem-
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
94
based curriculum5 ; problem-based learning’; ‘problem-based contexts’; ‘problem-
based instruction’; ‘problem-based track’, ‘problem-based trained students’; and
‘problem-based school’. While ‘problem-based learning’ could be considered one
form of ‘problem-based instruction’, the reverse of this statement is not correct.
Problem-based instruction is an open-ended generic term where the specific
‘problem’ in question could take almost any shape. A didactic lecture that uses a
patient presentation to convey information could theoretically be called problem-
based instruction. On the other hand, some, but not all, within the medical education
community share an understanding of PBL as an instructional method embracing
certain fundamental principles that conform to educational objectives originally
suggested by Barrows (1986). Unfortunately, Barrows (1986) may have initiated
confusion around this issue at the same time he proposed a taxonomy to better guide
PBL development when he stated: “The increasingly popular term ‘problem-based
learning’ does not refer to a specific educational method. It can have many different
meanings depending on the design of the educational method employed and the
skills of the teacher” (p. 481). This statement is found to be more curious when one
considers that in an earlier work Barrows’ (1983) clearly described procedural steps
specific to a PBL format (see chapter 2). Nevertheless, review of the literature
demonstrates that many authors describing PBL share enough consensus (Albanese
& Mitchell, 1993; Bligh, 1995; Distlehorst & Robbs, 1998; Maudsley, 1999; Norman
& Schmidt, 1992) to provide meaningful definitional parameters, parameters that
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
95
have clearly taken considerable time to establish. Yet, because the literature does not
contain complete parametric unanimity, authors must appreciate the importance of
considering the term ‘problem-based learning’ as it is most commonly described in
the literature. They must also appreciate the importance of clarifying their own
definitions to allow readers to accurately interpret their studies or reviews. Schmidt
(2000) cited Blumberg & Michael (1992), a study utilizing PBL students, and refers
to this group as “students in the problem-based track” (p. 244). It appears from this
notation that Schmidt (2000) is equating ‘problem-based instruction’ with PBL, but
the reader is still left to wonder whether Schmidt (2000) is suggesting that PBL
supports SDL or whether any problem-based instruction supports SDL. It is very
unfortunate that such confusion can arise from the use of the word Teaming’. This
confusion probably stems from the fact that the word learning can be used either as a
noun, to describe a mental state or change arising as a result of knowledge
acquisition, or used as a verb when it refers to the external processes involved with
knowledge acquisition. ‘Problem-based learning’ literally defined can acceptably
mean any learning that results from the study of an identified problem. However, it
does not mean that in the context of medical education. It is argued here that the use
of the phrase ‘problem-based learning’ must be considered in this context as a whole
entity, not as a concept divisible into two separate words with individual definitions.
While there is flexibility in how PBL is implemented, it must be understood as an
educational methodology, and philosophy, [Maudsley, 1999 #180] containing certain
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
96
instructional principles and objectives. Those principles suggested necessary to label
an instructional technique PBL involves learning that: is performed in smail-groups
that are student-centered and faculty facilitated; centers around the progressive
presentation of a patient case without the benefit of prior study; requires students to
activate prior knowledge, think critically, and self-assess current learning and
knowledge deficits; promotes active knowledge integration and deep-level learning;
requires student identification and individual research of learning issues, as well as
critical incorporation of knowledge gained from research back into the case; requires
students to evaluate their learning upon completion of the case; occurs over a
minimum of two sessions; and does not include correct identification of the patient
problem as a learning objective. The educational objectives central to PBL are those
set forth by Barrows (1986) as discussed in chapter 2.
Three similar examples of this problem were encountered in the
interpretation of SDL. Thomas (1997) included a heading in his article called, “The
Effect of PBL on Self-learning Skills.” References to differences between PBL and
CC students on these ‘self-learning skills’ included data regarding resource use,
study approaches, studying for meaning, number of hours studied per day, and effect
of PBL expert facilitators on students’ study habits and level of satisfaction. Vernon
& Blake (1993) discuss PBL and CC students’ learning approaches, learning
resources and library use under the subheading “Academic Progress.” The authors
state that reports suggest “a greater degree of independent study in the PBL
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
97
program s...” (p. 557), later followed by, . .the pattern of resource use suggested
more self-directed learning in PBL programs.. (p. 557). Moore et al. (1994) was
particularly confusing on this point. Stated as one of three major objectives for
Harvard’s New Pathway curriculum was, “to have students acquire skills in and
positive attitudes toward self-directed learning” (p. 983). However, while the study
abstract identified findings for the other two major educational objectives, it did not
refer to SDL as a study variable. After discussing two study categories pertaining to
student ‘Learning styles and preferences’ (e.g., teacher-directed versus discovery-
directed learning, memorizing versus conceptualization) and ‘Student experiences’
(e.g., preference for faculty support, degree of autonomy, educational responsibility),
Moore et al. (1994) conclude “Putting these evaluation findings together, we believe
the NP curriculum.. .fostered students’ development of self-directed learning skills”
(p. 988). Exactly what are these authors defining as self-directed learning skills, and
why is there no mention of SDL being a study variable? No literature reviewed for
this paper addressed what SDL was not, and perhaps that is one reason authors use
various terms equally to refer to SDL. SDL is not considered here to be synonymous
with ‘independent learning’ or with ‘lifelong learning’. Lifelong learning may be
interpreted simply to mean ‘learning throughout one’s lifetime’. It may or may not
be self-directed; attending college is an example of lifelong learning, but probably
features a good deal of teacher-directed, passive learning. Independent learning
refers to individualized learning. This could refer to learning that Is accomplished by
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
98
one’s self or to a learning plan that an individual creates for him or herself. In either
case independent learning is not synonymous with SDL. A student doing homework
alone or listening to a foreign language tape in a lab are examples of learning by
oneself. Creating an individual learning plan is routinely done every time a student
studies for school outside of a classroom. SDL could subsume examples of lifelong
learning or independent learning; however, the reverse is not true. Perhaps if authors
in this field describe more often what SDL and PBL are not, terminology substitution
would diminish.
The critical problem with failing to clearly define PBL/SDL or using
unorthodox, or non-synonymous terminology is that it greatly obscures research
interpretation, which in turn eclipses further understanding of the postulated
association between these two variables. How can studies on SDL in PBL be
interpreted or replicated when the variables originally measured remain cryptic?
‘Problem-based instruction’ does not have clearly articulated educational objectives
attached to its meaning; PBL does. ‘Independent study’ is not postulated to be more
effectively developed in PBL than in traditional curriculum; SDL is. PBL
methodology has been hypothesized and spiritedly debated for many years to be
superior to traditional medical education in developing SDL. For any author to
proclaim evidential assertion supporting this hypothesis without first carefully
considering how the literature does and does not interpret PBL and SDL, and without
then unequivocally defining his or her own conceptual and operational definitions of
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
99
the same, is to effect a great disservice to this field of research. It unnecessarily
makes more turbid these concepts educators have been laboring to clarify. Referring
to the definitional equivocation of PBL, one author states, “Such semantic
uncertainty compromises the evidence-base on the added value of problem-based
versus traditional approaches and the main messages for good practice” [Maudsley,
1999, p. 178 #180]. There are insightful and discriminating definitions in the
medical education literature for SDL, definitions that are burgeoning and malleable,
but durable enough to withstand research scrutiny without crumbling. Until more
educators and researchers express a collective commitment to establishing a greater
unified, disciplined understanding of both of these constructs, however, SDL in PBL
research will be forced to continue to study a moving target, and our understanding
of the true nature of the relationship between PBL and SDL will remain static.
SDL Operationalization
The narrow parameters used to define SDL in these reviewed studies led to
operationalization of the concept in rather fragmented form. By far, the most widely
utilized dependent measure of SDL was students’ use of library resources. While
consideration of which learning resources students use does indeed have merit (i.e.,
this is an accepted part of SDL process definition), it is measuring SDL in a very
limited way if used as a sole measure of the construct, as was the case in many of
these studies.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
100
Two other potentially significant concerns become apparent in reviewing the
literature’s operationalization of SDL through resource utilization. The first
concerns the logistical organization of the two curricula in question. A traditional
medical education curriculum customarily requires students to buy designated
textbooks and provides them with syllabi and supplemental materials, whereas a
PBL curriculum typically does not have assigned texts, and does not provide students
with handouts. Rather, materials may be placed on reserve in the library, where
many different textbooks also are naturally available. Therefore, the increased
library utilization seen in the PBL students may simply be a reflection of their
curriculum structure rather than a true measure of SDL process behavior. This
hypothesis perhaps is supported by Rankin (1992), who reported that curriculum
group was found to be the most significant predictor of library use, accounting for
18.6% of the variability in that study. While learning how to reference various types
of resources may be of value to students, the uncertainty of whether library
utilization is truly indicative of SDL behavior or occurs as an extension of
curriculum design invites potential criticism of its use as an authentic measure of
SDL. In other words, library utilization arguably possesses poor construct validity as
a dependent measure of SDL behavior.
The second problem with using library utilization as a measure of SDL is that
it simply quantifies usage, but cannot tell us anything about the quality of learning
that occurs as a result of that usage. PBL students’ greater use of library resources
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
101
and possible greater resource-accessing ability does not necessarily translate into
better learning outcomes. Berkson (1993) alluded to this point when she commented
that none o f her three reviewed studies which reported increased library utilization
among PBL students (Saunders, Northup & Mennin, 1985; Anderson, Camp &
Philip, 1990; Blumberg & Michael, 1992) explored the content of the consulted
material. As noted in chapter 1, Brookfield (1984) also expressed concern that
embracing quantitative SDL instruments risked overemphasizing the quantity of
SDL to the exclusion of assessing its quality or effectiveness. Rankin (1992) noted
this limitation in her own study of library resource use when she remarked, “Serious
discussions in the literature point to limitations of quantitative assessments and
question whether more use of the library and information resources is necessarily
better for the user” (p. 41). Viewing more frequent library usage as somehow better
than less frequent library usage as an indication of more developed SDL behavior
constitutes a bias belief that repetition of behavior automatically equates to more
skilled behavior. Practice of any behavior is necessary for progressive skill
development; however, it is not sufficient, and to associate a cause and effect
relationship between SDL and frequency or range of library utilization is invalid.
Two of the six empirical studies reviewed measured SDL using
Guglielmino’s SDLRS scale. Although the SDLRS has received criticism (Field,
1989; Candy, 1991) since its introduction in 1977, it remains widely used in studies
of SDL. Brockett (1985), in a review of the instrument, offers findings from several
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
102
studies (Hassan, 1981; Long & Agyekum, 1983; Sabbaghian, 1980; Torrance &
Mourad, 1978) that support the construct and predictive validity of the SDLRS.
Brockett’s (1985) main criticism of the scale, that it appears to be inappropriate for
groups with low formal educational attainment, does not apply in the case of medical
students. Field (1989) offered strong criticism of the SDLRS in several areas,
including the use of a Delphi technique to construct the instrument, the lack of
definition of the term self-directed learner, use of negatively phrased items, and the
method of factor analysis used to support the reliability and validity claims of the
instrument. The author concludes that the SDLRS measures a homogenous construct
that is not SDL readiness, but rather one related to love of and enthusiasm for
learning. In response to this criticism, Guglielmino (1989) defended the reliability
and validity testing and analysis of the SDLRS, as have other authors (Long, 1989;
McCune, 1989b). Kreber (1998) concluded that the SDLRS was appropriate for use
in her study, but acknowledged the criticisms of the SDLRS made by previous
authors (Brockett, 1985; Field, 1989; Candy, 1991). The author reasoned that the
self-response nature of the SDLRS indirectly addresses Candy’s (1991) point that
people’s willingness to engage in SDL activities is shaped by their self-concept as
self-directed learners. As such, students with stronger self-concepts as autonomous
learners could be expected to score higher on the SDLRS, and those with weaker
self-concepts in this area could be expected to achieve lower scores. Therefore, one
could plausibly conclude that the SDLRS measures people’s willingness and
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
103
perceived capacity to engage in SDL, and, to a certain extent, their seif-concept as
autonomous learners. Evidence that found self-concept was indeed positively related
to SDLRS scores (Sabbaghian, 1980) was previously noted in Mann & Kaufman
(1995). Kreber (1998) remarks that it is this willingness and capacity for SDL which
institutions of higher learning wish to foster.
O f the two studies in this review that featured the SDLRS, Shokar et a l
(2002) cited one additional study utilizing the SDLRS with medical students (Pilling-
Cormick & Bulik, 1999). Although perhaps tenuous, as precedent support for the
reliability and validity of the SDLRS is noted in the general education literature, the
use of the instrument in medical education SDL studies would enable researchers to
gather further reliability and validity data specific to medical students and medical
education. As it is a self-report instrument, measures from this scale need correlation
with objective SDL measures to help support or discount the construct and predictive
validity of the SDLRS. Importantly, both studies in this present analysis also used
other measures of SDL behavior or attitude in their studies to help compare and
contrast with SDLRS scores.
The issue of measuring both attitudinal and behavioral components of SDL
was addressed by Shokar et al. (2002), who gathered clinical clerkship information
on third-year former PBL students to determine if clerkship performance evaluations
correlated with SDLRS scores. The authors found significant correlations (p < .01)
of students’ SDLRS scores and preceptor evaluations for two clerkships. Although a
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
104
strength of this particular study was obtaining behavioral measures o f SDL to
support student self-reports, the research design itself was limited, and this topic will
be further discussed shortly.
A significant limitation not well addressed in the SDL medical education
literature is the lack o f ‘gold standard’ methods available to objectively measure
SDL. This omission is not surprising given the extent to which SDL remains
conceptually and behaviorally underdeveloped in the field. As operationalization of
SDL is intimately related to its definition, this issue also represents an area in need of
further evolution.
Sampling
It is well known that PBL studies cannot truly be randomized, as non-
negotiable assignment of students into an experimental PBL or traditional medical
curriculum raises ethical concerns. However, many of the previous studies
acknowledged the limitations inherent in using non-randomized sampling and were
conscientious in reporting relevant sampling characteristics to note group
similarities. Two studies (Mennin et al., 1993; Moore et ah, 1994) reported the use
of ‘randomizing’ students into the PBL curriculum by establishing an original PBL
volunteer admission pool, and then randomly selecting students into the PBL track.
Other studies (Blumberg & Michael, 1992) addressed the issue of group
comparability by reporting no statistical differences in areas such as group
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
105
demographics (e.g., age, sex ratios), entering grades, basic science background, or
grades in medical school. In her study of library usage at four medical schools with
varying curricula, Rankin (1992) matched an entirely PBL institution with a control
school on basis of size, mission, geographic location and library data, and controlled
for user characteristics such as gender, age, education, academic standing, and prior
library experience in her data analysis. Weng (1989) strengthened the quality of his
SDL study by using stratified random sampling (e.g., geographical province, national
entrance exam score, previous educational background) of entering medical students
and subsequently assigned teachers with identical academic ranks and comparable
teaching abilities (as ascertained from student and departmental evaluations) to the
SDL and CC groups. In general, the research reviewed was diligent about
acknowledging sampling limitations and using comparable PBL and CC groups in
these studies.
Research Design
As noted by previous authors (Berkson, 1993; Vernon & Blake, 1993), some
of the reviewed research possessed limitations in its research design. Two of the six
empirical studies reviewed included only a PBL group, with no control group
comparison in their study data (Dolmans & Schmidt, 1994; Shokar et a l, 2002). The
purpose of the Dolmans & Schmidt (1994) study was to analyze the extent to which
various elements of a PBL curriculum influenced students’ decisions on what to
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
106
study. However, one could argue the authors’ conclusion - that PBL students
become better self-directed learners as a result of being in the curriculum for a longer
time and becoming more experienced - is also a conclusion that is basically expected
for any medical student progressing through a generic curriculum. This study would
have been strengthened perhaps by comparing how the designated study factors may
have differentially influenced PBL and CC students across a four-year curriculum.
In this way, differences in how PBL and CC students pursue SDL may be better
clarified. In the Shokar et al. (2002) study, although it was noteworthy that the
authors obtained behavioral measures to support students’ self-reported SDLRS
scores, this study also did not have a control group. It also constituted a posttest-
only design study, as the SDLRS was administered to PBL students on a single
occasion during a third-year clerkship. Here again, the authors’ conclusion that
medical students who participated in a two-year preclinical PBL curriculum were
found to have a higher degree of readiness for SDL than general adult learners based
on their SDLRS scores is open to criticism. A plausible argument could be that most
medical students, regardless of type of curriculum, would be expected to have higher
than average SDL scores, not only as measured during third-year clerkships, but
potentially before they even begin their medical school education. Studies utilizing
the SDLRS have shown that individuals with more years of formal education tend to
demonstrate higher self-directed readiness (Brockett & Hiemstra, 1991), a finding
which supports the argument that no correlation between higher student SDLRS
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
107
scores and a PBL curriculum can validly be established for this study. However, this
study would have been strengthened had a matching control group from a CC
curriculum also been administered the SDLRS, after which correlation analyses with
their clerkship evaluations could have been compared to those of the PBL students
(this medical school instituted a PBL curriculum in 1998 so previous classes
theoretically may have been used as controls).
Another limitation of research design concerned researchers5 restriction of
study data to the preclinical years, or the same time period in which most students
remain in a PBL curriculum. The continuation of SDL research into the clinical
years, when both PBL and CC students participate in the same curriculum, provides
insight into whether any potentially increased SDL behavior measured in PBL
students also continues into the clinical curriculum or if this behavior decays after
leaving a PBL environment. Blumberg & Michael (1992) was the only empirical
study to research effects of SDL in both the preclinical and clinical years. Although
their variables were limited to data regarding library utilization and study resources,
it is also noteworthy that these authors obtained triangulation of data (e.g., student
questionnaires, library circulation data, and student- and faculty-reported perceptions
regarding students’ SDL skills), which potentially strengthens a study by allowing
for clearer interpretation of results. The authors’ discovery that library circulation
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
108
data confirmed students’ self-report data on library usage, for example, lends
credibility to the accuracy of student self-assessments, a topic further discussed in
the next section.
Instrumentation
Two issues of concern in this area include researchers’ heavy reliance on
self-report questionnaires or surveys and the lack of description or availability of
study instruments. Relying on self-reports for accurate measurement of SDL, as for
any behavior or attitude, becomes problematic if self-report is the sole data source
used for the obvious reason that subjects may not be reliable self-assessors, and
because self-reports do not measure actual behavior. In a review of self-assessment
in health professions training, Gordon (1991) found low to moderate validity of
students’ self-assessed performance that did not improve with time. Additionally,
self-assessed performance was reported to be closely related to generalized self
attributions and was minimally influenced by external feedback in the form of test
scores, grades, or faculty assessments. However, in five programs specifically
emphasizing explicit self-assessment goals and training strategies (one medical, two
dental programs), moderate-to-high validity outcomes or improvements were
demonstrated over time.
Although studies are clearly strengthened when supplemental measures
corroborate student self-reports, the medical education literature is also replete with
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
109
research demonstrating poor correlation between medical student/resident and
faculty ratings (Calhoun, Ten Haken, & Wooliiscroft, 1990; Kolm & Verhulst, 1987;
Paiva, 1979; Stuart, Goldstein, & Snope, 1980; Wooliiscroft, Ten Haken, Smith, &
Calhoun, 1993). One cannot automatically assume supervisor ratings to be more
accurate as studies demonstrating both poor supervisor interrater reliability
(Thompson, Lipkin Jr., Gilbert, Guzzo, & Roberson, 1990) as well as high interrater
reliability have been reported (Paiva, 1979). However, even in studies replacing
general criteria with more objective and specific criteria (e.g., videotaped physical
exam, Calhoun,Ten Haken & Wooliiscroft, 1990; clerkship exams, board scores,
Wolliscroft et al., 1993), correlation of student assessments with faculty or with
external measures was still poor. Other studies applying objective criteria
(Wolliscroft et al., 1993; Gordon, 1991) found that lower-performing medical
students demonstrated overconfidence by rating their skills higher than their higher-
performing peers, a finding also supported in studies involving medical residents
(Hodges, Regehr, & Martin, 2001).
Although faculty and student assessments may demonstrate poor correlation,
potential discrepancy should not dissuade the use of faculty assessment for
comparison with student self-assessments. While clearer or more explicit assessment
measures may be needed, the argument presented in the literature on student self-
assessment justifies using faculty assessment to supplement self-report data.
Utilization of faculty assessment also potentiates the opportunity for data
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
110
triangulation. As previously noted, a few of the reviewed SDL studies also used
other data measures in conjunction with student self-reports (e.g., library circulation
data, clerkship evaluations/OSCE results).
The second concern in this area involves the lack of description and/or
availability of research instruments. Description of research instruments becomes
particularly important when a study has attempted to behavioraily measure SDL, as
is often the case with faculty or supervisor evaluations. When conceptual or
operational definitions of SDL are not overtly discussed, reviewing a study’s
instrument provides the reader with an otherwise absent understanding of the
particular attitudes and/or behaviors individual researchers are classifying as SDL
measures. Readers may be interested in learning how faculty and supervisors are
specifically being asked to evaluate students’ SDL ability. Instrument provision
allows for better purview of SDL measurement and may help to increase general
understanding, hasten measurement consensus, or provide direction in SDL; it may
also serve to identify or clarify issues concerning instrument reliability/validity.
Additionally, result analysis may be clarified for readers when study instruments are
available, as this provides them with an opportunity to understand individual
measure data analysis if data is reported only for grouped or global measures. In
Mann & Kaufman (1995), the authors provided a table of seven measures they
considered to be SDL-fostering features. These features were part of what the
authors referred to as a ‘Student Opinions about Courses’ evaluation. By including
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
I l l
these, the authors allowed the reader to specifically understand what particular
features they considered to he SDL-enhancing and how students from the PBL or CC
curriculum rated each one.
Two of the studies reviewed used instruments involving faculty assessment.
In the Shokar et al. (2002) study, which found a significant correlation between
medical students’ SDLRS scores and clerkship preceptor evaluations, the authors
provided web addresses to allow interested readers to download the actual preceptor
evaluation forms used in this study. This provided specific information of the
behavioral and attitudinal characteristics that were evaluated and found to
significantly correlate with SDLRS scores. What may have provided further
valuable information in these evaluation/SDLRS correlations would have been an
analysis of individual correlations of the four specific areas faculty evaluated -
patient evaluation, patient management, communication skills, and learning - with
SDLRS scores to ascertain whether the category of Teaming’, which most closely
represented components of SDL behavior, had a higher correlation with SDLRS
scores than the total evaluation.
The second study that utilized faculty assessment in SDL data collection
(Blumberg & Michael, 1992) used faculty-reported perceptions regarding CC and
PBL students’ SDL skills as one of three sources of study data. Although this
assessment was from an unpublished internal program evaluation, the authors
provided the reader with a partial understanding of the SDL assessment tool
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
112
measurement. Faculty reported higher SDL skill ratings for PBL students in such
areas as willingness to admit, and consequently pursue, self-perceived lack of
understanding, ability to find relevant references and subsequently obtain necessary
information, and frequency of asking questions. Although it is unclear whether the
faculty were specifically asked to comment on these attributes, or whether faculty
themselves individually defined SDL in this way, this point is secondary to
understanding what SDL behavior the instrument measured.
Interpretation of Results
The last area of discussion in this analysis, interpretation of results, is closely
interwoven with the issues of SDL definition, operationalization, and research
design. Interpretation of results follows from a study design, which is based on how
a person conceptually defines SDL and then operationalizes it. In Blumberg &
Michael (1992), the authors define SDL as, “recognizing the need for new learning,
setting one’s own learning objectives, defining relevant questions for study,
accessing relevant information, [and] testing one’s depth of understanding of what
one has learned” (p. 3). Their three SDL measures were student-reported data on
learning resource use, library circulation data, and student- and faculty-reported
perception of certain SDL skills. From these data, the authors concluded, “students
in the mixed PBL curriculum,.. .seem to develop or manifest self-directed learning
behaviors to a greater extent than do students in the teacher-centered RC [regular
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
113
curriculum]” (p. 7). As noted under ‘SDL Operationalization’, the quality of
learning that occurred here is completely unknown. Accordingly, data on ‘testing
one’s depth of understanding of what one has learned’, the last component o f the
authors’ SDL definition, was not reported here, and apparently the study design did
not include a method of measuring this. The authors’ definition of SDL also does
not include significant procedural components such as critical thinking and logical
application of new knowledge to solve a learning problem. To conclude that PBL
students develop or manifest SDL behaviors more than CC students without
considering the very heart of SDL - students’ quality of learning and appropriate and
critical use of that learning - is a very large stretch.
In contrast, the Marshall et al. (1993) study, also concerned with library
utilization, concluded that PBL students used the library more frequently, for longer
periods of time, and as a greater source of study materials than CC students.
However, the authors did not conclude that PBL students developed SDL skills
better than CC students. In discussing possible factors that may have influenced
these differences, the authors stated, “More likely, the nature of the PBL curriculum
itself, with its emphasis on self-directed learning and resource identification and use,
has a major impact” (p. 304, italics added). Regrettably, the authors did not define
SDL, and whether they intended for ‘resource identification’ to be subsumed within
SDL or not, they did not interpret the greater library use by PBL students as
constituting more effective SDL.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
114
Two additional examples further illustrate the global problem concerning
how SDL study definition, operationalization, research design and result
interpretation can be incongruous or skewed to distort the understanding of SDL.
The first example was previously noted in the Dolmans & Schmidt (1994) study that
looked at the influence of different factors on SDL activities in PBL students. The
authors noted that first-year students relied more on cited reference literature, content
covered in tests, and lectures more than students in the other three curriculum years,
and that, in general, influence of these elements diminished over the four years.
Although not statistically significant, tutorial group discussion influence tended to
increase over the four years. The authors concluded, “These findings suggest that
students in a problem-based curriculum become more accomplished self-directed
learners over the four curriculum years...” (p. 372). This study was conducted in the
Netherlands, and although European medical training does vary from U.S. training,
typically speaking, lectures and exams become less frequent as the curriculum
proceeds, and students’ clerkship grades (years 3-4), which largely include
evaluation of patient care duties, generally are not as dependent on exam scores as
are first- and second-year students’ basic science grades. Additionally, most first-
year medical students do not know a lot of medically relevant basic science and are
therefore naturally more dependent on teacher guidance and reference lists for their
preliminary learning. Aside from this potential difference partially accounting for
the authors’ findings, the more significant point here is the authors’ suggestion that
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
115
solely due to less dependence on lecture notes and exams for their studying, or in
other words, a change in choice of study resources, PBL students became more
accomplished self-directed learners over the four-year curriculum. One can easily
argue that the choice of resources a student uses to study teaming issues in PBL is
not indicative of the level of accomplishment of a student’s SDL skills, particularly,
once again, when we know nothing about the quality of learning or how appropriate
the choice of learning resources was for a particular learning issue. How is the
reader to know that lecture material was not the best material to be studied for any
given learning issue? The authors simply did not provide enough pertinent
information to validly conclude that less frequent use of lecture material or cited
reference literature was equivalent to more accomplished SDL.
The last example illustrating the concern with result interpretation accuracy is
the Schmidt (2000) article in which the author states that the “few empirical studies
available generally seem to support the assumption that problem-based instruction
encourages SDL. Students in these programmes tend to spend more self-study time
using a greater variety of learning resources, and describe themselves as more
intrinsically interested in subject-matter” (p. 244). The prior discussion on
Schmidt’s (2000) unfortunate choice of the term ‘problem-based instruction’
notwithstanding, there were other concerns with this author’s analysis. Review of
Peng (1989) revealed that not only was PBL not utilized in the curriculum, but the
author made no mention of using any ‘problem-based instruction’ whatsoever. Peng
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
116
(1989) and Lawrence et al. (1992) were also reviewed for analysis of the author’s
references to ‘more self-study time’, ‘greater variety of learning resources’, and
‘more intrinsically interested in subject-matter’. The curriculum in the Peng (1989)
study was specifically designed to give the SDL students a significant (30% - 57%)
reduction in lecture hours, but measured neither students’ utilization of learning
resources, nor their intrinsic interest in subject-matter. Lawrence et al. (1992) stated
that the MCM (i.e., ‘problem-based’) group reported spending significantly more
hours preparing for the topic sessions and consulting a significantly greater number
of sources than the CC students. However, review of this study revealed that the
MCM group received case-logs in which they were asked to identify a clinic patient
possessing one of the designated disease topics for study, and subsequently answer
assigned research questions that they were to discuss within their group; the CC
group was not issued case-logs. Additionally, while the MCM group did report
higher satisfaction ratings on a number of course topics, including ‘motivation to
leam more about the topics’, assumedly representing Schmidt’s (2000) ‘intrinsic
interest in subject-matter’, this rating was not statistically significant.
The examples scrutinized in this result interpretation section are used to
emphasize how incomplete or weakly substantiated conclusions, while neither
categorically erroneous nor without partial merit, can be open to challenge and
criticism. Educators’ desire for novel instructional techniques to succeed in
improving medical students’ knowledge and ability must not influence the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
117
interpretation of research outcomes. Particularly in education, where variables may
suffer from ambiguous conceptual or operational definition more than in other fields,
it thus becomes essential that research be well designed and logically analyzed.
Amorphousness or lack of congruity between SDL definition, measurement, study
design and result interpretation allows SDL to remain poorly understood and
impedes its development.
Research Discussion Epilogue
A significant question to be addressed in this analysis of SDL in PEL
research is whether or not the research findings support the hypothesis that PEL
instruction more effectively imparts SDL development than a traditional curriculum.
One could argue that given the research presented, no conclusion to this question can
be drawn. SDL research, either within PEL instruction or as an independent entity,
is in a developing stage in medical education. Analysis of the research presented
possessed many positive characteristics, such as attempting to triangulate data, using
behavioral measures of SDL to supplement self-reports, providing instrument
measures, and ensuring the comparability of non-randomized PEL and CC groups.
However, other than providing group comparability data, much of this was
inconsistent, and done randomly in one or two individual studies, not collectively.
Most, if not all, of these studies contained both strong and limited research
characteristics. The underlying lack of conformity within the areas of SDL
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
118
definition, operationalization, research design and result interpretation do not allow
one to make valid conclusions about differential effectiveness of SDL development
within PBL and CC curricula. Although evidence was previously presented that
theoretically supported the development of SDL within a PBL curriculum,
limitations within this research field preclude educators from legitimately proposing
that PBL curricula undisputedly offer more effective and sustained development of
SDL behavior than do traditional medical education curricula.
Recommendations for SDL Instruction and Research in Medical Education
Salient points emerging from chapters 2 and 3 reveal that SDL: a) is not a
prevalent, well-established, formal part of PBL medical education instruction or
research, and b) variation in the overall understanding of the concept continues to be
widely discrepant. With a focus on advancing the awareness, incorporation, and
research of SDL in medical education, the following recommendations are hereby
proposed:
1) Establish a meaningful SDL definition appropriate to medical education. Medical
educators need to adopt a clear and reasonably acceptable working definition of
SDL. After nearly four decades of research and discussion, the field of adult
education is not totally united in its SDL definition, and it is therefore probably not
realistic to expect the field of medical education to find unanimous agreement any
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
119
time soon. However, among adult education scholars there is general and reasonable
accord for defining SDL as a two-dimensional process-product construct, and that is
the model recommended here. SDL process is best understood as a learning skill set
that is directly teachable to medical students and emphasizes learner self-awareness,
critical thinking, and knowledge application. Slightly modified from chapter I, the
proposed definition for the procedural dimension of SDL in medical education is:
the set of skills necessary for a student to successfully fulfill a
designated learning task, including:
♦ the accurate self-assessment of current level of knowledge
and/or skill;
♦ the articulation of learning needs and objectives necessary to
successfully satisfy the learning task;
♦ appropriate utilization and critique of learning resources;
♦ use of effective learning strategies;
♦ critical and appropriate application of new information or
skill resulting in a meaningful enhancement of learning; and
♦ the objective and accurate self-evaluation of learning outcomes,
including the identification of potential areas of further study.
The proposed definition of SDL product or goal in medical education is:
the identification of learning needs consistently generated over
time that leads to the internally motivated, self-initiated, and
sustained behavior required to effect intentional, objective-based,
and permanent learning for the purpose of enhancing one’s
professional development.
2) Establishment of SDL as a formal and significant goal of PBL and non-PBL
education. As one of the four advocated PBL objectives, SDL development is not
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
120
consistently addressed throughout PBL curricula. Evidence was presented in chapter
2 that supported the promotion of SDL process within a PBL methodology. Medical
educators are encouraged to strengthen the development of students’ SDL skills in a
PBL curriculum by consciously reinforcing those procedural skills proposed under
recommendation 1. Towards this goal, educators are also encouraged to utilize those
instructional techniques previously discussed and endorsed by Candy (1991): a)
making use of learners’ existing knowledge; b) encouraging deep-level learning; c)
increasing question-asking by learners; d) developing critical thinking; e) enhancing
reading skills; f) improving comprehension monitoring; and g) creating a supportive
climate for learning. Previous analysis of PBL methodology revealed how each of
these strategies could be reinforced during a patient case analysis.
Although this paper’s main focus concerns the review of SDL within a PBL
curriculum, a majority of U.S. medical schools either do not employ PBL, or do so
minimally. It becomes most essential, therefore, for these schools to categorically
identify SDL development as a formal educational goal. Aside from PBL, other
possible techniques non-PBL institutions can institute to enhance SDL skills include:
the use of patient cases for classroom analysis, self- and peer-assessment, individual
or small-group teaching responsibility, discussion and analysis of evidenced-based
medicine and clinical research, and assignment of student learning projects. It is
recommended that all medical educators and administrators examine their curricula
to determine the extent to which SDL is presently promoted and to consider the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
121
creation of innovative opportunities to further support its development. Additional
potential SDL-enhancing techniques are further discussed in chapter 6.
3) Formative and summative SDL assessment. A long-proposed educational axiom
is “assessment drives learning.” Students perceive greater value in those curricular
elements that will be examined. Explicit declaration of SDL assessment to students
emphasizes the importance of acquiring this skill. Students first need repeated
opportunities to practice SDL subcomponent skills, where formative assessment can
determine their progress and self-confidence. The TIE, previously described in this
chapter, offers one form of SDL summative assessment. This assessment could be
implemented using either standardized patients or paper cases. Although initially
labor intensive both in content development and faculty training, the TIE is capable
of measuring higher cognitive skills and critical thinking; it could potentially be used
to measure knowledge as well. The TJE also has the important benefit of simulating
an authentic clinical context. The TJE could be implemented in both PBL and non-
PBL curricula. The Southern Illinois University of Medicine TJE previously
discussed offers a possible model for other medical schools to follow. Alternative
SDL evaluation methods may include the submission of written analyses, learning
issues and corresponding research for patient case scenarios, subject essay exams
designed to evaluate specific SDL skills, write-ups of possible patient problems or
issues encountered in an Introduction to Clinical Medicine course, and research or
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
122
teaching projects. Curriculum committees are encouraged to create meaningful SDL
evaluation tools, outlining specific skills they wish to assess, and to implement SDL
evaluation throughout the four-year medical school curriculum.
4) Overt discussion, advocacy and role-modeling of SDL as a professional goal.
PBL was characterized as an effective means of enhancing SDL procedural skills.
However, one of the most challenging goals facing educators in the area of SDL
development is to identify methods capable of promoting the desired goal behavior
of SDL in students. Although PBL theory suggests that this instructional technique
is more effective than a traditional curriculum in developing both SDL process and
outcome behavior, there is no convincing evidence for this. Studies examining this
hypothesis are largely conducted at the post-graduate level, and as SDL in post
graduate medicine is the subject of chapter 4, this issue will be further addressed
there. One suggestion that does not appear to be well explored in the literature is to
approach SDL goal development with students from the perspective that it is an
essential component of medical professionalism. Faculty are encouraged to involve
students in meaningful dialogue concerning the importance o f engaging in SDL
behavior in a responsible and critical manner throughout their careers to improve
their clinical acumen, skills and patient care management. Instructors are
encouraged to share personal examples of SDL behavior and to role model SDL in
their teaching through researching and discussing newly learned information with
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
123
students, sharing literature, discussing new developments in medicine, etc. Medical
educators may benefit from encouraging students to visualize what is entailed in
being an actual clinician more often by discussing with them educational issues that
arise in the course of practice. Discussing such topics as specialty certification, the
regular need for re-certification, and participating in continuing education may help
to sharpen students’ awareness of some of the professional educational expectations
that await them. Emphasizing other aspects of practice such as the importance of
regular journal reading, being aware of new treatment guidelines or pharmaceutical
developments, using evidenced-based medicine, or being asked to address patients’
questions on unfamiliar topics (e.g., the use of alternative health care supplements)
also may demonstrate to students what a significant role continuing education plays
in the everyday life of a physician.
5) Enhancement of SDL research. Strengths and limitations of current SDL research
in undergraduate medical education were described in this chapter. In consideration
of this discussion, it is recommended that future SDL researchers: a) conceptually
define SDL in the beginning of a study, using a definition similar to that stated under
recommendation number 1; b) operationalize SDL in a manner logically consistent
with the proposed conceptual definition, specifying those particular SDL skills a
study is attempting to measure; c) focus on quality over quantity of learning
outcomes through objective assessment measures; d) attempt to triangulate data both
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
124
to increase the opportunity to obtain more complete research answers and to avoid
overdependence on SDL measurement from any one angle (Patrick & Middleton,
2002; Perry et a l, 2002); e) use control and experimental groups that are statistically
comparable on relevant background variables when random sampling is not possible;
f) use pretest-posttest research designs wherever possible, which are superior to
posttest-only designs in establishing internal validity (Isaac & Michael, 1997) of
educational intervention; g) sufficiently describe all instrumentation used, provide
hard copies or make electronic copies available via the internet; h) limit result
interpretation to only those specific skills or behaviors designated by the research
design and SDL operation; and i) extend SDL research into the clinical years
whenever possible.
6) Dissemination of SDL ideas at the national level. The medical education
community holds annual national conferences that attract sizeable numbers of
educators and administrators. It is recommended that conference coordinators
designate more time to the topic of SDL within these forums. Retaining medical
educators familiar with instituting or researching SDL as conference speakers would
help to promulgate ideas, research, instructional techniques, etc. in the field and
extend educational awareness and understanding of SDL.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
125
Chapter Summary
Chapter 3 presented an overview of SDL in PBL medical education.
Analysis of the literature revealed that many disparate notions of SDL exist in this
field and in many cases SDL definition was either limited or absent. Literature
discussing SDL assessment was found to be scarce, however, two more descriptive
examples in this area and supplementary references revealed the triple jump
examination to be an established evaluation tool of SDL in PBL education.
Considerable attention was focused on exploring SDL in PBL research in
undergraduate medical education. Numerous studies were categorized and
discussed, and research strengths and limitations were addressed in detail.
Recommendations were then presented for enhancing SDL in medical education and
strengthening SDL undergraduate research.
Critical to the measure of success of any curricular intervention is its ability
to have a lasting effect on students. Chapter 3 was concerned with reviewing the
development of SDL in PBL within undergraduate medical education, but of larger
significance is determining whether any perceived increase in SDL behavior in PBL
students extends into their postgraduate training. In other words, do PBL graduates
develop and demonstrate the goal or attitude of SDL more effectively than CC
graduates? To address this question, review of the research on SDL behavior in CC
and PBL postgraduates will be presented in chapter 4.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
126
CHAPTER 4
SDL IN PBL RESEARCH IN POSTGRADUATE MEDICINE
In this chapter, factors prohibiting the examination of post-graduate SDL in
PBL research using meta-analysis are described and a rationale is provided for
reviewing this research using a non-statistical meta-analysis as proposed in the
literature (Bland, Meurer, & Maldonado, 1995). Central components of this
technique, including the creation of an SDL model and the development of a study
coding form and rating scale are discussed. Selection of coding form categories
based on significance to study validity is described. Results from nine studies
utilized in the analysis are discussed and several data tables are presented. Three of
nine studies reported significant differences between the PBL and CC graduates on
measures of SDL in favor of the PBL graduates. Study strengths were found to
include comparison of study respondents versus non-respondents; sample sizes with
adequate power; and the use of multiple SDL operational measures. Study
limitations included the absence of a PBL/SDL model; use of post-test only research
design; inadequate reporting of reliability/validity instrument measures; and lack of
SDL conceptual and operational definitions. Discussion of results focused on the
areas of SDL definition, research design, data reporting and instrumentation.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
127
Rationale and Literature Review for Proposed Statistical Meta-analysis
While it may be debatable exactly when (or if) a student internalizes the
attitude indicative of SDL outcome, most students must first spend an appropriate
period o f time developing and practicing SDL skills in a guided environment. Since
the ultimate goal of SDL development is practice of this behavior throughout one’s
lifetime, SDL is then best researched after the passage of time to determine the
extent to which a learner continues to practice this behavior after leaving an SDL-
enhancing environment. The original intent of this chapter was to perform a meta
analysis on relevant post-graduate research studies to determine effect sizes for
potential SDL behavior differences between PBL and CC post-graduates, or, in other
words, well-after PBL learners had left their SDL-associated curriculum.
Considering meta-analysis of SDL behavior to determine effect sizes is appropriate
to this level of training then and not to undergraduate education since PBL learners
are still engaging in intentional SDL-reinforcing instruction.
The literature search used to locate studies for this proposed meta-analysis
was previously described in chapter 3. As meta-analysis is based on the quantitative
treatment of an aggregation of study results, this method could determine the
magnitude of effect size regardless of whether individual studies were significant or
not (Lipsey & Wilson, 1993). This approach also has the advantage of compensating
for any small sample sizes that may not have the statistical power to achieve
significance. Greater use of meta-analysis and concern for effect size is also
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
128
recommended over the use of the statistical significance test due to the high error
rate in this test’s interpretation (Hunter, Schmidt, & Jackson, 1982; Michalczyk &
Lewis, 1980; Schmidt & Hunter, 1995).
The available research literature was reviewed for studies potentially
conformable with meta-analysis requirements. Two preliminary inclusion criteria
required both the use of clearly defined and comparable PBL/CC groups, as well as
independent measurement indices of SDL or a potentially equivalent construct. One
example of a study excluded from consideration based on the first criterion used a
single sample of PBL graduates who ranked themselves and their CC peers on SDL
skill (Schmidt & van der Molen, 2001). Another example from this category
compared a group of Middle Eastern post-PBL interns to ‘graduates from other
schools’ (Al-Haddad & Jayawickramarajah, 1991). An example of a study excluded
based on the latter criterion included one in which SDL was measured within an
aggregate group of traits entitled ‘clinical competence’ (Barnsley et ah, 1994). Nine
of the studies reviewed conformed to the necessary preliminary criteria. However,
after careful analysis of these studies and subsequent consultation with a research
statistician, it was concluded that a meta-analysis could not be meaningfully
performed on the available data. Reasons included: 1) inadequate statistical data
reporting (e.g., only 2/9 studies included means and standard deviations); 2)
undefined dependent measures of SDL (‘self-directed learning’ in one study,
‘independent learning’ measure in another study not further defined; e-mail request
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
129
for a copy of the latter study’s instrument went unanswered); 3) incomparable
dependent measures of SDL (e.g., ‘mean number of times/month spent finding
information electronically’; ‘> 5 h of medical discussions [per week]’; ‘continually
evaluate my own performance’; etc.); 4) different reporting groups (e.g., self-report
only; self-report + supervisor report; supervisor only); and 5) dissimilar instrument
scales (e.g., Likert scales with upper values ranging from 5-10; values reported in
absolute hours or as ‘more than/quite/less than well prepared/competent’).
Non-statistical Meta-analysis
Too many incongruities were found within these nine studies to allow a
meaningful meta-analysis to be performed. However, the SDL in PBL post-graduate
research literature warrants examination similar to the undergraduate research
literature to determine potential strengths and limitations that may help guide the
planning of future research. Thus, it becomes important to review this research using
another form of analysis. Bland, Meurer & Maldonado (1995) argue that topic areas
where the research is not of the quality or consistency necessary to conform to
statistical meta-analysis may be more appropriate for non-statistical analysis and
synthesis. In their description of a non-statistical meta-analysis, or literature
synthesis, these authors also note the presence of similar difficulties as described
above pertaining to studies on primary care career choice, such as definitional
inconsistencies, significant variation in independent and dependent variables, and
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
130
incomparable instruments. Using this method, the nine SDL in PBL post-graduate
studies will be reviewed to explicate the current quality of research in this area.
Bland et ah (1995) propose that the steps involved in conducting a non-
statistical meta-analysis are comparable to those of a statistical meta-analysis,
including model development, literature retrieval, literature coding, rating references
for quality, and annotating quality references. Literature retrieval and corresponding
inclusion/ exclusion criteria were previously discussed, and due to the limited
number of retrieved studies, the annotation of quality references step was omitted.
The remaining model development, literature coding and reference rating steps were
modified from Bland et al.’s (1995) original design to better reflect the issues
relevant to the study analysis of SDL.
A Model of SDL
Bland et al. (1995) maintain that a model is fundamental in the front end of
an individual study to guide the variables studied and determine study design and
analysis, as well as at the end of a study to help interpret results. Similarly, the
authors propose that using a model to govern a literature synthesis is also essential.
A model helps provide literature search direction; allows for the grouping of findings
into themes, which collectively may offer explanations in the larger area of interest;
and may reveal further areas of study. To provide a framework for the current
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
131
literature analysis of SDL in PBL and CC post-graduate physicians, a model of SDL
in medical practice is proposed in Figure 1.
The two spheres in this model represent the two SDL domains of process and
product. The conceptual and operational definitions stated for the process dimension
of SDL (inner circle) are equivalent to those proposed in chapter 3. To reiterate, the
skill set considered fundamental to the procedural development of SDL includes: 1)
Self-assessment of current learning state; 2) identification of learning goals and
needs; 3) learning resource selection/critique; 4) use of effective learning strategies;
5) application of new knowledge; and 6) self-evaluation of learning. In a feedback
process, the identification of potential further learning needs may act to recycle back
to skill one. Proposed external forces capable of influencing SDL process
development include previous SDL instruction, medical education curriculum,
repeated practice and SDL assessment.
The outcome or goal dimension of SDL (outer circle) includes a conceptual
definition similar to the one in chapter 3 except with a stated difference in learning
purpose. Rather than learning being for the purpose of ‘enhancing one’s professional
development’, as applicable to medical students, for physicians the goal of SDL is
now ‘for the purpose of changing clinical management practices or expanding one’s
professional knowledge base’. O f central importance to this definition Is the need
for SDL to be internally motivated. Externally motivated SDL is not considered
authentic SDL behavior as some form of potential extrinsic gain serves as the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
FIGURE 1 - SDL PROCESS/PRODUCT MODEL IN MEDICAL PRACTICE
SDL Product/Outcome Dimension
The product of SDL is defined as the identification of learning
needs consistently generated over time that leads to the internally
motivated, self-initiated, and sustained behavior required to effect
intentional and permanent learning for the purpose of changing clinical
management practice or expanding one’s professional knowledge base.
Level of motivation may be influenced by four factors: 1) perceived value of the
learning task; 2) contextually dependent self-efficacy beliefs; 3) mood in relationship
to task, context or self; and 4) external influences (e.g., task agency and competing
interests).
SDL produet' is operationalised by: AND
® Independent reading
® Use of supplemental materials
® Medical consultations
® CME participation
® Non-CME activities
® Volunteer activity
® Change in patient
care practices
SDL Process Dimension
The process of SDL is defined as the skill set
needed to fulfill a purposeful learning task.
SO I process Is opmaSionmSimd by
• Self-assessment of current learning state
® Identification of learning goals & needs
® Learning resource selection/critique
® Use of effective learning strategies
® Application of new knowledge
® Self-evaluation of learning
AND
Influenced by:
* Medical education curriculum
Previous SDL development
* Repeated practice
» Assessment
infiuemed by:
* Role-modeling
* Medical education
or training
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
133
learning impetus, rather than learning to improve one’s patient care abilities or
learning for personal interest; examples of this (not depicted) include learning for
such reasons as improving job income, obtaining necessary continuing medical
education (CME) credit, or attending conferences based on geographical location or
socialization opportunities. Motivation itself is postulated to be influenced by four
factors: 1) Perceived value of the learning task; 2) contextually dependent self-
efficacy beliefs; 3) mood in relationship to the task, the context or oneself; and 4)
external influences, such as task agency (e.g., task permissibility - “Will someone
cover my patients while I’m gone?”) and competing interests (e.g., distractions - “I’d
love to attend this conference but I have a sick relative who needs my attention”).
SDL outcome behavior is proposed to be influenced by role-modeling or medical
education/training.
One noteworthy difficulty pertaining to the product dimension of SDL
concerns operationalizing the construct as a goal/product. To this point, the
literature has not promoted any clear or unanimous description of specific measures
as appropriate dependent variables for SDL outcome. As an initial attempt to offer
direction in this area, the following measures are proposed as potential dependent
variables for SDL outcome behavior with the important caveat that the motivation
associated with these variables must concurrently be assessed and found to be
internally driven: 1) Independent reading, including medical journals, texts, and
relevant computerized materials (e.g., literature searches); 2) use of supplemental
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
134
relevant materials (e.g., audiovisual aids, technical programs); 3) consults with other
medical professionals; 4) CME participation; 5) participation in non-CME medical
conferences, meetings, workshops, etc; 6) volunteer activity for the purpose of
attaining or reinforcing new knowledge/skills (e.g., teaching); and 7) change in
patient care practices.
The process and product dimensions of SDL are specifically depicted as two
spheres, one subsumed within the other, for two reasons. First, the process of SDL is
represented as the smaller sphere as it is viewed as less critical than SDL outcome to
the overall concept - while learning a set of skills is important, particularly for more
naive self-directed learners, the absence of these skills is less consequential than the
absence of the attitude and motivation necessary for the goal of SDL. It is certainly
plausible that a motivated self-directed learner may achieve effective learning
outcomes without being trained in the procedural skills of SDL, but highly unlikely
that a person skilled in the procedural components of SDL could achieve the same
learning outcomes without the corresponding motivation to do so (the combination
of the two dimensions, however, maximizes SDL behavior). Secondly, the sphere
overlay also represents the reciprocal influence the two dimensions exert on one
another, depicted in Figure 1 as the two arrows between the circles. Possessing
procedural SDL skills may influence a person’s learning motivation toward SDL
goal behavior (e.g., self-efficacy) and SDL goal motivation may certainly influence
one’s participation in the SDL process.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
135
As previously stated, the research in this area is currently too sparse to use a
model to provide meaningful direction in a literature search; paring down nine
articles any further is not practical. However, a model will provide a basis for
interpretation of the literature synthesis findings.
Literature Coding and Rating
At this time the other two steps in a non-statistical meta-analysis, literature
coding and reference rating, will be considered. Four types of validity are of concern
in research analysis - internal, construct, statistical conclusion and external validity;
different factors potentially jeopardizing study validity are known as ‘threats to
validity’. A review of the threats inherent to each type of validity was summarized
from the works of Campbell & Stanley (1963), Cook & Campbell (1979), and Isaac
& Michael (1997) and is presented in Table 3. It is not feasible in research analysis
to consider every threat listed under each of the four validity categories; rather, some
threats are more substantial than others in different types of analysis. The selection
of coding variables to be used in this literature synthesis was based on validity
threats considered most pertinent to SDL in PBL research. The rationale behind the
selection of each category and its point weighting is discussed below. The coding
form and associated rating scale created for this analysis is a heavily modified
version of the original form created by Bland et al. (1995), and can be found in
Appendix A.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
136
TABLE 3 - SUMMARY OF RESEARCH THREATS TO VALIDITY*
Class
Internal Validity
History
Maturation
Testing
Instrumentation
Statistical regression
Selection
Experimental mortality
Interactions with selection
Ambiguity about the direction of
of causal influence
Diffusion or imitation of treatments
Compensatory equalization of treatments
Compensatory rivalry by respondents
receiving less desirable treatments
Explanation
External events occurring between the
pretest and posttest that may influence
results
Biological and/or psychological
processes occurring as a result of the
passage of time between pretest and
posttest that may affect results
Test familiarity that may influence
posttest results
Change in measuring instruments, raters
or observers between pretest and
posttest that may account for posttest
results
Selection of groups based on pretest
extreme scores may affect posttest
results
Initial differences between groups that
may affect posttest results
Difference in subject losses between
groups that may affect posttest results
Initial selection differences between
groups that act in combination with any
of the previous threats to affect results
Order of cause-effect temporal
precedence is unclear; may occur in
cross-sectional studies
Communication between groups
regarding treatment may affect results
Control group receives desirable goods
or services
similar to experimental treatment
Public knowledge of treatment provides
extra performance incentive to control
group
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
137
TABLE 3 - SUMMARY OF RESEARCH THREATS TO VALIDITY*
(CONTINUED)
Class Explanation
Internal Validity
Resentful demoralization of respondents
receiving less desirable treatments
Construct Validity
Inadequate preoperational explication
of constructs
Mono-operation bias
Mono-method bias
Hypothesis-guessing within
experimental conditions
Evaluation apprehension
Experimenter expectancies
Confounding constructs and levels
of constructs
Interaction of different treatments
Interaction of testing and treatment
Restricted generalizability across
constructs
Knowledge of receiving less desirable
treatment causes control group to
retaliate by intentionally lowering
performance
Formal conceptual analysis of essential
constructs is lacking
Using a single measure for cause-and-
effect constructs
Using a single means to either deliver
treatment or to record results
Respondents behave according to
anticipated experimenter expectancies
Respondents behave to gain a favorable
personal evaluation
Experimenter treatment expectancies
bias data
Multiple discrete levels of an
independent continuous variable are not
all measured In determination of a
cause-and-effect association
Respondents experience more than one
treatment
Pretest or repeated posttest sensitization
limits generalizability of results
Potentially associated constructs of
effect are not measured
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
138
TABLE 3 - SUMMARY OF RESEARCH THREATS TO VALIDITY*
(CONTINUED)
Class Explanation
Statistical Conclusion Validity
Low statistical power
Violated assumptions of statistical tests
Fishing and error rate problem
Reliability of measures
Reliability of treatment implementation
Random irrelevancies in the experimental
setting
Random heterogeneity of respondents
External Validity
Interaction of selection and treatment
Interaction of setting and treatment
Interaction of history and treatment
Small sample sizes increase chance of
making an incorrect no-difference
conclusion
Certain statistical assumptions not
tested for a given data set
Failure to recalculate significance level
when conducting multiple statistical
tests
Low reliability of measures decreases
opportunity to identify true differences
Unstandardized treatment delivery
decreases opportunity to identify true
differences
Environmental variables other than
treatment may affect results
Random variation among subjects may
influence results
Subject selection factors may limit
generalization
Distinction of setting may limit
generalization
Temporal conditions may limit
generalization
* Adapted from Cook & Campbell (1979). Campbell & Stanley (1963), & Isaac &
Michael (1997).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
139
Primary Validity Concerns
Internal validity is considered the most fundamental validity of research
design, which Campbell & Stanley (1963) describe as, “the basic minimum without
which any experiment is uninterpretable: Did in fact the experimental treatments
make a difference in this specific experimental instance” (p. 5)? Internal validity,
therefore is concerned with the accuracy with which observed group differences can
be attributed solely to the treatment intervention. It is largely determined by the
design of the research, and thus, according to Bland et al. (1995), “each research
design can be given a score that reflects how likely it is to yield results that are
attributable to the variables hypothesized by the researcher” (p. 645). As internal
validity, and hence study design, is considered most fundamental to research, this
category was awarded the highest point value.
The two main threats to internal validity considered in this analysis are
‘selection’ and ‘interactions with selection’. As discussed in chapter 3, selection is
of primary importance, as original PBL groups are comprised of volunteer medical
students. O f the nine available studies, one conducted a randomized control trial,
one utilized a nonequivalent control group design (with pretest), and the other seven
employed static-group comparisons, also referred to as a posttest-only design with
nonequivalent groups. Aside from the first experimental study noted, the other eight
studies did not use randomization and are therefore considered quasi-experimental.
A significant flaw in the static-group comparison study design is the absence of a
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
140
pretest, leading to the possibility that posttest differences may be attributed to
selection differences between the groups as easily as they may be attributed to the
treatment effect. Therefore, as Cook & Campbell (1979) suggest, “interpretability
depends in large measure on how well selection artifacts can be explicitly ruled out
or rendered less plausible” (p. 136). Cook & Campbell (1979) also note that in the
absence of a pretest, and when interaction predictions have not been made to
strengthen inferences about cause, substitution for lack of pretest measures has been
attempted in some research fields (e.g., economics, sociology). This is done by
seeking out pretest measures (e.g., age, sex, race) that correlate with the posttest
within experimental groups but are not measured on the same scale as the posttest.
In an effort to stratify the quality of the static-group design studies further, additional
points were awarded to those studies that attempted to compare initial PBL and CC
groups on variables other than the SDL dependent measure (e.g., age, previous
grades, MCAT, etc.). Similarly, because the actual subsets of PBL and CC study
respondents could also be classified as volunteers, points were awarded for those
studies that reported demographic comparisons between study respondents and/or
non-respondents. Additional points were also allotted for any study using PBL and
CC graduates from the same medical school rather than two or more institutions, as
different governing educational philosophies of individual institutions could be
viewed as an ‘interaction with selection’ internal validity threat. It must be noted,
however, that stratifying individual studies belonging to the overall quasi-
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
141
experimental design category is an artificial creation; it is conducted here only as a
means of identifying distinctions within a small pool of studies.
Cook & Campbell (1979) further suggest that for research primarily
performed for theory testing, construct validity is next in importance, followed by
statistical conclusion validity and lastly by external validity. In PBL/SDL research,
there is a fundamental theory that PBL curricula develop students’ SDL behavior
more effectively than traditional curricula, as was detailed in chapter 2. Isaac &
Michael (1997) define a theory as, “a set of interrelated hypotheses that have been
derived from one or more definitional statements - super hypotheses (axioms,
postulates, or conceptual assumptions that define the major given features of the
theory)” (p. 3). One may propose that the underlying theory on the relationship
between PBL and SDL may be loosely summarized as the following set of
hypotheses:
• Hypothesis 1: A PBL curriculum encourages participation in SDL procedural
skills to a greater extent than a conventional, didactic-based medical education
curriculum.
• Hypothesis 2: When engaged in PBL instruction, medical students will practice
SDL procedural skills more than medical students in a conventional medical
education curriculum.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
142
• Hypothesis 3: The repeated practice of procedural SDL skills will make PBL
medical students more effective procedural self-directed learners than their
conventional counterparts.
• Hypothesis 4: The results of the use of SDL procedural learning skills will
motivate PBL students to demonstrate SDL outcome behavior more than their
conventional counterparts.
• Hypothesis 5: The application of SDL outcome behavior will transfer when PBL
students are removed from a PBL learning environment, and will cause them to
engage in more effective and more frequent SDL as practicing clinicians than their
conventional counterparts.
The undergraduate medical education literature is inconclusive regarding
these first five hypotheses, largely due to methodological limitations (see chapter 3
for discussion). Additionally, as presented above, one may argue the veracity of any
individual hypothesis, but in particular hypothesis 4 - that procedural SDL skills
necessarily increase one’s motivation to achieve SDL goal behavior - a premise that
will be further discussed later in this paper; however, the current point is that a
semblance of this theory nevertheless serves as the framework for the proposed
cause and effect association between PBL and SDL. Cook & Campbell (1979)
suggest that for construct validity of putative causes and effects, a researcher must
critically: 1) Reason out how a particular construct should be defined; 2) distinguish
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
143
the construct from other cognate constructs; 3) decide which measures or
manipulations to use to index the particular construct; and 4) utilize multiple
measures or manipulations wherever possible. Since SDL in PBL research is
primarily concerned with theory testing, rating scale points were allotted according
to sequential validity importance as stated by Cook & Campbell (1979) (construct
validity- statistical conclusion validity- external validity). For this literature
synthesis, therefore, points were awarded to studies based on the extent to which the
constructs of PBL and SDL were clearly defined; secondly, whether a PBL/SDL
associative model or theory was described; thirdly, on whether SDL was
operationalized as separate process and product measures and how closely these
measures correlated with those suggested in the SDL model proposed in Figure 1;
and lastly, on whether multiple measures of SDL were utilized as dependent
measures.
In the area of statistical conclusion validity, the threats considered most
relevant were statistical power and instrument reliability. Scoring for statistical
power was based on power table values (Gall, Borg, & Gall, 1996) and utilized
values given for an independent samples t-test with alpha = .05, a corresponding
medium effect size of 0.5 (equivalent to the difference between the 50th and 69th
percentiles in a normal distribution [Schmidt & Hunter, 1995]), and a statistical
power of .7 (N = 100) or .5 (N = 64) as general guideline parameters. The inclusion
of a rating category based on whether or not studies provided adequate statistical
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
144
reporting to calculate an effect size was initially considered, but subsequently
dismissed due to the complicated nature of this issue. Consultation with a
biostatistician revealed that for some studies reporting data other than means and
standard deviations, the calculation of an effect size was possible, but complicated
and not practical; in these cases limited study data would have required extensive
manipulations that necessitated making unknown data assumptions and may have
yielded effect sizes that contained significant error. The issue of effect size will be
further addressed in the ‘Discussion’ section. Lastly, scoring of reliability was based
on the degree to which reliability (or validity) indices were reported for results
obtained with a particular study measurement tool (i.e., questionnaire); scoring
consideration was also given to studies that attempted to increase reliability through
data triangulation.
Finally, external validity was briefly considered in the coding process. Points
were awarded based on whether demographics and/or institutional descriptors (e.g.,
educational mission, curriculum characteristics) were reported for the medical
schools from which the study populations were drawn.
Non-statistical Meta-analysis Results
Results of this non-statistical meta-analysis are briefly presented here and
will be further considered in the ‘Discussion’ section to follow. The research
populations and institutional programs utilized by each of the studies are listed in
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
145
Table 4. Somewhat surprisingly, 4/9 studies utilized physician populations who
were in most cases well beyond residency training. Eight studies compared
populations from undergraduate PBL and CC curricula, while the remaining study
compared PBL versus CC instruction within its own residency program. Two of the
studies resembled program evaluations as they were designed to measure residents5
attitudes regarding curricular preparedness for postgraduate training.
The range of overall scores for these studies was 39 - 76 out of a possible
127 points, with a mean rating of 57.7 points. Ratings for each individual study can
be found in Appendix A and a summary of results is presented in Table 5. Review
of this table indicates that certain trends can be identified. Overall strengths of this
study set included the following: 1) 8 of 9 studies compared study respondents from
non-respondents on > 2 demographic variables in consideration of response bias; 2)
6 of 9 studies included a sample size (> 98 subjects) with sufficient statistical power
to reject a false null hypothesis, i.e., reducing a Type II error; 3) 6 of 9 studies also
pretested their instrument before use; 4) 5 of the 9 studies operationalized SDL as
either a process or product behavior; and 5) these same 5 studies operationalized
SDL as > 3 measures. Limitations discovered in this literature synthesis included the
following: 1) A PBL/SDL theory or model was absent in every study; 2) of the 7
studies using a posttest-only design with nonequivalent groups, only 1 used pretest
substitute measures; 3) 6 of the 9 studies provided either a cursory conceptual
definition or no conceptual definition for both PBL and SDL; 4) 4 of the 9 studies
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
146
TABLE 4 - RESEARCH POPULATIONS AND INSTITUTIONAL PROGRAMS
UTILIZED BY STUDIES REVIEWED FOR NON-STATISTICAL
META-ANALYSIS
Study
Ozuah et al.
2001
Research Population
Yr. 1-3 Residents
Institutional Program(s)
Comparison
None
Peters et al. Post-residents
2000 8-9 yrs. post-graduation
Santos-Gomez Yr. 1-3 Residents
et al. 1990
Harvard
Other
Residency program
conducted curricular
intervention (study
conducted at
Children’s Hospital,
Bronx, N.Y.)
New Mexico 1985-1987 graduates
Shin et al. Post-residents
1993 5-16 yrs. post-graduation
Mennin et al. Post-residents
1996 4-7 yrs. post-graduation
Tolnai Post-residents
1991 9-15 yrs. post-graduation
Mann & 2n d yr. Residents
Kaufman 1999
Hill et al. Yr. 1 Interns
1998
McMasters - PBL
U of Toronto - CC
New Mexico 1983-1986 graduates
(some pop. overlap
with prior NM 1990
study)
McMasters - PBL
U of Ottawa - CC
Dalhousie Univ. Program evaluation
Halifax, Nova Scotia design
Newcastle - PBL Program evaluation
U of Sydney/U of design
New South Wales - CC
Rolfe et al. Yr. 1 Interns Newcastle - PBL
1995 U of Sydney / U of
New South Wales - CC
(also included international grads.)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
147
TABLE 5 - NON-STATISTICAL META-ANALYSIS RESULTS
Rating Index No. of studies possessing rating index
Research Design
Experimental I
Quasi-experimental 8
Utilized pretest design 1
Did not utilize pretest design 7
Used pretest substitute measures 1
Population drawn from single institution 3
Study Group Comparison
> 3 demographic variables 5
2 demographic variables 3
1 demographic variable/not reported 1
PBL/SDL Conceptual Definition
PBL and SDL both adequately/partially defined 1
PBL only adequately/partially defined 1
SDL only adequately/partially defined 1
PBL and SDL given cursory definition/no definition 6
PBL and SDL Theory or Model Base
Theory or model extensively/partially presented 0
SDL Operationalization
Operationalized as both process and product 1
Operationalized as a process only 0
Operationalized as a product only 4
SDL not adequately operationalized 4
SDL operationalized as > 3 measures 5
Statistical Power/Sample Size
Studies using > 98 subjects 6
Studies using > 64 and < 97 subjects 3
Studies using < 64 subjects 0
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
TABLE 5 - NON-STATISTICAL META-ANALYSIS RESULTS
(CONTINUED)
Rating Index No. of studies possessing rating index
Instrument Reliability/Validity Measures
Explicit reliability or validity measures reported 3
Instrument reliability or validity measures suggested 1
Instrument pretested/field tested 6
Instrument pretested/field tested only 4
No instrument reliability or validity indices discussed 1
Data triangulation attempted 1
(Self-report, nurse, doctor-supervisor)
Self-report only 7
Supervisor only 1
Institution Demographics/Descriptors
> 3 institutional demographic or descriptors reported 1
2 institutional demographic or descriptors reported 3
0-1 institutional demographic or descriptor reported 5
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
149
did not adequately operationalize SDL, while only one study operationalized SDL
using both process and product measures; 5) only 3 of the 9 studies provided explicit
reliability or validity measures for their instrument; and 6) 5 of 9 studies provided
little to no institutional demographic data.
A further look at SDL operationalization is presented in Table 6, which lists
all SDL dependent measures utilized in these studies. In total, 34 different measures
of SDL were used, discounting 3 non-operationalized SDL measures (‘independent
learning’; ‘ability as a self-directed learner’ and ‘self-directed learning’). Some
overlap existed between studies and the operations most frequently cited included:
1) Reading/studying of medical journals and texts (5); 2) engaging in medical
discussions/consultations with colleagues (4); 3-4) performing literature
searches/engaging in computer use; keeping up-to-date (3 each); 5) CME activities
(2). Nineteen of the 34 items were found to be distinctively individual measures of
SDL, and of the total 37 measures, 26 were directed toward SDL behavior, while 11
were designed to measure SDL attitude.
A summary of the individual study findings is presented in Table 7. This
summary indicates that 3 of the 9 studies reviewed reported significant differences
between the two groups on measures of SDL in favor of PBL graduates. One study
found a significant difference in favor of CC graduates on 1/14 SDL measures.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
150
TABLE 6 - COMPLETE LISTING OF SDL OPERATIONALIZED MEASURES
USED IN STUDIES REVIEWED FOR NON-STATISTICAL META
ANALYSIS
* denotes 0-10 scaled questionnaire item (0 = complete disagreement/lack of
confidence; 10 = total agreement/confidence)
c p denotes 1-5 scaled evaluation rating form (no descriptors provided)
** denotes 1-7 scaled questionnaire item (1 = rarely: 7 - frequently)
© denotes 1-7 scaled questionnaire item (1 = unsatisfactory; 7 = excellent)
N denotes 1-6 scaled questionnaire item (1 = very inadequately; 6 = very adequately)
♦ denotes 1-7 scaled evaluation rating form (1 = unsatisfactory; 7 = outstanding)
(A) denotes SDL attitude measure
Study 1 - Ozuah et al., 2001
1) average time/week over the past 1 month spent in independent study of medical
text and journals
2) average time/week over the past 1 month spent in medical discussions with
colleagues, outside of scheduled didactics and teaching rounds
3) average time/week over the past 1 month spent in performing computer literature
searches
Study 2 - Peters et al., 2000
4) Were you a chief resident?
5) Did you take any additional training beyond residency, including fellowships or
other advanced degrees?
*6) Compared with other doctors in my medical specialty, I frequently look up
information in textbooks and journals.
7) In a typical week, how many hours do you spend reading journal articles?
8) In a typical month, how many times do you find medical information through
electronic sources?
*9) In making medical decisions, I like to consult with other physicians
frequently. (A)
*10) I find it useful to reach decisions through discussion with non-physician health
professionals. (A)
*11) I will be able to stay up-to-date with developments in medicine outside my
field. (A)
*12) Within the scope of your practice, how confident do you feel in your ability to
interpret the scientific strength of clinical research evidence? (A)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
151
TABLE 6 - COMPLETE LISTING OF SDL OPERATIONALIZED MEASURES
USED IN STUDIES REVIEWED FOR NON-STATISTICAL META
ANALYSIS (CONTINUED)
Study 3 - Santos-Gomez et al., 1990
(pi3) independent learning
Study 4 - Shin et al., 1993
14) mean score on a 52 multiple-choice questionnaire in five areas of current
knowledge of the hypertension management of hypertension
Study 5 - Mennin et a l, 1996
**15) topics studied: basic science, clinical medicine, community health, health care
organization; techniques/skills, office management
**16) preferred educational methods: journals, books, patient files, computer,
continuing medical education courses, collegial network, videotape, audiotape
**17) motivation for education: concern with patient problems, keep up with
colleagues, to teach, be up to date, interest, curiosity
Study 6 - Tolnai, 1991
18) hours/week spent reading journals and books
19) hours/week spent using audiovisual materials
20) hours/week attending hospital rounds
21) frequency of participation as CME course coordinator
22) frequency of participation in CME clinical days
23) frequency of participation in computer use
24) frequency of participation in correspondence course
25) frequency of participation in journal club, seminars
26) frequency of participation in research
27) frequency of participation in self-evaluation programmes
28) frequency of participation in study leave
29) frequency of participation in MSc & residency programmes [MSc = Master of
Science]
Study 7 - Mann & Kaufman, 1999
[rate of preparation for...]
® 30) ability as a self-directed learner (A)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
152
TABLE 6 - COMPLETE LISTING OF SDL OPERATIONALIZED MEASURES
USED IN STUDIES REVIEWED FOR NON-STATISTICAL META
ANALYSIS (CONTINUED)
Study 8 - Hill et al, 1998
[my undergraduate medical training prepared me to ...]
N 31) take responsibility for my own learning (A)
N32) continually evaluate my own performance (A)
N33) evaluate my educational experience (A)
N34) invest time in developing my skills (A)
N35) identify my own educational needs (A)
N36) keep up to date with medicine (A)
Study 9 - Rolfe et al, 1995
#37) self-directed learning
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
153
TABLE 7 - FINDINGS REPORTED BY STUDIES REVIEWED FOR NON-
STATISTICAL META-ANALYSIS
Study
Ozuah et al.
2001
Population Comparison
Yr. 1-3 pediatric
residents
(undergrad, institutions
unknown)
Study Design
Interrupted time series
SDL self-report
questionnaire
PBL vs. CC instruction
given to resident groups
Peters et al.
2000
Harvard PBL/CC MDs Telephone
8-9 yrs. post-grad. interviews
(original groups randomized)
Santos-Gomez
et al. 1990
Yr. 1-3 New Mexico
PBL/CC residents
Shin et al.
1993
Mennin et al.
1996
PBL - McMasters U
CC - U of Toronto
5-16 yrs. post-grad.
New Mexico PBL/CC MDs
4-7 yrs. post-grad.
Tolnai
1991
PBL -McMasters U
C C - U of Ottawa
9-15 yrs. post-grad.
Mann &
Kaufman 1999
Year 2 Dalhousie
PBL/CC residents
Key Findings
PBL group with
sig. higher SDL
during exposure;
no sig. diff. at
pre-exposure or at
3 mo. post-exposure
No sig. diff.
between groups on
9 measures of
‘lifelong learning’
No sig. diff.
between groups
on 1 measure of
‘independent learning’
Sig. mean diff. on
overall score in
favor of PBL grads.
(p<0.01)
No sig. diff.
between groups on
3 categories of SDL
after Bonferroni adjust.
Sig. diff. on 1/14
SDL measures
(‘attending hospital
rounds’) in favor of
CC grads (p < 0.02)
Self-report No sig. diff.
questionnaire between groups on 1
(re: perceptions ‘ability as a self-directed
of curr. preparation) learner’ measure
Supervisor/nurse
self-report
questionnaire
52 MCQ self-report
questionnaire on
hypertension
management
Self-report
questionnaire
Self-report
questionnaire
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
154
TABLE 7 - FINDINGS REPORTED BY STUDIES REVIEWED FOR NON-
STATISTICAL META-ANALYSIS (CONTINUED)
Study Population Comparison Study Design Key Findings
Hill et al. PBL - U of Newcastle, Australia Self-report Sig. mean diff. on
1998 CC - U of Sydney/U of New questionnaire 6-item SDL subscale
South Wales; all Yr. 1 residents (re: perceptions of in favor of PBL
curr. preparation) grads (p < 0.01)
adjusted for age &
gender
Sig. mean diff. on 1
measure of ‘self
directed learning’ in
favor of PBL grads
95% Cl stated; no
p-value given
adjusted for age &
gender
Rolfe et al. PBL - U ofNewcastle, Australia Clinical
1995 CC — U of Sydney /U of New supervisor
South Wales/International grads, evaluation
all Yr. 1 residents
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
155
Discussion
O f the nine studies reviewed, eight were conducted by researchers who were
faculty members from the same medical school as (one of) their respective study
populations. Some of these authors also published studies included within the
chapter 3 literature review. This is mentioned simply to point out that there appears
to be a good deal of integration between the undergraduate and the post-graduate
literature in this field. As one may expect, many of the student populations studied
in undergraduate research later become the physician populations utilized in post
graduate research. Although these nine studies represent only a very small portion of
the SDL/PBL post-graduate literature, it is not surprising then that the research
limitations identified by this literature synthesis are similar to those discussed in
chapter 3. Comparable problem areas, particularly in the areas of SDL conceptual/
operational definition and research design are a further indication that the difficulty
in understanding SDL is not at the level of an individual researcher, an
individual institution, or even an educational level, but throughout the overall field of
SDL within medical education research. However, this literature synthesis also
identified some positive developments that can serve as a framework for continuing
SDL research. A further look at some of the SDL strengths and limitations identified
in this analysis follows below.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
156
SDL Conceptual and Operational Definitions
In comparing the proposed SDL model measures with the list of operations
used in the current study set, 4 of the 5 most frequently mentioned study measures
had been correspondingly designated as SDL model operations (reading/studying of
medical journals and texts; engaging in medical discussions/consultations with
colleagues; performing literature searches/engaging in computer use; CME
activities). The fifth study measure, ‘keeping up-to-date’, is viewed here as
problematic in that it may be widely interpreted, with one interpretation simply being
that ‘keeping up-to-date’ is, by virtue of engaging in designated SDL activities,
occurring in the very process and hence does not constitute a separate measure. In
the one study that operationalized SDL as a process, these measures also
corresponded closely with the measures proposed in the model. The five studies
that adequately operationalized SDL also used multiple measures (> 3) as
recommended by Cook & Campbell (1979), suggesting that many researchers are
aware of the mono-operation bias inherent in using only one measure of a construct.
Although the proposed model measures were not utilized in all nine studies, it
is encouraging that post-graduate research may have enough of an SDL operational
common ground from which to shape the future scope of SDL research. However,
there were still 4/9 studies that were rated as having inadequate SDL
operationalization. The study that used the results of a hypertension management
evaluation (Shin et al., 1993) as the sole SDL measure was one of these; however,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
157
one may argue that knowing the latest practice guidelines for a specific treatment -
particularly if those guidelines have changed since a physicians’ medical education -
may indeed be an reasonable SDL measure as knowing those guidelines may be a
result o f ‘reading journals/texts’, ‘performing literature searches’, or ‘CME
participation’. Contrastingly, the remaining three studies simply measured ‘self
directed learning’ or ‘independent learning’ without providing further definition.
This creates the obvious problem of using a study measure with unlimited definition
interpretations.
The positive trend initially identified for SDL operationalization
unfortunately did not extend to the area of SDL/PBL conceptualization, as 6/9
studies provided little or no definition of these concepts and only one study offered a
(partial) definition for both (Shin et a l, 1993). This problem was compounded by
the fact that there was a unanimous absence of an SDL theory or model in these
studies. Establishing and structuring conceptual and operational definitions of
constructs, theorizing on their potential associations, and developing means to test
these theories form the cornerstone of any research field. The field of PBL/SDL
research needs to collectively complete this first step of becoming grounded in its
definitions in order to evolve. Medical educators continue to hypothesize that there
is a strong correlation between PBL and SDL, but to date there appears to be
minimal consensus in defining and measuring these concepts, and no substantial
theory or model on which to ground this research. It is in this area where
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
158
undergraduate and graduate medical education face their most immediate and critical
challenge. To this end, it is imperative that research articles provide clear conceptual
and operational definitions of SDL and provide a concise narrative account or a table
that lists the key features of both a particular institution’s method of PBL instruction
as well as how SDL is incorporated within that instruction. It is further
recommended that researchers use a very minimum of three SDL measures in their
studies, preferably more (> 5). While it is best to use both process and product
measures, post-graduate research should emphasize product measures as it is this
behavior with which medical education is most ultimately concerned. Including
these steps will greatly improve the construct validity in this field.
Correspondingly, the theory by which PBL is proposed to cause SDL
requires fuller elucidation. It is in theory development where the distinction between
SDL process and product becomes particularly critical. A theory hypothesizing that
PBL students become more effective self-directed procedural learners than CC
students is not difficult for most critics to accept; however, a theory suggesting that
these skills will then transfer into self-directed procedural learners: a) demonstrating
the internally motivated, self-initiated behavior representative of SDL outcome; and
b) demonstrating this behavior far more effectively than CC students over the course
of their careers, is where the theory is weak. Researching SDL behavior in post
graduate education is crucial for this very reason. This link between SDL process
transfer into SDL product lies at the foundation of PBL/SDL theory and yet it simply
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
159
has not been established. The finding that only 3/9 studies reviewed for this analysis
reported significant findings in favor of PBL graduates, with one of those being a
self-report measure on curriculum preparation effectiveness, illustrates this point.
The suggestion exists that providing authentic, integrated patient cases to
undergraduate medical students and leading them through stimulating problem
solving situations increases their motivation for learning. Indeed, this is the fourth
PBL educational objective. While some studies suggest PBL students find this
instruction more enjoyable and satisfying than a traditional curriculum (Albanese &
Mitchell, 1993; Bernstein, Tipping, Bercovitz, & Skinner, 1995; Kaufman & Mann,
1996), others argue that the effects of PBL on student motivation are among the least
well documented outcomes of the curriculum (Berkson, 1993), and that no direct
evidence exists as to how specific aspects of PBL problems stimulate learning
applicable to patient care (Thomas, 1997). Even if PBL students may indeed exhibit
increased motivation for learning during the time spent in this curriculum, PBL/SDL
learning theory does not explain how or why that motivation is presumed to transfer,
for example, to a PBL graduate being more likely than a CC graduate to engage in a
literature search on the newest available diabetes medications ten years down the
road. The (intrinsic) motivational component of SDL has simply not been identified
or well researched in the literature currently reviewed for chapters 3 and 4.
Unfortunately, in this present literature synthesis only 1 of the 9 studies included any
kind of question concerning motivation for SDL (Mennin, Kalishman, Friedman,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
160
Pathak, & Snyder, 1996). Understanding the source of motivation underlying
physicians’ learning is a crucial aspect of developing SDL behavior, and this is
further addressed in chapter 5.
Research Design
Ozuah et al. (2001) is the single study in this review that utilized an
experimental research design with a pretest (nonequivalent control group design).
Residents were divided into PBL and CC groups, and self-report questionnaires were
administered at baseline (pre-exposure), during exposure, and at three months post
exposure. Although this study did not find significant SDL behavioral differences
between the groups at follow-up, the inclusion of a pretest (which identified no SDL
significant differences between the groups at baseline) is considered critical as it
minimizes the selection threat to internal validity, which, as noted, is considered a
significant threat to PBL/SDL research due to the volunteer effect.
Peters et al. (2000) is the only one of the remaining eight quasi-experimental
studies that mitigated the threat to selection, in this case by utilizing random
populations. The original 125/280 entering Harvard medical students in 1985 and
1986 who agreed to be randomized into the PBL (63 students) or CC curriculum (62
students) were matched for race and gender. Since the current study was limited
solely to these students, the threat to selection was minimized as using this
homogenous population negated (or equalized) any experimental volunteer bias.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
161
The lack of pretest employed in the remaining seven static group design
studies is more of an undergraduate level issue as SDL pretesting needs to be
administered to research populations prior to the commencement of an educational
intervention; at the post-graduate level researchers then must resort to using pretest
substitute measures in an attempt to minimize selection threat. Only Mann &
Kaufman (1999) included pretest substitute measures in their study, reporting that
the PBL and CC groups from Dalhousie University were statistically similar on all
admission characteristics. The authors note that these groups were also compared in
earlier studies, including a “perceived self-directed learning readiness” measure (see
chapter 3, Mann & Kaufman, 1995). Unfortunately, due to less than 100% response
rates for both studies, it is unknown whether the current post-graduate respondents
all had associated SDL readiness data available; otherwise, this would have
constituted a pretest research design. Nevertheless, these authors were diligent in
their comparisons of the two groups.
Three of the eight static-group design studies utilized populations from a
single institution. Although it was previously mentioned that this feature aids in
reducing potential interaction with selection threats, post-graduate research
realistically cannot be limited to those medical schools that either happen to be
undergoing curriculum revision or possess dual PBL and CC curricular tracks. This
rating category was assigned minimal point value to reflect this limitation. However,
this aside, researchers are encouraged to provide further institutional demographic or
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
162
descriptive data than did most in this review. Briefly describing features such as
class size and demographic composition, curricular program emphasis (e.g., organ
system versus discipline-based, community health emphasis), etc. provides
preliminary external validity criteria.
Further recommendations for improving research design are straightforward.
Although more difficult to implement, experimental designs reduce more threats to
internal validity than quasi-experimental designs and should be considered if at all
possible. Those researchers affiliated with institutions that possess both PBL and CC
curricula are encouraged to implement SDL pretesting shortly after admission if
there is potential opportunity to follow these students as post-graduates. The
creation of new SDL instruments is greatly needed, however, it was noted in chapter
3 that the SDLRS may provide one pretest option, and if compared with subsequent
SDL assessment measures, would serve as an opportunity for predictive validity
testing. Although again not often feasible, the randomization of students into PBL
and CC groups, as was done at Harvard, in combination with pretesting would
further strengthen SDL research. Where pretesting is not available, substitute
measures are highly encouraged and should be fairly accessible to researchers using
their own post-graduate populations, as admission offices or medical education
departments are typically replete with detailed statistics on entering students. If
legally permissible, those researchers utilizing unfamiliar populations may consider
contacting the home institution for these records.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
163
Data Reporting
Associated with research design is the issue of data reporting. Further
potential selection effect may be present if differences exist either between the two
respondent study groups or between study respondents and non-respondents
(response bias). Five of the nine studies included some degree of reporting in this
area, and thus were conscientious of this fact; however, it was often unclear how
groups were compared, for example, whether PBL and CC respondents were each
compared against their same group non-respondents, or whether total respondents,
regardless of group membership, were compared with total non-respondents. For
ease of rating, studies were awarded points if they simply reported any comparison
group data. Depending on how one conducts the comparison, however, finding no
difference between respondents and non-respondents could still yield two
considerably different - and potentially biased - study groups. If both 50 total
respondents and non-respondents were women, these two study groups overall would
be equivalent, yet what may be unreported is that 38/50 female respondents were
from the PBL group and only 12/50 were from the CC group. Conversely, if we are
told that 20 women from both the PBL and the CC group responded, then the
respondent groups would appear equivalent, but in this case there may have been 20
total women in the PBL group and 50 total women in the CC group. There may be
many possible interpretations to comparison group data, therefore the proposed
recommendation is for studies to report: 1) Comparison data for within- group
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
164
respondents and non-respondents, included with the original PBL and CC group
sample numbers for the variables being compared; and 2) comparison data for PBL
versus CC respondents overall.
A related recommendation concerns statistical data reporting. Some
researchers (Hedges, 1992; Michalczyk & Lewis, 1980; Schmidt & Hunter, 1995)
promote the reporting of effect sizes over significance values in individual studies, as
nonparametric tests are unable to provide estimates of the magnitude of the effects
being considered (Hedges, 1992). Most studies do not report effect size, however,
and thus it becomes important for investigators to provide appropriate statistical data
to allow interested researchers to perform their own calculations. As noted earlier,
determination of effect size would have been problematic for some of the present
studies due to incomplete data reporting. Table 8 presents a summary of the
statistical information provided by each of the studies. Minium, King & Bear (1993)
report that the 1983 Publication Manual of the American Psychological Association
(APA), which is frequently used in education, specifies that authors must provide the
statistic symbol (such as t), degrees of freedom, the obtained value of the statistic,
and the p-value in their research. The most straightforward determination of
experimental effect size according to Glass, McGaw & Smith (1981) is measured
using the mean difference between the two study groups divided by within-group
standard deviation (or, in some cases, the control group standard deviation). In a
review of 33 articles randomly selected from the Journal o f Medical Education,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
165
TABLE 8 - SDL DATA REPORTING IN STUDIES REVIEWED FOR NON-
STATISTICAL META-ANALYSIS
Study
Ozuafa et al.
2001
Peters et al.
2000
Statistical Test
Chi-square/Fisher exact
test
Analysis of variance
Santos-Gomez et al. Fisher exact test
1990
Data Reported
Absolute number of residents &
percentages on each measure;
p- value where significant <0.01
Mean scores on 0-10 Likert
scale for each measure; ‘n.s’ for
p-value
n value & percentages of
residents ranked on 1-5 Likert
scale for ‘independent
leaming’measure; specific p-
value stated
Shin et al.
1993
t-test Mean exam scores and 95%
confidence intervals; p-value
where significant either < 0.05
or < 0.01
Mennin et al. 2-way analysis of variance; Mean scores on 1 -7 Likert scale
1996
Tolnai
1991
Newman-Keuls multiple comparisons for each SDL measure; specific
test; significance levels adjusted p-value stated
by Bonferroni methods
t-test
Mann & Kaufman
1999
t-test
Mean scores in hours/week for
each measure & standard
deviations; p-value stated for 1
significant finding only (p <
0.02)
Mean scores and standard
deviations on 1-7 Likert scale
for ‘ability as a self-directed
learner’ measure; specific t-
score and p-value stated for
overall subscale (6 items)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
TABLE 8 -
Study
Hill et al.
1998
Rolfe et al.
1995
166
SDL DATA REPORTING IN STUDIES REVIEWED FOR NON-
STATISTICAL META-ANALYSIS (CONTINUED)
Statistical Test Data Reported
Analysis of covariance Mean scores and 95%
confidence intervals on 1-6
Likert scale for 6-item SDL
subscale measure; p-value <
0.01
Analysis of covariance Mean scores and 95%
confidence levels on 1-7 Likert
scale for 1 ‘self-directed
learning’ measure; finding
called significant, but no
p-value stated
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
167
Michalczyk & Lewis (1980) reported that almost half of the studies did not report a
sufficient amount of data to calculate an effect size. In addition to the above APA
guidelines, it is strongly recommended that studies also report all means and standard
deviations, thus allowing for effect size calculation in the clearest manner. It is
further recommended that researchers estimate the anticipated effect size prior to
conducting their actual research to determine appropriate sample size, as suggested
by Michalczyk & Lewis (1980); investigators may find the consultation of a power
curve chart as discussed in Minium, King & Bear (1993) helpful in this process. It is
highly encouraging that 6/9 studies utilized sample sizes with adequate power (0.7)
to detect a true difference, while the remaining three included sample sizes
acceptable for a lesser 0.5 power level.
Instrumentation
Issues In the area of instrumentation concern data triangulation, instrument
reliability/validity, and the measurement of SDL behaviors versus attitudes.
Questionnaires were the sole form of instrumentation used in these studies, with 7/9
using self-reporting measures only, one using a clinical supervisor evaluation only,
while the remaining study attempted to triangulate data by obtaining supervisor,
nurse and self-reports. This last study (Santos-Gomez, Kalishman, Rezler, Skipper,
& Mennin, 1990) evaluated residents on eight different dimensions, including
‘independent learning’. Interestingly, the highest correlation between independent
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
168
learning and one of the other seven dimensions evaluated was different for each of
the three raters - for supervisors it was ‘critical thinking’ (r = 0.69), nurses’ highest
correlation was with ‘knowledge’ (r = .66), and for residents themselves it was
‘communication with patients’ (r = 0.52). This perhaps illustrates the different
perceptions that may exist between raters when the construct of ‘independent
learning’ is not defined. However, it is noteworthy that this study obtained multiple
rater evaluations, and as discussed in chapter 3, data triangulation is encouraged to
offer additional data in answering a research question. Chapter 3 also discussed the
need for research to create more objective SDL measures overall. The assessment of
SDL process at the undergraduate level was previously addressed, and while
objective procedural skill assessment may also have some value at the post-graduate
level, the larger difficulty for future SDL research lies in the task of creating
objective measures of SDL outcome behavior to which self-reports can be compared.
When constructs are not unanimously conceptualized or operationalized, as is
the case with SDL research, establishing the reliability and/or validity of study
results has great potential utility in helping to shape a clearer understanding of the
construct. Pretesting an instrument is always recommended and was performed in
6/9 current studies. However, only 3/9 studies reported explicit reliability or validity
measures for the results obtained with their instruments. Two studies reported alpha
coefficients, one for a 6-item SDL subscale (Hill, Rolfe, Pearson, & Heathcote,
1998), and the other for a 6-item learning subscale that included one specific SDL
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
169
question (Mann & Kaufman, 1999). Shin et al. (1993) discussed previous validation
of their hypertension management questionnaire by reporting that three prior studies
found significant negative coefficients between the questionnaire scores and time
since graduation. As studies are encouraged to preferably use > 5 measures to
operationalize SDL, it is recommended that researchers provide an alpha coefficient
as a report of the internal consistency of their instruments. As stated earlier,
researchers may also consider establishing predictive validity for an SDL instrument
if undergraduate SDL assessment measures are also available. Improving instrument
reliability will also increase the power of a study by decreasing standard deviation
(Minium, King & Bear, 1993).
Six of the nine studies measured SDL as a behavior, two measured it as an
attitude (e.g., program evaluation studies), and the remaining study measured SDL as
both. Measuring SDL behavior and attitude are both encouraged. Behavioral
measures should be given particular priority, however, as physicians’ actual SDL
participation is the most essential goal of SDL. Measuring physicians’ SDL attitudes
is also encouraged, however, as attitude may be associated with the issue of self-
efficacy, or self-concept as a self-directed learner. Candy (1991) and Kreber (1998)
addressed the value of exploring this association and evidence of a positive
relationship between SDLRS scores and self-concept was previously noted
(Sabbaghian, 1980).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
170
Included within this recommendation for researchers to measure both SDL
behavior and attitude is the previous recommendation as stated in chapter 3 for
authors either to include a hard copy of their instrument with the study, or to make a
copy available electronically, either via a web site or an electronic mail request.
Two studies provided a copy of their instrument as part of their journal article and
one of two study authors provided a copy as a result of an electronic mail request.
Chapter Summary
Chapter 4 reviewed the research comparing SDL in PBL and CC post
graduate physicians, primarily at a resident training level. Justification for
conducting a non-statistical meta-analysis over a traditional meta-analysis was
presented and the design on which this analysis was based as originally developed by
Bland et al. (1995) was described. An SDL model was initially proposed, the
development of a study rating form was discussed and the rationale for the inclusion
of each category based on different threats to validity was provided. Results from
the analysis of nine studies were presented and subsequent discussion focused on
strengths and limitations primarily identified in the areas of SDL conceptual/
operational definition, research design, data reporting and instrumentation.
The research reviewed for this chapter’s non-statistical meta-analysis
comprised a very narrow and specific portion of the CME literature. However,
expanding analysis of this literature beyond a PBL and CC post-graduate comparison
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
171
context may offer a broader perspective on physician SDL from which enhanced
understanding of the subject may emerge. Therefore, chapter 5 will focus on
describing germane features of physician learning in the clinical environment as
discussed in the CME literature. By probing such issues as how physicians are likely
to leam in practice, or what motivates learning or change in practice behavior, it is
hoped that additionally relevant SDL characteristics as yet undiscovered in this
paper’s literature review may come to light and provide utility in the consideration of
SDL development at the undergraduate level.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
172
CHAPTER 5
PHYSICIAN SELF-DIRECTED LEARNING IN CLINICAL PRACTICE
This chapter presents literature from the field of continuing medical
education in an effort to better typify the SDL behavior of practicing physicians.
Characterizing SDL in this setting more clearly may suggest further implications for
SDL development at the undergraduate medical education level. Three models of
physician learning are presented, two of which follow from qualitative research
conducted on physician learning. Supplementary CME research is then presented in
which additional aspects of physician SDL are discussed, including physicians’ use
of the medical literature, and their observed learning behavior during actual patient
practice. The chapter concludes with a discussion on the motivational factors
associated with SDL.
The CME literature is reviewed here to locate additional theory and research
that may enhance understanding of the learning in which physicians continually
engage throughout their careers. Exploring those elements that shape physician
learning may generate ideas or methods that, if implemented at the undergraduate
level, would assist in developing more effective self-directed learners. If
examination of the CME literature identifies common approaches or themes that
clearly effect or facilitate physician learning, then medical educators can utilize this
knowledge to reinforce similar learning strategies that will promote SDL in medical
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
173
students. The more closely undergraduate training in SDL reflects the authentic
learning environment of clinical practice, the better prepared medical school
graduates will be for the future SDL challenges that await them.
Models of Learning in Clinical Practice
Review of the CME literature reveals a limited number of physician SDL
models that generated recurrent citations by different authors in the field. A few of
these models are initially discussed here to provide a closer analysis of physician
learning within a clinical context and to compare the extent to which agreement
concerning physician SDL is shared. Consideration of supplemental CME research
findings will follow this presentation. Where appropriate, implications these
findings may have for undergraduate SDL training will be initially raised, and later
incorporated within a subsequent closing overview of SDL.
Schon’s Cycle of Learning
The first model of physician learning widely cited in the literature is an
iterative five-stage theoretical model developed by Schon (1983, 1987). The fust
stage in the process of physician learning is ‘knowing-in-action ’ which Schon (1983)
uses to describe the extensive base of knowledge and skills that physicians apply in
day-to-day practice. This working knowledge is often tacit and unconscious,
constituting what some authors refer to as automated knowledge; whether a
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
174
physician is aware or unaware of having learned this practical information, he is
usually unable to articulate the knowledge used in his decision-making. Schon
(1983) warns that this automation carries a danger of a physician becoming so
repetitive and routine that he may miss opportunities to actively think about what he
is doing, which may lead to patterns of error. If a physician becomes selectively
inattentive to phenomena that do not fit with his ‘ knowing-in-action ’, he may suffer
from boredom or “bum-out” that may then affect his patients. When this happens
Schon (1983) suggests that a practitioner has “overleamed what he knows.”
When intuitive performance does not yield the expected results, the second
stage of learning, an encounter with a “surprise”, is experienced, which serves as a
triggering event to begin a reflective cycle. Physicians then enter the third learning
stage, referred to as ‘reflecting-in action’. The physician now intentionally makes
conscious the relevant premises that previously had been automated; he reflects on
this information, critiques and restructures it, and reframes this knowledge to decide
what further action to take in his attempt to solve the puzzling situation. Reflection
focuses interactively on the implicit knowledge, the course of action itself, and the
anticipated action outcome. The physician considers previous similar examples in
his repertoire of experiences to contemplate the resemblance and differences in the
case at hand. This reflection-in-action may or may not be rapid, and is said to be
bounded by the “action-present,” the zone of time in which an action can still make a
difference. Although the action-present may extend over minutes, hours, days,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
175
weeks or months, depending on situational boundaries, this period is commonly
typified as the time limited to the patient visit. Emerging from reflection-in-action is
stage four, identified as the ‘experiment stage’, in which a physician considers
alternative approaches to solving the problem. He constructs a new understanding of
his usual automated knowledge and tests his new description by an on-the-spot
experiment. The chosen experiment is decided upon after discriminating between
multiple hypotheses originally generated to potentially explain the unfamiliar
problem.
The last stage in Schon’s (1987) learning model is ‘reflection-on-action’, a
time in which a physician is said to reflect back on his chosen course of action after
the fact and to re-examine the nature of the ‘surprise’. He reviews and assesses the
relevance of new information encountered in solving the problem. This process
eventually leads to the acquisition of new knowledge that then becomes incorporated
into a physician’s ‘ knowing-in-action ’ knowledge base. This process of reflection
in, and on, action often leads to other questions that motivate the physician to
participate in new educational activities that can reinforce current or newly acquired
knowledge (Borduas, Gagnon, Lacoursiere, & Laprise, 2001).
A Model of Physician Learning and Change
Fox, Mazmanian & Putnam (1989b) offer a description of physician learning
and change based on an extensive study that examined why physicians initiate
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
176
change, the types of changes they make, and how change occurs. The authors’
discussion is derived from interview data collected on 775 reported changes made by
340 physicians nationwide. Findings from each of these research areas are presented
below.
Reasons for Change
Physicians stated that episodes of self-evaluation prompted their motivation
to change, and from their accounts, Fox et al. (1989) identified the emergence of ten
discrete forces, originating in professional, personal, or social dimensions, which
served as impetuses for change in medical practice. Professional forces were
identified most often as reasons for change, and included desire for enhanced
competence (24%) and a perception that one’s clinical environment pressed for a
change (14%). Effort to improve financial status (9%) and desire to move on to a
further stage of career development (8%) were identified as a combination of
professional/personal forces. Similarly, a combination of professional/social forces
included relationships with colleagues in the same institution (11%) or the profession
(9%), and a perceived need to respond to regulations (10%). Purely personal forces
were noted to be personal well-being (8%) and pure curiosity (4%). Roles in the
family or community, outside of a professional context, accounted for 5% of change
forces. The driving force underlying change therefore occurred as a result of the
interrelationship between professional, personal and social forces. Although force
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
177
for change was not typically identified as singular, the authors suggest that one force
seemed to establish a more central and powerful role than others. This primary force
was then supported or opposed by one or more other forces.
How Change Occurred
Fox et al. (1989) state that some form of learning was necessary to make
changes in more than two-thirds of the cases, while the remaining events involved
changes for which the necessary knowledge or skill had previously been acquired.
The extent of learning involved in physician changes was related to the reason, or
force, behind the change. Moreover, the amount of learning was greater when the
change was perceived as internally motivated, and less when it was seen as coerced.
Learning was most often associated with changes caused by purely professional
forces and was described by its purpose, method and resources. Learning purpose
was identified as being either problem-specific or conceptual in nature. Problem- '
specific learning was directed toward finding the best steps to take to solve a well-
defined, concrete problem; the focus of conceptual learning, on the other hand, was
to create better defined and more organized ideas and thoughts, with an intent to
develop intellect rather than to apply a solution to a specific problem. Physicians
identified problem-specific learning as the purpose of learning more often than
conceptual learning (59% to 41%).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
178
Method of learning was identified as either deliberative, in which the order
and structure of resources fostered contemplation and consideration, or experiential,
in which learning resources and information were structured and ordered to enable
direct experience in knowledge and skill application. Deliberative learning would
describe the learning process of a physician who wished to locate information on a
new procedure or piece of equipment, then sought out various articles on the issue to
understand the strengths and limitations involved, and subsequently discussed the
pros and cons with a consultant before arriving at a decision. An example of
experiential learning is illustrated by a physician who, upon deciding to become
more proficient in his radiology interpretation skills, would then seek practice under
the supervision of a radiologist rather than read or listen to lectures on the subject.
An essential difference between these two learning methods therefore is the extent to
which the method emphasizes thought over interaction. Physicians characterized
53% of their changes as having been accomplished through experiential methods and
47% as having occurred through deliberative methods.
Learning resources were categorized by their importance and legitimacy and
were grouped according to the extent that their learning value was viewed as
formally sanctioned. Those resources clearly sanctioned (e.g., received professional
or academic body endorsement) were classified as formal, and all others were
classified as informal (e.g., not considered to be expert opinion, lacking in CME
credit). Formal resources were most often identified as being primary to learning
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
179
overall, particularly when learning was associated with professional forces for
change; however, the authors note that this classification was difficult and their
“confidence in the rightness of it is limited.” They therefore did not offer further
discussion on this issue.
Types of Changes Made
Fox et al. (1989) identified four kinds of changes that each was found to be
associated with different types of learning and forces for change. The first types of
changes were accommodations (16%), which were small and simple changes often
accompanied by apathy, reluctance or anger (e.g., the adoption of a newly required
regulation). Accommodative changes were said to be driven by social or mixed
professional/social forces, and were less likely to result in learning than other types
of changes. These types of changes were not actively sought, rather they were acts
of acquiescence. Adjustments constituted the second type of change, and these
consisted o f generally larger changes that showed incremental differences in some
element or aspects of practice; these were the most common type of change reported
(62%). Adjustments were usually driven by professional forces, with a desire to
increase competence or respond to the clinical environment accounting for nearly
half of these changes. The authors postulate that professional forces are more likely
than other forces to lead to new learning activities, and indeed, new learning was
associated with professional forces for change in 88% of these cases. Being highly
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
180
associated with professional forces for change, adjustment changes therefore were
more likely to be problem-specific and experiential, and the clarity o f the change
rationale was better understood than for other types of changes. The third and fourth
type of changes, redirections and transformations, were considered to be structural
changes. Redirections (18%) commonly resulted from personal forces for change,
and involved the addition, subtraction or change of a major element of one’s
practice, such as an ob/gyn practitioner giving up obstetrics to concentrate fully on
gynecology. This type of change commonly occurred in response to personal forces.
Transformational changes (4%) entailed the restructuring or redefinition of many
elements in the physician’s practice, as, for example, the case of an internist who
progressively enjoyed counseling opportunities with his patients to the extent that he
decided to change specialties and pursue a psychiatric residency. Only one of
twenty-seven transformational changes reported occurred as a result of purely
professional forces. Structural changes were reported to be: less clearly understood
than other types of changes before the change was made; associated more with
personal forces of change, and involved learning that was lengthy, complicated and
conceptually oriented.
Competency-based Learning and Change
Putnam & Campbell (1989) dedicate a chapter in Fox et al. (1989) to the
discussion of the characteristics of physician change when driven by the desire for
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
181
competence. The authors suggest that the motivation behind change driven by a
desire for competence or excellence - the single largest group of changes identified
in this study - pervaded all of the physicians’ changes to some extent, even when the
drive for competence was not the primary force for change. Physicians’ desire for
competence led to change based on one of three reasons: 1) A desire to excel, or live
up to high personal standards of professional behavior; 2) an openness to change was
present and acted upon when an innovation was anticipated to lead to a new or better
way of practicing; and 3) dissatisfaction was experienced with current practice
procedures or outcomes and more acceptable alternatives were pursued. An
openness to change, coupled with an awareness of a desirable innovation, constituted
the largest group of changes driven by the desire for competence (54.3%). The
distinction in these cases was that the triggering motivation was an opportunity to
change as the result of something external to the physician, as contrasted with a
willingness to change as a result of a problem. Physicians making changes governed
by this reason were noted to, “show evidence of an ongoing internalized desire to be
up-to-date, a personal standard adopted during the process of “professionalization”
(p. 93).
The types of changes physicians made covered a broad spectrum and were all
subsumed under the category of adjustments. The most common changes occurred
in prescribing habits and the use of procedures. Other changes were noted in areas
of history-taking, physical examination, and ordering of laboratory and radiological
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
182
investigations, as well as in such non-patient care areas as how physicians organized
their practices, taught students, and thought and felt about their profession. Learning
was almost unanimously associated with the changes (92%) related to increased
competence; 60% of cases were directed toward solving a problem, while 40%
focused on increasing conceptual understanding. Most changes were associated with
a relatively well-defined, discrete learning strategy; however, a subset of 20
physicians each described a change in sufficient enough detail that the authors were
able to identify a pattern of learning common to all of these changes. Three separate
learning phases emerged from these descriptions: preparing to change, making the
change and solidifying the change. Learning resources were found to vary with
stage. In the preparing to change phase, characterized as “taking it all in,” physicians
gathered as much information as possible on the proposed change, and used an
average of two to three different learning resources. The resources physicians most
utilized included: CME programs (15 [number of physicians utilizing resource]),
which encompassed such activities as skill development courses, local or state
professional meetings, grand round sessions, etc.; journal reading (12); Interaction
with other physicians (12); use of other instructional materials (6), such as
videotapes or textbooks; and interaction with non-physicians (5). CME activities
were either specifically sought out to provide information after preparing to change
was initiated, or served as the impetus for preparing to change. Once the decision to
initiate the change was affirmed, the learning involved with making the change
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
183
became more active and focused as physicians developed and experimented with
newly acquired specific skills or information. Fewer learning resources were used in
this phase (averaging just over 1 per physician) and they assumed different priority
than in the first phase. Resources utilized in making the change included:
interaction with other physicians (10); CME programs (6); use of other instructional
materials (4); journal reading (3) and interaction with non-physicians (2).
Physicians’ needs for learning continued in the solidifying the change phase, for
example, by practicing or modifying a newly learned procedure or realizing
additional uses for a newly prescribed drug. In this phase, interactions with other
physicians were again the most widely utilized source (11), followed by CME
programs (8), journal reading (8), teaching (5), use of other instructional materials
(4), and interactions with non-physicians (3). The appearance of ‘teaching’ as a
learning resource in this phase is noteworthy, supporting the belief that the teaching
process is effective in reinforcing learning. In all three learning phases interactions
with other physicians were more likely to be with peers or colleagues rather than
experts or consultants, and this occurred with greatest frequency in the final learning
phase (10/11 physicians learned from colleagues). Review of resources utilized in
these three learning phases indicates that interaction with other physicians remained
quite stable as a learning resource across phases, and was the top resource used in
making the change and solidifying the change phases. The considerable variation in
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
184
journal reading and CME participation indicates that physicians use these medical
learning resources differently depending on the specific learning purpose.
In their conclusions, Putnam & Campbell (1989) state that the overwhelming
tendency for physicians to make small, more controlled changes that are
characteristic of adjustment changes is associated with the slow and careful way
advances are made in clinical science and clinical practice. A hesitancy to react to
every new idea or finding is suggested to represent a basic conservative approach to
medical practice that is taught and modeled in physicians’ training years, and thus
would preserve the edict of “do no harm.” Although this incremental approach
serves to protect patients, the authors suggest that it also may retard appropriate
changes in thinking and directions in practice.
Implications for SDL
In a discussion emerging from this study on the topic of curiosity and the
process of change, Lanzilotti (1989) suggests that encouraging curiosity in medical
students plays an important part in developing SDL. Toward this goal, the author
recommends increasing problem solving activities to promote exploratory behavior
while decreasing lecture hours that promote memorization. Further benefit would be
achieved by helping students develop strategies that encourage observation,
questioning, decision-making and metacognition. Lanzilotti (1989) cites work by
Gibbons et al. (1980) who propose a set of principles for faculty to consider in
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
185
designing strategies to help students develop self-education skills. Among others,
these principles include: 1) integrating theoretical studies with technical training and
practical application as a method of learning; 2) helping learners generate their own
drive toward their own goals rather than stimulating them to pursue goals established
by others; 3) promoting, modeling, and rewarding the development of personal
integrity rather than the opportunistic pursuit of offered rewards; and 4) creating an
active environment in which learners’ self-directed activities are supported and in
which there are opportunities to work cooperatively with others. Lanzilotti (1989)
additionally suggests that positive differences in physicians’ learning behavior may
be observed if, as medical students, they experienced more flexibility and choice in
their curriculum, and if they “were exposed to patient problems from the first day
and charged with the responsibility to learn medicine by solving some of these
problems” (p. 41). Furthermore, the author suggests that medical teaching be given
more value in comparison to research and clinical activity, with more rewards and
awards designated for excellence in teaching, and that greater in-service training be
devoted to help faculty members leam how to support SDL activities for medical
students.
Slotnick’s Theory of Physician SDL
A final learning model frequently cited in the CME literature is offered by
Slotnick (1999). Based on previous CME research and study results derived from
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
186
personal interviews concerning learning needs with 32 physicians, Slotnick (1999)
proposes that physicians’ SDL episodes follow a four-stage process. These stages
include: scanning for potential problems (Stage 0); deciding whether to take on the
problem (Stage 1); learning the required skills and knowledge (Stage 2); and gaining
experience with what has been learned (Stage 3). These stages of learning are
characterized by goals, discrepancies, learning resources, reflections, and criteria for
completion. Suggesting that research has confirmed physicians’ learning motivation
to be generated by problems of either a specific or general nature, Slotnick (1999)
further distinguishes between the resources, reflection and criteria for completion
involved in each problem type.
Specific problems (e.g., a particular patient issue) are said to involve semi
structured learning forms such as reading available journals or consulting colleagues
and result in incremental changes to a doctor’s knowledge or skill. From his
physician interviews, Slotnick (1999) found that learning in response to specific
problems progressed with a sense of urgency. In deciding whether to take on the
problem (Stage 1), the physician sensed a need for immediate action and decided
either on the spot or after reading and talking with colleagues briefly whether to
tackle the problem. In contrast, general problems (e.g., gaps in skills and
knowledge) are described as necessitating formal learning such as planned learning
projects or specialty courses and are associated with redirections in clinical practice.
Slotnick (1999) found physician learning in response to general problems to be more
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
187
deliberate (and thus slower) and made use of resources requiring planning before
they could be accessed (e.g., specialty society courses).
Slotnick (1999) reflects on some of his study findings to draw potential
implications for undergraduate medical education in the area of SDL. On the issue
of pursuing learning problems, the author observes that students are not given a
choice o f whether or not to undertake a particular learning problem (e.g., a PBL
patient case); they simply are given problems and are told to solve them. Moreover,
encountered problems are often well circumscribed with fairly straightforward
solutions. This mode of instruction may lead students to form two unconscious
assumptions: all medical problems encountered should be undertaken; and all
medical problems have clearly established solutions. In findings pertinent to this
second assumption, Slotnick (1999) discerned that many doctors interviewed
described learning events that were terminated before completion of Stage 3 (gaining
experience through application of the new knowledge/skill). Additionally, these
physicians often could not articulate what criteria they used to end their learning
during Stage 2 (i.e., learning the required skills and knowledge), particularly when
available resources were far from exhausted. Although often justified in their
employment of general learning heuristics, such as moving forward if information
encountered from different sources was consistent or provided a plan of action,
Slotnick (1999) cautions that physicians’ casual application of these principles may
also lead to unjustified termination of learning. Referring to a study (Gorman, Ash,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
188
& Wykoff, 1994) in which primary care physicians were found to frequently
underestimate the likelihood that specific patient problems had solutions, Slotnick
(1999) illustrates a potentially dangerous belief held by some physicians: if
solutions do not readily present themselves within a relatively short period of time -
as they tended to in undergraduate training - then they do not exist, thus leading to
the premature termination of SDL issues.
A second implication for SDL in medical education concerns the use of
learning resources. Slotnick (1999) notes that practicing clinicians rely little on
formal textbooks in their SDL activities, depending rather on colleagues and readily
available literature such as journals. A later study by Slotnick (2001) supported this
finding as physicians (n = 152) identified their most commonly used learning
resources to be same specialty physicians, subscribed journals and review articles,
and specialty society CME meetings. Findings confirming the primacy of
physicians’ use of journals and/or colleague consultation as learning resources are
well documented in the CME literature (Covell, Uman, & Manning, 1985; Geertsma,
Parker, & Whitboume, 1982; Mann & Chaytor, 1992; McClaran, Snell, & Franco,
1998; Rothenberg et al., 1982; Stinson & Mueller, 1980). Medical students, on the
other hand, rely heavily on textbooks for their learning. Slotnick (1999)
recommends that educators must decide when in medical training to shift the focus
of learning from textbooks to medical journals and how to implement this change. A
related issue to learning resources is Slotnick’s (1999) suggestion that the functional
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
189
unit of learning for physicians appears not to be medical topics per se, but rather
medical problems (general or specific). This finding is noted by the author to be
described in other studies as well (Jennett, Jones, Mast, Egan, & Hotvedt, 1994;
McClaran et al., 1998; Slotnick, Kristjanson, Raszkowski, & Moravec, 1997;
Slotnick, Kristjanson, Raszkowski, & Moravec, 1998), and is further supported by
the previously discussed findings of Fox et al. (1989). Slotnick (1999) discusses this
issue as an implication for designing CME activities, but it also has direct bearing on
the subject of learning resources. Medical textbooks are typically written and
organized by organ systems and associated topics, for example, a pathology textbook
that contains a chapter on cardiac disease (organ system), that is additionally
subdivided by different notable pathologies or topics (e.g., diseases of the
myocardium, valvular disease, etc.). As such, students’ learning is based largely
around medical topics, and not on medical problems as they are likely to encounter
in future clinical practice. Journals are organized predominantly by medical
problems, but also contain topic information in the form of review articles. These
two contexts represent quite different shifts in thinking when pursuing learning
activities. For students who are not exposed to learning and finding answers to
problem-based issues, SDL in clinical practice will present a bigger challenge.
Perhaps unfamiliarity with this learning framework serves to impede physicians in
their SDL pursuits, or leads them to erroneously think that answers to many
questions simply do not exist. Medical students have ample opportunities to practice
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
190
educating patients in disease prevention and management; however, they may feel
lost if a patient were to say, “Doctor, I ’ve starting taking milk thistle because of the
wonderful health benefits I’ve heard are associated with its use, but I just wanted to
ask you - is it okay for me to be taking it with my diabetes and blood pressure
medication or is there any potentially harmful interaction?” Consulting medical
textbooks for help with this issue will likely prove fruitless, and yet physicians are
routinely faced with patient questions similar to this throughout the course of
practice. It would behoove medical educators to intentionally pose learning
problems to students that cannot be answered by referencing a textbook as part of
SDL development. Providing SDL opportunities that simulate potential practice
situations will help to build students’ skills and can reinforce the idea that SDL is an
expected part of medical professionalism.
Supplementary CME Literature on Physician Learning
Supporting CME literature is presented here to call attention to additional
fundamental characteristics of physician learning that possess added relevance to a
subsequent closing overview of SDL. A study by Bergman & Pantell (1986)
illustrates an important and rather surprising point regarding physicians’ facility with
the medical literature. Eighty-three pediatricians, pediatric residents, and family
practitioners were presented with a common, potentially serious problem of a < 1
year-old infant with a fever of 38.5° C, an unremarkable physical exam, an elevated
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
191
white blood cell count and a normal lumbar puncture. They were subsequently
asked to estimate the probabilities of this infant having bacteremia (bacteria in the
bloodstream) and acquiring meningitis, and to choose one of four possible
management options. They were next asked to read a published scientific report
addressing the risks of meningitis in febrile infants and then re-answer the questions
relating to the clinical problem.
The authors found that physicians’ decisions were not based on logical
processing of information, as there was no correlation between their estimate of the
risk of meningitis and the underlying risk of bacteremia as well as no correlation
between their decisions to hospitalize or use antibiotics and their estimated risk of
the patient developing meningitis. Bergman & Pantell (1986) concluded that
“physicians appear to have considerable difficulty in using probability data and
appear to base estimates of serious disease and subsequent management on intuition
rather than calculation” (p. 380). Additionally, the authors found no correlation
between physicians’ changes in probability estimates or management decisions and
their years of training or whether they had recalled or previously read this particular
study. Bergman & Pantell (1986) also cite other studies in which physicians
misunderstood and misused quantitative and probability information (Berwick,
Fineberg, & Weinstein, 1981; Casscells, Schoenberger, & Graboys, 1978; Eddy,
1982). Examples of this included misinterpreting study results; drawing unsupported
study conclusions; misunderstanding statistical concepts such as “p value” and “false
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
192
positive rate”; an inability to use prevalence rate and false positive rates to estimate
the probability of a positive screening test being a true positive result; and
overestimation of the true positive rate of a positive screening test.
Bergman & Pantell (1986) suggest that their results as well as the previously
cited studies support a growing body of evidence that physicians do not readily
understand statistical concepts and misuse probability information. The authors
propose that physicians should rightfully be responsible for the appropriate use of
clinical results, further adding that the failure to find any difference between
residents and attending physicians in the skills of probability reasoning and statistical
influence in their study or in Berwick’s (1981) study suggests that these skills are not
being taught. The authors note that if medical students and physicians “are not
taught the skills to translate clinical research into clinical practice as part of a core
curriculum in medical decision-making and the use of medical information, they face
a persistent risk o f incorrect interpretation and application of medical research” (p.
385).
Kanouse & Jacoby (1988) adopt a somewhat different theoretical perspective
on the relationship between physicians’ clinical management and the medical
literature. In discussing the difficulty involved with changing physicians’ practice
behavior, the authors state that CME and other informational program evaluations
have demonstrated that providing physicians with clinically relevant information will
not automatically lead to behavioral change. Kanouse & Jacoby (1988) suggest that
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
193
physicians make frequent and numerous adjustments in response to new information,
but that the inundation of information resulting from the exponential growth in
biomedical knowledge justifies their selectivity in making these adjustments. It is
due partly to this reason, the authors state, that most informational interventions will
not change most practitioners’ behavior, even when clear behavioral implications are
delineated. These authors appear to be subtlety implying that the impetus toward
physician change depends more on the Information itself than on the physician. This
implication becomes more overt as they comment, “Basic biomedical information as
published in the scientific literature is not always usable by clinical practitioners,
even if it is made accessible. The need to “repackage” information to make it
clinically useful is widely appreciated” (pp. 29-30). To this, Kanouse & Jacoby
( 1 9 8 6 )add:
The perspective of a clinical decision maker may require the
integration of disparate elements of information that do not map
easily into the research categories in the biomedical literature...
It seems evident that unless a recommendation prescribes defined
actions to be taken in defined circumstances for specific groups of
patients, it is unlikely to have a uniform or measurable effect on
practice, (p. 30)
Within these two articles discussing physician interaction with the medical
literature, then, one finds contrasting views on where the appropriate onus for
facilitating change in patient management lies - with the physician and his/her
ability to interpret and apply the literature to clinical practice or with the literature
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
194
and investigators’ ability to present their findings in a more effectively usable
format. A third, and seemingly intermediate opinion, is offered by Davis & Fox
(1994). These authors suggest that using reading as an effective vehicle for ‘keeping
up5 has become increasingly difficult due to the size and increasing complexity of
the literature. However, they also point out that problems have been uncovered with
the skills physicians need for critical reading and interpretation of the literature. For
these reasons, then, reading by itself has not been found necessarily to be a direct
path to new knowledge.
Physician Learning in Actual Clinical Practice
Two similar studies examining features of physician learning in actual
practice evoke another particularly important issue that must be considered in a
discussion regarding SDL development. In the first of these studies, Covell, Uman
& Manning (1985) studied physicians’ information needs by interviewing them
following patient encounters in their clinics during a half day of typical office
practice. After patient visits, physicians were asked if questions arose regarding
patient management issues to which they would like answers if they had access to
information sources of their choice. Physicians were also subsequently interviewed
at the end of their office sessions to elicit their perceptions of the information sources
they used, additional sources they would like to have used, and barriers to obtaining
these sources. Researchers initially had physicians complete a questionnaire
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
195
regarding information needs prior to the beginning of their interviews. Forty-seven
physicians raised 269 questions during their interviews following a combined total of
409 patient visits, for an average of two questions for every three patients. Most of
the questions raised reflected a need for highly specific non-recurrent points of
information related to individual patients’ problems. The nature of the questions
identified related to: treatment of specific conditions (31%); diagnosis of symptoms,
physical findings, or syndromes (25%); requests for drug information (14%);
obtaining records and communicating with other health professionals (11%); patient
services (8%); general review of a specific disease topic (6%); and use of
interpretation of laboratory tests (5%). O f the 35 out of 47 physicians who were
internal medicine subspecialists (e.g., cardiologists, gastroenterologists, etc.), cross
tabulation of physician subspecialty with subspecialty of the questions posed
revealed that 69% of these questions were outside of a physician’s own subspecialty.
On the issue of question identification, a particularly noteworthy finding reported by
Covell et ai. (1985) was that most of the questions posed by physicians (recorded
verbatim) “were worded in a non-generalized but practice-related fashion, and not in
terms that would make finding the answers easy” (p. 597). The authors include the
example of a physician who asked, “Should I test the serum procainamide level in
this patient?” as opposed to asking, “What are the indications for measuring serum
procainamide?”
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
196
Another important finding in this study concerned the gross discrepancy
between the data on informational needs physicians recorded on their questionnaires
and their actual resources used in practice. Questionnaire responses indicated that
information was needed an average of once per week, although physicians were
found to generate two questions for every three patients seen in a mere half-day of
typical practice. Physicians also reported using print sources such as textbooks,
journals, and drug information sources significantly more often (p < 0.001) than
consultations with other physicians or related health care personnel, with the most
frequently used source reported to be a drug compendium (average reported use,
once a week). However, in practice the source most frequently providing answers to
identified questions was another physician (29%), followed by other health
professionals (24%), and laboratory data (20%). Nine percent of answers were
elicited from drug information texts, 7% each from journals and self-made
compendia, while less than 3% of answers were derived from medical textbooks.
Print sources were reportedly used 61% of the time, while observed use was 26.6%;
similarly, human resource use was reportedly 31.6%, while actual use was found to
be 53.3%.
Related study findings were also indicated for physicians’ satisfaction with
information sources and perceived barriers to obtaining necessary information.
Although most of the participating physicians indicated in the questionnaire that the
sources available to them provided needed information, 30% of physicians
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
197
considered their books and drug information sources to be inadequately indexed for
rapid retrieval of information. Thirty-three percent of physicians had no organized
method for storing and retrieving journal articles, while half of the participants
reported being “somewhat dissatisfied” or “very dissatisfied” with the organization
of their journals, due to haphazard storage and inability to keep up with tearing out
and filing articles. Physicians’ selection of resources was based on perception of
quality, or trustworthiness that the source had the correct answer, 47% of the time,
and on the accessibility of the source 38% of the time; however, 34% of the
information found was not considered helpful, while 25% only partially answered
the question and 9% was deemed unreliable. A particularly striking finding in this
study was that answers to questions raised at the time of the patient visit were found
for only 80 of the 269 questions formulated, or 30%. An average of four questions
per physician were not immediately answered, a finding constituting yet another
disparity from questionnaire responses, in this case conflicting with data indicating
that physicians were unable to find answers only approximately once per month.
Reasons indicated for an inability to answer questions included: awaiting laboratory
reports or patient response (36.5%); appropriate source to locate answer unknown
(24%); human source not immediately contacted (21%); print source not available
(6.5%); source not reliable (5.5%); and too time-consuming (2%).
Two very important findings were revealed in this study, the first of which
was that in one typical half-day of practice, four management decisions might have
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
198
been altered if needed information had been available during patient visits.
Secondly, Covell et ah (1985) note that it was likely that the method of interviewing
physicians following patient visits stimulated questions that otherwise may not have
been considered (explaining the questionnaire response discrepancies), but that the
habit of seeking answers to questions was not in place, and that when stated,
questions were formulated in terms that did not trigger a search for answers.
In a more recent study adapted from Covell et al. (1985), Green, Ciampi &
Ellis (Green, Ciampi, & Ellis, 2000) interviewed 64 internal medicine residents
following a total of 401 patient encounters during 114 half-day clinic sessions to
determine whether their interactions led to the formulation of clinical questions.
Similar to Covell et al.’s (1985) findings, two questions were generated for every
three patients seen and the majority of the 280 questions formulated pertained to
therapy (38%) and diagnosis (27%). Asked to categorize their questions, residents
indicated that 70% of their questions would change patient management, 34% might
involve harming the patient if not answered, and 24% were urgent. Number of
questions generated was modestly associated with earlier postgraduate year, a greater
number of patient issues addressed during the visit, and having had a preceptor see
the patient.
Residents were contacted one week later regarding pursuit of their questions.
A sobering 71% of all questions (200/280) questions went unanswered. Reasons for
not pursuing answers included: lack of time (60%); forgetting the question (29%);
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
199
lack of interest (4%); lack of urgency (3%); belief that the answer would not change
management (3%); and a perception of inadequate resources (2%). Residents’ use of
information sources included textbooks (31%), original articles (21%), attending
physicians (17%), review articles (9%), fellow residents (5%), the Physician’s Desk
Reference (5%), specialty consultants (3%), and other (9%). Resident view of the
questions was significantly associated with question pursuit for two categories: a
belief that a patient expected the answer (p - 0.004) and fear of malpractice liability
(p - 0.05). Although 34% of questions were originally identified as potentially
involving patient harm if left answered, a disturbing finding indicated that this
perception was not significantly associated with question pursuit; among residents
sharing this perception, only 36% of such labeled questions were answered.
The authors in this study distinguished the questions generated as
‘foreground’ or ‘background’ questions. Foreground questions inquire about explicit
or implied outcomes resulting from a particular intervention in a patient case.
Background questions concern general information about a disease process or patient
presentation, such as the description of a disease, the general approach to evaluation,
available therapeutic options, or possible causes of a condition. Sixty-six percent of
questions were foreground related, and advanced level residents asked
proportionately more foreground questions (p = 0.04). Green et al. (2000) cite
evidence that this question classification distinction can guide information seeking
(Northup, Moore-West, Skipper, & Teaf, 1983; Richardson & Wilson, 1997) and
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
200
may assist educators in developing customized algorithms and curricula. The
authors maintain that background questions can be answered by textbooks and
narrative reviews, whereas foreground question often require a search for original
research or evidence-based summaries. An identified example of an informational
source using a similar distinction was ‘SUMSearch’, a recently developed internet-
based hierarchical search engine which distinguishes resources for ‘broad questions’
from those more appropriate for ‘specific questions’ (Badgett, 2000).
In their discussion, Green et al. (2000) cite Covell et al.’s (1985) findings as
well as those from another similar study (Gorman & Helfand, 1995) reporting that
practicing physicians pursued 44% of their clinical questions within 5 days of
question identification. Possible suggested explanations for these low pursuit rates
were commonly reported time constraints among both residents and practicing
physicians (Covell et ah, 1985; Williamson, German, & Weiss, 1989) and
physicians’ complaints of lack of availability, lack of familiarity, or poor
organization of information resources (Williamson et al., 1989).
The authors urge medical educators to direct more attention to SDL in terms
of effectiveness of alternative strategies, curriculum development, and resource
access and usability. Related to their study finding that residents neglected many
questions due to forgetting them, Green et al. (2000) suggest that attending
physicians devote more attention to reminding residents of questions, using such
approaches as ‘educational prescriptions’ and requiring standardized forms for
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
201
answering questions. The authors also include an example of an inpatient clinical
team who uses a mobile ‘evidence cart’ during rounds, a source containing selected
resources which are used to promptly answer emerging questions (Sackett & Strauss,
1998).
Factors Contributing to Physician SDL Motivation
A final area identified in the CME literature having potential implications for
undergraduate SDL training concerns the role of physician motivation in SDL
behavior. An insightful discussion on this topic is offered by Mann & Kibble (1994).
The authors suggest a framework for thinking about motivation in SDL that
incorporates specific factors within three dimensions - the individual, the
environment, and the learning process itself.
Factors within the individual are identified as: forethought capability; goal-
setting capability; perception of a gap or need; self-efficacy; previous success with
SDL; locus of control; achievement; curiosity; professionalism; and attitudes toward
and readiness for SDL. Goal-setting is said to be a motivational force by directing
energy and effort, mobilizing attention and action and generates strategies toward
goal achievement; goals are most effective when they are of average difficulty, as
opposed to being very high or too easy. As the authors note, Bandura (1982)
suggests that self-efficacy is the major determinant of the goals an individual will set
and dictates the energy, effort, and perseverance that will be dedicated to their
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
202
achievement. Mann & Ribble (1994) state that experience is the most effective
enhancer o f self-efficacy, and therefore previous successful experience with SDL
will be likely to lead to higher perceived self-efficacy. This, in turn, will result in
higher motivation for further involvement in SDL experiences. Additionally, having
an internal locus-of-control, and thus believing that one’s actions will exert an effect,
may be associated with self-efficacy, and the authors comment that internal locus-of-
control has been linked to increased satisfaction with SDL in college students.
Mann & Ribble (1994) recount findings by Fox et al. (1989) involving
professionalism as an important factor in the learning process. They borrow from
Fox et ah’s (1989) description of professionalism in defining it as the internalization
of societal and other external expectations that occurs through professional
socialization and involves the interaction of these external expectations with an
individual’s internal personal experience, values, etc. The authors suggest that
individuals frequently lose the distinction between internal and external factors, and
this combination then serves as a potent internal motivator that may impact SDL.
In their discussion on attitudes toward and readiness for SDL, Mann &
Ribble (1994) borrow from Candy’s (1991) analysis of SDL, citing categories that
emerged from his composite list of competencies that have been associated with
successful independent learning. These categories suggest that successful
independent learners are: 1) methodical and disciplined; 2) logical and analytical; 3)
reflective and self-aware; 4) able to demonstrate curiosity, openness and motivation;
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
203
5) flexible; 6) interdependently and interpersonally competent; 7) responsible; 8)
venturesome and creative; 9) confident and have a positive self-concept; 10)
independent and self-sufficient; 11) competent in information-seeking and retrieval
skills; and 12) knowledgeable and skillful about learning processes and can develop
and use criteria for evaluation. The authors are careful to note, however, that while
positive attitudes towards SDL may make SDL participation more likely than not,
attitudinal predispositions are insufficient on their own to effect behavioral change;
for this, skills and an enabling environment are also necessary.
The next set of specific factors related to motivation are those factors existing
within the environment. Qualifying that the environment of SDL is not subject to
distinct boundaries, general environmental factors are described as those involving
the practice environment, including the immediate environment of the physician and
his or her patients, the larger environment involving physician colleagues in the
practice and the community, and lastly the community from which both patients and
physicians come which exerts its own set of expectations. Mann & Ribble (1994)
categorize environmental factors that motivate SDL by their impact on three stages
of the SDL process: 1) asking the question; 2) finding the answer; and 3) integrating
the new information or skill into practice.
Asking the question refers to the process of identifying a learning need;
factors involved here include: time to reflect; availability of information systems;
interaction with colleagues; community expectations; and mechanisms to identify
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
204
needs. The authors suggest that time to reflect, whether in response to Schon’s
(1987) ‘surprise5 situation, or deliberatively in self-evaluation, is the first, critical
step in SDL, as reflection may give rise to a need to leam about additional
knowledge or skills that may impact patient care. As was reported in studies
previously discussed, Mann & Ribble (1994) propose that work and patient
obligations can frequently preclude time for formal reflection or prevent questions
that are raised from being properly acknowledged or answered. An earlier study by
Mann & Chaytor (1992) found that only 47% of family practitioners (n = 390) and
61% of specialists (n = 371) conducted ongoing assessments of competence,
generally using consultation, self-assessment, and structured examinations to do so.
On the subject of availability of information systems, the authors also refer to
a finding from the above study (Mann & Chaytor, 1992) in which 77% of family
practitioners and 67% of specialists rated their computer skills as low. Resource
availability, therefore, must also consider physicians’ ability to appropriately access
those learning resources that are made available.
Observation of and interaction with colleagues may also serve as influential
factors in learning behavior. Behavior modeled by others can provide a standard by
which to measure one’s own performance. Mann & Ribble (1994) suggest the
absence of collegial interaction may exert a negative effect on the pursuit of SDL as
physicians who are in solo practice appear more likely over time to experience
difficulty in maintaining professional competence. They note that their risk appears
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
205
to substantially exceed that of the physician belonging to a practice group (McAuley
& Henderson, 1984; McAuley, Paul, Morrison, Beckett, & Goldsmith, 1990).
In their discussion of community expectations, Mann & Ribble (1994) raise
an important point pertaining to the perception of SDL. They suggest that not only
do increasingly sophisticated and well-informed patients have expectations of their
physicians regarding the understanding of current advances, but they also have the
expectation that physicians will be able to address emerging issues of psychosocial,
socioeconomic, and other dimensions that significantly affect overall health. These
are abilities, the author suggest, that are not systematically addressed during most
physicians’ undergraduate medical education. In their mention of psychosocial and
socioeconomic issues, the authors may be reminding the reader that not all SDL
pertains to the acquisition of specific clinical knowledge or skills; a simple example
is illustrated in a population of homeless patients with substance abuse - a
physician’s SDL focus for these patients could be to leam where shelters and/or
rehabilitation centers are in his/her particular practice radius and provide that
information to patients.
The second stage of the SDL process on which environmental factors have
impact is finding the answer. Three factors proposed to motivate physicians to
successfully complete SDL include: 1) assistance with focusing the question; 2)
assistance with locating resources; and 3) access to those resources. The authors
discuss the development of information systems utilizing librarians and/or other
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
206
physicians that are designed to assist clinicians through clinical problem clarification
and resource support. Other discussion of resource use has previously been
discussed.
Factors integral to the third stage of SDL, integration into practice, are said to
further motivate SDL. Personal factors play a role, as successful integration of
learning into practice increases satisfaction with attained goals or performance,
enhancing perceptions of self-efficacy, that, in turn, increase motivation.
Environmental factors, such as feedback from patients and colleagues in the form of
approval and respect, as well as improved patient health outcomes, also provide
motivation.
Factors inherent within the learning process that motivate SDL include:
control of the learning experience; coaching/corrective feedback; effect of past
experience; social interaction; educational components and strategies; enjoyment;
and achievement of a goal. The authors suggest that (CME) program developers
build in incentives within these areas that will sustain an individual learner’s
attention. Some of these efforts could include using multimedia applications of
computer programming; offering a variety of learning options; providing structured
self-assessment programs and telephone hotlines; using material that builds on prior
knowledge; providing interactive discussion groups or forming study groups;
devoting energy to making material more animated, and thus more enjoyable;
teaching on-line literature searching skills; using contract learning and information
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
207
brokering (linking physicians with consultants and community resources). Although
this discussion was directed toward SDL in the CME arena, most of these
educational strategies are suited to undergraduate medical education as well.
In their discussion of questions on SDL that remain unanswered, Mann &
Ribble (1994) suggest that future research include: 1) more inquiry into the nature
and usefulness of resources used by self-directed learners; 2) attempts to identify and
describe personal attributes associated with participation in SDL and to clarify their
developments; 3) the effect of interventions on the development of SDL; and 4) the
predictive value of measurement scales. The authors also consider current
implications for the practice and providers of CME, in the last of which they state:
Finally, recognizing the importance of the continuum of
medical education, CME providers must promote the importance
of acquiring skills in self-directed learning in undergraduate
curricula and residency training. If these skills are successfully
acquired early in the individual’s career, they are likely to
become an embedded source of motivation, (p. 86)
Chapter Summary
Chapter 5 considered whether review of the CME literature could offer new
Insight into physician SDL behavior by providing further clarification o f the nature
of their learning practices. Three models of physician SDL were discussed and
certain common trends of learning were observed. Physician learning purpose
appears to be defined as problem-specific or conceptual in nature, and may be
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
208
undertaken either deliberativeiy or in response to exposure to an unfamiliar problem.
Learning may also be formal or informal and resources used vary depending on this
classification. Overall, physician SDL is heavily dependent on journal reading,
interactions with colleagues, and CME courses. Supplementary research
subsequently presented revealed that physicians may have difficulty interpreting
clinical research findings, report learning needs that are discrepant from actual
observed learning needs, and answer only a small percentage of questions that they
generate following patient encounters, due to time constraints, lack of proper
resource identification or poor organization of available resources. A concluding
discussion on physician motivation in SDL suggested that motivation results from a
combination of specific factors operating within the individual, the environment and
the learning process itself. Implications these findings may have for undergraduate
medical education is discussed in the closing summary.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
209
CHAPTER 6
A CLOSING DIGEST ON SDL
This paper has presented extensive discourse on the biography of SDL, from
its birth within general education to its maturation within medical training.
Description of SDL’s genesis as a budding formal academic concept within the field
of adult education established an historical grounding from which to consider its
growth and diffusion into the arena of medical education. In the half-century since
the medical education iconoclasts emerged to challenge the stronghold of Flexner’s
(1910) manifesto, medical educators have been searching for panacean curricular
reform. The once deeply intractable didactic teaching philosophy began to yield to
the sociocultural demand for more modem, integrated medical education curricula.
PBL appeared as a promising, untested curricular innovation that grew in credibility
with Barrows’ (1980, 1983, 1986) establishment of formal educational objectives
and more sharply defined instructional philosophy. SDL gained greater exposure
within medical education when it became one of the formally proposed, although
initially lesser-considered, educational objectives of PBL. Like a virus lying
dormant until cellular division occurs, SDL remained quiescent until the branches of
curricular reform were shaken once more. Medical education was still too passive;
students needed to take more responsibility for their own learning. The influential
GPEP Report (1984) recognized SDL as a set of skills that required development to
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
210
allow medical students to learn throughout their professional lives, rather than
simply to master current information. The LCME followed suit by requiring the
incorporation of SDL into the curriculum. SDL was dusted off and brought to the
forefront o f medical education to undergo closer scrutiny by the educational
community.
And what, then, has been gleaned from this closer scrutiny of SDL? This
paper focused spectral analysis on a subset of four such dimensions of SDL - the
consideration of its conceptual and quantitative nature; understanding its association
with PBL; analyzing research comparing its development in PBL versus non-PBL
students and graduates; and examining its behavioral manifestations in practicing
physicians. The myriad of results emerging from this dissection have invoked the
commentary offered in the antecedent pages; what remains to be contemplated are
the implications that can be drawn from this amalgamation of data.
Considerations remaining unaddressed from this compendium of findings
include the following:
♦ How does SDL become better delineated and more unanimously
accepted within the medical education community?
♦ If PBL curricula are theoretically designed to facilitate the
development of SDL, why does undergraduate and graduate
research not support this hypothesis?
♦ Is the inclusion of SDL as an independent curricular strategy
in medical education more theory or reality?
♦ How can CME research findings on physician SDL more clearly
assist in SDL development at the undergraduate level?
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
211
Before addressing each of these questions, a definition of SDL relevant to
medical education/training is re-iterated for the reader. The conceptual process of
SDL, as stated in chapters 1 and 4, refers to the skill set needed to fulfill a purposeful
learning task, and operationally includes the: 1) accurate self-assessment of current
knowledge/learning state; 2) identification and articulation of learning needs and
objectives; 3) appropriate utilization and critique of learning resources; 4) use of
effective learning strategies; 5) application of new knowledge/skills; and 6) self-
evaluation of learning outcomes. SDL product or outcome is conceptually defined
as: the identification of learning needs consistently generated over time that leads to
the intemhlly motivated, self-initiated, and sustained behavior required to effect
intentional and permanent learning for the purpose of expanding one’s professional
knowledge/skill base or changing clinical management practice. SDL outcome
operationally includes the following behaviors (when learning is internally
motivated): 1) independent reading (e.g., medical journals, texts); 2) use of
supplemental materials (e.g., computer programs, AV aids); 3) medical
consultations; 4) CME participation; 5) non-CME participation (e.g., local
conferences); 6) volunteer activity; and 7) change in patient care practices.
Towards a More Unified Definition of SDL
The answer to achieving better SDL definition and acceptance is tri-fold: 1)
recognized SDL scholars in medical education need to be established and accepted;
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
212
2) the creation of innovative, well defined quantitative and qualitative SDL
instruments must be pursued; and 3) SDL must be awarded greater attention at local,
state and national educational conferences. Each of these assessments was
separately mentioned elsewhere in this paper. The suggestion that there existed a
lack of medical education SDL scholars who were comparable to those in adult
education was raised in chapter 1. In the chapter 4 discussion, it was mentioned that
integration between the undergraduate and post-graduate literature could be seen
from the re-emergence of authors previously cited in the chapter 3 analysis. Only
limited research was incorporated into the previous non-statistical meta-analysis;
however, the term ‘post-graduate’ literature is purposely used here, as research
utilizing residents, although technically CME literature, appears to constitute a
somewhat separate category of literature. The point of this is to remark that in these
potentially three distinct literature bases, there was one author who appeared at each
level - Mann (University of Dalhousie, Nova Scotia, Canada). Perhaps with
multiple studies and theoretical discussions on SDL, this author is emerging (or has
emerged) as a leader in this field. As seen in adult education with scholars such as
Brookfield and Candy, any newly developing field must await the presence of a few
committed authors whose work then offers further direction within the field.
The need for medical education to develop innovative research instruments to
further measure SDL was discussed in chapter 3. Guglielmino (1979) and Oddi
(1984) continue to be referenced in the literature as researchers who have contributed
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
213
to this area. In the comprehensive literature reviewed for this paper, no examples of
other SDL research instruments were discovered at the undergraduate level Two
SDL self- assessment tools, which may hold value as potential research tools, were
referenced in the CME literature. As briefly discussed by Mann & Gelula (2003),
Fox (2000) has developed a change readiness inventory for physicians in CME that
allows respondents to characterize their current state of knowledge and goals through
a series of objectives strictly related to a specific disease process. Also noted by
these authors was the development of a PC Diary, a software-based program
designed to record SDL progress (Campbell et a l, 1996).
Thirdly, as mentioned within the recommendations proposed in chapter 3, in
order for SDL to gain consistency in its theoretical and operational definitions, more
attention needs to be devoted to this subject at academic conferences. In this way,
the promotion of wider dissemination of SDL ideas, strategies and research can serve
to facilitate discussion and understanding of SDL within the medical education
community. As these three essential pieces of the SDL puzzle take greater shape,
greater accord in the field will follow.
The Missing Evidence in SDL in PBL Research
The theoretical analysis of the skills and strategies used to facilitate SDL
(Candy, 1991) within a PBL instructional context revealed that this educational
strategy appears well designed to enhance SDL skills in medical students. The
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
214
methods used in PBL to encourage each of the seven strategies described by Candy
(1991) were reviewed and found to be quite representative of SDL-reinforcing
techniques. Why, then, does research not empirically support this theory? It is
argued here that a complexity of issues exist, spanning the instructional to research
gamut, that prevent SDL in PBL research from generating supportive data. Some of
these difficulties include: 1) Inconsistent definitions of both PBL and SDL; 2)
inability to control for lack of standardization of PBL methodology; 3) variable
quality o f PBL facilitation; 4) minimal use of formal SDL assessment; 5) insufficient
time spent practicing SDL; 5) inconsistent research operationalization of SDL; and
6) lack o f clear understanding regarding the direct relationship between SDL skill
acquisition and demonstration of future SDL behavior.
Many of these points have been addressed elsewhere, and are briefly re
addressed here in consideration of their collective identification as research obstacles
to establishing support for a positive and sustained association between SDL and
PBL. The inconsistency in PBL, and particularly SDL, definitions has been
thoroughly discussed. The point of re-emphasis is simple: medical educators cannot
hope for consistent research results for what they cannot consistently define.
Nebulous or equivocal research constructs stand little chance of producing clear
research results. Even when PBL is clearly defined, standardizing instruction is
simply not possible, and the extent to which this lack of standardization translates
into inconsistent research findings is difficult to estimate. Although the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
215
methodological steps in PBL may have predictable uniformity, much of the process
must remain spontaneous and flexible in order to optimize educational value.
Student curiosity and reasoning ideally guide the flow of instruction and these
elements will naturally lack predictability. There is an array of instructional
variables that may play a role in research uncertainty - Does SDL change in any way
if there are five students in the group, as opposed to eight? How do different student
social dynamics affect the SDL process? Does the nature of the cases themselves
change the use of SDL? Does depth of current relevant case knowledge alter
students’ practice of SDL? A further consideration is the quality of the facilitator
him/herself. PBL research has studied the effects of expert versus non-expert
facilitation on such outcomes as student satisfaction, use of learning issues, time
spent in ‘self-study’, etc. However, the simple identification of facilitators as expert
or non-expert still does not tell us about the quality of the facilitation. Facilitation
quality is, in fact, highly variable and difficult to measure without observational
analysis.
The apparent dearth of formal SDL assessment associated with PBL
instruction was previously noted. Understanding how this may impact research also
deserves consideration. Certainly, the assessment of a student’s SDL skills as
evaluated by a casual presentation of a learning issue to the group is distinct from the
formal assessment offered by a TIE. Similarly, the number of PBL sessions, and
thus opportunities for SDL, deserves much more attention than was found in the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
216
research literature. While many authors spoke of PBL being employed in the two-
year basic science curriculum, the answer to the question, “And how many actual
PBL sessions did that entail?” was absent. It must be intuitive that the more
opportunities a student is given to engage in SDL, the greater the possibility for SDL
skill development. While opportunity never guarantees skill acquisition, it certainly
increases its probability.
The inconsistent operationalization of SDL has been given sufficient
commentary elsewhere as well. As was acknowledged in referring to the
inconsistency in PBL and SDL definitions, this point of re-emphasis is also obvious:
measuring SDL as apples and oranges will yield research results of apples and
oranges; dissimilar measurements cloud the issue more than clarify it, and serve to
decrease the likelihood of finding evidence to support the theoretical link between
PBL and SDL development. This is a bit more complex perhaps than it appears on
the surface and deserves a little closer analysis. Although use of multiple measures
of a construct was recommended in the chapter 4 discussion on postgraduate
research, those measures in the field of CME are more clearly delineated than they
are in undergraduate education. For example, “number of hours spent discussing
cases with colleagues/week” is a logical measure In a CME context, but obviously
does not fit within an undergraduate educational context. Operational SDL variables
in undergraduate medical education have been less well discriminated and accepted.
Although multiple measures are still needed here, by using measures that may have
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
217
too many confounding intermediaries between the proposed PBL-SDL link, more
confusion is likely to arise when attempting to formulate a clear measure-hypothesis-
theory model. Research in this field should still be considered embryonic in nature,
and when this is the case, articulating measures that more closely reflect the
proposed fundamental conceptual definition of SDL is crucial if one is looking for
evidence to support its proposed theoretical association with another construct, in
this case PBL. In the case of library utilization, for example, the argument was
previously proposed that this use could simply be a reflection of PBL curricular
design (a confounding variable), rather than a measure of SDL. What may subtlety
have occurred in the process of PBL-SDL model formulation is that the theoretical
dimension of the model expanded too far ahead of its measurement counterpart.
Simply put, PBL-SDL theory was unquestioningly accepted before enough research
was generated to support it. Because the primary focus on PBL research has until
late been on the more visible, and ‘in vogue’ educational objectives of structuring
knowledge for use In clinical contexts, and development of an effective clinical
reasoning process, people may have simply assumed vicarious support for SDL’s
association with PBL when supportive findings were being reported for an
association between PBL and its first two objectives. What we may be witnessing
with SDL’s relatively newly rising credibility in medical education is a backpedaling
to try to determine how to provide evidentiary proof of an accepted theoretical
association with PBL that was left previously unexplored.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
218
A direct extension of this dilemma may be reflected in the PBL-SDL
theoretical weak link that does not clarify how SDL skill acquisition necessarily
leads to the adoption of SDL outcome behavior. It was encouraging that the SDL
outcome operations most utilized in the chapter 4 studies reflected the SDL
behaviors physicians also engaged in most often as reported in studies presented in
chapter 5. The use of measures that coincide with actual behavior is beneficial to
research; however, these study results reviewed in chapter 4 did not support the
theory that PBL graduates develop more effective SDL behavior than CC graduates.
In their discussion of motivational factors contributing to SDL, Mann & Ribble
(1994) suggested that repeated experience (with SDL) is the most effective enhancer
of self-efficacy; previous successful experience with SDL will likely lead to higher
perceived self-efficacy. Higher self-efficacy, in turn, was said to result in higher
motivation for further involvement in SDL experiences. While this may be true, the
PBL-SDL research has not produced evidence to support this hypothesis. O f course,
Mann & Ribble (1994) also suggest that motivation to engage in SDL stems from a
combination of factors, including individual attitudinal predispositions, skills and an
enabling environment. As suggested in chapter 2, SDL research may also benefit by
examining the SRL literature to consider better the nature of self-efficacy and the
integrative effect of self-efficacy and motivation on academic achievement.
With the identification of what may appear to be a cornucopia of obstacles
facing PBL-SDL research, researchers should be given a great deal of credit for their
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
219
efforts to contribute to the field using what in many cases are less than optimal
conditions. Overcoming hurdles in the pursuit of educational research is, in some
respects, the “nature of the beast.” Identifying definitive measures for such internal
constructs as motivation is thorny in the best of controlled situations. Man perhaps
pays a price for his cognitive sophistication when the focus becomes educational
research; though we possess the highest spot on the evolutionary ladder, it is less
exasperating and far more sufferable to train with repeated precision the behavior of
a colony of laboratory rats than it is to propose with conviction an understanding of a
single reason behind a human being’s motivation. While elucidation of PBL-SDL
educational theory is inherently problematic, two suggestions are worth re-stating
here that would benefit research: 1) provide enough clarity and comprehensiveness
of PBL/SDL definitions and descriptions to allow for reproducibility of a study; and
2) use SDL operations in undergraduate education that better reflect the skill set that
has gained increased acceptance (i.e., as previously discussed and presented in
Figure 1) and rephrase study questions to more meaningfully examine these SDL
skills. When important information is omitted from study descriptions, perhaps due
to a belief that a common understanding exists and therefore certain information
does not merit inclusion, opportunities for understanding the specificity of the
possible conditions under which research may (or may not) produce evidentiary
support for the proposed PBL-SDL association are lost. Using library utilization
(I.e., representing the SDL skill of learning resource selection/critique) as an
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
220
example to illustrate the second suggestion, rather than defining a study question in
terms o f whether PBL students use the library with greater frequency than CC
students, perhaps a more revealing question may be to ask whether or not there are
differences in the learning resources selected by PBL and CC students for the same
learning task, and if so, what are they and why specific resources were chosen.
SDL in Medical Education - Fact or Theory?
The question of whether SDL exists more in theory than it does in reality in
medical education curricula is perhaps the most difficult of the four issues raised in
this summary to answer. One reason for this is that proper discussion of this
question must consider SDL as both specific to a PBL curriculum and as a separate
educational curricular intervention. This paper’s focus was primarily on the former,
and therefore a literature search pertaining to SDL outside of a PBL curriculum was
not emphasized. Nevertheless, after discussing SDL within PBL, an opinion will be
offered regarding SDL as an independent curricular intervention.
In the recommendation section in chapter 3, it was suggested that SDL was
not a prevalent, well-established, formal part of PBL medical education instruction
or research. This assessment followed from the analysis of a fair volume of PBL
literature. However, this finding should not surprise anyone familiar with PBL
education. It would seem that SDL is a ‘newer’ PBL educational objective.
Although there is no indication that Barrows (1986) openly prioritized the four
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
221
objectives assigned to PBL, by surveying the literature there is clear indication that
both interest and research of these objectives has progressed in chronological order.
The extensive literature base available concerning PBL and its association with
clinical knowledge structure and clinical reasoning skills was cited. And while
literature on SDL in PBL is still relatively sparse in comparison, a person interested
in pursuing reading on PBL’s fourth objective, increased motivation for learning,
would have an even harder time amassing literature. Motivation for learning was
also suggested to be the least well documented of the PBL objectives by Berkson
(1993). What intuitively stands to reason, then, is that an educational objective that
is not well-defined conceptually or operationally in the literature, has minimal
references to its formal evaluation in the literature, and is completely absent as a
PBL objective in other discussions in the literature, is not likely to exist as a well
established, purposeful part of a PBL curriculum.
Then the appropriate question stemming from this becomes - why not? First
of all, it appears that SDL overall is increasingly becoming discussed and recognized
as a legitimate educational objective - although whether increasing interest in
clarifying SDL as an objective in PBL led to more discussion of SDL as a separate
curricular intervention or vice versa is difficult to say. Additionally, more
comprehensive discourses are appearing concerning the theory of SDL in PBL (e.g.,
see Evenson & Hmelo, 2000), and the always-influential LCME, with their explicit
reference to SDL in the accreditation passage on general curricular design, have sent
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
222
notice to medical schools that SDL is being recognized as an important element in
medical education. But here again, while many academics may think SDL “is a
great idea,” where is the evidence that more is actively being done to further
facilitate SDL?
The reticence with moving SDL still forward may have to do with its
assessment. Formal and informal educational goals and/or objectives are quite
different, naturally, as the former necessitate formal assessment while the latter do
not. So then, for SDL in PBL, which has functioned almost exclusively as an
informal objective, to now move into the limelight as a formal objective, curriculum
directors will be pressured to assess it. Assessing what has traditionally been poorly
described is not a most comfortable job. If curriculum directors then turn to the PBL
literature for help on SDL assessment, they will likely find very little of it, and what
they do find they will not like - the TJE. The TJE, like PBL itself, theoretically
supports SDL development, and, as mentioned, is also extremely labor intensive.
Curriculum administrators do not like educational solutions that are labor intensive.
Any educator who has taken part in meaningful curricular reform can attest to how
very taxing this process can be. Given the choice between making SDL a formal
educational objective or keeping it as an informal objective, and hoping it gets
accomplished in the process, well - hope springs eternal.
The same difficulty surfaces when considering the implementation of SDL as
an independent curricular strategy, although as an independent curricular strategy,
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
223
assessment creativity may be a little more flexible. It may be that when SDL is
formally assessed as part of a PBL curriculum, educators feel a greater responsibility
for that assessment to more closely reflect the PBL process itself, and in the case of a
TJE, the exam can be designed to reflect many elements of the PBL process. After
all, it is educationally sound to formally assess students’ skills using the same
methods they have been practicing in formative assessment, particularly for less
experienced medical students. When implemented as an independent curricular
approach, SDL is not tied to PBL. As more is learned about SDL in medical
education, newer, creative methods for developing SDL may lend themselves to
more assessment options.
Further consideration will be given to SDL development strategies during
discussion of the implications of CME research findings on SDL undergraduate
development. Before leaving the current issue, however, there is a final point that
bears substantial impact on SDL incorporation into medical education. This point
concerns the entrenched culture inherent to academic medicine. The hierarchical
structure of a medical university is powerful dictator of the flow of operations. It is a
poorly kept secret that the pecking order of the hierarchy is - clinical service first,
research a close second, and medical education a distant third. Service work and
research bring in money; medical education does not (apparently, 20 million dollars
in tuition each year doesn’t count). There is a clear message sent with considerable
decibels to those who care to listen in on the inner bowels of a medical school: very
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
224
few, and often not a single faculty member, is hired strictly to teach; clinicians are
expected to teach as part of their job description, yet educators routinely scramble to
fill uncovered lab slots because supervisors will not let subordinates out of patient
care responsibilities due to anticipated lose of revenue; and medical school
‘education’ tuition increases every year while ‘educational’ budgets remain frozen.
After being informed that the LCME accreditation body was exerting pressure on
administrators to change a stagnant curriculum, one medical school dean replied,
“That’s fine, just don’t let it cost anything.” Quite simply, medical education is
valued only by a handful of educators who remain passionate about the quality of
instruction that is imparted to medical students. These same few also feel an ethical
responsibility to graduate young physicians who are clinically sound.
Curricular reform is made that much more difficult by the lack of value given
to it. To many, it represents an unnecessary expenditure of finances and human
resources - “if w e’ve trained medical students successfully for years using the
traditional system, why do we need to change?” SDL faces a particularly difficult
challenge in this regard. Not only are many educators still not quite sure what it
even is, but it does not have the popular student appeal of innovative curricular
techniques such as PBL often enjoys. Students do not clamor for educators to let
them become more responsible for their own learning. In their educational careers to
date, learning was passive and most medical students are content to leave it that way.
Anyone searching for a reason why educators should not bother with SDL
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
225
development needs to look no farther than the student body where s/he is guaranteed
to find vocal agreement. Most medical students still enter medical school fresh out
of college and not many U.S. colleges employ active SDL within their curricula.
Students are more likely to view SDL as punitive rather than constructive,
particularly in their initial year of medical school in which they require time to adjust
to the intellectual demands expected of them. Most medical students have difficulty
seeing their futures clearly (“if it doesn’t apply to me now, I’ll worry about it
later”...), and in many ways, medical education has not helped to clarify that vision;
that is the whole point behind SDL. SDL is their future - it is their entire
professional future. This is why implementing SDL is an educational imperative. A
valid argument can be made in support of SDL skill development being as important,
and probably more important, than anything else medical students will learn in
medical school. What needs to be determined is whether anybody at the helm is
listening.
Making CME Research Work in Undergraduate Education
The last issue to be considered in this closing overview concerns the
implications that can be drawn from the CME literature that pertain to SDL
development at the undergraduate level. The study findings and physician SDL
models reviewed in chapter 5 reveal exciting possibilities for incorporating SDL in
undergraduate medical education.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
226
There were a number of significant points arising in chapter 5 that educators
should consider: 1) authors either directly or indirectly repeatedly mentioned the
importance of reflection and self-assessment in SDL; 2) physicians often reported
learning needs that contrasted sharply with their actual learning behavior; 3) journals
and colleagues were consistently found to be the most important learning resources
used in physicians’ SDL activities, and overall, different learning resources were
used for different learning goals ; 4) physicians often could not articulate what
criteria they used to end their learning and begin implementing an intended change;
5) physicians had difficulty phrasing their learning issues in terms conducive to
finding answers; 6) physicians were found to misinterpret clinical study data and
make unjustified conclusions; 7) SDL issues were formulated as medical problems,
not medical topics; problems were either problem/patient-specific or conceptual in
nature, and were more often reported as problem/patient-specific; 8) learning was
described as deliberative or experiential; 9) most physician changes were motivated
by the professional desire for increased competence; 10) physicians underestimated
the likelihood that specific patient problems had solutions; and 11) most questions
generated by physicians following patient encounters went unanswered; in many
cases the reason stated concerned not knowing what the appropriate learning source
was.
These findings should cause concern for all medical school curriculum
directors. In this one sample of studies, findings clearly indicated that physicians
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
227
had difficulty with SDL skills. And in one study that specifically addressed the
issue, more than half of the respondents indicated that they wanted expanded
assistance with SDL (Mann & Chaytor, 1992). It is worth recalling that this same
study found that greater then 70% of physicians also rated their computer skills as
low. The suggestion was made that SDL activities in undergraduate education
should simulate the actual SDL issues clinicians face in practice. Consultation of the
CME literature would greatly benefit educators in understanding the specific nature
of those SDL issues. In turn, they can use these to create similar situations,
appropriate to a medical student level, to purposefully assist students in developing
more effective SDL skills. Table 9 is presented in an effort to provide concrete
examples of learning activities educators might employ toward this goal.
Further elements involved with SDL promotion were included in the
recommendations given in chapter 3. One of these included the active discussion of
SDL as part of the professionalization process. As we saw in chapter 5, the attitude
of wanting to remain competent as part of one’s professional responsibility was the
most frequent reason prompting physicians’ changes in the Fox et ah (1989) study.
Faculty role modeling was also suggested as adding significant influence to this
process. One study proposed that the role modeling of SDL may be thought of as a
deliberate role that can then be assigned to the ‘educational influential’ (Mann &
Ribble, 1994). One research question posed to practicing physicians that may
provide utility in designing undergraduate SDL activities is to ask them what SDL
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
228
TABLE 9 - POSSIBLE SDL EXAMPLES/ACTIVITIES FOR
UNDERGRADUATE MEDICAL EDUCATION BASED ON
STUDY FINDINGS FROM THE CME LITERATURE
♦ Provide training In computer use, instruction in use of different resources; instruct
students specifically in literature search techniques and assign periodic learning tasks
that require students to use these skills
4 Provide students with educational choices
Example: Employ student portfolios - allow students to choose from a list of
options (e.g., presenting and interpreting clinical study findings to fellow students;
teaching fellow students how to interpret chest x-rays; spending an evening at an AA
meeting, interviewing patients about their psychosocial concerns and writing a report
of the experience; developing a self-study module for students on how to read EKGs;
facilitating a PBL session, etc.); also consider novel proposals by students
All choices must be assessed for quality using specific guidelines known to students
in advance of the project. Many activities should require teaching/interacting with
students and faculty who may be asked to assess the student’s performance; some
should have potential for raising new learning issues that a student would then need
to research and report back on; students should be required to participate in different
types of activities (e.g., requiring: teaching; patient interaction; writing; etc.)
♦ Use patient cases, PBL or otherwise, frequently in teaching
Focus on having students:
-identify what information is considered critical to understanding the case
and why
-identify what information they do not understand, and why they do not
understand It
-understand how phrasing learning questions differently can lead to different
research results
-identify in advance their intended learning strategy(ies) and resources and
their rationale for choosing them; have them determine second choices
-identify different learning assignments that are patient-specific or conceptual
in nature; are deliberative and experiential; and include assignments that
cannot be answered from a textbook
-making comparisons and contrasts of the present case to other cases or other
previously learned concepts
-justify their thinking for all links in the case (e.g., “why do you want to order
that lab test?”, “why would you ask the patient that question?”, etc.)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
229
TABLE 9 - POSSIBLE SDL EXAMPLES/ACTIVITIES FOR
UNDERGRADUATE MEDICAL EDUCATION AS BASED ON
STUDY FINDINGS FROM THE CME LITERATURE (CONTINUED)
♦ Use patient cases, PBL or otherwise, frequently in teaching (continued)
Focus on having students:
-teach their classmates some element of the case
-use diagramming, such as concept maps (e.g., see Guerrero, 2001) to
visually connect ideas
-identify how the information learned in their assignments changes their
thinking about the case and why
-practice reasoning through a case from start to finish (e.g., “tell me all of the
steps involved with your thinking for a patient who initially presents with
polyuria, polydipsia and polyphagia and ends up In renal failure 10 years
later.”)
♦ Journal reading and interpretation
Give students repeated practice In working with the medical literature
consider instruction in the principles of: how to use evidence-based
medicine; how to determine if clinical study results are applicable to another
theoretical patient population; interpreting statistical probabilities;
reconciling contradictory information
Introduce literature specifically on physician SDL for discussion, include
such topics as how a physician monitors his/her competence; why and when
change in practice behavior may be necessary
♦ Self-assessment/Peer-assessment
May include such areas as interactions with patients; cooperative, small-
group learning skills; learning strategies; teaching or presentation ability;
required analysis of erroneous exam answers to determine where their
understanding was faulty, etc.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
230
TABLE 9 - POSSIBLE SDL EXAMPLES/ACTIVITIES FOR
UNDERGRADUATE MEDICAL EDUCATION AS BASED ON
STUDY FINDINGS FROM THE CME LITERATURE (CONTINUED)
♦ Present actual or theoretical patient cases for the specific purpose of exploring
learning and/or change behavior
Example: You have a 58-year-old patient whom you newly diagnosed with
hypertension 2 months ago and started on a calcium channel blocker. She has had 4
visits to monitor her blood pressure during this time. Her blood pressure continues
to run between 134-140/90-96 mm Hg. Do you decide to change her medication?
Why or why not? If you decide to change her medication, do you pursue any active
learning in reaching your decision?
Use examples that have prompted recent controversial findings, such as hormone
replacement therapy
Use examples that require the need for SDL and discuss when learning is considered
sufficient in order to initiate the change in treatment
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
231
skills they would have liked to have practiced more in medical school. Also
important to this process is to provide students with feedback, and to demonstrate
praise or respect for their SDL efforts. Successfully implementing SDL education is
contingent upon faculty members being facile in SDL themselves, and therefore,
faculty development deserves critical attention in the course of designing SDL
education.
It is hoped that the extensive analysis of SDL provided in these pages can be
used to promote the understanding and development of SDL within medical
education. SDL must rightfully be considered the oldest education technique known
to man, but only in the past twenty years has medical education focused its attention
on dissecting its theory and research enough to consider its underlying layers. While
much remains unanswered about its full potential, what is clear is that the rapid rate
of growth of medical knowledge and medical advances demands that current and
future physicians be able to navigate through this vast informational jungle. While it
remains possible for physicians to blindly find their way, teaching them effective
SDL skills from the inauguration of their undergraduate training is, in effect,
providing them with an internal compass to steer their long professional journey.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
232
Bibliography
Adams, L., Kasserman, I., Yearwood, A., Perfetto, G., Bransford, J.,
& Franks, J. (1988). The effect of fact versus problem oriented acquisition. Memory
& Cognition, 16, 167-175.
Albanese, M. A., & Mitchell, S. (1993). Problem-based learning: A
review o f literature on its outcomes and implementation issues. Academic Medicine.
68, 52-81.
Al-Haddad, M. K., & Jayawickramarajah, P. T. (1991). Problem-
based curriculum: outcome evaluation. Medical Teacher, 13(4), 273-279.
Anderson, S., Camp, M. G., & Philp, J. R. (1990). Library Utilization
by Medical Students in a Traditional or Problem-based Curriculum. In W. Bender, R.
J. Hiemstra, A. Scherpbier, & R. Zwierstra (Eds.), Teaching and Assessing Clinical
Competence (pp. 77-80). Groningen, The Netherlands: Boek Werk Publications.
Attarian, L., Fleming, M., Barron, P., & Strecher, V. (1987).
Comparison of health promotion practices of general practitioners and residency
trained family physicians. Journal of Community Health. 12. 31-39.
Badgett, R. G. (2000). SUMSearch: A new way to locate medical
evidence. SGIM Forum. 23. 6.
Bandura, A. (1982). Self-efficacy mechanism in human agency.
American Psychologist. 37, 122-147.
Bandura, A. (1989). Regulation of cognitive processes through
perceived self-efficacy. Developmental Psychology. 25(5). 729-735.
Bandura, A. (1996). Multifaceted impact of self-efficacy beliefs on
academic functioning. Child Development. 67. 1206-1222.
Barnsley, L., Cameron, R., Engel, C. E., Feletti, G. I., Hazell, P.,
McPherson, J., Murphy, L. B., Pearson, S., Powis, D. A., Rolfe, I., Smith, A. J.,
Saunders, N. A., & Wallis, B. J. (1994). Ratings of performance of graduates from
traditional and nontraditional medical schools. Teaching and Learning in Medicine,
6(3), 179-184.
Barrows, H. S. (1983). Problem-based, self-directed learning. Journal
of the American Medical Association. 250(22). 3077-3080.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
233
Barrows, H. S. (1986). A taxonomy of problem-based learning
methods. Medical Education. 20.481-486.
Barrows, H. S., & Tamblyn, R. M. (1980). Problem-based Learning:
An Approach to Medical Education (Vol. 1). New York: Springer Publishing
Company.
Bergman, D. A., & Pantell, R. H. (1986). The impact of reading a
clinical study on treatment decisions of physicians and residents. Journal of Medical
Education. 61, 380-386.
Berkson, L. (1993). Problem-based learning: Have the expectations
been met? Academic Medicine. 68. S79-S88.
Bernstein, P., Tipping, J., Bercovitz, K., & Skinner, H. A. (1995).
Shifting students and faculty to a PBL curriculum: Attitudes changed and lessons
learned. Academic Medicine. 70, 245-247.
Berwick, D. M., Fineberg, H. V., & Weinstein, M. C. (1981). When
doctors meet the numbers. American Journal of Medicine. 71. 991-998.
Bland, C. J., Meurer, L. N., & Maldonado, G. (1995). A systematic
approach to conducting a non-statistical meta-analysis or research literature.
Academic Medicine. 70(7). 642-653.
Bligh, J. (1995). Problem-based learning in medicine: an
introduction. Postgraduate Medical Journal, 71, 323-326.
Blumberg, P. (2000). Evaluating the Evidence that Problem-based
Learners are Self-directed Learners. In D. H. Evensen & C. E. Hmelo (Eds.),
Problem-based Learning: A Research Perspective on Learning Interactions (pp. 199-
226). Mahwah, NJ: Lawrence Erlbaum Associates.
Blumberg, P., & Michael, J. A. (1992). Development o f self-directed
learning behaviors in a partially teacher-directed problem-based learning curriculum.
Teaching and Learning in Medicine. 4(1). 3-8.
Blumberg, P., Michael, J. A., & Zeitz, H. (1990). Roles of student
generated learning issues in problem-based learning. Teaching and Learning in
Medicine. 2, 149-154.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
2 3 4
Borduas, F., Gagnon, R., Lacoursiere, Y., & Laprise, R. (2001). The
longitudinal case study: From Schon’ s model to self-directed learning. The Journal
o f Continuing Education in the Health Professions. 21. 103-109.
Bransford, J. D., Franks, J. I., Vye, N. J., & Sherwood, R. D. (1989).
New approaches to instruction: Because wisdom can't be told. In S. Vosniadou & A.
Ortony (Eds.), Similarity and analogical reasoning (pp. 470-497). New York:
Cambridge University Press.
Bransford, J. D., & Nitsch, K. E. (1978). Coming to understand things
we could not previously understand. In J. F. Kavanaugh & W. Strange (Eds.), Speech
and language in the laboratory, school, and clinic (pp. 267-307). Cambridge, MA:
MIT Press.
Bransford, J. D., Sherwood, R., Vye, N. J., & Rieser, J. J. (1986).
Teaching thinking and problem solving. American Psychologist, 41. 1078-1089.
Bransford, J. D., & Stein, B. S. (1993). The IDEAL problem solver.
New York: Freeman.
Brockett, R. G. (1985). Methodological and substantive issues in the
measurement of self-directed learning readiness. Adult Education Quarterly, 36(1).
15-24.
Brockett, R. G., & Hiemstra, R. (1991). Self-direction in adult
learning: Perspectives on theory, research, and practice. New York: Routledge.
Brookfield, S. (1984). Self-directed adult learning: A critical
paradigm. Adult Education Quarterly. 35(2). 59-71.
Brookfield, S. (1985). Self-directed learning: a conceptual and
methodological exploration. Studies in the Education of Adults, 17( 1), 19-32.
Brookfield, S. D. (1987). Developing Critical Thinkers: Challenging
Adults to Explore Alternative Wavs of Thinking and Acting. San Francisco: Jossey-
Bass.
Bruning, R. FL, Schraw, G. J., & Ronning, R. R. (1999). Cognitive
psychology and instruction (Third ed.). Upper Saddle River, NJ: Prentice Hall.
Bulter, D. L. (2002). Qualitative approaches to investigating self
regulated learning: Contributions and challenges. Educational Psychologist, 37(1).
59-63.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
235
Caffarella, R. S., & O'Donnell, J. M. (1989). Self-Directed Learning.
Nottingham, England: The Department of Adult Education, University of
Nottingham.
Calhoun, J. G., Ten Haken, J. D., & Woolliscroft, I. O. (1990).
Medical students' development of self- and peer-assessment skills: A longitudinal
study. Teaching and Learning in Medicine, 2(1), 25-29.
Camp, M. G., Mitchell, R., Blumberg, P., Kalishman, S., & Zeitz, H.
J. (1992). Students' perceptions of the learning environments in problem-based and
traditional medical school curricula: Exploring the implications for educational
planning. Paper presented at the Symposium presented at the annual meeting of the
American Educational Research Association, San Francisco, CA.
Campbell, C., Parboosingh, I., Gondocz, T., Babitskaya, G., Lindsey,
E., & DeGuzman, R. C. (1996). Study of physicians' use of a software program to
create a portfolio of their self-directed learning. Academic Medicine. 71. S49-S51.
Campbell, D. T., & Stanley, J. C. (1963). Experimental and Ouasi-
experimental Designs for Research. Chicago: Rand McNally College Publishing
Company.
Candy, P. C. (1991). Self-direction for Lifelong Learning. San
Francisco: Jossey-Bass, Inc.
Cariaga-Lo, L. D., Richards, B. F., Hollingsworth, M. A., & Camp, D.
L. (1996). Non-cognitive characteristics of medical students: entry to problem-based
and lecture-based curricula. Medical Education. 30.179-186.
Casscells, B. S., Schoenberger, A., & Graboys, T. B. (1978).
Interpretation by physicians of laboratory results. New England Journal of Medicine.
299. 999-1000.
Chapman, J. A., Westmorland, M. G., Norman, G. R., Durrell, K., &
Hall, A. (1993). The structured oral self-directed learning evaluation: one method of
evaluating the clinical reasoning skills o f occupational therapy and physiotherapy
students. Medical Teacher. 15(2/3). 223-236.
Chickering, A. W. (1964). Dimensions of independence. Journal of
Higher Education. 35. 38-41.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
236
Coles, C. R. (1985). Differences between conventional and problem-
based curricula in their students' approaches to studying. Medical Education, 19.
308-309.
Collins, A., Brown, J. S., & Newman, 8. E. (1989). Cognitive
apprenticeship: Teaching the crafts of reading, writing and mathematics. In L. B.
Resnick (Ed.), Knowing, learning and instruction: Essays in honor of Robert Glaser
(pp. 453-494). Hillsdale, NJ: Lawrence Erlbaum Associates.
Collins, R., & Hammond, M. (1987). Self-directed learning to educate
medical educators, part 2: why do we use self-directed learning? Medical Teacher,
9(4), 425-432.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation:
Design & Analysis Issues for Field Settings. Boston: Houghton Mifflin Company.
Covell, D. G., Uman, G. C., & Manning, P. R. (1985). Information
needs in office practice: Are they being met? Annals of Internal Medicine. 103. 596-
599.
David, T. J., Dolmans, D. H. J. M., Patel, L., & van der Vleuten, C. P.
M. (1998). Problem-based learning as an alternative to lecture-based continuing
medical education. Journal of the Royal Society of Medicine, 91, 626-630.
Davis, W. K., Naim, R., & Paine, M. E. (1992). Effects of expert and
non-expert facilitators on the small-group process and on student performance.
Academic Medicine. 67, 470-474.
Della-Dora, D., & Blanchard, L. J. (1979). Moving Toward Self-
Directed Learning. Alexandria: Association for Supervision and Curriculum
Development.
Distlehorst, L. H., & Robbs, R. S. (1998). A comparison of problem-
based learning and standard curriculum students: Three years of retrospective data.
Teaching and Learning in Medicine, 10(3), 131-137.
Dolmans, D., & Schmidt, H. (1996). The advantages o f problem-
based curricula. Postgraduate Medical Journal. 72(851), 535-538.
Dolmans, D. H. J. M., & Schmidt, H. G. (1994). What drives the
student in problem-based learning? Medical Education. 28, 372-380.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
237
Eagle, C. I., Harasym, P., & Mandin, H. (1992). Effects of M ors with
case expertise on problem-based learning issues. Academic Medicine. 67. 465-469.
Eddy, D. M. (1982). Probabilistic Reasoning in Clinical Medicine:
Problems and Opportunities. In D. Kahneman, P. Slovic, & A. Tversky (Eds.),
Judgment Under Uncertainty: Heuristics and Biases (pp. 249-268). Cambridge:
England: Cambridge University Press.
Education, L. C. o. M. (2003). Functions and Structure of a Medical
School: Standards for Accreditation of Medical Education Programs Leading to the
M.D. Degree . Washington, D.C.: Association of American Medical Colleges.
Entwistle, N. J., & Hounsell, D. J. e. (1979). Higher Education. 8(4),
(entire issue).
Evensen, D. H., & Hmelo, C. E. (2000). Problem-Based Learning: A
Research Perspective on Learning Interactions. Mahwah, NJ: Lawrence Erlbaum
Associates.
Field, L. (1989). An investigation into the structure, validity, and
reliability o f Guglielmino's self-directed learning readiness scale. Adult Education
Quarterly. 39(3). 125-139.
Finucane, P. M., Johnson, S. M., & Prideaux, D. J. (1998). Problem-
based learning: its rationale and efficacy. Medical Journal of Australia. 168.445-
448.
Flexner, A. (1910). Medical Education in the United States and
Canada: A Report to the Carnegie Foundation for the Advancement of Teaching.
Boston, MA: Updyke.
Foley, R. P., Poison, A. L., & Vance, J. M. (1997). Review of the
literature on PBL in the clinical setting. Teaching and Learning in Medicine. 9(1). 4-
9.
Fox, R. (2000). Using theory and practice to shape the practice of
continuing professional development. Journal of Continuing Education in the Health
Professions. 20. 238-246.
Fox, R. D., Mazmanian, P. E., & Putnam, R. W. (1989a). Change and
I .earning in the Lives of Physicians. New York: Praeger.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
238
Fox, R. D., Mazmanian, P. E., & Putnam, R. W. (1989b). Changing
and Learning in the Lives of Physicians. New York: Praeger Pblishers.
Gall, M. D., Borg, W. R., & Gall, I. P. (1996). Educational Research:
An introduction (6th ed.). White Plains: Longman Publishers USA.
Garrison, D. R. (1997). Self-directed learning: Toward a
comprehensive model. Adult Education Quarterly. 48(1), 18-33.
Geertsma, R. H., Parker, R. C., & Whitboume, S. K. (1982). How
physicians view the process of change in their practice behavior. Journal of Medical
Education. 57, 752-761.
Gibbons, M. e. a. (1980). Toward a theory of self-directed learning:
A study of experts without formal training. Journal of Humanistic Psychology. 20(2),
41-56.
Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in
Social Research. Beverly Hills: Sage Publications, Inc.
Gordon, M. J. (1991). A review of the validity and accuracy of self-
assessments in health professions training. Academic Medicine, 66, 762-769.
Gorman, P., Ash, J., & Wykoff, L. (1994). Can primary care
physicians' questions be answered using the medical journal literature? Bulletin of
the Medical Library Association, 82, 140-146.
Gorman, P. N., & Helfand, M. (1995). Information seeking in primary
care: How physicians choose which clinical questions to pursue and which to leave
unanswered. Medical Decision Making, 15. 113-119.
Graham, H. J., Seabrook, M. A., & Woodfield, S. J. (1999).
Structured packs for independent learning: a comparison of learning outcome and
acceptability with conventional teaching. Medical Education, 33, 579-584.
Graham, S. (1997). Executive control in the revising o f students with
learning and writing difficulties. Journal of Educational Psychology, 89, 223-234.
Green, M. L., Ciampi, M. A., & Ellis, P. J. (2000). Residents’ medical
information needs in clinic: Are they being met? American Journal o f Medicine,
109.218-223.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
239
Guerrero, A. P. S. (2001). Mechanistic case diagramming: A tool for
problem-based learning. Academic Medicine. 76, 385-389.
Guglielmino, L. M. (1978). Development of the self-directed learning
readiness scale (Vol. 38, pp. 6467-A). (Doctoral Dissertation, University of Georgia,
1977): Dissertation Abstracts International.
Guglielmino, L. M. (1989). Reactions to Field's investigation into the
SDLRS. Adult Education Quarterly, 39(4), 235-240.
Gunzburger, L. (1980). Characteristics identified upon entrance to
medical school associated with future participation in professional education (Vol.
41, pp. 2572A). (Doctoral dissertation, University of Chicago, 1980): Dissertation
Abstracts International.
Hassan, A. M. (1981). An investigation of the learning projects
among adults of high and low readiness for self-direction in learning (Vol. 42, pp.
3838A). (Doctoral dissertation, Iowa State University): Dissertation Abstracts
International.
Hedges, L. V. (1992). Meta-analysis. Journal of Educational
Statistics. 17(4). 279-296.
Hill, J., Rolfe, I. E., Pearson, S., & Heathcote, A. (1998). Do junior
doctors feel they are prepared for hospital practice? A study of graduates from
traditional and non-traditional medical schools. Medical Education. 32(1), 19-24.
Hmelo, C. E. (1998). Cognitive consequences of problem-based
learning for the early development of medical expertise. Teaching and Learning in
Medicine. 10(2). 92-100.
Hmelo, C. E., & Lin, X. (2000). Becoming Self-Directed Learners:
Strategy Development in Problem-Based Learning. In D. H. Evensen & C. E. Hmelo
(Eds.), Problem-Based Learning: A Research Perspective on Learning Interactions
(pp. 227-250). Mahwah, N.J.: Lawrence Erlbaum Associates.
Hodges, B., Regehr, G., & Martin, D. (2001). Difficulties in
recognizing one's own incompetence: Novice physicians who are unskilled and
unaware of it. Academic Medicine. 76. S87-S89.
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis:
Cumulating research findings across studies. In A. P. Association (Ed.), Studying
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
240
Organizations; Innovations in methodology . Beverly Hills, CA: Sage Publications
Inc.
Isaac, S., & Michael, W. B. (1997). Handbook in Research and
Evaluation (3rd ed.). San Diego: Educational and Industrial Testing Services.
Jennett, P., Jones, D., Mast, T., Egan, K., & Hotvedt, M. (1994).
Characteristics of self-directed learning. In D. A. Davis & R. D. Fox (Eds.),
Physicians as Learners (pp. 47-65). Chicago: American Medical Association.
Jennett, P. A. (1992). Self-directed learning: A pragmatic view. The
Journal of Continuing Education in the Health Professions. 12. 99-104.
Jennett, P. A., & Swanson, R. W. (1994). Lifelong, self-directed
learning: Why physicians and educators should be interested. The Journal of
Continuing Education in the Health Professions. 14. 69-74.
Kaufman, A., Mennin, S., Waterman, R., Duban, S., Hansbarger, C.,
Silverblatt, H., Obenshain, S., Kantrowitz, M., Becker, T., Samet, J., & Wiese, W.
(1989). The New Mexico experiment: Educational innovation and institutional
change. Academic Medicine. 64. 285-294.
Kaufman, D. M., & Mann, K. V. (1996). Comparing students'
attitudes in problem-based and conventional curricula. Academic Medicine. 71.
1096-1099.
Kaufman, D. M., & Mann, K. V. (1998). Comparing achievement on
the Medical Council of Canada Qualifying Examination Part I of students in
conventional and problem-based learning curricula. Academic Medicine. 73. 1211-
1213.
Kelson, A. C. M. (2000). Epilogue: Assessment o f Students for
Proactive Lifelong Learning. In D. H. Evensen & C. E. Hmelo (Eds.), Problem-based
Learning: A Research Perspective on Learning Interactions (pp. 315-345). Mahwah,
NJ: Lawrence Erlbaum Associates.
Knowles, M. S. (1975). Self-directed Learning: A Guide for Learners
and Teachers. Chicago: Follett Publishing Company.
Kolm, P., & Verhulst, S. J. (1987). Comparing self- and supervisor
evaluations: A different view. Evaluation & the Health Professions. 10(1). 80-89.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
241
Kreber, C. (1998). The relationships between self-directed learning,
critical thinking, and psychological type, and some implications for teaching in
higher education. Studies in Higher Education, 23(1), 71-86.
Landers, K. (1990). The Oddi Continuous Learning Inventory: An
alternative measure of self-direction in learning (Vol. 50, pp. 3824A). (Doctoral
dissertation, Syracuse University, 1989): Dissertation Abstracts International.
Lanzilotti, S. S. (1989). Curiosity. In R. D. Fox, P. E. Mazmanian, &
R. W. Putnam (Eds.), Changing and Learning in the Lives of Physicians (pp. 29-43).
New York: Praeger Publishers.
Lawrence, S. L., Grosenick, D. J., Simpson, D. E., & Susteren, T. J.
(1992). A comparison of problem-based and didactic approaches to learning on an
ambulatory medicine clerkship. Teaching and Learning in Medicine. 4. 221-224.
Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of
psychological, educational, and behavioral treatment: Confirmation from meta
analysis. American Psychologist. 48(12). 1181-1209.
Lloyd-Jones, G., Margetson, D., & Bligh, J. G. (1998). Problem-based
learning: a coat of many colours. Medical Education. 32(492-494).
Long, H. B. (1989). Some additional criticisms of Field's
investigation. Adult Education Quarterly. 39(4). 240-243.
Long, H. B., & Agyekum, S. K. (1983). Guglielmino's self-directed
learning readiness scale: a validation study. Higher Education, 12(1). 77-87.
Mann, K., & Ribble, J. (1994). The Role of Motivation in Self
directed Learning. In D. A. Davis & R. D. Fox (Eds.), The Physician as Learner:
Linking Research to Practice (pp. 67-88). Chicago: American Medical Association.
Mann, K. V. (1994). Educating medical students: Lessons from
research in continuing education. Academic Medicine. 69. 41-47.
Mann, K. V., & Chaytor, K. M. (1992). Help! Is anyone listening?
An assessment of learning needs of practicing physicians. Academic Medicine. 67.
S4-S6.
Mann, K. V., & Kaufman, D. (1995). Skills and attitudes in self
directed learning: The impact of a problem-based curriculum. Paper presented at the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
242
Proceedings of the Sixth Ottawa conference on medical education, Toronto, ON,
Canada.
Mann, K. V., & Kaufman, D. M. (1999). A comparative study of
problem-based and conventional undergraduate curricula in preparing students for
graduate medical education. Academic Medicine. 74. S4-S6.
Mann, K. & Putnam, R. W. (1989). Physicians' perceptions of
their role in cardiovascular risk reduction. Preventive Medicine. 18. 45-58.
Marshall, J. G., Fitzgerald, D., Busby, L., & Heaton, G. (1993). A
study of library use in problem-based and traditional medical curricula. Bulletin of
the Medical Library Association. 81(3). 299-305.
Martenson, D., Myklebust, R., & Stalsberg, H. (1992). Implementing
problem-based learning within a lecture-dominated curriculum: Results and process.
Teaching and Learning in Medicine. 4(4). 233-237.
Marton, P., Hounsell, D. J., & Entwistle, N. H. (1984). The
Experience of Learning. Edinburgh: Scottish Academic Press.
Maudsley, G. (1999). Do we all mean the same thing by "problem-
based learning"? A review of the concepts and a formulation of the ground rules.
Academic Medicine, 74. 178-185.
McAuley, R. G., & Henderson, H. W. (1984). Results of the peer
assessment program of the College of Physicians and Surgeons of Ontario. Journal of
the Canadian Medical Association. 131, 557-561.
McAuley, R. G., Paul, W. M., Morrison, G. H., Beckett, R. F., &
Goldsmith, C. H. (1990). Five-years results of the peer-assessment program of the
College of Physicians and Surgeons of Ontario. Journal of the Canadian Medical
Association, 143, 1193-1199.
McClaran, J., Snell, L., & Franco, E. (1998). Type of clinical problem
is a determinant of physicians’ self-selected learning methods in their practice
settings. Journal of Continuing Education in the Health Professions, 18, 107-118.
McCloskey, M., Caramazza, A., & Green, B. (1980). Curvilinear
motion in the absence of external forces: naive beliefs about the motion of objects.
Science. 2 1 0 ,1139-1141.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
243
McCune, S. K. (1989a). A meta-analytic study of adult self-direction
in learning: A review of the research from 1977 to 1987 (Vol. 49, pp. 3237A).
(Doctoral Dissertation, Texas A & M University, 1988): Dissertation Abstracts
International.
McCune, S. K. (1989b). A statistical critique of Field’ s investigation.
Adult Education Quarterly. 39(4).
Mennin, S. P., Friedman, M., Skipper, B., Kalishman, S., & Synder, J.
(1993). Performances on the NBME I, II, and III by medical students in the problem-
based learning and conventional tracks at the University of New Mexico. Academic
Medicine. 68, 616-624.
Mennin, S. P., Kalishman, S., Friedman, M., Pathak, D., & Snyder, J.
(1996). A survey of graduates in practice from the University of New Mexico's
conventional and community-oriented problem-based tracks. Academic
Medicine! 1079-1089).
Meyer, D. K., & Turner, J. C. (2002). Using instructional discourse
analysis to study the scaffolding of student self-regulation. Educational Psychologist,
37(1), 17-25.
Michalczyk, A. E., & Lewis, L. A. (1980). Significance alone is not
enough. Journal o f Medical Education. 55. 834-838.
Miflin, B. M., Campbell, C. B., & Price, D. A. (2000). A conceptual
framework to guide the development of self-directed, lifelong learning in problem-
based medical curricula. Medical Education, 34, 299-306.
Minium, E. W., King, B. M., & Bear, G. (1993). Statistical Reasoning
in Psychology and Education (3rd ed.). New York: John Wiley & Sons, Inc.
Moore, G. T., Block, S. D., Style, C. B., & Mitchell, R. (1994). The
influence of the New Pathway curriculum on Harvard medical students. Academic
Medicine, 69(12), 983-989.
Morris, A., & Stewart-Dore, N. (1984). Learning to Learn from Text:
Effective Reading in the Content Areas. North Ryde, Australia: Addison-Wesley.
Needham, D. R., & Begg, I. M. (1991). Problem-oriented training
promotes spontaneous analogical transfer. Memory-oriented training promotes
memory for training. Memory & Cognition. 19. 543-557.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
244
Neufeld, V. R., Woodward, C. A., & MacLeod, S. M. (1989). The
McMaster M.D. program: A case study of renewal in medical education. Academic
Medicine. 64. 423-432.
Newbie, D. I., & Clark, R. M. (1986). The approaches to learning of
students in a traditional and in an innovative problem-based medical school. Medical
Education. 20. 267-273.
Nolte, J. (1986). A problem-solving approach to teaching basic
sciences: A final report: [Unpublished report to the National Fund for Medical
Education, grant #DM 14/85].
Nolte, J. (1989). Allocation of educational time by first-year medical
students in traditional and in partially problem-based curricula. Paper presented at
the Proceedings of the Twenty-eighth Annual Conference on Research in Medical
Education, Washington, D.C.
Norman, G. R., & Schmidt, H. G. (1992). The psychological basis of
problem-based learning: A review of the evidence. Academic Medicine. 67. 557-
565.
Northup, D. E., Moore-West, M., Skipper, B., & Teaf, S. R. (1983).
Characteristics of clinical information-searching: Investigation using critical
incident technique. Journal of Medical Education. 58. 873-881.
Oddi, L. F. (1986). Development and validation of an instrument to
identify self-directed continuing learners. Adult Education Quarterly. 36(2). 97-107.
Oddi, L. F. (1987). Perspectives on self-directed learning. Adult
Education Quarterly. 38(1). 21-31.
Olson, J. O. (1987). The McMaster philosophy: A student's
perspective on implementation. Medical Education. 21. 293-296.
Ozuah, P. O., Curtis, J., & Stein, R. E. K. (2001). Impact of problem-
based learning on residents' self-directed learning. Archives of Pediatric &
Adolescent Medicine. 155(6). 669-672.
Page-Voth, V., & Graham, S. (1999). Effects of goal setting and
strategy use on the writing performance and self-efficacy of students with writing
and learning problems. Journal o f Educational Psychology. 91, 230-240.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
245
Paiva, R. E. A. (1979). Reliability in follow-up evaluations of
graduates as residents. Paper presented at the Proceedings of the Eighteenth Annual
Conference on Research in Medical Education, Washington, D.C.
Panel on the general professional education of the physician and
college preparation for medicine. (1984). Physicians for the twenty-first century:
The GPEP report . Washington, B.C.: Association of the American Medical
Colleges.
Papa, F. J., & Harasym, P. H. (1999). Medical curriculum reform in
North America, 1765 to the present: a cognitive science perspective. Academic
Medicine. 74. 154-164.
Paris, S. G., & Paris, A. H. (2001). Classroom applications of research
on self-regulated learning. Educational Psychologist. 36(2). 89-101.
Patel, V. L., Groen, G. J., & Norman, G. R. (1991). Effects of
conventional and problem-based medical curricula on problem solving. Academic
Medicine. 66. 380-389.
Patrick, H., & Middleton, M. I. (2002). Turning the kaleidoscope:
What we see when self-regulated learning is viewed with a qualitative lens.
Educational Psychologist. 37(1). 27-39.
Peng, W. (1989). Self-directed learning: A matched control trial.
Teaching and Learning in Medicine. 1(2). 78-81.
Perfetto, G. A., Bransford, J. D., & Franks, J. J. (1983). Constraints
on access in a problem solving context. Memory & Cognition, 11. 24-31.
Perkins, D. N., & Salomon, G. (1989). Are cognitive skills context-
bound? Educational Researcher, 18(1), 16-25.
Perry, N. E., VandeKamp, K. O., Mercer, L. K., & Nordby, C. J.
(2002). Investigating teacher— student interactions that foster self-regulated learning.
Educational Psychologist. 37(1). 5-15.
Peters, A. S., Greenberger-Rosovsky, R., Crowder, C., Block, S. D.,
& Moore, G. T. (2000). Long-term outcomes of the new pathway program at
Harvard medical school: A randomized controlled trial. Academic Medicine. 75,
470-479.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
246
Pilling-Cormick, J., & Bulik, R. I. (1999). A preliminary study
exploring the use of the Self-directed Learning Perception Scale in a clinical setting.
In L. H. Associates (Ed.), Contemporary ideas and practices in self-directed learning
(pp. 103-116). Norman, OK: Public Managers Center, College of Education,
University o f Oklahoma.
Richardson, W. S., & Wilson, M. C. (1997). On questions,
background and foreground (pp. 6): Evidence-Based Healthcare Newsletter.
Ripkey, D. R., Swanson, D. B., & Case, S. M. (1998). School-to-
school differences in step 1 performance as a function of curriculum type and use of
step 1 in promotion/graduation requirements. Academic Medicine. 730 0 ), S16-S18.
Rothenberg, E., Wolk, M., Scheidt, S., Schwartz, M., Aarons, B., &
Pierson, R. N. (1982). Continuing medical education in New York County:
Physician attitudes and practices. Journal of Medical Education. 57, 541-549.
Sabbaghian, Z. S. (1980). Adult self-directedness and self-concept:
An exploration of relationships (Vol. 40, pp. 3701A). (Doctoral Dissertation, Iowa
State University, 1979): Dissertation Abstracts International.
Sackett, D. L., & Strauss, S. E. (1998). Finding and applying evidence
during clinical rounds: the "evidence cart". Journal of the American Medical
Association, 2 8 0 .1336-1338.
Salomon, G., & Perkins, D. N. (1989). Rocky roads to transfer:
Rethinking mechanisms of a neglected phenomenon. Educational Psychologist. 24,
113-142.
Santos-Gomez, L., Kalishman, S., Rezler, B., Skipper, B., & Mennin,
S. P. (1990). Residency performance of graduates from a problem-based and a
conventional curriculum. Medical Education, 24, 366-375.
Saunders, K., Northup, D. E., & Mennin, S. P. (1985). The library in a
problem-based curriculum. In A. Kaufman (Ed.), Implementing Problem-based
Medical Education: Lessons from Successful Innovations . New York: Springer.
Schmidt, F., & Hunter, J. E. (1995). The impact of data-analysis
methods on cumulative research knowledge. Evaluation & the Health Professions.
18(4), 408-427.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
2 4 7
Schmidt, H. G. (1983). Problem-based learning: rationale and
description. Medical Education, 17. 11-16.
Schmidt, H. G. (1989). New Directions for Medical Education:
Problem-based Learning and Community-oriented Medical Education. New York:
Springer-Verlag.
Schmidt, H. G. (2000). Assumptions underlying self-directed learning
may be false. Medical Education. 34(4). 243-245.
Schmidt, H. G., Dauphinee, W. D., & Patel, V. L. (1987). Comparing
the effects of problem-based and conventional curricula in an international sample.
Journal of Medical Education. 62. 305-315.
Schmidt, H. G., Machiels-Bongaerts, M., Hermans, H., ten Cate, T. J.,
Venekamp, R., & Boshuizen, H. P. A. (1996). The development of diagnostic
competence: Comparison of a problem-based, an integrated, and a conventional
medical curriculum. Academic Medicine. 71. 658-664.
Schmidt, H. G., van der Arend, A., Moust, J. H. C., Kokx, I., & Boon,
L. (1993). Influence of tutors' subject-matter expertise on student effort and
achievement in problem-based learning. Academic Medicine, 68. 784-791.
Schmidt, H. G., & van der Molen, H. T. (2001). Self-reported
competency ratings of graduates of a problem-based curriculum. Academic
Medicine. 7 6 .466-468.
Schon, D. (1987). Educating the Reflective Practitioner: Toward a
New Design for Teaching and Learning in the Professions. San Francisco: Jossey-
Bass.
Schon, D. A. (1983). The Reflective Practitioner: How Professionals
Think in Action. New York: Basic Books, Inc.
Shin, J. H., Haynes, B., & Johnston, M. E. (1993). Effect of problem-
based, self-directed undergraduate education on life-long learning. Journal of the
Canadian Medical Association. 148(6). 969-976.
Six, J. E. (1989). The generality of the underlying dimensions of the
Oddi Continuing Learning Inventory. Adult Education Quarterly, 40f 1), 43-51.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
248
Skager, R., Dave, R. H., & Robinson, K. G. (1977). Curriculum
Evaluation for Lifelong Education. Elmsford, NY: Pergamon Press.
Slotnick, H. (1999). How doctors leam: Physicians' self-directed
learning episodes. Academic Medicine. 74. 1106-1117.
Slotnick, H. B., Harris, T. B., & Antonenko, D. R. (2001). Changes in
leaming-resource use across physicians' learning episodes. Bulletin of the Medical
Library Association, 89. 194-203.
Slotnick, H. B., Kristjanson, A. F., Raszkowski, R. R., & Moravec, R.
(1997). How doctors leam: mechanisms of action. Paper presented at the Annual
Meeting of the American Educational Research Association, Chicago, IL.
Slotnick, H. B., Kristjanson, A. F., Raszkowski, R. R., & Moravec, R.
(1998). A note on mechanisms of action in physical learning. Professional Education
Res. Quarterly, 15(2), 5-12.
Smith, R. M. (1993). The triple-jump examination as an assessment
tool in the problem-based medical curriculum at the University of Hawaii. Academic
Medicine. 68. 366-372.
Spencer, J. A., & Jordan, K. R. (1999). Learner centred approaches in
medical education. British Medical Journal. 318(7193). 1280-1283.
Stinson, E. R., & Mueller, D. A. (1980). Survey of health
professionals' information habits and needs. Journal of the American Medical
Association. 243(2). 140-143.
Stuart, M. R., Goldstein, H. S., & Snope, F. C. (1980). Self-evaluation
by residents in family medicine. The Journal of Family Practice. 10(4). 639-642.
Thomas, R. E. (1997). Problem-based learning: measurable
outcomes. Medical Education. 31. 320-329.
Thompson, W. G., Lipkin Jr., M., Gilbert, D. A., Guzzo, R. A., &
Roberson, L. (1990). Evaluating evaluation: Assessment of the American Board of
Internal Medicine resident evaluation form. Journal of General Internal Medicine. 5.
214-217.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
249
Torrance, E. P., & Mourad, S. (1978). Some creativity and style of
learning and thinking correlates of Guglielmino's self-directed learning readiness
scale. Psychological Reports. 43., 1167-1171.
Tough, A. (1971). The adult's learning projects. Toronto: Ontario
Institute for Studies in Education.
Tough, A. (1978). Major learning efforts: Recent research and future
directions. Adult Education, 28, 250-263.
Towle, A., & Cottrell, D. (1996). Self directed learning. Archives of
Disease in Childhood. 74(4), 357-359.
van Luijk, S. I., & van der Vleuten, C. P. M. (2001). Assessment in
problem-based learning (PEL). Annals of the Academy of Medicine Singapore, 30„
347-352.
Vernon, D. T. A., & Blake, R. L. (1993). Does problem-based
learning work? A meta-analysis of evaluative research. Academic Medicine. 68.
550-563.
Walton, H. I., & Matthews, M. B. (1989). Essentials o f problem-
based learning. Medical Education. 23. 542-558.
Wertsch, J. V., & Bivens, J. A. (1993). The social origins of
individual mental function: Alternatives and perspectives. In R. Cocking & K. A.
Renninger (Eds.), The development and meaning of psychological distance (pp. 203-
218). Hillsdale, NJ: Lawrence Erlbaum Associates.
Williamson, J. W., German, P. S., & Weiss, R. e. a. (1989). Health
science information management and continuing education of physicians: A survey
of U.S. primary care practitioners and their opinion leaders. Annals o f Internal
Medicine. 110. 151-160.
Woolliscroft, J. O., Ten Haken, J., Smith, J., & Calhoun, J. G. (1993).
Medical students' clinical self-assessments: Comparisons with external measures of
performance and the students' self-assessments of overall performance and effort.
Academic Medicine, 68, 285-294.
Zimmerman, B. J. (1986). Becoming a self-regulated learner: Which
are the key subprocesses? Comtemporary Educational Psychology. 11, 307-313.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
250
Zimmerman, B. J. (1990). Self-regulated learning and academic
achievement: An overview. Educational Psychologist. 25(1). 3-17.
Zimmerman, B. J. (1998). Academic studying and the development of
personal skill: A self-regulatory perspective. Educational Psychologist 33(2/3). 73-
86.
Zimmerman, B. J., & Bandura, A. (1994). Impact of self-regulatory
influences on writing course attainment. American Educational Research Journal. 31.
845-862.
Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-
motivation for academic attainment: The role of self-efficacy beliefs and personal
goal setting. American Educational Research Journal. 29(3). 663-676.
Zimmerman, B. J., & Lebeau, R. B. (2000). A Commentary on Self-
Directed Learning. In D. H. Evensen & C. E. Hmelo (Eds.), Problem-Based
Learning: A Research Perspective on Learning Interactions (pp. 299-313). Mahwah,
NJ: Lawrence Erlbaum Associates.
Zimmerman, B. J., & Martinez-Pons, M. (1986). Devleopment of a
structured interview for assessing student use of self-regulated learning strategies.
American Education Research Journal. 23. 614-628.
Zimmerman, B. J., & Martinez-Pons, M. (1988). Construct validation
of a strategy model of student self-regulated learning. Journal of Educational
Psychology. 80(3). 284-290.
Zimmerman, B. J., & Martinez-Pons, M. (1990). Student differences
in self-regulated learning: Relating grade, sex, and giftedness to self-efficacy and
strategy use. Journal of Educational Psychology. 82(1). 51-59.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
251
APPENDIX A
Rating and Summary Form Used for Noil-statistical Meta-analysis on PEL and SDL
Research in Post-graduate Medical Training
I. Citation_Ozuah, Curtis, & Ruth; OVERALL SCORE 57
Arch Pediatr Adolesc Med.2001;155: 669-672 (127 points possible)
Internal Validity Rating Categories
II. Type of Study
_____ A. Experimental (40)
(Pretest-Posttest Control Group Design, Solomon Four-Group Design)
B. Quasi-experimental (Variable)
X Nonequivalent Control Group Design (30)
Posttest-Only Design with Nonequivalent Groups (20)
i . ____ Used pretest substitute measures (3)
ii . ____ Population drawn from single institution (2)
_ _ _ C. Pre-experimental (10)
(One-Group Pretest-Posttest Design, One-Shot Case Study)
D. Observational (5)
(Causal-Comparative, Correlational)
III. Study Group Comparison
A. Respondents/non-respondents compared on > 3 demographic
variables (10)
B. Respondents/non-respondents compared on 2 demographic variables (5)
X _C. Respondents/non-respondents compared on 1 demographic variable or not
reported (0)
Construct Validity Rating Categories
IV. PBL/SDL Conceptual Description
A. Both PBL and SDL adequately defined (15)
X B. Either PBL or SDL adequately defined (8) [PBL]
C. Both PBL and SDL partially defined (8)
D. Either PBL or SDL partially defined (4)
E. PBL and SDL given cursory or no definition (0) [SDL]
V. PBL and SDL Theory or Model Base
A. Theory or model extensively presented (15)
B. Theory or model partially presented (10)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
252.
C. Theory or model superficially presented or implied (5)
X D. Hypothesis present or implied (2)
VI. SDL Operationalization
A. SDL operationalized as both process and product measures and
operationalized measures highly correlated with process and product
measures proposed in Figure 1 model (15)
B. SDL operationalized as both process and product measures and
operationalized measures somewhat correlated with process and product
measures proposed in Figure 1 model (7)
X C. SDL operationalized as either process or product measures and
operationalized measures highly correlated with either process or product
measures proposed in Figure 1 model (5)
D. SDL operationalized as either process or product measures and
operationalized measures somewhat correlated with either process or
product measures proposed in Figure 1 model (2)
E. SDL not adequately operationalized (0)
X _F. > 3 measures used to operationalize SDL (5)
_____ 2 measures used to operationalize SDL (2)
0-1 measure used to operationalize SDL (0)
Statistical Validity Rating Categories
VII. Statistical Power/Sample Size
A . > 98 total subjects (10)
X B. > 64 and < 97 total subjects (5) [80]
C. < 64 total subjects (0)
VIII. Instrument R eliab ility alid ity Measures
A. Explicit instrument reliability/validity measures reported (e.g., alpha
coefficient) (10)
B. Data triangulation attempted (10)
__C. Instrument reliability/validity measures suggested/inferred (5)
X D. Instrument pretested (2)
JE. Instrument reliability/validity indices not discussed (0)
External Validity R ating Categories
IX. Institution Demographics/Descriptors
A . > 3 institutional demographic or descriptors reported (5)
B. 2 institutional demographic or descriptors reported (2)
_ X _ C . 0-1 institutional demographic or descriptor reported (0)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
253
APPENDIX A (CONTINUED)
Rating and Summary Form Used for Non-statistical Meta-anaiysis on PBL and SDL
Research in Post-graduate Medical Training
I. Citation: Peters, Greenberger-Rosovsky, OVERALL SCORE 76_
Crowder, Block & Moore; Acad Med. (127 points possible)
2Q0Q;75: 470-479
Internal Validity Rating Categories
II. Type of Study
X_ A. Experimental (40)
(Pretest-Posttest Control Group Design, Solomon Four-Group Design)
_____ B. Quasi-experimental (Variable)
Nonequivalent Control Group Design (30)
Posttest-Only Design with Nonequivalent Groups (20)
i . _____Used pretest substitute measures (3)
ii . Population drawn from single institution (2)
C. Pre-experimental (10)
(One-Group Pretest-Posttest Design, One-Shot Case Study)
D. Observational (5)
(Causal-Comparative, Correlational)
III. Study Group Comparison
X A. Respondents/non-respondents compared on > 3 demographic variables (10)
[graduating class; program; gender; chosen specialty]
B. Respondents/non-respondents compared on 2 demographic variables (5)
C. Respondents/non-respondents compared on 1 demographic variable or not
reported (0)
C onstruct Validity R ating Categories
IV. PBL/SDL Conceptual Description
A. Both PBL and SDL adequately defined (15)
B. Either PBL or SDL adequately defined (8)
C. Both PBL and SDL partially defined (8)
D. Either PBL or SDL partially defined (4)
X E. PBL and SDL given cursory or no definition (0)
V. PBL and SDL Theory or Model Base
_ A. Theory or model extensively presented (15)
B. Theory or model partially presented (10)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
254
C. Theory or model superficially presented or implied (5)
X D. Hypothesis present or implied (2)
VI. SDL Operationalization
__ A. SDL operationalized as both process and product measures and
operationalized measures highly correlated with process and product
measures proposed in Figure 1 model (15)
B. SDL operationalized as both process and product measures and
operationalized measures somewhat correlated with process and product
measures proposed in Figure 1 model (7)
X C. SDL operationalized as either process or product measures and
operationalized measures highly correlated with either process or product
measures proposed in Figure 1 model (5)
D. SDL operationalized as either process or product measures and
operationalized measures somewhat correlated with either process or
product measures proposed in Figure 1 model (2)
E . SDL not adequately operationalized (0)
X _F. > 3 measures used to operationalize SDL (5)
2 measures used to operationalize SDL (2)
0-1 measure used to operationalize SDL (0)
Statistical Validity Rating Categories
VII. Statistical Power/Sample Size
X A. > 98 total subjects (10) [100]
B. > 64 and < 97 total subjects (5)
C. < 64 total subjects (0)
VIII. Instrument Reliability/V alidity Measures
_A . Explicit instrument reliability/validity measures reported (e.g., alpha
coefficient) (10)
B. Data triangulation attempted (10)
C. Instrument reliability/validity measures suggested/inferred (5)
X D. Instrument pretested (2)
E. Instrument reliability/validity indices not discussed (0)
External Validity Rating Categories
IX. Institution Demographics/Descriptors
A . > 3 institutional demographic or descriptors reported (5)
X B. 2 institutional demographic or descriptors reported (2)
C. 0-1 institutional demographic or descriptor reported (0)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
255
APPENDIX A (CONTINUED)
Rating and Summary Form Used for Non-statistical Meta-anaiysis on PBL and SDL
Research in Post-graduate Medical Training
I. Citation: Santos-Gomez, Kalishman, Rezler, OVERALL SCORE 51__
Skipper & Mennin; Med Ed. 1990; 24: 366-375 (127 points possible)
Internal Validity R ating Categories
II. Type of Study
_ _ _ A. Experimental (40)
(Pretest-Posttest Control Group Design, Solomon Four-Group Design)
_ _ _ _ B. Quasi-experimental (Variable)
Nonequivalent Control Group Design (30)
X Posttest-Only Design with Nonequivalent Groups (20)
i . Used pretest substitute measures (3)
ii. X Population drawn from single institution (2)
C. Pre-experimental (10)
(One-Group Pretest-Posttest Design, One-Shot Case Study)
D. Observational (5)
(Causal-Comparative, Correlational)
III. Study Group Comparison
A. Respondents/non-respondents compared on > 3 demographic
variables (10)
X B. Respondents/non-respondents compared on 2 demographic variables (5)
[gender; year of residency]
C. Respondents/non-respondents compared on 1 demographic variable or not
reported (0)
Construct Validity R ating Categories
IV. PBL/SDL Conceptual Description
A. Both PBL and SDL adequately defined (15)
6. Either PBL or SDL adequately defined (8)
C. Both PBL and SDL partially defined (8)
D. Either PBL or SDL partially defined (4)
X E. PBL and SDL given cursory or no definition (0)
V. PBL and SDL Theory or Model Base
A. Theory or model extensively presented (15)
B. Theory or model partially presented (10)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
256
C. Theory or model superficially presented or implied (5)
X D. Hypothesis present or implied (2)
VI. SDL Operationalization
A. SDL operationalized as both process and product measures and
operationalized measures highly correlated with process and product
measures proposed in Figure 1 model (15)
B. SDL operationalized as both process and product measures and
operationalized measures somewhat correlated with process and product
measures proposed in Figure 1 model (7)
C. SDL operationalized as either process or product measures and
operationalized measures highly correlated with either process or product
measures proposed in Figure 1 model (5)
D . SDL operationalized as either process or product measures and
operationalized measures somewhat correlated with either process or
product measures proposed in Figure 1 model (2)
X E. SDL not adequately operationalized (0)
F. > 3 measures used to operationalize SDL (5)
_____ 2 measures used to operationalize SDL (2)
0-1 measure used to operationalize SDL (0)
Statistical Validity Rating Categories
VII. Statistical Power/Sample Size
X A. > 98 total subjects (10) [101 with complete returns (3 raters)/185 total]
B. > 64 and < 97 total subjects (5)
C. < 64 total subjects (0)
VIII. Instrument Reliability/V alidity Measures
A . Explicit instrument reliability/validity measures reported (e.g., alpha
coefficient) (10)
X 6. Data triangulation attempted (10)
C. Instrument reliability/validity measures suggested/inferred (5)
X D. Instrument pretested (2)
E. Instrument reliability/validity indices not discussed (0)
External Validity Rating Categories
IX. Institution Demographics/Descriptors
A . > 3 institutional demographic or descriptors reported (5)
B. 2 institutional demographic or descriptors reported (2)
X C. 0-1 institutional demographic or descriptor reported (0)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
257
APPENDIX A (CONTINUED)
Rating and Summary Form Used for Non-statistical Meta-analysis on PBL and SDL
Research in Post-graduate Medical Training
I. Citation: Shin, Haynes & Johnson; OVERALL SCORE 55
Can Med Assoc J. 1993; 148: 969-976 (127 points possible)
Internal Validity R ating Categories
II. Type of Study
_____ A. Experimental (40)
(Pretest-Posttest Control Group Design, Solomon Four-Group Design)
_____ B. Quasi-experimental (Variable)
Nonequivalent Control Group Design (30)
X Posttest-Qnly Design with Nonequivalent Groups (20)
i. Used pretest substitute measures (3)
ii . _____ Population drawn from single institution (2)
C. Pre-experimental (10)
(One-Group Pretest-Posttest Design, One-Shot Case Study)
D. Observational (5)
(Causal-Comparative, Correlational)
III. Study Group Comparison
X A. Respondents/non-respondents compared on > 3 demographic
variables (10) [several]
B. Respondents/non-respondents compared on 2 demographic variables (5)
C. Respondents/non-respondents compared on 1 demographic variable or not
reported (0)
Construct Validity Rating Categories
IV. PBL/SDL Conceptual Description
A. Both PBL and SDL adequately defined (15)
6. Either PBL or SDL adequately defined (8)
X C. Both PBL and SDL partially defined (8)
D. Either PBL or SDL partially defined (4)
E. PBL and SDL given cursory or no definition (0)
V. PBL and SDL Theory or Model Base
A. Theory or model extensively presented (15)
B. Theory or model partially presented (10)
C. Theory or model superficially presented or implied (5)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
258
X D. Hypothesis present or implied (2)
V I SDL Operationalization
A. SDL operationalized as both process and product measures and
operationalized measures highly correlated with process and product
measures proposed in Figure 1 model (15)
6 . SDL operationalized as both process and product measures and
operationalized measures somewhat correlated with process and product
measures proposed in Figure 1 model (7)
C. SDL operationalized as either process or product measures and
operationalized measures highly correlated with either process or product
measures proposed in Figure 1 model (5)
_ D. SDL operationalized as either process or product measures and
operationalized measures somewhat correlated with either process or
product measures proposed in Figure 1 model (2)
X E. SDL not adequately operationalized (0)
F. > 3 measures used to operationalize SDL (5)
2 measures used to operationalize SDL (2)
_ _ _ 0-1 measure used to operationalize SDL (0)
Statistical Validity R ating Categories
VII. Statistical Power/Sample Size
A. > 98 total subjects (10)
X 6. > 64 and < 97 total subjects (5) [82]
_C . < 64 total subjects (0)
VIII. Instrument R eliab ility alid ity Measures
X A. Explicit instrument reliability/validity measures reported (e.g., alpha
coefficient) (10)
B. Data triangulation attempted (10)
_ C. Instrument reliability/validity measures suggested/inferred (5)
D. Instrument pretested (2)
E. Instrument reliability/validity indices not discussed (0)
External Validity R ating Categories
IX. Institution Demographics/Descriptors
A. > 3 institutional demographic or descriptors reported (5)
B. 2 institutional demographic or descriptors reported (2)
X C. 0-1 institutional demographic or descriptor reported (0)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
259
APPENDIX A (CONTINUED)
Rating and Summary Form Used for Non-statistical Meta-analysis on PBL and SDL
Research in Post-graduate Medical Training
I. Citation: Mennin, Kalishman, Friedman, OVERALL S C O R E _ 7 3 _
Pathak & Snyder; Acad. Med. 1996: (127 points possible)
71: 1079-1089
Internal V alidity Rating Categories
II. Type of Study
A. Experimental (40)
(Pretest-Posttest Control Group Design, Solomon Four-Group Design)
B. Quasi-experimental (Variable)
N onequivalent Control Group Design (30)
X Posttest-Only Design with Nonequivalent Groups (20)
i. Used pretest substitute measures (3)
ii. X Population drawn from single institution (2)
C. Pre-experimental (10)
(One-Group Pretest-Posttest Design, One-Shot Case Study)
D. Observational (5)
(Causal-Comparative, Correlational)
III. Study Group Comparison
X A. Respondents/non-respondents compared on > 3 demographic
variables (10) [age; gender; marital status]
B. Respondents/non-respondents compared on 2 demographic variables (5)
C. Respondents/non-respondents compared on 1 demographic variable or not
reported (0)
C onstruct Validity R ating Categories
IV. PBL/SDL Conceptual Description
A. Both PBL and SDL adequately defined (15)
B. Either PBL or SDL adequately defined (8)
_ C. Both PBL and SDL partially defined (8)
D. Either PBL or SDL partially defined (4)
X E. PBL and SDL given cursory or no definition (0)
V. PBL and SDL Theory or Model Base
A. Theory or model extensively presented (15)
B. Theory or model partially presented (10)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
260
C. Theory or model superficially presented or implied (5)
X D. Hypothesis present or implied (2)
VI. SDL Operationalization
A. SDL operationalized as both process and product measures and
operationalized measures highly correlated with process and product
measures proposed in Figure 1 model (15)
B . SDL operationalized as both process and product measures and
operationalized measures somewhat correlated with process and product
measures proposed in Figure 1 model (7)
X C. SDL operationalized as either process or product measures and
operationalized measures highly correlated with either process or product
measures proposed in Figure 1 model (5)
D. SDL operationalized as either process or product measures and
operationalized measures somewhat correlated with either process or
product measures proposed in Figure 1 model (2)
E. SDL not adequately operationalized (0)
X _F. > 3 measures used to operationalize SDL (5)
2 measures used to operationalize SDL (2)
_____ 0-1 measure used to operationalize SDL (0)
Statistical Validity Rating Categories
VII. Statistical Power/Sample Size
_ _ X _ A . > 98 total subjects (10) [120]
B. > 64 and < 97 total subjects (5)
_C. < 64 total subjects (0)
VIII. Instrument Reliability/V aiidity Measures
A. Explicit instrument reliability/validity measures reported (e.g., alpha
coefficient) (10)
B. Data triangulation attempted (10)
X C. Instrument reliability/validity measures suggested/inferred (5)
X D. Instrument pretested (2)
E. Instrument reliability/validity indices not discussed (0)
External Validity R ating Categories
IX. Institution Demographics/Descriptors
___ _A. > 3 institutional demographic or descriptors reported (5)
X B. 2 institutional demographic or descriptors reported (2)
C. 0-1 institutional demographic or descriptor reported (0)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
261
APPENDIX A (CONTINUED)
Rating and Summary Form Used for Mon-statistical Meta-analysis on PBL and SDL
Research in Post-graduate Medical Training
I. Citation: Tolnai; Med Ed. 1991; OVERALL SCO RE_62___
25:414-420 (127 points possible)
Internal Validity Rating Categories
II. Type of Study
A. Experimental (40)
(Pretest-Posttest Control Group Design, Solomon Four-Group Design)
B. Quasi-experimental (Variable)
Nonequivalent Control Group Design (30)
X Posttest-Only Design with Nonequivalent Groups (20)
i . Used pretest substitute measures (3)
ii. Population drawn from single institution (2)
C. Pre-experimental (10)
(One-Group Pretest-Posttest Design, One-Shot Case Study)
_____ D. Observational (5)
(Causal-Comparative, Correlational)
III. Study Group Comparison
X A. Respondents/non-respondents compared on > 3 demographic
variables (10) [geo. location; gender; solo vs. group practice; field of
practice; specialty certification]
B. Respondents/non-respondents compared on 2 demographic variables (5)
C. Respondents/non-respondents compared on 1 demographic variable or not
reported (0)
C onstruct Validity Rating Categories
IV. PBL/SDL Conceptual Description
A. Both PBL and SDL adequately defined (15)
B. Either PBL or SDL adequately defined (8)
C. Both PBL and SDL partially defined (8)
D. Either PBL or SDL partially defined (4)
X E. PBL and SDL given cursory or no definition (0)
V. PBL and SDL Theory or Model Base
A. Theory or model extensively presented (15)
B. Theory or model partially presented (10)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
262
C. Theory or model superficially presented or implied (5)
X D. Hypothesis present or implied (2)
VI. SDL Operationalization
A. SDL operationalized as both process and product measures and
operationalized measures highly correlated with process and product
measures proposed in Figure 1 model (15)
_B. SDL operationalized as both process and product measures and
operationalized measures somewhat correlated with process and product
measures proposed in Figure 1 model (7)
X C. SDL operationalized as either process or product measures and
operationalized measures highly correlated with either process or product
measures proposed in Figure 1 model (5)
D. SDL operationalized as either process or product measures and
operationalized measures somewhat correlated with either process or
product measures proposed in Figure 1 model (2)
E, SDL not adequately operationalized (0)
X _F. > 3 measures used to operationalize SDL (5)
_ _ _ 2 measures used to operationalize SDL (2)
0-1 measure used to operationalize SDL (0)
Statistical Validity Rating Categories
VII. Statistical Power/Sample Size
X A. > 98 total subjects (10) [342]
_B. > 64 and < 97 total subjects (5)
C. < 64 total subjects (0)
VIII. Instrument Reliability/Validity Measures
A. Explicit instrument reliability/validity measures reported (e.g., alpha
coefficient) (10)
B. Data triangulation attempted (10)
_C. Instrument reliability/validity measures suggested/inferred (5)
D. Instrument pretested (2)
X E. Instrument reliability/validity indices not discussed (0)
External Validity Rating Categories
IX. Institution Demographics/Descriptors
A. > 3 institutional demographic or descriptors reported (5)
B. 2 institutional demographic or descriptors reported (2)
X C. 0-1 institutional demographic or descriptor reported (0)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
263
APPENDIX A (CONTINUED)
Rating and Summary Form Used for Non-statistical Meta-analysis on PBL and SDL
Research in Post-graduate Medical Training
Internal Validity Rating Categories
II. Type of Study
_____ A. Experimental (40)
(Pretest-Posttest Control Group Design, Solomon Four-Group Design)
_ _____ B. Quasi-experimental (Variable)
Nonequivalent Control Group Design (30)
X Posttest-Only Design with Nonequivalent Groups (20)
i. X Used pretest substitute measures (3) [“...statistically
similar applicant pools and similarities on all
admission characteristics”]
ii. X Population drawn from single institution (2)
C. Pre-experimental (10)
(One-Group Pretest-Posttest Design, One-Shot Case Study)
D. Observational (5)
(Causal-Comparative, Correlational)
III. Study Group Comparison
X A. Respondents/non-respondents compared on > 3 demographic
variables (10) [gender; grades; basic science knowledge; course opinion
quest.; clerkship OSCE; MCCQE Parts I and II]
B. Respondents compared on 2 demographic variables (5)
_ C. Respondent comparison on 1 demographic variable or not reported (0)
Construct Validity Rating Categories
IV. PBL/SDL Conceptual Description
A. Both PBL and SDL adequately defined (15)
B. Either PBL or SDL adequately defined (8)
C. Both PBL and SDL partially defined (8)
D. Either PBL or SDL partially defined (4)
X E. PBL and SDL given cursory or no definition (0)
V. PBL and SDL Theory or Model Base
A. Theory or model extensively presented (15)
I. Citation: Mann & Kaufman; Acad Med.
1999; 74: S4-S6
OVERALL SCORE 56
(127 points possible)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
264
B. Theory or model partially presented (10)
C. Theory or model superficially presented or implied (5)
X D. Hypothesis present or implied (2)
VI. SDL Operationalization
A. SDL operationalized as both process and product measures and
operationalized measures highly correlated with process and product
measures proposed in Figure 1 model (15)
B. SDL operationalized as both process and product measures and
operationalized measures somewhat correlated with process and product
measures proposed in Figure 1 model (7)
C. SDL operationalized as either process or product measures and
operationalized measures highly correlated with either process or product
measures proposed in Figure 1 model (5)
D. SDL operationalized as either process or product measures and
operationalized measures somewhat correlated with either process or
product measures proposed in Figure 1 model (2)
X E. SDL not adequately operationalized (0)
F. > 3 measures used to operationalize SDL (5)
2 measures used to operationalize SDL (2)
______ 0-1 measure used to operationalize SDL (0)
Statistical Validity Rating Categories
VII. Statistical Power/Sample Size
A. > 98 total subjects (10)
X 6 . > 64 and < 97 total subjects (5) [96]
C. < 64 total subjects (0)
VIII. Instrument R eliabilityalidity Measures
X A. Explicit instrument reliability/validity measures reported (e.g., alpha
coefficient) (10) [alpha coefficient for subscale including SDL = .85]
B . Data triangulation attempted (10)
C. Instrument reliability/validity measures suggested/inferred (5)
X D. Instrument pretested (2)
E. Instrument reliability/validity indices not discussed (0)
External Validity Rating Categories
IX. Institution Demographics/Descriptors
A. > 3 institutional demographic or descriptors reported (5)
X B. 2 institutional demographic or descriptors reported (2)
C. 0-1 institutional demographic or descriptor reported (0)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
265
APPENDIX A (CONTINUED)
Rating and Summary Form Used for Non-statisticai Meta-analysis on PBL and SDL
Research in Post-graduate Medical Training
I. Citation: Hill, Rolfe, Pearson OVERALL SCORE 70__
& Heathcote; Med Ed. 1998; 32: 19-24 (127 points possible)
Internal Validity Rating Categories
II. Type of Study
A. Experimental (40)
(Pretest-Posttest Control Group Design, Solomon Four-Group Design)
B. Quasi-experimental (Variable)
Nonequivalent Control Group Design (30)
X Posttest-Only Design with Nonequivalent Groups (20)
i . Used pretest substitute measures (3)
ii. _ Population drawn from single institution (2)
_ __ _ C. Pre-experimental (10)
(One-Group Pretest-Posttest Design, One-Shot Case Study)
D. Observational (5)
(Causal-Comparative, Correlational)
III. Study Group Comparison
A. Respondents/non-respondents compared on > 3 demographic
variables (10)
X B. Respondents/non-respondents compared on 2 demographic variables (5)
[covariates - age; gender]
C. Respondents/non-respondents compared on 1 demographic variable or not
reported (0)
C onstruct Validity R ating Categories
IV. PBL/SDL Conceptual Description
A. Both PBL and SDL adequately defined (15)
B. Either PBL or SDL adequately defined (8)
C. Both PBL and SDL partially defined (8)
X D. Either PBL or SDL partially defined (4) [SDL]
E. PBL and SDL given cursory or no definition (0)
V. PBL and SDL Theory or Model Base
A. Theory or model extensively presented (15)
B. Theory or model partially presented (10)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
266
C. Theory or model superficially presented or implied (5)
X D. Hypothesis present or implied (2)
VI. SDL Operationalization
_A . SDL operationalized as both process and product measures and
operationalized measures highly correlated with process and product
measures proposed in Figure 1 model (15)
X B. SDL operationalized as both process and product measures and
operationalized measures somewhat correlated with process and product
measures proposed in Figure 1 model (7)
C. SDL operationalized as either process or product measures and
operationalized measures highly correlated with either process or product
measures proposed in Figure 1 model (5)
D. SDL operationalized as either process or product measures and
operationalized measures somewhat correlated with either process or
product measures proposed in Figure 1 model (2)
E. SDL not adequately operationalized (0)
X JF. > 3 measures used to operationalize SDL (5)
2 measures used to operationalize SDL (2)
_ _ _ 0-1 measure used to operationalize SDL (0)
Statistical Validity Rating Categories
VII. Statistical Power/Sample Size
X A. > 98 total subjects (10) [139]
B . > 64 and < 97 total subjects (5)
C. < 64 total subjects (0)
VIII. Instrument R eliab ility alid ity Measures
X A. Explicit instrument reliability/validity measures reported (e.g., alpha
coefficient) (10) [alpha coefficient for SDL subscale = 0.82]
B. Data triangulation attempted (10)
C. Instrument reliability/validity measures suggested/inferred (5)
X D. Instrument pretested (2)
E. Instrument reliability/validity indices not discussed (0)
External Validity R ating Categories
IX. Institution Demographics/Descriptors
X A. > 3 institutional demographic or descriptors reported (5)
6. 2 institutional demographic or descriptors reported (2)
C. 0-1 institutional demographic or descriptor reported (0)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
267
APPENDIX A (CONTINUED)
Rating and Summary Form Used for Non-statistical Meta-analysis on PEL and SDL
Research in Post-graduate Medical Training
I. Citation: Rolfe, Andren, Pearson, Hensley OVERALL SCORE 49
& Gordon; Med Ed. 1995; 29: 225-230 (127 points possible)
In tern al Validity Rating Categories
II. Type of Study
_ _ _ A. Experimental (40)
(Pretest-Posttest Control Group Design, Solomon Four-Group Design)
_____ B. Quasi-experimental (Variable)
Nonequivalent Control Group Design (30)
X Posttest-Only Design with Nonequivalent Groups (20)
i. Used pretest substitute measures (3)
ii . ____ Population drawn from single institution (2)
C. Pre-experimental (10)
(One-Group Pretest-Posttest Design, One-Shot Case Study)
_____ D. Observational (5)
(Causal-Comparative, Correlational)
III. Study Group Comparison
A. Respondents/non-respondents compared on > 3 demographic
variables (10)
X B. Respondents/non-respondents compared on 2 demographic variables (5)
[gender; age]
C. Respondents/non-respondents compared on 1 demographic variable or not
reported (0)
C onstruct Validity R ating Categories
IV. PBL/SDL Conceptual Description
A. Both PBL and SDL adequately defined (15)
B. Either PBL or SDL adequately defined (8)
C. Both PBL and SDL partially defined (8)
D. Either PBL or SDL partially defined (4)
X E. PBL and SDL given cursory or no definition (0)
V. PBL and SDL Theory or Model Base
A. Theory or model extensively presented (15)
B. Theory or model partially presented (10)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
268
C. Theory or model superficially presented or implied (5)
X D. Hypothesis present or implied (2)
VI. SDL Operationalization
_ A . SDL operationalized as both process and product measures and
operationalized measures highly correlated with process and product
measures proposed in Figure 1 model (15)
6. SDL operationalized as both process and product measures and
operationalized measures somewhat correlated with process and product
measures proposed in Figure 1 model (7)
C. SDL operationalized as either process or product measures and
operationalized measures highly correlated with either process or product
measures proposed in Figure 1 model (5)
D. SDL operationalized as either process or product measures and
operationalized measures somewhat correlated with either process or
product measures proposed in Figure 1 model (2)
X E. SDL not adequately operationalized (0)
F. > 3 measures used to operationalize SDL (5)
_ 2 measures used to operationalize SDL (2)
0-1 measure used to operationalize SDL (0)
Statistical Validity Rating Categories
VII. Statistical Power/Sample Size
X A. > 98 total subjects (10) [486]
B. > 64 and < 97 total subjects (5)
C. < 64 total subjects (0)
VIII. Instrument Reliability/V alidity Measures
A. Explicit instrument reliability/validity measures reported (e.g., alpha
coefficient) (10)
B. Data triangulation attempted (10)
C. Instrument reliability/validity measures suggested/inferred (5)
X D. Instrument pretested (2)
E. Instrument reliability/validity indices not discussed (0)
External Validity Rating Categories
IX. Institution Demographics/Descriptors
A. > 3 institutional demographic or descriptors reported (5)
B. 2 institutional demographic or descriptors reported (2)
X C. 0-1 institutional demographic or descriptor reported (0)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Development and validation of the Cooper Quality of Imagery Scale: A measure of vividness of sporting mental imagery

PDF

Gender role conflict, personality, and help -seeking in adult men

PDF

A case study: The application of learning theory to teaching logic

PDF

A comparative study of American, Australian, British, and Canadian museum visitors' understanding of the nature of evolutionary theory

PDF

An evaluation of select athletic training education programs administration and educational structure

PDF

An evaluation of perceived task value, self-efficacy, and performance in a geography blended distance course

PDF

A pilot study to investigate the relationship between student self -regulatory resource management strategies and academic achievement in a Web-based hybrid graduate nursing course

PDF

Can individual creativity be enhanced by training? A meta analysis

PDF

Districtwide instructional improvement: A case study of Algiers Elementary School in the Orchard Unified School District

PDF

Commitment to deliberate practice in the development of writing expertise: A survey of the motivational patterns of academic writers in Tier One research universities

PDF

Cognitive efficiency of animated pedagogical agents for learning English as a second language

PDF

Implementing and assessing problem -based learning in non -traditional post -secondary aviation safety curricula: A case study

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in Los Coyotes High School District

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Media Unified School District

PDF

College students' attributions about non -condom use

PDF

Gender differences in symptom presentation of depression in primary care settings

PDF

Inaccuracies in expert self -report: Errors in the description of strategies for designing psychology experiments

PDF

Comparing cognitive task analysis to behavior task analysis in training first year interns to place central venous catheters

PDF

Factors that influence students' investment of mental effort in academic tasks: A validation and exploratory study

PDF

A phenomenological inquiry into the essential meanings of the most intense experience of religiosity and spirituality

Asset Metadata

Creator Tsoulas, Karen D. (author)

Core Title Does problem -based learning in medical education lead to more self -directed learning by physicians? A review of comparative studies, theory and analysis

Contributor Digitized by ProQuest (provenance)

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, educational psychology,Health Sciences, Education,OAI-PMH Harvest

Language English

Advisor Clark, Richard (committee chair), Abbott, Allan (committee member), Hoffman, Kaaren (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-497135

Unique identifier UC11340138

Identifier 3133345.pdf (filename),usctheses-c16-497135 (legacy record id)

Legacy Identifier 3133345.pdf

Dmrecord 497135

Document Type Dissertation

Rights Tsoulas, Karen D.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA