University of Southern California Dissertations and Theses

Problem-based learning in a dental school: measuring change in students' critical thinking skills

(USC Thesis Other)

Problem-based learning in a dental school: measuring change in students' critical thinking skills

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

PROBLEM-BASED LEARNING IN A DENTAL SCHOOL:

MEASURING CHANGE IN STUDENTS’ CRITICAL THINKING SKILLS

by

Bens Pardamean

A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

December 2007

Copyright 2007 Bens Pardamean
ii
DEDICATION
I would like to express a special dedication to my late parents, who
relentlessly stressed the importance of education.

iii
ACKNOWLEDGEMENTS
I wish to dedicate this dissertation to my wife, Angy, and my daughter/
editor, Carissa. Their unwavering love, continuous encouragement, and endless
support have made this dissertation possible. Additionally, a special accolade goes to
Algy, our beloved tiny entertainer and stress reliever.
I wish to thank you my dissertation committee. My deepest appreciation goes
to my committee chairperson, Dr. Robert Keim, for his leadership, mentorship, and
understanding. You have made possible for this dissertation to complete in a
relatively short amount of time. My special gratitude also goes to the members of my
committee: Dr. Dennis Hocevar and Dr. Simeon Slovacek, for their advice, insightful
feedbacks, and wisdoms.
The professional opportunity granted to me by Dr. Stanley Azen during my
doctoral program deserves special acknowledgment. His extended support was an
instrumental factor in making the pursuit of this degree possible.
My appreciation extends to my sister, Maya, and her husband, Robert, for
being a kind host for many of my stays after long-distance commutes during the
writing of this dissertation. And lastly, I express my thanks to my humble cousin,
Daniel, for always keeping things in perspective for me.

iv
TABLE OF CONTENTS
DEDICATION ii

ACKNOWLEDGEMENTS iii

LIST OF TABLES vi

LIST OF FIGURES vii

ABSTRACT viii

CHAPTER ONE: OVERVIEW OF THE STUDY 1
Problem-Based Learning 1
Critical Thinking 3
Statement of the Problem 5
Purpose of the Study 6
Research Questions 7
Hypothesis 8
Significance of the Study 8

CHAPTER TWO: LITERATURE REVIEW 9
Introduction 9
Concepts of Problem-Based Learning 10
Definition and Process of Problem-Based Learning 13
Problem-Based Learning and Cognitive-Structural Development Theories 16
Piaget’s schemata and Perry’s positions 17
Epistemological reflection framework 19
Reflective judgment model 20
Problem-Based Learning and Typology Theories 23
Kolb’s learning styles theory 23
Self-directed learning 26
Self-regulated learning 28
Effectiveness of Problem-Based Learning 29
Learning achievement 31
Attitudes and feedback 34
Critical Thinking Skills 36
Effects of Problem-Based Learning on Critical Thinking Skills 38
Tests for Assessing the Critical Thinking Skills 40
The Health Sciences Reasoning Test (HSRT) 46
Implications 48

v
CHAPTER THREE: METHODOLOGY 52
Introduction 52
Study Design 53
Sample and Population 53
Instrumentation and Data Collection Procedures 54
Validity and Reliability 55
Data Analysis and Reporting Procedures 56
Ethical Consideration 57
Limitations of the Study 58

CHAPTER FOUR: RESULTS 59
Introduction 59
Participants 60
Hypothesis 61
Descriptive Analysis of Research Findings 61

CHAPTER FIVE: DISCUSSION 76
Summary 76
Hypothesis 76
Rationale of the Study 77
Results 79
Discussions 82
Limitations of Study 85
Recommendations for Further Study 87

REFERENCES 90

APPENDICES
APPENDIX A: THE HEALTH SCIENCES REASONING TEST 99
APPENDIX B: CAPSCORE™ RESPONSE FORM 108
APPENDIX C: DEMOGRAPHICS/ACADEMIC QUESTIONNAIRE 109

vi
LIST OF TABLES
Table 1: HSRT Sub-scales and KR-20 Internal Consistency Coefficients 47

Table 2: CCTST Correlations with Other Measures 47

Table 3: Study Participants 60

Table 4: Demographics Description of Study Participants 61

Table 5: Descriptive Statistics of Study Participants’ Age and Academic Data 62

Table 6: Statistics of HSRT Total Score for Entire Study Participants 63

Table 7: Rubric of HSRT Delphi Sub-scales 64

Table 8: Mean (SD) and ANOVA of HSRT Total Score and Sub-scales Scores 65

Table 9: HSRT Total Score Based on Demographics Data 70

Table 10: Summary of Mean HSRT Sub-scales Scores in Contemporary
Category Based on Demographics Data 71

Table 11: Summary of Mean HSRT Sub-scales Scores in Classical Category
Based on Demographics Data 72

Table 12: ANCOVA Results for HSRT Total Score and Sub-scales Scores 73

Table 13: Correlations Between HSRT Scores and Academic Data 74

vii
LIST OF FIGURES
Figure 1: Frequency Distribution Histogram of HSRT Total Score 63

Figure 2: Mean HSRT Total Score 67

Figure 3: Mean HSRT Sub-scales Scores in Contemporary Category 68

Figure 4: Mean HSRT Sub-scales Scores in Classical Category 69

viii
ABSTRACT
This study measured the change of critical thinking skills (CTS) of dental
students in a problem-based learning (PBL) pedagogical method. The quantitative
analysis was focused on measuring students’ CTS achievement starting from their
first- through third-year of dental education at the University of Southern California.
This non-experimental study used a cross-sectional design. Evaluation was
based on a volunteer sample of 98 dental students who completed a demographics/
academic questionnaire and a psychometric assessment known as the Health
Sciences Reasoning Test (HSRT) that consisted of 33-item multiple choice format
test. The HSRT produced the overall CTS score or the total score. In addition, the
HSRT also generated five sub-scales scores relating to CTS: analysis, inference,
evaluation, deductive reasoning, and inductive reasoning. Both descriptive and
inferential statistical analyses were carried out.
The results of this study concluded that students showed no continuous and
significant incremental improvement in their overall CTS score achievement during
their PBL-based dental education. Except for the inductive reasoning score, this
result was very consistent with other four sub-scales scores as well. Moreover, after
performing the statistical adjustment (ANCOVA) on total score and sub-scales
scores, no significant statistical differences were found among the three student
groups. On the other hand, the results of this study also showed some aspects of
critical thinking achievements that differed by categories of gender, race, and
English as first language.
1
CHAPTER ONE
OVERVIEW OF THE STUDY
Problem-Based Learning
Problem-based learning (PBL) is an alternative pedagogical model, which
was first introduced to medical education at the McMaster University School of
Medicine in Ontario, Canada. Since its introduction in 1969, PBL has become an
integral part of medical curricula and teaching strategy around the world (Fincham &
Shuler, 2001; Newman, Ambrose, Corner, Evans, Morris-Vincent, Quinn,
Stephenson, & Vernon, 2003). Although the precise format of PBL implementation
varies, its main key features remain consistent: an emphasis on learner-centered
exploration of case-based patient problems and the utilization of patient case
histories to help students identify learning needs that become the focus of individual
and group problem solving (Kenny & Beagan, 2004). The general goal of PBL is “to
develop successful problem solving in two dimensions: the acquisition of knowledge
and the application of knowledge” (Gijbels, Dochy, van den Bossche, & Segers,
2005, p. 30). At the University of Southern California (USC), PBL curriculum has
been developed and implemented in the School of Dentistry since 1995. This new
PBL program “[offer] its first twelve students a different approach to the doctoral
dental curriculum, emphasizing integration of basic and clinical science through
inquiry based learning” (About USCSD, n.d.; Fincham & Shuler, 2001).
According to numerous medical educators (e.g., Albanese & Mitchell, 1993;
Barrows & Tamblyn, 1980; Boud, 1987; Norman & Schmidt, 1992), the theory
2
behind PBL is based upon the idea that learning occurs at its optimum when it is
associated with actual clinical problems. In USC dental education, PBL students
explore basic scientific knowledge by discussing and hypothesizing about clinical
problems. The patient case is selected for its relevance to the curriculum and for its
similarity to real life situations. Students are the center and the source of discussion
for all of the subsequent PBL activities. They must actively draw upon their prior
knowledge and experience in order to generate their learning ideas. Presentation of
case facts is followed by preliminary identification of the problems and a thorough
iteration of relevant factual information from the case. Accordingly, the students
produce hypotheses that can potentially explain the problem and identify their
learning needs. After researching and finding resource material, the students are then
challenged to validate and elaborate on their original thinking through a process of
testing hypotheses. They are also challenged to develop new hypotheses when they
discover gaps in knowledge. This inquiry-driven process of real life clinical case
examination contributes to meaningful knowledge generation and the building of a
structured approach to problem-solving (PBL Workshop Series, USC School of
Dentistry, February 2005).
Barrows (1986) suggests that active participations in PBL environment
produce longer-lasting impacts than the passive absorption of knowledge, which
occurs in non-PBL forms of pedagogy. Moreover, Zimmerman and Lebeau (2000)
observe that the student-centered instruction of PBL utilizes students’ prior
knowledge, thus, encouraging motivation for self-directed learning. According to
3
Albanese and Mitchell (1993), problem-based approaches also facilitate the
development of students’ collaborative skills without inhibiting individual
responsibility.
Medical students who are trained with a PBL method have exceptional
research assessment proficiency, resources utilization skills, as well as better
preparation to function effectively in an environment of ambiguity and uncertainty
(Barrows & Tamblyn, 1980; Dolmans, Wolfhagen, van der Vleuten, & Wijnen,
2001; Friedman & Deek, 2002; Major & Palmer, 2001; Nendaz & Tekian, 1999;
Patel, Groen, & Norman, 1991). These researchers also note that students who are
enrolled in PBL program demonstrate sound data collection skills and assessment
techniques. Furthermore, students are also able to organize the data into a concise,
problem-oriented record, carry out the analysis, and perform the presentation.
Many researchers (e.g., Bridges & Hallinger, 1997; Burch, 2000; Lloyd-
Jones, Margetson, & Bligh, 1998) find that medical students who are trained in a
PBL setting are able to demonstrate excellent interpersonal communication skills and
cultural awareness. These skills and awareness are necessary for facilitating patient
communication and understanding of medical problems.

Critical Thinking
Critical thinking can be defined as the process of determining the
authenticity, accuracy, and worth of information or knowledge claims. Norris and
Ennis assert that critical thinking is demonstrated by the individual ability to judge
4
the soundness of information, assess conclusions, and make good inferences
(Christen, Angermeyer, Davison, & Anderson, 1994). Critical thinking improves the
ability of students to make decisions or conclusions about problems that may not
always have a correct solution. Hence, according to Facione (2007), there are three
core critical thinking abilities: analyzing, drawing, and evaluating inferences.
In 1990, a panel of 46 experts from philosophy and education participated in
the American Philosophical Association’s Critical Thinking: A Statement of Expert
Consensus for Purposes of Educational Assessment and Instruction. They produced
the Delphi Report that defines critical thinking as a “purposeful, self-regulatory
judgment which results in interpretation, analysis, evaluation, and inference, as well
as explanation of the evidential, conceptual, methodological, criteriological, or
contextual considerations upon which that judgment is based” (Edman, Robey, &
Bart, 2002; Facione, 1990; Spicer & Hanks, 1995).
Rather than students being taught to gather and learn soon-to-be obsolete
information, they ought to learn how to become critical thinkers; that is, students
need to “learn how to learn.” This strategy has been integrated into the Department
of Education’s National Education Goals for the Year 2000. One of the goals is to
“assess the ability of college graduate to think critically, to communicate effectively,
and to solve problems ...” (NEGR, 1991).
At the USC School of Dentistry, through its PBL curriculum, students must
entail thoughtful decision-making based on their ability to make purposeful
judgments that involve analysis, interpretation, inference, evaluation, and
5
explanation (Fincham & Shuler, 2001). In short, students are expected to engage in
critical thinking. PBL dental students are taught these cognitive skills. Accordingly,
the dental program ought to show evidence that its students have developed the
critical thinking skills.

Statement of the Problem
Student assessment refers to the evaluation of students’ performance using
appropriate methods and proven testing instruments (Hsu, 1999). Many educators
and education leaders acknowledge and emphasize the importance of student
assessment on learning. However, not many studies have been conducted on the
methodology of assessment in a PBL approach. Furthermore, the results of some
studies on the cognitive advantages of PBL have shown mixed results (Hmelo,
Gotterer, Bransford, 1994; Hsu, 1999).
PBL students are assumed to be more capable in learning new knowledge due
to the activation of prior knowledge and reorganization of newly acquired
knowledge; thus, improving their critical thinking skills (Zimmerman & Lebeau,
2000). It is frequently implied that clinical PBL courses are expected to improve
students’ professional knowledge acquisition and critical thinking skills (Birgegard
& Lindquist, 1998; Maudsley & Strivens, 2000). PBL can also help students develop
advanced cognitive abilities, such as critical thinking, problem solving, and
interpersonal communication skills (Thomas, 1997).
6
Although the USC School of Dentistry has been implementing and practicing
PBL curriculum for the past ten years, there has been no assessment to measure the
critical thinking skills improvement among these dental students.

Purpose of the Study
One of the main goals of higher education is to develop and to enhance
critical thinking skills (Association of American Colleges, 1985). The Association
strongly recommends that students learn critical analytical skills, abstract logical
thinking skills, and inquiry skills. Accordingly, the purpose of this study is to assess
whether or not problem-based learning pedagogical method in dental education
improves critical thinking skills of the students. Subsequently, this study has three
goals. The first goal is to understand the concept of problem-based learning. How do
cognitive-structural development and typology theories provide a lens in
understanding problem-based learning? Furthermore, this study will define critical
thinking skills in a problem-based learning framework.
The second goal is to select a testing instrument for measuring students’
critical thinking skills. Notwithstanding the existence of empirical evidence
supporting the positive effects of PBL on students’ critical thinking (e.g., Birgegard
& Lindquist, 1998; Dolmans, Wolfhagen, van der Vleuten, & Wijnen, 2001;
Friedman & Deek, 2002; Kamin, O’Sullivan, Deterding, & Younger, 2003), a
number of colleges and universities that practice PBL curriculum are searching for
better ways to evaluate students’ critical thinking skills.
7
The third goal is to administer critical thinking skills assessment to USC
dental students and to measure specific critical thinking changes that are purported to
be affected by the PBL curriculum. The quantitative data will be analyzed to
determine the outcome of critical thinking achievements.

Research Questions
This quantitative research study intends to evaluate the critical thinking skills
performance of the dental students in a PBL teaching program. The following
research questions are to guide the literature review of this study:
What are the concepts, definitions, and process of problem-based learning?
How do cognitive-structural development theories provide a lens in
understanding problem-based learning?
How do typology theories provide a lens in understanding problem-based
learning?
How does problem-based learning affect learning achievement and students’
attitudes and feedback?
What cognitive skills are associated with critical thinking skills?
What role does problem-based learning play in critical thinking skills
development?
What tests can be used for assessing critical thinking skills achievement?
What is the Health Sciences Reasoning Test (HSRT)?
What are the HSRT’s psychometric validity and reliability?
8

Hypothesis
Starting from their first-year of dental education, PBL students are expected
to gain continuous and incremental improvement in their critical thinking skills
achievement through their years of study.
.
Significance of the Study
The PBL theory has continued to gain support from a number of educational
educators (e.g., Albanese & Mitchell, 1993; Barrows, 1986; Bridges & Hallinger,
1997; Friedman & Deek, 2002; Major & Palmer, 2001; Savin-Baden, 2000) who
contend that by using problems from actual clinical cases to approach learning
content, students develop exceptional skills in critical thinking. Many higher
education institutions are looking to assess and validate students’ critical thinking
skills improvement either for accountability or quality satisfaction enhancement
objectives (Stein, Haynes, & Unterstein, 2003). Moreover, Maudsley and Strivens
(2000) contend that PBL method can promote professional knowledge acquisition,
critical thinking, problem-solving, and experimental learning.
Although this study investigated dental students, it should interest and benefit
educators in other health sciences disciplines because critical thinking skills are
fundamental components of clinical reasoning (Birgegard & Lindquist, 1998;
Maudsley & Strivens, 2000). This study is expected to validate that PBL, as a
teaching method, can improve students’ critical thinking skills.
9
CHAPTER TWO
LITERATURE REVIEW
Introduction
The notion of learning through solving problem provides educational
strategies that are designed to help students acquire and develop knowledge
(Barrows & Tamblyn, 1980; Boud, 1987; Albanese & Mitchell, 1993). As this
approach converges with other contextual constructs, problem-based learning (PBL)
emerges as an innovative approach to education. Subsequently, the purpose of this
chapter is to conduct a literature review on PBL curriculum in health sciences (i.e.,
medical, dental, nursing) schools settings by utilizing student cognitive development
and learning theories.
Although innovative in some aspects, PBL is based upon the ideas that have
an extensive history and have been fostered by many educators and researchers (e.g.,
Piaget, Perry, Kolb, King & Kitchener). Accordingly, the theories that will be used
are derived from the cognitive-structural development and the typology frameworks.
In cognitive-structural development framework, the analysis will focus on Piaget’s
schemata or Perry’s positions, Baxter Magolda’s epistemological reflection, and
King and Kitchener’s reflective judgment model. Whilst in the typology framework,
the analysis will focus on Kolb’s learning styles theory, Hmelo’s self-directed
learning, and Zimmerman’s self-regulated learning.
Many studies on the effectiveness of PBL in the field of medical education
have been conducted. Although the reported results show some advantages of PBL
10
over non-PBL method, the studies’ authors also reach different overall conclusions
about the effectiveness of PBL (Newman, Ambrose, Corner, Evans, Morris-Vincent,
Quinn, Stephenson, & Vernon, 2003). For the purpose of this study, the focus of
PBL effectiveness will relate to learning achievement and students’ attitudes and
feedback.
In this study, a specific focus is placed on the critical thinking skills
outcomes in a problem-based learning framework. The effects of PBL on critical
thinking skills are examined. Additionally, tests for assessing critical thinking skills
achievement are discussed and evaluated.

Concepts of Problem-Based Learning
As an alternative medical education approach, problem-based learning (PBL)
was first introduced at The School of Medicine at McMaster University in Hamilton,
Canada in 1969 after many years of planning. Following their research into the
reasoning aptitude of medical students at this university’s medical school, Barrows
and Tamblyn (1980), who are often credited with the creation of PBL, established
PBL as a pedagogical strategy and curricular model. In reality, PBL was developed
after many years of faculty and student discontent with problem-solving learning
method in the traditional lecture-centered curriculum. Faculty perceived the urgent
need to produce graduates who could think critically and solve complex problems
and who were prepared to deal with the information explosion (Major & Palmer,
2001). In problem-solving learning, individual students answer a series of questions
11
from information supplied by a lecturer (Savin-Baden & Major, 2004). However,
absorbing and retaining the enormous amount of content inherent in a medical
education were turning into more unrealistic expectations. Using problem-based
learning, students used problem scenarios to engage themselves in the learning
process (Savin-Baden & Major, 2004). McMaster’s experiment certainly focused on
simulating patient problems and diagnoses in a manner consistent with that of a
practicing physician.
According to the 1984 report of the Project Panel on the General Professional
Education of the Physician (GPEP), as cited in Hmelo (1994), lectures are the
predominant method of instruction in traditional medical education. The traditional
curriculum inclines to promote rote memorization and a task orientation (Schmidt,
Dauphinee, & Patel, 1987). In fact, Levine and Forman (1973) find that students who
were learning in traditional curriculum were ineffective because they forget most of
the content. The GPEP panel recognizes that medical students retain little from this
method of traditional lectures that will not help them in acquiring the self-directed
learning skills that they will need for their future professional careers. The panel
recommends that basic sciences and clinical education should be integrated; the
medical students should incorporate basic science concepts into their clinical
problem-solving. Medical schools, such as Case Western Reserve and Harvard,
develop curriculum in which students take a certain number of traditional lecture
courses interspersed with PBL courses (Savin-Baden & Major, 2004). This
curriculum integrates basic science and clinical medicine throughout the entire four
12
years. Students work with a medical tutor/facilitator, who stays with them in the
same PBL group throughout their studies.
At USC School of Dentistry, the PBL curriculum is based upon the following
eight concepts (PBL Workshop Series, USC School of Dentistry, February 2005):
Inquiry based
The initial stimulus for inquiry in PBL is the data presented in a patient
case. Material presented in the case is selected for its relevance to the
curriculum and for its similarity to real life situations.
Contextualization
Problem based learning is an educational approach that is tailor-made to
make optimal use of context and it should also be understood that this is by
design and not by accident. PBL problems are designed as case
presentations so that learning of basic science information is stimulated by,
and organized around, thinking about a patient’s signs and symptoms.
Knowledge organization
Using the structure of the PBL process, the students begin to organize their
knowledge into a structure that is linked to evaluating clinical problems as
they learn. This establishes, earlier in the educational experience, a
foundation for future mentoring experiences during which the knowledge
organization can be expanded and refined.
Learner centered
In the broader sense, the term learner-centered implies educational
approaches that are focused on the learner – on his or her attributes,
capabilities, or needs – rather than the more typical emphasis on such
things as the objectives of the instructor, or the perceived requirements of
the curricular content.
Motivation
Students become motivated to learn because they cannot solve the problem
without acquiring understanding of the new material, and because they can
see that having mastered the material will be valuable to them in the future.
Prior knowledge
The idea that people learn by using existing knowledge to build new
understanding is a central theme of the school of thought known as
“constructivism”. Students entering any kind of graduate program bring
with them a wide variety of previously acquired knowledge and ideas that
may have a bearing on their graduate education.
Meta-cognition
The PBL process followed by the students provides a meta-cognitive
scaffolding for learning about all aspects of the situation presented in each
13
case or problem. Following the PBL process leads the students through the
identification of facts, elaboration of hypotheses, identification of learning
needs, application of new knowledge to the problem, and re-evaluation of
the hypotheses.
Cooperative learning
The most essential of this is individual accountability. Each member’s
contribution is essential to the function of the group and its ability to reach
its goals. Likewise each member is expected to maximize the overall group
effort by being able to contribute something unique to the group, sharing
unique resources, taking a special role or responsibility that promotes
successful completion of the task.

Definition and Process of Problem-Based Learning
In the literature, PBL has been defined and described in many various ways.
Barrows and Tamblyn (1980) define the concept of PBL as “the learning that results
from the process of working toward the understanding or resolution of a problem”
(p. 18). Meanwhile, Boud (1987) declares that “the starting point for learning should
be a problem, a query or a puzzle that the learner wishes to solve” (p. 13).
Furthermore, Albanese and Mitchell (1993) state that “PBL at its most fundamental
level is an instructional method characterized by the use of patient problems as a
context for students to learn problem-solving skills and acquire knowledge about the
basic and clinical sciences” (p. 53).
Depending on the nature of the domain and specific goals of the programs,
PBL can adopt various models (Barrows, 1986; Boud, 1987). Savin-Baden (2000)
argues that there merely are no narrowly defined characteristics of PBL because
individuals working in various contexts may use various PBL approaches. Despite
the many variations of PBL that intend to match it with specific educational contexts,
14
a core model is required to provide a basis of comparison to other education
methods.
Barrows and Tamblyn identify five characteristics of PBL model (Barrows &
Tamblyn, 1980): 1) Real world and complex problems that have no clear answer are
the learning focus; 2) students work in groups to define the problem, to identify
learning gaps, and to develop possible solutions; 3) students acquire new knowledge
through self-directed learning; 4) teachers act as facilitators; 5) problems lead to the
development of clinical problem-solving skills. Later, Barrows (1986) asserts four
important objectives in PBL: 1) Organizing knowledge for use in clinical context; 2)
developing effective clinical reasoning process; 3) developing effective self-directed
learning skills; 4) increased motivation for learning.
Boud (1987) categorizes Barrows and Tamblyn’s classic model of problem-
based learning as a problem-centered program. He expands this model to become
more student-centered by adding eight other characteristics, such as a focus on the
process rather than the products of knowledge acquisition and a focus on
communication and interpersonal skills. Moreover, Barrows (1986) proposes a
taxonomy of problem-based learning methods that describes varieties of differing
meanings and uses of problem-based learning. In this taxonomy, PBL involves
medical students to meet with a patient in some form of simulated setup that allows
for free inquiry to occur.
Burch (2000) defines PBL as a teaching strategy that shifts the classroom
focus from teaching to learning. He argues that as a type of active and discovery
15
learning, PBL requires students to take responsibility for their learning in
cooperative group setup to address specific and real problems. In the shift from a
teacher-centered to a peer-centered instructional method, students have a greater
opportunity to establish and exercise teamwork and leadership skills (Friedman &
Deek, 2002). The teacher plays a role as an observer, answers students’ questions,
provides feedback, and corrects misunderstandings of the content (Bridges &
Hallinger, 1997). Hmelo (1994) details the learning process of medical education
that utilizes the problem-based curriculum: students work in groups, collectively
pool their expertise and experience, and together they wrestle with the complexities
of the case that must be solved. As they work through the problem, they identify
their knowledge gaps, set their learning goals, and conduct research to close these
gaps. Facilitators guide students in facilitating learning of the cognitive skills needed
for problem-solving and helping them in reflecting on their experience.
As an innovative educational approach, PBL is gaining prominence in higher
education. Major and Palmer (2001) state that “PBL is an educational approach in
which complex problems serve as the context and the stimulus for learning” (p. 4).
Moreover, they describe that PBL students work in groups to solve complex and real
life problems. First, they define the problem and then they develop skills in
collecting, evaluating, and synthesizing resources. At the end, they summarize and
present the solution through their learning experiences. Unlike in a traditional class,
the instructor in a PBL class facilitates the learning process by guiding students in
searching out appropriate resources and moving forward in the problem-solving
16
process. As Khoo (2003) suggests, “the absence of a teacher part of the time resulted
in increased spontaneous participation” (p. 407) from students.
Mostly, problem-based learning represents an integrative approach to
students’ learning and cognitive development. PBL draws on a number of learning
theories while at the same time recognizes the importance of learning through
experience. Thus, as Savin-Baden and Major (2004) argue, PBL is an approach to
learning that is affected by the structural and pedagogical environment into which it
is placed. In higher education, PBL is “intended to guide students to become experts
in a field of study, capable of identifying the problems of the discipline and
analyzing and contributing to the solutions” (Gijbels, Dochy, van den Bossche, &
Segers, 2005, p. 30).

Problem-Based Learning and Cognitive-Structural Development Theories
According to Savery and Duffy (n.d.), PBL as an instructional model, is
consistent and coherent with the principles of instruction emerging from
constructivism. Thus, cognitive-structural theories, which concentrate on
psychological development processes, can provide a fascinating lens for
understanding the foundation of problem-based learning construct. Cognitive
theories are utilized to understand how medical students learn and what occurs inside
the mind when learning occurs in a PBL setup. In fact, Savin-Baden and Major
(2004) argue that one of the primary goals of problem-based learning is to focus “on
17
cognitive structuring which is essential for developing capacity and skills for better
learning or to learn how to learn” (p. 24).
As advocates of the cognitive tradition, Piaget and Perry contend that the
existing cognitive structure is the principal factor influencing meaningful learning
(Evans & Guido-Dibrito, 1998; Savin-Baden & Major, 2004). They believe that new
knowledge must be interpreted based on prior knowledge. Furthermore, Baxter
Magolda and King and Kitchener argue that new knowledge has to be interpreted in
terms of both prior knowledge and shared perspectives (Evans & Guido-Dibrito,
1998; King & Kitchener, 2004). Problem-based learning promoters, such as Barrows
and Boud, acknowledge that students embark on any learning environment with prior
knowledge and pre-existing cognitive structuring (Savin-Baden, 2000). Thus, one of
the main foci of PBL method centers on assisting students to apply their previous
knowledge and ways of thinking, and constructing them into new forms of
knowledge that are understandable and meaningful to them.

Piaget’s schemata and Perry’s positions
Piaget’s schemata is perhaps the first theory that examine how students as
learners grow and develop over time and how the actual act and process of learning
transform (Piaget, 1952). His cognitive development theory depends on the concept
of cognitive structures. He believes that the learning activities of students could
entirely correspond and match their cognitive stage, skill, or readiness. Later on,
Perry expands Piaget’s schemata by formulating nine positions that describe the
18
evolution of students’ conceptions of knowledge (Perry, 1970). He places students’
learning experiences at the center stage. Furthermore, he argues that students proceed
through a sequence of development stages. The nature of these development stages is
identified in Strange’s (1994) Proposition 5 to 9. As indicated by Proposition 5 to 7,
cognitive development occurs when students are ready for learning experiences,
respond to new knowledge, and evaluate the learning tasks that ensure change. While
Proposition 8 contends with the qualitative and cyclical developmental changes,
Proposition 9 maintains that development occurs as an interactive process between
students and their environment. These degrees of development or integration,
Strange argues, are often understood in the context of a hierarchical sequence in
which each stage represents an inherently more complex structure.
Problem-based learning adheres to Perry’s theory as the basis of cognitive-
structural development theories: students compare new knowledge to their existing
cognitive structures and they ascertain the overall structure of the problem. Perry
refers to this theory as “forms” of intellectual development, which are the structures
that shape students’ learning experiences (Perry, 1970). Learning, in Perry’s dualism
view, is essentially information exchange since knowledge as facts is considered to
be having the right answers. However, Perry acknowledges multiplicity as
recognizing diverse views when right answers are yet unknown. In this situation, all
views from multiplistic perspectives are equally valid. As students progress through
multiplicity, they shift from dependent learners toward the more independent type.
During this progression, peers become sources of knowledge and students are
19
expected to improve their analytical skills. In problem-based learning, students’
learning capabilities can be expanded through facilitator guidance and peer
collaborations (Savin-Baden & Major, 2004). Students learn when they are able to
see the meaning in learning through progressions of their experiences. Perry’s theory
is an important contributor to PBL method for gaining a basic understanding of how
students make meaning. Thus, problem-based learning is essentially an attempt to
create educationally purposeful environments and optimal contexts for learning.

Epistemological reflection framework
In Baxter Magolda (2001)’s epistemological reflection, absolute knowing is
similar to Perry’s dualism that assumes knowledge exists in an absolute form.
Students often enter problem-based learning by assuming that right or wrong
answers exist in all areas of knowledge and expecting the authorities to know these
answers. Thus, as learners, absolute knower focus on obtaining the information; they
do not expect their peers to have valid knowledge. The pathway from Baxter
Magolda’s absolute knowing to transitional knowing involves learners’ realization
that some knowledge is uncertain. Medical students frequently face uncertainty in
dealing with some medical problems. PBL helps students in shifting their focus from
acquiring knowledge to understanding it. Peer interactions play more active roles
because understanding requires more exploration than knowledge acquisition does.
The emergence of Baxter Magolda’s independent knowing prompts self-authored
knowledge in which students view their opinions as valid. Learning independently
20
involves expressing one’s views as well as hearing others (Evans & Guido-Dibrito,
1998). In PBL setup, facilitators are no longer responsible for providing knowledge
but rather providing the context in which to explore knowledge. Peers become a
valid source of knowledge rather than part of the process of knowing (Baxter
Magolda, 2001; Savin-Baden & Major, 2004). In its practice, PBL promotes
independent thinking and the exchange of opinions. Lastly, Baxter Magolda’s
contextual knowing is demonstrated only by a few undergraduate students.
Contextual knowing entails a principle that the legitimacy of knowledge claims is
determined contextually (Evans & Guido-Dibrito, 1998). Contextual knowers are
defined as students who examine all aspects of an issue, seek out expert advice, and
integrate their own and others’ views in deciding the answer. Thus, the role of PBL
in the medical curricular model now involves the creation of a learning environment
that supports contextual applications of knowledge among medical students.

Reflective judgment model
Hmelo (1994) argue that evidence from cognitive psychology shows that
prior knowledge facilitates the subsequent processing of new information. She
concludes that problem-based learning provides a suitable context for learning basic
science since all of the knowledge content is learned in the context of a clinical
problem, which should increase recall of this information when it is needed. Central
to King and Kitchener’s model is “the people’s assumptions about what and how
something can be known provide a lens that shapes how individuals frame a problem
21
and how they justify their beliefs about it in the face of uncertainty” (Evans &
Guido-Dibrito, 1998, p. 161). The seven stages of King and Kitchener (2004)’s
Reflective Judgment Model, which may be clustered into three categories (pre-
reflective, quasi-reflective, and reflective thinking), can be reflected to describe how
PBL students learn new medical and clinical knowledge. Pre-reflective thinkers do
not realize or recognize the presence of real problems that lack an absolute correct
answer. They usually do not use evidence in reasoning to form a conclusion. PBL
students at the pre-reflective thinking stage can obtain knowledge with certainty by
direct observation, through the senses, or via authority figures. As students reach the
quasi-reflective thinking mode, they have the ability to see knowledge as an
abstraction and recognize its construction. PBL students as quasi-reflective thinkers
realize the existence of ill-structured problems and are able to identify problematic
issues as well. Although they use evidence, they are still struggling in drawing
reasoned conclusions and justifying their beliefs. Finally, at the reflective thinking
stage, PBL students uphold that knowledge must be viewed in relation to the context
in which it is actively constructed. Using relevant data, they contentedly and
consistently use evidence and reason in support of their judgment.
Schmidt (1983) describes how PBL creates three conditions that are optimal
for cognitive development and learning. First, PBL stimulates students to activate
their prior knowledge in processing newly acquired information through the
selection and sequencing of problems. This prior knowledge along with the way it is
stored in memory facilitates initial understanding of the new information and its later
22
recall. Second, PBL constructs a framework that resembles the environment in which
students later will apply their knowledge. This framework is the problem that they
will confront during their professional practice. The various elements of this problem
provide signals that enable recall of the knowledge acquired in a similar context
during learning. Third, PBL urges students to elaborate their newly acquired
knowledge when they initially learned. Elaboration deepens understanding and
provides opportunities for students to examine their comprehension of the subject
matter against the interpretations of their peers. Furthermore, Friedman and Deek
(2002, p. 241) condense Schmidt’s PBL into three principles:
(a) activation of prior-learning via problem;
(b) encoding specificity such as that resemblance of the problem to intended
applications domains facilitates later transfer (leading to an emphasis on
authentic learning environments);
(c) elaboration of knowledge via discussion and reflection to consolidate
learning experiences.

PBL stimulates cognitive effects, which are assumed to positively contribute
towards students’ ability to apply knowledge. (Dolmans, Wolfhagen, van der
Vleuten, & Wijnen, 2001). Moreover, PBL enhances intrinsic interest in subject
matter.
Finally, the cognitive-structural development theories provide a construct for
the problem-based learning to operate in which students compare new information to
their existing cognitive structures, conceptualize the overall structure of the problem,
broaden their learning capabilities through guidance and collaboration, and learn
through progression of experience. Students have the ownership of the problem and
23
the facilitation is not knowledge driven, but rather it is focused on meta-cognitive
processes (Savery and Duffy, n.d.).

Problem-Based Learning and Typology Theories
Typology theories, which reflect on the concept of learning styles, can
provide a lens on how learners take in, organize, and process information in the
problem-based learning environment. Typology operates as a framework within
which cognitive-structural development occurs and also offers essential information
about sources of support and challenge for PBL students who are otherwise
developmentally similar (Evans & Guido-Dibrito, 1998). In PBL construct, typology
theories are particularly beneficial in providing guidance concerning the design of
classes, clinical training sessions, and other structured medical education
experiences.

Kolb’s learning styles theory
Similar to the notion of the problem-based learning context, Kolb defines
learning as the transformation process of creating knowledge through students’
learning experiences (Kolb, 1984). Kolb (1981) uses a series of steps in describing
the learning cycle: concrete experience, reflective observation, abstract
conceptualization, and active experimentation. Each step, which provides a
foundation for the subsequent one, may be viewed as a dimension to a four-
dimension learning style model: feeling dimension, watching dimension, thinking
24
dimension, and doing dimension. Burch (2000) maps Kolb’s learning cycle to
describe the PBL process that involves four stages: the problem, initial analysis,
research, and reporting. At each stage, students as group members discuss the case
and receive feedback from peers and the facilitator.
Kolb’s concrete experience stage or feeling dimension provides a model for
the problem stage in PBL construct. Typically, as Burch (2000) describes, the
problem is well-devised and open-ended that it does not provide information that are
necessary for quick solutions. The main purpose for such problems is to challenge
students in order to motivate learning. Students must identify, search, and use
appropriate resources. Thus, by identifying learning issues in this method, students
learn what knowledge they are required to acquire by focusing their efforts in
integrating the information they gather.
Reflective observation stage in Kolb’s learning cycle model associates with
the watching dimension that observes and reflects on the concrete experience from
multiple perspectives. Burch (2000) describes this stage as the initial analysis stage
in the PBL model. The initial analysis stage elicits and activates students’ existing
knowledge, a critical step in learning new knowledge. In this stage, students are
required to assure that each member of the group understands the basic concepts and
the nature and scope of the problem. Moreover, students need to identify specific
learning issues by inventorying what is known from the problem, prior knowledge,
and personal experience.
25
In the abstract conceptualization stage or the thinking dimension, Kolb
describes how students formulate concepts that integrate their observations into
theories (Kolb, 1984). In the PBL structure, Burch (2000) describes this stage as the
research stage in which students work individually or in subgroups to gather
information on identified learning issues. Research may be conducted at the library,
on the Internet, or from interviews of authorities. Initial research can lead to further
questions and inquiries. The primary focus of this stage is for students to recognize
researching as a skill and to manage problems competently.
Kolb’s active experimentation stage or the doing dimension puts theories to
use in making decisions and solving problems (Kolb, 1984). Burch (2000) describes
this stage as the reporting stage in the PBL context as students report their
discoveries to the group and in the process become the expert of the subject matter.
Students explain the concept to their peers and field questions from the members.
Explanations that are based on their research teach students to put ideas into their
own words. As students report and share their findings, they realize the complexity
and characteristics of the problem. Burch (2000) argues that education research
consistently shows that active experimentation is a key to learning, as opposed to
memorization or short-term recalling.
Kolb defines individual learning style based on individual preferences as a
habitual way of responding to a learning environment (Kolb, 1984). Four individual
learning styles emerge namely convergers, divergers, assimilators, and
accommodators. The learning style of convergers is inclined toward the learning
26
style of problem-based learners as this type of students is typically good problem
solvers and decision makers. They are effective in applying ideas to practical
situations, excelling at task involving research, and demonstrating deductive
reasoning.

Self-directed learning
Because of the self-discovery characteristics of learning in PBL, skills needed
for self-directed learning are attained as students oversee their learning goals while
they are coping with the problems they are trying to solve. Researches show that
students who participate in problem-based learning curricula demonstrate self-
directed learning skills (Blumberg, 2000; Hmelo, 1994). At some medical schools,
PBL is used to assist students to learn basic science integrated with clinical
knowledge and clinical reasoning skills along with self-directed learning skills
(Barrows & Tamblyn, 1980; Hmelo, 1994).
Hmelo develops seven features of PBL that specifically support the
development of self-directed learning skills. In this self-directed learning model,
students are part of the student-centered nature of PBL, identify a problem with their
existing knowledge, produce learning goals through knowledge gaps identification,
perform independent research activities, critique the resources used for research,
apply new knowledge to the problem, and collaborate and reflect on the answer.
Each of these features lends an important function in supporting the development of
self-directed skills.
27
Furthermore, Hmelo develops a self-directed learning model within the PBL
construct. Based on this model, self-directed activities that occur during the problem
solving itself can be summarized into a five-step process. In the first step, students
assess their knowledge relative to problems that they face before continuing into the
second step in which they formulate the learning issues. During the third step,
students develop and implement a plan to address learning issues. In the fourth step,
students utilize new knowledge in their problem-solving processes. Finally, in the
fifth step, students perform an evaluation to find out if the goals have been met. If
the goals have not been met, the process can be repeated, starting from step two to
step five.
In a problem-based learning setting, studies have revealed that students
constantly apply their new knowledge and practice their self-directed strategies in
problem-solving contexts (Hmelo, 1994). Subsequently, students in a PBL
curriculum have opportunities to utilize their knowledge in variety of cases; thus,
they are able to apply their self-directed learning strategies in solving new cases.
Since knowledge transforms rapidly, PBL approach fosters students’ self-
directed learning skills so that they will automatically continue their own learning for
the rest of their lives (Barrow, 1986). Hsu (1999) argues that PBL encourages
students to determine what and how to learn on their own in order to ensure their
lifelong learning skills and the ability to be independent and effective learners.

28
Self-regulated learning
The roles of self-directed learning in PBL construct are highly similar to what
has been termed self-regulated learning in typology theories by Zimmerman and
Lebeau (2000). Zimmerman defines self-regulated learning as self-generated
thoughts and actions intended to achieve specific educational goals, such as
analyzing a reading assignment, preparing to take an exam, or writing a paper.
Therefore, self-regulated learning is designed to develop students’ understanding of
self-efficacy and control over their learning process.
Self-regulated learning can play an integral role in the development of study
skills among PBL students. The four-step of Zimmerman’s cyclic model of self-
regulated learning can be viewed as an integral part of the PBL construct. The cycle
begins with self-evaluation and monitoring phase, which supports Kolb’s feeling
dimension and self-directed learning’s problem identification. The goal setting and
strategic planning phase utilizes Kolb’s thinking dimension and operates similarly
with the self-directed learning model’s goals. As the third phase in the cycle, the
strategy-implementation monitoring follows Kolb’s doing dimension and the self-
directed learning’s new knowledge application. Lastly, Kolb’s watching dimension
and self-directed learning’s reflection are reflected in the strategic-outcome
monitoring phase that links learning outcomes and strategic processes to determine
effectiveness.
Zimmerman’s cyclic model of self-regulated learning provides students with
self-monitoring feature on each learning stage that provides information that can
29
alter subsequent learning goals, strategies, or performance efforts. By establishing
this self-regulatory cycle, Zimmerman, Bonner, and Kovach (1996) argue that it can
help students learn to recognize and acknowledge the relations between their study
behavior and learning outcomes. Hence, with careful self-monitoring during PBL
practice, students are able to monitor the effectiveness of a newly acquired strategy.
The topology theories provide the problem-based learning construct with the
students’ transformation process in acquiring new knowledge through their learning
experiences. The theories contend that learners develop and progress through
transitory stages. The theories also establish that learning in the PBL environment
involves self-development person through self-directed and self-regulated learning.

Effectiveness of Problem-Based Learning
The effects of PBL have not been adequately assessed for making an
informed decision (Hmelo, Gotterer, Bransford, 1994). PBL method in medical
education is supposed to enhance the integration of basic knowledge and clinical
science. Medical educators (e.g., Barrows, 1986; Schmidt, 1983) have argued that
PBL improves basic science learning, clinical reasoning, and lifelong learning.
Norman and Schmidt (1992) contend that there is good empirical evidence to support
at least two of the key aspects of PBL in the cognitive psychology literature. First,
learning is improved where there is activation of prior knowledge. Second,
elaboration of knowledge at the time of learning enhances information retrieval.
30
However, the literature on the cognitive benefits of PBL has shown mixed
results (Hmelo, Gotterer, Bransford, 1994; Hsu, 1999). Outcome studies on PBL
effectiveness have also revealed controversial results (Newman, Ambrose, Corner,
Evans, Morris-Vincent, Quinn, Stephenson, & Vernon, 2003). One of these studies
indicates that medical students at PBL schools scored lower on basic science
examinations and viewed themselves as less well prepared in basic sciences in
comparison with their colleagues at non-PBL or traditional medical schools
(Albanese and Mitchell, 1993). Furthermore, PBL graduates tended to engage in
backward reasoning rather than the forward reasoning that experts engage in.
Langendyk (2006) argues that PBL curriculum does not guarantee the appropriate
and true development of self-assessment skills.
The use of PBL methods has been widespread in the preclinical curricula of
medical schools in the U.S. According to a survey conducted by Kinkade (2005),
70% or 86 out of 123 medical schools used PBL. However, the full use of PBL is
limited, since less than 6% of medical education programs utilize it more than 50%
of their instruction. This is due to the fact that in preclinical curriculum, during the
first two years of medical school, most learning typically occurs in classrooms,
lecture halls, and labs, rather than in the presence of patients.
As a pedagogical method, many educators (e.g., Barrows, 1986; Hmelo,
1994; Savin-Baden, 2000; Schmidt, 1983) advocate that PBL promotes the active
learning. The focus of PBL pedagogy is primarily on learning to learn and less on
mastery of a particular body of knowledge. Study conducted by Major and Palmer
31
(2001) shows that PBL strategy provides students with the opportunity to acquire
theory and content knowledge. PBL also contributes in improving students’ attitude
toward learning. Additionally and more importantly, PBL helps students to develop
advanced cognitive abilities, such as critical thinking and problem solving,
interpersonal communication skills (Thomas, 1997), and self-regulated skills (van
den Hurk, 2006).
Schmidt, Vermeulen, and van der Molen (2006) study the long term effect of
PBL on the professional competencies of medical training graduates. Study’s
participants were requested to rate themselves on 18 professional competencies
derived from the literature. The findings suggest that PBL not only affects the PBL-
related competencies in the cognitive and interpersonal skills area, but also the
professional-related skills as well that are important in their medical practice.

Learning achievement
By comparing to traditional instruction, a number of studies on PBL
effectiveness focus on the change in knowledge and skill levels. Based upon
quantitative and qualitative analyses conducted by Liu, Hsieh, Cho and Schallert
(2006), findings indicate an increase in students’ science achievement for learning
science after their computer-enhanced PBL instruction. Likewise, Yip and Ghafarian
(2002) prove that the PBL approach in information systems courses is far better than
the traditional approach in many aspects (i.e., learning achievement, level of
interaction, self-regulated learning). Moreover, Distlehorst, Dawson, Robbs, &
32
Barrows (2005) find that in several of the clerkship performance measures, PBL
students perform significantly and consistently better than non-PBL students.
At USC School of Dentistry, PBL-track students demonstrate a statistically
significant (t = -6.5, p = .000) superior performance on the Part I National Dental
Board Examination, at all subjects level, as compared both with their peers in the
traditional track and with national means (Fincham & Shuler, 2001). Furthermore,
this significant difference is also shown in all the mean scores for all four of the sub-
tests: Anatomical Sciences, Biochemistry/Physiology, Microbiology/Pathology, and
Dental Anatomy.
The effect of group size in PBL on problem-solving skill, self-directedness,
and technical knowledge among students in a dental education program is examined
by Lohman and Finkelstein (2000). Analysis of the data finds that the development
of self-directedness varies with group size: an increase in small and medium size
groups, but a decrease in large groups. A significant difference is yielded between
the medium and large groups on this measure, F = 3.321 (2.68). However, no
significant differences among group sizes are found on the transfer of problem-
solving skills measures and the development of technical knowledge.
Maudsley and Strivens (2000) argue that PBL can promote professional
knowledge acquisition, critical thinking, problem-solving, and experimental learning.
At the University of Missouri School of Medicine, Hoffman, Hosokawa, Blake,
Headrick, and Johnson (2006) report that PBL curriculum has resulted in higher
performance of PBL class cohorts on United States Medical Licensing Examination.
33
They also find that PBL program better prepare graduates with knowledge and skills
needed to practice within a complex health care system. Similarly, the introduction
of PBL curriculum coincides with the improved students’ academic performance at
the University of Transkei School of Medicine in South Africa (Iputo and Kwizera,
2005). The PBL approach to medical education may have contributed to this
academic improvement. Moreover, students in PBL setting have developed stronger
clinical competencies (Hmelo, 1994).
On the contrary, Albanese and Mitchell (1993) find that most medical
education studies show no significant difference between PBL students and non-PBL
students in acquiring sciences. Similarly, a study conducted by Prince, van Mameren,
Hylkema, Drukker, Scherpbier, and van der Vleuten (2003) in the Netherlands
reveals that PBL does not result in a lower level of anatomy knowledge than the
more traditional medical educational approaches.
Another study conducted by Schmidt, Dauphinee, and Patel (1987) shows
that students in non-PBL programs score higher on tests of medical knowledge than
students in PBL curriculum. Vernon and Blake (1993) find that non-PBL students
performed significantly better than their PBL counterparts on the National Board of
Medical Examiners Part I. However, it also finds that students who acquired
knowledge in PBL context are more likely to use it spontaneously to solve new
problems than students who acquire the same knowledge under non-PBL method
through lectures.

34
Attitudes and feedback
Albanese and Mitchell (1993) report that students in PBL courses experience
greater satisfaction than non-PBL students. Students find PBL to be more nurturing
and enjoyable than traditional instruction. Consequently, PBL students experience
substantially more positive attitudes toward the learning environment than do
students in non-PBL programs (Vernon & Blake, 1993).
PBL students also tend to give high ratings for their training than non-PBL
students (Schmidt, Dauphinee, & Patel, 1987). Accordingly, Roche, Scheetz, Dane,
Parish, & O’Shea (2003) find a significant and positive change of attitudes among
first year medical school students in PBL environment. Similar results are also
reported by Winning, Skinner, Townsend, Drummond, & Kieser (2004) on students’
attitude toward PBL at two dental schools in Australia and New Zealand. It seems
that these changes in attitude have impacted on student retention as Lieux (1996)
discover that student attendance is significantly higher in PBL classes than in
lecture-based format.
Birgegard and Lindquist (1998) discover that medical students generally had
a low opinion in their attitudes toward traditional preclinical studies at University
Hospital in Sweden. However, when PBL was introduced, there was a substantial
change and significant improvement in students’ attitudes. There were no other
changes in the curriculum or the teaching methods other than the introduction of
PBL. It seems that PBL improved the students’ opinion about the medical school
35
curriculum pertaining to critical thinking and problem-solving. Similar results from
other medical school in Sweden are reported by the study’s authors.
Although feedback is an essential component of PBL medical education, the
types of feedback that students receive have not been methodically studied. Jacobs,
Dolmans, Wolfhagen, and Scherpbier (2003) validate a short questionnaire to assess
the levels of complexity and structuredness of a problem as perceived by students
that can provide facilitators with feedback about the quality of problems. Parikh,
McReelis, and Hodges (2001) find significant differences exist in the types of
feedback final year medical students receive at five medical schools in Canada. The
types of feedback include peer feedback, facilitator or tutor feedback, and self-
assessment. While rated highly by students at all schools, the use of feedback is
mostly limited. Gilkison (2003) reveals that both expert and non-expert facilitators
use similar techniques to facilitate students’ learning process in undergraduate
medical curriculum. In addition, both type of facilitators rise students’ awareness and
feedback.
Based on two major meta-analyses on the effectiveness of PBL (Albanese &
Mitchell, 1993 and Vernon & Blake, 1993), Bridges and Hallinger (1997) conclude
that 7 out of 10 measures favor PBL approach over non-PBL methods. However, a
review of the literature by Farrow (2003) concludes that “despite the additional
resources required, PBL is no more effective than traditional methods” (p. 1131). He
also criticizes the validity of measuring attributes of interest to medical educators or
36
patients. Colliver (2000) concerns that PBL students are developing fragmented
conceptualizations (e.g., learning isolated case-specific facts).
Since PBL is an innovative educational method in which complex problems
serve as the context and the motivation for learning, it brings with it unique
challenges to traditional assessment. Alternative assessment may provide additional
insight into the effectiveness of PBL as an alternative pedagogy (Major & Palmer,
2001). An alternative assessment measure should consider that learning is a
multidimensional activity that includes knowledge, abilities, values, attitudes, and
habits of the mind (Evans & Guido-Dibrito, 1998).

Critical Thinking Skills
The definitions of critical thinking skills vary and often overlap. According to
the Critical Thinking Community (n.d.), critical thinking is “self-directed, self-
disciplined, self-monitored, and self-corrective thinking. . . . It entails effective
communication and problem-solving abilities . . .” As the term is presently used,
critical thinking means “reasonable and reflective thinking focused on deciding what
to believe or do” (CriticalThinking.net, n.d.).
Norris and Ennis define critical thinking as “reasonable and reflective
thinking that is focused upon deciding what to do or believe” (Christen, Angermeyer,
Davison, & Anderson, 1994). Furthermore, Norris and Ennis argue that critical
thinking is demonstrated by the individual ability to judge the accuracy of
information, evaluate conclusions, and make good inferences. Based upon these
37
definitions, students are expected to utilize critical thinking skills to reach a
conclusion based on available information. In fact, critical thinking improves the
ability of students to make decisions or to attain conclusions about problems or
issues that may not always have a single correct answer.
According to the Delphi Report (1990) of the American Philosophical
Association’s Critical Thinking: A statement of expert consensus for purposes of
educational assessment and instruction, critical thinking is “purposeful, self-
regulatory judgment which results in interpretation, analysis, evaluation, and
inference, as well as explanation of the evidential, conceptual, methodological,
criteriological, or contextual considerations upon which that judgment is based”
(Edman, Robey, & Bart, 2002; Facione, 1990; Spicer & Hanks, 1995).
In medical education, problem solving process has been identified as the
clinical reasoning process, which associates with the cognitive process that is
necessary to evaluate and solve a patient’s problem (Hsu, 1999). Halpern (1999)
argues that the use of cognitive skills or strategies in critical thinking increases the
probability of desirable outcome as defined by the individual. As summarized by
Facione (1990), the panel of 46 experts from philosophy and education that produce
the Delphi Report specifies critical thinking to include cognitive skills in:
Interpretation (sub-skills: categorizing, decoding significance, clarifying
meaning)
Analysis (sub-skills: examining ideas, identifying arguments, analyzing
arguments)
38
Evaluation (sub-skills: assessing claims, assessing arguments)
Inference (sub-skills: querying evidence, conjecturing alternatives, drawing
conclusions)
Explanation (sub-skills: stating results, justifying procedures, presenting
arguments)
Self-regulation (sub-skills: self-examination, self-correction)

Effects of Problem-Based Learning on Critical Thinking Skills
Although descriptions of critical thinking skills are wide-ranging and
overlapping, the selection of a particular assessment methodology should be based
on how thinking is defined. Many efforts have been made to develop instruments
reflecting the reasoning process, but it appears that no unanimity exists on the
methods in assessing critical thinking skills (Nendaz & Tekian, 1999; Yip &
Ghafarian, 2002), or on the methodologies for assessment in general (Hsu, 1999).
Medical education researchers (e.g., Patel, Groen, & Norman, 1991; Hmelo,
1994; Moore, Blocks, Briggs-Style, & Mitchell, 1994) argue that there is evidence
that different PBL schools show different outcomes on student’s critical reasoning
ability. From a critical thinking skills perspective, PBL students are assumed to be
more able to learn new knowledge due to the activation of prior knowledge and
restructuring of newly acquired knowledge (Zimmerman & Lebeau, 2000).
It is often implied that clinical courses will automatically improve students’
critical thinking and problem-solving skills (Birgegard & Lindquist, 1998). Several
39
studies (e.g., conducted by Albanese & Mitchell, 1993; Dolmans, Wolfhagen, van
der Vleuten, & Wijnen, 2001; Friedman & Deek, 2002) have produced empirical
evidence that PBL stimulates cognitive effects and leads to elaboration of knowledge
and enhances underlying interest in the subject matter.
Kamin, O’Sullivan, Deterding, and Younger (2003) conduct an exploratory
study to examine how case modality presentations (face-to-face with a text case,
face-to-face with a digital video case, and virtual with a digital video case) affect the
critical thinking of the third-year medical students in PBL format. Utilizing an
existing coding scheme, each distinct codable unit of discourse is placed into one of
the 35 indicators that is reflective of five critical-thinking stages. The results show
that students who learned in a virtual modality with a digital video case engaged in
more critical thinking.
Reflective judgment, a student development theory promoted by King and
Kitchener (2004), is the foundation for critical thinking concept encompassed by the
Norris and Ennis definition in a study conducted by Christen, Angermeyer, Davison,
and Anderson (1994). According to reflective judgment theory, students develop
through a specified sequence of cognitive maturity in which they show
characteristics changes along three dimensions (prereflective, quasi-reflective, and
reflective thinking). The study shows that the correlations between the Cornell
Critical Thinking Test and the Reflective Judgment Performance Assessment are
low. This suggests that there is a possibility that the study measures different forms
of critical thinking.
40
Neo and Neo (2005) conduct a study to examine how learning in multimedia
could be enhanced through the use of PBL. The study’s survey results show that
students participate actively in their own learning process and worked together in
collaboration with their peers. By practicing PBL method, students successfully learn
to develop their critical thinking, problem-solving, and interpersonal skills. In their
preliminary study, Hmelo, Gotterer, and Bransford (1994) report that there is a
significantly greater use of hypothesis-driven reasoning in the PBL group compared
to the non-PBL group. Likewise, PBL group also shows a greater coherence in their
explanations. PBL instruction appears to have distinct cognitive effects that may
influence medical professionals throughout their careers and may shape the learning
strategies they use in lifelong learning.

Tests for Assessing the Critical Thinking Skills
Critical thinking consists of a number of discrete and distinct skills. Critical
thinking is not a general ability but rather a complex set of general and specific
factors (Facione, 2007). Based on information obtained from U.S. Department of
Education (2000), there are twelve standardized critical thinking tests available
(Stein, Haynes, & Unterstein, 2003). Additionally, there are several performance
assessment approaches that can be used as outcome measures within various
subjects. Ennis and Weir (1985) suggest that in lieu of appropriate multiple choice
tests, open-ended assessment tests are needed and other measures may include
interviews. In formulating their means of assessment, educators should consider
41
guidelines concerning meaningful contexts in assessment tests. The Delphi Report
issues a specific recommendation regarding the critical thinking assessment: “In
evaluating the acceptability of a critical thinking assessment strategy or instrument
one should consider content validity, construct validity, reliability, and fairness”
(Facione, 1990).
There were four instruments available commercially for the assessment of
critical thinking skills at college levels by 1994 (Facione & Facione, 1994): the
Watson-Glaser Critical Thinking Appraisal (first developed in the 1940’s and revised
most recently in 1980), the Cornell Critical Thinking Test (1985), the Ennis-Weir
Critical Thinking Essay Test (1985), and the California Critical Thinking Skills Test
(1990).
The Watson-Glaser Critical Thinking Appraisal test measures five type of
thinking: inference, recognition of assumptions, deduction, interpretation, and
evaluation of arguments (Blai, 1992). The Cornell Critical Thinking Test, Level Z
focuses on seven types of thinking: induction, credibility, prediction, experimental
planning, fallacies, deduction, and identification of assumptions (Blai, 1992). The
Ennis-Weir Critical Thinking Essay Test consists of open-ended assessments of
thinking. The California Critical Thinking Skills Test (CCTST) provides five types
of thinking: inductive reasoning, deductive reasoning, analysis, inference, and
evaluation (Facione, 2007).
Williams, Wise, and West (2001) conduct a multifaceted measurement of
critical thinking skills in college students by using several instruments. California
42
Critical Thinking Dispositions Inventory obtains a fairly stable estimated reliability
(.89) and covers four domain facets. Scores on the Critical Thinking Appraisal (.83)
are more reliable than the scores on the Cornell Critical Thinking Test, Level Z (.63).
However, both of these tests cover only two domain facets.
Similar to Williams, Wise, and West (2001)’s research is a study conducted
by Edman, Robey, and Bart (2002). The study use the Watson-Glaser Critical
Thinking Appraisal and the Ennis-Weir Critical Thinking Essay Test along with the
Minnesota Test of Critical Thinking-II, the Multi-Dimensional Aptitude Battery, and
the Epistemological Questionnaire to measure both critical thinking skills and the
willingness to critically evaluate arguments that are congruent with one’s own goals
and beliefs. The results show the correlations of the scores on the Minnesota Test of
Critical Thinking-II with the scores on the Watson-Glaser Critical Thinking
Appraisal, the Ennis-Weir Critical Thinking Essay Test, and the Multi-Dimensional
Aptitude Battery are in the range hypothesized and support the concurrent validity of
the test. Specially, the Minnesota Test of Critical Thinking-II most highly correlates
with the Watson-Glaser Critical Thinking Appraisal since both tests purport to test
critical thinking abilities and both use multiple-choice methodology.
Van Gelder, Bissett, and Cumming (2004) utilize California Critical Thinking
Skills Test (CCTST) to measure university students’ informal reasoning after
completing 12 weeks of deliberate practice in informal reasoning. The results show
that students gain a large improvement and practice related to amount of
improvement in informal reasoning. In another study, Anderson and Saucier (1999)
43
show significant differences between the CCTST scores and educational preparation
among faculty in several nursing programs. Additionally, there are also significant
differences between the CCTST scores and academic rank.
Meanwhile, Tiwari, Lai, So, and Yuen, K. (2006) utilize California Critical
Thinking Disposition Inventory (CCTDI) to measure and compare the effects of PBL
and traditional lecturing approaches on students’ critical thinking development. The
study uses a randomized, controlled trial to compare the effects of PBL and
traditional lecturing methods on undergraduate nursing students over a 3-year period.
The CCTDI that measures students’ critical thinking has a 75-item Likert scale tool
with 7 subscales. A mix method, a combination of quantitative and qualitative data,
was employed. The study results show that there is a significant improvement in the
overall development of students’ critical thinking dispositions for those who
undertake the PBL compared to traditional lecture courses, respectively. Similarly,
Profetto-McGrath, Hesketh, Lang, and Estabrooks (2003) find a significant positive
correlation between the total CCTDI score and research utilization among nurses.
Moreover, CCTDI demonstrates very good reliability estimates in the evaluation of
critical thinking that are administered to nursing students (Bondy, Koenigseder,
Ishee, and Williams, 2001).
On the other hand, Bartlett and Cox (2002) use both CCTST and CCTDI to
determine the change in critical thinking dispositions and skills among physical
therapy students over academic and clinical portions in one year. The results show
statistically significant (p < .001) improvements in both total scores and all
44
subscales. Likewise, Rapps, Riegel, and Glaser (2001) also utilize CCTST and
CCTDI to test a model of cognitive development among registered nurses. The
outcomes indicate that critical thinking skill is a significant contributor only to the
dualistic level of cognitive development. Meanwhile, critical thinking dispositions
contribute to all three levels of cognitive development. These results suggest that the
development of a critical thinker may require time and experience.
Sungur and Tekkaya (2006) use Motivated Strategies for Learning
Questionnaires to investigate the effectiveness of PBL approach on students’
learning strategies, motivation, and self-regulated learning. The study design is a
randomized true experiment, which consists of an experimental group (PBL) and a
control group (non-PBL). The results reveal that students in the PBL group show
higher level of intrinsic goal orientation, learning strategies, critical thinking, and
meta-cognitive self-regulation compared with the non-PBL group.
Boshuizen, Machiels-Bongaerts, Schmidt, and Hermans (1995) explore the
validity of the Maastricht Progress Test by comparing student scores and subscores
on the different categories with a newly developed Clinical Reasoning Test.
Maastricht Progress Test is an alternative assessment instrument that was developed
at the University of Limburg Medical School in Netherlands. This test was designed
to measure the growth of knowledge and clinical reasoning performance in a PBL
curriculum. The results reveal that both tests show the same pattern of increasing
scores over the years and have a high correlation as well.
45
Cheung, Rudowicz, Kwan, and Yue (2002) perform a broader assessment of
critical thinking to adapt to countries where English is not the primary language and
whose cultures, value, and lifestyles are different from those of the West. This
adaptation is necessary in order to avoid the impediment of critical thinking
assessment due to language, beliefs, and habits in educational setting as argued by
Ennis and Norris (1990). Measures of critical thinking skills include nine items
adapted from Watson-Glaser Critical Thinking Appraisal and fourteen items adapted
California Critical Thinking Disposition Inventory. Results show that eight measures
are reliable and contribute significantly to general critical thinking. Furthermore,
they show convergence in the predictive validity of alternative scores of general
critical thinking.
Setterstein and Lauver (2004) use the Test of Everyday Reasoning (TER) to
show the relation between critical thinking and participation in health behaviors
among community-dwelling adults. The TER is adapted from the CCTST and is
recommended for use with high school and general adult populations (Facione &
Blohm, 2001).
Stein, Haynes, and Unterstein (2003) at the Tennessee Technological
University develop a core set of critical thinking skills for their graduates. The test
results demonstrate good face validity and high criterion validity when the scores are
correlated with American College Test (ACT), r = .659 and with the CCTST, r =
.645. However, there is no report on significant p-values.

46
The Health Sciences Reasoning Test (HSRT)
The Health Sciences Reasoning Test (HSRT) is a test of critical thinking
skills for health care professionals (i.e., dentistry, nursing, pharmacy, optometry,
occupational therapy, physical therapy, medical technology). The HSRT was
developed for use by educators and researchers to assess the critical thinking skills of
health sciences students and health sciences professionals. However, no discipline-
specific health sciences content knowledge is presumed on the HSRT. The HSRT is
a comparable instrument to California Critical Thinking Skills Test (CCTST), but it
is especially designed to suit more directly to those in the health sciences (Facione &
Facione, 2006). The Test consists of a standardized, 33-item multiple choice test that
targets the three core college-level critical thinking abilities: analyzing, drawing, and
evaluating inferences.
The HSRT was piloted in a number of college level and professional
employee settings (Facione & Facione, 2006). A large validation sample was
collected at an international conference of health care providers of mixed
professional field. Other groups were predominantly nursing students at college
level. The data from 2004-2005 validation studies with 444 participants shows Kuder
Richardson-20 (KR-20) internal reliability coefficient for HSRT total score ranging
from .77 to .83, with an overall internal consistency of .81 (Facione & Facione,
2006). The KR-20 is comparable statistic to Cronbach’s alpha used for
dichotomously scored instruments and scales. Facione and Facione (2006) claim that
an instrument with multidimensional scales, a KR-20 above .70, indicates a high
47
level of internal consistency. Table 1 shows the HSRT subscale scores along with
HR-20 internal consistency coefficients (Facione & Facione, 2006).

Table 1. HSRT Sub-scales and KR-20 Internal Consistency Coefficients
HSRT Subscale KR-20 Coefficient
Inductive Reasoning .76
Deductive Reasoning .71
Analysis .54
Inference .52
Evaluation .77

Table 2 shows CCTST score positive correlations with other measures such
as GRE, ACT, SAT, Watson-Glaser CTA, and college GPA (Facione & Facione,
2006). An interesting fact, this table reports that age and the number of units earned
from college work shows no significant correlation with critical thinking skills.

Table 2. CCTST Correlations with Other Measures
Measure r N p-value
GRE Total .719 143 < .001
GRE Analytic .708 143 < .001
GRE Verbal .716 143 < .001
GRE Quantitative .582 143 < .001
ACT .402446< .001
SAT Verbal .545 123 < .001
SAT Math .422 123 < .001
CCTDI Total .201 1557 < .05
Watson-Glaser CTA .405 139 < .001
College GPA .20 473 < .001
Age -.006479.449
College Units Earned .03 473 .262

48
Implications
There are three important implications that can be drawn relating to how
problem-based learning (PBL) affects the critical thinking skills development of
medical and dental students.
The first implication is related to critical thinking skills and the associated
cognitive representations. PBL students generate more coherent explanations and
reflect a more developed and well connected representation. This resonates with the
cognitive structural development traditionalist, such as Piaget and Perry, who argue
that new information must be interpreted in terms of both prior knowledge and
shared perspectives (Barrows & Tamblyn, 1980; Evans, Forney, & Guido-Dibrito,
1998; Savin-Baden & Major, 2004; Schmidt, Dauphinee, & Patel, 1987). Thus, the
existing cognitive structure becomes the most important factor in shaping
meaningful learning. PBL advocates argue that students embark in any learning
environment with pre-existing cognitive structure and knowledge (Savin-Baden &
Major, 2004; Savin-Baden & Wilkie, 2004). In practice, Hmelo (1994) finds that for
PBL students, scientific concepts and clinical applications are integrated in a causal
network. This is responsible for the relatively greater access to science for the PBL
students. Therefore, this suggests that for the PBL students, science is not merely a
set of facts learned for a test rather it is an instrument for understanding. For PBL
students, learning science and clinical knowledge occur simultaneously and in
parallel that leads to a more integrated representation of knowledge. However, for
non-PBL students, the learning occurs only in the scientific knowledge.
49
The second implication is related to the relationship between students’
learning style and their critical thinking development. Kolb reports that there is a
positive relationship between student-discipline learning style and the development
of their academic performance, social adaptation, and career commitment (Evans,
Forney, & Guido-Dibrito, 1998; Friedman & Deek, 2002). Subsequently, if academic
disciplines were to be available to students with diverse learning styles, efforts must
be made to provide various methods of instructions. For medical and dental
education, PBL method is the answer for student’s discontent with problem-solving
learning method in the traditional lecture-centered curriculum. Savin-Baden and
Major (2004) suggest that curricula in which PBL is central to the learning are in fact
largely constructivist in nature because students make decisions about what counts as
knowledge. In practices, PBL method provides both supports to help students in
connecting with subject matter and to assist them in developing their preferred styles
so they can achieve the level of flexibility needed to respond to differing demands of
learning environment.
The third implication is related to self-directed learning and self-regulated
learning styles in the development of students’ critical thinking skills. In her study,
Hmelo (1994) finds that the PBL students’ self-directed learning is consistent with
their reasoning strategies. They are more likely to take a hypothesis-driven approach
to identify their medical and clinical knowledge deficiencies (Nendaz & Tekian,
1999). As described by Savin-Baden and Major (2004), students take the initiative to
recognize their learning needs, formulate learning goals, identify resources for
50
learning, implement learning strategies, and evaluate learning outcomes. In essence,
students become responsible in selecting, managing, and assessing their own
learning activities, which can be pursued at any time, in any place, and through any
means. For the students, self-directed learning involves initiating personal challenge
activities and developing the personal qualities to pursue them successfully.
Zimmerman and Lebeau (2000) interpret self-regulated learning approach as
backward hypothesis-driven reasoning of PBL students when dealing with
uncertainty of many patients’ medical problems. As PBL students acquire learning
experience, they construct hypothesis-related learning objectives with increasing
frequency in response to a problem. Moreover, the PBL students are expected to
define their learning issues in terms of relevant hypotheses. Hmelo (1994) also finds
that the PBL students have a better starting point for their self-directed learning due
to the extended amount of practice and reflection. Finally, deriving from Savin-
Baden and Major (2004)’s analysis, the most important implication of self-directed
learning in medical student development is the ability of PBL students to be better at
integrating new knowledge into their explanations of the clinical case and the
knowledge that has been integrated would be related to their hypotheses. This
inference is made because in program-based learning curriculum, students are
encouraged to apply the new knowledge to the problem rather than lecturing to each
other on their chosen learning issues. Thus, their educational experience prepares
them to use new knowledge as a tool for problem-solving.
51
Together, these implications should impact the development of students’
critical thinking skills in a PBL setting, specifically in helping students to utilize
their prior knowledge and cognitive structure, then forming them into a new
construct through self-directed learning initiative that is understandable and
meaningful to the students themselves.
Lastly, this literature review has revealed that certain attitudes and
dispositions are essential to critical thinking. Attitudes may foster or hinder the
development of critical thinking. Helping students develop sound critical thinking
skills is one of the principal goals of education. Educators have the task of defining
critical thinking, creating educational methods to support its development, and
assessing that students have achieved some acceptable level of the skill.
52
CHAPTER THREE
METHODOLOGY
Introduction
At the University of Southern California (USC), the problem-based learning
(PBL) program has been designed to address the recommendations developed by the
National Academy of Sciences Institute of Medicine in its report, Dental Education
at the Crossroads: Challenges and Change. This program presents the identical set
of curricular learning outcomes that are approved by the American Dental
Association Council on Dental Accreditation. In fact, the dental PBL program
represents an alternative approach to the School of Dentistry curriculum rather than a
new curriculum. Subsequently, the objective of this PBL program is to educate
students who, as professionals, will be committed to a lifelong, self-motivated
learning, skilled in the methods of problem solving in a clinical setting, well-
prepared to deal with the future advances in dental therapy and dental care delivery,
able to deal with the medical presentations of dental patients, effective in group
learning, and highly skilled in the delivery of dental health care of outstanding
quality (About USCSD, n.d.).
This chapter on methodology describes the study design, sample and
population, instrumentation and data collection, validity and reliability, and data
analysis process of the proposed study. As previously mentioned in chapter one, the
purpose of this quantitative study is to assess whether or not problem-based learning
(PBL) pedagogical method in dental education improves critical thinking skills of the
53
students. The focus is to illuminate the impact of PBL curriculum that has been
implemented at the University of Southern California School of Dentistry (USCSD)
toward students’ critical thinking skills.

Study Design
The study utilized a quantitative research design methodology and a
psychometric instrument called the Health Sciences Reasoning Test (HSRT). A
cross-sectional, parametric multiple independent groups analysis of variance
(ANOVA), comparing critical thinking skills among the three year levels of dental
students, was used in this non-experimental study. The independent variable for
ANOVA was the year level, and the dependent variables were the HSRT total score
and sub-scales scores.
Analysis of covariance (ANCOVA) with gender, race, age, English as first
language, and education level as covariates, was a part of the study. The correlations
between the HSRT total score and sub-scales scores with Dental Admission Test
(DAT) score and GPA was evaluated.

Sample and Population
The USC School of Dentistry admits 144 students each year for the PBL
curriculum leading to the Doctor of Dental Surgery (About USCSD, n.d.). The target
population of the study was the first-year, second-year, and third-year students of the
dental program. The fourth-year students were excluded since they have already
54
graduated when the test was administered. Also, the fourth-year students are
typically no longer in PBL classes anymore.
Convenience sample by way of volunteerism was used to identify the
participants of the study who were drawn during the same academic year for each
level. The sample consisted of three year levels or groups between 30 to 36
participants for each year level. The total number of participants was planned for 100
students.

Instrumentation and Data Collection Procedures
A psychometric instrument, namely the Health Sciences Reasoning Test
(HSRT), was administered to the first- through third-year USCSD students. Between
30 to 36 dental students at each year level took the HSRT which is shown in
Appendix A. Each student or subject took the HSRT one time.
Data for this study were collected on the HSRT answer sheets (CapScore™)
through the use of the HSRT instrument that consisted of 33-item multiple choice
format test. Items presented necessary informational content in text-based and
diagrammatic formats. Questions encouraged students to draw inferences, to make
interpretations, to analyze information, to identify claims and reasons, and to
evaluate the quality of arguments.
Insight Assessment scanned and scored the CapScore.™ All scores were
returned in a digital file format that contains: 1) total and subscale HSRT scores for
55
each test taker by ID number; 2) descriptive statistics for the group of test takers as a
whole.
Additionally, students who took the HSRT also self-reported their
demographics information (i.e., gender, race, age, English as first language) and
academic information (i.e., education level, GPAs, Dental Admission Test (DAT)
score) both on the CapScore™ (Appendix B) and Demographics/Academic
Questionnaire (Appendix C).

Validity and Reliability
Psychometric validity can be considered as content validity and construct
validity. The Delphi Report (Facione, 1990) describes content validity: “The strategy
or instrument should be based on an appropriate conceptualization of critical
thinking and a clear understanding of which aspects of critical thinking the
assessment targets.” Furthermore, the Report describes construct validity: “each task
or question should have been evaluated to insure that students who answer correctly
do so on the basis of good critical thinking and that inadequate or wrong responses
are the result of weak or inadequate critical thinking.” Finally, the Report describes
reliability: “In acceptable critical thinking assessment each task or question should
have been evaluated to insure that good critical thinkers generally do better on that
item than weak critical thinkers.”
The data from 2004-2005 validation studies produced Kuder Richardson-20
(KR-20) internal reliability coefficient for HSRT total score ranging from .77 to .83
56
with an overall internal consistency of .81 (N = 444) (Facione & Facione, 2006). The
KR-20 is comparable statistic to Cronbach’s alpha used for dichotomously scored
instruments and scales. Facione and Facione (2006) argue that an instrument with
multidimensional scales, a KR-20 above .70 indicates a high level of internal
consistency.
The subscale of inductive and deductive reasoning has KR-20 of .76 and .71,
respectively. Likewise, the subscale of analysis, inference, and evaluation has KR-20
of .54, .52, and .77, respectively.

Data Analysis and Reporting Procedures
Both descriptive and inferential statistical analyses were carried out. The
overall critical thinking skills or the HSRT total score were reported. The HSRT total
score targets the strength or weakness of students’ skill in making reflective,
reasoned judgments about what to believe or what to do (Facione & Facione, 2006).
Additionally, the HSRT also generates five sub-scales scores relating to
critical thinking. Sub-scales scores by contemporary categories are analysis,
inference, and evaluation. Sub-scales scores by the classical categories are inductive
reasoning and deductive reasoning.
Electronic file reporting was in an Excel or a tab-delimited text file format.
The file contained the total and subscales HSRT scores for each test taker identified
by ID number. Additionally, the content of the file also included descriptive statistics
for the group of test takers as a whole.
57
The HSRT data along with demographics and academic data were merged to
create the study database. Data analysis that involved statistical tests was achieved
using statistical software such as SPSS and/or SAS.
As stated in the purpose of the study, the analysis was primarily focus on
measuring students’ critical thinking skills achievement starting from their first-
through third-year of dental education. Students are expected to gain continuous and
incremental improvement in their critical thinking skills achievement during their
PBL-based dental education at USCSD.

Ethical Consideration
The study involved a normal educational practice in an established PBL
educational setting within USDSD. The research was conducted on the effectiveness
of PBL instructional technique or curriculum. The study was conducted using the
HSRT, a psychometric educational test.
The study adhered to the Institutional Review Board (IRB) rules, regulations,
and procedures to the fullest in order to achieve the highest ethical standards
possible. The informed consent form was provided for participants of the study. The
informed consent form ensured study’s participants about the key facts and purpose
of the research study and what their participation will involve. The human subjects in
the study understood their participation is on volunteer basis, after having been
adequately informed about the research.

58
Limitations of the Study
The study had some limitations. Subsequently, the findings warrant looking
at what limitations might have influenced the results of this study.
The internal validity of the study must carefully be evaluated. Internal
validity is not concerned with the generality but with the integrity of the study itself
(Locke, Silverman, & Spirduso, 2004). In this study, it is important to consider if
there are any other factors besides PBL curriculum used by the USCSD that may
have caused the outcome of critical thinking skills measurement.
There is no agreed approach to measuring critical thinking skills (Newman,
Ambrose, Corner, Evans, Morris-Vincent, Quinn, Stephenson, & Vernon, 2003). The
HSRT, which was derived from the CCTST is based on the consensus view of the
critical thinkers produced by the American Philosophical Association and has
undergone extensive testing and validating (Facione, 1990). Numerous criticisms
have been made to the CCTST. The main one lies in the way that critical thinking is
conceptualized independently of context (Newman et al., 2003). PBL is based on
principles derived from cognitive psychology that contends knowledge is structured
in semantic networks (Norman & Schmidt, 1992). PBL cases create a semantic
structure for learning of new knowledge that is similar to semantic structure in which
the new knowledge will enable the recall of required knowledge. Therefore, it may
be a possible source of invalidity to use context free critical thinking tests to measure
outcomes achieved by PBL method.
59
CHAPTER FOUR
RESULTS
Introduction
The purpose of this study was to assess whether problem-based learning
(PBL) pedagogical method in dental education improved critical thinking skills of
the students. The quantitative analysis was mainly focused on measuring students’
critical thinking skills achievement starting from their first- through third-year of
dental education. The study used a psychometric instrument known as the Health
Sciences Reasoning Test (HSRT) that consisted of 33-item multiple choice format
test. The HSRT produced the overall critical thinking skills score or the total score.
In addition, the HSRT also generated five sub-scales scores relating to critical
thinking: analysis, inference, evaluation, deductive, and inductive reasoning.
This chapter describes the research findings of this non-experimental study in
which both descriptive and inferential statistical analyses were carried out. A cross-
sectional, parametric multiple independent groups analysis of variance (ANOVA),
comparing critical thinking skills among the three-year levels of dental students, was
utilized. Moreover, analysis of covariance (ANCOVA) was used with age, gender,
race, English as first language, and education level serving as the covariates.
ANCOVA was carried out using the General Linear Model (GLM) approach by
employing SPSS statistical software. GLM was used as multivariate design because
it can provide regression analysis and analysis of variance for multiple dependent
variables by one or more factor variables or covariates. Lastly, the correlation
60
between the total score and sub-scales scores with Dental Admission Test (DAT)
score and Grade Point Average (GPA) was evaluated.

Participants
The total number of participants was 98 students (98% out of 100 students).
The participants consisted of the first-, second-, and third-year students of the dental
program at the University of Southern California. Fourth-year students, who have
graduated, were not included. This exclusion was also decided due to the fact that the
fourth-year dental students were no longer engaged in PBL classes. A convenience
sample by way of volunteerism was used to identify the participants of the study who
were withdrawn during the 2007-2008 academic year. The HSRT was administered
during summer 2007 and at the same time additional demographics and academic
data were also collected.
Table 3 shows that the sample consisted of three groups between 28 to 36
participants for each year level or class.

Table 3. Study Participants
Year Level Class of N %
First 2010 36 36.7
Second 2009 34 34.7
Third 2008 28 28.6

61
Hypothesis
Students are expected to gain continuous and incremental improvement in
their critical thinking skills achievement during their PBL-based dental education at
the University of Southern California School of Dentistry.

Descriptive Analysis of Research Findings
Table 4 provides the demographics and characteristics description of study
participants. Of these 98 dental student participants, 60 (61.2%) were male and 38
(38.8%) were female. The majority of participants (62.9%) considered themselves to

Table 4. Demographics Description of Study Participants
1
st
Year 2
nd
Year 3
rd
Year Total
Demographics Category
N (%)
Gender:
Male
Female
20 (55.6)
16 (44.4)
23 (67.6)
11 (32.4)

17 (60.7)
11 (39.3)
60 (61.2)
38 (38.8)
Race:
African American
Anglo
Asian/Pacific Islander
Hispanic/Latino
Native American
Mixed/Other
3 (8.3)
16 (44.4)
11 (30.6)
1 (2.8)
0 (0.0)
5 (13.9)
0 (0.0)
26 (78.8)
4 (12.1)
0 (0.0)
0 (0.0)
3 (9.1)

0 (0.0)
19 (67.9)
4 (14.3)
0 (0.0)
0 (0.0)
5 (17.9)
3 (3.1)
61 (62.9)
19 (19.6)
1 (1.0)
0 ( 0.0)
13 (13.4)
English as First Language:
Yes
No
27 (77.1)
8 (22.9)
30 (90.9)
3 (9.1)

23 (82.1)
5 (17.9)
80 (83.3)
16 (16.7)
Highest Education Level:
Bachelor
Masters
32 (88.9)
4 (11.1)
32 (97.0)
1 (3.0)

26 (92.9)
2 (7.1)
90 (92.8)
7 (7.2)

62
be Anglo American or White. Eighty-three percents of the participants stated that
English was their first language. Most of the participants, 92.8 percents, had at least
bachelor’s degree before they enrolled into the dental program.
Table 5 describes the descriptive statistics of the study participants’ age and
academic data. The mean age was 25.66 (SD = 2.761). Academically, the
participants had a mean undergraduate GPA of 3.53 (SD = .220), a mean graduate
GPA of 3.60 (SD = .240), and a mean DAT score of 20.60 (SD = 2.171).

Table 5. Descriptive Statistics of Study Participants’ Age and Academic Data

1
st
Year 2
nd
Year 3
rd
Year Total

Mean (SD)
Age 24.33 (1.912) 26.06 (3.418) 26.89 (2.097) 25.66 (2.761)
Undergrad. GPA 3.52 (.219) 3.52 (.217) 3.55 (.229) 3.53 (.220)
Graduate GPA 3.68 (.273) 3.53 (.205) 3.58 (.210) 3.60 (.240)
DAT Score 21.11 (1.997) 20.10 (2.249) 20.48 (2.238) 20.60 (2.171)

Table 6 summarizes the statistics for HSRT total score for all participants as a
group. The overall mean total score was 22.30 (SD = 4.633). The median total score
was 23.00 and the mode was 26. The minimum and maximum total score were 8 and
32, respectively.
The frequency distribution histogram of HSRT total score is displayed in
Figure 1. The histogram graph appears jagged due to the number of scores in the
sample. The maximum point of the normal curve can be viewed around the mean
total score, which is 22.30.

Table 6. Statistics of HSRT Total Score for Entire Study Participants
N 98
Mean 22.30
Std. Error of Mean .468
Median 23.00
Mode 26
Std. Deviation 4.633
Variance 21.468
Skewness -.668
Range 24
Minimum 8
Maximum 32

Figure 1. Frequency Distribution Histogram of HSRT Total Score
HSRT Total Score
32.5 30.0 27.5 25.0 22.5 20.0 17.5 15.0 12.5 10.0 7.5
30
20
10
0

63
64
Table 7 describes the rubric of HSRT Delphi Sub-scales as explained by
Facione and Facione (2006). Each of the contemporary category sub-scales uses 6
items to measure three areas: analysis, inference, and evaluation. For each of these
sub-scales, a score of 0, 1, or 2 indicates weakness, scores of 3 or 4 indicates
average, and scores of 5 or 6 indicates strength in this critical thinking skills area.
Correspondingly, each of the classical category sub-scales uses 10 items to measure
two areas: deductive and inductive reasoning. For each of these sub-scales, a score of
0, 1, 2, or 3 indicates weakness, scores of 4, 5, 6, or 7 indicates average, and scores
of 8, 9, or 10 indicates strength in this critical thinking skills area.

Table 7. Rubric of HSRT Delphi Sub-scales
Measured Areas Weak Average Strong
Analysis
Inference
Evaluation
0, 1, 2 3, 4 5, 6
Deductive
Inductive
0, 1, 2, 3 4, 5, 6, 7 8, 9, 10

Table 8 outlines the participants’ mean critical thinking total score and sub-
scales scores for each year level or class. Using Table 6 of the HSRT Delphi Sub-
scales description, the results of students’ critical thinking sub-scales scores can be
interpreted.

65
Table 8. Mean (SD) and ANOVA of HSRT Total Score and Sub-scales Scores
Measured Area First Year Second Year Third Year F p
Total 22.31 (4.503) 23.47 (3.979) 20.86 (5.254) 2.519 .086
Analysis 4.33 (1.069) 4.74 (1.163) 4.29 (1.013) 1.693 .189
Inference 3.83 (1.298) 3.82 (1.466) 3.36 (1.393) 1.158 .318
Evaluation 5.14 (1.222) 5.26 (.864) 4.71 (1.718) 1.515 .225
Deductive 6.94 (2.229) 7.24 (1.810) 6.86 (1.880) .318 .729
Inductive 7.67 (1.474) 8.03 (1.359) 6.89 (2.266) 3.507 .034*
* indicates p < 0.05 for difference among means

Analysis as used on the HSRT sub-scale means “to comprehend and express
the meaning or significance of a wide variety of experiences, situations, data, events,
judgments, conventions, beliefs, rules, procedures or criteria (Facione & Facione,
2006, p. 9).” Furthermore, it also means “to identify the intended and actual
inferential relationships among statements, questions, concepts, descriptions or other
forms of representation intended to express beliefs, judgments, experiences, reasons,
information or opinions” (Facione & Facione, 2006, p. 9). The mean (SD) analysis
scores for first-, second, and third-year students were 4.33 (1.069), 4.74 (1.163), and
4.29 (1.013), respectively. The overall group’s mean (SD) analysis score was 4.46
(1.095), which was considered to be in between average and strong.
Inference, as used on the HSRT, means to identify elements that were needed
to draw reasonable conclusions, to form hypotheses, and to consider relevant
information (Facione & Facione, 2006). The mean (SD) inference scores from first-
to third-year students decreased from 3.83 (1.298) to 3.36 (1.393). The overall
66
group’s mean (SD) inference score was 3.69 (1.388), which was considered to be the
average.
Evaluation, as used on the HSRT, means to assess the credibility of
statements and the logical strength of the actual or intended inferential relationships
among statements, descriptions, or questions. It also means to state the results of
reasoning, to justify reasoning in terms of the evidential, conceptual, methodological,
and contextual considerations (Facione & Facione, 2006). The mean (SD) evaluation
scores from first- to third-year students were 5.14 (1.222), 5.26 (.864), and 4.71
(1.718), respectively. The overall group’s mean (SD) evaluation score was 5.06
(1.291), which was considered to be strong.
Deductive reasoning, as used on the HSRT sub-scale, means “the assumed
truth of the premises purportedly necessitates the truth of conclusion” (Facione &
Facione, 2006, p. 10). The mean (SD) deductive scores from first- to third-year
students increased were 6.94 (2.229), 7.24 (1.810), and 6.86 (1.880), respectively.
The overall group’s mean (SD) deductive score was 7.02 (1.979), which was
considered to be average.
Inductive reasoning, as used on the HSRT, means “an argument’s conclusion
is purportedly warranted, but not necessitated, by the assumed truth of its premises”
(Facione & Facione, 2006, p. 10). The mean (SD) inductive scores for first-, second,
and third-year students were 7.67 (1.474), 8.03 (1.359), and 6.89 (2.266),
respectively. The overall group’s mean (SD) inductive score was 7.57 (1.747), which
was considered to be near strong.
Figure 2 illustrates the changes in mean critical thinking total score of the
Health Science Reasoning Test (HSRT) across the class levels of dental program.
With maximum total score of 33, the mean score for first-, second-, and third-year
class respectively were 22.31, 23.47, and 20.86 as shown in Table 8.
One-way ANOVA was used to test for significant differences among the
three year class levels with respect to participants’ mean HSRT total score. The
result in Table 8 shows that the mean total scores among the three classes did not
differ significantly, F(2,95) = 2.519, p = .086.

Figure 2. Mean HSRT Total Score
19.5
20
20.5
21
21.5
22
22.5
23
23.5
24
1st Year 2nd Year 3rd Year

67
Similar results also emerged when comparing first-year class with second-
year class (t(68) = -1.149, p = .255), and first-year class with third-year class (t(62) =
1.164, p = .250). However, there was a significant difference between second-year
class and third-year class, t(60) = 2.169, p = .035.
Figure 3 illustrates the changes in mean critical thinking sub-scales scores in
contemporary category of the HSRT across class levels of dental program as shown
in Table 8. The mean scores of analysis skills for first-, second-, and third-year class
were 4.33, 4.74, and 4.29, respectively. The mean scores of inference skills for first-,
second-, and third-year class were 3.83, 3.82, and 3.36, respectively. The mean
scores of evaluation skills for first-, second-, and third-year class were 5.14, 5.26,
and 4.71, respectively.

Figure 3. Mean HSRT Sub-scales Scores in Contemporary Category
0
1
2
3
4
5
6
1st Year 2nd Year 3rd Year
Analysis
Inference
Evaluation

68
One-way ANOVA was used to test for significant differences among the
three class levels with respect to participants’ HSRT sub-scales scores in
contemporary category. Table 8 shows that the mean scores for analysis skills among
the three classes did not differ significantly, F(2,95) = 1.693, p = .189. The same
result also emerged for both inference and evaluation skills, F(2,95) = 1.158, p =
.318 and F(2,95) = 1.515, p = .225, respectively.
Figure 4 illustrates the changes in mean critical thinking subs scores in
classical category of the HSRT across class levels of dental program. Table 8 shows
that the mean scores of deductive reasoning skills for first-, second-, and third-year
class were 6.94, 7.24, and 6.86, respectively. The mean scores of inductive reasoning
skills for first-, second-, and third-year class were 7.67, 8.03, and 6.89, respectively.

Figure 4. Mean HSRT Sub-scales Scores in Classical Category
6.2
6.4
6.6
6.8
7
7.2
7.4
7.6
7.8
8
8.2
1st Year 2nd Year 3rd Year
Deductive
Inductive

69
70
One-way ANOVA was used to test for significant differences among the
three class levels with respect to participants’ HSRT subs scores in classical
category. Table 8 shows that the mean scores among the three classes differ
significantly for inductive reasoning skills (F(2,95) = 3.507, p = .034), but not for
deductive reasoning skills (F(2,95) = .318, p = .729).
Table 9 summarizes the mean HSRT total scores for participants based on
their demographics and academic profiles. Independent-samples T-test was used to
test for significant differences among gender, race, English as first language, and
highest education level with respect to participants’ HSRT total score. The results
show that the mean total score for male students differ significantly from those of
female students, t(96) = 3.005, p = .004. Likewise, the results also show that there

Table 9. HSRT Total Score Based on Demographics Data
Demographics Category N Mean SD p
Gender:
Male
Female

60
38
23.38
20.58
4.357
4.589

.004*
Race:
White
Non White

61
36
23.02
21.11
4.349
4.967

.061
English as First Language:
Yes
No

80
16
22.84
19.75
4.350
5.520

.048*
Highest Education Level:
Undergraduate
Graduate

90
7
22.67
18.57
4.500
4.429

.051
* indicates p < 0.05 for difference between means

71
was a significant difference in the mean scores between English as first language
group and the non English group, t(94) = 2.110, p = .048.
Table 10 summarizes the mean HSRT sub-scales scores in contemporary
category for participants based on their demographics and academic profiles. T-test
was used to test for significant differences among gender, race, English as first
language, and highest education level with respect to participants’ analysis,
inference, and evaluation scores. There were significant differences in the mean
scores in the analysis skills between male and female students (t(96) = 2.152, p =
.035), and between White and non White students (t(95) = 3.054, p = .003).

Table 10. Summary of Mean HSRT Sub-scales Scores in Contemporary Category
Based on Demographics Data

Analysis Inference Evaluation
Demogra-
phics
Category
N
Mean (SD) p Mean (SD) p Mean (SD) p
Gender:
Male
Female

60
38

4.65 (1.022)
4.16 (1.151)

.035*

3.92 (1.430)
3.34 (1.258)

.040*

5.17 (1.167)
4.89 (1.467)

.338
Race:
White
Non White

61
36

4.70 (1.054)
4.03 (1.055)

.003*

3.66 (1.353)
3.81 (1.451)

.616

5.20 (1.123)
4.81 (1.527)

.186
English as 1st
Language:
Yes
No

80
16

4.56 (1.041)
3.94 (1.289)

.084

3.71 (1.398)
3.69 (1.401)

.949

5.20 (1.095)
4.31 (1.922)

.012*
Highest Edu-
cation Level:
Undergrad.
Graduate

90
7

4.50 (1.114)
4.14 (.690)

.245

3.76 (1.401)
3.00 (1.155)

.142

5.13 (1.265)
4.14 (1.464)

.127
* indicates p < 0.05 for difference between means

72
Moreover, there was a significant difference in the mean scores in the inference
skills between male and female students, t(96) = 2.088, p = .040. Lastly, a significant
difference in the mean scores also existed in the evaluation skills between English as
first language group and non English group, t(94) = 2.563, p = .012.
Table 11 summarizes the mean HSRT sub-scales scores in classical category
for participants based on their demographics and academic profiles. T-test was used
to test for significant differences among gender, race, English as first language, and
highest education level with respect to participants’ deductive and inductive
reasoning scores. For deductive reasoning, there were significant differences in the

Table 11. Summary of Mean HSRT Sub-scales Scores in Classical Category Based
on Demographics Data

Deductive Inductive
Demographics Category N
Mean (SD) p Mean (SD) p
Gender:
Male
Female

60
38
7.40 (1.796)
6.42 (2.126)
.021*

7.92 (1.576)
7.03 (1.881)
.018*
Race:
White
Non White

61
36
7.39 (1.706)
6.44 (2.273)
.022*

7.74 (1.611)
7.28 (1.966)
.239
English as First Language:
Yes
No

80
16
7.30 (1.782)
5.75 (2.490)
.004*

7.73 (1.630)
6.75 (2.206)
.110
Highest Education Level:
Undergraduate
Graduate

90
7
7.16 (1.937)
5.71 (1.976)
.105

7.70 (1.706)
6.14 (1.773)
.060
* indicates p < 0.05 for difference between means

73
mean scores between male and female students (t(96) = 2.355, p = .021), between
White and non White students (t(95) = 2.334, p = .022), and between English as first
language group and the non English group (t(94) = 2.959, p = .004). Meanwhile for
inductive reasoning, the significant difference in the mean scores only occurred
between male and female students, t(96) = 2.427, p = .018.
All three groups of the study’s participants were not randomly assigned to
each of their respected group. Consequently, it cannot assume that the level of
participants’ prior critical thinking skills were equal to begin with. Therefore, it was
necessary to perform the ANCOVA analysis in order to control or partial out the
effect of age, gender, race, English as first language, and education level.
Table 12 shows the results of ANCOVA analysis to test the effect between
the groups on the total and sub-scales scores. The first result concludes that the
adjusted mean total score of critical thinking skills among the three class levels was

Table 12. ANCOVA Results for HSRT Total Score and Sub-scales Scores

Measured Area F p
Total 2.260.110
Analysis 1.082.343
Inference .398.673
Evaluation 1.262.288
Inductive .172.842
Deductive 2.283.108
Covariates: age, gender, race, English as first language, and education level

74
not statistically significant, F(2,95) = 2.260, p = .110. Likewise, other results for
analysis, inference, evaluation, inductive, and deductive skills also yielded the same
conclusion.
Pearson’s correlations between the different measures are found in Table 13.
Pearson’s correlations represent the linear relationship between two variables.
Accordingly, Table 13 shows the overall correlations between total score and sub-
scales scores with DAT score, graduate GPA, and undergraduate GPA were low.
Furthermore, correlations were remarkably low between all HSRT scores and DAT
score. This might be an effect of lack of relevancy between the two variables.

Table 13. Correlations Between HSRT Scores and Academic Data
DAT
Graduate
GPA
Undergraduate
GPA
Total
-.048
p = .655
.252
p = .017*
.064
p = .538
Analysis
-.131
p = .220
.138
p = .196
-.001
n = .995
Inference
.059
p = .582
.172
p = .104
-.001
p = .995
Evaluation
-.138
p = .196
.261
p = .013*
.138
p = .201
Deductive
-.032
p = .766
.217
p = .040*
.011
p = .914
Inductive
-.084
p = .431
.306
p = .003*
.088
p = .396
* indicates p < .05

75
Statistically significant relationships existed only between the correlations of
graduate GPA with: total score (r(90) = .252, p = .017), evaluation score (r(90) =
.261, p = .013), deductive score (r(90) = .217, p = .040), and inductive score (r(90) =
.306, p = .003).

76
CHAPTER FIVE
DISCUSSION
Summary
This study measured the change of critical thinking skills of dental students
in a problem-based learning (PBL) pedagogical method. Based on the results
described in Chapter 4, it was concluded that students’ critical thinking skills showed
no significant improvement as they progress from first- through third-year of PBL-
based dental education at the University of Southern California (USC). In a matter of
fact, the mean total score for second year students increased by +1.16 (+5.2%)
compared to that of first year students. However, the mean total score for third year
students decreased by -2.61 (-11.12%) compared to that of second year students.
ANCOVA analysis using age, gender, race, English as first language, and education
level as covariates also yielded the same conclusion.

Hypothesis
Students are expected to gain continuous and incremental improvement in
their critical thinking skills achievement during their PBL-based dental education at
the University of Southern California School of Dentistry.
Based on the evidence in this study, the hypothesis is rejected.

77
Rationale of the Study
Assessing critical thinking is an important undertaking. PBL method that has
been a part of USC School of Dentistry curriculum, encourages students to engage in
critical thinking. Therefore, students must involve thoughtful decision-making
process based on their ability to make purposeful judgments that involve analysis,
inference, evaluation, and explanation. Since PBL dental students are taught these
cognitive skills, the dental program ought to show evidence that its students have
developed the critical thinking skills.
According to Zimmerman and Lebeau (2000), PBL students are expected to
gradually improve their critical thinking skills. They are typically to be more capable
in learning new knowledge due to the activation of prior knowledge and
reorganization of newly acquired knowledge. Thomas (1997) also discovers that
PBL can actually help students develop advanced cognitive abilities, such as critical
thinking, problem solving, and interpersonal communication skills. Clinical PBL
courses are expected to improve students’ professional knowledge acquisition and
critical thinking skills (Birgegard & Lindquist, 1998; Maudsley & Strivens, 2000).
Furthermore, a study that was conducted by Fincham and Shuler (2001) at USC
School of Dentistry concludes that PBL-track students demonstrated a statistically
significant (t = -6.5, p = .000) superior performance on the Part I National Dental
Board Examination as compared with their peers in the traditional track.
On the contrary, studies conducted by Barrows (1994) and Hoffman,
Hosokawa, Blake, Headrick, and Johnson (2006) find that students who participated
78
in PBL curriculum score the same or lower on standardized examinations than do
traditional curriculum students. Previously, Colliver (2000) concludes that the
effectiveness of PBL curriculum in medical education in randomized studies showed
no effect for PBL and for most of non-randomized studies demonstrate small effect
that may be explained by selection differences among PBL students. Earlier in 1993,
Albanese and Mitchell conclude that most medical education studies in their meta-
analysis study showed no conclusive evidence that PBL students performed better
than non-PBL students in acquiring sciences. PBL graduates were inclined to engage
in backward reasoning rather than the forward reasoning experts engage in and there
appeared to be gaps in their cognitive knowledge base. Moreover, a 1993 study
conducted by Vernon and Blake find that non-PBL students performed significantly
better than their PBL counterparts on the National Board of Medical Examiners Part
I (dw = -.18, CI.95 = -.10 to -.26). Similarly, another study conducted by Schmidt,
Dauphinee, and Patel in 1987 shows that students in non-PBL programs score higher
on tests of medical knowledge than students in PBL curriculum.
The PBL theory has continued to gain support from a number of educational
educators (e.g., Barrows, 1986; Bridges & Hallinger, 1997; Friedman & Deek,
2002; Major & Palmer, 2001; Savin-Baden, 2000) who argue that by using problems
from actual clinical cases to approach learning content, students develop exceptional
skills in critical thinking. Numerous higher education institutions are looking to
assess and validate students’ critical thinking skills improvement either for
accountability or quality satisfaction enhancement objectives (Stein, Haynes, &
79
Unterstein, 2003). Although the USC School of Dentistry has been implementing
and practicing PBL curriculum for more than ten years, there has been no assessment
to measure the critical thinking skills improvement among these dental students.
While this study investigated dental students, it should interest and benefit educators
in other health sciences disciplines as well because critical thinking skills are
fundamental components of clinical reasoning (Birgegard & Lindquist, 1998;
Maudsley & Strivens, 2000). This study was expected to validate that PBL, as a
teaching method, can improve students’ critical thinking skills.

Results
As more and more higher education institutions look to teaching and testing
the critical thinking skills of students, assessments that are reliable in many settings
and accurately test critical thinking skills are needed. The results of this study
indicate that the Health Sciences Reasoning Test (HSRT) may be a valuable
instrument for assessing critical thinking skills, particularly in the field of health
sciences. Accordingly, the results of this study indicate the following findings:
1. The mean HSRT total scores among the three level of classes did not differ
significantly (F(2,95) = 2.519, p = .086) using a one-way ANOVA. Thus,
students showed no continuous and incremental improvement in their critical
thinking skills achievement during their PBL-based dental education.
2. Although there was significant difference in the mean total score between
second- and third-year class (t(60) = 2.169, p = .035), the results of
80
comparing first- with second-year class (t(68) = -1.149, p = .255), and first-
with third-year class (t(62) = 1.164, p = .250) revealed no significant
difference in the mean total score.
3. The mean scores for analysis skill among the three classes did not differ
significantly (F(2,95) = 1.693, p = .189). The same result also emerged for
both inference and evaluation skills, F(2,95) = 1.158, p = .318 and F(2,95) =
1.515, p = .225, respectively. The results of these three sub-scales scores that
represented the critical thinking contemporary category were consistent with
that of the total score.
4. While the mean scores among the three classes differ significantly for
inductive reasoning skills (F(2,95) = 3.507, p = .034), the mean scores among
the three classes did not differ significantly (F(2,95) = .318, p = .729) for
deductive reasoning skills (F(2,95) = .318, p = .729).
5. The results show that the mean total score for male students differed
significantly (t(96) = 3.005, p = .004) from that of female. Likewise, the
results also show that there were significant differences (t(94) = 2.110, p =
.048) between English as first language group and the non English group.
6. In the contemporary category, there were significant differences in mean
scores in the analysis skills between male and female students (t(96) = 2.152,
p = .035), and between White and non White students (t(95) = 3.054, p =
.003). Moreover, there was also significant difference in mean scores in the
inference skills between male and female students (t(96) = 2.088, p = .040).
81
7. In the classical category, for deductive reasoning, there were significant
differences in mean scores between male and female students (t(96) = 2.355,
p = .021), between White and non White students (t(95) = 2.334, p = .022),
and between English as first language group and the non English group (t(94)
= 2.959, p = .004). Meanwhile for inductive reasoning, the significant
difference in mean scores only occurred between male and female students
(t(96) = 2.427, p = .018).
8. The adjusted mean total score of critical thinking skills among the three class
levels was not statistically significant (F(2,95) = 2.260, p = .110). Likewise,
other results for analysis, inference, evaluation, inductive, and deductive
skills yielded the same conclusion.
9. The overall correlations between total score and sub-scales scores with DAT
score, graduate GPA, and undergraduate GPA were low. Statistically
significant relationships existed only between the correlations of graduate
GPA with:
a) total score (r(90) = .252, p = .017);
b) evaluation skills score (r(90) = .261, p = .013);
c) deductive reasoning score (r(90) = .217, p = .040);
d) inductive reasoning score (r(90) = .306, p = .003).

82
Discussions
The results of this study concluded that students showed no continuous and
significant incremental improvement in their critical thinking skills achievement
during their PBL-based dental education in the University of Southern California.
This conclusion was drawn as a result of the analyses of the mean total HSRT score
comparison among the three year levels of dental students. Except for the inductive
reasoning score, this result was very consistent for other four sub-scales scores as
well. Moreover, after performing the statistical adjustment on total score and sub-
scales scores, the same outcomes were produced. These results are in-line with
previous studies that have found students who participated in PBL curricula scored
the same or lower on standardized knowledge tests than do traditional curricula
students (Goodman, Brueschke, Bone, Rose, Williams, & Paul, 1991; Moore, Block,
Style, & Mitchell, 1994; Norman & Schmidt, 1992).
Goodman, Brueschke, Bone, Rose, Williams and Paul (1991) found that at
Rush Medical College, PBL curriculum students did not differ significantly from
their traditional curriculum peers on National Board of Medical Examiners, Part I
and Part II total scores. In their study to find the experimental evidence supporting
the difference in students’ learning that can be attributed to PBL, Norman and
Schmidt (1992) discover that there was no evidence that PBL curricula result in any
improvement in general, context-free problem-solving skills. Another study of 121
students from the entering classes of 1989 and 1990 at the Harvard Medical School
that was conducted by Moore, Block, Style, and Mitchell (1994) concludes that there
83
was no difference in problem-solving skills or biomedical knowledge base between
the randomly assigned the New Pathway (PBL) group and the traditional group.
On the other hand, the results of this study also showed that critical thinking
achievements differed by gender, race, and English as first language categories. In
race category, the comparison was performed by regrouping the students in White
(62.9%) and non White category (37.1%). Other than in the total score, the
differences also occurred in analysis, inference, deductive, and inductive scores.
In utilizing HSRT testing instrument for this study, English language
proficiency played an important role in measuring the critical thinking achievement.
The 33-item HSRT testing instrument was built based on an elaborate English
structure that required an advanced English knowledge. The language barrier was a
factor confounding measurement of critical thinking. Without a sufficient English
knowledge, participants would not have their critical thinking skills adequately
assessed. Cheung, Rudowicz, Kwan, and Yue (2002) conduct an assessment of
critical thinking to adapt to countries where English is not the primary language.
This adaptation is necessary in order to avoid the impediment of critical thinking
assessment due to language knowledge and to demonstrate the importance of
language in critical thinking assessment as argued by Ennis and Norris (1990).
Dental students who completed master’s degree did not differ in their critical
thinking skills from those with bachelor’s degree. These indifferences consistently
appeared in total score and all sub-scales scores. Thus, it seemed that the level of
higher education did not influence the critical thinking skills achievement.
84
At the same time, the results of this study also revealed students’
development in five dimensions of critical thinking disposition: analysis, inference,
evaluation, deductive reasoning, and inductive reasoning. The results ultimately
showed students’ strengths and weaknesses in the different dimensions, thus making
it possible for the necessary intervention.
The low correlations were reported between DAT score, graduate GPA, and
undergraduate GPA with HSRT total score and sub-scales scores. A possible
explanation for these findings might be the self-reporting of DAT score, graduate
GPA, and undergraduate GPA. The self-reporting tended to be an unreliable source
of data. Nevertheless, based on Table 2, the correlation between college GPA and
CCTST (California Critical Thinking Skills Test) score (r = .20, p < .001) was
comparable with correlation between graduate GPA and HSRT total score (r = .252,
p < .05). CCTST was the general test version of HSRT, which also tested five types
of thinking: analysis, inference, evaluation, deductive reasoning, and inductive
reasoning (Facione, 2007). Using the Watson-Glaser Critical Thinking Appraisal
(WGCTA), Scott and Markert (1994) found almost a similar result in which
correlation between WGCTA and GPA scores for the first two years of medical
school students was .33.
Although the correlations between HSRT total score and sub-scales scores
with DAT score and GPA were low within classes, the trend in mean total and sub-
scales scores across classes was very similar. The low correlations mean that within
a class, the students who performed best on the HSRT were not necessarily the same
85
students who performed best in DAT, college, and dental school. This may suggest
that the HSRT measured different forms of critical thinking.
Whether utilizing a PBL method or others, it is essential that one understands
the nature of critical thinking dispositions to the teaching, assessment, and practice
of critical thinking itself. These concerns require further research on critical thinking.
The initial evidence from this study indicates that the HSRT may be a useful
instrument for testing critical thinking abilities and researching the questions raised
in this study.

Limitations of Study
Experts have not agreed on an approach to measure critical thinking skills
(Newman, Ambrose, Corner, Evans, Morris-Vincent, Quinn, Stephenson, & Vernon,
2003). The HSRT, which was derived from the CCTST, is based on the consensus
view of the critical thinkers produced by the American Philosophical Association
and has undergone extensive tests and validations (Facione, 1990). Many criticisms
have been made on the CCTST. The main one lies in the way that critical thinking is
conceptualized independently from context (Newman et al., 2003). PBL is based on
principles derived from cognitive psychology that contends knowledge is structured
in semantic networks (Norman & Schmidt, 1992). PBL cases create a semantic
structure for learning of new knowledge that is similar to semantic structure in which
the new knowledge will enable the recall of required knowledge. Therefore, it seems
86
invalid to use context free critical thinking tests to measure outcomes achieved by
PBL method.
Although this study investigated critical thinking skills improvement of PBL
dental students, it should interest and benefit educators in other health sciences
disciplines as well. However, some limitations of the study exist. Consequently, the
findings warrant looking at what limitations might have impacted the results of the
study.
This study is limited in that it examines critical thinking achievement in one
school without establishing the pre-treatment performance prior to an educational
intervention. Furthermore, the internal validity of this study must cautiously be
evaluated. Internal validity is not only concerned with the generality but with the
integrity of the study itself. In this study, it is important to consider if there are any
other factors besides PBL curriculum used by the USC School of Dentistry that may
have caused the outcome of critical thinking skills measurement. Moreover, internal
validity is also concerned with selection bias that can result from pre-existing
conditions differences on any number variables. Selection bias can be avoided by
random assignment of a study’s participants. However, this was not an option in this
study.
Another limitation to this study was with the sample size. The study was
limited to approximately 30 students per class or group. The relatively small number
of samples could be a reason for the lack of statistical significance and could produce
spurious results. Moreover, the small number increases the possibility of individual
87
differences masking the effects of prolonged exposure to PBL. Consequently, the
small sample size could threaten the external validity of a study.

Recommendations for Further Study
Critical thinking is not a general cognitive ability but rather a complex set of
general and specific factors. There are at least seven standardized critical thinking
tests available, and several performance assessment approaches can be used as
outcome measures within various subjects. Standardized tests can provide useful
information for the purpose of diagnostic that may help to guide class room
instruction. However, multiple measures of critical thinking should be used in PBL
assessment. Educational psychologists generally prefer multiple measures of critical
thinking because no single test would cover the dimensions of a good conceptual
definition of critical thinking.
Ennis and Norris (1990) suggest that in lieu of appropriate multiple choice
tests, open-ended assessment tests are necessary, and other measures could include
interviews. Higher education educators should initially decide what students should
be able to demonstrate and what they know and can do. Then, they should decide
what to teach students. When educators are clear about the intended performance and
results, they will have a set of criteria for selection of content. Subsequently in
devising their means of assessment, educators should consider guidelines concerning
meaningful contexts in new knowledge, exams, relevant products and performances,
and the various levels of student ability (Spicer & Hanks, 1995).
88
Educators believe the ability to think critically to be one of the most
important educational outcomes. Although descriptions of critical thinking skills
vary and often overlap, the selection of particular assessment methodology should be
based as closely as possible on how thinking is defined. While a multiple choice
measure like the Health Science Reasoning Test may have a solid theoretical basis
and sound technical characteristics, it does not lend itself to the design of PBL
educational and instructional strategies. Norris and Ennis (1990) define critical
thinking as reasonable and reflective thinking that is focused upon deciding what to
do or believe. Stated this way, educators can expect students to use critical thinking
skills to reach a conclusion based on available information. According to Norris and
Ennis, critical thinking is demonstrated by the ability to: 1) analyze the accuracy of
information, 2) evaluate conclusions, and 3) make good inferences. Norris and
Ennis’ definition helps to clarify the assessment task. Instead of having the ability to
solve “puzzles”, critical thinking becomes the ability to make decisions or attain
conclusions about problems and issues that may not always have a single correct
answer.
Despite the effort to collect data from a diverse sample of dental program
students, this study was still unable to secure a sample that closely represents the
population of university students at USC. The sample for this study did not represent
a randomly selected one. Perhaps, it might over-represent certain subgroups and
under-represent others. This was particularly true in that it over-represented a certain
89
race group and gender category. Therefore, a future study that is based on a
randomized sample is strongly recommended.
Further study can also examine in more detail the variation in critical
thinking as a consequence of different background characteristics. While this study
discovered some significant differences due to gender, race, and English language
usage, other variables such as actual GPA, GRE, and DAT scores are desirable over
self-reported scores in further research. In order to accomplish this goal, further
research to enhance the database is needed, particularly in limiting the reliance on
self-reporting data by participants. The reliability and validity of the database is
certainly the utmost important element.
90
REFERENCES
About USCSD. (n.d.). University of Southern California School of Dentistry.
Retrieved February 20, 2007, from
http://www.usc.edu/hsc/dental/about/history.htm

Albanese, M. A., & Mitchell, S. (1993). Problem-based learning: A review of
literature on its outcomes and implementation issues. Academic Medicine,
68(1), 52-81.

Anderson, S., & Saucier, B. L. (1999). The critical thinking skills of faculty in
baccalaureate and masters in nursing programs. Journal of Thinking &
Problem Solving, 9(2), 51-62.

Association of American Colleges. (1985). Integrity in the college curriculum: A
report to the academic community. Washington, DC.

Barrows, H. S. (1986). A taxonomy of problem-based learning methods. Medical
Education, 20, 481-486.

Barrows, H. S. (1994). Practice-Based Learning: Problem-Based Learning Applied
to Medical Education. Springfield, IL: Southern Illinois University School of
Medicine.

Barrows, H. S., & Tamblyn, R. M. (1980). Problem-based learning: An approach to
medical education. New York: Springer.

Bartlett, D. J., & Cox, P. D. (2002). Measuring change in students’ critical thinking
ability: Implications for health care education. Journal of Allied Health,
31(2), 64-69.

Baxter Magolda, M. B. (2001). Complex lives. Making their own way. In M. E.
Wilson & L. E. Wolf-Wendel (Eds.), ASHE Reader on College Student
Development Theory, (pp. 81-100). Boston, MA: Pearson Custom Publishing.

Birgegard, G., & Lindquist, U. (1998). Change in student attitudes to medical school
after the introduction of problem-based learning in spite of low ratings.
Medical Education, 32, 46-49.

Blai, B., Jr. (1992). Assessment of critical thinking in postsecondary education.
EDRS.

91
Blumberg, P. (2000). Evaluating the evidence that problem-based learners are self-
directed learners: A review of the literature. In D. H. Evensen & C. E. Hmelo
(Eds.), Problem-based learning: A research perspective on learning
interactions, (pp. 199-226). Mahwah, NJ: Lawrence Erlbaum Associates.

Bondy, K. N., Koenigseder, L. A., Ishee, J. H., & Williams, B. G. (2001).
Psychometric properties of the California Critical Thinking Tests. Journal of
Nursing Measurement, 9(3), 309-328.

Boshuizen, H. P. A., Machiels-Bongaerts, M., Schmidt, H. G., & Hermans, H.
(1995). Monitoring the development of expertise in a problem-based
curriculum. Paper presented at the Annual Meeting of the American
Educational Research Association, San Francisco, CA.

Boud, D. (1987). Problem-based learning in perspective. In D. Boud (Ed.), Problem-
based learning in education for the professions (pp. 13-18). Sidney: Higher
Education Research and Development Society of Australia.

Bridges, E. M., & Hallinger, P. (1997). Using problem-based learning to prepare
educational leaders. Peabody Journal of Education, 72(2), 131-146.

Burch, K. (2000). A primer on problem-based learning for international relations
courses. International Studies Perspectives, 1, 31-44.

Cheung, C., Rudowicz, E., Kwan A. S. F., & Yue, X. D. (2002). Assessing
university students’ general and specific critical thinking. College Student
Journal, 504-525.

Christen, R., Angermeyer, J., Davison, M. L., & Anderson, K. (1994). Performance
assessments of critical thinking: The Reflective judgment approach.
University of Minnesota Research/Practice Newsletter, 2(1).

Colliver, J. A. (2000). Effectiveness of PBL curricula. Medical Education, 34(11),
959.

Critical Thinking Community. (n.d.). Retrieved February 20, 2007, from
http://www.criticalthinking.org/aboutCT/ourConceptCT.shtml

CriticalThinking.net. (n.d.). Retrieved February 20, 2007, from
http://www.criticalthinking.net/SSConcCTApr3.html

Distlehorst, L. H., Dawson, E., Robbs, R. S., & Barrows, H. S. (2005). Problem-
based learning outcomes: The glass half-full. Academic Medicine, 80(3), 294-
299.
92

Dolmans, D. H. J. M, Wolfhagen, I. H. A. P., van der Vleuten, C. P. M., & Wijnen,
W. H. F. W. (2001). Solving problems with group work in problem-based
learning: Hold on to the philosophy. Medical Education, 35, 884-889.

Edman, L. R. O., Robey, J., & Bart, W. M. (2002). Critical thinking, belief bias,
epistemological assumptions, and the Minnesota Test of Critical Thinking.
Paper presented at the Annual Meeting of the American Educational
Research Association, New Orleans, LA.

Ennis, R. H., & Norris, S. P. (1990). Critical thinking assessment: Status, issues,
needs. In C. Cheung, E. Rudowicz, A. S. F. Kwan, & X. D. Yue (2002).
Assessing university students’ general and specific critical thinking. College
Student Journal, 504-525.

Ennis, R. H., & Weir, E. (1985). The Ennis-Weir critical thinking essay test. Pacific
Grove, CA: Midwest.

Evans, N. J., Forney, D. S., & Guido-Dibrito, F. (1998). Student development in
college: Theory, research, and practice. New York: Wiley.

Farrow, R. (2003). The effectiveness of PBL: The debate continues. Is meta-analysis
helpful? Medical Education, 37, 1131-1132.

Facione, P. A. (1990). Executive summary: The Delphi report. Millbrae, CA:
California Academic Press.

Facione, P. A. (2007). Critical thinking: What it is and why it counts. Millbrae, CA:
California Academic Press.

Facione, P. A., & Blohm, S. (2001). The test of everyday reasoning: A measure of
thinking skills: Test manual - 2001 edition. Millbrae, CA: California
Academic Press.

Facione, N. C., & Facione, P. A. (1994). The California Critical Thinking Skills Test
and the National Leagues for Nursing accreditation requirement in critical
thinking. Millbrae, CA: California Academic Press.

Facione, N. C., & Facione, P. A. (2006). The health sciences reasoning test: Test
manual - 2006 edition. Millbrae, CA: California Academic Press.

Fincham, A. G., & Shuler, C. F. (2001). The changing face of dental education: The
impact of PBL. Journal of Dental Education, 65(5), 406-421.

93
Friedman, R. S., & Deek, F. P. (2002). Problem-based learning and problem-solving
tools: Synthesis and direction for distributed education environments. Journal
of Interactive Learning Research, 13(3), 239-257.

Gijbels, D., Dochy, F., van den Bossche, P., & Segers, M. (2005). Effects of
problem-based learning: A meta-analysis from the angle of assessment.
Review of Educational Research, 75(1), 27-61.

Gilkison, A. (2003). Techniques used by ‘expert’ and ‘non-expert’ tutors to facilitate
problem-based learning tutorial in an undergraduate medical curriculum.
Medical Education, 37, 6-14.

Goodman, L. J., Brueschke, E. E., Bone, R. C., Rose, W. H., Williams, E. J., & Paul,
H. A. (1991). An experiment in medical education. A critical analysis using
traditional criteria. Journal of the American Medical Association, 265(18),
2373-2376.

Halpern, D. (1999). New Directions for Teaching and Learning, 80, 69-74. Retrieved
February 20, 2007, from
http://www.scottsdalecc.edu/ricker/critical_thinking/index.htm

Hmelo, C. E. (1994). Development of independent learning and thinking: A study of
medical problem solving and problem-based learning. UMI.

Hmelo, C., Gotterer, G. S., & Bransford, J. D. (1994). The cognitive effects of
problem-based learning: A preliminary study. Paper presented at the Annual
Meeting of the American Educational Research Association, New Orleans,
LA.

Hoffman, K., Hosokawa, M., Blake, R., Jr., Headrick, L., & Johnson, G. (2006).
Problem-based learning outcomes: Ten years of experience at the University
of Missouri—Columbia School of Medicine. Academic Medicine, 81(7), 617-
625.

Hsu, Y. (1999). Evaluation theory in problem-based learning approach. Paper
presented at the National Convention of the Association for Educational
Communications and Technology, Houston, TX.

Iputo, J. E., & Kwizera, E. (2005). Problem-based learning improves the academic
performance of medical students in South Africa. Medical Education, 39,
388-393.

94
Jacobs, A. E. J. P., Dolmans, D. H. J. M., Wolfhagen, I. H. A. P., & Scherpbier, A. J.
J. A. (2003). Validation of a short questionnaire to assess the degree of
complexity and structuredness of PBL problems. Medical Education, 37,
1001-1007.

Kamin, C., O’Sullivan, P., Deterding, R., & Younger, M. (2003). A comparison of
critical thinking in groups of third-year medical students in text, video, and
virtual PBL case modalities. Academic Medicine, 78(2), 204-211.

Kenny, N. P., & Beagan, B. L. (2004). The patient as text: A challenge for problem-
based learning. Medical Education, 38, 1071-1079.

Khoo, H. E. (2003). Implementation of problem-based learning in Asian medical
schools and students’ perceptions of their experience. Medical Education, 37,
401-409.

King, P. M., & Kitchener, K. S. (2004). Reflective judgment: Theory and research on
the development of epistemic assumptions through adulthood. In M. E.
Wilson & L. E. Wolf-Wendel (Eds.), ASHE Reader on College Student
Development Theory (pp. 487-504). Boston, MA: Pearson Custom
Publishing.

Kinkade, S. (2005). A snapshot of the status of problem-based learning in U.S.
medical schools, 2003-04. Academic Medicine, 80(3), 300-301.

Kolb, D. A. (1981). Learning styles and disciplinary differences. In A. W.
Chickering & Associates, The modern American college: Responding to the
new realities of diverse students and a changing society (pp. 232-255). San
Francisco: Jossey-Bass.

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and
development. Englewood Cliffs, NJ: Prentice Hall.

Langendyk, V. (2006). Not knowing that they do not know: Self-assessment
accuracy of third-year medical students. Medical Education, 40, 173-179.

Levine, H. G., & Forman, P. B. (1973). A study of retention of knowledge of
neurosciences information. Journal of Medical Education, 48, 867-869.

Lieux, E. M. (1996). A comparative study of learning in lecture versus problem-
based format. About Teaching, 50, 25-27.

95
Liu, M., Hsieh, P., Cho, Y., & Schallert, D. L. (2006). Middle school students’ self-
efficacy, attitudes, and achievement in a computer-enhanced problem-based
learning environment. Journal of Interactive Learning Research, 17(3), 225-
242.

Lloyd-Jones, G., Margetson, D., & Bligh, J. G. (1998). PBL in action: A coat of
many colours. Medical Education, 32(5), 492.

Locke, L. F., Silverman, S. J., & Spirduso, W. W. (2004). Reading and
understanding research. Thousand Oaks, CA: Sage Publications.

Lohman, M. C., & Finkelstein, M. (2000). Designing groups in problem-based
learning to promote problem-solving skill and self-directedness, 28, 291-307.

Major, C. H., & Palmer, B. (2001). Assessing the effectiveness of problem-based
learning in higher education: Lessons from the literature. Academic Exchange
Quarterly, 5(1), 4-9.

Maudsley, G., & Strivens, J. (2000). Promoting professional knowledge, experiential
learning and critical thinking for medical students. Medical Education, 34,
535-544.

Moore, G. T., Block, S. D., Briggs-Style, C., & Mitchell, R. (1994). The influence of
the New Pathway curriculum on Harvard medical students. Academic
Medicine, 69(12), 983-989.

National Educational Goals Resources Group (NERG). (1991). Interim Report. U.S.
Department of Education, Office of Educational Statistics and Research.

Nendaz, M. R. , & Tekian, A. (1999). Assessment in problem-based learning medical
schools: A literature review. Teaching and Learning in Medicine, 11(4), 232-
243.

Neo, M., & Neo, T. (2005). A multimedia-enhanced problem-based learning
experience in the Malaysian classroom. Learning, Media and Technology,
30(1), 41-53.

Newman, M., Ambrose, K., Corner, T., Evans, J., Morris-Vincent, P., Quinn, S.,
Stephenson, J., & Vernon, L. (2003). Evaluating educational impact: The
approach followed by The Project on the Effectiveness of Problem Based
Learning (PEPBL). Paper presented at the Annual Meeting of the American
Educational Research Council, Chicago, IL.

96
Norman, G. R., & Schmidt, H.G. (1992). The psychological basis of problem-based
learning: A review of the evidence. Academic Medicine, 67, 557-565.

Onwuegbuzie, A. J. (2001). Critical thinking skills: A comparison of doctoral- and
master’s-level students. College Student Journal, 477-480.

Parikh, A., McReelis, K., & Hodges, B. (2001). Student feedback in problem based
learning: A survey of 103 final year students across five Ontario medical
schools. Medical Education, 35, 632-636.

Patel, V. L., Groen, G. J., & Norman, G. R. (1991). Effects of conventional and
problem-based medical curricula on problem solving. Academic Medicine,
66(7), 557-565.

Perry, W. G., Jr. (1970). Forms of intellectual and ethical development in the college
years: A scheme. New York: Holt, Rinehart & Winston.

Piaget, J. (1952). The origins of intelligence in children. New York: International
Universities Press.

Prince, K. J. A. H., van Mameren, H., Hylkema, N., Drukker, J., Scherpbier, A. J. J.
A., & van der Vleuten, G. P. M. (2003). Does problem-based learning lead to
deficiencies in basic science knowledge? An empirical case on anatomy.
Medical Education, 37, 15-21.

Profetto-McGrath, J., Hesketh, K. L., Lang, S., & Estabrooks, C. A. (2003). A study
of critical thinking and research utilization among nurses. Western Journal of
Nursing Research, 25(3), 322-337.

Rapps, J., Riegel, B., & Glaser, D. (2001). Testing a predictive model of what makes
a critical thinker. Western Journal of Nursing Research, 23(6), 610-626.

Roche, W. P. III, Scheetz, A. P., Dane, F. C., Parish, D. C., & O’Shea, J. T. (2003).
Medical students’ attitudes in a PBL curriculum: Trust, altruism, and
cynicism. Academic Medicine, 78(4), 398-402.

Savery, J. R., & Duffy, T. M. (n.d.). Problem-based learning: An instructional model
and its constructivist framework. In B. Wilson (Ed.), Constructivist learning
environments: Case studies in instructional design. Englewood Cliffs, NJ:
Educational Technology Publications. Retrieved November 9, 2006, from
http://www3.uakron.edu/edfound/people/savery/papers/sav-duff.html

Savin-Baden, M. (2000). Problem-based learning in higher education: Untold
stories. Philadelphia: SRHE & Open University Press.
97

Savin-Baden, M., & Major, C. H. (2004). Foundations of problem-based learning.
New York: McGraw-Hill.

Savin-Baden, M., & Wilkie, K. (2004). Challenging research in problem-based
learning. New York: McGraw-Hill.

Schmidt, H. G. (1983). Problem-based learning: Rationale and description. Medical
Education, 17, 11-16.

Schmidt, H. G., Dauphinee, W. D., & Patel, V. L. (1987). Comparing the effects of
problem-based and conventional curricula in an international sample. Journal
of Medical Education, 62, 305-315.

Schmidt, H.G., Norman, G. R., & Boshiuzen, H. P. A. (1990). A cognitive
perspective on medical expertise: Theory and implications. Academic
Medicine, 65, 611-621. In C. E. Hmelo (1994), Development of independent
learning and thinking: A study of medical problem solving and problem-
based learning. UMI.

Schmidt, H. G., Vermeulen, L., & van der Molen, H. T. (2006). Longterm-effects of
problem-based learning: A comparison of competencies acquired by
graduates of a problem-based and a conventional medical school. Medical
Education, 40, 562-567.

Scott, J. N., & Markert, R. J. (1994). Relationship between critical thinking skills and
success in preclinical courses. Academic Medicine, 69(11), 920-924.

Setterstein, L., & Lauver, D. R. (2004). Critical thinking, perceived health status, and
participation in health behaviors. Nursing Research, 53(1), 11-18.

Spicer, K., & Hanks, W. E. (1995). Multiple measures of critical thinking skills and
predisposition in assessment of critical thinking. Paper presented at the
Annual Meeting of the Speech Communication Association, San Antonio,
TX.

Stein, B. S., Haynes, A. F., & Unterstein, J. (2003). Assessing critical thinking skills.
Paper presented at SACS/COC Annual Meeting, Nashville, TN.

Strange, C. (1994). Student development: The evolution and status of an essential
idea. In M. E. Wilson & L. E. Wolf-Wendel (Eds.), ASHE Reader on College
Student Development Theory, (pp. 25-41). Boston, MA: Pearson Custom
Publishing.

98
Sungur, S., & Tekkaya, C. (2006). Effects of problem-based learning and traditional
instruction on self-regulated learning. The Journal of Educational Research,
99(5), 307-317.

Thomas, R. E. (1997). Problem-based learning: Measurable outcomes. Medical
Education, 31, 320-329.

Tiwari, A., Lai, P., So, M., & Yuen, K. (2006). A comparison of the effect of
problem-based learning and lecturing on the development of students’ critical
thinking. Medical Education, 40, 547-554.

Van den Hurk, M. (2006). The relation between self-regulated strategies and
individual study time, prepared participation and achievement in problem-
based curriculum. Active Learning in Higher Education, 7(2), 155-169.

Van Gelder, T., Bissett, M., & Cumming, G. (2004). Cultivating expertise in
informal reasoning. Canadian Journal of Experimental Psychology, 58(2),
142-152.

Vernon, D. T. A., & Blake, R. L. (1993). Does problem-based learning work? A
meta-analysis of evaluative research. Academic Medicine, 68(7), 550-563.

Williams, K., Wise, S. L., & West, R. F. (2001). Multifaceted measurement of
critical thinking skills in college students. Paper presented at the Annual
Meeting of the American Educational Research Association, Seattle, WA.

Winning, T., Skinner, V., Townsend, G., Drummond, B., & Kieser, J. (2004).
Developing problem-based learning packages internationally: An evaluation
of outcomes. Innovations in Education and Teaching International, 41(2),
125-144.

Yip, W., & Ghafarian, A. (2002). Problem-based learning assessment for
information systems courses. Paper presented at the International Academy
for Information Management Annual Conference, Brisbane, Australia.

Zimmerman, B. J., Bonner, S., & Kovach, R. (1996). Developing self-regulated
learners: Beyond achievement to self-efficacy. Washington, DC: American
Psychological Association.

Zimmerman, B. J., & Lebeau, R. B. (2000). A commentary on self-directed learning.
In D. H. Evensen & C. E. Hmelo (Eds.), Problem-based learning: A research
perspective on learning interactions, (pp. 299-313). Mahwah, NJ: Lawrence
Erlbaum Associates.

APPENDIX A

THE HEALTH SCIENCES REASONING TEST
99
100
101
102
103
104
105
106
107
APPENDIX B

CAPSCORE™ RESPONSE FORM
108
APPENDIX C

DEMOGRAPHICS/ACADEMIC QUESTIONNAIRE

University of Southern California
School of Dentistry
925 West 34
th
Street
Los Angeles, CA 90089-0641

PBL EFFECTIVENESS IN CRITICAL THINKING SKILLS STUDY

Study ID: __ __ __

Do you consider English to be your first language?
1. Yes
2. No

Age: __ __ years

Undergraduate GPA: __ . __ __

Graduate GPA: __ . __ __

Best DAT Score: __ __

109

Abstract (if available)

Abstract This study measured the change of critical thinking skills (CTS) of dental students in a problem-based learning (PBL) pedagogical method. The quantitative analysis was focused on measuring students' CTS achievement starting from their first- through third-year of dental education at the University of Southern California.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

The impact of diversity courses on students' critical thinking skills

PDF

Implementing problem-based learning to develop student supply chain skills

PDF

A comparsion of traditional face-to-face simulation versus virtual simulation in the development of critical thinking skills, satisfaction, and self-confidence in undergraduate nursing students

PDF

The key essentials for learning in the 21st century: programs and practices

PDF

Facilitation of authentic teaching and learning in a problem-based learning (PBL) environment

PDF

The process secondary administrators use to implement twenty‐first century learning skills in secondary schools

PDF

The effectiveness of a district's social skills curriculum for students with disabilities

PDF

Assessing the impact of diversity courses on student-faculty interactions, critical thinking and social engagement

PDF

The relationship between learning style and student satisfaction in the Problem Based Learning Dental Program at the University of Southern California

PDF

Globalization and the need for 21st-century skills: implications for policy education in science, technology, engineering, mathematics, and project-based learning in schools in Ireland

PDF

Mind, motivation, and meaningful learning: A cognitive science approach to learning how to learn

PDF

Measuring OVID MEDLINE information literacy and search skill retention in medical students

PDF

Problem-based learning and its influence on college preparation knowledge, motivation, & self-efficacy in high school students

PDF

Globalization in curricular elements and instructional practices in California schools: A high school case study

PDF

Improving student achievement at a restructured high school academy of health sciences using an innovation gap analysis approach

PDF

The effects of culturally responsive standards based instruction on African American student achievement

PDF

Integrating project-based learning and work-based learning into a coherent program: is the sum worth more than its parts?

PDF

Emerging practices for a changing world: a case study of 21st century learning

PDF

An evaluation of project based learning implementation in STEM

PDF

Influence of globalization, school leadership, and students’ participation in science competitions on 21st-century skill development, instructional practices, and female students’ interest in sci...

Asset Metadata

Creator Pardamean, Bens (author)

Core Title Problem-based learning in a dental school: measuring change in students' critical thinking skills

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 09/20/2013

Defense Date 10/17/2007

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag critical thinking skills,CTS,dental school,Health Sciences Reasoning Test,HSRT,OAI-PMH Harvest,PBL,problem-based learning

Place Name California (states), Los Angeles (counties), University of Southern California (geographic subject), USA (countries)

Language English

Advisor Keim, Robert G. (committee chair), Hocevar, Dennis J. (committee member), Slovacek, Simeon P. (committee member)

Creator Email pardamea@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m898

Unique identifier UC1144730

Identifier etd-Pardamean-20071031 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-559355 (legacy record id),usctheses-m898 (legacy record id)

Legacy Identifier etd-Pardamean-20071031.pdf

Dmrecord 559355

Document Type Dissertation

Rights Pardamean, Bens

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu