University of Southern California Dissertations and Theses

Teaching interpersonal and communication feedback skills to standardized patients: assessment of a cognitive model

(USC Thesis Other)

Teaching interpersonal and communication feedback skills to standardized patients: assessment of a cognitive model

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

TEACHING INTERPERSONAL AND COMMUNICATION

FEEDBACK SKILLS TO STANDARDIZED PATIENTS:

ASSESSMENT OF A COGNITIVE MODEL

by

Denise M. Souder

A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

May 2009

Copyright 2009 Denise M. Souder

ii
Dedication
This dissertation is dedicated to my husband, Lee Alan Souder, for his
unconditional love and endless support throughout this journey. You have kept me on
task with encouragement, enthusiasm, and belief in my abilities. Through the tears and
the celebrations, I could not have asked for a better partner and friend. We share this
achievement together.
This work is also dedicated to the memory of my father, Joseph H. Brown, and to
the honor of my mother, Florence E. Brown for their love and support as I pursued my
dreams.

iii
Acknowledgements
I would like to acknowledge my family for their support and belief in my abilities
throughout this doctoral program. Each and every one of you has made an important
contribution to my success. Thank you Mickee for letting me talk, you were always there
when I needed an ear. I am especially indebted to Mary and Bob for their help, support,
and understanding when I was stressed.
I am very grateful for the support, time, and contributions of my dissertation
chair, Dr. Maura Sullivan who has keep me on task with her vision. I am indebted to my
committee members, Dr. Win May, my mentor, whose guidance and expertise made this
achievement possible, and Dr. Rodney Goodyear, whose valuable work contributed to the
foundation for this study.
It is with sincere appreciation that I acknowledge my fellow KSOM faculty,
Dr. Janet Trial, and Dr. Madeleine Bruning. They willingly shared their knowledge and
expertise, and provided encouragement when I most needed it.
I pay special recognition to Dr. Sarah Peyre and Dr. Moreen Logan for their time
and expertise as my expert raters. No matter the task, they never said no. Thank you my
friends, for going above and beyond to support me in this effort.
A special acknowledgement is paid to Joy Cruz for her support as I pursued my
scholarly activities. Joy, I am grateful for your help in lightening my load and for your
friendship.

iv
Table of Contents
Dedication ii
Acknowledgements iii
List of Tables vi
Abstract vii
Chapter One: Conceptual and Empirical Foundations for the Study 1
Background and Statement of the Problem 2
Purpose of the Study 4
Review of the Literature 5
Feedback 6
Cognitive Learning Theory 12

Chapter Two: Method 17
Intervention 18
Measures 23
Standardized Patient Satisfaction Questionnaire 23
Faculty Workshop Feedback Form 24
Student Online Evaluation 24
Quality of Standardized Feedback Form 25
Population and Sample 25
Sampling Strategy 25
Validity and Confidence in Findings 26

Chapter Three: Results 27
Research Question One 27
Theme Event Examples 29
Research Question Two 30
Research Question Three 31
Research Question Four 32

Chapter Four: Discussion 34
Conclusions 34
Conclusion One 34
Conclusion Two 37
Conclusion Three 39
Conclusion Four 41
Limitations 44
Implications for Medical Educators 45

v
Bibliography 47

Appendix A 49
Appendix B 51
Appendix C 56
Appendix D 57

vi
List of Tables
Table 1: Training Protocols 17
Table 2: Standardized Patient Satisfaction Questionnaire 27
Table 3: Frequency of Comments Based Upon Themes 28
Table 4: Faculty Workshop Feedback Form 30
Table 5: Faculty Comments of SP Feedback by Frequency 31
Table 6: Student Web Evaluation of SP Workshop 31
Table 7: Independent Rating of Sample Videos Utilizing the QSF 33

vii
Abstract
The purpose of this mixed-methods retrospective study was to examine the
effectiveness of a training intervention for standardized patients to improve their verbal
feedback to medical students. Although feedback is acknowledged as important for
medical student development, actual interventions to improve effective feedback to
medical students are scarce in the literature.
In the 2006-2007 academic year, the Standardized Patient Program at the Keck
School initiated a revised training protocol for standardized patients playing patient cases
in the Year I and Year II Introduction to Clinical Medicine interviewing workshops.
Assessment instruments consisted of a Standardized Patient Training Satisfaction
Questionnaire, a Faculty Workshop Feedback Form, student online evaluations of the
workshops, and a Quality of Standardized Patient Feedback form. The Standardized
Patient Training Satisfaction Questionnaire measured the knowledge, skills and
confidence of the standardized patients in providing verbal feedback. The Faculty
Workshop Feedback Form assessed whether the standardized patients followed the seven
steps for providing verbal feedback as taught in the intervention. The student online
evaluations determined if the post-intervention standardized patient feedback was more
useful than the pre-intervention feedback. The Quality of Standardized Feedback form
was utilized by two independent raters to score the feedback given to the medical
students from video review of the workshops.
Descriptive statistics from the Standardized Patient Satisfaction Questionnaire,
the Faculty Workshop Feedback Form and student online evaluations indicated

viii
standardized patients provided effective verbal feedback post-intervention. However,
direct observation by two independent raters of videos of workshop feedback revealed
that standardized patients often omitted feedback steps.
The main conclusion reached was that although three of the four assessment
instruments indicated the standardized patients provided quality feedback, it cannot be
determined conclusively if the improved feedback was the effect of the feedback
intervention in the revised training protocol. Direct observations of the independent
raters must be considered as the standardized patients had less than satisfactory scores on
the Quality of Standardized Patient Feedback form, indicating a lower quality of
feedback. Further research with psychometrically sound instruments is needed to
delineate variables contributing to the knowledge and skills of SPs as they learn to
provide quality verbal feedback.

1
Chapter One: Conceptual And Empirical Foundations For The Study

The impact of feedback on the performance of medical students is recognized in
the literature as being critical to their improvement of clinical and communication skills
(Bienstock, Katz, Cox, Hueppchen et al. 2007; Howley & Martindale, 2004; Sachdeva,
1996; Westberg & Jason, 2001; Wood 2000). Medical educators endeavor to assist
medical students in becoming competent practitioners who are not only skillful, but who
also practice self-reflection and possess professional attitudes (Westberg & Jason, 2001).
According to Westberg and Jason (2001), reflection and feedback on patient care
experiences are necessary components in the learners’ clinical and professional
development. Although feedback is acknowledged as an important factor for medical
student development, actual interventions to improve effective feedback to medical
students are scarce in the literature.
In the training of health professions, it is undisputed that patients offer the best
learning experiences because they are in a position to know if they are satisfied with the
interaction with the physician, if they liked how they were examined, if they are willing
to follow treatment recommendations, and if they will return to that physician. When
health care experiences with providers are positive and patients are active participants in
their own care, patient outcomes are improved (Westberg & Jason, 2001).
Real patients however, rarely offer feedback to their healthcare providers for fear
that their treatment may be compromised. Thus, many medical schools now rely on
actors to portray real patients in order to standardize the curriculum and ensure that all
students have adequate exposure to essential cases and skills. The trained actors called

2
standardized patients (SPs) have the responsibility to accurately portray a patient
scenario, evaluate the students using an objective structured checklist, and provide
feedback when indicated. Standardized patients have been used as a teaching
methodology for many years (Jason et al., 1971; Kahn et al., 1979), and it has been
shown that SPs are able to simulate the patient scenario (Tamblyn, Klass, Schnabl et al.,
1990) and accurately evaluate the clinical and communication skills of medical students
from recall

(De Champlain, Margolis, King et al., 1997; Heine, Garman, Wallace et al.,
2003; MacRae, Vu, Graham et al. 1995; Vu, Barrows, Marcy et al.,1992; Williams,
2004). However, there is little to no attention given to the quality of feedback that the
SPs provide to students or the methodologies utilized for teaching the skills of feedback.
Since effective verbal feedback is considered vital in the process for enhanced learning in
adults and in helping students achieve their goals (Howley & Martindale 2004; Sachdeva,
1996; Wood, 2000), it is important that the SPs know how to give effective verbal
feedback to students.
Background and Statement of the Problem
In an effort to help the medical students develop more accurate perceptions about
their interpersonal skills, the Keck School of Medicine (KSOM) utilizes workshops in the
Introduction to Clinical Medicine (ICM) course throughout the first two years of
curriculum that require SPs to not only portray various patient cases, but also to provide
verbal feedback with regard to interpersonal and communication skills to the medical
students. ICM workshops have two sessions on the same day and consist of 14 groups of
medical students per session; thus 28 student groups are in each workshop. Each of the

3
28 groups for one workshop consists of six medical students, a standardized patient, and a
faculty facilitator assigned to provide guidance and individual feedback to the student
learners. The SP provides verbal feedback to the students as a group and does not
provide individual feedback. The student groups change from session one to session two,
however the faculty and the standardized patients remain the same for both groups of
students. These workshops are videotaped. The Standardized Patient Program at the
KSOM trains 14 actors to portray a single case and uses five cases in the first year
curriculum, and one case in the second year curriculum; thus 84 actors need
comprehensive training for case portrayal and feedback delivery within a six-month
period of each academic year.
The training methodology for each patient case portrayed in the first two years of
ICM curriculum workshops has historically included two three-hour sessions: the first
instructional session devoted to correctly portray the case, and the second three-hour
session focused on how to give feedback in the first hour, with the final two hours
devoted to practicing the case with mock students. This method of training was found to
be ineffective in that the observed verbal feedback given by the SPs to medical students
in the workshops was not what was instructed in the training sessions. The SPs often
provided vague feedback which did not provide medical students with the exact
behaviors on which they could improve.
In response to the inadequate SP feedback provided to medical students in the
ICM workshops, the Standardized Patient Program introduced an additional three-hour
curricular intervention in the 2005-2006 academic year into each series of workshop

4
training sessions. The first training session for each workshop remained devoted to
portrayal of the patient, in essence, the gestalt of the case; the new second training
session concentrated on teaching the SPs how to provide effective verbal feedback; and
the third training session required each actor to practice portraying the case and providing
verbal feedback to mock students.
The additional feedback training session was a three -hour workshop utilizing
power point slides, handouts, interactive quizzes, and analysis of sample feedback videos.
The analysis of feedback videos was accomplished using a Quality of SP Feedback Form
(QSF), a step-by-step outline of how to give effective feedback (Appendix A). The SPs
watched video examples of feedback and rated the quality by indicating which
components of the QSF were included in the feedback.
Purpose of the Study
The purpose of this study was to address the gap of feedback training
interventions in medical education literature by evaluating the effectiveness of an
additional three-hour curricular intervention designed to improve the quality of feedback
given by SPs. A mixed methods approach was used to evaluate the quality of the verbal
feedback provided by the SPs in the ICM Year I and Year II workshops for medical
students. Specifically, this two-fold study was designed to: 1) determine whether the
addition of a third training session, which focused solely on feedback principles,
improved the quality of feedback provided by the SPs to the medical students, and 2)
determine to what extent the standardized patients utilized the QSF format in their verbal
feedback to medical students.

5
This study added to the medical education literature by assessing the effectiveness
of an educational intervention for improving the quality of feedback given by SPs to
medical students. In addition, it provided useful information to medical educators to
better inform their efforts and optimize the feedback skills of standardized patients. The
research questions addressed were:
1. Did the standardized patients find the revised training protocol useful in
providing knowledge, skills and confidence in giving verbal feedback?
2. Did workshop faculty perceive standardized patients as providing effective
feedback?
3. Did more students perceive standardized patients as providing effective
feedback after the revised training protocol?
4. Did the standardized patients utilize the QSF from the revised training
protocol when providing verbal feedback?
Review of the Literature
The literature review for this study is organized into two sections. Section one
describes the functions of feedback with inclusion of its background, definitions, features
and principles. Section two addresses theoretical frameworks relevant to learning how to
provide effective feedback with emphasis given to cognitive learning theory and social
cognitive theory. An extensive search was performed in the databases of PsychINFO,
PubMed, and Ovid MEDLINE using the search words of feedback, feedback
interventions, medical students, undergraduate medical education, standardized patients,
and simulated patients.

6
Feedback
While the term feedback has been in existence for many years, the concept was
formalized in the early part of the 1900’s in the field of electrical engineering and process
controls whereby a sensor in a system determined if output varied from a targeted value
(Claiborn, Goodyear & Horner, 2001; Veloski, Boex, Grasberger, Evans & Wolfson,
2006). A controller in the system provided a feedback signal allowing the system to
adjust itself to the target output value. This form of feedback mechanism allowed the
electrical system to maintain homeostasis for optimum operating conditions.
From its origins in electrical engineering, feedback as human communication was
transferred into the social sciences by Kurt Lewin in the 1920’s (Claiborn et al., 2001).
While feedback may seem intuitive on the surface, Lewin’s formula in the 1920’s states
that B=f (P,E) where behavior (B) is a function of the interaction between the person (P)
and the environment (E) (Ash, 1992). Thus, a person’s behavior can be adjusted by
changes in the environment or by conscious choice on the part of the person. Lewin
further expanded his concept of feedback in his studies of human relations to include not
only the interaction between the environment and the person, but to also include the
impact of behaviors between people (Claiborn et al., 2001). Involvement of interactions
between people facilitated the transfer of feedback into various social sciences disciplines
including education, psychology, and medical education.
Definitions of Feedback. Feedback has been defined differently across disciplines
and although the intent is the same, no consensus exists for defining feedback. As
mentioned above, the earliest definition of feedback from electrical engineering was

7
information provided to maintain homeostasis within a closed system. However, human
interactions are influenced by a myriad of factors and thus are considered open systems
rather than closed. In the field of psychotherapy, Claiborn et al., (2001) define feedback
as information or a response to another person’s behavior or its effects. Therefore,
feedback has the ability to serve two functions, as a response to another person’s
behavior with information and as a potential power to influence the continuance or
discontinuance of the exhibited behaviors (Claiborn & Goodyear, 2005).
In medical education literature, Sender-Liberman and colleagues (2005) define
feedback as specific information given to the learner in order to promote reflection on
performance, a definition which introduces the construct of self-reflection in the feedback
process. Bienstock, Katz, Cox et al. (2007) further define feedback in medical education
as the constructive and objective appraisal of performance given in order to improve the
learner’s skills.
Although little to no research has been conducted on standardized patient
feedback to medical students, one study emerged which does address this issue. Howley
and Martindale (2004) define feedback from SPs as an informed, non-evaluative,
objective, oral appraisal of performance. Their feedback definition is based on a study of
standardized patient feedback to students investigating student satisfaction with the
feedback and the quality of the SP feedback. Their findings indicated that the students
were satisfied with SP feedback and that the quality of feedback improved with training.
Although one objective of the study was to determine if the SPs could provide

8
constructive feedback to medical students after a short training intervention, the article
did not include the SP training methodology.
Medical students, residents, and faculty often vary in their views of what
constitutes feedback (Sender-Liberman, Liberman, Steinert et al., 2005). To address this
variation and seemingly lack of feedback in medical education, van de Ridder, Stoking,
McGaghie et al., (2008) reviewed literature from the social sciences and medical
education for a consensus of feedback definition with the goal of finding conceptual and
operational features. The authors report three underlying concepts in the literature: (1)
feedback as information; (2) feedback as a response which includes information; and (3)
feedback as a cycle which includes both information and a reaction (van de Ridder et al.,
2008). Feedback considered as a cycle includes self-reflection on the part of the learner
(Sender-Liberman et al., 2005), as well as information from the instructor, both necessary
for improving the skills and performance of the learner (Westberg & Jason, 2001).
Feedback is therefore a potential for change on the part of the learner. As a
response to the interaction between two individuals, one the source of information and
the other the receiver, variables of feedback during the process influence its effectiveness
(Claiborn & Goodyear, 2005; Wood, 2000). These features in turn, form principles for
providing effective feedback and serve to guide the process in medical education.
Features of Effective Feedback. Generalizing from psychology and
psychotherapeutic literature to feedback in medical education, variables contributing to
the effectiveness of deliberate oral feedback include a structured delivery; descriptive of
behavior based upon unbiased observation of the sender; evaluative in nature against a

9
known standard; the use of positive comments to reinforce good skills and behaviors;
constructive rather than negative comments to correct deficiencies; an emotional response
of sender tied to the behavior of the receiver; and acknowledgement of any receiver
resistance (Claiborn & Goodyear, 2005; Wood, 2000). Care must be taken on the part of
the sender to avoid overloading the receiver with too much negative information in which
case emotions can interfere with attention in information processing thereby decreasing
learning by the receiver. Negative emotions can also cause the receiver of feedback to
attribute poor performance to external causes, a situation addressed by the feedback
sender in the study intervention.
Features of effective feedback to help learners learn better and faster are the
foundations for principles of providing feedback, and are the bases for the SP feedback
intervention in this study.
Principles of Effective Feedback. Effective feedback is structured and starts
immediately after the encounter, with self-reflection by the feedback receiver on the
interpersonal processes of the encounter (Claiborn et al., 2001; Wood, 2000). This
allows the students or recipients to identify strengths and weaknesses from their
perspective prior to receiving information from the sender. The feedback must be
unbiased and based upon observable and modifiable behaviors (Claiborn et al., 2001;
Wood, 2000). These observable behaviors are linked to emotions experienced by the
sender during the encounter process and are reflected back as information and a response
to the behaviors.

10
The information and responses contained in the feedback message are presented
in a “sandwich” format and are most effective when they are delivered with a positive
comment first. According to Claiborn et al., (2001) positive comments serve to affirm
self-perceptions and help to shape the receiver’s self-concept. Thus, positive feedback
confirms the self-image of the receiver as a person of worth possessing high or adequate
skills and behaviors. Negative or constructive feedback is provided next and serves to
correct attitudes and behaviors of the feedback receiver (Claiborn et al., 2001; Wood,
2000). Since constructive feedback does not reinforce the receiver’s self-image,
resistance to these comments may occur and the sender needs to address this cognitive
dissonance by acknowledging it and providing support that the intent of constructive
comments is to provide insight on the effects of behaviors from another perspective.
Wood (2000) also states that the constructive feedback must be within the ability of the
learner or the zone of proximal development (Vygtosky, 1978) to change so as to avoid
frustration and a breakdown in the dialogue and communication between the feedback
sender and receiver. Constructive comments need to be limited so as not to induce
cognitive overload in the receiver. Positive feedback would end the “sandwich”
technique by allowing the receiver to once again experience a positive self-concept.
Wood (2000) believes that it is important in the feedback process that the receiver
verify the information from the sender. Allowing the receiver to restate the feedback in
their own words verifies that they heard it and more importantly, understood it (Wood,
2000). Thus, the feedback process is a dialogue of information and responses from the

11
sender, with understanding and clarification by the receiver. Such a process involves
skill on the part of the sender to provide useful feedback.
The sources of feedback in this study were standardized patients who are adult
learners and need education in how to provide effective feedback to medical students. As
mentioned above, section two of the literature review examines current learning theories
as applied to the overall feedback intervention to assist the standardized patients learning
how to provide effective feedback. The two learning theories explored in relation to
adult learning are that of cognitive learning using constructivism, and social cognitive
learning.

12
Cognitive Learning Theory
As a learning theory, constructivists believe that individuals construct meaning
and knowledge through their experiences. Previous experiences provide knowledge and
serve to guide the learner when presented with new challenges and experiences.
Constructivism is a complex construct involving thoughts and experiences of individuals
necessary to acquire knowledge and skills. The incorporation of conceptual and
experiential knowledge into interactions with others is part of the complexity of cognitive
and socio-cognitive learning. Key principles to constructivist theory are that knowledge
is socially constructed, learning is an active process, knowledge is constructed from past
experiences, and that learning is self-regulated (Ormrod, 2006). Two modern
constructivists contributing to cognitive learning theory are Jean Piaget and Lev
Vygotsky whose theories are presented to support the adult learning needs of the
standardized patients in acquiring effective feedback skills.
Piaget’s Cognitive Learning Theory. Jean Piaget, a developmental psychologist,
predicted a sequential process of cognitive development throughout the human lifespan
(Ormrod, 2006). The stages of cognitive development start in infancy as the individual
attempts to make meaning of the world through the understanding of concepts in relation
to their experiences. Knowledge in the form of schemata are formed and become the
basis for learning as individuals apply previously learned knowledge to new situations.
The cognitive processes continue throughout an individual’s life into adulthood as they
build upon existing knowledge to adapt to novel experiences.

13
Learning is a dynamic process triggered when an individual experiences a
discrepancy or disequilibrium when presented with a new situation that does not match
existing knowledge. As with all living beings, humans seek homeostasis and equilibrium
in their environment and when disequilibrium exists, they will use either assimilation of
previous knowledge to the new experience, or accommodation to manipulate the concept
or experience in order to achieve meaning. Thus, the trigger for learning is that of
disequilibrium whereby the learner reorganizes knowledge and forms new schemata.
Formal operational thought in cognitive learning theory is the ability for abstract thinking
in generating new knowledge. This higher level of cognitive processing allows the
individual to solve complex problems using previously learned constructs when presented
with situations containing new or changing information.
The cognitive processing learning theory is applicable to this study in that SPs are
adult learners who have previous experience in acting, and in giving and receiving
feedback in their acting careers. Although the process of providing feedback is different
in this study than what the SPs experience from artistic directors in stage, theatre and
movies, the basic concepts of feedback exist in long term memory and can be recalled
into working memory when the SPs encounter new information related to feedback
principles and process. The SPs are able to construct new knowledge, forming more
elaborate schemata as they apply and encode novel information to previously learned
constructs.

14
As a stage theory of cognitive development, Piaget does not take into account the
influence of the environment on the individual as part of the learning process, or the
influences of behaviors as they contribute to learning.
Vygotsky’s Cognitive Learning Theory. Unlike Piaget who focused solely on the
individual in the learning process, Vygotsky posits that learning is also dependent upon
the individual’s socio-historical and cultural experiences. These variables interact and
play a role in the learning process.
One of Vygotsky’s key principles for learning is that challenging tasks promote
maximum cognitive growth (Ormrod, 2006). The “zone of proximal development” is the
area for learners where tasks cannot be performed on their own but can performed with
the guidance of experienced others (Ormrod, 2006). One of the methods to assist
learning in this zone of proximal development is that of scaffolding whereby more
competent individuals provide guidance and structure that enables the learner to perform
more challenging tasks. As the learner gains success with the novel task, guidance is
gradually removed allowing the student to perform the task independently. Vygotsky
believed that opportunities need to exist to motivate individuals beyond their current level
of knowledge and skills in order for learning to occur.
Cognitive learning theories focus on an individual’s ability to construct meaning
from their experiences. However, knowledge is also constructed through interactions
with others where individuals have opportunities to use their constructed knowledge and
learn the circumstances in which to apply the knowledge. The motivation for learning in

15
social interactions can be through the observation of others and the modeling of
behaviors in order to reach individual goals.
Social Cognitive Theory. Albert Bandura’s (1986) social cognitive theory asserts
that there is an interaction between the learner, the environment and the learner’s
behaviors known as triadic reciprocality (Bandura, 1989; Schunk, Pintrich & Meece,
2008). Social cognitive theory states that people learn skills and behaviors by observing
others and will exhibit these later when they are motivated to do so (Schunk et al., 2008).
Ultimately, learning is demonstrated by changes in cognition, behaviors, and affective
states that are acquired through observation of others, particularly from models who are
deemed competent, credible, similar and enthusiastic (Schunk et al., 2008).
Modeling serves the functions of inhibition/disinhibition, response facilitation,
and observational learning with motivation on the part of the learner being dependent
upon observed consequences of modeled behaviors (Schunk et al., 2008). The
motivation to behave in a certain manner is related to the value in activities that have an
expectation of positive outcomes (Schunk et al., 2008). Accordingly, a learner can be
motivated to exhibit attitudes and behaviors that are deemed good or poor, but behaviors
are ultimately based upon the consequences from personal experience or the observed
consequences of others.
A person’s behavior can determine the outcome of a situation and in turn, beliefs
about the outcome can be related to the amount of the person’s self-efficacy (Schunk et
al., 2008). Self-efficacy plays a major role in the motivation of learners as when they
believe they are capable of acquiring knowledge and skills, and when they believe that

16
they are able to behave in a certain manner, they are more motivated to their goals
(Bandura, 1989; Schunk et al., 2008). Thus, role modeling can play an important part in
raising the self-efficacy of learners and affect motivation as they engage in modeled tasks
and behaviors with positive outcomes.
Bandura’s (1986) social cognitive theory is applicable to this study in that the
feedback workshop utilizes videos demonstrating examples of good and poor feedback
delivered by SPs to medical students. The SPs in the workshop videos role model the
process of providing good feedback to medical students. These examples demonstrate
that providing effective feedback to medical students is possible and serve to motivate the
SPs towards the goal of acquiring the skills necessary to do so. The video examples are
used to promote the principles and processes of providing effective feedback through the
inhibition and disinhibition of outcomes based upon SPs behaviors of feedback
deliverance in the videos.
As discussed in this literature review, effective feedback is considered vital in the
process for enhanced learning in adults and in helping students achieve their goals
(Howley and Martindale, 2004; Sachdeva, 1996). The learners in this study were
standardized patients who provided the feedback, therefore it is important that they were
aware of how to give effective verbal feedback to students. The learning theories
discussed provide the basis for the study intervention.

17
Chapter Two: Method
In order to improve the quality and accuracy of SP verbal feedback, the SP
program at KSOM has developed a set of materials consisting of a Feedback Manual,
How to Give Effective Feedback, by Win May, M.D., Ph.D. and Dixie Fisher, Ph.D.
(2004). The manual consists of a CD, a DVD, and a Quality of SP Feedback Form (QSF)
(Appendix A) which are described below. Using these materials, the SP Program
implemented an additional training session to teach SPs how to give effective feedback.
During this training session the SPs learn the importance of feedback, the principles of
giving feedback, and have the opportunity to critically examine examples of feedback
using the QSF rating form. The pre- and post-intervention training protocols are shown
in Table 1.
Table 1
Training Protocols
Historical Training Protocol
Prior to 2005-2006
Academic Year
Revised Training Protocol
2005-2006 and 2006-2007
Academic Years
First session, 3 hours:
Case read-through in group

Second session, 3 hours:
- 1 hour of didactic feedback
instruction
- 2 hours of case and feedback
practice in small groups
First session, 3 hours:
Case read-through in group

Second session, 3 hours:
- 3 hours of interactive feedback
workshop utilizing handouts, pre-quiz,
audience-response quizzes, and
practice with the 7-step feedback
process by using the QSF form with
video examples of feedback

Third session, 3 hours:
- 3 hours of patient case portrayal and
feedback practice in small groups
using the QSF form to rate verbal
feedback

18
This additional feedback training session incorporated goals for the standardized
patients, structure to the process of feedback, instructional aids, and modeling from
examples in an effort to address the learning needs of the standardized patients.
Intervention
The feedback training intervention was dedicated to teaching the SPs how to
provide quality verbal feedback and occurred in the second session of the revised training
protocol. A feedback manual, developed in 2004 by Win May, M.D., Ph.D. and Dixie
Fisher, Ph.D., was utilized and consisted of a CD, a DVD, and QSF form. Particular
attention was paid to the QSF form in this study, as it provided the foundation for the SPs
to provide quality and effective feedback to the medical students. Divided into seven
major sections with each section containing from one to four components, each section is
discussed due to the fact that the mean of each section is dependent upon its respective
number of components.
Section one of the QSF form is composed of four components and is designed to
promote self-reflection by the students as prompted by the SP. This section is vital as
effective feedback starts with learner self-reflection (Sender-Liberman et al., 2005;
Westberg & Jason, 2001). As well, sufficient time for students to respond is included as
an item, as the wait time for learner response is typically not sufficient for the feedback
recipient. The total possible score for this section is four with zero as not done and four
completely done.
The second major section of the QSF form asks the SPs to provide positive
feedback first. Not only do the SPs need to provide positive feedback first, they must

19
also give the feedback from the patient’s perspective and the feedback must be limited to
specific changeable behaviors. Effective feedback needs to be structured (Claiborn et al.,
2001; Wood, 2000) with positive feedback given first in order to confirm the student’s
self-concept as a person possessing adequate skills (Claiborn et al., 2001). The total
score of this section is three, with zero indicating not done and three indicating
completely done.
Section three of the QSF form determines if the SPs gave constructive feedback to
the workshop students. According to Claiborn and Goodyear (2005) constructive or
negative feedback serves to close the gap between a recipient’s current behavior and a
standard of behavior (p. 211). The term “constructive” in the QSF form though, is
deliberate in order to remove the emotional aspect of the term “negative” (Claiborn &
Goodyear, 2005, p. 211). It is important to determine if the SPs provided students in the
ICM workshops with constructive feedback to help them improve their skills. Another
component of this section is that SP did not overload the feedback receivers with too
much constructive feedback. An overload of feedback interferes with information
processing, resulting in decreased learning. Three points are possible in this section with
zero equaling not done and three indicating completely done.
Section four of the QSF form asked the SPs to recognize any distress that their
constructive verbal feedback may be causing to the medical students. In contrast to
positive feedback, constructive feedback does not reinforce a learner’s self-concept, but
exposes deficiencies in the performance. High emotions can then occur in the learner
which interferes with the processing of information (Ormrod, 2006/2008). Therefore, the

20
SPs must monitor and acknowledge the students’ reactions to verbal feedback to help
reduce anxiety or distress and aid in learning. This section contains a possible total score
of three, with zero equaling not done and three indicating completely done.
The fifth major section of the QSF form determined if the SPs finished their
verbal feedback on a positive note. This section contained one component with a
possible score of either two (completely done) or zero (not done). The closing of the
verbal feedback on a positive note completes the recommended format of providing
feedback according to the “sandwich” technique.
Section six of the QSF form determines if the SPs verified the students’ learning
by summarizing the feedback given, and by the students’ understanding of what
behaviors could be improved in future patient encounters. The total score possible is three
with zero equaling not done and three indicating completely done.
Section seven of the QSF form rates whether the SPs asked the students if they
had questions regarding the feedback provided, and if the SPs thanked the students. Two
points are possible for this section. The total score possible for section eight of the QSF
form is 20 points.
These QSF feedback components were integrated in the additional three-hour
group feedback training session for the SPs to learn the seven-step process of giving
verbal feedback. The feedback training session was organized as follows:
SPs first completed a pre-workshop paper 10 item multiple choice quiz to
assess baseline knowledge of feedback and feedback principles. Although
the quiz was part of the intervention, its items were self-graded by the SPs

21
as they progressed through the training session, were not evaluated by the
session instructor, and thus are not included in the results.
The CD consisting of slides provided a training session overview and
addressed goals of the session, challenges of the SPs in providing
feedback, principles of feedback, and a few multiple choice quizzes
assessing the SPs’ understanding of feedback principles. The multiple
choice quizzes utilized an audience response system in an effort to add
interactive engagement to the training and were not intended for individual
evaluation, but rather as points of group discussion. Thus, the training
session was dynamic and interactive as the SPs were encouraged to
discuss their current limitations in giving feedback and as a group, work
towards solutions to providing effective feedback based upon their past
experiences and the principles of feedback presented.
Next in the feedback session, the Quality of Standardized Patient
Feedback form (QSF) (Appendix A) was introduced with an explanation
on how to utilize it to assess examples of good and poor feedback from the
video clips on DVD. The QSF is a step-by-step organized instructional
aid to guide the SPs in their delivery of oral feedback to the medical
students. It was used as an internal training tool to assess the quality of
verbal feedback provided by the SPs, and earlier work has shown that the
reliability of the QSF form ranges from 0.75 to 0.83 (W. May & D. Fisher,
personal communication, October 25, 2006).

22
The SPs then watched examples of good and poor feedback from the
DVD. Next, they viewed feedback examples on the DVD for analysis of
the verbal feedback and rated the feedback using the QSF form to
determine which elements of verbal feedback are omitted in the examples,
if any. The QSF scores from each video example vary depending on
which parts of the feedback are omitted or are erroneous in that particular
video example. The scores of each example were posted on a whiteboard
to demonstrate how interpretation of what constitutes feedback can affect
the QSF score. The instructor worked through the QSF form after each
video example with the group, discussing each feedback feature for its
presence or absence until group consensus is reached. These QSF forms
were for internal training purposes only, were not evaluated by the training
instructor, and thus are not included in the results of this study. The
feedback intervention concluded with discussion of the goals for the third
and final training session.
The third and final training session involved having the SPs to break into three
small groups so that each SP was able to practice portraying the patient and practice
providing feedback to fellow SPs who played mock students. The SPs rotated roles
throughout the third training session so that each SP played the patient at least once and
provided oral feedback to the mock students at least once. The QSF form was utilized in
the third session to rate each SP’s adherence to providing feedback according to the
feedback principles of the training session, however the QSF forms were not graded. At

23
the end of the third training session which incorporated the knowledge of the first training
session for case portrayal, the second session for knowledge and skills of providing
effective feedback, and the third session utilizing case portrayal and feedback, the SPs
were asked to fill out an anonymous Standardized Patient Satisfaction Questionnaire
(SPSQ) (Appendix B). The questionnaire was used to determine if the SPs felt that the
three-session training protocol prepared them to portray the case and to provide effective
verbal feedback.
Measures
The effectiveness of the feedback workshop was evaluated according to the
following measures:
1. The Standardized Patient Satisfaction Questionnaire (SPSQ) (Appendix B) at the end
of the last training session measured the SPs’ confidence of their ability to provide
effective feedback based upon the addition feedback interventional session, as well as
their satisfaction with the revised training protocol. This questionnaire consists of 11
items rated on a 1 to 5 Likert scale with 1= strongly disagree and 5= strongly agree to
provide quantitative data; two open-ended questions to provide qualitative data; and an
open comment section for additional qualitative data. The questionnaire was developed
by the PI in 2005 to seek improvements in the program to address learners’ needs. Data
from the Standardized Patient Questionnaire were analyzed from 11 post-intervention
workshop learning sessions for the 2006-2007 and the 2007-2008 academic years. As of
this date, the SPSQ has not been investigated for reliability or validity.

24
2. Faculty evaluations from the Faculty Workshop Feedback Form (FWFF) (Appendix
C) reflected whether the SPs adhere to the seven-step guidelines for providing feedback
to the medical students at the end of the workshops. This evaluation form was developed
by Win May, M.D., Ph.D. for faculty assessment of whether the SP feedback followed
the steps of the QSF form. This FWFF instrument consists of 8 dichotomous responses
for quantitative data and offers space for open-ended comments for qualitative data.
Patton (2002) states that the advantage of using quantitative instruments that limit
responses to predetermined categories is that these allow measurement of “the reactions
of many respondents to a limited set of questions, thus facilitating comparison and
statistical aggregation of the data” (p. 227). Data from the FWFF have been analyzed
from the post-intervention 2006-2007 and 2007-2008 academic years. These data were
collected as part of the routine ICM and SP program evaluation process, with any faculty
and SP identifiers were deleted by the SP Program Administrator, Joy Cruz, so that the PI
was blinded for data analysis. To date, neither reliability nor validity of this form has
been established.
3. Student Online Evaluations (Appendix D) of the Year I and Year II ICM workshops.
This evaluation instrument consisted of four items utilizing a 5-point Likert scale with 1=
Strongly Disagree to 5=Strongly Agree for quantitative data. This evaluation was
developed by the Introduction to Clinical Medicine office in conjunction with Dr. Win
May and the PI, and differs from the workshop faculty evaluation instrument. Student
online evaluations of the Year I and Year II SP workshops have been collected for the
2004-2005 and 2005-2006 academic years which is pre-intervention; and for the 2006-

25
2007 and 2007-2008 academic years for post-intervention data. Student evaluation data
were analyzed for differences between pre-feedback workshop evaluation and post-
feedback workshop evaluations.
4. Direct observation of SPs providing feedback to students on video review by two
outside reviewers. These voluntary faculty reviewers were identified as Moreen Logan,
R.N., Ed.D., and Sarah Peyre, Ed.D., who were trained by the PI to observe SP feedback
from workshop videos and rate the feedback against the QSF form (Appendix A). Both
independent raters were oriented to the QSF form by the PI for inter-rater reliability, after
which they reviewed random videos of SP verbal feedback (n = 25) which were
randomized from the 240 available videos using the software program GraphPad. This
quantitative data assessed the SPs’ utilization of the QSF form in their verbal feedback in
the ICM workshops after participating in the training intervention.
Population and Sample
The units of analysis or participants in this study included the total number of
students in Year I (n = 174) and Year II (n = 160) classes, the faculty for the Year I
(n = 33) and Year II (n = 31) ICM workshops, and the individual standardized patients
(n = 84). The time periods evaluated for pre-intervention data were the 2004-2005 and
2005-2006 academic years, and the 2006-2007 and 2007-2008 academic years for post-
intervention data.
Sampling Strategy
This study utilized total population sampling of the SPSQ and total population
sampling of the medical students’ and faculties’ assessments of the feedback provided for

26
each workshop. Simple random sampling of the videotaped workshop encounters was
used to allow generalization to the total SP population who participated in the additional
training intervention.
Validity and Confidence in Findings
Multiple-unit sampling and the use of mixed methods validated findings through
triangulation. Quantitative data were derived from student web-based assessments of the
workshop, faculty assessments, the SPs’ ratings on the questionnaire, and the two
independent reviewers with direct observation comparing video recorded feedback to the
QSF form used in training. Qualitative data were obtained from the SPs’ questionnaire
and faculty comments from the ICM workshop.

27
Chapter Three: Results

This chapter presents the study’s results. Each section begins with the research
question that was used to guide the study and then presents the obtained results.
Research Question One
Did the standardized patients find the revised training protocol useful in providing
knowledge, skills and confidence in giving verbal feedback?
Table 2
Standardized Patient Satisfaction Questionnaire
Mean SD
1. The expectations of the program and SP responsibilities were clearly
explained.
4.76 0.45
2. The role of SPs in the curriculum was clearly explained. 4.67 0.50
3. The objectives for the SPs were achievable. 4.62 0.62
4. Trainings were well organized. 4.58 0.60
5. The content was appropriate and the quizzes helped me learn my
material better.
4.50 0.76
6. Trainings were presented at a level I could understand. 4.83 0.40
7. The training activities included active involvement of the
standardized patients.
4.75 0.47
8. Feedback from the trainer was helpful for improving performance. 4.72 0.47
9. The Feedback Training Handout/Manual was helpful. 4.41 1.02
10. The comprehensive training activities helped to develop the
knowledge, skills, and confidence needed to meet the objectives in
portraying the case and in providing effective written/verbal feedback
to students.
4.50 0.55
11. I was treated as a valuable member of the team and felt that my
efforts were respected.
4.81 0.43
*5-point Likert scale; 5 = strongly agree, 1= strongly disagree

28
Each question from the instrument was analyzed using descriptive statistics for
the mean of each question (n = 129). The results indicate that overall, the SPs were
satisfied with the comprehensive training sessions based on the revised training protocol.
The means of each question were high, providing preliminary findings that the revised
training protocol which is comprehensive for teaching how to play the case, how to
provide effective verbal feedback, and practice time to play the case and provide
feedback, contributed to the knowledge, skills, and confidence of the SPs for the Year I
and Year II ICM workshops.
In addition, two major themes emerged from an open-ended question (No. 12),
which training activity did you find most useful and effective? why?, as to which training
activity the SPs found most useful. Table 3 reports the frequencies of the training
activities that the SPs felt were most useful in helping them prepare for providing
effective verbal feedback to the medical students.
Table 3
Number of Comments Based Upon Themes
Small Group Role-play with
feedback
Feedback Training Session
Number of
Comments

88

26

The first major theme was that of SPs liking small groups where they role-play the case
and practice providing verbal feedback, (n = 88). The high frequency of this theme
supported the quantitative findings whereby the SPs found the comprehensive training
activities, including the additional feedback training session, to contribute to their
knowledge, skills, and confidence in playing the case and in providing effective verbal
feedback.

29
The second major theme, the feedback training session utilizing the CD, DVD and
QSF as being helpful in learning how to give feedback, had the next highest frequency,
(n = 26). It is worth noting that each SP who mentioned the second theme also
commented on the first major theme.
Theme Examples
The following examples represent one of the two themes that emerged from the
standardized patients’ responses to the open-ended question (#12).
Small Group Role Play with Feedback Practice. Breaking into small groups and
watching/commenting, asking questions as we each went through it; because
different people will bring different things to the table and you learn from all of it.
I think that the training we receive for this work is very thorough, clear and total.
We are encouraged to ask questions, we are given answers when we do; it is very
well explained to us that there are difficulties because of the subjective aspects of
being an SP and I feel well provided for.

It is so helpful to have the “pretend session.” To watch what everyone else
chooses [to say] and just hear different possible interactions.

Using the QSF to evaluate other people’s encounters. It provided a cohesive
structure for us to use in our own feedback.

Feedback Training Session. The feedback training session was great for learning
the framework for responses.

Feedback training session is most helpful. Performing the case and memorizing
the script is the easy part. Giving concise and constructive feedback is the hardest
part. More practice helps us give better feedback.

The feedback training session made things very clear and was helpful in
composing my feedback. The step-by-step process of feedback training was very
helpful in making things very standard and complete.

I like the feedback rating sheet very much.

The feedback training helped me develop better feedback skills.

30
The feedback training was well organized and very specific. The training and
particularly the written material was very helpful and well organized.

Feedback quizzes and intense training helped a lot.

Filling out the QSF form helped illustrate specific intents. I think this exercise
really helped pinpoint specific goals and expectations of the SP. It also made me
feel like an active and important part of this program. Feedback used to be a bit
cloudy – this really clarified our role and expectations.

Research Question Two

Did workshop faculty perceive standardized patients as providing effective feedback?
Frequencies were used to determine the percent at which the faculty felt the SPs
accomplished the individual steps of providing effective verbal feedback, (n = 143).
Results from the Faculty Workshop Feedback Form indicated that faculty perceived the
SPs gave verbal feedback to the students in the ICM workshops utilizing the steps of
feedback according to the QSF form.
Table 4
Faculty Workshop Feedback Form
Frequency
1. The SP portrayal was realistic. 100%
2. SP asked students to reflect. 94%
3. SP gave specific positive feedback. 100%
4. SP gave specific constructive feedback. 97%
5. SP finished with specific positive feedback. 96%
6. SP verified student learning. 93%
7. SP checked whether student(s) had any questions. 97%
8. SP thanked the students 94%

31
The Faculty Workshop Feedback Form also contained an open-ended section for
comments. Table 5 reflects the faculty comments related to the feedback provided by
their SP during the ICM workshops.
Table 5
Faculty Comments of SP Feedback by Frequency

SPs Gave Specific Feedback

SPs Gave Inadequate Feedback

Frequency

28

6

The total number of faculty comments (n = 72) were analyzed for themes relating
to SP feedback. Two themes emerged, the first was that the SPs gave specific feedback
to the students, although it is unknown if the feedback was positive or constructive.
Secondly, in a few instances, faculty felt that the feedback provided was inadequate. The
inadequacy of the feedback was not detailed further in the comments.
Research Question Three
Did more students perceive standardized patients as providing effective feedback
after the revised training protocol?
Table 6 summarizes the findings from student responses regarding SP feedback
pre-intervention and post-intervention.
Table 6
Student Online Evaluation of SP Workshop
Pre-Intervention
Mean
Post-Intervention
Mean
Value
1. The SP communicated how
the character (Case Name) felt
during the encounter.

4.44

4.48

p < 0.001
2. The SP gave positive
feedback.
4.38 4.44 p < 0.001
3. The SP gave constructive
criticism (feedback).

4.09

4.19

p < 0.001

32
Table 6, Continued
Student Online Evaluation of SP Workshop
4. The feedback provided by
the SP during the session
wrap-up will be useful in
future SP or real patient
encounters.

4.09

4.18

p < 0.001
* 5-point Likert scale; 5 = strongly agree, 1 = strongly disagree
A one-sample T-test was used to compare the means of the pre-intervention
student ratings to the post-intervention online student evaluations, (n = 24). The values
for each of the four questions indicate a significant improvement in students perceiving
that SPs provided more useful verbal feedback after the revised training protocol was
initiated.
Research Question Four
Did the standardized patients utilize the QSF from the revised training protocol
when providing verbal feedback?
Two independent raters viewed videos from the 240 ICM workshop videos
available on post-intervention digital recordings. It was decided that for the purposes of
this study, that the reviewers would each watch the same 25 videos, randomized from the
240 available using the software program GraphPad. These raters were oriented to the
QSF form, and then assessed whether the standardized patients in the ICM workshops
provided feedback according to the seven steps outlined in the QSF. Table 7 presents the
results of the descriptive analysis for each main section of the QSF form as well as inter-
rater reliability for these sections.

33
Table 7
Independent Rating of Sample Videos Utilizing the QSF
QSF Section Mean SD Inter-rater Reliability
1. SP asked students to reflect
Scale 0=not done, 4=completely done
2.94 .96 .95
2. SP gave positive feedback
Scale 0=not done, 3=completely done
2.31 .94 .95
3. SP gave constructive feedback
Scale 0=not done, 3=completely done
2.02 1.15 .88
4. SP showed empathy if students
appeared distressed
Scale 0=not done, 3=completely done

2.77

.82

1.00
5. SP finished with positive feedback
Scale 0=not done, 2=done
1.43 .90 .95
6. SP verified students’ learning
Scale 0=not done, 3=completely done
.85 1.13 .86
7. SP asked if students had
questions/thanked the students
Scale 0=not done, 2=completely done

1.35

.72

.93
8. Total Score
Scale 0=not done, 20=completely done
13.87 5.11 .98

The means of each section from direct observation were lower than expected.
Results indicated that the SPs in the reviewed videos did not fully utilize the format of the
QSF form when giving verbal feedback to the groups of students in the ICM workshops.

34
Chapter Four: Discussion
This chapter is organized into three sections. The first presents the conclusions to
be drawn from the study and discusses the relationship between them and the existing
literature. Second, the limitations of the study are presented. Lastly, implications for
medical educators will be offered.
Conclusions
This study provided valuable insight into the SPs’ ability to provide quality or
effective verbal feedback to medical students in the ICM workshops, as well as
perceptions of the faculty and medical students on the effectiveness and quality of
feedback given. Four conclusions were reached:
Conclusion One
The SPs were satisfied with the revised training protocol which included a session
dedicated to how to provide quality verbal feedback. The overall means from this
questionnaire, (n = 129) were high, yet it cannot be determined with certainty that the
SPs’ knowledge, skills or confidence in providing quality verbal feedback were achieved.
The SPSQ instrument was not specific to feedback which makes it difficult to definitively
conclude that the revised training protocol improved the SPs quality of feedback. Several
items on the SPSQ were double-barrel questions which made analysis specific to
feedback difficult to interpret. Specifically, the one question which did include
knowledge, skills, and confidence in providing feedback (Question 10) consisted of
confounding constructs and therefore its value in helping to answer research question one
was precluded.

35
Although the results showed a high mean for SPQS Question eight, feedback from
the trainer was helpful for improving performance, the question did not define the term
“performance” and therefore no definitive conclusions can be reached to determine if the
SPs interpreted it as portrayal of the case or in giving feedback. However, this question
did reflect the importance of effective feedback to learners in achieving their goals. If the
SPs found that feedback from the trainer contributed to their own knowledge and skills in
performing the case and in providing effective verbal feedback, then conversely it could
model how to provide effective verbal feedback and reinforce the importance of their
feedback to the medical students (Bandura, 1989). The trainer role modeled the process
of giving effective verbal feedback when providing feedback to the SPs in the third
training session as they portray the case and provide verbal feedback to mock students.
The lowest mean of the SPSQ was Question nine, the Feedback Training
Handouts/Manual [were] was helpful. Since the manual consisted of several
components, it could not be determined which part(s) were or were not useful to the SPs.
A more psychometrically sound instrument would help in determining the areas in the
manual which need further development. Also, providing the manual to the SPs in
advance of feedback training sessions might assist them in understanding the feedback
process, as prior knowledge aids in the construction of new knowledge and learning is
increased. Consideration must be given that three hours may be too long a session and
the SPs may become cognitively overloaded.
The training manual, How to Give Effective Feedback, developed by Dr. May and
Dr. Fisher (2004/2006) was the first known attempt to provide a cognitive and social

36
cognitive structure for teaching feedback to SPs. Standardized patient programs could
devote effort in building upon the authors’ work by further investigating learning issues
for SPs which are currently lacking in medical education literature.
While the quantitative data from the SPSQ was inconclusive with regard to
improving the quality of SP verbal feedback, the qualitative data was positive overall.
Two distinct themes emerged from the SP’s comments concerning their satisfaction with
the revised training protocol. The first theme was the SPs found benefit and value from
the small groups from the third training session where they role-played the case and
practiced providing verbal feedback to mock students, (n = 88). The frequency of this
theme demonstrated that the SPs found that the comprehensive training activities,
including the additional feedback training session, contributed to their knowledge, skills,
and confidence in playing the case and in providing effective verbal feedback. Training
session number three combined the knowledge learned in training sessions one and two,
and allowed the SPs to transfer their knowledge into skills. This final training session
utilized practice in small groups, which are more conducive than large groups for each SP
to play the patient role, provide verbal feedback to mock students, and receive feedback
from their peers and the trainer on their portrayal and their verbal feedback. Social
cognitive theory states that learning occurs by observing others, especially from models
who are deemed competent, credible, similar and enthusiastic (Schunk et al., 2008). An
example of the social cognitive theory in practice is the following SP comment:
“breaking into small groups and watching/commenting, asking questions as we each went

37
through it; because different people will bring different things to the table and you learn
from all of it.”
The second major theme, the feedback training session utilizing the CD, DVD,
and QSF (n = 26) was helpful for the SPs to learn how to provide effective verbal
feedback. Although the quantitative data indicated that the feedback training manual was
not helpful, comments from the SPs were positive concerning the training session
devoted solely to how to give effective verbal feedback. Therefore, it appeared that the
structured training session based upon the cognitive and social cognitive learning theories
in the literature review assisted some of the SPs in learning how to provide effective
verbal feedback.
The data needed to answer research question one were not obtained from the
SPSQ. The instrument was originally developed by the PI as the final project in a
Masters’ class for program evaluation with no reliability or validity established prior to
its use. Recognizing that is it important to teach and assess the SPs as cognitive learners,
more definitive data could be obtained from a sound psychometric instrument and this
will be pursued in future studies.
Conclusion Two
The faculty perceived that the SPs provided quality verbal feedback as evidenced
by the high frequencies in the data (93%-100%). The FWFF is specific to verbal
feedback as it is aligned with the major sections of the QSF form. Thus, it was designed
to measure if the SPs provided feedback according to the seven step feedback process as
trained in the revised training protocol utilizing the QSF form. ICM workshop faculty

38
were not oriented to the QSF form and therefore used their judgment of whether the SP
assigned to their ICM workshop groups provided feedback according to the steps in the
FWFF. Although the six students in the ICM workshop groups changed from the first to
the second session, the faculty and SPs remained the same, hence only one rating form
was used by faculty for both workshops sessions to rate their SP’s verbal feedback.
Although the faculty had no formal orientation to the FWFF nor to the steps of
verbal feedback, the quantitative data from the Faculty Workshop Feedback Form
indicated that faculty were able to distinguish between question three, SP gave specific
positive feedback and question five SP finished with specific positive feedback. They
were also able to make the distinction between question six, SP verified student learning,
from question seven, SP checked whether the student(s) had any questions. These results
indicated that although the faculty had received no formal instruction to the rating form,
they demonstrated an understanding of the seven step feedback process and were able to
evaluate the SPs’ verbal feedback. However, despite the indication that the faculty knew
how to rate the SP feedback, definitive data does not exist to conclusively determine this
conclusion.
Comments from the open-ended section of the FWFF noted 28 faculty perceived
the SPs provided specific feedback to the students in their two ICM workshop groups.
The comments do not delineate whether the specific feedback was positive or
constructive. Six faculty noted that the SPs provided inadequate feedback to their groups
of students, but gave no additional information as to why they considered it inadequate.

39
Thus, these comments did not reflect if the faculty felt that the feedback was general
rather than specific, or that one or more of the steps of feedback was omitted.
While the faculty did perceive that the SPs provided effective verbal feedback in
the ICM workshops, no pre-intervention data exists to conclusively determine whether
these findings were due to the revised training protocol with the feedback intervention or
other variables not measured. Future studies which include pre- and post- intervention
measurements of faculty’s perceptions of SP feedback would contribute to the literature
as a valuable source of data.
Conclusion Three
Students receiving feedback from SPs who participated in the revised training
protocol rated the feedback higher than students in the pre-intervention academic years.
Data from the student online evaluations specific to feedback provided by the SPs in the
ICM workshops showed a significant improvement of the verbal feedback. Post-
intervention means were significantly higher (p < .001) for all four questions than as
reflected in the pre-intervention means for the same questions. The student online
evaluation was based on a 6-point Likert scale of 0 (not applicable) to 5 (strongly agree),
however the count of those students who answered 0 for any question were omitted from
the means since this indicated that they did not participate in the workshops. Raw data
were not available for analyses, thus no standard deviations were reported for these
means.
Verbal feedback from the SPs to the medical students was provided to the group
as a whole and was not specific to individual students. Thus, student perceptions may

40
have varied as to how they interpreted if the feedback was positive or constructive in
nature, as well as if the feedback provided by the SPs will be useful in their future patient
encounters. The feedback format in the ICM workshops did not change from pre-
intervention to post-intervention. Therefore, the high means post-intervention indicated
that some transfer of learning did occur from the SP verbal feedback, however the degree
of learning by each individual student was not able to be measured. The format of group
verbal feedback in the ICM workshops is similar to that of attending faculty providing
verbal feedback to their groups of learners in a clinical setting, a format of feedback the
students will encounter as they begin their clinical rotations.
An important consideration emerging from this study is that of orienting the
students to feedback. In the KSOM, there is no such formal orientation and the
expectations of the students differ from the feedback task assigned to the SPs. Feedback
is given by the SPs to the students from their character’s point of view, with respect to the
interactions between the student(s) and SP within the encounter with the domains limited
to interpersonal and communication skills. Students however, seek feedback regarding
the content of the encounter in an effort to know if they asked the “right” questions.
These differing perceptions of feedback could be addressed as a short orientation to
feedback in the Year I ICM curriculum at the students’ first interviewing techniques
workshop, as well as noted in their ICM Student Manual. This would assist the students’
knowledge of the feedback processes they will experience in SP encounters and later in
their clinical years.

41
Conclusion Four
Standardized patients did not fully utilize the QSF when providing verbal
feedback to medical students in the ICM workshops. Total QSF scores rated by two
independent reviewers against videos were lower than expected. Video review provided
the following valuable and unexpected findings related directly to the SPs attempting to
provide verbal feedback. Twenty-three SPs did not provide verbal feedback utilizing the
seven steps as they were taught in the feedback training session either because their ICM
faculty interrupted the feedback or the SPs used case training materials as feedback. The
raters noted that out of the 25 ICM workshop videos, only one SP provided feedback
according to the QSF with a total score of 17. The SP was able to accomplish this
because her ICM workshop faculty did not interrupt her during the feedback and she
knew how to utilize the feedback steps. The raters observed that faculty interrupted SPs
at different points in the feedback process to interject their own perspectives and ended
up taking over the sessions. These interruptions prevented the SPs from finishing their
feedback and ran the SPs out of time. One SP was interrupted by faculty at the beginning
of feedback, thus affecting his total QSF score (1 out of 20). Overall, these faculty
interruptions made it unclear as to whether the SPs knew the process of providing
effective verbal feedback.
Additionally, standardized patients used case training materials as feedback which
also contributed to the low total QSF scores. These deviations from the QSF feedback
process tended to occur in the first and seventh sections of the QSF form. When students
were asked to self-reflect on the encounter (QSF section one), they focused on content of

42
the questions, specifically what they missed rather than on the process of communication
and professional skills. The SPs accommodated the students’ questions regarding case
content by providing training material facts rather than verbal feedback on the process of
the encounter. Additionally, at the end of the feedback when the SPs asked the students
if they had questions, the students asked about content which would lead the SPs to give
case training materials.
Although providing content of the training materials could be considered
feedback, in the context of the ICM workshops it is not the focus for SP feedback. As
discussed earlier, the ICM workshop faculty facilitate the student groups and thus are
responsible for teaching the students appropriate case content and providing feedback
based upon questions asked or not asked. The SPs on the other hand, are expected to
provide verbal feedback based upon the interpersonal and communication skills of the
students as they interacted with the SP. This distinction is important as it defines the
boundaries of the SP verbal feedback to the students.
Future program efforts need to focus on faculty development for SP feedback
since feedback is critical for improvement of the medical students’ communications
skills. Two issues are at task with having faculty become familiar with SP feedback:
(1) they need to increase their knowledge of the SPs’ provision for verbal feedback in
relation to student learning; and (2) they need to allow the SP to provide verbal feedback
without interruption so as not to impose their bias into the communication between the
SP feedback sender and the student feedback receiver(s).

43
Additionally, further effort needs to be devoted by SP programs in teaching the
SPs what constitutes feedback. Consideration must be given that the one three-hour
session devoted to how to provide feedback in the revised training protocol may not be
sufficient for the SPs to learn the steps or the components of verbal feedback.
Conversely, a single comprehensive session to practice playing the case and providing
feedback to mock students may not be enough time for the SPs to develop proficient
feedback skills.
It is important to note that for the purposes of this study, only 25 out of a total
possible 240 videos were reviewed for quality verbal feedback given according to the
seven steps of the QSF form. Therefore, the sample number of videos selected may not
accurately represent the targeted census.
A study to compare faculty rated feedback forms to the same forms rated by
independent reviewers on direct observation would provide valuable information as to the
feedback steps the SPs omit with greatest frequency. Studies of SPs focus groups would
also help determine the barriers to verifying their verbal feedback with students. One of
the most useful future studies would be investigation of whether the students from the
ICM workshop groups found the SP feedback useful in their real patient encounters.
Finally, inter-rater reliability for the QSF form was high for each major section,
ranging from .86 to 1.00 with a value of .98 for the total score. This strong reliability
provided evidence that the two independent raters gave similar scores across all sections
of the QSF.

44
Limitations
There are a few methodological limitations that must be considered when
discussing the findings in this study. This was a retrospective study which analyzed
previously collected data. Three of the four measurement instruments were not
psychometrically sound and so it cannot be determined with certainty if the feedback
intervention was effective. The SPSQ contained several questions that were not related
to feedback, and it also contained questions with confounding items.
The participants in this study were limited to SPs who fit the demographics of the
cases needed for the ICM workshops, and thus not all SPs in the program were included.
Several of the SPs were utilized in more than one workshop and thus received the revised
training protocol more than once. The expertise of providing verbal feedback in this
sample of SPs could be considerable secondary to gender, age, and life experience or lack
thereof, all of which were not used as variables in this study.
It is important to note that there are no pre-intervention data available from the
SPs, the faculty, or video reviews against the QSF to determine whether the additional
training session for feedback was significant in its efforts to teach the SPs how to give
better and more effective verbal feedback to the medical students. As well, only mean
scores were available from the student online evaluation of the SP feedback, with no raw
data available for student ratings. This study used self-reported, retrospective data from
the students, as they had several weeks in which to post their workshop evaluations
online. Several factors can cause people to incorrectly processor information about their
past experiences (Nisbett & Wilson, 1977). They may reshape the past to conform to

45
their current needs and circumstances, and they may remember more positive feedback
than constructive feedback unless negative emotions were associated with the feedback.
ICM workshops occasionally had to use substitute faculty who may not have been
familiar with the SP feedback process. Finally, this study was conducted at only one
institution, so that these findings may not be generalizable to other institutions or SP
programs.
Implications for Medical Educators
The previous conclusions produced valuable information that can be used by medical
educators and SP programs to enhance the learning of SPs in order to provide effective
verbal feedback and maximize positive outcomes for medical students in future real and
SP patient encounters. One of the more important findings from this study is that
standardized patients construct their own knowledge of feedback and provide it to the
students as they understand it. Therefore, SP programs need to shift the paradigm away
from training SPs which implies a behaviorist model, to one of teaching SPs utilizing a
cognitive model. As noted initially and has been shown in this study, SPs are learners
and their learning is influenced by cognitive and social cognitive factors.
Providing verbal feedback requires more than knowledge; it also requires the
transfer of the knowledge into skills. Teaching sessions utilizing cognitive and social
cognitive learning theories can assist the SPs to become knowledgeable and skillful
regarding the steps needed to give effective verbal feedback. A final teaching session is
essential for providing SPs the opportunity to practice their feedback skills with mock

46
students utilizing the social cognitive theory of role modeling, inhibition and disinhibition
(Schunk et al., 2008).
Not all standardized patient programs require SPs to give verbal feedback. For
those programs which do, it is imperative that the SPs receive some form of teaching
about how to give effective verbal feedback in order to ensure positive learning outcomes
for the medical students. In light that unstructured feedback may be harmful to students,
the benefits of having learned SPs give verbal feedback far outweigh the costs of
providing such teaching.
This study provided a foundation for future studies devoted to teaching feedback
which are currently lacking in medical education literature. The findings from this study
will be useful in future SP methodology studies at the KSOM and at other medical
schools who want to implement SPs giving verbal feedback and desire positive student
outcomes.

47
Bibliography

Ash, M.G. (1992). Cultural contexts and scientific changes in psychology. American
Psychologist, February, 198-207.

Bandura, A. (1989). Human Agency in Social Cognitive Theory. American Psychologist,
44(9), 1175-1184.

Bienstock, J.L., Katz, N.T., Cox, S.M., Hueppchen, N., Hueppchen, N., & Erickson, S.
(2007). To the point: medical education reviews – providing feedback.
American Journal of Obstetrics & Gynecology, June, 508-513.

Claiborn, C.D. & Goodyear, R.K. (2005). Feedback in Psychotherapy. JCLP/In
Session, 61(2):209-217. Retrieved August 12, 2007 from
www.interscience.wiley.com.

Claiborn, C.D., Goodyear, R.K., & Horner, P.A. (2001). Feedback. Psychotherapy,
Theory, Research, Practice, Training, 38(4):401-405.

Clark, R.E. & Estes, F. (2002) Turning Research Into Results: A Guide to Selecting the
Right Performance Solutions. Atlanta, GA: CEP Press.

De Champlain, A.F., Margolis, M.J., King, A., & Klass, D.J. (1997). Standardized
patients’ accuracy in recording examinees’ behaviors using checklists. Academic
Medicine, 72(10):S85-87.

Heine, N., Garman, K., Wallace, P., Bartos, R., & Richards, A. (2003). An analysis of
standardized patient checklist errors and their effect on student scores. Medical
Education, 37:99-104.

Howley, L.D. & Martindale, J. (2004). The efficacy of standardized patient feedback in
clinical training. A mixed methods analysis. Medical Education Online, 9:18.

MacRae, H.M., Vu, N.V., Graham, B., Word-Sims, M, Colliver, J.A., & Robbs, R.S.
(1995). Comparing checklists and databases with physicians’ ratings as measures
of students’ history and physical-examination skills. Academic Medicine,
70(4):313-317.

Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we know: Verbal reports
on mental processes. Psychological Review, 84, 231-259.

Ormrod, J.E. (2006). Educational Psychology: Developing Learners (5
th
ed.). Upper
Saddle River, NJ:Pearson Merrill Prentice Hall.

48
Ormrod, J.E. (2008). Human Learning (5
th
ed.). Upper Saddle River, NJ:Pearson
Prentice Hall.

Patton, M.Q. (2002). Qualitative research and evaluation methods (3
rd
ed.). Thousand
Oaks, California: Sage Publications, Inc.

Sachdeva, J. (1996). Use of Effective Feedback to Facilitate Adult Learning. Journal of
Cancer Education, 11(2), 106-118.

Schunk, D., Pintrich, P., & Meece, J. (2008). Motivation in education: theory, research,
and applications (3
rd
ed.). Upper Saddle River, NJ:Pearson Education, Inc.

Sender-Liberman, A., Liberman, M., Steinert, Y., McLeod, P. & Meterissian S. (2005).
Surgery residents and attending surgeons have different perceptions of feedback.
Medical Teacher, 27(5): 470-472.

Tamblyn, R.M., Klass, D.K., Schnabl, G.K., & Kopelow, M.L. (1990). Factors associated
with the accuracy of standardized patient presentation. Academic Medicine,
65:S55-6.

Van de Ridder, J.M. Monica, Stokking, K.M., McGaghie, W.C., & ten Cate, O. TJ.
(2008). What is feedback in clinical education. Medical Education, 42,189-197.

Veloski, J., Boex, J.R., Grasberger, M.J., Evans, A., & Wolfson, D.B. (2006).
Systematic review of the literature on assessment, feedback and physicians’
clinical performance. Medical Teacher, 28(2), 117-128.

Vu, N.V., Barrows, H.S., Marcy, M.L., Verhulst, S.J., Colliver, J.A., & Travis, T.A.
(1992). Six years of comprehensive, clinical performance-based assessment using
standardized patients at the southern illinois university school of medicine.
Academic Medicine, 67(1):42-50.

Vu, N.V., Marcy, M.L., Colliver, J.A., Verhulst, S.J., Travis, T.A., & Barrows, H.S.
(1992). Standardized (simulated patients’ accuracy in recording clinical
performance check-list items. Medical Education, 26(2):99-104.

Westberg, J. & Jason, H. (2001). Fostering Reflection and Providing Feedback: Helping
Others Learn from Experience. New York, NY:Springer Publishing Company.

Williams, R.G. (2004). Have standardized patient examinations stood the test of time
and experience? Teaching and Learning in Medicine, 16(2):215-222.

Wood, B. (2000). Feedback: A key feature of medical training. Radiology, 215, 17-19.

49
Appendix A
Quality of SP Feedback Form (QSF)
Date
Case ID
Rater ID

SP asked student to reflect
1. SP: So, how do you think it went? 1
2. SP: So, what are some things you think you did well? 1
3. SP: Is there anything you would do or say differently if you could do this over again? 1
4. Gave student adequate time to answer questions before continuing 1

SP gave positive feedback
5. SP first gave positive feedback 1
6. SP’s positive feedback referred to specific changeable behaviors 1
(Check if positive feedback MOSTLY specific—some generalization okay)
7. SP gave feedback from patient’s perspective 1

SP gave constructive feedback
8. SP’s negative feedback referred to specific changeable behaviors (feedback not destructive) 1
9. SP limited the constructive feedback to 2 or fewer points 1
10. SP gave constructive feedback from patient’s perspective. 1

SP showed empathy for distressed student (if student not distressed go to question #14)
11. SP stopped feedback and acknowledged students’ feelings 1
SP: I’m feeling that you might be upset by this feedback.
12. SP confirmed the feelings with student. 1
SP: Are you feeling [sad, angry, upset], or, Is this true?
13. SP reassured student about purpose of feedback 1
SP: Giving you feedback is our way to help you
14. If student didn’t appear distressed by feedback, check box at right and go to question #15. 3

SP finished with positive feedback (sandwich)
15. SP finished feedback on a positive note. 2

SP verified student’s learning
17. SP asked student to summarize feedback given. 1
SP: What have you learned from this feedback session?
18 The SP ensured that the student understood what s/he (the student) needed to work on 2

50
At end of session, SP asked student if s/he had other questions
19. SP continued to ask student if he/she had questions until student said “no.” 1
SP: Do you have any other questions or comments? Anything else you would like to ask?
20. SP thanked the student. 1
SP: Thank you for your effort here today. I feel privileged to be part of your education.

© 2006 Win May, Dixie Fisher TOTAL SCORE

51
Appendix B

Standardized Patient Program Training Questionnaire

Directions: For each question below, please circle the answer that you feel best describes
your experience with the Standardized Patient (SP) training sessions.

Please use the following to guide your responses:
5 = Strongly Agree 4 = Agree 3 = Neutral 2 = Disagree 1 = Strongly Disagree
N/A = Not Applicable

1. The expectations of the program and SP responsibilities were clearly explained.

5

4

3

2

1

N/A

2. The role of SPs in the curriculum was clearly explained.

5

4

3

2

1

N/A

52
3. The objectives for the SPs were achievable.

5

4

3

2

1

N/A

4. Trainings were well organized.

5

4

3

2

1

N/A

5. The content was appropriate and the quizzes helped me learn my materials better.

5

4

3

2

1

N/A

53
6. Trainings were presented at a level that I could understand.

5

4

3

2

1

N/A

7. The training activities included active involvement of the standardized patients.

5

4

3

2

1

N/A

8. Feedback from the trainer was helpful for improving performance.

5

4

3

2

1

N/A

54
9. The Feedback Training Handout/Manual was helpful.

5

4

3

2

1

N/A

10. The comprehensive training activities helped to develop the knowledge, skills, and
confidence needed to meet the objectives in portraying the case and in providing effective
written/verbal feedback to students.

5

4

3

2

1

N/A

11. I was treated as a valuable member of the team and felt that my efforts were respected.

5

4

3

2

1

N/A

55
12. Which training activity did you find most useful and effective? Why?

13. Was there anything that made training difficult? If Yes, state why.

Please add any comments or suggestions for Program improvement with regard to the training
sessions. Your comments are valuable to the program and we appreciate any feedback you wish
to share.

Thank you for taking the time to complete the questionnaire!

56
Appendix C
Faculty Workshop Feedback Form

1. The SP portrayal was realistic Yes
No

2. SP asked student to reflect Yes
No

3. SP gave specific positive feedback Yes
No

4. SP gave specific constructive feedback Yes
No

5. SP finished with specific positive feedback Yes
No

6. SP verified student learning Yes
No

7. SP checked whether students(s) had any questions Yes
No

8. SP thanked the students Yes
No

Please make any comments in the space below if you would like to elaborate on any of
the above statements.

57
Appendix D
Student Online Evaluation of SP Workshop

1. The SP communicated how the character (Case Name) felt during the encounter.

2. The SP gave positive feedback.

3. The SP gave constructive criticism (feedback).

4. The feedback provided by the SP during the session wrap-up will be useful in future
SP or real patient encounters.

0 = Not Applicable
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree

Comments:

Abstract (if available)

Abstract The purpose of this mixed-methods retrospective study was to examine the effectiveness of a training intervention for standardized patients to improve their verbal feedback to medical students. Although feedback is acknowledged as important for medical student development, actual interventions to improve effective feedback to medical students are scarce in the literature.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Medical students' reflections on professional behavior: cognitive style as a predictor of content and level of integrative complexity

PDF

The effect of cognitive task analysis based instruction on surgical skills expertise and performance

PDF

Advanced surgical skill assessment: task analysis as a model for the development of a valid and reliable objective procedural checklist

PDF

Clinical skills instruction: evaluation of a female gynecologic examination curriculum

PDF

Employing cognitive task analysis supported instruction to increase medical student and surgical resident performance and self-efficacy

PDF

21st century skills as equity pedagogy to teach academic content standards: a socioeconomically diverse district's implementation

PDF

The use of cognitive task analysis to capture exterptise for tracheal extubation training in anesthesiology

PDF

The effects of culturally responsive standards based instruction on African American student achievement

PDF

Factors influencing gifted students' preferences for models of teaching

PDF

The use of cognitive task analysis to investigate how many experts must be interviewed to acquire the critical information needed to perform a central venous catheter placement

PDF

Allocation of educational resources to improve student learning: case studies of California schools

PDF

The use of cognitive task analysis for the postanesthesia patient care handoff in the intensive care unit

PDF

An examination of teacher-centered Explicit Direct Instruction and student-centered Cognitively Guided Instruction in the context of Common Core State Standards Mathematics…

PDF

Measuring the alignment of high school and community college math assessments

PDF

Developing professional expertise in paramedic students using the gap analysis framework

PDF

Allocation of educational resources to improve student learning in a sample of California schools

PDF

The use of cognitive task analysis to determine surgical expert's awareness of critical decisions required for a surgical procedure

PDF

Examining urban high school English language arts teachers’ written feedback to student writing and their perceptions and applications of culturally relevant pedagogy

PDF

Assessing the impact of diversity courses on students’ values, attitudes and beliefs

PDF

An evaluation of the impact of a standards-based intervention on the academic achievement of English language learners

Asset Metadata

Creator Souder, Denise M. (author)

Core Title Teaching interpersonal and communication feedback skills to standardized patients: assessment of a cognitive model

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Degree Conferral Date 2009-05

Publication Date 05/11/2009

Defense Date 03/27/2009

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag cognitive,feedback,Learning and Instruction,OAI-PMH Harvest,standardized patient

Place Name educational facilities: Keck School of Medicine (geographic subject)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Sullivan, Maura (committee chair), Goodyear, Rodney K. (committee member), May, Win (committee member)

Creator Email dsouder@usc.edu,souderfam@roadrunner.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m2229

Unique identifier UC1280478

Identifier etd-Souder-2857 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-238303 (legacy record id),usctheses-m2229 (legacy record id)

Legacy Identifier etd-Souder-2857.pdf

Dmrecord 238303

Document Type Dissertation

Rights Souder, Denise M.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu