University of Southern California Dissertations and Theses

Building a framework for self-regulated learning in surgical education

(USC Thesis Other)

Building a framework for self-regulated learning in surgical education

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content
Running head: SRL IN SURGICAL EDUCATION 1

BUILDING A FRAMEWORK FOR SELF-REGULATED LEARNING IN
SURGICAL EDUCATION

By

Cali Elyse Johnson, MD

A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

August 2018

Copyright 2018 Cali Elyse Johnson

SRL IN SURGICAL EDUCATION 2

Acknowledgements
It is an honor to write this section to acknowledge the people who have been instrumental
throughout this process. I would like to first thank my chair, Dr. Kenneth Yates, for the
countless hours of dedication and comprehensive supervision he provided to me. Thank you for
your clear guidance, your selflessness, and your adaptability in making this dissertation possible.
Dr. Maura Sullivan, you have been an incredible advocate for me. I have developed into
a professional, an educator, and a researcher under your guidance. My future career path is
clearer because of your mentorship and your belief in me. Dr. Alison Muraszewski and Dr.
Helena Seli, I am grateful for your presence on my committee. Thank you for your personal
encouragement and thoughtful review of my work.
My personal growth and confidence have always been supported by my loving and
encouraging family, especially my mother, father, and step-father. The late night reviews, early
morning chats, and weekend resets have grounded me and continue to fill my emotional tank.
My sister, brother-in-law, and nephews continually bring joy and laughter into my life. Jacadi,
you are my best friend and at times understand me better than I do myself. Thank you for your
honesty, trustworthiness, and compassion. My little brother, Caleb, may be my biggest
cheerleader and fan. Thank you for believing I am a rock-star.
Greg, thank you for standing by my side, being a sounding board for my ambitions, and
embracing the unknown. To my closest friends, you have reminded me to be present. I have
grown tremendously and experienced incredible companionship in this two-year endeavor.
Thank you for keeping me grounded, broadening my horizons, and joining me on adventures.
The people named here, and many who are not, have believed in me even when it was hard,
ultimately making the big dreams feel manageable. Your support provided me the courage to

SRL IN SURGICAL EDUCATION 3

pursue an uncharted course and develop the gifts the Lord has given me. May my life be a
reflection of His grace.

SRL IN SURGICAL EDUCATION 4

Table of Contents
List of Tables 7
List of Figures 8
Abstract 9
CHAPTER ONE: OVERVIEW OF THE STUDY 10
Background of the Problem 10
Statement of the Problem 11
Purpose of the Study 12
Research Question 12
Methodological Framework 12
Significance of the Study 13
Limitations and Delimitations 13
Definition of Terms 14
Organization of the Study 16
CHAPTER TWO: REVIEW OF THE LITERATURE 18
Surgical Education in the United States 18
External Factors Affecting Surgical Training 19
Trainee Competency 23
Competency of Practicing Surgeons 26
Self-Regulated Learning in Educational Psychology 27
Metacognition and Self-Regulation 27
Self-Regulated Learning 29
Characteristics of Self-Regulated Learners 29
Forethought Phase 30
Performance Phase 35
Reflection Phase 37
Measuring SRL 39
Current Research of SRL Strategies in Surgical Education 41
Individualized Learning Plans 42
Forethought 43
Performance 43
Reflection 44
Summary 45

SRL IN SURGICAL EDUCATION 5

CHAPTER THREE: METHODS 47
Research Question 47
Methodology 47
Technical Considerations 49
Anonymity 49
Noise Reduction 50
Sample and Population 50
Data Collection 51
Validity and Reliability 51
Instrumentation 52
Round One Instrument 53
Round One Data Analysis 55
Round Two Instrument 61
Round Two Data Analysis 61
Saturation and Consensus Levels 61
CHAPTER FOUR: RESULTS 63
Participants 64
Summary of Results 64
Round One Quantitative Results 64
Round One Qualitative Results 65
Round Two Quantitative Results 67
Round Two Qualitative Results 69
Summary 72
CHAPTER FIVE: DISCUSSION 74
Purpose of the Study 75
Discussion of Findings 76
Discussion of Quantitative Findings 78
Discussion of Qualitative Findings 78
Alignment With SRL Frameworks 81
Limitations and Delimitations 82
Implications for Practice 83
Recommendations for Future Study 84
Conclusion 85

SRL IN SURGICAL EDUCATION 6

References 87
Appendix A: Round One Qualtrics Survey 111
Appendix B: Round Two Qualtrics Survey 121

SRL IN SURGICAL EDUCATION 7

List of Tables
Table 1. Round One Self-Regulated Learning Constructs 54
Table 2. Modified Round One Self-Regulated Learning Constructs 56-57
Table 3. Domains of Self-Regulated Learning in Surgical Education 58
Table 4. Organization of Domains and Subconstructs 59-61
Table 5. Round One Stem Ratings and Tabulation of Coded Comments 65
Table 6. Comment Coding Examples 67
Table 7. Round Two Level of Agreement 68-69

SRL IN SURGICAL EDUCATION 8

List of Figures
Figure 1. Delphi study structure for data collection and analysis. 53
Figure 2. Two-round Delphi results. 63
Figure 3. Self-regulated learning framework in surgical education. 77

SRL IN SURGICAL EDUCATION 9

Abstract
Practice-Based Learning and Improvement is a Core Competency for surgical residents and
requires demonstration of self-regulated learning, yet research regarding the development and
evaluation of these behaviors in this learning context is limited. This study utilized a two-round
modified Delphi survey technique to develop a framework for self-regulated learning in surgical
education. A panel of 38 experts in surgical education evaluated self-regulated learning
constructs derived from educational, professional and medical literature for their importance in
surgical residency. Based upon quantitative data and thematic coding of comments from round
one, the researchers adapted self-regulated learning constructs for applicability in the context of
surgical residency. These were then reorganized into seven domains: self-awareness, task
analysis, situation awareness, strategic planning, progress evaluation, learning and performance
management, goal attainment and refinement. Thirty-five experts from round one evaluated the
new domains and sub-constructs, with all individual items exceeding the predetermined
consensus level of 70%, and 94% agreement overall. Experts in surgical education believed self-
regulated learning skills are important in training, but emphasized distinct features of the
learning environment for surgical residents: patient care takes priority over learning, learners
have little control over their environment, external resources and support are necessary, and
metacognition is limited even among practicing faculty surgeons. This novel framework
provides an initial guide for directing curriculum and further study into self-regulated learning in
surgical education.
Keywords: self-regulated learning, surgical residency, core competency, practice based
learning and improvement, Delphi, framework, metacognition

SRL IN SURGICAL EDUCATION 10

CHAPTER ONE: OVERVIEW OF THE STUDY
Background of the Problem
For over a century, surgeons have been trained using the apprenticeship model, which
was heavily influenced by Halsted (1852-1922) and Churchill (1895-1972) with the belief that
surgical training should be long enough to provide adequate breadth and depth (Allen, 1907).
Surgical residents, aptly named trainees who resided in the hospital, would learn their craft
through a competitive, hierarchical approach where the strength of a surgeon was rooted on
one’s ability to overcome and excel in this demanding environment, which often exceeded 100
hours per week. Ultimately, societal pressures regarding patient safety and trainee well-being
became an impetus for national hour regulations, while expansion of subspecialty fellowships
and technologic innovations compounded the diversity of knowledge required for surgical
mastery (Pellegrini, 2006).
Recent efforts to improve and standardize the quality of surgical training with a national
competency-based medical education (Holmboe et al., 2015) and more humane duty-hours
(Imrie, Frank, & Parshuram, 2014) have been received with mixed results and unclear benefit to
the trainee or the patient (Jamal, Wong, & Whalen, 2014). When surveyed, faculty perceived a
decline in patient safety and the quality of resident education (Ahmed et al., 2014), believing that
two-thirds of graduates entering specialty training could not be left unsupervised in the operating
room for longer than 30 minutes (Mattar et al., 2013). Faculty cite resident preparation as the
main factor limiting educational experiences in the operating room (Torbeck et al., 2015), and
hold that “learners must learn to be self-directed in seeking assessment and feedback” (Holmboe
et al., 2015, p. 507). Although the national Accreditation Council for Graduate Medical
Education (ACGME) maintains that residents are responsible for their own reading and

SRL IN SURGICAL EDUCATION 11

preparation (ACGME, 2011), evidence suggests that self-regulated learning skills (SRL) do not
develop in medical school (Lucieer et al., 2016; M. H. Murad, Coto-Yglesias, Varkey, Prokop, &
Murad, 2010), and little data exists regarding the efficacy of residents’ SRL behaviors (Aho et
al., 2015).
Statement of the Problem
To better prepare trainees for the demands of residency under hour restrictions, medical
education has aimed to foster self-directed learning (SDL) during medical school. The Liaison
Committee for Medical Education (LCME, 2016) concluded that medical curriculum needs to
include “self-directed learning experiences and time for independent study to allow medical
students to develop the skills of lifelong learning” (p. 8). However, analyses of medical student
education have yielded mixed results, with lack of a clear framework for SDL (M. H. Murad &
Varkey, 2008). Furthermore, studies have demonstrated minimal improvement in outcomes after
SDL interventions (McGrath et al., 2015; Murad et al., 2010), and some evidence suggests that
SDL and SRL decreases as students proceed through medical school (Artino et al., 2012;
Jouhari, Haghani, & Changiz, 2015; Lucieer et al., 2016; Tackett et al., 2015).
The ACGME has recently implemented a set of Core Competencies for residents to
“guide curriculum development…provide explicit expectation for learners…encourage informed
self-assessment and self-directed learning… [and] guide personal action plans for improvement”
(Holmboe et al., 2015, p. 507). While other medical specialties have attempted to define SDL
and SRL (Li et al., 2010; Smith et al., 2011), a search of the literature does not reveal a
framework for SRL in surgical residency. It is imperative to have a solid framework for SRL in
surgical education, which can then serve as a foundation for future work to ensure that residents
are able to develop SRL skills in training and assessment measures of these skills.

SRL IN SURGICAL EDUCATION 12

Purpose of the Study
Thus, the purpose of this exploratory study was to capture expertise from nationally
recognized surgeon educators toward creating a framework for SRL in surgical residency. SRL
constructs were identified from an extensive review of the empirical literature in traditional K-16
educational and professional settings. Surgical experts were asked to rate the importance of
these attributes and strategies in the context of surgical training.
Research Question
The research question guiding this study was “What evidence-based self-regulated
learning attributes and strategies do experts in surgical education deem necessary for residents
during surgical training?”
Methodological Framework
A modified Delphi survey technique (Dalkey & Helmer, 1963) was used for this study.
The goal of a Delphi study is to achieve group consensus through a series of sequential surveys
that are refined based upon the previous responses of the group. This iterative process of
controlled feedback allows the panel of experts to maintain individual, anonymous responses, but
promotes convergence toward consensus (Fink, Kosecoff, Chassin, & Brook, 1984; Keeny,
Hasson, & McKenna, 2010; Hsu & Sanford, 2007). Although traditional Delphi approaches are
qualitative in that they solicit statements from participants, Delphi techniques have been
modified more recently to include quantitative responses, in that participants select from pre-
written responses rather than self- generating them. From a pragmatic view, the Delphi
technique has been chosen as a mixed-methods strategy of inquiry that focuses on the research
problem and utilizes “pluralistic approaches to derive knowledge about the problem” (Creswell,
2014, p. 11).

SRL IN SURGICAL EDUCATION 13

Significance of the Study
The structure of surgical training in the United States has undergone a dramatic shift in
the last century through diversification in subspecialty fellowships and early integration of
specialty training in residency (Chapman, 2016; Sachdeva et al., 2007). Because no two training
experiences are the same, each learner must monitor, adjust and direct their own education to
meet their individual learning needs. As one of the six Core Clinical Competencies, a resident
must engage in “self-initiated, self-directed learning activities” and demonstrate the “ability to
learn from the results of his/her practice” (ACGME & ABS, 2015). However, many residents
are not able to assess their own level of competency (Meyerson et al., 2014) and program
directors believe that many graduating residents are not capable of independent practice (Mattar
et al., 2013). Therefore, it is important to first clearly define and build a framework for SRL in
surgical education to then develop evidence-based curriculum and tools to assist trainees in
identifying and supporting their learning needs. Failure to target individual learners in this
diverse and evolving surgical training environment will continue to produce trainees who feel
unprepared for independent practice and seek additional training after five years of general
surgery residency (Coleman, Esposito, Rozycki, & Feliciano, 2013).
Limitations and Delimitations
There are a few limitations and delimitations that should be addressed. One of the
limitations of this study was the assumption and definition of expertise in the panel selected in
the Delphi method. Individuals selected for the panel were currently active in national surgical
education. There may have been some selection bias based upon respondents that chose to
participate and those may not have viewed themselves as experts despite having significant
experience in the field of surgical education. Another limitation of this study was the potential

SRL IN SURGICAL EDUCATION 14

for leading bias, in that respondents were initially provided SRL behaviors drawn from the
literature in the modified round. This was done to build upon prior research and ensure adequate
breadth of behaviors covered, however this may have created a tendency for respondents to agree
with such statements. Additionally, an inherent limitation was that the statements in round one
did not include the original area of work or quality of the research, and therefore all were
presented as empirical evidence.
The main delimitation of this study was the focus on surgical residents and results
therefore may not be generalizable to other medical specialty trainees or medical students. The
surgical trainee in residency represents a unique population of adult learners who enter the field
with little clinical knowledge or skill, and must leave residency with the ability to function as an
autonomous surgeon. For that reason, the study targeted this group of learners and did not ask
broadly about medical training across specialties or include medical school.
The nature of the study was to produce a framework, grounded in evidence, but built
from expert opinions. Although of great value as a starting point, the framework will ultimately
need to be validated with additional research to confirm accuracy of the results with residents.
Definition of Terms
Certain terms are used throughout this study. They are defined in chronological order
building on the previous definition.
Surgical Resident: A resident physician is a medical doctor who has completed medical school,
pursuing surgical specialty training. General surgical training involves five clinical years of
supervision, and often one or two additional years of research or professional development, prior
to reaching eligibility for national board certification for independent practice (ACGME, 2017).
Surgical Fellow: A surgical fellow is a surgeon in additional training, usually sought

SRL IN SURGICAL EDUCATION 15

immediately following surgical residency, in an area of subspecialty practice. The training
length varies between one and three years (ACGME, 2017).
Attending: Attending physicians are faculty members who have completed training and now
directly oversee surgical residents, fellows and medical students. They are required to devote
sufficient time to administrative and clinical responsibilities, while supporting resident
educational requirements in all competency areas (ACGME, 2017).
Liaison Committee for Medical Education (LCME): Recognized by the US Department of
Education as the accrediting body for US and Canadian medical schools to ensure appropriate
administration and achievement of established standards for medical student education (LCME,
2016).
Accreditation Council for Graduate Medical Education (ACGME): Accrediting body for all
residency programs, both medical and surgical. The ACGME sets standards for education and
training after medical school (ACGME, 2017).
ACGME Competencies: In a movement toward competency-based medical education, the
ACGME released six core competencies across all medical specialties that must be integrated
into resident curriculum. Resident progress toward competency must be documented and
addressed in all areas, including: a) patient care and procedural skills, b) medical knowledge, c)
practice-based learning and improvement, d) interpersonal and communication skills, e)
professionalism, and f) system-based practice (Holmboe, Edgar, & Hamstra, 2016).
Milestones: Released after ACGME core competencies, and developed by each specialty board
to provide specialty specific “performance levels residents and fellows are expected to
demonstrate for skills, knowledge, and behaviors in the six clinical competency domains”
(Holmboe, Edgar, & Hamstra, 2016, p. 9).

SRL IN SURGICAL EDUCATION 16

Metacognition: Awareness of one’s own thinking, as well as awareness of one’s cognitive and
emotional states and intentions (Papaleontiou-Louca, 2014).
Self-regulation: “Self-generated thoughts, feeling, and actions that are planned and cyclically
adapted to the attainment of personal goals” (Zimmerman, 2000, p. 14).
Self-regulated learning: “Learning that results from students’ self-generated thoughts and
behaviors that are systematically oriented toward the attainment of their learning goals” (Schunk,
2001, p. 125).
Organization of the Study
Chapter One of this study presents the background of the problem, the statement of the
problem, the purpose of the study, the question this study addresses, and introduces the
methodology used. It also provides a description of the significance of the study, the limitations,
delimitations, and definitions of terms.
Chapter Two is a review of relevant literature and is divided into three main sections.
The first section provides a historical perspective of the evolution of surgical education in the
United States and current factors that influence the training of surgeons. The second section
introduces SRL from a social cognitive perspective, provides definitions of specific learning
strategies, and implications of SRL on performance. The third section reviews literature on the
limited SRL principles that have been examined in surgical education.
Chapter Three describes the methodology for this study, including the sample and
population, the instrumentation, data collection, and data analysis. The selection of a Delphi
survey technique as the strategy for inquiry in this mixed-methods study is discussed in detail.

SRL IN SURGICAL EDUCATION 17

Chapter Four includes the participants and the response rates for each Delphi survey
round. The quantitative and qualitative results are reported as they relate to each round. Global
themes that emerged throughout the study are presented.
Chapter Five provides a summary of the findings, and the implications of these findings
in the context of available literature. Direction for future research is also presented.

SRL IN SURGICAL EDUCATION 18

CHAPTER TWO: REVIEW OF THE LITERATURE
This literature review is structured in three sections to develop a background that
supports the purpose of this study, building a framework for self-regulated learning (SRL) in
surgical education. The first section describes the history of surgical education in the United
States, and the training implications of the current competency-driven assessment model.
Section two provides an overview of research on SRL in other fields, discussing specific SRL
behaviors and the effect of SRL on learning and performance outcomes. The final section will
report the limited SRL principles that have been previously explored in the field of surgical
education.
Surgical Education in the United States
Early surgical training followed a Halstedian apprenticeship model. This was a
hierarchical or pyramidal design in which several individuals would undergo rigorous training
with extensive exposure to surgical disease, but only a few progress to mastery of skill and
knowledge (Halsted, 1904). This model could take over 12 years of training, during which
individuals would pride themselves on the grueling service, and only the top individuals acquire
sufficient skill to practice autonomously. Even at that level of rigor, the president of the
American Surgical Association felt this length of training was too short (Allen, 1907).
Recognizing the inefficient training strategy and years of lost skill, Churchill instituted a more
rectangular 5-year program in which those entering training should finish it (Grillo, 2004).
That sentiment has held through training today, in that training programs are held
accountable for ensuring that all trainees reach competence. Although unintended, the push to
finish all who enter training has resulted in the dominant five-year surgical residency model that
exists today. This fixed-time training has been challenged in recent years, notably during

SRL IN SURGICAL EDUCATION 19

another presidential address to the American Surgical Association, Pellegrini (2006) stressed that
ensuring competence of trainees has been a recurring theme. With an abundance of surgical
training in new and diverse subspecialties, the surgical community needs to embrace a learner-
centered model; one that provides structured curriculum without the risk of harming patients,
provides a learner the opportunity for practice and permission to fail, and measures competency
through objective evaluation. Understanding the challenges of providing learner-centered
surgical education involves not only recognizing the current rectangular training structure that
maximizes the number of surgeons trained, but also the number of diverse learning environments
that exist. Each of these learning environments and the increased number of individual
specialties that require a unique skill set bring external constraints into this already complex
training model.
External Factors Affecting Surgical Training
Technological and societal forces have shaped the current educational framework of
surgical residency, resulting in increased regulation and need for standardization. The field of
surgery has expanded greatly with technological advances in recent decades. Gone is the historic
adage of “see one, do one, teach one” as general surgery has continued to maximize the number
of diseases it cares far, while minimizing the physiological assault on the patient (Pellegrini,
2006). Demands for increased innovation, more domains of technical skill, and inability to fail,
have resulted in new governmental regulations, increased oversight from accrediting bodies, and
greater public accountability to ensure trainee competency.
Technologic innovation. The minimally invasive era of surgery, which began with
laparoscopic removal of the gallbladder, has pushed new technology on both practicing surgeons
and those in training. Trainees now care for greater numbers of patients, of higher acuity and

SRL IN SURGICAL EDUCATION 20

disease severity, with shorter hospitalization durations, and less direct patient interaction (Way,
1996). These limitations of space, time, and room for error have been met with each individual
specialty believing more time in training should be devoted to developing these unique skills.
The desire for increased exposure and training in advanced skills is multifocal, with
support from specialty societies, associated industries, and trainees. Simulation has been used to
address some of these concerns, but as demonstrated by Cooper and colleagues (2014), trainees
still perceive a lack in standardization for learning these new technologies. Residents reported
that access to simulation is not sufficient and curriculum must include quality teaching by
faculty. Yet many faculty are still early in their careers with simultaneous exposure to new
techniques, and external pressures of their own (Fuchshuber et al., 2015).
Work hour restrictions. Pressures on faculty represent one barrier that limits time for
resident instruction, and recent restrictions on work-hours for the learner is another.
Implemented without evidence to substantiate it, restricting residents to an 80-hour work-week
has not proved to be effective in improving the learning environment for residents (Barden,
Specht, McCarter, & Daly, 2002; Jamal et al., 2011; Jamal, Wong, & Whalen, 2014). Many
argue that hour restrictions significantly cut available time during training by limiting the hours
one can spend in the hospital. Restrictions also fail to recognize that the time actually spent in
the hospital is different, with more administrative work, higher acuity patients, less autonomy,
and with less longitudinal care (Pellegrini, 2006; Way, 1996).
The theoretical benefits of an 80-hour work-week restriction have not come to fruition.
Program directors have requested effective, efficient training with a structure that uses the
curriculum more wisely, and evidence to back up theory-based curriculum (Hutter, Kellogg,
Ferguson, Abbott, & Warshaw, 2006; Neumayer, 2012). Even with a well-structured

SRL IN SURGICAL EDUCATION 21

curriculum, probabilistic models suggest residents may not receive adequate case volume for
advanced skills, and would likely still violate work hours (Obi et al., 2015). Proponents of
increased simulation and deliberate practice recommend better utilization of the early training
years. Chapman (2016) cautions against this shift from patient to simulation as a solution. He
argues that decreased hospital presence would only worsen trust between full-time care providers
and trainees, would decrease teaching opportunities for senior residents, and would worsen a
growing concern of faculty and residents about less resident autonomy during training.
Decreased autonomy. Work-hour restrictions are most prohibitive for early learners,
having the unintended consequence of prolonging the learning curve for surgical skills. Yet, the
development of surgical skills in senior trainees has also been threatened. Historically, trainees
in their final years were not only allowed, but encouraged, to operate and make clinical decisions
autonomously. Medicare now requires that attending surgeons are present for all critical portions
of an operative case (Borman, Biester, Jones, & Shea, 2011; Damewood, Blair, Park, Lupi,
Newman, & Sachdeva, 2017). When surveyed, 47% of faculty saw work-hours as a barrier to
autonomy, with 13% believing the culprit was increased supervision requirements (Teman,
Gauger, Mullan, Tarpley, & Minter, 2014). Surveyed residents also believed they were making
less independent decisions, shifting complex skill development into fellowship training, and
requested further faculty development and engagement to help facilitate resident autonomy
(Franzone et al., 2015). In summary, both faculty and residents recognize the need for adequate
training and have struggled to find solutions to external barriers that affect training. This internal
recognition of external factors has also been met with increased external accountability, often
from public and governmental agencies.

SRL IN SURGICAL EDUCATION 22

Public accountability. The discussion of autonomy varies greatly depending upon the
audience. When surveyed in a multi-institutional study, significant proportions of both faculty
and residents believed trainees had too little autonomy (Kempenich, Willis, Rakosi, Wiersch, &
Schenarts, 2015). Although information and patient education increases willingness, patients are
hesitant to have trainees complete significant portions of complex cases (Kempenich et al.,
2016). Unfortunately, public awareness has not been routinely met with complete transparency,
as many trainees are not involved in all phases of patient care, and may meet the patient
immediately before surgery, or only get formally introduced if the patient stays in the hospital
after the procedure (Wojcik, Phitayakorn, Lillemoe, Chang, & Mullen, 2017).
Increased sub-specialization with evolving training paradigms. Increasing external
pressures to an already constrained system has required restructuring and removal of training
areas that were previously seen as essential. In recent years, up to 80% of residents who
complete five clinical years of general surgery have chosen to pursue additional training in the
form of a fellowship (Regenbogen, 2010), and may not feel adequately prepared to enter practice
after residency (Coleman et al., 2013). Recognizing this increased length of training and high-
quality outcomes in highly specialized surgeons, the surgical community has responded by
creating tailored experiences for residents based upon their future area of focus. In addition to
the traditional pathway for specialization via fellowship after a general surgery residency, many
new paradigms have been created, including early integration of residency and fellowship
straight out of medical school, and early specialization by combining fellowship with the final
year of training (Chapman, 2016).
Surgical training has increased in organization and volume to accommodate the number
of surgeons requiring training, and in response to external pressures that have challenged this

SRL IN SURGICAL EDUCATION 23

system. Some modified training paradigms have been created to provide training that is specific
to the learner’s needs, reducing the number of general surgery years and increasing specialty
exposure in early residency. This has led to an emphasis on examining the outcome or product
of training, in addition to accreditation of the program providing instruction.
Trainee Competency
A formal statement regarding residency redesign has recently been released by the
American Board of Surgery (Shiffer, 2016). The main concern with early specialization
programs that reduce general surgery to its core components is the ability to ensure individual
resident competency. The charge from this accrediting body was for the surgical community to
adopt a new framework for training that includes: a) learner-dependent education; b)
competency-based assessment tools; c) structured assessment to gauge promotion and
completion of training; d) enhanced surgical experience and early surgical exposure; e) faculty
development for teaching and assessing learners of various levels; and f) curricular and
evaluation standards for core and subspecialty training.
Competency-based medical education (CBME). Oversight committees have been
tasked with defining, establishing, and validating competency measures for surgical trainees in
these new educational environments, as well as ensuring competency of surgeons adapting their
practice with new innovations. Assessment of competency in surgical education has been a
major undertaking as the quality and content of assessments have evolved over time (Potts,
2016). Without a clear understanding of the final product, it is difficult to develop, and
ultimately assess, the intermediate educational milestones. In addition, it is also hard to assess
the learner when the training paradigms continually shift, and new technology is adopted
constantly (Norcini & Talati, 2009). Competency-based education has been further complicated

SRL IN SURGICAL EDUCATION 24

by a need to have common outcomes that are required of all practicing physicians and surgeons,
while simultaneously developing different competencies for individual specialties within
surgery.
Accreditation Council for Graduate Medical Education (ACGME). Minimum
standards and oversight of all residency and fellowship programs occurs through voluntary
accreditation by the ACGME. Between 1999 and 2001, the ACGME approved and released the
General Core Competencies as part of their Outcome Project. The expanding number of
subspecialists were the impetus for creating common core requirements for training (Batalden,
Leach, Swing, H. Dreyfus, & S. Dreyfus, 2002). Building upon this initial movement and the
difficulty in operationalizing these general competencies, the ACGME released The Milestone
Project, which served as a blueprint for formative assessments within each specialty (Holmboe,
Edgar, & Hamstra, 2016). The Milestones are specialty-specific and provide guidance for
performance standards by level of training. These are linked to the six Core Competencies, and
are designed to promote learner-centered, assessment-based education in medical training. The
guidebook for implementing the Milestones calls for learner awareness of their own knowledge
and skills development, and is predicated on an active learner that is self-directed in receiving
assessment and feedback (Holmboe et al., 2016).
American Board of Surgery: Formative and summative assessments. One of the
ACGME requirements for all programs is to provide an annual exam as formative assessment of
a resident’s clinical knowledge. The American Board of Surgery (ABS) utilizes an in-training
assessment that is standardized across all general surgery training programs (Pugh & Britt,
2013). The ultimate goal is to provide formative assessment of a resident’s performance prior to
their summative assessment after the completion of residency, which includes the ABS

SRL IN SURGICAL EDUCATION 25

qualifying and certifying exams. Unfortunately, current assessments including ABS In-Training
Exam (ABSITE) scores and faculty evaluation of the Milestones fail to correlate with
performance on these board examinations (Jones, Biester, Buyske, Lewis, & Malangoni, 2014;
Ray et al., 2016), indicating a need for continued development of assessment and competency
measures.
American Board of Surgery: Minimum case logs. While the surgical community
works to define and measure competency, there continues to be debate about the required
minimum exposure to core and complex surgeries. An operative case can be logged as
completed by the trainee if a resident or fellow participates at a predetermined minimum level in
the operation. Analysis of specialty training indicates that residents and fellows may fail to meet
current case requirements within the current work-hour constraints (Obi et al., 2015). Of
procedures that program directors deem as an essential skill for trainees finishing general
surgery, only 15% were performed at least 10 times by the average graduating resident (Bell et
al., 2009). Even more alarming, an in-depth analysis of case logging behaviors showed
significant logging inaccuracy by residents (Balla et al., 2016), and discrepancies between
residents and attendings regarding whether a trainee’s involvement met minimum standards to
warrant logging of a case (Morgan, Kauffman, Doherty, & Sachs, 2017). It therefore becomes
difficult to hold trainees to a) milestones for performance if they ultimately do not correlate with
summative assessments; b) annual examinations if they are not predictive of achieving board
certification; c) or minimum case requirements if they cannot be feasibly obtained within work-
hour regulations. Although progress has been made toward competency-based training in
surgical residency, it is still unclear whether any of these measures denote competency.

SRL IN SURGICAL EDUCATION 26

Competency of Practicing Surgeons
Despite difficulties determining competency in trainees, many have argued that learners
do direct their learning to support sufficient skill development, often by delaying independent
practice for additional training in fellowship (Regenbogen, 2010). For those who enter
fellowship, program directors have reported concern as to whether the trainee could have entered
independent practice from residency (Mattar et al., 2013). In this survey, two-thirds of
fellowship directors felt uncomfortable leaving the graduate unsupervised in the operating room
for longer than 30 minutes. Graduating residents report similar concerns, with 27% reporting
apprehension about performing surgical procedures independently (Yeo et al., 2009).
Transition to practice. Increased concern regarding the competency of graduating
residents and the need for fellowship to gain proficiency has resulted in a focus on improving the
transition from residency to independent practice. During this transition, increasing oversight
and coaching early in practice has been suggested as a solution to a lack of autonomy in the final
years of residency (Hashimoto, Bynum, Lillemoe, & Sachdeva, 2016; Sachdeva et al., 2014).
This paradigm views the transition from residency to independent practice as a continuum for
professional development, and an opportunity to use competency measures for increasing
autonomy, both during and after training.
Lifelong and individualized learning in surgery. Viewed on a continuum of
competency, the surgical community has recognized the need to incorporate ongoing
competency evaluation in the setting of advanced technologies, aging surgeons, and new practice
models. This has been addressed with Continuing Medical Education credits and Maintenance
of Certification assessments, as well as suggestions for next steps including video-based
coaching, and teaching surgeons lifelong learning skills. In order to support surgeons throughout

SRL IN SURGICAL EDUCATION 27

their career, leaders have called for use of multi-modality curriculum and integrated training
approaches to meet surgeon-specific needs (Sachdeva, Blair, & Lupi, 2016). Peer coaching
programs have emerged to provide practicing surgeons with training in technical, cognitive, and
interpersonal skills (Greenberg & Klingensmith, 2015; Greenberg, Dombrowski, & Dimick,
2016; Stefanidis et al., 2016).
Ultimately, individualized education and assessment have been suggested to support not
only trainees and recent graduates, but career surgeons as well. In the current environment of
surgical training, many agencies work together to ensure adequacy of training and maintenance
of surgical skill, however these are still in premature development and will require research
focus in the future. One sustainable model is to teach the learner to be self-regulated and be able
to manage their learning at each stage of their career.
Self-Regulated Learning in Educational Psychology
Self-regulated learning (SRL) is purposeful, proactive, and learner-driven. SRL includes,
and is often confused with, self-directed learning, yet these two processes are distinctly different
in theory and practice. Often driven by an externally created curriculum, self-directed learning
follows a more linear path from instruction to acquisition of knowledge (M. H. Murad & Varkey,
2008). The broader educational theory of SRL includes the entire learning process, from goal
formation to the quality of knowledge demonstration, and how successful or unsuccessful goal
attainment shapes future learning experiences (Zimmerman & Schunk, 2001).
Metacognition and Self-Regulation
Both metacognition and self-regulation are fairly abstract concepts, but educating
individuals about these processes allows the individual to intentionally interact with their
environment, rather than simply reacting to maintain homeostasis. Taken together,

SRL IN SURGICAL EDUCATION 28

metacognition and self-regulation have been used extensively with any field predicated on goal-
directed behavior, including sports, education, and health (Clark & Zimmerman, 2014;
McInerney, 2011).
Metacognition. Metacognition can generally be thought of as thinking about one’s
thinking. It is the process of an individual recognizing and evaluating his or her personal
thoughts and actions to understand the intent of those thoughts and actions (Papaleontiou-Louca,
2014). For simplicity, metacognition can be considered a conscious, “cognitive” process of
deliberate actions.
Self-regulation. In general, the ability for one to moderate their personal thoughts,
emotions, and actions represents the concept of self-regulation. The study of self-regulation
includes investigation and theories as to why individuals elect to perform certain behaviors, how
those activities help achieve personal goals, and the manner in which outcomes of goal-directed
behavior modifies one’s own beliefs about oneself, and his or her selection of future goals and
actions. Counter to initial motivation theories which focused on reward and punishment,
including behavioral theories of classical (Pavlov, 1927), and operant (Skinner, 1963)
conditioning, self-regulation theories have emerged from a social cognitive perspective
(Zimmerman, 2000). Originally described by Bandura (1986), social cognitive theory recognizes
the dynamic interaction of personal, behavioral, and environmental processes. This theory
places the individual in control of his or her interaction with the environment, creating a sense of
personal agency to interpret thoughts and emotions, and act intentionally (Bandura, 2001).
Understanding self-regulation requires appreciation of the development of oneself, which
viewed through a social cognitive lens, involves belief formation through interaction with the
environment and observation of skilled models. When placed in a learning context, self-

SRL IN SURGICAL EDUCATION 29

regulation is a reciprocal interaction of self and the environment in a goal-directed fashion, that
is proactive and intentional, through metacognitive strategies of planning, monitoring and
adapting one’s thoughts, emotions, and actions.
Self-Regulated Learning
To examine self-regulation in the context of learning, it is important to first define
learning in various contexts. Natural learning episodes are spontaneous or self-initiated, are not
usually instructor driven, and therefore are often very different than those that are instructor led.
Boekaerts and Niemivirta (2000) recognize that natural learning goals are “cumulative,
sequential, and affectively charged, and always socially situated” (p. 418). Classroom learning is
different in that goals are usually teacher-defined, and must be evaluated and potentially
internalized by the student. Additionally, learning objectives defined by a curriculum often have
limited outcomes in that learning goals are met, not met, or deferred. Higher order goals are
created from within, and represent the ideal-self that drives goal-setting behaviors.
The identification and appraisal of a learning opportunity depends on environmental
factors presented to the learner, the previous motivational, emotional, and knowledge states of
that learner, as well as current and future ideals or goals the learner holds for him or herself.
Self-regulated learners display specific characteristics, processes, and strategies before, during
and after a learning task to enhance their learning effectiveness, regardless of whether goals are
externally or internally derived.
Characteristics of Self-Regulated Learners
Self-regulated learners actively seek knowledge and skill through metacognitive,
motivational, and behavioral processes. Through these processes, learners are “active
participants in their own learning” (Zimmerman, 1990, p. 4). Multiple models exist and have

SRL IN SURGICAL EDUCATION 30

been used to evaluate these processes within different contexts with a variety of learners.
Conceptually, all models seek to address “learning and motivational processes and to understand
their interactions” (Zimmerman, 2015, p. 543), and pertain to goal formation, pursuit,
modification, and achievement.
Although there are many ways to conceptualize SRL and define the characteristics of the
learner and their interaction with the environment, a systematic and operationalized definition of
SRL is helpful in investigating differences between learners, between different learning
environments, and when empirically examining the effect of learning interventions. Most
definitions systematically divide SRL into three cyclic phases of forethought, performance or
volition, and self-reflection, occurring before, during, and after a learning episode, respectively.
Forethought Phase
This phase attempts to explain processes that occur in preparation for goal-related
activities. This anticipatory phase is better developed in expert learners than with novice
learners (Zimmerman, 2015). Expert performers exhibit high levels of forethought before
beginning an activity, compared to novices who fail to do this systematically and are more
reactionary (Zimmerman, 2002). Classically defined, two fundamental processes occur in the
forethought phase proceeding a learning task: task analysis and self-motivational beliefs. Goal-
setting and strategic planning are both related more specifically to the task during this phase,
while goal orientation, outcome expectations, and intrinsic interest are elements that pertain more
to motivation and beliefs about oneself.
Goal-setting. Significant research has been done in the area of goal setting. Locke and
Latham (2002) emphasize the necessity of goals, which serve to purposefully direct effort and
attention. Over four decades of research on the relationship between goals, motivation, and

SRL IN SURGICAL EDUCATION 31

learning have shown that goals should be: a) set by or with active engagement from the learner;
b) task specific, explicitly stated and measurable; c) proximally attainable; and d) slightly
challenging and hierarchically ordered (Locke & Latham, 2002; Schunk, 2001; Zimmerman &
Schunk, 2001). Many of the other SRL behaviors overlap with goal-setting and therefore will be
discussed further in sections below.
Strategic planning. Strategic planning involves well thought out goals, strategic
processes that match the task and those goals, as well as determining the time and resources
necessary for success. Planning of preparation activities has repeatedly been associated with
high achievement (Kitsantas, 2002; Zumbrunn, Tadlock, & Roberts, 2011). This involves both
strategic planning of how to approach a task, what steps to take to ensure task completion and
quality, and appropriate allocation of resources (Sexton, Harris, & Graham, 1998).
The identification of appropriate learning strategies is an entire field of study beyond the
scope of this literature review (Elliot, McGregor, & Gable, 1999; Pintrich, 2000; Pintrich, Smith,
Garcia, & McKeachie, 1991). Briefly, learning strategies that are selected must match both the
learner, the task, and the available time and resources. Evidence suggests that different types of
knowledge, such as factual versus procedural, are acquired better through certain learning
strategies (Anderson et al., 2001). However, many learning strategies, such as elaboration and
integration, have been shown to globally improve learning and performance (Winne, 1996;
Zimmerman, 2000) as they increase information relevance and coherence.
Self-efficacy. During the forethought phase, evidence suggests improved performance
and learning for individuals who are able to evaluate a task and have a positive belief that they
are able to complete it with their current skill level (Bandura, 1986; Zimmerman, 2000). Self-
efficacy focuses on performance capabilities rather than an individual’s traits or characteristics.

SRL IN SURGICAL EDUCATION 32

Those with higher self-efficacy demonstrate greater motivation to pursue a goal and will persist
longer at challenging activities (Schunk & Usher, 2011; Schunk & Zimmerman, 1997).
Furthermore, self-efficacious learners tend to set higher personal goals, and are more likely to
use other SRL strategies (Zimmerman & Bandura, 1994).
Many factors determine a learner’s level of self-efficacy, including previous performance
on similar task, observing success or coping strategies in others, and positive emotional states
(Dembo & Seli, 2016 citing Bandura, 1977; Zimmerman, 2011). Identifying multiple sources of
positive reinforcement to pursue a goal outperformed those with only one source for self-efficacy
(Linnenbrink-Garcia & Patall, 2015). Ultimately, self-efficacy has been identified as a distinct
motivational construct, one that can be fostered, but also one that responds to previous
performances (Zimmerman, 2000).
Outcome expectations. The quality of outcome expectations increases with experience,
and involves one’s ability to predict multiple possible outcomes and the ability to provide
contingency plans to still accomplish desired goals (Bandura, 1997; Schunk 1984). The product
of this SRL is more challenging task choice, longer persistence and effort toward a goal, and
higher skill acquisition.
An interesting aspect of outcome expectation theory is how it affects satisfaction with
goal attainment. Wigfield, Tonks, and Klauda (2009) provide a detailed description of the
relationship between outcome expectations and personal value or satisfaction in their discussion
of expectancy value theory. This model combines an outcome expectation, one’s self-efficacy of
actually achieving that outcome, the value an individual attributes to goal attainment and the
emotional response generated (Pintrich & DeGroot, 1990). One of the ways that outcome
expectations affect learning and motivation is in the level of satisfaction or contempt an

SRL IN SURGICAL EDUCATION 33

individual feels with inadequate goal attainment. Those with higher end goals are dissatisfied
with less discordance, meaning they must be very close to achieving a goal to actually be
satisfied with the result. As one might expect, this becomes an upward propagating effect of
pushing high achievers to achieve more. Value expectancy theory is a major area of research
that has linked cognitive beliefs of achievement to motivation, effort, persistence, achievement
and performance (Pintrich & DeGroot, 1990; Wigfield, Tonks, & Klauda, 2009; Zimmerman &
Kitsantas, 1999).
Intrinsic interest. Often incorrectly used interchangeably with motivation, intrinsic
interest as a SRL factor is better thought of as value. Schunk, Meece and Pitrich (2012) stress
that the value of an activity is in relation to the individual and is always context specific in that
value changes based upon the presence of other activities. As discussed with outcome
expectations, value expectancy theory underscores that the value of an item is not only
dependent upon predictive factors and anticipatory gains based upon the outcome, but also in the
intrinsic interest of task completion (Pintrich & DeGroot, 1990). Consistent with a social
cognitive theory, higher value attributed to any of these aspects of a task is associated with a
reciprocal increase in the other emotional, behavioral, or cognitive components.
Viewing learning activities and goals associated with them as having high value increases
learning, achievement, self-efficacy, and positive affectivity (Gottfried, 1990; Lepper et al.,
2005). Furthermore, higher intrinsic interest is associated with higher cognitive strategic use and
other SRL behaviors (Pintrich & DeGroot, 1990). In contrast, extrinsic interest is often
described as assigning external value to an activity. If this external value is internalized and
viewed as consistent with an individual’s own goals, both intrinsic and extrinsic interest can be

SRL IN SURGICAL EDUCATION 34

compatible with a sense of autonomy. Therefore, if external rewards are internally reconciled,
they are not always detrimental to learning (Linnenbrink-Garcia & Patall, 2015).
Goal orientation. It is important to define or at least recognize goal-orientation during
goal-setting to determine the relationship of the goal to the intended process and outcome. Goal
orientation is shaped by the learning context, and personal beliefs about the outcome
(Linnenbrink-Garcia & Patall, 2015). Broadly speaking, placing goal orientation into a binary
categorization of mastery versus performance orientation is used within the context of
achievement goal theory (Dweck & Leggett, 1988) that seeks to understand individual
motivation toward goal pursuit. The effect of each goal orientation is not linear with
achievement goal theory as it includes an individual’s emotional motivation state of avoidance
versus an approach state (Linnenbrink-Garcia et al., 2012).
An individual with a mastery goal orientation thinks, acts and reacts to learning activities
and goals in a manner that focuses on developing competency (Elliot & Church, 1997). The
cognitive and learning strategies selected for a mastery goal orientation seek to understand the
process and develop intentional habits. Individuals with a mastery orientation learn as much as
possible for the purpose of self-improvement, irrespective of the performance of others.
An orientation toward learning in which outperforming others is a major concern is often
described as a performance oriented goal. The learner focuses attention and effort toward
demonstration of competence, or at least meeting some external standard. The benefit or
impairment of a performance orientation depends upon the type of task, with evidence to suggest
worse delayed performance when learning a complex task with a performance goal orientation
(Sitzmann & Ely, 2011). The connection between goal orientation and SRL behaviors, such as

SRL IN SURGICAL EDUCATION 35

monitoring, is complex and remains extremely context dependent (Boekaerts, Pintrich, &
Zeidner, 2000).
Performance Phase
The learner’s beliefs regarding their own ability, learning, and motivation are shaped by
and in turn impact the analysis of task specific components and expectations. Clearly identifying
one’s self-motivational beliefs and analyzing a task enables students to subsequently deploy task-
specific learning strategies and self-monitoring strategies during the performance phase of a
learning task (Zimmerman, 2002; 2015). The performance phase of a learning task involves
processes related to how a learner controls his or her attention and actions, as well as key aspects
of self-monitoring, which are classically divided into two major classes of self-control and self-
observation (Zimmerman, 2000). These are considered conscious thought processes or
strategies, and actions under volitional control that “seek to optimize learning efforts”
(Zimmerman, 2015, p. 542).
Use of task strategies. Self-control SRL strategies are processes that are planned and
executed during learning. The planning of strategic use was discussed in the forethought SRL
phase, and is dependent upon self-efficacy and outcome expectations. The execution of these
planned strategies, and the ability to deviate from the initial plan based upon progress and
premeditated contingencies are self-control strategies (Sitzmann & Ely, 2011). Persistence to
continue with a strategic plan enables continued effort toward goals despite failure to progress or
other emotional states (Elliot, McGregor & Gable, 1999), and increases the likelihood of success.
Imagery. Imagery is mental visualization of a learning or performance process, with
subsequent completion of step-by-step actions. Planning learning through an organized
structure, such as formulating an outline or ordered steps, has been shown to improve learning

SRL IN SURGICAL EDUCATION 36

and final products in many fields including writing, sports, and graphic design (Zimmerman &
Risemberg, 1997). This cognitive strategy includes two phases, the pre-emptive mental image or
structure, followed by recall or reference of that scaffold during task completion.
Self-instruction. Also called self-talk, self-instruction is similar to providing cognitive,
behavioral and emotional coaching to oneself. Accomplished through an internal or external
dialogue, this strategy utilizes reorientation and self-motivation to continue at a task, and
includes activation of strategies for emotional control (Sitzmann & Ely, 2011). This self-talk
maintains attention on the learning task, can reinforce emotional control strategies, and can
highlight irrational thinking patterns (Dembo & Seli, 2016). Emotional control, often through
relaxation exercises and self-encouragement, limits disruptive effects of anxiety and negative
emotional states (Kanfer, Ackerman, & Heggestad, 1996; Pintrich, 2000). Regulation of
emotion during learning improves performance by maintaining focus on the current tasks rather
than dividing attention toward unrelated concerns (Keith & Frese, 2005; Porath & Bateman,
2006).
Attention focusing. Often described from a cognitive lens, attention focusing includes
cognitive strategies of task prioritization and time management. Specifically, attention is the
cognitive ability to maintain focus and concentration during training (Zimmerman, 2015). From
a resource allocation framework, this SRL behavior is a trainee’s ability to use metacognitive
strategies to determine value and allocate appropriate cognitive resources to the task at hand, and
manage off-task needs and one’s own emotional state (Kanfer, Ackerman, & Heggestad, 1996).
Off-task items are considered external and internal distractors, and management of these items
includes pre-emptive awareness and in the moment monitoring (Dembo & Seli, 2016).

SRL IN SURGICAL EDUCATION 37

Self-recording. During a learning activity, learners use metacognitive awareness to self-
monitor and execute SRL strategies. Self-awareness of one’s learning events and performance is
termed self-recording (Sitzmann & Ely, 2011). The development of self-recording skills often
requires objective measures and recordings, including diaries, progress notes, and confirmation
of findings with external observers. Accurate recording and self-assessment enables growth in
future cycles of learning because it reveals current knowledge and directs focus to known
limitations (Schunk, 2001).
Metacognitive monitoring. Moving beyond the collecting of data, metacognitive
monitoring includes appraisal of progress and evaluation of current strategies for effectiveness.
This SRL skill includes the ability to detect a gap in current performance with ideal performance,
and adjust effort or strategy to narrow the gap (Sitzmann & Ely, 2011). During the performance
phase, self-regulated learners use behavioral and metacognitive strategies to monitor attention
and progress toward goals.
Reflection Phase
The results of continuous cognitive and motivational engagement in the performance
phase is evaluated against initial criteria of the forethought phase during the final reflection
phase. In the regulatory phase that occurs after a learning task, SRL behaviors include self-
judgment and self-reaction to provide feedback of performance to goals or targets to affect future
learning activities. Self-reflection is evaluation of past performance, and influences forethought
in subsequent activities (Zimmerman, 2002).
Self-judgment. Self-judgment involves assessment of one’s current performance
compared to a specific standard, preferably predetermined in goal-setting, with inferences
regarding causal attributions. In this self-evaluation phase, a learner recognizes the final

SRL IN SURGICAL EDUCATION 38

outcome of his or her efforts compared to a set standard, criterion, or his or her own previous
performance (Zimmerman, 1998). Self-evaluation affects a learner’s emotional state, and can
lower self-satisfaction unless the learner has adequate self-efficacy about their ability to adapt
SRL processes (Bandura, 2001).
Causal attributions. Learners hold certain beliefs regarding the cause of their
performance or outcome. These can be externally or internally oriented causes, as well as those
within the control of the learner or out of their control, and perceived as stable or not stable
(Linnenbrink-Garcia & Patall, 2015). This is especially important if the outcome is unexpected
or negative. Interpreting the cause of an outcome as something that is within the learner’s
control, and that can be changed or modified can have positive effects on a learner’s motivation
in future learning activities. Understanding a learner’s attribution assumptions helps to
understand how individuals respond differently to the same outcome (Sexton, Harris & Graham,
1998).
Self-reaction. Learners’ behavioral and emotional reactions to their performance is
termed self-reaction. How they respond to their achievements or inability to reach a goal affects
how they approach future learning events and goal setting behaviors. After a learning episode,
learners exhibit either positive or negative emotions that may affect their identity as a learner,
and self-efficacy for similar events in the future (Zimmerman, 2000).
Adaptive inferences and help seeking. Based upon the outcome of a learning endeavor
and the causes a learner attributes to that outcome, the learner will exhibit a behavioral and
cognitive response, which can be adaptive or maladaptive. For example, after attaining a goal,
the learner may set another higher goal; if unsuccessful, the learner may develop another strategy
for the next attempt, or abort effort toward the goal (Zimmerman, 2001). This SRL behavior is

SRL IN SURGICAL EDUCATION 39

related to other SRL skills. More accurate self-monitoring and causal attribution judgments will
lead to more appropriate and specific help seeking behaviors (Karabenick & Dembo, 2011) and
identify modifications needed for future learning activities. Ultimately, multiple processes are
involved in effective self-regulated learning behaviors, many overlap and are affected by
elements in other phases of the learning cycle.
Measuring SRL
Traditional instruments for measuring SRL included questionnaires and interviews, but
developments in educational psychology and other fields have integrated technology and in-task
measurements of SRL strategies and behaviors. Most research has been conducted with limited
options for assessing SRL constructs, restricted to behaviors that can be observed, or those
reported in learner and teacher surveys. Newer research designs have included external
evaluation of learning materials, as well as think aloud protocols or during-task interviews.
Traditional psychometric surveys. Many validated instruments are used in whole as
measures of SRL. The Motivated Strategies for Learning Questionnaire (MSLQ) is one often
used as the sole measure of SRL, but only measures a few elements (Pintrich et al., 1991). Some
of these only assess specific elements of SRL, such as planning, monitoring, self-efficacy, or
motivation (Sitzmann & Ely, 2011). Metacognition scales often cover more elements, including
planning, monitoring, and attention together, rather than as separate constructs. Importantly,
none of these metacognition scales assess all aspects of self-regulation. Thus, there is a
disconnect between theory and measurement that limits the field’s understanding of the role of
metacognition in self-regulated learning. The Learning and Study Strategies Inventory (LASSI;
Weinstein, Palmer, & Schulte,1987) assesses student’s self-reported strategies for studying. It
does include several subscales that assess elements of motivation, regulation, skill, and

SRL IN SURGICAL EDUCATION 40

metacognition. The Self-Regulated Learning Interview Scale (SRLIS; Zimmerman & Martinez-
Pons, 1988) assesses students’ motivation, metacognition, and learning strategies. Several
constructs are addressed through subscales, and all are answered via self-report.
Importantly, none of these scales or instruments correlate perfectly with learning and
performance for all level of learners (Nandagopal & Ericsson, 2012). This suggests that
quantitative measurements may not capture all elements of SRL, or that the measurement tools
should be context and learner dependent. Although validated metrics are still utilized, more
recent research has proposed alternate options for measuring SRL in a diverse population of
learners.
E-learning. Use of computer programs and detailed informatics about the users has
enabled the use of sophisticated and more objective techniques for measuring SRL. These
strategies attempt to analyze learners’ use of SRL strategies by observing actual learning
activities. Tested measures include use of trace logs for capturing electronic clicks and time on
task (Zimmerman & Schunk, 2011). Caution must be taken in the interpretation of computer
learning environments, as the relationship of time on task is not linear and should not be viewed
as such. Many of these programs measure the use of specific tools to assess SRL behaviors,
including the use of notes, indexes, concept maps, chats, and coaching (Zimmerman, 2008).
Importantly, these elements do not correlate with self-reports, either suggesting inaccurate
measurements or lack of self-awareness.
Microanalysis. A newer, alternate way of assessing learners’ intentions and strategies
involves non-delayed self-report. During a microanalysis, a researcher asks targeted open- and
closed-ended questions to assess predetermined areas of interest (Zimmerman, 2008). For
example, the researcher may ask how a learner is planning on approaching a problem, or what

SRL IN SURGICAL EDUCATION 41

they are thinking after they are given negative feedback. In this manner, the researcher can
collect both quantitative and qualitative data before, during, and after a learning experience. The
topics of focus are decided before the encounter, and the questions are designed to be brief and
acquire information without significant disruption of attention or interference with learning.
Although not without its own limitations, microanalysis methodologies have provided interesting
insights into the SRL behaviors, cognitive processes, and motivational sources in sports, music
and medicine (Cleary & Zimmerman, 2001; Cleary, Durning, & Artino, 2016). With this
technique, researchers have been able to determine significant differences in the thought and
action patterns of novices and experts in these fields.
Although there are multiple avenues to measure SRL behaviors, evidence suggests that
learners may not be accurate in their own awareness of SRL use. In order to accurately assess
the utilization of SRL strategies and the correlation of such behaviors with learning and
performance outcomes, it is important to select the assessment that targets the SRL behaviors of
interest. Due to the complexity of this topic, studies investigating SRL should thoroughly
describe their methodology and identify the benefits and limitations of their study designs.
Current Research of SRL Strategies in Surgical Education
Mainstream educational research has moved to the broader investigation of self-
regulatory processes and strategies in a holistic fashion to improve learning in traditional
classroom environments, yet surgical education is just beginning to explore individual
components of self-regulation, such as motivation, self-efficacy, goal orientation, self-
assessment, and metacognition with medical settings. The introduction of simulation into
surgical training has greatly expanded the conversation on SRL in surgical education, with a new
emphasis on initial demonstration of basic skill competency in the laboratory rather than the

SRL IN SURGICAL EDUCATION 42

operating room (Brydges et al., 2015). The majority of these curricula have viewed SRL as
learning alone, with few actually teaching SRL skills. This section will discuss the SRL
constructs that have been explored with surgical residents as the targeted learner population.
Individualized Learning Plans
While documentation of individual progress is required by the ACGME, only a few
programs have implemented formalized individual learning plans for surgical residents. These
educational programs revolve around the ACGME Core Competency of Practice Based Learning
and Improvement (PBLI). O’Conner and colleagues (2010) discuss a four-week course for
junior residents with the goal of improving quality improvement projects and the development of
an individual learning plan. Self-reported comfort with identifying learning needs was improved
after this curriculum was adapted from the Dreyfus Model of Skills Acquisition (Dreyfus, H. &
Dreyfus, S, 1986).
Another model instituted a learning portfolio for both junior and senior level residents,
and analyzed the topics identified as individual learning points (Webb & Merkley, 2011; 2012).
Juniors discussed foundational topics related to specific diseases, while senior level residents
focused on complications as learning points. These initial studies were followed with a review
of the portfolio entries for milestones related to the PBLI Core Competency and found seven
themes that address this competency (Webb et al., 2014). Overall, most individualized learning
plans that have been evaluated demonstrate resident acceptance and feasibility for integration
into a curriculum for the ACGME core competencies. If implemented within a structured
program, individualized learning plans have the potential for appropriate goal setting, progress
monitoring, resident self-awareness and critical reflection.

SRL IN SURGICAL EDUCATION 43

Forethought
Most of the research in the surgical education literature that includes elements of the
forethought phase of SRL targets goals. This includes goal setting, goal orientation, and
outcome expectations. The surgical literature about goal setting is concurrent with that published
in traditional education settings. The presence of goals improves learning and performance and
has been demonstrated with early laparoscopic skill acquisition (Madan, Harper, Taddeucci, &
Tichansky, 2008; Stefanidis, Acker, & Greene, 2010), and during daily hospital work (Chung &
Ahmed, 2007). Selection of appropriate goals also improves performance as demonstrated with
realistic performance goals during laparoscopic training, compared with inflated goals (Kishiki et
al., 2016).
Performance
Imagery. Mental models are often used by experts to hold an image or physical
arrangement temporarily in working memory (Zimmerman & Kitsantas, 1999). Often discussed
in sports and writing, the ability to accurately imagine and describe a mental model is a SRL that
has been tested in surgery. In a randomized control trial, the group of residents and faculty who
received mental training in visualization and self-talk performed better in task-specific
maneuvers on a simulated laparoscopic cholecystectomy (Immenroth et al., 2007). The two
comparison groups received (a) additional time on the simulator, with access to scientific articles
and a surgical atlas; (b) control without additional training.
The difference in the quality of mental imagery between faculty, resident, and novice was
tested with a video prediction model (Leff et al., 2017). After video review of a laparoscopic
cholecystectomy, faculty and residents were more likely to select the safest next operative
maneuver, and function neuroimaging demonstrated discordance when the wrong technique was

SRL IN SURGICAL EDUCATION 44

chosen, compared with novice acceptance of the outcome as correct. The ability to form a
mental representation of operative steps develops with experience, and the quality of this
imagery is correlated with improved performance (Nordin, Cumming, Vincent, & McGrory,
2007).
Self-assessment. Self-assessment has been a topic that has been studied recently in
surgical education. The ability to accurately self-assess is dependent on the type of skill being
assessed and the level of expertise of the one being assessed. For technical skills, self-
assessment of skill during the task improves with experience (de Blacam, O'Keeffe, Nugent,
Doherty, & Traynor, 2012; Moorthy et al., 2006). Vyasa et al. (2017) showed that performance
can be improved with video review of one’s own performance or an expert performance.
However improvement of self-assessment occurred only with review of expert videos, and not
with self-review.
Reflection
Studies suggest that resident self-evaluation may be inaccurate in the clinical and
interpersonal domain (Brewster et al., 2008; Lipsett, Harris, & Downing, 2011; Vyasa et al.,
2017; Watson et al., 2017). Factors that have been shown to create a discrepancy between self-
and expert-assessment of non-technical skills include performance level, years in training, type
of training, and gender (Lipsett et al., 2011; Vyasa et al., 2017; Watson et al., 2017). There are
two main components that have been examined in surgical literature regarding self-reflection: (a)
self-evaluation or judgment, meaning retrospective evaluation of one’s current abilities and the
source of skills or inadequacies; (b) self-reaction, one’s own satisfaction and reactive behaviors
of performance.

SRL IN SURGICAL EDUCATION 45

Self-judgment. Reflective exercises on one’s knowledge and skill level, or performance
is considered self-judgment. This retrospective evaluation of learning has been examined in
surgical residents in regards to the ACGME Core Competencies. Higher performing residents
tend to be more critical of their performance than attending faculty, while lower performing
residents demonstrate a lack of insight into their knowledge and skill deficits (Gow, 2013;
Watson et al., 2017). This discrepancy of assessment is consistent with educational psychology
literature, which has shown a relationship between self-satisfaction and achievement-goal
discrepancies that depend on the learner’s level of performance (Bandura, 2001).
The type of learning reported by residents also changes with training level (Monaghan et
al., 2012). In regard to the six Core Competencies, early learners report most of their learning is
occurring with patient care and knowledge, while professionalism was not a prominent area until
residents became more senior. Interestingly, Monaghan et al. concluded that “self-learning is not
a significant source of learning, and the majority of the learning is experiential,” (p. 618) taking
place on the wards or in the operating room.
Self-reaction. The effect of understanding and acknowledging one’s level of success of
learning can be improved when this SRL is deliberately taught. The addition of post-operative
video-based coaching increased the number of teaching points, both resident-initiated and overall
(Hu et al., 2016). This study also promoted more complex topics during discussion and learners
set more future learning goals.
Summary
A critical search of the literature reveals several SRL strategies that improve learning and
performance. Promotion of these strategies for learners should be based upon context specific
research, and newer methods of investigation can be utilized after creating a framework for SRL.

SRL IN SURGICAL EDUCATION 46

Advancing SRL research in surgical education begins with a clear definition of SRL, based upon
the following strategies from empirical research in other educational settings.
Individual components of SRL have been researched with surgical residents, mainly: goal
setting, the accuracy of self-assessment, and the use of self-reflective practices. Most research
has examined the reception of these interventions or short term impact, with little focus on
learning outcomes or learning behaviors. For this reason, it is imperative that a framework of
self-regulation exists for surgical trainees as a specific population of learners and works toward
ensuring a profession of lifelong learners that are highly skilled self-regulated learners.

SRL IN SURGICAL EDUCATION 47

CHAPTER THREE: METHODS
Current surgical education often challenges residents with an unstructured curriculum and
minimal instructional guidance, resulting in most reporting dissatisfaction with their current
study habits (Yeh et al., 2013). Although surgical residents are asked to be self-regulated
learners, they may not be effectively using these techniques and need guidance from faculty and
staff to assist in developing learning strategies and self-regulation learning (SRL) skills. The
current evidence suggests that SRL skills do not develop in medical school (Lucieer et al., 2016;
Murad et al., 2010), and even practicing surgeons have difficulty identifying and directing their
own learning needs (Gagliardi et al., 2009). Thus, the evidence confirms that SRL skills need to
be deliberately taught and practiced to foster individual mastery (Zimmerman, 2015).
The evaluation and development of SRL skills should be domain specific (Sitzmann & Ely,
2011), yet little research has examined SRL in its entirety in medical education, let alone surgical
education. The purpose of this study was to build a comprehensive framework of SRL in
surgical education and determine which attributes are most valued by surgical educators. This
was important because before one can develop a process to teach and assess SRL skills,
educators must agree on which skills are important and have a common language to describe
them.
Research Question
What evidence-based self-regulated learning attributes and strategies do experts in surgical
education deem necessary for residents during surgical training?
Methodology
This mixed-method study utilized a Delphi survey technique as the methodologic
framework. The Delphi method (Dalkey & Helmer, 1963) is a consensus-building technique that

SRL IN SURGICAL EDUCATION 48

narrows the opinions of a panel of experts through a series of sequential surveys, occurring over
the course of several weeks. The panel of experts remains the same for all surveys. The
responses from each survey round are analyzed and serve as the foundation for the subsequent
round. This systematic process allows for the emergence of a group consensus, and is well
suited for research questions in which no single correct answer exists, and the goal is to gather a
collective opinion of informed individuals (McKenna, 1994). The Delphi method has been used
in social sciences and educational policy, to set clinical and research priorities, and to help
determine health and education resource utilization (Hsu & Sanford, 2007; Keeny, Hasson, &
McKenna, 2010; McKenna, 1994; Stefanidis et al., 2015).
As a mixed-methods framework, the Delphi survey technique utilizes both qualitative and
quantitative sources of data. The flexibility of this methodologic framework holds to the core
theoretical underpinnings of its design; to allow a group of individuals to explore a complex
topic, and discuss it until a statistical consensus is reached (Brady, 2015). Parts of the Delphi
method use quantitative data collection to determine saturation or sufficient levels of consensus,
and qualitative data is used in a different fashion to explore and understand the complexity of the
topic. The flexibility of this mixed-methods design is often necessary in moving a group toward
consensus (Habibi, Sarafrazi, & Izadyar, 2015).
Although the traditional Delphi method uses an open-ended questionnaire to generate a
list of opinion statements on a topic (Dalkey & Helmer, 1963), modified versions of the Delphi
method are also well described in the literature, specifically those in which the initial round of
opinion statements are gathered from either an extensive literature review of another field, or by
focus groups and interviews with experts (Fink et al., 1984; Keeny, Hasson, & McKenna, 2010).
This study utilized a modified first round, constructed from an extensive review of SRL research

SRL IN SURGICAL EDUCATION 49

in traditional educational settings (Zimmerman & Kitsantas, 2014; Zumbrunn, Tadlock, &
Roberts, 2011), professional environments (Sitzmann & Ely, 2011), and those available in
medical literature (Brydges et al., 2015; Murad & Varkey, 2008; Woods, Mylopoulos, &
Brydges, 2011). Evidence-based SRL strategies appropriate to adult learning were selected.
Similar to a traditional Delphi approach, a modified Delphi utilizes qualitative methods to
ensure each iteration of the survey incorporates written comments from the panel. Additionally,
the Delphi survey technique offers several key advantages, outlined below, compared to
traditional consensus methods, such as a focus group and the nominal group technique (Fink et
al., 1984; Jones & Hunter, 1995):
Technical Considerations
Traditional group consensus methods can be costly and time intensive if experts are
required to be in the same location for an extended period of time (Fink et al., 1984). The Delphi
technique offers the unique benefit that panelists can respond within a short survey time window,
without travel costs, and without significant interference into the professional schedules of the
panelists (Hsu & Sanford, 2007; Keeny, Hasson, & McKenna, 2010).
Anonymity
A benefit of the fact that respondents are not in direct communication is the anonymity
that is provided, particularly if the topic of conversation is emotionally charged or contains areas
of contention. Initially, individuals are able to answer the first round in an unbiased fashion, and
the responses may provide more immediate reactions to a topic. Subsequent rounds in a Delphi
provide the panelist information about the collective opinions of the group, but in a way that the
conversation is not dominated by only a few individuals, which is often a flaw of other
consensus methods (Keeny, Hasson, & McKenna, 2010). This subtle introduction of group

SRL IN SURGICAL EDUCATION 50

normative information promotes delayed consensus, while maintaining initial anonymity and
non-conformity of opinions.
Noise Reduction
The study design of a Delphi technique captures the collective knowledge of a talented
group of individuals and focuses their communication around key issues through a multi-stage
process. The dialogue is considered “clean” (McKenna, 1994) because the individuals are not in
direct communication, and therefore the discussion can often be more focused. This is
particularly helpful when an issue may otherwise have too many moving parts, and this
separation may aid in the removal of static or background noise created from extraneous
conversations unrelated to the specific research topic.
Sample and Population
This study used a purposeful sample of experts to develop a framework for SRL in
surgical education. Keeney, Hasson, and McKenna (2010) make specific recommendations
when selecting experts in a Delphi study, including those with practical experience, as well as
those that would be considered members of a professional or specialty society. Therefore,
experts were selected for the panel if they were recognized on a national level as experts in
education within the field of surgery, beginning with members of the national Association for
Surgical Educators and the Association for Program Directors in Surgery. The panel therefore
consisted of individuals who had dedicated their careers to the advancement of surgical
education and were active on a national level, thereby understanding the current environmental
demands.
Forty-two individuals were identified for the expert panel, with an aim to select enough
individuals to capture a variety of teaching environments. No minimum panel size is

SRL IN SURGICAL EDUCATION 51

recommended in the literature, but panels as small as 15-20 members have been able to reach
consensus when the responses were expected to be homogeneous or if a modified initial round
was used (Keeny, Hasson, & McKenna, 2010; McKenna, 1994), as done in this study. Only the
individuals from the expert panel who successfully complete the round one survey were invited
to complete the round two survey.
Data Collection
Panelists were contacted via email, with a brief introduction to the purpose of the study,
the Delphi method, the research question, and a link to participate. Surveys were distributed and
collected electronically, using Qualtrics online software (Qualtrics, Provo, UT, 2018).
Quantitative data from Round One was collected using a slider function from zero to 100, with
the selected whole-number integer displayed. Round Two quantitative data was collected in
binary form: agree, disagree. Qualitative data was collected via free response forms embedded
in Qualtrics, and were transferred to excel without modification. Round One remained open for
three weeks, and Round Two remained open for two weeks. Reminder emails were sent weekly
for incomplete and unopened surveys.
Validity and Reliability
Qualitative validity and panelist confirmation of accurate data analysis was inherent in
the Delphi survey design, as Round Two served as an opportunity for the panelists who provided
the initial comments in Round One to confirm the accuracy of the new items in Round Two.
This form of validity, use of member checking for accuracy of interpretation, was a specific
strength of choosing the Delphi method for this study (Habibi et al., 2015). Heavy use of
quotations was another strategy for methodologic qualitative validity, as recommended by

SRL IN SURGICAL EDUCATION 52

Creswell (2014), that was used during the final analysis of themes that persisted through both
Round One and Round Two.
Additional factors reported in the literature to increase the validity of the Delphi
technique include retaining panel members between rounds and limiting the duration of time
between rounds (Landeta, 2006). For the present study, only three individuals that participated
in Round One did not participate in Round Two, representing a drop-out rate of eight percent.
Additionally, the time between completion of Round One data collection and release of Round
Two was five weeks, far less than the suggested maximum interval of three months (Landeta,
2006). Methodologic reliability was ensured with adherence to the six-step process of qualitative
analysis recommended by Creswell (2014).
Instrumentation
Given that SRL has been extensively researched in other fields (Sitzmann & Ely, 2011;
Zimmerman, 2015), this study used a modified Delphi technique that converts the traditional first
round of purely open-ended questions into one in which the expert respondents were asked to
evaluate a set of items pulled from an extensive literature review of published material in other
fields, and describe how each item applied to surgical education. The Round Two survey was
constructed after analysis of Round One, and is discussed in order below. Figure 1 represents a
visual mapping of the study design.

SRL IN SURGICAL EDUCATION 53

Figure 1. Delphi study structure for data collection and analysis. Building a framework for self-
regulated learning in surgical education with a two-round modified Delphi survey.
Round One Instrument
SRL attributes and skills were identified from an extensive literature review, holding to a
social cognitive model of SRL according to Bandura (1986) and Zimmerman (2000, 2015). A
total of 25 individual items were identified, with attempts to reduce redundancy while capturing
the breadth of literature (see Table 1). The panel was asked to evaluate each SRL characteristic
for relevancy in surgical residency, and rate level of importance on a scale from zero to 100
(Appendix A). Space was provided for written comments for each item. Additionally,
respondents were also asked to identify any aspects of SRL in surgical residency that may not
have been included.

SRL IN SURGICAL EDUCATION 54

Table 1.
Round One Self-Regulated Learning Constructs
Utilize Feedback: Receive, analyze and incorporate feedback
Goal Setting: Establish their own learning goals
Goal Orientation: Choose goals based upon mastery of the content rather an a need to demonstrate
ability
Value Appraisal: Make choices based upon their interests, as well as the perceived costs and benefits
of learning
Planning: Analyze and develop a plan to accomplish their learning goals
Self-Efficacy: Have confidence in their ability to complete a task
Strategic Use: Use learning strategies to reach their learning goals or complete a task (such as
checklists, mnemonics, or reminders)
Outcome Expectations: Visualize and articulate the final product of their efforts
Environmental Structuring: Choose settings that enhance their learning
Time Management: Prioritize and manage competing demands
Attention: Concentrate and direct their focus
Emotional Control: Monitor and regulate their emotions
Use of Models: Learn by watching people who are more skilled than them
Help-Seeking: Know when and how to ask others for help
Motivation: Know what motivates them to pursue a learning goal
Metacognition: Monitor their thought processes and how they are affected by their own actions
Attribution Judgments: Identify reasons for good or bad outcomes
Monitoring: Track their progress toward goals, and adapt as necessary
Performance Calibration: Accurately match their confidence with performance
Self-Reaction: Accept or forfeit rewards based upon success, progress, or need to delay gratification
Innovative: Be open to using new technology or techniques
Evidence-Based: Uses evidence-based medicine to inform practice
Collaboration: Able to learn and grow when working with others
Effort: Work hard, and persist through adversity
Efficiency: Work smart to accomplish goals in a timely manner

SRL IN SURGICAL EDUCATION 55

Round One Data Analysis
Data from completed surveys were de-identified. Quantitative analysis was used to
provide a general sense of importance and agreement for each SRL stem by calculating the
median and range. Consistent to using the mixed-method Delphi technique framework, these
quantitative measures were not taken in isolation, but rather were used in conjunction with
qualitative analysis (Brady, 2015).
The six-step model for analysis of qualitative data outlined by Creswell (2014) was
followed for Round One. All data was organized and prepared for analysis, including de-
identification of the participant, but maintaining connection between quantitative scoring and
written comments to ensure appropriate interpretation during analysis. Next, all scores and
comments were reviewed in entirety to gain a general sense of the responses and to begin to
identify themes. Space was provided after each of the 25 constructs for expert comments, and
their responses were systematically coded into the following six categories: (1) affirmation of
importance (A); (2) suggestions for applicability to surgical training (+); (3) critiques regarding
relevance to surgical training (-); (4) those pertaining to other SRL stems (O), (5) queries
relaying confusion about the term or definition (Q); and (6) statements not applicable to SRL
(N/A). In addition to the 25 constructs, participants were asked to provide any additional
attributes of SRL that were not previously listed. The open form responses were also coded into
the above six categories. If the open responses did not align with a construct within any of the
six categories, then it was coded as a novel construct (ON). Comments that were coded as
affirmations (A), suggestions (+), critiques (-) and queries (Q) were used to modify the original
25 construct definitions shown in Table 2.

SRL IN SURGICAL EDUCATION 56

Table 2.

Modified Round One Self-Regulated Learning Constructs
Utilize
Feedback
- actively seek, analyze, and incorporate feedback from multiple sources, including peers
- evaluate the feedback quality and determine its relevancy
Goal Setting
- establish specific, achievable goals to guide learning toward a known standard
- utilize mentors to refine targets based upon current performance and trajectory
Goal
Orientation
- adapt learning goals based upon situational factors
- hold a mastery orientation or growth mindset during initial skill development
- switch to a performance orientation when needing to demonstrate ability
Value
Appraisal
- choose learning tasks based upon a balance between intrinsic interest or motivation,
and competing situational needs
Strategic
Planning
- after identifying learning goals, develop specific steps for achievement
- choose different learning techniques or strategies based upon the knowledge level of
the objective (declarative, conceptual, procedural, etc) Bloom's Taxonomy of learning
- choose appropriate learning strategies (mnemonics, checklists)
Self-Efficacy
- understand that to complete a task, success is based upon both student current skill
level and task specific factors
- belief in one's capabilities to reach a goal if given sufficient effort
Strategic Use
- adhere or adapt proposed learning strategies based upon effectiveness
- use self-instruction or self-talk to verbalize key points or strategies for task completion,
such as repetitive checkpoints, mnemonics
Outcome
Expectations
- visualize and articulate what success looks like
- understand how their own and others' actions can influence the success of a goal
Environmental
Structuring
- identify, select, and adapt learning environments to increase goal attainment
Time
Management
- aware of task requirements and time restraints
- manage competing demands
- prioritize and adapt learning goals based upon these demands
Attention
Control
- concentrate and direct focus to the most pressing task
- ability to limit cognitive and behavioral resources to prioritized tasks
Emotional
Control
- aware of and compensates for positive and negative emotional responses
- able to regulate emotional response to maintain and focus performance and learning
Use of Models

- finds, observes, and seeks to learn coping strategies from more competent peers or
experts
- develops mental models of desired performance
Help-Seeking

- know when and how to identify others to correct specific deficiencies
- differentiate help based upon the demands of the task

SRL IN SURGICAL EDUCATION 57

Table 2.

The newly-defined constructs and definitions from Table 2 were organized into seven
themes or domains, listed and defined in Table 3. Table 4 shows the newly-defined constructs
listed under each of the seven domains. The domains and constructs listed in Table 4 were used
in Round Two data collection.

Modified Round One Self-Regulated Learning Constructs (Continued)
Motivation

- aware of own motivation toward a learning goal
- utilizes techniques to remain motivated through adversity
Metacognition - aware of thinking patterns and how they affect behavior and learning
Causal
Judgments
- appropriately attributes errors or success to internal and external factors
- Assesses ability to change unhelpful components, or need to personally adapt if
barriers to goal attainment are unmalleable
Monitoring

- observe and track one's own performance toward a goal
- accurate self-assessment and comparison to specified criteria
Performance
Calibration
- matches self-confidence to actual capabilities
Adaptive
Reactions
- sustains or modifies current strategies or behaviors based upon progress toward a goal
Innovative

- evaluates new technologies and techniques
- adapts practice when necessary to remain current
Evidence-
Based
- seeks published empiric or consensus data to inform current practice
- interprets relevancy of research to current situation
Collaboration

- understands one's strengths and weaknesses in a group learning environment
- able to learn and grow individually when working with others
Effort

- works hard and persists through adversity in pursuit of a longer-term goal
- believes that effort can influence the outcome of a learning situation
Efficiency

- selects strategies to accomplish goals in a timely manner
- participates in purposeful practice

SRL IN SURGICAL EDUCATION 58

Table 3.
Domains of Self-Regulated Learning in Surgical Education
Domain Definition
Self-Awareness
Awareness of one's own skills and deficits, as well as their cognitive,
emotional, and motivational states
Task-Analysis
Appraisal and development of one's goals for an activity, as well as
its learning value and expected outcomes
Situational Awareness
Understands and appreciates concurrent responsibilities and patient
care, resource and time availability, and interpersonal aspects of
their environment
Strategic Planning
Incorporation of personal, task and environmental factors to define
goal orientation, mental and physical resource allocation, self-
preparation, and process planning
Progress Evaluation
Recognizes discrepancies between goals and current state, and
appraises the situation to determine the cause
Manage Learning and
Performance
Utilizes individual and systemic strategies for improvement
Goal Attainment and
Refinement
Uses specific short-term goals to develop mid-term goals, which
ultimately build upon long-term goals or a main driving principle.

SRL IN SURGICAL EDUCATION 59

Table 4.
Organization of Domains and Subconstructs
Domain Sub-Construct Definition
Self-Awareness
Awareness of one's own skills and deficits, as well as their
cognitive, emotional, and motivational states

Self-awareness
(knowledge/skill level)
Recognizes current knowledge and skill level, based upon
self-assessment and external evaluation/measures

Self-awareness
(of limitations)
Cognizant of personal limitations

Strategic planning
(learning strategies)
Aware of personal learning strategies that have been
previously successful

Metacognitive
(awareness)
Metacognitively aware of personal patterns of thought,
emotion, and motivation, and mindful of their effects

Task-Analysis
Appraisal and development of one's goals for an activity,
as well as its learning value and expected outcomes

Self-efficacy Appraises task level and need for guidance

Goal setting
Establishes specific, achievable, and progressive individual
goals to guide learning, in alignment with known standards,
such as established curricula or national mandates

Help-seeking
Utilizes mentors to refine goals based upon current
performance and trajectory

Outcome expectations Visualizes and articulates success

Resourcefulness
Seeks published empiric evidence or consensus data to
inform current practice when relevant

Value appraisal
Understands the costs and benefits of task engagement,
balance intrinsic interest and external demand

Situational
Awareness

Understands and appreciates concurrent responsibilities
and patient care, resource and time availability, and
interpersonal aspects of their environment

Understands priorities Prioritization of competing demands

Understands priorities Aware of time constraints

Resourcefulness Knows or researches available resources

Self-awareness
(in a team)
Understands team roles, strengths/weaknesses, and
dynamic

SRL IN SURGICAL EDUCATION 60

Table 4.
Organization of Domains and Subconstructs (Continued)
Domain Sub-Construct Definition
Strategic Planning
Incorporation of personal, task and environmental factors to
define goal orientation, mental and physical resource
allocation, self-preparation, and process planning

Strategic planning
(learning strategies)
Develops specific steps for goal achievement, seeking
guidance when necessary

Strategic planning
(learning strategies)
Identifies learning and performance strategies, such as self-
instruction, checkpoints, mnemonics, or teaching

Goal orientation
Alternates between a performance goal orientation if needing to
demonstrate ability, and a mastery or growth orientation during
skill/construct development

Environmental
structuring
Actively shapes the learning environment by communicating
goals, and requesting autonomy, teaching, and/or feedback

Progress
Evaluation

Recognizes discrepancies between goals and current state,
and appraises the situation to determine the cause

Monitoring Observes and tracks performance toward a goal

Metacognitive
(awareness)
Identifies gaps in knowledge, skill, or motivation

Causal attributions
Appropriately attributes errors or success to internal and
external factors

Self-reaction Incorporates success and failures into future goal generation

Self-reaction
Determines the need to adhere or adapt current learning
strategies or goals, based upon effectiveness and need for
deeper learning

Manage Learning
and Performance
Utilizes individual and systemic strategies for improvement

Resourcefulness Seeks and uses available resources

Help seeking
Actively seeks assessment and feedback from multiple
sources, including peers

Self-awareness
Evaluates quality and relevancy of feedback, and incorporates
appropriately

Self-awareness
Differentiates help based upon the demands of the task and
specific deficits

Strategic use
Finds, observes, and learns from peers or experts to develop
appropriate mental and motor skills and coping strategies

Self-assessment
Evaluates current knowledge or processes, and adapts practice
when necessary

Environmental
control
Identifies, selects, and adapts learning environments to
increase goal attainment as circumstance permits

Attention control
Concentrates and directs focus to the most pressing task, and
manages distractions

Attention control
Regulates emotional response to maintain focused
performance and learning

Metacognitive
(monitoring)
Identifies individual weaknesses in motivation, effort, or
efficiency

SRL IN SURGICAL EDUCATION 61

Table 4.
Organization of Domains and Subconstructs (Continued)
Domain Sub-Construct Definition
Goal Attainment
and Refinement

Uses specific short-term goals to develop mid-term goals,
which ultimately build upon long-term goals or a main driving
principle.

Goal setting
Continually references or builds an overall mission or long-
term goal, and uses this to persist through adversity and find
career purpose

Goal setting
Develops strategic goals to reach long-range goals, with the
assistance of mentors and self-reflection

Goal setting
Creates, refines, and adapts short-term goals depending on
environmental circumstances, and availability of personal and
external resources

Round Two Instrument
The purpose of Round Two was to capture the agreement (Agree or Disagree) of the
expert panel with SRL constructs listed in Table 4. The survey for Round Two was sent to panel
members who responded to Round One (see Appendix B). Respondents were asked to agree or
disagree with the refined definition for each construct in the domain. Space was made available
after each domain to provide overall comments or suggestions.
Round Two Data Analysis
Both qualitative and quantitative data were analyzed during Round Two. The goal of
Round Two was to determine the level of consensus for the domains and constructs. The results
are reported in Chapter Four.
Saturation and Consensus Levels
As the Delphi technique is an iterative process of evaluation and controlled feedback for
subsequent rounds, it is important to stop the survey process once consensus has been reached or
it appears the topic has reached a level of saturation (Keeney, Hasson, & McKenna, 2010).
Although the number of rounds may vary between studies, a modified Delphi design typically
requires between two and four rounds, with most convergence of opinion occurring between

SRL IN SURGICAL EDUCATION 62

Rounds One and Two (Murry & Hammons, 1995). Consistent with similar Delphi techniques,
70%was the predetermined level of consensus needed for agreement, and this achieved on each
item for this study, thereby ending the Delphi technique after two rounds (Hsu & Sandford,
2007; Keeney, Hasson, & McKenna, 2011).

SRL IN SURGICAL EDUCATION 63

CHAPTER FOUR: RESULTS
The methodology selected for this study, a modified Delphi technique, enabled a rich,
context-specific discussion for self-regulated learning (SRL) in surgical education, that was
grounded in the extensive research of SRL conducted in other fields. This well described
modification (Hsu & Sandford, 2007) facilitated a richer discussion of the applicability of SRL
to surgical residency, while providing a scaffold for SRL as a whole from research in other
fields. The mixed-method nature of a Delphi study recognizes that qualitative and quantitative
data have distinct purposes (Clayton, 2006). Round One relied on qualitative analysis for the
adaption of SRL definitions for the surgical education. Round Two utilized quantitative methods
to determine whether adequate consensus had been obtained to end the Delphi process (Keeney
et al., 2011). See Figure 2 for results by round.

Figure 2. Two-round Delphi results. This mixed-method study utilized quantitative and
qualitative data during both Round One and Round Two analysis.

SRL IN SURGICAL EDUCATION 64

Participants
This study utilized a purposeful sampling approach to select a Delphi panel of experts in
surgical education. All participants were members of a national organization involved in
resident education (Association for Surgical Education, Association of Program Directors in
Surgery), and were actively involved in educational research. Thirty-eight of 42 individuals
responded to round one, resulting in a 90.5% response rate, representing 29 different medical
systems nationally. Six (14.3%) were recruited from a community hospitals. The response rate
for Round Two was 92% with 35 of the 38 participants from Round One completing Round
Two.
Summary of Results
The survey design of Round One collected both quantitative and qualitative data.
Participants rated SRL behaviors, described in other fields, based upon the importance of that
behavior for surgical residents. Many comments suggested that participants rated items lower
because the language used in other fields could not be applied to surgical trainee directly. The
ratings of each item are shown in Table 1, and were used in conjunction with qualitative data to
build the survey for Round Two.
Round One Quantitative Results
Quantitative analysis demonstrated a high overall value of SRL attributes and behaviors
in surgical education. All 25 stems received a median score of 50 or greater for the importance
to surgical residency, on a scale from zero to 100 (Not Important to Very Important). See Table
5 for the median and associated range for each SRL item.

SRL IN SURGICAL EDUCATION 65

Table 5.

Round One Qualitative Results
There were 291 comments coded in Round One for the 25 SRL. In total, 194 (67%)
comments were coded as affirmative (A) or with suggestions for application within surgical
residency (+), and 18 (6%) critiquing relevance to surgical training (-). These were used to
Round One Stem Ratings and Tabulation of Coded Comments
Rank Round 1 Item N Median Range A + - Q NA O
1 Utilize Feedback 38 100 60-100 3 4

2 Time Management 38 95 51-100 7

2

3 Goal Setting 38 94 25-100 12 12 3

1 4
4 Effort 37 91 50-100 2 2

1

3
5 Help-Seeking 38 91 25-100 7 2

2

6 Planning 38 90 50-100 9 4 2

1 1
7 Attention 38 85 50-100 3 2

3

8 Evidence-Based 38 82 0-100 2 4

9 Self-Efficacy 38 80 40-100 7

1 2

2
10 Self-Reaction 37 79 10-100 4 1

11 Monitoring 38 79 40-100 4

1

2
12 Attribution Judgments 38 78 26-100 6 2

3

13 Efficiency 38 76 26-100

3 2
14 Outcome Expectations 38 76 25-100 7 3

1 3 2
15 Collaboration 38 76 1-100 2

2 1 1 1
16 Emotional Control 38 75 23-100 6

2

2

17 Use of Models 38 75 25-100 6 4

18 Strategic Use 38 69 1-100 5 5 1

2
19 Goal Orientation 37 68 25-100 3 12

8 4

20 Metacognition 38 67 10-100 5 1

2 2
21 Environmental Structuring 38 67 25-100 2 10 2

3

22 Performance Calibration 38 63 20-100 4 2

1
23 Innovative 38 60 1-100 4 2 2

24 Motivation 38 60 21-100 3 1 2

2 1
25 Value Appraisal 38 50 24-100 1 7

4 4 3

Total A + - Q NA O

Total Stem Comments

291 114 80 18 17 36 26

Total ON NA O

Free Response
Comments

59

9 18 32

SRL IN SURGICAL EDUCATION 66

modify the individual SRL definitions. There were 79 (27%) comments that were: (a) not
directly applicable to the stem SRL behavior, either commenting on the general topics of SRL or
surgical training (NA: 36, 12%); (b) expressing confusion regarding the SRL stem (Q: 17, 6%),
or relating to a different stem (O: 26, 9%). Coding frequency was tabulated in Table 5, with
examples of comments from each coding category listed in Table 6. General comments about
learning, residency or the study design (NA: 18, 31%), as comments that could be incorporated
into previous stems (O: 32, 54%), and those identifying other novel SRL behaviors not present in
the survey (ON: 9, 15%). These novel items included self-awareness, self-assessment,
resourcefulness, and patient care priorities.

SRL IN SURGICAL EDUCATION 67

Table 6.
Comment Coding Examples
Code Comment
A “This is fundamental to self-regulated learning.”
+
“Learners need to set their own goals, though this is best done in consultation with
supervisors/ teachers.”
- “Not sure that the use of particular learning strategies is all that important.”
NA
“I feel in GME we don't do a good job of at least teaching self-regulated strategies early on
to residents, rather we just expect them to do it.”
Q
“This seems too broad. Do you mean "ability” to complete a LEARNING task? A surgical
task?”

Although all items of Round One were rated as at least somewhat important, the written
comment coding revealed a need for definition modifications to improve relevance to surgical
training. Clarifying questions (Q), suggestions (+), and crossover to other stems (O), suggested
that the construct as it exists in educational psychology is too broad or too narrow, or irrelevant
in the clinical setting. The stem definitions were modified to incorporate coded feedback, yet
still had significant overlap with several themes emerging. Therefore, the written comments
were also analyzed for themes and the SRL behaviors were reorganized into the following seven
domains: self-awareness, task-analysis, situation awareness, strategic planning, progress
evaluation, learning and performance management, and goal attainment and refinement.
Round Two Quantitative Results
Taken in whole, there was 94% agreement by the expert panel to the items on round two.
All items reached the predetermined consensus level of 70% agreement, as demonstrated in
Table 7. With consensus obtained for all items, and after review of the comments, it was
determined that saturation had been reached and addition rounds would not be necessary (Hsu &
Sandford, 2007). Only one participant agreed with less than 70% of the items (64%), compared
to 30 participants (86%) who agreed with 90% or more of the items.

SRL IN SURGICAL EDUCATION 68

Table 7.

Round Two Level of Agreement

Self-Awareness: Awareness of one's own skills and deficits, as well as their
cognitive, emotional, and motivational states
Overall: 95%
Recognizes current knowledge and skill level, based upon self-assessment and
external evaluation/measures
100%

Cognizant of personal limitations 97%

Aware of personal learning strategies that have been previously successful 100%

Metacognitively aware of personal patterns of thought, emotion, and motivation, and
mindful of their effects
83%

Task-Analysis: Appraisal and development of one's goals for an activity, as well
as its learning value and expected outcomes
Overall: 92%
Appraises task level and need for guidance 97%

Establishes specific, achievable, and progressive individual goals to guide learning, in
alignment with known standards, such as established curricula or national mandates
100%

Utilizes mentors to refine goals based upon current performance and trajectory 97%

Visualizes and articulates success 85%

Seeks published empiric evidence or consensus data to inform current practice when
relevant
94%

Understands the costs and benefits of task engagement, balance intrinsic interest and
external demand
77%

Situation Awareness: Understands and appreciates concurrent responsibilities
and patient care, resource and time availability, and interpersonal aspects of
their environment
Overall: 92%
Prioritization of competing demands 97%

Aware of time constraints 97%

Knows or researches available resources 89%

Understands team roles, strengths/weaknesses, and dynamic 86%

Strategic Planning: Incorporation of personal, task and environmental factors to
define goal orientation, mental and physical resource allocation, self-
preparation, and process planning
Overall: 91%
Develops specific steps for goal achievement, seeking guidance when necessary 100%

Identifies learning and performance strategies, such as self-instruction, checkpoints,
mnemonics, or teaching
89%

Alternates between a performance goal orientation if needing to demonstrate ability,
and a mastery or growth orientation during skill/construct development
83%

Actively shapes the learning environment by communicating goals, and requesting
autonomy, teaching, and/or feedback
91%

SRL IN SURGICAL EDUCATION 69

Table 7.
Round Two Level of Agreement (Continued)
Progress Evaluation: recognizes discrepancies between goals and current
state, and appraises the situation to determine the cause
Overall: 97%
Observes and tracks performance toward a goal 97%

Identifies gaps in knowledge, skill, or motivation 97%

Appropriately attributes errors or success to internal and external factors 97%

Incorporates success and failures into future goal generation 100%

Determines the need to adhere or adapt current learning strategies or goals,
based upon effectiveness and need for deeper learning
94%

Manage Learning and Performance: utilizes individual and systemic
strategies for improvement
Overall: 95%
Seeks and uses available resources 97%

Actively seeks assessment and feedback from multiple sources, including peers 100%

Evaluates quality and relevancy of feedback, and incorporates appropriately 94%

Differentiates help based upon the demands of the task and specific deficits 83%

Finds, observes, and learns from peers or experts to develop appropriate mental
and motor skills and coping strategies
97%

Evaluates current knowledge or processes, and adapts practice when necessary 100%

Identifies, selects, and adapts learning environments to increase goal attainment
as circumstance permits
83%

Concentrates and directs focus to the most pressing task, and manages
distractions
94%

Regulates emotional response to maintain focused performance and learning 97%

Identifies individual weaknesses in motivation, effort, or efficiency 100%

Goal Attainment and Refinement: uses specific short-term goals to develop
mid-term goals, which ultimately build upon long-term goals or a main
driving principle.
Overall: 98%
Continually references or builds an overall mission or long-term goal, and uses
this to persist through adversity and find career purpose
94%

Develops strategic goals to reach long-range goals, with the assistance of
mentors and self-reflection
100%

Creates, refines, and adapts short-term goals depending on environmental
circumstances, and availability of personal and external resources
100%

Round Two Qualitative Results
Respondents provided 51 comments during Round Two after each domain and regarding
the overall framework. Compared to comment coding in Round One that focused adjusting SRL

SRL IN SURGICAL EDUCATION 70

definitions for surgical education, Round Two comments were analyzed thematically, with
several key themes emerging from the data. Upon further review, many of these themes also
resonated in the original comments from Round One.
Theme One: Clinical Prioritization
Many SRL behaviors discussed in Chapter Two and included in Rounds One and Two
were related to executive functioning and awareness of one’s learning environment, managing
multiple tasks, and attention control. Multiple participants emphasized labeling patient care as
its own entity and not just another task. Clearly labeling patient care as a top priority,
superseding individual learning was mentioned on more than one occasion. This theme was
incorporated into the situational-awareness domain of Round Two. Therefore, patient care was
labeled as an independent factor that must be weighed against other competing responsibilities,
and with limited time and resources.
Theme Two: Learner Control
Participants reiterated throughout both rounds that resident learners have little control
over their learning topics and learning environment. One of the lowest ranked items of Round
One was the SRL behavior “environmental structuring”, and although it was valued in theory,
respondents emphasized limited control over learning, clinical, and working environments.
Respondents from Round One stated “it is likely helpful to be forced into sub-optimal
environments,” that “sometimes the learning needs to be fitted into existing life conditions,” and
that “the learner must be able to adapt to the situation.”
Transitioning from a choice in learning environments to learner adaptability was
incorporated into multiple domains for Round Two, including situational awareness, strategic
planning, and learning and performance management. Qualitative analysis of Round Two

SRL IN SURGICAL EDUCATION 71

indicated respondent difficulty in interpreting this topic as presented, with some respondents
stating again that this is out of the control of the resident. Few solutions were offered for
situations deemed a poor learning environment.
Theme Three: External Resources
Awareness, utilization, and reliance on external resources was a theme common to
several SRL domains in Round Two. One of the participants emphasized that self-awareness
cannot be limited to self-assessment. True self-regulation involving self-awareness includes the
awareness of external assessment and calibration. Elements of utilizing external assessment
were present in Round Two, but the behavior “performance calibration” was found to be
confusing and was rated similar to environmental structuring with a low median score in Round
One.
More emphasis was placed upon the infrastructure that supports the resident, rather than
the resident’s self-initiated use of external resources. One response stated that a learner’s
recognition of his or her own knowledge and skill level “obviously requires a residency that is
supplying appropriate, meaningful, and timely external evaluation.” Another suggested a
rephrasing of a learner-initiated search for available resources, to learner appraisal of “value and
limitations” of resources provided. Although this is slightly nuanced, the comment suggests that
the learner’s role is more cognitive, and that resources should be made available to the learner.
Theme Four: Limited Metacognition in General or with Faculty
A major aspect of SRL is one’s ability to recognize his or her own thoughts, actions, and
patterns. Throughout both rounds, and in multiple domains, metacognition was viewed by the
panelists as a lofty goal and one not learned in residency or by many practicing surgeons.
Comments stated that “metacognition is acquired over years of self-reflection… I doubt that

SRL IN SURGICAL EDUCATION 72

many people…are especially good at that”, and that “the majority [of people don’t] really get
there.” The concept of partial metacognition was a theme of a few comments: “an emerging
awareness,” “a partial awareness,” or awareness but not “the ability to modify their
metacognitive attributes.”
Theme Five: Goal Orientation
Statements surrounding goal orientation remained nebulous and received comments in
both rounds, as well as receiving a low median score in round one. Goal orientation was
described as “way too dense of a construct”, with “too much verbiage.” The dichotomy between
performance and mastery orientation was an issue. It was expressed that “mastery orientation is
not inconsistent with needing to demonstrate an ability.” It is possible that the term “mastery” is
misinterpreted in this sample as reaching a level of autonomy. One participant stated that
“residents achieve relatively little true mastery in their GME years, but must develop many skills
to the level of basic ability,” while another clearly stated, “mastery of content could mean ability
to perform independently… there are very few opportunities for surgery residents to function
independently.”
Summary
The response rates for both Round One and Two were high, with an 8% drop-out rate for
the panel between rounds. The panel found value in all of the SRL behaviors supported by
literature in other fields. Incorporation of context-specific descriptions for each increased the
level of agreement for all items to at least 70%, reaching the pre-defined level for consensus.
Qualitative analysis for both rounds highlighted the importance of the surgical environment and
its limitations for learners. Additionally, the panel consistently expressed the essential nature of

SRL IN SURGICAL EDUCATION 73

external support for these SRL behaviors. Finally, many SRL behaviors, and metacognition in
particular, were viewed on a spectrum and that emerging awareness may be sufficient.
Although there was consensus that all contextualized SRL behaviors were important, the
qualitative results and findings of this study demonstrate that presentation of SRL behaviors in a
list format insufficiently describes SRL in surgical residency. The Delphi technique study design
confirmed a set of ideal SRL attributes, while also examining the feasibility of SRL through a
qualitative lens. Therefore, interpretation of these results and subsequent interventions should
recognize the external constraints emphasized by the panel of experts.

SRL IN SURGICAL EDUCATION 74

CHAPTER FIVE: DISCUSSION
The practice of educating surgeons has evolved over the years from a pure apprenticeship
model (Halsted, 1904) to one that maximizes the training of many residents in a diverse set of
training environments and instructors (Pellegrini, 2006). This shift has coincided with a change
in how regulatory agencies view accreditation, from evaluating programs directly to assessing
the products of these training programs, the trainees themselves (Williams & Dunnington, 2004).
The success of competency-based assessment for surgeons during and after completion of
surgical training requires a clear understanding of the objectives of training, and shared
definitions of competency. The development, implementation, and refinement of these
competencies have become clearer in surgical education over the last two decades, with six Core
Competencies outlined by the Accreditation Council of Graduate Medical Education (ACGME)
(Mainiero & Lourenco, 2011) which set clear standards for all residency programs regardless of
specialty.
To accommodate the growing diversity of training paradigms and the ever-changing body
of knowledge, the medical education community has seemingly embraced the concepts of
learner-centered education and lifelong learning (ACGME & ABS, 2015; Holmboe et al., 2016).
One of the six ACGME Core Competency domains, Practice-Based Learning and Improvement,
defines several skills required of lifelong learners that fall within the broader concept of self-
regulated learning (SRL), such as self-assessment, recognition of personal skills and limitations,
establishment of learning goals, reflection on practice, and incorporation of feedback for
personal improvement (ACGME, 2017). Therefore, SRL offers an ideal conceptual framework
for teaching and evaluating Practice-Based Learning and Improvement in surgical education.

SRL IN SURGICAL EDUCATION 75

SRL has been defined and examined in other educational settings (Zimmerman, 2002),
and has been found to be a context-specific concept (Cleary, 2011). Furthermore, SRL has been
shown not to be an innate skill but one that must be taught (Schunk & Ertmer, 2000). Although
new requirements for medical school curriculum include a mandate to teach lifelong learning
skills (LCME, 2016), application of the SRL model defined by Zimmerman has led to mixed
results for medical students. Utilization of SRL behaviors during medical school is variable
(Sullivan, 2003), may decrease over the course of training (Artino, Cleary, Dong, Hemmer, &
Durning, 2014; Artino et al., 2012), and does not correlate with defined measures of success
(Lucieer, et al., 2016).
Purpose of the Study
This study was designed to confront (a) the inconsistent and discouraging results about
SRL behaviors in medical students who enter surgical training; (b) address a lack of SRL
research in surgical trainees; and (c) provide guidance for the incorporation of SRL into surgical
trainee competencies. The purpose of this study was to build a framework for SRL in surgical
residency. The research question guiding this study was “What evidence-based self-regulated
learning strategies do experts in surgical education deem necessary for residents during surgical
training?”
This study used a mixed-method design via a modified Delphi survey technique, with two
survey rounds, and a panel of national experts in surgical education. SRL behaviors were
collected from published literature in other educational fields via an extensive literature review.
In Round One, participants were asked to rate the level of importance of each SRL construct for
surgical residency. Based upon the quantitative and qualitative results of Round One, the same

SRL IN SURGICAL EDUCATION 76

panel of experts in Round Two was asked to agree or disagree with SRL constructs modified and
reorganized for use in surgical residency.
Discussion of Findings
In a Delphi method, the validity and value of the results depends upon the panel of
experts, their commitment to the research, and adherence to core principles of the Delphi method
(Hsu & Sandford, 2007), including use of (a) a purposeful sample; (b) iterative rounds; (c)
controlled feedback of previous results; and (d) incorporation of panel suggestions. In this study,
the panel captures significant expertise from the field of surgical education, with over 20% of the
panel serving as a president of either the Association for Surgical Education or the Association
for Program Directors in Surgical Education. The panel demonstrated commitment to the
research, with response rates for both rounds exceeding 90% and over 350 written comments.
The content of the two survey rounds were distinctly different to incorporate feedback from the
panel via written comments that SRL is different in surgical education compared to other
educational fields. Figure 3 provides a visual representation of the overall framework for SRL in
surgical education.

SRL IN SURGICAL EDUCATION 77

Figure 3. Self-regulated learning framework in surgical education. Self-regulated learners in
surgical education must appraise the task, the situation, and be self-aware; regulate their learning
in a proactive, adaptive, and reflective manner; pursue and refine hierarchical learning goals.

This study was exploratory in nature and was designed to use empirical evidence from
other fields as a foundation for expert opinion in surgical education. Although it has been
hypothesized that SRL attributes are important in residency, research in medical students and
other medical residencies has demonstrated an inconsistent pattern in SRL behavior usage.
Consistent with a mixed-method design (Creswell, 2014), initial assumptions about SRL were
collected from the literature, but the qualitative data from the panel was used to modify these
predictions and understand the conceptual differences between SRL in surgical education
compared with traditional education settings. Therefore, the individual SRL behaviors identified

SRL IN SURGICAL EDUCATION 78

by Zimmerman (2015) served as a working hypothesis, but the adaption of these and their
relationship to each other emerged from the data.
Discussion of Quantitative Findings
The goal of the Delphi technique is to bring together many opinions about a complex
topic and achieve a consensus of greater than 70% (Clayton, 2006; Hsu & Sandford, 2007). In
Round One, although all original 25 SRL constructs were felt to be at least somewhat important,
there was decent variability among respondents with 80% of items receiving at least one score
below 50 on a 100 point scale. After restructuring the SRL constructs and framework to include
appropriate surgical education language and opinions from Round One, 75% of the items
received over 90% consensus, and all items had greater than 70% agreement. A movement from
20% to 75% of agreement represents a significant methodologic success (Holey, Feeley, Dixon,
& Whittaker, 2007), achieving the goal of moving a group of individuals toward a consensus
without allowing conversation to be dominated by a few individuals (Dalkey & Helmer, 1963).
Discussion of Qualitative Findings
Initially, SRL in surgical education may appear similar to learning in other domains, but
qualitative analysis of this study demonstrated learning constraints inherent to the surgical
residency learning environment. These factors are specific to surgical training, which is a unique
learning environment that is also a work environment. According to the ACGME, both faculty
and residents must recognize that work and patient care supersedes learning and personal interest
in surgical residency (ACGME, 2017). In this learning environment, there is an intricate balance
of needs between resident education and patient care (Pellegrini, 2006).
Surgical education has slowly embraced the transition from a teacher-centered, process-
oriented teaching approach to a learner-centered competency-based educational model. Taken

SRL IN SURGICAL EDUCATION 79

from other educational domains, this paradigm shift fails to account for the patient at the center
of the surgical environment, ultimately creating an incomplete picture of the learning
environment for surgical faculty, trainees and governing bodies. Because the surgical
community is still early in this transition (ACGME & ABS, 2015; Holmboe et al., 2016), it is
understandable that the expert panel maintained the need for a process-oriented teaching
philosophy and described the educational environment as inflexible. Throughout both rounds,
there was an emphasis on the limited options available to learners. Importantly, most comments
that described limitations of the surgical environment were not endorsing or condemning this
fact, but simply recognizing the restrictions. However, the panel also offered no suggestions for
correcting this or even an opinion that learner choice was important. Empowering learners to
select and modify their learning environments has been shown to benefit learning in other
contexts (Bierer & Dannefer, 2016; Hoffman, 2015; Zumbrunn, Tadlock & Roberts, 2011), but
this may not be possible in surgical residency.
A similar theme from the panelists was a tendency to focus on processes in education,
rather than the learner directly. Specifically with the concept of goal orientation, comments
emphasized performance over mastery, or a need to demonstrate competency. Attention control
was similarly viewed as a prioritization and time-management skill, with less focus on cognitive
resources or internal thought processes. In strategic planning, one panelist wanted additional
specific examples of learning strategies to be included, such as recall and repeat testing. These
are very externally-driven processes that can be observed, in contrast to more cognitive and
metacognitive strategies that are driven by the learner.
Consistent with a focus on behaviors that can be observed from the outside, the panel
placed priority on externally validated resources and assessment over learner selected materials

SRL IN SURGICAL EDUCATION 80

or self-assessment. There could be benefit to this thought process, in that the panel held the
program accountable for timely and accurate feedback. As discussed in Chapter Two, it is
important for learners to validate their outcome attributions with others and seek help during the
reflective phase of traditional SRL models (Karabenick & Dembo, 2011). This focus on external
support is viewed as learner initiated in Zimmerman’s SRL model (2015), yet the panel focused
on the responsibility of the infrastructure and educators to provide appropriate support.
The current surgical training paradigm that mandates programs to provide feedback and
resources to residents supports this idea of a learner relying on external support over their own.
Although not discussed by the panel, this emphasis on providing an organized structure for the
learner may exist to counter the learner’s limited choice in learning environment that was
repeated mentioned by the panel. Logically, to accommodate diverse surgical learners and
unmalleable learning environments, training programs may need to be the reliable source for
learning resources and assessment. Furthermore, this arrangement is one that can be supported
by current surgical training models with oversight provided by the ACGME and ABS as
accrediting bodies.
There is a growing awareness of the topic of metacognition and its implications in
surgical education. A test preparatory company used by some general surgery residency
programs asks learners to predict their performance prior to taking practice tests (Arnold, 2018).
Their blog describes the rationale for their testing strategies and provides evidence to support
metacognition in adult learning environments. In fact, the opening line states “One of the most
difficult aspects of preparing for a medical licensure test is determining how to regulate your
own learning” (para. 1). Although metacognition has been viewed as a difficult construct, there

SRL IN SURGICAL EDUCATION 81

is support for implementation of this SRL from governing bodies, contemporary surgical
education materials, and results from the study of experts in the field of surgical education.
Alignment With SRL Frameworks
Consistent with Bandura’s (2001) social cognitive model, panel members were not
completely satisfied with the discrete domains. They recognized that there still was overlap in
subconstructs and some expressed interest in how all items would relate together in a formal
factor analysis. Comments highlighted the reciprocal nature of the learner’s thoughts, actions,
and the environment. The first three domains were intimately tied together, with suggestions to
expand “cognizant of personal limitations” to reflect how a learner’s awareness of time
management and prioritization skills translate into effective action; include constraints of the
situation when interpreting “costs and benefits of task engagement” recognize the individual’s
relationship to the task, both with “personal comfort” over absolute task difficulty, and with the
variable definition of “success” for each individual. These complex SRL constructs represent a
dynamic relationship between Self, Task, and Situation that is difficult to capture in a static
model of SRL, and should be examined with greater depth in future studies.
The traditional model of SRL by Zimmerman and Schunk (2011) follows the social
cognitive theory conceptual model of Bandura (2001). Major themes of this model are learners
as the central agent in their learning, and the reciprocal, dynamic nature of the environment, the
learner’s thoughts and emotions, and the learner’s behavior. The results indicate that this
relationship may not be as clear in surgical education, as others have shown with medical
students (Cleary, Dong, & Artino, 2015; Sullivan, 2003). Specifically, themes of the study
suggest that learners have little choice in their environment and resources, and that patient care
and performing at a level of competency take precedence over learning. Surgical residency is

SRL IN SURGICAL EDUCATION 82

therefore a five- to nine-year trainee paradigm, that includes 80-hours of work each week that
has little room for mastery-oriented learning, and requires learners to perform competently. In
this work environment, it is not surprising that the panel emphasized the importance of external
support and processes to ensure appropriate growth and efficient workflow.
Limitations and Delimitations
As discussed in Chapter One, there are specific limitations and delimitations in this study.
The results of this study are the opinions of the panel, and would classically be described as
nongeneralizable. However, the intent of this study was to provide guidance for educators and
learners who seek to understand SRL in a surgical context. The methodology of this study was
deliberately selected, in addition to the analysis and presentation of data. As described by
Merriam (2009) regarding similar limitations with case studies, “it is the reader, not the
researcher, who determines what can apply to his or her context” (p. 51). Although the opinions
expressed here are those of the distinct panel of experts and their impact on each other through
iterative rounds, the hope is that the reader recognizes this not only as a limitation but also in the
power of the collective opinions of the panel, which was intentionally selected to represent
expertise in the field of surgical education.
The main delimitation of this study is the narrow focus on one learner population.
Surgical residents represent a unique category of adult learners, with specific initial knowledge
and learning needs. Although the framework can be used as a starting point in other study
populations, the results of this study demonstrate the context-specific nature of SRL and
therefore are intended to inform the education of surgical residents.

SRL IN SURGICAL EDUCATION 83

Implications for Practice
This study represents a needs assessment for learner-centered education in surgery.
Governing bodies have mandated inclusion of many SRL principles into surgical curriculum,
including self-assessment, recognition of one’s own limitations, creation of learning goals based
upon these deficits, individualized learning plan development, and reflection on individual
practice. This study affirms that expert surgical educators perceive value in SRL behaviors. The
available research, for medical school and medical specialties, suggests that the surgical
community poorly teaches and evaluates SRL behaviors. The results of this study can be used to
guide further exploration into specific SRL behaviors, eventually developing more refined
learning objectives, curriculum, and assessments in a component fashion, while valuing the
unique patient care aspects of the surgical educational environment.
One specific area recognized during this study was metacognition. The panel recognized
the difficulty of this topic for residents and faculty. Although, likely unintentional, many of the
comments mirrored the structure of the developmental Milestones currently to evaluate residents
(ACGME & ABS, 2015; Holmboe et al., 2016). Metacognition was viewed as a spectrum, with
distinct skills that could be mastered at different stages in residency. In increasing order of
difficulty, the panel identified a learner having:
● a basic understanding or definition of metacognition
● an emerging awareness of one’s metacognitive thought processes
● awareness of one’s thought processes and emotional response patterns
● an appreciation of how thought patterns affect action
● a utilization of metacognitive strategies and altering thought processes for
behavioral change.

SRL IN SURGICAL EDUCATION 84

This study provides direction on ways to educate and integrate metacognition into surgical
culture, potentially as a training milestone. As the panel emphasized, metacognition may be too
complex of a topic to teach all at once and may require gradual introduction into surgical
curriculum for both faculty and residents.
Although all items were rated as important for an ideal self-regulated learner in surgical
residency, panel members reiterated the difficult nature of these items, both in terms of
complexity, and also in stating that even lesser forms of these constructs would be an
improvement and potentially satisfactory for some residents. If incorporated into residency
Milestones, it should be recognized that varying levels of SRL may be acceptable depending
upon the individual and the specific SRL construct.
Recommendations for Future Study
There are three main directions for future studies based upon the findings. The results
can be explored with future studies that seek to provide finer detail to the behaviors described in
this study. In an exploratory direction, topics that were identified as conceptually complex in
round two can be further explored in their more expanded forms. These included outcome
expectations, value appraisal, goal orientation, environmental structuring, and metacognition.
Future study may be able to address the complexity of these topics by examining the nuances
within the accompanying educational theory, such as expectancy value theory or adaptive goal
theory. Topics from this study that reached high levels of agreement can be further measured
specifically with surgical residents. There are many psychometric tools that in original or
adapted formats explore individual components of SRL. Some of these were discussed in
chapter two, and include measures of self-efficacy, motivation, and strategic planning, among
others. Additionally, the identified domains of SRL and broader concepts such as metacognition

SRL IN SURGICAL EDUCATION 85

or environmental structuring can be evaluated in a context-specific manner via a microanalysis
technique, a newer methodology discussed in chapter two. Microanalysis requires a trained
researcher to probe a learner during a learning activity with open- and closed-ended questions for
specific mental processes that may not be readily apparent or sufficiently captured via external
observation, self-report, or post-activity reflection. Findings from this study provide the optimal
foundation for microanalysis by defining SRL behaviors that are most valued by expert surgical
educators and suggestions for individual components that need to be explored specifically within
the context of surgical residency.
A second direction for future research would be to expand the context of the study to
other learner populations, such as fellows or young surgeons early in their careers. The SRL
needs for surgeons after residency may be different than the studied population. Similarly, if the
study were expanded for medical students, the focus may be on different SRL processes in
preparation for surgical residency.
Lastly, the study provides several SRL behaviors and attributes that are familiar to
surgeons and found to have significant agreement among the panel members. SRL curriculum
exist in other domains including K-12 education, sports, and music (Boekaerts et al., 2000;
Schunk & Zimmerman, 1997; Zimmerman & Schunk, 2011). Future research could include
implementation of a SRL curriculum that targets a few SRL behaviors. Ideally, this would be
analyzed via rigorous methods that include control and intervention groups, with comparative
analysis of learner performance and outcomes.
Conclusion
This study is the first of its kind to systematically assess SRL for surgical residents from
the perspective of the educator. The methodology has demonstrated not only the importance of

SRL IN SURGICAL EDUCATION 86

SRL behaviors in surgical residency, but done so with overwhelming and unanimous support
from experts in surgical education from a diverse set of teaching environments, both
geographically and in regard to academic affiliation. This consensus study provides a new
framework tailored for surgical residency, and distinctly different from previously published
models. This study should serve as a foundation for future work to validate and elaborate upon
the findings in a context-specific manner. Further exploration of SRL constructs and curriculum
for specific SRL behaviors in surgical education may provide additional guidance for educators
and learners during the transition to competency-based surgical education, and support the call to
develop surgeons who are lifelong learners.

SRL IN SURGICAL EDUCATION 87

References
Accreditation Council for Graduate Medical Education. (2017). ACGME Program
Requirements for Graduate Medical Education in General Surgery. Retrieved from
https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/440_general_surgery
_2017-07-01.pdf
Accreditation Council for Graduate Medical Education. (2011). The ACGME 2011 Duty Hour
Standard. (I. Philibert & S. Amis, Eds.) (pp. 1–113). Chicago. Retrieved from
http://www.acgme-2010standard...monographs/jgme-monograph.pdf
Accreditation Council for Graduate Medical Education, & American Board of Surgery. (2015).
The General Surgery Milestone Project, 1–19.
Ahmed, N., Devitt, K. S., Keshet, I., Spicer, J., Imrie, K., Feldman, L., et al. (2014). A
Systematic Review of the Effects of Resident Duty Hour Restrictions in Surgery. Annals
of Surgery, 259(6), 1041–1053. http://doi.org/10.1097/SLA.0000000000000595
Aho, J. M., Ruparel, R. K., Graham, E., Zendejas-Mummert, B., Heller, S. F., Farley, D. R., &
Bingener, J. (2015). Mentor-Guided Self-Directed Learning Affects Resident Practice.
Journal of Surgical Education, 72(4), 674–679.
http://doi.org/10.1016/j.jsurg.2015.01.008
Allen, D. P. (1907). The American Surgical Association. The Boston Medical and Surgical
Journal, 156(23), 748–751. http://doi.org/10.1056/NEJM190706061562308
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich,
P. R., ... & Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A
revision of Bloom’s taxonomy of educational objectives, abridged edition. White Plains,
NY: Longman.

SRL IN SURGICAL EDUCATION 88

Arnold, A. (2018) Medical Licensure Test Predictions: The Importance of Metacognition.
Accessed May 9, 2018: https://truelearn.com/blog/test-predictions-importance-
metacognition/
Artino, A. R., Cleary, T. J., Dong, T., Hemmer, P. A., & Durning, S. J. (2014). Exploring
clinical reasoning in novices: a self-regulated learning microanalytic assessment
approach. Medical Education, 48(3), 280–291. http://doi.org/10.1111/medu.12303
Artino, A. R., Dong, T., DeZee, K. J., Gilliland, W. R., Waechter, D. M., Cruess, D., &
Durning, S. J. (2012). Achievement goal structures and self-regulated learning:
relationships and changes in medical school. Academic Medicine : Journal of the
Association of American Medical Colleges, 87(10), 1375–1381.
http://doi.org/10.1097/ACM.0b013e3182676b55
Balla, F., Garwe, T., Motghare, P., Stamile, T., Kim, J., Mahnken, H., & Lees, J. (2016).
Evaluating Coding Accuracy in General Surgery Residents' Accreditation Council for
Graduate Medical Education Procedural Case Logs. Journal of Surgical Education,
73(6), e59–e63. http://doi.org/10.1016/j.jsurg.2016.07.017
Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change.
Psychological review, 84(2), 191.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.
Englewood Cliffs, NJ, US: Prentice-Hall, Inc.
Bandura, A. (2001). Social cognitive theory: an agentic perspective. Annual Review of
Psychology, 52, 1–26. http://doi.org/10.1146/annurev.psych.52.1.1

SRL IN SURGICAL EDUCATION 89

Barden, C. B., Specht, M. C., McCarter, M. D., Daly, J. M., & Fahey, T. J. (2002). Effects of
limited work hours on surgical training. Journal of the American College of Surgeons,
195(4), 531-538.
Batalden, P., Leach, D., Swing, S., Dreyfus, H., & Dreyfus, S. (2002). General Competencies
And Accreditation In Graduate Medical Education. Health Affairs, 21(5), 103–111.
http://doi.org/10.1377/hlthaff.21.5.103
Bell, R. H., Jr., Biester, T. W., Tabuenca, A., Rhodes, R. S., Cofer, J. B., Britt, L. D., & Lewis,
F. R., Jr. (2009). Operative Experience of Residents in US General Surgery Programs.
Annals of Surgery, 249(5), 719–724. http://doi.org/10.1097/SLA.0b013e3181a38e59
Bierer, S. B., & Dannefer, E. F. (2016). The Learning Environment Counts: Longitudinal
Qualitative Analysis of Study Strategies Adopted by First-Year Medical Students in a
Competency-Based Educational Program. Academic Medicine : Journal of the
Association of American Medical Colleges, 91(11 Association of American Medical
Colleges Learn Serve Lead: Proceedings of the 55th Annual Research in Medical
Education Sessions), S44–S52. http://doi.org/10.1097/ACM.0000000000001363
Boekaerts, M., & Niemivirta, M. (2000). Self-Regulated Learning: Finding a balance between
learning goals and ego-protective goals. In M. Boekaerts, P.R. Pintrich, & M. Zeidner
(Eds.), Handbook of Self-regulation (pp. 417-450). Burlington, MA: Elsevier.
Boekaerts, M., Pintrich, P. R., & Zeidner, M. (2000). Handbook of Self-regulation. (M.
Boekaerts, P. R. Pintrich, & M. Zeidner, Eds.). Elsevier.
Boekaerts, M., Pintrich, P.R., & Zeidner, M. (2000). Self-Regulation: An Introductory. In M.
Boekaerts, P.R. Pintrich, & M. Zeidner (Eds.), Handbook of Self-regulation (pp. 1-9).
Burlington, MA: Elsevier.

SRL IN SURGICAL EDUCATION 90

Borman, K. R., Biester, T. W., Jones, A. T., & Shea, J. A. (2011). Sleep, Supervision,
Education, and Service: Views of Junior and Senior Residents. Journal of Surgical
Education, 68(6), 495–501. http://doi.org/10.1016/j.jsurg.2011.07.012
Brady, S. R. (2015). Utilizing and adapting the Delphi method for use in qualitative research.
International Journal of Qualitative Methods, 1–6.
http://doi.org/10.1177/1609406915621381
Brewster, L. P., Risucci, D. A., Joehl, R. J., Littooy, F. N., Temeck, B. K., Blair, P. G., &
Sachdeva, A. K. (2008). Comparison of resident self-assessments with trained faculty
and standardized patient assessments of clinical and technical skills in a structured
educational module. American Journal of Surgery, 195(1), 1–4.
http://doi.org/10.1016/j.amjsurg.2007.08.048
Brydges, R., Manzone, J., Shanks, D., Hatala, R., Hamstra, S. J., Zendejas, B., & Cook, D. A.
(2015). Self-regulated learning in simulation-based training: a systematic review and
meta-analysis. Medical Education, 49(4), 368–378. http://doi.org/10.1111/medu.12649
Chapman, W. C. (2016). Surgical Training in the United States: Is It Time for a Paradigm Shift?
Journal of the American College of Surgeons, 223(1), 1–7.
http://doi.org/10.1016/j.jamcollsurg.2016.01.055
Chung, R. S., & Ahmed, N. (2007). How Surgical Residents Spend Their Training Time.
Archives of Surgery, 142, 249–252.
Clark, N. M., & Zimmerman, B. J. (2014). A social cognitive view of self-regulated learning
about health. Health education & behavior: the official publication of the Society for
Public Health Education (2nd ed., Vol. 41, pp. 485–491). SAGE Publications.
http://doi.org/10.1177/1090198114547512

SRL IN SURGICAL EDUCATION 91

Clayton, M. J. (1997). Delphi: a technique to harness expert opinion for critical decision-
making tasks in education. Educational Psychology, 17(4), 373–386.
http://doi.org/10.1080/0144341970170401
Cleary, T. J. (2011). Emergence of Self-Regulated Learning Microanalysis. In Handbook of
Self-Regulation of Learning and Performance. Routledge.
http://doi.org/10.4324/9780203839010.ch21
Cleary, T. J., & Zimmerman, B. J. (2001). Self-Regulation Differences during Athletic Practice
by Experts, Non-Experts, and Novices. Journal of Applied Sport Psychology, 13(2),
185–206. http://doi.org/10.1080/104132001753149883
Cleary, T. J., Durning, S. J., & Artino, A. R. (2016). Microanalytic Assessment of Self-
Regulated Learning During Clinical Reasoning Tasks: Recent Developments and Next
Steps. Academic Medicine : Journal of the Association of American Medical Colleges,
91(11), 1516–1521. http://doi.org/10.1097/ACM.0000000000001228
Coleman, J. J., Esposito, T. J., Rozycki, G. S., & Feliciano, D. V. (2013). Early
Subspecialization and Perceived Competence in Surgical Training: Are Residents
Ready? Journal of the American College of Surgeons, 216(4), 764–771.
http://doi.org/10.1016/j.jamcollsurg.2012.12.045
Cooper, M. A., Tinmouth, J. M., Yong, E., Walsh, C. M., Carnahan, H., Grover, S. C., & Ritvo,
P. (2014). Surgical Residents’ Perceptions on Learning Gastrointestinal Endoscopy:
More Hours and More Standardization Wanted. Journal of surgical education, 71(6),
899-905.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches. Sage publications.

SRL IN SURGICAL EDUCATION 92

Dalkey, N., & Helmer, O. (1963). An Experimental Application of the DELPHI Method to the
Use of Experts. Management Science, 9(3), 458–467.
http://doi.org/10.1287/mnsc.9.3.458
Damewood, R. B., Blair, P. G., Park, Y. S., Lupi, L. K., Newman, R. W., & Sachdeva, A. K.
(2017). “Taking Training to the Next Level”: The American College of Surgeons
Committee on Residency Training Survey. Journal of Surgical Education.
http://doi.org/10.1016/j.jsurg.2017.07.008
de Blacam, C., O'Keeffe, D. A., Nugent, E., Doherty, E., & Traynor, O. (2012). Are residents
accurate in their assessments of their own surgical skills? American Journal of Surgery,
204(5), 724–731. http://doi.org/10.1016/j.amjsurg.2012.03.003
Dembo, M., & Seli, H. (2016). Motivation and Learning Strategies for College Success, 1–372.
Dreyfus HL and Dreyfus SE. Mind over Machine: the power of human intuition and expertise in
the age of the computer. Oxford, Basil Blackwell, 1986.
Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and
personality. Psychological review, 95(2), 256.
Elliot, A. J., & Church, M. A. (1997). A hierarchical model of approach and avoidance
achievement motivation. Journal of personality and social psychology, 72(1), 218.
Elliot, A. J., McGregor, H. A., & Gable, S. (1999). Achievement goals, study strategies, and
exam performance: A mediational analysis. Journal of educational psychology, 91(3),
549.
Fink, A., Kosecoff, J., Chassin, M., & Brook, R. H. (1984). Consensus methods: characteristics
and guidelines for use. American Journal of Public Health, 74(9), 979–983.
http://doi.org/10.2105/AJPH.74.9.979

SRL IN SURGICAL EDUCATION 93

Franzone, J. M., Kennedy, B. C., Merritt, H., Casey, J. T., Austin, M. C., & Daskivich, T. J.
(2015). Progressive Independence in Clinical Training: Perspectives of a National,
Multispecialty Panel of Residents and Fellows. Journal of Graduate Medical Education,
7(4), 700–704. http://doi.org/10.4300/JGME-07-04-51
Fuchshuber, P. R., Robinson, T. N., Feldman, L. S., Brunt, L. M., Madani, A., Jones, S. B., et
al. (2015). Fundamental Use of Surgical Energy (FUSE): Closing a Gap in Medical
Education. Annals of Surgery, 262(1), 20–22.
http://doi.org/10.1097/SLA.0000000000001256
Gagliardi, A. R., Wright, F. C., Victor, J. C., Brouwers, M. C., & Silver, I. L. (2009). Self-
directed learning needs, patterns, and outcomes among general surgeons. Journal of
Continuing Education in the Health Professions, 29(4), 269–275. http://doi.org/DOI:
10.10020chp.20046
Gottfried, A. E. (1990). Academic intrinsic motivation in young elementary school children.
Journal of educational psychology, 82(3), 525.
Gow, K. W. (2013). Self-evaluation: how well do surgery residents judge performance on a
rotation? The American Journal of Surgery, 205(5), 557–562.
http://doi.org/10.1016/j.amjsurg.2013.01.010
Greenberg, C. C., & Klingensmith, M. E. (2015). The Continuum of Coaching: Opportunities
for Surgical Improvement at All Levels. Annals of Surgery, 262(2), 217–219.
http://doi.org/10.1097/SLA.0000000000001290
Greenberg, C. C., Dombrowski, J., & Dimick, J. B. (2016). Video-Based Surgical Coaching: An
Emerging Approach to Performance Improvement. JAMA Surgery, 151(3), 282–283.
http://doi.org/10.1001/jamasurg.2015.4442

SRL IN SURGICAL EDUCATION 94

Grillo, H. C. (2004). Edward D. Churchill and the “rectangular” surgical residency. Surgery,
136(5), 947–952. http://doi.org/10.1016/j.surg.2004.09.002
Habibi, A., Sarafrazi, A., & Izadyar, S. (2015). Delphi technique theoretical framework in
qualitative research. The International Journal of Engineering and Science, 3(4), 8–13.
Halsted, W. S. (1904). The Training of the Surgeon. Bulletin of the Johns Hopkins Hospital,
15(162), 267–275.
Hashimoto, D. A., Bynum, W. E., Lillemoe, K. D., & Sachdeva, A. K. (2016). See More, Do
More, Teach More: Surgical Resident Autonomy and the Transition to Independent
Practice. Academic Medicine : Journal of the Association of American Medical
Colleges, 91(6), 757–760. http://doi.org/10.1097/ACM.0000000000001142
Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey
technique. Journal of Advanced Nursing. http://doi.org/10.1046/j.1365-2648.2000.t01-1-
01567.x
Hoffman, B. D. (2015). Using Self-Determination Theory to Improve Residency Training.
Academic Medicine, 90(4), 408–410. http://doi.org/10.1097/ACM.0000000000000523
Holey, E. A., Feeley, J. L., Dixon, J., & Whittaker, V. J. (2007). An exploration of the use of
simple statistics to measure consensus and stability in Delphi studies. BMC Medical
Research Methodology, 7(1), 52. http://doi.org/10.1186/1471-2288-7-52
Holmboe, E. S., Edgar, L., & Hamstra, S. J. (2016). The Milestones Guidebook, 1–41.
Holmboe, E. S., Yamazaki, K., Edgar, L., Conforti, L., Yaghmour, N., Miller, R. S., & Hamstra,
S. J. (2015). Reflections on the First 2 Years of Milestone Implementation. Journal of
Graduate Medical Education, 7(3), 506–511. http://doi.org/10.4300/JGME-07-03-43

SRL IN SURGICAL EDUCATION 95

Hsu, C. C., & Sandford, B. A. (2007). The Delphi technique: making sense of consensus.
Practical Assessment, 12, 1–8.
Hu, Y.-Y., Mazer, L. M., Yule, S. J., Arriaga, A. F., Greenberg, C. C., Lipsitz, S. R., et al.
(2016). Complementing Operating Room Teaching With Video-Based Coaching. JAMA
Surgery, 1–8. http://doi.org/10.1001/jamasurg.2016.4619
Hutter, M. M., Kellogg, K. C., Ferguson, C. M., Abbott, W. M., & Warshaw, A. L. (2006). The
impact of the 80-hour resident workweek on surgical residents and attending surgeons.
Annals of Surgery, 243(6), 864–71– discussion 871–5.
http://doi.org/10.1097/01.sla.0000220042.48310.66
Immenroth, M., Bürger, T., Brenner, J., Nagelschmidt, M., Eberspächer, H., & Troidl, H.
(2007). Mental training in surgical education: a randomized controlled trial. Annals of
Surgery, 245(3), 385–391. http://doi.org/10.1097/01.sla.0000251575.95171.b3
Imrie, K. R., Frank, J. R., & Parshuram, C. S. (2014). Resident duty hours: past, present, and
future. BMC Medical Education, 14(Suppl 1), S1. http://doi.org/10.1186/1472-6920-14-
S1-S1
Jamal, M. H., Rousseau, M. C., Hanna, W. C., Doi, S. A. R., Meterissian, S., & Snell, L. (2011).
Effect of the ACGME Duty Hours Restrictions on Surgical Residents and Faculty: A
Systematic Review. Academic Medicine, 86(1), 34–42.
http://doi.org/10.1097/ACM.0b013e3181ffb264
Jamal, M. H., Wong, S., & Whalen, T. V. (2014). Effects of the reduction of surgical residents’
work hours and implications for surgical residency programs: a narrative review. BMC
Medical Education, 14(Suppl 1), S14. http://doi.org/10.1186/1472-6920-14-S1-S14

SRL IN SURGICAL EDUCATION 96

Jones, A. T., Biester, T. W., Buyske, J., Lewis, F. R., & Malangoni, M. A. (2014). Using the
American Board of Surgery In-Training Examination to predict board certification: a
cautionary study. Journal of Surgical Education, 71(6), e144–8.
http://doi.org/10.1016/j.jsurg.2014.04.004
Jones, J., & Hunter, D. (1995). Consensus methods for medical and health services research.
British Medical Journal, 311(7001), 376.
Jouhari, Z., Haghani, F., & Changiz, T. (2015). Factors affecting self-regulated learning in
medical students: a qualitative study. Medical Education Online, 20(0), 28694.
http://doi.org/10.3402/meo.v20.28694
Kanfer, R., Ackerman, P. L., & Heggestad, E. D. (1996). Motivational skills & self-regulation
for learning: A trait perspective. Learning and individual differences, 8(3), 185-209.
Karabenick, S. A., & Dembo, M. H. (2011). Understanding and facilitating self-regulated help
seeking. New Directions for Teaching and Learning, 2011(126), 33–43.
http://doi.org/10.1002/tl.442
Keeney, S., Hasson, F., & McKenna, H. (2010). Conducting the Research Using the Delphi
Technique. In The Delphi Technique in Nursing and Health Research (Vol. 3, pp. 69–
83). Oxford, UK: Wiley-Blackwell. http://doi.org/10.1002/9781444392029.ch5
Keeney, S., Hasson, F., & McKenna, H. (2010). How to Get Started with the Delphi Technique.
In The Delphi Technique in Nursing and Health Research (Vol. 23, pp. 43–68). Oxford,
UK: Wiley-Blackwell. http://doi.org/10.1002/9781444392029.ch4
Keith, N., & Frese, M. (2005). Self-regulation in error management training: emotion control
and metacognition as mediators of performance effects. Journal of applied psychology,
90(4), 677.

SRL IN SURGICAL EDUCATION 97

Kempenich, J. W., Willis, R. E., Blue, R. J., Fayyadh, Al, M. J., Cromer, R. M., Schenarts, P. J.,
et al. (2016). The Effect of Patient Education on the Perceptions of Resident
Participation in Surgical Care. Journal of Surgical Education, 73(6), e111–e117.
http://doi.org/10.1016/j.jsurg.2016.05.005
Kempenich, J. W., Willis, R. E., Rakosi, R., Wiersch, J., & Schenarts, P. J. (2015). How do
Perceptions of Autonomy Differ in General Surgery Training Between Faculty, Senior
Residents, Hospital Administrators, and the General Public? A Multi-Institutional Study.
Journal of Surgical Education, 72(6), e193–201.
http://doi.org/10.1016/j.jsurg.2015.06.002
Kishiki, T., Lapin, B., Tanaka, R., Francis, T., Hughes, K., Carbray, J., & Ujiki, M. B. (2016).
Goal setting results in improvement in surgical skills: A randomized controlled trial.
Surgery, 160(4), 1028–1037. http://doi.org/10.1016/j.surg.2016.07.022
Kitsantas, A. (2002). Test Preparation and Performance: A Self-Regulatory Analysis. The
Journal of Experimental Education, 70(2), 101–113.
http://doi.org/10.1080/00220970209599501
Landeta, J. (2006). Current validity of the Delphi method in social sciences. Technological
Forecasting and Social Change, 73(5), 467–482.
http://doi.org/10.1016/j.techfore.2005.09.002
Liasion Committee on Medical Education. (2016). Functions and structure of a medical school
(pp. 1–34). Standards for Accreditation of Medical Education Programs Leading to the
MD Degree. Retrieved from http://www.lcme.org/publications
Leff, D. R., Yongue, G., Vlaev, I., Orihuela-Espina, F., James, D., Taylor, M. J., et al. (2017).
“Contemplating the Next Maneuver”: Functional Neuroimaging Reveals Intraoperative

SRL IN SURGICAL EDUCATION 98

Decision-making Strategies. Annals of Surgery, 265(2), 320–330.
http://doi.org/10.1097/SLA.0000000000001651
Lepper, M. R., Corpus, J. H., & Iyengar, S. S. (2005). Intrinsic and extrinsic motivational
orientations in the classroom: Age differences and academic correlates. Journal of
educational psychology, 97(2), 184.
Li, S. T. T., Tancredi, D. J., Co, J. P. T., & West, D. C. (2010). Factors associated with
successful self-directed learning using individualized learning plans during pediatric
residency. Academic pediatrics, 10(2), 124-130.
Linnenbrink-Garcia, L., Middleton, M. J., Ciani, K. D., Easter, M. A., O'Keefe, P. A., & Zusho,
A. (2012). The strength of the relation between performance-approach and performance-
avoidance goal orientations: Theoretical, methodological, and instructional implications.
Educational psychologist, 47(4), 281-301.
Linnenbrink-Garcia, L. & Patall, E. A. (2015). Motivation. In L. Corno & E. Anderman (Eds.),
Handbook of Educational Psychology, 3rd Edition, (pp. 91-103).
Lipsett, P. A., Harris, I., & Downing, S. (2011). Resident self-other assessor agreement:
influence of assessor, competency, and performance level. Archives of Surgery
(Chicago, Ill. : 1960), 146(8), 901–906. http://doi.org/10.1001/archsurg.2011.172
Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and
task motivation. A 35-year odyssey. The American Psychologist, 57(9), 705–717.
Lucieer, S. M., Jonker, L., Visscher, C., Rikers, R. M. J. P., & Themmen, A. P. N. (2016). Self-
regulated learning and academic performance in medical education. Medical Teacher,
38(6), 585–593. http://doi.org/10.3109/0142159X.2015.1073240

SRL IN SURGICAL EDUCATION 99

Lucieer, S. M., van der Geest, J. N., Elói-Santos, S. M., de Faria, R. M. D., Jonker, L., Visscher,
C., et al. (2016). The development of self-regulated learning during the pre-clinical stage
of medical school: a comparison between a lecture-based and a problem-based
curriculum. Advances in Health Sciences Education : Theory and Practice, 21(1), 93–
104. http://doi.org/10.1007/s10459-015-9613-1
Madan, A. K., Harper, J. L., Taddeucci, R. J., & Tichansky, D. S. (2008). Goal-directed
laparoscopic training leads to better laparoscopic skill acquisition. Surgery, 144(2), 345–
350. http://doi.org/10.1016/j.surg.2008.03.016
Mainiero, M. B., & Lourenco, A. P. (2011). The ACGME core competencies: changing the way
we educate and evaluate residents. Medicine and Health, 94(6), 164–166.
Mattar, S. G., Alseidi, A. A., Jones, D. B., Jeyarajah, D. R., Swanstrom, L. L., Aye, R. W., et al.
(2013). General Surgery Residency Inadequately Prepares Trainees for Fellowship:
Results of a Survey of Fellowship Program Directors. Annals of Surgery, 258(3), 440–
449. http://doi.org/10.1097/SLA.0b013e3182a191ca
McGrath, D., Crowley, L., Rao, S., Toomey, M., Hannigan, A., Murphy, L., & Dunne, C. P.
(2015). Outcomes of Irish graduate entry medical student engagement with self-directed
learning of clinical skills. BMC Medical Education, 15(1), 21.
http://doi.org/10.1186/s12909-015-0301-x
McInerney, D. M. (2011). Culture and Self-Regulation in Educational Contexts. In Handbook of
Self-Regulation of Learning and Performance (pp. 442–464). Routledge.
http://doi.org/10.4324/9780203839010.ch28

SRL IN SURGICAL EDUCATION 100

McKenna, H. P. (1994). The Delphi technique: a worthwhile research approach for nursing?
Journal of Advanced Nursing, 19(6), 1221–1225. http://doi.org/10.1111/j.1365-
2648.1994.tb01207.x
Merriam, S. B. (2009). Qualitative case study research. Qualitative research: A guide to design
and implementation, 39-54.
Meyerson, S. L., Teitelbaum, E. N., George, B. C., Schuller, M. C., DaRosa, D. A., & Fryer, J.
P. (2014). Defining the autonomy gap: when expectations do not meet reality in the
operating room. Journal of Surgical Education, 71(6), e64–72.
http://doi.org/10.1016/j.jsurg.2014.05.002
Monaghan, S. F., Blakely, A. M., Richardson, P. J., Miner, T. J., Cioffi, W. G., & Harrington,
D. T. (2012). The reflective statement: a new tool to assess resident learning. The
Journal of Surgical Research, 178(2), 618–622. http://doi.org/10.1016/j.jss.2012.07.034
Moorthy, K., Munz, Y., Adams, S., Pandey, V., Darzi, A., Imperial College--St. Mary's
Hospital Simulation Group. (2006). Self-assessment of performance among surgical
trainees during simulated procedures in a simulated operating theater. The American
Journal of Surgery, 192(1), 114–118. http://doi.org/10.1016/j.amjsurg.2005.09.017
Morgan, R., Kauffman, D. F., Doherty, G., & Sachs, T. (2017). Resident and Attending
Perceptions of Resident Involvement: An Analysis of ACGME Reporting Guidelines.
Journal of Surgical Education, 74(3), 415–422.
http://doi.org/10.1016/j.jsurg.2016.10.012
Murad, M. H., & Varkey, P. (2008). Self-directed learning in health professions education.
Annals of the Academy of Medicine, Singapore, 37(7), 580–590.

SRL IN SURGICAL EDUCATION 101

Murad, M. H., Coto-Yglesias, F., Varkey, P., Prokop, L. J., & Murad, A. L. (2010). The
effectiveness of self-directed learning in health professions education: a systematic
review. Medical Education, 44(11), 1057–1068. http://doi.org/10.1111/j.1365-
2923.2010.03750.x
Murry, J. W., Jr, & Hammons, J. O. (1995). Delphi: A Versatile Methodology for Conducting
Qualitative Research. The Review of Higher Education, 18(4), 423–436.
http://doi.org/10.1353/rhe.1995.0008
Nandagopal, K., & Ericsson, K. A. (2012). An expert performance approach to the study of
individual differences in self-regulated learning activities in upper-level college
students. Learning and Individual Differences, 22(5), 597–609.
http://doi.org/10.1016/j.lindif.2011.11.018
Neumayer, L. (2012). Changing the surgical education paradigm for the 21st century. American
Journal of Surgery, 203(203), 282–286. http://doi.org/10.1016/j.amjsurg.2011.09.008
Norcini, J., & Talati, J. (2009). Assessment, surgeon, and society. International Journal of
Surgery (London, England), 7(4), 313–317. http://doi.org/10.1016/j.ijsu.2009.06.011
Nordin, S. M., Cumming, J., Vincent, J., & McGrory, S. (2007). Mental Practice or
Spontaneous Play? Examining Which Types of Imagery Constitute Deliberate Practice
in Sport. Journal of Applied Sport Psychology, 18(4), 345–362.
http://doi.org/10.1080/10413200600944124
O'Connor, E. S., Mahvi, D. M., Foley, E. F., Lund, D., & McDonald, R. (2010). Developing a
Practice-Based Learning and Improvement Curriculum for an Academic General
Surgery Residency. Journal of the American College of Surgeons, 210(4), 411–417.
http://doi.org/10.1016/j.jamcollsurg.2010.01.017

SRL IN SURGICAL EDUCATION 102

Obi, A., Chung, J., Chen, R., Lin, W., Sun, S., Pozehl, W., et al. (2015). Achieving
Accreditation Council for Graduate Medical Education duty hours compliance within
advanced surgical training: a simulation-based feasibility assessment. American Journal
of Surgery, 210(5), 947–50.e1. http://doi.org/10.1016/j.amjsurg.2015.03.012
Papaleontiou-Louca, E. (2014). Metacognition. In D. C. Phillips (Eds.), Encyclopedia of
educational theory and philosophy. (pp. 523-525). Thousand Oaks, CA: SAGE
Publications
Pavlov, I. P. (1927). Conditioned reflexes.
Pellegrini, C. A. (2006). Surgical education in the United States: navigating the white waters.
Annals of surgery, 244(3), 335.
Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientation in
learning and achievement. Journal of educational psychology, 92(3), 544.
Pintrich, P. R., & DeGroot, E. V. (1990). Motivational and Self-Regulated Learning
Components of Classroom Academic Performance. Journal of Educational Psychology,
82(1), 33–40.
Pintrich, P. R., Smith, D., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the
Motivated Strategies for Learning Questionnaire (Technical Report 91-B-004). The
Regents of the University of Michigan.
Porath, C. L., & Bateman, T. S. (2006). Self-regulation: from goal orientation to job
performance. Journal of applied psychology, 91(1), 185.
Potts, J. R. (2016). Assessment of Competence: The Accreditation Council for Graduate
Medical Education/Residency Review Committee Perspective. Surg Clin North Am,
96(1):15–24.

SRL IN SURGICAL EDUCATION 103

Pugh, C. M., & Britt, L. D. (2013). In-training assessments used in the United States surgical
residency programmes. ANZ Journal of Surgery, 83(6), 460–465.
http://doi.org/10.1111/ans.12217
Ray, J. J., Sznol, J. A., Teisch, L. F., Meizoso, J. P., Allen, C. J., Namias, N., et al. (2016).
Association Between American Board of Surgery In-Training Examination Scores and
Resident Performance. JAMA Surgery, 151(1), 26–31.
http://doi.org/10.1001/jamasurg.2015.3088
Regenbogen, S. E. (2010). Weathering a “perfect storm” in surgical education. Surgery, 148(3),
602–604. http://doi.org/10.1016/j.surg.2010.06.013
Sachdeva, A. K., Bell, R. H., Britt, L. D., Tarpley, J. L., Blair, P. G., & Tarpley, M. J. (2007).
National efforts to reform residency education in surgery. Academic Medicine, 82(12),
1200–1210. http://doi.org/10.1097/ACM.0b013e318159e052
Sachdeva, A. K., Blair, P. G., & Lupi, L. K. (2016). Education and Training to Address Specific
Needs During the Career Progression of Surgeons. The Surgical Clinics of North
America, 96(1), 115–128. http://doi.org/10.1016/j.suc.2015.09.008
Sachdeva, A. K., Flynn, T. C., Brigham, T. P., Dacey, R. G., Jr., Napolitano, L. M., Bass, B. L.,
et al. (2014). Interventions to address challenges associated with the transition from
residency training to independent surgical practice. Surgery, 155(5), 867–882.
http://doi.org/10.1016/j.surg.2013.12.027
Schunk, D. H. (1984). Self-efficacy perspective on achievement behavior. Educational
Psychologist, 19(1), 48-58.

SRL IN SURGICAL EDUCATION 104

Schunk, D. H. (2001). Social Cognitive Theory and Self-Regulated Learning. In B. J.
Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement
theoretical perspectives (2nd ed., pp. 125–152). Mahwah, NJ: Erlbaum.
Schunk, D. H., & Ertmer, P. A. (2000). Self-Regulation and Academic Learning. In M. Zeidner,
P. R. Pintrich, & M. Boekaerts (Eds.), Handbook of Self-Regulation (pp. 631–649).
Burlington, MA: Elsevier. http://doi.org/10.1016/B978-012109890-2/50048-2
Schunk, D., & Usher, E. (2011). Assessing self-efficacy for self-regulated learning. Handbook
of self-regulation of learning and performance, 282-297.
Schunk, D. H., & Zimmerman, B. J. (1997). Developing self-efficacious readers and writers:
The role of social and self-regulatory processes. Reading engagement: Motivating
readers through integrated instruction, 34, 50.
Schunk, D. H., Meece, J. R., & Pintrich, P. R. (2012). Intrinsic and Extrinsic Motivation. In
Schunk, D. H., Meece, J. R., & Pintrich, P. R. (Eds.), Motivation in education: Theory,
research, and applications. Pearson Higher Ed. (p. 239-272)
Sexton, M., Harris, K. R., & Graham, S. (1998). Self-regulated strategy development and the
writing process: Effects on essay writing and attributions. Exceptional Children, 295–
311. http://doi.org/10.1177/001440299806400301
Shiffer, C. (2016). ABS Statement Regarding Residency Redesign. Retrieved from
https://www.absurgery.org/default.jsp?news_resredesign0416 on June 13, 2017.
Sitzmann, T., & Ely, K. (2011). A meta-analysis of self-regulated learning in work-related
training and educational attainment: What we know and where we need to go.
Psychological Bulletin, 137(3), 421–442. http://doi.org/10.1037/a0022777
Skinner, B. F. (1963). Operant behavior. American Psychologist, 18(8), 503.

SRL IN SURGICAL EDUCATION 105

Smith, S. J., Kakarala, R. R., Talluri, S. K., Sud, P., & Parboosingh, J. (2011). Internal medicine
residents' acceptance of self-directed learning plans at the point of care. Journal of
Graduate Medical Education, 3(3), 425–428. http://doi.org/10.4300/JGME-03-03-30
Stefanidis, D., Acker, C. E., & Greene, F. L. (2010). Performance goals on simulators boost
resident motivation and skills laboratory attendance. Journal of Surgical Education,
67(2), 66–70. http://doi.org/10.1016/j.jsurg.2010.02.002
Stefanidis, D., Anderson-Montoya, B., Higgins, R. V., Pimentel, M. E., Rowland, P.,
Scarborough, M. O., & Higgins, D. (2016). Developing a coaching mechanism for
practicing surgeons. Surgery, 160(3), 536–545.
http://doi.org/10.1016/j.surg.2016.03.036
Stefanidis, D., Cochran, A., Sevdalis, N., Mellinger, J., Phitayakorn, R., Sullivan, M., et al.
(2015). Research priorities for multi-institutional collaborative research in surgical
education. American Journal of Surgery, 209(1), 52–58.
http://doi.org/10.1016/j.amjsurg.2014.08.032
Sullivan, M. E. (2003). Self-regulated learning of medical students: Assessment of a social
cognitive model (Doctoral dissertation, University of Southern California).
Tackett, S., Shochet, R., Shilkofski, N. A., Colbert-Getz, J., Rampal, K., Abu Bakar, H., &
Wright, S. (2015). Learning environment assessments of a single curriculum being
taught at two medical schools 10,000 miles apart. BMC Medical Education, 15(1), 105.
http://doi.org/10.1186/s12909-015-0388-0
Teman, N. R., Gauger, P. G., Mullan, P. B., Tarpley, J. L., & Minter, R. M. (2014). Entrustment
of general surgery residents in the operating room: factors contributing to provision of

SRL IN SURGICAL EDUCATION 106

resident autonomy. Journal of the American College of Surgeons, 219(4), 778–787.
http://doi.org/10.1016/j.jamcollsurg.2014.04.019
Torbeck, L., Williams, R. G., Choi, J., Schmitz, C. C., Chipman, J. G., & Dunnington, G. L.
(2014). How much guidance is given in the operating room? Factors influencing faculty
self-reports, resident perceptions, and faculty/resident agreement. Surgery, 156(4), 797–
803. http://doi.org/10.1016/j.surg.2014.06.069
Vyasa, P., Willis, R. E., Dunkin, B. J., & Gardner, A. K. (2017). Are General Surgery Residents
Accurate Assessors of Their Own Flexible Endoscopy Skills? Journal of Surgical
Education, 74(1), 23–29. http://doi.org/10.1016/j.jsurg.2016.06.018
Watson, R. S., Borgert, A. J., O Heron, C. T., Kallies, K. J., Sidwell, R. A., Mellinger, J. D., et
al. (2017). A Multicenter Prospective Comparison of the Accreditation Council for
Graduate Medical Education Milestones: Clinical Competency Committee vs. Resident
Self-Assessment. Journal of Surgical Education.
http://doi.org/10.1016/j.jsurg.2017.06.009
Way, L. W. (1996). General surgery in evolution: technology and competence. The American
Journal of Surgery, 171(1), 2–9. http://doi.org/10.1016/S0002-9610(99)80064-6
Webb, T. P., & Merkley, T. R. (2011). The surgical learning and instructional portfolio: what
residents at a single institution are learning. Journal of Graduate Medical Education,
3(1), 104–108. http://doi.org/10.4300/JGME-D-10-00034.1
Webb, T. P., & Merkley, T. R. (2012). An evaluation of the success of a surgical resident
learning portfolio. Journal of Surgical Education, 69(1), 1–7.
http://doi.org/10.1016/j.jsurg.2011.06.008

SRL IN SURGICAL EDUCATION 107

Webb, T. P., Merkley, T. R., Wade, T. J., Simpson, D., Yudkowsky, R., & Harris, I. (2014).
Assessing competency in practice-based learning: a foundation for milestones in
learning portfolio entries. Journal of Surgical Education, 71(4), 472–479.
http://doi.org/10.1016/j.jsurg.2014.01.019
Weinstein, C. E., Palmer, D., & Schulte, A. C. (1987). Learning and study strategies inventory
(LASSI). Clearwater, FL: H & H Publishing.
Wigfield, A., Tonks, S., & Klauda, S. L. (2009). Expectancy-value theory. Handbook of
motivation at school, 55-75.
Williams, R. G., & Dunnington, G. L. (2004). Accreditation Council for Graduate Medical
Education core competencies initiative: the road to implementation in the surgical
specialties. The Surgical Clinics of North America, 84(6), 1621–46– xi.
http://doi.org/10.1016/j.suc.2004.06.018
Winne, P. H. (1996). A metacognitive view of individual differences in self-regulated learning.
Learning and individual differences, 8(4), 327-353.
Wojcik, B. M., Phitayakorn, R., Lillemoe, K. D., Chang, D. C., & Mullen, J. T. (2017).
Preoperative Disclosure of Surgical Trainee Involvement: Pandora's Box or an
Opportunity for Enlightenment? Annals of Surgery, 265(5), 869–870.
http://doi.org/10.1097/SLA.0000000000002136
Woods, N. N., Mylopoulos, M., & Brydges, R. (2011). Informal self-regulated learning on a
surgical rotation: uncovering student experiences in context. Advances in Health
Sciences Education : Theory and Practice, 16(5), 643–653.
http://doi.org/10.1007/s10459-011-9285-4

SRL IN SURGICAL EDUCATION 108

Yeh, D. D., Hwabejire, J. O., Imam, A., Mullen, J. T., Smink, D., Velmahos, G., & DeMoya, M.
(2013). A survey of study habits of general surgery residents. Journal of Surgical
Education, 70(1), 15–23. http://doi.org/10.1016/j.jsurg.2012.08.006
Yeo, H., Viola, K., Berg, D., Lin, Z., Nunez-Smith, M., Cammann, C., et al. (2009). Attitudes,
training experiences, and professional expectations of US general surgery residents: a
national survey. JAMA, 302(12), 1301–1308. http://doi.org/10.1001/jama.2009.1386
Zimmerman, B. J. (1998). Academic studing and the development of personal skill: A self-
regulatory perspective. Educational Psychologist, 33(2), 73–86.
http://doi.org/10.1207/s15326985ep3302&3_3
Zimmerman, B. J. (2000). Attaining Self-Regulation: A social cognitive perspective. In M.
Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of Self-regulation (pp. 13-39).
Burlington, MA: Elsevier.
Zimmerman, B. J. (2000). Self-Efficacy: An Essential Motive to Learn. Contemporary
Educational Educational Psychology, 25(1), 82–91.
http://doi.org/10.1006/ceps.1999.1016
Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview.
Educational Psychologist, 25(1), 3–17. http://doi.org/10.1207/s15326985ep2501_2
Zimmerman, B. J. (2000). Attaining Self-Regulation. In M. Zeidner, P. R. Pintrich, & M.
Boekaerts (Eds.), Handbook of Self-Regulation (pp. 13-39). Elsevier.
Zimmerman, B. J. (2002). Becoming a Self-Regulated Learner: An Overview. Theory Into
Practice, 41(2), 64–70. http://doi.org/10.1207/s15430421tip4102_2
Zimmerman, B. J. (2008). Investigating Self-Regulation and Motivation: Historical
Background, Methodological Developments, and Future Prospects. American

SRL IN SURGICAL EDUCATION 109

Educational Research Journal, 45(1), 166–183.
http://doi.org/10.3102/0002831207312909
Zimmerman, B. J. (2011). Motivational Sources and Outcomes of Self-Regulated Learning and
Performance. In Handbook of Self-Regulation of Learning and Performance. Routledge.
http://doi.org/10.4324/9780203839010.ch4
Zimmerman, B. J. (2015). Self-Regulated Learning: Theories, Measures, and Outcomes. In
International Encyclopedia of the Social & Behavioral Sciences (pp. 541–546). Elsevier.
http://doi.org/10.1016/B978-0-08-097086-8.26060-1
Zimmerman, B. J., & Bandura, A. (1994). Impact of Self-Regulatory Influences on Writing
Course Attainment. American Educational Research Journal, 31(4), 845.
http://doi.org/10.2307/1163397
Zimmerman, B. J., & Kitsantas, A. (1999). Acquiring writing revision skill: Shifting from
process to outcome self-regulatory goals. Journal of Educational Psychology, 91(2),
241–250. http://doi.org/10.1037/0022-0663.91.2.241
Zimmerman, B. J., & Martinez-Pons, M. (1988). Construct validation of a strategy model of
student self-regulated learning. Journal of educational psychology, 80(3), 284.
Zimmerman, B. J., & Risemberg, R. (1997). Becoming a self-regulated writer: A social
cognitive perspective. Contemporary educational psychology, 22(1), 73-101.
Zimmerman, B. J., & Schunk, D. H. (2001). Reflections on Theories of Self-Regulated
Learning and Academic Achievement. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-
regulated learning and academic achievement theoretical perspectives (2nd ed., pp.
289–308). Mahwah, NJ: Erlbaum.

SRL IN SURGICAL EDUCATION 110

Zimmerman, B. J., & Schunk, D. H. (2011). Self-Regulated Learning and Performance. In
Handbook of Self-Regulation of Learning and Performance. Routledge.
http://doi.org/10.4324/9780203839010.ch1
Zumbrunn, S., Tadlock, J., & Roberts, E. D. (2011). Encouraging self-regulated learning in the
classroom: A review of the literature. Metropolitan Educational Research Consortium,
1-28.

SRL IN SURGICAL EDUCATION 111

Appendix A: Round One Qualtrics Survey

Default Question Block
Self-regulated learning is a process in which the learner takes responsibility for their learning. The purpose of this
study is to understand what self-regulated learning characteristics are most important in surgical residency.

This study design is a Delphi method that seeks to compare and synthesize the opinions of experts, such as
yourself. This initial survey will take approximately 7-10 minutes to complete, and will remain open for the next 3
weeks.
Your participation is completely voluntary and you can withdraw at any time. You can also save and return to the
survey via the link in your email, if needed.

Your survey responses will be coded and will remain confidential. If you have questions at any time about the survey
or procedures, you may contact Cali Johnson at 951-333-0501 or by email at cali.johnson@med.usc.edu . You can
also contact my dissertation committee: Kenneth Yates, EDD at kenneth.yates@usc.edu or Maura Sullivan, PhD at
mesulliv@med.usc.edu .

We want to thank you in advance for completing the survey.

Proceeding with the survey will demonstrate your voluntary consent to participate *
Qualities of a Self-Regulated Resident
To increase their self-regulated learning, please rate how important it is for residents to do the following behaviors
(space is provided below for any comments, suggestions or examples):
Goal Setting: Establish their own learning goals

Comments:
Goal Orientation: Choose goals based upon mastery of the content rather than a need to demonstrate ability

Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 112

Comments:
Value Appraisal: Make choices based upon their interests, as well as the perceived costs and benefits of learning

Comments:
Planning: Analyze and develop a plan to accomplish their learning goals

Comments:
Self-Efficacy: Have confidence in their ability to complete a task
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 113

Comments:
Strategic Use: Use learning strategies to reach their learning goals or complete a task (strategies such as checklists,
mnemonics, or reminders)

Comments:
Outcome Expectations: Visualize and articulate what success looks like to them

Comments:
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 114

Environmental Structuring: Choose settings that enhance their learning

Comments:
To increase their self-regulated learning, please rate how important it is for residents to do the following behaviors
(space is provided below for any comments, suggestions or examples):
Time Management: Prioritize and manage competing demands

Comments:
Attention: Concentrate and direct their focus

Comments:
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 115

Emotional Control: Monitor and regulate their emotions

Comments:
Use of Models: Learn by watching people who are more skilled than themselves

Comments:
Help-Seeking: Know when and how to ask others for help

Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 116

Comments:
Motivation: Understand what motivates them to pursue a learning goal

Comments:
Metacognition: Monitor their thought processes to better understand how they are affected by their own actions

Comments:
Attribution Judgments: Be able to identify reasons for good or bad outcomes

Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 117

Comments:
To increase their self-regulated learning, please rate how important it is for residents to do the following behaviors
(space is provided below for any comments, suggestions or examples):
Monitoring: Track their progress toward goals, and adapt as necessary

Comments:
Performance Calibration: Display appropriate confidence for their level of performance

Comments:
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 118

Self-Reaction: Appropriately persist or be satisfied based upon perceived success, progress, or the need to delay
gratification

Comments:
Innovative: Be open to using new technologies or techniques

Comments:
Evidence-Based: Use evidence-based medicine to inform practice

Comments:
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 119

Collaboration: Work cooperatively and problem-solve with others

Comments:
Effort: Work hard, and persist through adversity

Comments:
Efficiency: Work efficiently to accomplish goals in a timely manner

Comments:
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 120

Utilize Feedback: Receive, analyze and incorporate feedback

Comments:
Please provide other important characteristics of self-regulated resident learners not previously addressed (please
number each one)
Submit
Thank you for participating in this research.
The responses from the expert panel (including you) will be analyzed shortly. You can expect instructions for the next
round in a few weeks.
Your responses will be displayed on the next page and can be downloaded as a PDF. Pressing the back arrow after
the summary will allow you to move back in the survey and change your answers.
Please email me if you have any trouble and need your survey reopened: cali.johnson@med.usc.edu
Not Important Somewhat Important Very Important
0 25 50 75 100

SRL IN SURGICAL EDUCATION 121

Appendix B: Round Two Qualtrics Survey

Introduction
Thank you for participating in this multi-round survey.

The purpose of this final round is to provide validation that I have accurately interpreted your comments,
and that this new framework is supported by the panel as a whole. It should take between 5 and 10 minutes to
complete.

Analysis of round one revealed significant overlap among specific terms. Many panel members stressed that
learning does not exist within a vacuum, the choice between learning environments is limited, and that a resident
learner must also be mindful that patient care takes precedence. Reflection, assessment, and adaptation were also
key themes that emerged.

Of note, some comments seemed to describe the general qualities of a "good resident", but were not specific to the
resident's learning. Therefore, I am including a historical perspective of self-regulated learning.

Self-Regulated Learning Key Elements: (Handbook of SRL and Performance, Schunk & Greene, 2018)
-ACTIVE LEARNER: The learner is behaviorally, cognitively, metacognitively, and motivationally active
-GOAL DIRECTED: Goal setting and striving maintain goal focus and trigger task-relevant strategies
-DYNAMIC: learners monitor progress by modify behavior or strategies
-CYCLIC: goal attainment leads to new goal setting
Self-Regulated Learning in Surgical Education
Please evaluate the following 7 constructs of self-regulated learning for surgical residents. If you disagree with a
specific element, please describe on your reasoning in the comment box.
Self-Awareness: Awareness of one's own skills and deficits, as well as their cognitive, emotional, and motivational
states
A self-regulated learner in surgical residency:

Agree Disagree
- Recognizes current knowledge and skill level, based upon self-assessment and external
evaluation/measures

- Cognizant of personal limitations
- Aware of personal learning strategies that have been previously successful
- Metacognitively aware of personal patterns of thought, emotion, and motivation, and mindful of their
effects

Comments/Suggestions:
Task-Analysis: Appraisal and development of one's goals for an activity, as well as its learning value and expected

SRL IN SURGICAL EDUCATION 122

outcomes
A self-regulated learner in surgical residency:

Agree Disagree
- Appraises task level and need for guidance
- Establishes specific, achievable, and progressive individual goals to guide learning, in alignment with
known standards, such as established curricula or national mandates

- Utilizes mentors to refine goals based upon current performance and trajectory
- Visualizes and articulates success
- Seeks published empiric evidence or consensus data to inform current practice when relevant
- Understands the costs and benefits of task engagement, balance intrinsic interest and external
demand

Comments/Suggestions:
Situational Awareness: Understands and appreciates concurrent responsibilities and patient care, resource and time
availability, and interpersonal aspects of their environment
A self-regulated learner in surgical residency:

Agree Disagree
- Prioritization of competing demands
- Aware of time constraints
- Knows or researches available resources
- Understands team roles, strengths/weaknesses, and dynamic
Comments/Suggestions:
Strategic Planning: Incorporation of personal, task and environmental factors to define goal orientation, mental and
physical resource allocation, self-preparation, and process planning
A self-regulated learner in surgical residency:

Agree Disagree
- Develops specific steps for goal achievement, seeking guidance when necessary
- Identifies learning and performance strategies, such as self-instruction, checkpoints, mnemonics, or
teaching

- Alternates between a performance goal orientation if needing to demonstrate ability, and a mastery
or growth orientation during skill/construct development

- Actively shapes the learning environment by communicating goals, and requesting autonomy,

SRL IN SURGICAL EDUCATION 123

teaching, and/or feedback
Comments/Suggestions:
Progress Evaluation: recognizes discrepancies between goals and current state, and appraises the situation to
determine the cause
A self-regulated learner in surgical residency:

Agree Disagree
- Observes and tracks performance toward a goal
- Identifies gaps in knowledge, skill, or motivation
- Appropriately attributes errors or success to internal and external factors
- Incorporates success and failures into future goal generation
- Determines the need to adhere or adapt current learning strategies or goals, based upon
effectiveness and need for deeper learning

Comments/Suggestions:
Manage Learning and Performance: utilizes individual and systemic strategies for improvement
A self-regulated learner in surgical residency:

Agree Disagree
- Seeks and uses available resources
- Actively seeks assessment and feedback from multiple sources, including peers
- Evaluates quality and relevancy of feedback, and incorporates appropriately
- Differentiates help based upon the demands of the task and specific deficits
- Finds, observes, and learns from peers or experts to develop appropriate mental and motor skills and
coping strategies

- Evaluates current knowledge or processes, and adapts practice when necessary
- Identifies, selects, and adapts learning environments to increase goal attainment as circumstance
permits

- Concentrates and directs focus to the most pressing task, and manages distractions
- Regulates emotional response to maintain focused performance and learning
- Identifes individual weaknesses in motivation, effort, or efficiency
Comments/Suggestions:

SRL IN SURGICAL EDUCATION 124

Goal Attainment and Refinement: uses specific short-term goals to develop mid-term goals, which ultimately build
upon long-term goals or a main driving principle.
A self-regulated learner in surgical residency:

Agree Disagree
- Continually references or builds an overall mission or long-term goal, and uses this to persist through
adversity and find career purpose

- Develops strategic goals to reach long-range goals, with the assistance of mentors and self-reflection
- Creates, refines, and adapts short-term goals depending on environmental circumstances, and
availability of personal and external resources

Comments/Suggestions:
Summary
Thank you for your time and effort in helping us with this important work. Feel free to provide any additional overall
comments or suggestions

Abstract (if available)

Abstract Practice-Based Learning and Improvement is a Core Competency for surgical residents and requires demonstration of self-regulated learning, yet research regarding the development and evaluation of these behaviors in this learning context is limited. This study utilized a two-round modified Delphi survey technique to develop a framework for self-regulated learning in surgical education. A panel of 38 experts in surgical education evaluated self-regulated learning constructs derived from educational, professional and medical literature for their importance in surgical residency. Based upon quantitative data and thematic coding of comments from round one, the researchers adapted self-regulated learning constructs for applicability in the context of surgical residency. These were then reorganized into seven domains: self-awareness, task analysis, situation awareness, strategic planning, progress evaluation, learning and performance management, goal attainment and refinement. Thirty-five experts from round one evaluated the new domains and sub-constructs, with all individual items exceeding the predetermined consensus level of 70%, and 94% agreement overall. Experts in surgical education believed self-regulated learning skills are important in training, but emphasized distinct features of the learning environment for surgical residents: patient care takes priority over learning, learners have little control over their environment, external resources and support are necessary, and metacognition is limited even among practicing faculty surgeons. This novel framework provides an initial guide for directing curriculum and further study into self-regulated learning in surgical education.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Digital portfolios for learning and professional development: a faculty development curriculum

PDF

Employing cognitive task analysis supported instruction to increase medical student and surgical resident performance and self-efficacy

PDF

Incorporating teacher self-care into teacher education: a curriculum for pre-service teachers to increase teacher wellness and prevent teacher burnout

PDF

An exploratory study on flipped learning and the use of self-regulation amongst undergraduate engineering students

PDF

Building a framework for guiding fundraiser learning and practice in higher education advancement

PDF

High school curriculum: Self-regulation for the secondary student

PDF

The relationship of students' self-regulation and self-efficacy in an online learning environment

PDF

The impact of the mindful method Youth Empowerment Seminar (YES!) on students' self-efficacy, self-regulation, and academic performance for becoming college- and career-ready

PDF

Approaches to encouraging and supporting self-regulated learning in the college classroom

PDF

Mind, motivation, and meaningful learning: A cognitive science approach to learning how to learn

PDF

Metacognition and self-regulation strategies to support high school student athletes

PDF

A marriage and family therapy trainee curriculum: college student success in academic self-regulation

PDF

Supporting faculty in preparing entrepreneurship: an exploration in the context of active learning

PDF

Teachers' experiences implementing social and emotional learning in the elementary classroom

PDF

Educational technologies as tools for self-regulated learning for students with autism in post-secondary settings

PDF

Facilitation of authentic teaching and learning in a problem-based learning (PBL) environment

PDF

Impact of technology on teaching and learning practices at high‐technology use K-12 schools: a case study

PDF

The academic implications of providing social emotional learning in K-12: an evaluation study

PDF

Institutional advancement in higher education: managing gift officer performance and turnover

PDF

Strategic support for philanthropic fundraising: a needs analysis of development officers in higher education

Asset Metadata

Creator Johnson, Cali Elyse (author)

Core Title Building a framework for self-regulated learning in surgical education

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 08/11/2020

Defense Date 05/30/2018

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag core competency,Delphi,framework,metacognition,OAI-PMH Harvest,practice based learning and improvement,self-regulated learning,surgical residency

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Yates, Kenneth (committee chair), Muraszewski, Alison (committee member), Seli, Helena (committee member), Sullivan, Maura (committee member)

Creator Email calijohn@usc.edu,calijohnsonMD@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-69655

Unique identifier UC11671738

Identifier etd-JohnsonCal-6729.pdf (filename),usctheses-c89-69655 (legacy record id)

Legacy Identifier etd-JohnsonCal-6729.pdf

Dmrecord 69655

Document Type Dissertation

Format application/pdf (imt)

Rights Johnson, Cali Elyse

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA