Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Using cognitive task analysis to capture palliative care physicians' expertise in in-patient shared decision making
(USC Thesis Other)
Using cognitive task analysis to capture palliative care physicians' expertise in in-patient shared decision making
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Running head: COGNITIVE TASKANALYSIS
1
USING COGNITIVE TASK ANALYSIS TO CAPTURE PALLIATIVE CARE
PHYSICIANS’ EXPERTISE IN IN-PATIENT SHARED DECISION MAKING
by
Deidre Lee Larson
A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2015
Copyright 2015 Deidre Lee Larson
COGNITIVE TASK ANALYSIS
2
Dedication
This dissertation is dedicated to my parents and my daughter. To my parents, Leland and
Elizabeth, I am extremely grateful for your support and words of encouragement. Papa, when
you tell me how proud you are of me, it motivates me to continue to move forward. Nana, you
have always been my “rock,” and I would not be where I am without your support. To my
daughter, Britnee, I appreciate your understanding and independence. You are a beautiful and
smart young woman and I appreciate you always allowing me to pursue my dreams. I love you
and I am proud of you!
COGNITIVE TASK ANALYSIS
3
Acknowledgements
I would like to acknowledge Dr. Kenneth Yates, who has served as an educator, mentor
and great motivator around this challenging topic. Thank you for the time you spent meeting
with me to interview experts, as well as discussing, and sometimes debating, the content of this
dissertation. Our conversations made me think about the deeper aspects of CTA and palliative
care. Moreover, I am forever thankful for your patience in trying to teach not only the content of
CTA, but also the process of writing a dissertation. Your encouragement and telling me to “just
take a stab at it,” and then knowing when I had reached cognitive overload, demonstrates the
strength of your ability as an educator and coach. Because of you, I am a true believer in
Cognitive Task Analysis, and its benefits for effective and efficient instruction in all content
areas. I would also like to thank Dr. Susan Enguidanos and Dr. Suan Wang for their willingness
to take the time, and energy to be meet with me and be part of my dissertation committee.
I would also like to acknowledge the four palliative care experts who participated in this
study. I deeply appreciate the time you sacrificed to meet with me for the interview processes in
this study. Your dedication and contributions to the field of palliative care are genuine and will
make a difference in helping patients and family members understand the end-of-life process,
and move forward with a sense of peace.
To the Thursday night cohort and my CTA cohort members, Christine Gayle Corpus,
Chad Hammitt, Megan McGuiness, Douglas Weiland, Kari Cole, Charlotte Ann Garcia, Milo
Jury, Nicholas Lim, Acquillahs Muteti Mutie, and Judith Franco thank you for your support!
The Saturday afternoons were a time to learn and have a bit of fun, and I will always be grateful
for the time we shared together. To my new friend Judith, thank you for being a fellow trouble
maker and making me feel like I wasn’t so alone and lost.
COGNITIVE TASK ANALYSIS
4
Finally, thank you to my friends and family. Michelle, your support through the tough
times at work and stimulating academic conversations kept me going when I wanted to give up.
Chris, thank you for the time away that allowed me to write with no distractions. Brenna, in
your own way you have pushed me to be something great, while at the same time, allowed me to
be me, and for that, I thank you.
COGNITIVE TASK ANALYSIS
5
Table of Contents
Dedication 2
Acknowledgements 3
List of Tables 7
List of Figures 8
Abstract 10
Chapter One: Overview of the Study 11
Statement of the Problem 11
Purpose of the Study 16
Methods of the Study 16
Definition of Terms 17
Organization of the Study 19
Chapter Two: Literature Review 20
Shared Decision-Making in Palliative Care 20
Addressing the Needs of an Aging and Chronically Ill Population 20
Addressing the Needs of Younger Populations with Chronic Conditions 22
Medical Complexity and Palliative Care 23
Communication and Shared Decision-Making in Palliative Care 26
Palliative Care Education and Training 29
Summary 31
Using Subject Matter Experts to Train Non-experts 32
Knowledge Types 34
Declarative Knowledge 34
Procedural and Conditional Knowledge 36
Automaticity 38
Expertise 41
Building Expertise 42
Consequences of Expertise 44
Expert Omissions 45
Cognitive Task Analysis (CTA) 46
CTA History 46
Cognitive Task Analysis Methodology 47
Effectiveness of CTA 49
Benefits of CTA Instruction and Design 50
Chapter Three: Methods 53
Participants 53
Phase 1: Collect Preliminary Knowledge 55
Phase 2: Identify Knowledge Types 55
Phase 3: Apply knowledge elicitation techniques 56
Phase 4: Data analysis 58
Phase 5: Formatting the results 59
Data Analysis for Question 2 61
Spreadsheet Analysis 61
Chapter Four: Results 62
Research Questions 62
COGNITIVE TASK ANALYSIS
6
Research Question One 62
Research Question Two 70
Chapter Five: Discussion 74
Overview of the Study 74
Selection of Experts 75
Selection of the Cognitive Task 77
Collection of Data 77
Discussion of Findings 80
Research Question 1 80
Research Question 2 83
Limitations 85
Confirmation Bias 85
Internal Validity 86
External Validity 86
Implications 87
Future Research 88
Conclusion 90
References 91
Appendix A: Interview Protocol 108
Appendix B: Inter-Rater Reliability Code Sheet for SME A 111
Appendix C: Job Aid for Developing a Gold Standard Protocol 112
Appendix D: SME A Individual Protocol Flowchart 114
Appendix E: Gold Standard Protocol 119
Appendix F: Incremental Coding Spreadsheets 124
COGNITIVE TASK ANALYSIS
7
List of Tables
Table 1: Cumulative Action and Decision Steps Captured for Each SME 66
Table 2: Additional Expert Knowledge Captured, in Action and Decision Steps, During
Follow-up Interviews 68
Table 3: Number and Percentage of Action and Decision Steps that are Highly Aligned,
Partially Aligned, Slightly Aligned, Not Aligned 69
Table 4: Total Action and Decision Steps, or Expert Knowledge, Omissions by SME when
Compared to the Gold Standard Protocol 72
COGNITIVE TASK ANALYSIS
8
List of Figures
Figure 1: Visual Representation of CTA 3i + 3r Method of Data Collection 60
Figure 2. Example of Aggregating Action and Decision Steps for the Preliminary Gold Standard
Protocol (PGSP). 64
Figure 3. Number of Action Steps, Decision Steps, and Action and Decision Steps for SME A,
SME B, and SME C Captured through CTA. 67
Figure 4. Percentage of Action and Decision Steps that are Aligned with the Final Gold
Standard Protocol. 70
Figure 5. Total non-repeating action and decision steps from the CTA process represented in the
final gold standard protocol. 73
COGNITIVE TASK ANALYSIS
9
List of Abbreviations
ABIM American Board of Internal Medicine
ABMS American Board of Medical Specialties
ACA Affordable Care Act
ACGME Accreditation Council of Graduate Medical Education
CDM: Critical Decision Method
CPP: Concepts, Processes, and Principles
CTA: Cognitive Task Analysis
GSP: Gold Standard Protocol
IRB: Institutional Review Board
LCME Liaison Committee for Medical Education
MDL: Model of Domain Learning
PARI: Precursors, Action, Result, and Interpretation
PGSP: Preliminary Gold Standard Protocol
SDM Surrogate Decision Maker
SME: Subject Matter Expert
WHO World Health Organization
COGNITIVE TASK ANALYSIS
10
Abstract
The purpose of this study was to apply Cognitive Task Analysis (CTA) methods to capture the
tacit and unconscious action and decision steps of palliative care physicians as they engage in
shared-decision making at the end of life. Additionally, the study sought to identify the number
and percentage of critical action and decisions steps omitted by experts as they describe how
they perform this complex cognitive task. CTA has been shown to be effective at acquiring the
automated knowledge of experts as they perform cognitively complex tasks. Shared decision-
making has been shown to have a positive impact on patients and family member’s perceived
quality of medical care and sense of contentment at the end of life. Three expert palliative care
physicians were and interviewed to capture the action and decision steps they use when engaging
with patients and their family members during shared decision-making in end-of-life
conversations. The interview data was aggregated into a preliminary gold standard protocol and
reviewed by a fourth palliative care physician expert. Overall, the study found there were five
main steps or procedures experts completed when engaging in shared decision-making. The
study’s findings indicate that palliative care physician experts omitted 53.19% of action steps
and 66.67% of decision steps when compared to a final gold standard protocol, supporting the
research on expert omissions. The expert knowledge and skills captured by this CTA may be
used for further training in medical schools, residency and fellowship programs as they relate to
palliative care, patient-centered communication, and shared decision-making.
COGNITIVE TASK ANALYSIS
11
CHAPTER ONE: OVERVIEW OF THE STUDY
Statement of the Problem
Educational and technological advances in medicine have extended an individual’s life
expectancy (Bunker, 2001). In fact, since the late 1900s, there has been a dramatic increase in
the amount of medical research and new treatments, and these new medical technologies have
been shown to prolong lives and increase a person’s life expectancy. For example, the
treatments developed for heart disease, hypertension, diabetes, end stage kidney disease,
appendicitis, influenza, and pneumonia have increased the life expectancy for individuals
diagnosed with these conditions. In addition, screening for life limiting forms of cancer and
chronic conditions, as well as preventative clinical services, such as immunizations, have been
shown to increase life expectancy (Bunker, Frazier, & Mosteller, 1994). With each new medical
advance and cutting edge treatment, there are a multitude of choices for the patient and the
treating physician. Physicians are required to discuss the risks and benefits of treatments, and
patients are asked to make choices given the risks and benefits for each treatment option. To
illustrate, patients diagnosed with chronic conditions or life-limiting illnesses can make choices
about undergoing surgery or having treatments that may extend their life, but may also limit their
quality of life. Indeed, medical advances have created a wealth of new treatment options that can
increase life expectancies, while at the same time, the educational and technological medical
achievements have created a complex medical system, with many different treatment choices
that may prolong life, but may not be beneficial to the patients or families (Murphy, Xu, &
Kochanek, 2010; Landefeld, Winker, & Chernof, 2009).
Along with the array of life extending medical treatment options patients have, many
patients have the option to receive palliative care. In fact, patients and family members can
COGNITIVE TASK ANALYSIS
12
utilize palliative care services concurrent to life extending treatments, or patients and family
members can choose palliative care, foregoing life-saving treatments (Rocque, & Cleary, 2013).
Palliative care is a specialized approach to treating people with serious illnesses and diseases,
generally delivered by an interdisciplinary team that includes a physician, nurse, social worker,
and chaplain (Clark, 2007). While palliative care arose out of the development of hospice
programs, palliative care is a more comprehensive medical specialty, designed and intended for
patients with serious illnesses at any stage of their condition. Essentially, the goal of palliative
care is to enhance the quality of life for individuals with serious illness, through pain and
symptom control, comfort, and communication and decision-making.
Compared to other medical subspecialties, palliative care is fairly new, with the World
Health Organization officially defining palliation in 1989, and palliative care being recognized as
a subspecialty by the American Board of Medical Specialties in 2006 (Clark, 2007). According
to the WHO, palliative care is a distinctive approach to treating patients and families with life
threatening diseases (WHO, 1989). This specialized approach works by treating the physical
symptoms and pain associated with life-threatening illnesses, as well as the psychological,
spiritual and communication needs of the patient and family members. In fact, palliative can be
used in conjunction with other life prolonging treatments to relieve pain and control the
symptoms of the illness, in addition to the symptoms related to the treatments and therapies.
Furthermore, palliative care works with patients and family members with the psychological and
spiritual aspects of death and dying.
Because palliative care is considered specialized care, and it is a relatively new field of
medicine, effective training of medical personnel in the area of palliative care is essential. One
of the most critical components of palliative care is effective communication and shared
COGNITIVE TASK ANALYSIS
13
decision-making (Back, Arnold, Baile, Fryer-Edwards, Alexander, Barley, & Tulsky, 2007; Lo,
Quill, Tulsky, 1999). For effective palliative care team consultations, there needs to be patient-
centered communication and shared decision making, during which a caregiver, often a
physician, informs the patient and their family of the diagnoses, and together, the physician,
patient and family identify the course of treatment based on the patient’s values and preferences
(Freytag, 2012). When patients, family members and physicians engage in effective patient-
centered communication and shared decision-making, palliative care services are utilized more
often, leading to greater satisfaction and significant improvement in the quality of life for the
patients and family members (Freytag, 2012; Temel, et al., 2010). Moreover, shared decision
making at the end of life, specifically physicians providing appropriate information about a
patient’s prognosis and required care, in addition to the alignment of possible treatment options
to the values and preference of the patient, can significantly influence how a patient reacts and
adjusts to further medical care and the patient’s physical and psychological quality of life
(Enguidanos, Housen, Penido, Mejia, & Miller, 2014).
Beyond the benefits of effective patient-centered communication and shared decision-
making for patients and family members at the end of life, the Affordable Care Act (ACA)
includes provisions that focus care around shared decision-making (Kocher, Emanuel, &
DeParle, 2010; Oshima Lee & Emanuel, 2013). Specifically, a section of the ACA seeks to
facilitate shared decision-making by providing funding for the development of decision aids for
patients, as well as consensus-based standards for shared decision-making between physicians
and patients (Oshima Lee & Emanuel, 2013). Furthermore, the ACA delineates shared decision-
making as an important aspect of patient care by specifying that the patient and family members
be included in understanding the benefits and risks of treatment options and developing shared
COGNITIVE TASK ANALYSIS
14
goals for treatment, interventions, and outcomes. Accordingly, the ACA provides physicians
with incentives to deliver better information and change clinical practices, including shared
decision-making with patients regarding what medical approaches and treatment goals best suit
the patient, given the patient’s values and preferences.
With recommendations from the Accreditation Council for Graduate Medical Education,
medical schools are required to teach basic communication skills, with coursework typically
including lectures, care actors that portray patient encounters, and direct feedback from faculty
(Levinson, Lesser, Epstein, 2010). However, after medical school, education and training on
effective patient-centered communication is significantly reduced, with more focus placed on
teaching patient management along with diagnostic and treatment skills, technology, and science.
Although recent requirements have increased training in physician-patient communication during
medical schools and residency programs, communicating complex life-altering information,
including, shared decision making and the need for palliative care, often requires more in-depth
education and training (Bickel-Swenson, 2007; Hammel, Sullivan, Block, & Twycross (2007;
Van Aalst-Cohen, Riggs, & Byock, 2008). And while the ACA incentivizes patient-centered
communication and shared decision-making, there are little training opportunities provided to
practicing physicians. In fact, many practicing physicians do not realize the importance of such
skills, or are forced to learn patient-centered communication and shared decision-making skills
on their own through trial and error (Levinson, Lesser, Epstein, 2010).
Given that the ACA is requiring patient-centered communication and shared decision-
making, and that communication between extremely ill patients and palliative care physicians
can be extremely complex and highly emotional, there is a need for more effective and efficient
training. Indeed, there are limited models for training practicing physicians in the complex task
COGNITIVE TASK ANALYSIS
15
of shared decision-making, with the most common training methods being observing more senior
physicians, who are considered experts, or by practicing with actors representing patients with
given scenarios, with feedback from senior physicians who are considered experts (Levinson,
Lesser, Epstein, 2010; Levinson & Pizzo, 2011).
The main source of training and education for medical personnel relies on the expertise of
other, more senior physicians. However, research shows that educational programs that rely on
the expertise of others for this type of training are not the most effective or efficient type of
education (Clark, Feldon, van Merrienboer, Yates, & Early, 2008; Sullivan, Yates, Baker, &
Clark, 2010). In fact, research shows that experts may omit up to 70% of the critical information
novices require for successfully completing a complex task. Expert physicians are at an
increased disadvantage because their knowledge and skills are highly automated, making it
difficult for them to describe how they perform complex cognitive tasks, such as patient-centered
communication and shared decision-making. One characteristic of expertise is that experts are
able to automate their action and decision steps for complex cognitive tasks allowing them free-
up mental resources so they can manage more difficult or unusual problems.
Cognitive Task Analysis (CTA) refers to a set of methods that have been shown to be
effective in capturing the knowledge and skills experts use to solve difficult problems and
perform complex tasks (Clark et al., 2008). CTA is a method of qualitative research that uses a
semi-structured interview technique to elicit knowledge from three to five qualified experts.
During the semi-structured interview, procedural and conditional knowledge is captured
regarding the tacit action and decision steps of an expert as they perform a cognitively complex
task. Engaging with patients and family members in patient-centered communication and shared
decision-making during end-of-life discussions is a highly cognitively complex task. Thus, the
COGNITIVE TASK ANALYSIS
16
knowledge from the CTA can be used to develop a gold standard protocol that could potentially
be used for the training and education of medical personnel.
Purpose of the Study
The purpose of this CTA study is to capture the automated knowledge of expert
physicians when they describe how to engage patients in shared decision-making. Given the
complexity and critical importance of shared decision making, and its implications for a patient’s
future palliative care, developing a systematic approach to capturing expertise in communication
and shared decision-making may have significant consequences for palliative care and future
medical training.
The research questions that guide this study are:
1. What are the essential decision and actions steps palliative care physicians recall when
they describe how they involve patients and family members in in-patient shared-decision
making about end-of-life treatment?
2. What percentage of critical action and/or decision steps, when compared to a gold
standard, do expert palliative care physicians omit when they describe how they conduct
shared decision-making with patients during end-of-life communication?
Methods of the Study
A Cognitive Task Analysis (CTA) was used to identify the explicit, observable and
implicit, unconscious procedural knowledge of expert palliative care physicians as they engage
in patient centered-communication and shared decision-making with patients and families. The
palliative care physicians consisted of four experts from hospitals in Southern California. Three
subject matter experts (SME) participated in an extensive interview to elicit the unconscious
action and decision steps used as they engage in shared decision-making, and a fourth SME was
COGNITIVE TASK ANALYSIS
17
used to verify the data collected from the three experts. The CTA followed a five-step process,
as suggested by Clark et al. (2008):
1. A preliminary phase or preliminary knowledge collection frequently called
“bootstrapping”, is completed to build familiarity with the topic of the study.
2. The second stage identifies knowledge representations, including declarative,
procedural and conditional knowledge.
3. The third stage is the application of knowledge elicitation techniques that are best
suited for the study’s purpose.
4. The fourth stage is the review, verification, analysis and possible modification of
the knowledge elicited from experts.
5. The fifth stage formats the results from the analysis as a basis for an expert
system or expert cognitive model.
Definition of Terms
The following terms related to cognitive task analysis are defined as suggested by
Zepeda, McZeal (2014).
Adaptive expertise: When experts can rapidly retrieve and accurately apply appropriate
knowledge and skills to solve problems in their fields or expertise; to possess cognitive
flexibility in evaluating and solving problems (Gott, Hall, Pokorny, Dibble, & Glaser, 1993;
Hatano & Inagaki, 2000).
Automaticity: An unconscious fluidity of task performance following sustained and
repeated execution; results in an automated mode of functioning (Anderson, 1996a; Ericsson,
2004).
COGNITIVE TASK ANALYSIS
18
Automated knowledge: Knowledge about how to do something: operates outside of
conscious awareness due to repetition of task (Wheatley & Wegner, 2001).
Cognitive load: Simultaneous demands placed on working memory during information
processing that can present challenges to learners (Sweller, 1988).
Cognitive tasks: Tasks that require mental effort and engagement to perform (Clark &
Estes, 1996).
Cognitive task analysis: Knowledge elicitation techniques for extracting implicit and
explicit knowledge from multiple experts for use in instruction and instructional design (Clark et
al., 2008; Schraagen, Chipman, & Shalin, 2000).
Conditional knowledge: Knowledge about why and when to do something; a type of
procedural knowledge to facilitate the strategic application of declarative and procedural
knowledge to problem solve (Paris, Lipson, & Wixson, 1983).
Declarative knowledge: Knowledge about why or what something is; information that is
accessible in long-term memory and consciously observable in working memory (Anderson,
1996a; Clark & Elen, 2006).
Expertise: The point at which an expert acquires knowledge and skills essential for
consistently superior performance and complex problem solving in a domain; typically develops
after a minimum of 10 years of deliberate practice or repeated engagement in domain-specific
tasks (Ericsson, 2004).
Procedural knowledge: Knowledge about how and when something occurs; acquired
through instruction or generated through repeated practice (Anderson, 1982; Clark & Estes,
1996).
COGNITIVE TASK ANALYSIS
19
Subject matter expert: An individual with extensive experience in a domain who can
perform tasks rapidly and successfully; demonstrates consistent superior performance or ability
to solve complex problems (Clark et al., 2008).
Organization of the Study
Chapter Two of this study will review the literature in two central sections. In the first
section, the literature review will examine relevant literature related to palliative care, and
patient-centered communication and shared decision-making. In the second section of the
literature review, the Cognitive Task Analysis method of knowledge elicitation is reviewed. In
Chapter Three, the methods used for the study will be described along with how the approach of
using a CTA sought to answer each of the research questions. Chapter Four will review the
results of the study for each research question. Finally, Chapter Five will discuss the findings
and implications for patient-centered communication and shared decision-making, as well as
discuss the need for future research in the area of palliative care and shared decision-making.
COGNITIVE TASK ANALYSIS
20
CHAPTER TWO: LITERATURE REVIEW
Shared Decision-Making in Palliative Care
According to the United Nations (2012), the current projection for the future world
population indicates a large increase in the number of individuals over the age of 65. And, as the
age of the population increases, so does the need for medical education and training to treat
chronic and terminal diseases. Palliative care, a specialized approach to caring for the
chronically and terminally ill, continues to emerge as a beneficial medical practice for this
population (Reyes-Ortiz, Williams, & Westphal, 2014). The following review of literature will
identify the needs of individuals with chronic illnesses and life-limiting conditions, the mounting
complexity of the medical care system and demand for palliative care, as well as the necessity
for increased training for physicians involved with palliative care. In addition, the literature
review will examine the challenges of effective communication and shared decision making at
the end of life, how medical education relies on experts to train physicians in patient-centered
communication and shared-decision making, and the requirements of shared decision making as
a condition of the Affordable Care Act (ACA). Finally, the literature review will define
Cognitive Task Analysis (CTA) and distinguish it as an effective method for capturing expert
knowledge, and developing instruction and training protocols.
Addressing the Needs of an Aging and Chronically Ill Population
Globally, the average life span of the world’s population is increasing. Data indicates
that by the year 2047, the number of people over the age of 60 will outnumber the amount of
children, and by 2050, one in five individuals will be age 60 or older (United Nations, 2012).
The increase in the age of the population will bring about challenges for the health care system
as well as for health care providers. Chronic conditions, including hypertension, renal failure,
COGNITIVE TASK ANALYSIS
21
Alzheimer’s disease, dementia, and cancer are associated with longer life expectancy.
Specifically, heart and lung disease make up two of the top three causes of death, and
Alzheimer’s disease, kidney disease, and diabetes, all diseases indicated for older individuals, are
in the top eight of the fifteen leading causes of death (Murphy, Xu, & Kochanek, 2010).
Accordingly, medical care, along with medical professionals, will be required to shift the focus
from traditional curative methods of treatment and the prolongation of life, to the preservation of
the quality of life, and aligning treatment to an individual patient’s needs and goals (Metzger,
Norton, Quinn, & Gramling, 2013; Yong, bin Main, Tan, Low, & Quah, 2013).
Research indicates that as individuals age and develop chronic illnesses, their needs are
not being addressed adequately (Landefeld, Winker & Chernof, 2009; Winer & Angelis, 2010).
Moreover, there is evidence that there is a significant increase in the physical, mental, emotional
and financial burdens on the patient, family members, and caretakers, especially as the patient
declines clinically and functionally (Fitzsimons, et al., 2007; Lorenz, et al, 2008). Given the
growing age of the world population, along with the unmet needs of the patients and caretakers
with chronic and terminal illnesses, it is imperative that the medical community change to
address those needs.
Though technological and educational advances have proven to be beneficial in
improving life expectancies, there are medical challenges in supporting an individuals’ quality of
life, and at the same time, containing medical costs. Indeed, studies show that today a patient
with a chronic illness can benefit from better access to physicians, advanced treatments and
therapies, and updated medical facilities (United Nations, 2012). For example, advanced
technologies have been proven to assist in the diagnosis, treatment, and monitoring of patients
with chronic illnesses (Cutler, Rosen & Vijan, 2006). In effect, technological and educational
COGNITIVE TASK ANALYSIS
22
advances have proven to extend, and sometimes improve, the long-term quality of life (Cutler,
2007). However, research also indicates that while medical spending for technology and
improved care has increased, and proven to be of value, medical treatments are not always as
cost effective for the elderly (Orzag & Ellis, 2007; Smith & Hillner, 2011; Tilden & Thompson,
2009). In fact, studies show that caring for an aging person’s medical needs requires more
individualized attention, thus increasing the actual cost for medical treatment (Cutler, Rosen &
Vijan, 2006). Furthermore, a study completed by Cutler, Rosen, Vijan (2006) found that the cost
value for caring for persons over the age of 65 actually exceeds the cost for every additional year
of life expectancy. Additionally, Orszag & Ellis (2007) suggest that in certain cases, patients
with chronic illness would have the same medical outcomes with less expensive care. In effect,
advances in medicine have proven to be beneficial, and at the same time they have presented
complex challenges and financial concerns for patients, families, the health care community, and
society as a whole.
Addressing the Needs of Younger Populations with Chronic Conditions
Similar to older adults with chronic and terminal conditions, pediatric patients and their
families also require specialized care when they are diagnosed with a life limiting or life
threatening diseases (Zernikow, Michel Craig & Anderson). And like older adults, educational
and technological advances have improved the quality of treatment and extended the lives of
children with chronic illnesses. While the total number of deaths in children and adolescents are
not as significant as those of older adults, in 2011, there were almost 21,000 deaths in U.S.
children ages 1-19 (Hamilton, Hoyert, Martin, Strobino & Guyer, 2012). Of those pediatric
patients, almost a quarter of them were diagnosed with life threatening and life limiting medical
conditions. Research shows that the complexity of pediatric medical conditions, and the
COGNITIVE TASK ANALYSIS
23
multifaceted needs of the patients and families, especially regarding implications of a child’s
death, requires special considerations (Moody, Siegel, Scharbach, Cunningham & Cantor, 2011).
Specifically, families and pediatric patients with life limiting conditions require physician
and patient/family centered communication, opportunities to engage in shared decision making,
specialized psychosocial care, legal and ethical considerations, and bereavement counseling.
Furthermore, a medical review conducted by Himelstein, Hilden, Boldt and Weissman (2004)
identified the benefits and barriers to providing specialized medical care to pediatric patients and
families, regardless of the potential curative treatments available. While the use of
interdisciplinary medical care, including palliation, is still rare, the medical review identified
specific indicators of providing pediatric patients and families with life limiting conditions a
multidisciplinary palliative care team. These indicators include improved pain management and
better symptom control by the pediatric patient, as well as greater satisfaction with the quality of
communication and shared decision making between the medical community, pediatric patients
and family members. Medical care and decisions about medical treatments are complex for
pediatric patients. Fundamentally, effective patient care, whether it is an adult or a child,
requires a combination of a multidisciplinary medical approach balanced with the psychosocial
and physical and needs of patients and their families.
Medical Complexity and Palliative Care
Just as human beings are dynamic and complex, so too is the medical system. The
complexity of the medical system, with highly specialized treatment, has created fragmentation
and a lack of communication during the various stages of a serious illness. By introducing
palliative care during any stage of chronic illness or life limiting disease, the complex medical
COGNITIVE TASK ANALYSIS
24
needs of the patient, family, and caretakers can be better addressed (Fitzimons et al., 2007;
Temel et al., 2010).
Palliative care is a specialized medical approach to treating people with serious illness
and diseases. A palliative care team is generally made up of an interdisciplinary team that
includes a specially trained physician, nurse, social worker, and chaplain (Clark, 2007). Initially,
palliative care developed from the need to reform and improve oncological care and the
expansion of hospice programs. Since its inception, palliative care has advanced worldwide. In
1982, the World Health Organization (WHO) developed protocols for the use of pharmaceuticals
to assist in pain relief, and in 1989
the WHO formally defined palliative care. Similarly,
palliative care has progressed in the United States. To illustrate, in 1982 Medicare first provided
funding for hospice programs, and since then, it is the most rapidly growing benefit accessed by
Medicare patients (Clark, 2007; Connor, 2007). Equally important to the growth of palliative
care in the U.S., was the 2006 American Board of Medical Specialties (ABMS) recognition of
palliative care as a subspecialty (Clark, 2007). Further evidence of the growth of palliative care
is confirmed by the growth in palliative care within hospital settings. In fact, data indicates that
in 2006, 52.8% of hospitals in the United States with fifty beds or more had palliative care
available to patients (Goldsmith, Dietrich, Qingling & Morrison, 2008). Finally, the growth in
palliative care in the U. S. is demonstrated by the growth in the use of palliative care and hospice
services for all types of chronic and terminal illnesses (NHPCO, 2014).
Research studies show that by introducing palliative care during the treatment of a
serious illness, patients and families show greater satisfaction with the health care system and
have a greater quality of life (Brumley, Enguidanos, Jamison, Seitz, Morgenstern, Saito,
Mcllewane, Hillary and Gonzalez, 2007; Fitzsimons, et al., 2007; Higginson, Finlay, Goodwin,
COGNITIVE TASK ANALYSIS
25
Hood, Edwards, & Norman, 2003). For example, Brumley, et al., (2007) showed that patients
receiving in-home palliative care were less likely to visit the emergency room, be admitted
during to the hospital, and overall, palliative care patients showed greater satisfaction with the
medical care they received. Further research indicates that patients with access to palliative care
during the end stages of life had greater pain and symptom management, leading to less hospital
stays and lower rates of depression and feelings of isolation (Fitzsimons, et al., 2007; Higginson,
Finlay, Goodwin, Hood, Edwards, & Norman, 2003). Additionally, a study by Temel, et al.,
(2010) found that lung cancer patients receiving early palliative care concurrent with oncology
care demonstrated significant improvements in their quality of life, and more importantly,
patients receiving palliative care concurrent with oncology care lived an average of two months
longer than the control group that received standard oncology care without palliative care. As
such, it appears that patients and family members receiving palliative care demonstrate overall
better physical, emotional and mental quality of life than individuals without palliative care.
Finally, the use of palliative care and palliative care teams has been shown to be
beneficial in navigating the complex medical systems at the end-of-life (Fitzsimons et al., 2007;
Temel et al., 2010). Communication is an important factor in determining treatment decisions
for the patient. Research studies show that there is incongruity between the complex medical
decisions between medical personnel and patients and family members. For instance, patients
and family members may not receive the beneficial physical and psychological resources a
palliative care team can provide during end-of-life care. In fact, Fitzsimons et al., (2007) and
Temel et al., (2010) identify communication as a key factor in shared decision making
surrounding curative medicine with medical specialists and specialized medical therapies, and
the philosophies of patients and family members who choose to shift away from aggressive
COGNITIVE TASK ANALYSIS
26
curative treatments towards therapies that provide for greater comfort and quality of life (Oshima
& Emanuel, 2013). In addition, studies indicate that the lack of communication between medical
staff and patients and family members regarding medically complex situations, including
treatment decisions with realistic expectations, is a major barrier for quality end-of-life care
(Smith & Hillner, 2011; Tiden & Thompson). Specifically, patients and family members
identify greater satisfaction with communication, shared decision-making, as well as greater
feelings of comfort and quality of life when provided with palliative care and a palliative care
team (Enguidanos, Housen, Penido, Mejia, & Miller, 2014; Fitzsimons, et al., 2007). In sum,
research has demonstrated that the use of palliative care increases the satisfaction of patients and
family members as they navigate the complex medical system associated with end-of-life care
and communication.
Communication and Shared Decision-Making in Palliative Care
Over the last half century, there has been a shift in the communication patterns between
patients and physicians (Gaston & Mitchell, 2005). Historically, the communication patterns
between physicians and patients were paternalistic, or an illness-centered medical practice, with
the locus of control being held by the physician and patients playing a more passive role(Charles,
Gafni, Whelan, 1997; De Haes, 2006; Taylor, 2009). The model of paternalism in medical
practice assumed that the physician held the dominant role as the expert, while the patient
employed a more passive role as the sick person (Charles, Gafni, Whelan, 1997). Today,
physicians are required to have sophisticated medical knowledge, as well as provide patient
centered communication in which they are able to discuss relevant medical information, actively
listen, and assess a patients goals, priorities and values through effective questioning techniques
(Levinson, Lesser & Epstein, 2010; Levinson & Pizzo, 2011). Essentially, there has been a shift
COGNITIVE TASK ANALYSIS
27
in the way patients and physicians interact, with a distinct move toward shared decision-making
and patient-centered communication models. In fact, patient-centered communication and
shared-decision making arose from the effort toward individual autonomy, and the principle of
informed consent, whereby patients have the right to understand their medical condition and
treatment prior to consenting to any medical intervention (Barratt, 2008; Charles, Gafni &
Whelan, 1997; Taylor, 2009). As a result of the shift in medicine and medical practices, there is
an increasing recognition that the medical community needs to focus more on patient-centered
communication and shared-decision making. To emphasize a greater need for physicians and
medical schools to incorporate physician training of patient-centered communication and shared
decision making, the Medicare Payment Advisory Commission presented a report to Congress in
June 2009 recommending a financial link between Medicare payments and patient centered
communication skills taught in graduate medical education (Levinson, Lesser & Epstein, 2010).
Patient-centered communication is a complex cognitive skill that, when done well, has a
positive impact on the patient and family members (Levinson, Lesser & Epstein, 2010). Shifting
away from physician directives to shared decision-making between physicians and patients
establishes the fact that patient-communication is a pertinent part of health care. Current
research has shown that efficacious communication between patients and physicians is a key
component for quality health care, being more patient-centered, and reducing health care costs
(Berkhof, van Rijssen, Schellart, Anema & van der Beek, 2011; Legare & Witteman, 2013;
Oshima Lee & Emanuel, 2013). Indeed, patient-centered communication and shared decision-
making recognizes that the patient has the right to make decisions about his/her own medical
care based on the patient’s values, beliefs and goals (Moulton, Collins, Burns-Cox & Coulter,
2013).
COGNITIVE TASK ANALYSIS
28
While it is clear that patient-centered communication and shared-decision making are
critical to quality health-care, it is considered even more essential for patients and family
members at the end stages of life. In fact, research shows that end-of-life decisions are highly
emotional, with complex treatments options, thus communication of life- altering information
around palliative care is critical (Back, et al., 2007; Freytag, 2012; Slort, et al., 2014). Studies
posit that patient-physician communication and engaging patients in shared-decision making
during end-of-life can significantly influence how patients and their families react and adjust to
further medical care (Back et al., 2007; Kirk, P., Kirk, I. & Kristjanson, 2004; Slort et al., 2014).
Specifically, patient-centered communication during palliative care is complex and multi-
dimensional, and, one study of patient-physician communication found that uncertainty between
a patient and a physician about the patient’s values and goals resulted in greater uncertainty
about the efficacy of the patient’s choice of treatments (LaBlanc, Kenny, O’Conner & Legare,
(2009). In addition, research concerning physician communication and advance directives found
that the quality of the communication between a patient and the physician influenced the
completion of an advance directive, but more importantly, the study illuminated the discussion
patterns of physicians, illustrated that physicians spoke for two thirds of the time, rarely
questioning patients about their goals and values (Tulsky, Fischer, Rose & Arnold, 1988). More
importantly, research in palliative care and patient-centered communication has shown that the
ethnic background, race, education levels and socio-economic factors of a patient and their
family significantly affects the processes of communication and shared-decision making between
physicians, patients and family member during the end stages of life (Allen, R., Allen, J.,
Hilgeman & DeCoster, 2008; White, Braddock, Bereknyei & Curtis, 2007). In sum, medical
COGNITIVE TASK ANALYSIS
29
practices are changing, and research indicates that effective communication and shared decision-
making are critical components of high quality medical care.
While patient centered communication is proven to be beneficial at providing high
quality health care, greater levels of patient satisfaction, and cost savings, it is now a necessity
given a provision in the Affordable Care Act (ACA) signed into law in March of 2010, and the
Heath Care Education Reconciliation Act (HCER) of 2010. In fact, section 3506 of the ACA
requires health care providers to engage patients and caregivers in informed decision-making
(Kocher, Emanuel & DeParle, 2010; Legare & Witteman, 2013). Primarily, the ACA provision
for shared decision making has implications for medical professionals because it ensures that
physicians engage in patient-centered communication and shared decision-making around a
patient’s goals and values, better aligning with a patient’s preferences for different treatment
options (Kocher, Emanuel & DeParle, 2010). Fundamentally, patient-centered communication
and engaging with patients in shared-decision making is an essential part of a changing medical
system. While the active participation between physicians and patients in shared-decision
making and patient-centered communication is a new requirement of the ACA, it is vital for
patients and families as a best practice for medical care, especially as they proceed through the
end stages of life.
Palliative Care Education and Training
Traditionally, education and specialization in palliative care has been missing in medical
school and residency programs (Billings & Block, 1997). The tradition of curative medical
education has been the dominant focus for medical training. This focus is evidenced by
curriculum and instruction centered on developing expertise in diagnosing and prescribing
aggressive treatments to cure illness and disease (Bickel-Swenson, 2007; Hammel, Sullivan,
COGNITIVE TASK ANALYSIS
30
Block & Twycross, 2007; Van Aalst-Cohen, Riggs, & Byock, 2008). Given the importance and
the realization about the need for palliative care education, the Institute of Medicine requested
that professional training be part of medical training programs, and in 2006, the Liaison
Committee for Medical Education (LCME) began requiring all accredited medical schools in the
United States to implement education in end-of-life care (Horowitz, Gramling & Quill, 2014;
Van Aalst-Cohen, Riggs, & Byock, 2008).
Although the LCME requires that the training and education in end-of-life care become a
mandatory component for U.S. medical schools, the curricula is not standardized, with most
medical schools using an integrated approach to training medical students, and little clinical
experience with palliative care patients and family members (Bickel-Swenson, 2007; Hammel,
Sullivan, Block & Twycross, 2007; Horowitz, Gramling & Quill, 2014; Van Aalst-Cohen, Riggs,
& Byock, 2008). In one study, Horowitz, Gramling and Quill (2014) found that the variations of
medical training in end-of-life care varied from some medical students receiving two hours of
classroom instruction to other student receiving multiple weeks of clinical rotations with
palliative care and hospice patients. Since the 2006 mandate by LCME that medical schools
provide end-of-life and palliative care instruction, there has been advances in the medical
education for students. While there has been an increase in the number of medical schools,
residency programs and fellowships focused on palliative care, there remains inconsistent
training and preparation for dealing with the complex medical needs of chronically ill patients is
and the aging population. The inconsistent training and educational opportunities illustrates the
need for greater change and adaptation within the medical education system, as well as the need
for more explicit instruction regarding patient centered communication and shared decision
making at the end of life.
COGNITIVE TASK ANALYSIS
31
All physicians are required to demonstrate competencies in patient care, technical skills,
medical knowledge, practice based learning, system utilization, and communication skills
(Brigham, 2013). The complexities of palliative care require that palliative care physicians be
trained beyond medical school and residency. Indeed, palliative care physicians must develop
competencies that will enable them to treat the physical, psychological and spiritual needs of
patients with serious illnesses. Moreover, a joint initiative between the ACGME, the American
Board of Internal Medicine (ABIM), and other medical subspecialties developed a Milestone
Project for Hospice and Palliative Care Medicine in January 2015. The Milestone Project
provides for a semi-annual review and reporting of a fellow’s performance within a fellowship
program. Each fellow is evaluated on their progress in eight competencies, or milestones. The
goal of the milestone reporting is to document the learning and competency of fellows as they
progress through a fellowship program towards graduation.
Summary
The medical system is changing and more individuals will need access to medical care
due to life limiting chronic conditions and terminal illnesses. Further, individuals will require
greater access to physicians with not only medical knowledge, but also patient-centered
communication skills. As the medical community adjusts to the changing dynamics of medical
care and the provisions of the ACA, there is going to be a need for effective training for
physicians and other medical care providers in dealing effectively with chronic conditions,
terminal illnesses, as well as patient-centered communication, including shared-decision making.
Consequently, it is essential to determine the implicit and explicit knowledge of expert palliative
care physicians as they engage in patient-centered communication and shared decision making
during end-of-life discussions.
COGNITIVE TASK ANALYSIS
32
Using Subject Matter Experts to Train Non-experts
The challenges in education and training of physicians in patient-centered communication
and shared decision making skills may be ameliorated by using proven methods, such as
Cognitive Task Analysis (CTA) to capture and aggregate the knowledge and skills from expert
palliative care physicians to design effective instruction.
In almost every type of educational setting, experts are often called upon to communicate
their knowledge and skills, to inform the curricular content and instructional materials, and
mentor or coach others to perform complex tasks and solve difficult problems. Jackson (1985)
succinctly suggests that the fundamental purpose of education and training is to replicate
knowledge. Although, traditional education originated with the model of masters or experts,
demonstrating their knowledge to their apprentices or novices (Bloom, 1984), as education and
training evolved, the number of novices increased and the expert became the instructor to many
students. To this day, the model of the expert providing instruction for the novice to master
continues to be important in effective pedagogy (Bandura, 1977)
With the rise of industrialization and the development of new technologies, it is critical
that the contemporary model of education, where the expert imparts their knowledge to novices,
be re-evaluated. Current researchers recognized that much of the knowledge of experts is implicit
and unobservable (Clark, 2006; Clark, 2014; Clark & Feldon). Recent research has shown that
experts omit up to 70% of the procedural and conditional knowledge needed to complete a task
(Clark, 2006; Clark, 2014; Clark & Feldon). Specifically, Clark 2006 found that when reporting
a cognitive task, experts demonstrate high levels of incomplete and inaccurate information. With
this, novices on the receiving end of such incomplete, or inaccurate information use learning
COGNITIVE TASK ANALYSIS
33
strategies where they fill in their own information, leading to high levels of misconception
(Clark, 2006).
As individuals develop expertise in their domain, much of the procedural and conditional
knowledge they have becomes highly automated due to the repeated practices (Clark, Yates,
Early & Moulton, 2009; Clark, Feldon, van Merriënboer, Yates & Early 2008; Feldon & Clark,
2006; Hoffman & Militello, 2009). While the automaticity of experts allows for more
adaptability and greater working memory capacity, an expert’s automaticity renders the
information unconscious, and less available for recall (Clark, et al, 2008; Feldon & Clark, 2006;
Hoffman & Militello, 2009). And, as a result of the automaticity in the procedural and
conditional knowledge of experts, there is a gap in the information that is accessible to novices
for instruction (Clark, 2006; Clark, 2014; Clark & Feldon). Cognitive Task Analysis (CTA) was
developed as a method to address the issue of automaticity of cognitively complex tasks.
Essentially, CTA is a knowledge elicitation technique specifically developed to capture the
automated knowledge of experts within a specific domain (Hoffman & Militello, 2009).
Research has shown that using the knowledge elicitation methods of CTA as part of the design of
instruction has been indicated as effective and efficient for novice learners in complex cognitive
tasks across many different domains (Clark & Estes, 1996; Clark, Yates, Early & Moulton, 2009;
Clark, et al., 2008; Feldon & Clark, 2006; Hoffman & Militello, 2009; Means & Gott, 1988;
Toefel-Grehl & Feldon, 2013). To further understand why CTA is effective, the following
sections examine the two types of knowledge, the nature of automaticity and the characteristics
of expertise.
COGNITIVE TASK ANALYSIS
34
Knowledge Types
Research into the acquisition of cognitive skill suggests that the development of
knowledge, or learning, is acquired through a two-stage process. Anderson (1982) identifies this
process of cognitive skill acquisition, Adaptive Character of Thought (ACT), as a framework in
which declarative knowledge, the why or what, is converted and embodied into procedural
knowledge, the how and when. In effect, the acquisition of cognitive skill is a process whereby
declarative knowledge becomes procedural knowledge. All knowledge types, declarative,
procedural, and conditional knowledge develop as information and skills are acquired and an
individual transitions from a novice to an expert (Paris, Lipson & Wixson, 1983). The
differences between declarative knowledge and procedural knowledge are important to note, in
that, the different knowledge types allow for different performances and outcomes (Ambrose,
Bridges, DiPietro, Lovett & Norman, 2010)
Declarative Knowledge
There are various knowledge types and structures that are included in the cognitive
architecture of learning and knowledge. One of the knowledge types, declarative knowledge, is
hierarchical factual knowledge about events, objects, or why something is the way it (Anderson
& Fincham, 1994; Clark & Estes, 1996). Declarative knowledge is knowledge about information
that is represented in segments or chunks that are characterized by the quality of learning
(Anderson & Schunn, 2000). In other words, declarative knowledge is overt knowledge about
the why something is, or the that something is (Clark & Estes, 1996). According to Anderson
and Krathwohl (2001), declarative knowledge is isolated and discreet information that is content
specific. Furthermore, declarative knowledge is goal independent knowledge about terms and
facts, as well as larger, more organized factual knowledge about principles, concepts, theories or
COGNITIVE TASK ANALYSIS
35
models (Anderson & Krathwohl, 2001; Corbett & Anderson, 1995). There are several types of
declarative knowledge, however, the most common types of declarative knowledge is
information about concepts, processes, principles, and procedures (Merrill, 1983; 1994).
Anderson & Fincham’s (1994) research into learning, memory, and cognition finds that
most knowledge arrives in the cognitive architecture in the form of declarative knowledge. Once
in the working memory, declarative knowledge becomes information that is easily retrievable
and can be used to answer questions about why or what something is with regards to concepts
and facts (Anderson & Schunn, 2000). Additionally, research into learning and the cognitive
architecture identifies declarative knowledge as knowledge that it is characterized by its speed
and conscious quality, thus allowing for this type of knowledge to be easily learned and modified
in an individual’s working memory (Clark & Estes, 1996; Merrill, 1983). Because of the speed
and conscious quality of declarative knowledge, it is posited that this type of knowledge allows
individuals to create frameworks for comprehension based on domain specific and abstract
schemas and conceptual understanding (Clark & Estes, 1996). Furthermore, research indicates
that declarative knowledge, once in the long-term memory, can be converted into procedures and
frameworks that allow for experts to understand, comprehend, and quickly analyze new and
novel tasks (Anderson & Fincham, 1994; Clark & Estes, 1996).
While nearly all knowledge enters the working memory as declarative knowledge, factual
information about the why and that, declarative knowledge is inadequate for the execution or
performance of a skill or procedure (Anderson, 1982; Anderson & Fincham, 1994; Anderson &
Schunn, 2000). Conceptually, declarative knowledge is only the foundation of knowledge
acquisition, first learned in the conscious mind, and over time and with practice, transferred into
long-term memory. As such, declarative knowledge is considered by many to be passive
COGNITIVE TASK ANALYSIS
36
knowledge that lacks context, and must therefore be transferred into procedural knowledge prior
to the execution of any skill, performance, or procedure (Anderson, 1982; Anderson & Fincham,
1994; Anderson & Schunn, 2000). Essentially, declarative knowledge, knowing the why and
what of something, supports the acquisition of the how and when something is of procedural
knowledge.
Procedural and Conditional Knowledge
All types of knowledge, declarative, procedural, and conditional are required for
completing complex cognitive tasks (Anderson, 1982; Anderson & Fincham, 1994; Anderson &
Schunn, 2000). Beyond the factual and passive information of declarative knowledge, there is
procedural knowledge or production knowledge, that is the knowledge about the when and how a
task is performed (Clark & Estes, 1996). Specifically, procedural knowledge is the information
that allows individuals to know when and how to apply specific methods, skills, or procedures,
including the processes, sequences and steps of simple or complex tasks (Ambrose, Bridges,
DiPietro, Lovett & Norman, 2010; Anderson & Krathwohl, 2001). Procedural knowledge is
subject matter, content specific knowledge that is and goal-directed or goal oriented, lending to
an individual’s ability to solve more complex problems (Corbett & Anderson, 1995; Paris &
Lipson & Wixson, 1983). And, similar to declarative knowledge, repetition and practice allow
for procedural knowledge to become fluid, with rapid, automated responses (Corbett &
Anderson, 1995). Simply put, procedural knowledge allows experts to use their vast declarative
knowledge to perform complex tasks and solve intricate and novel problems (Anderson, 1982;
Anderson & Fincham, 1994; Anderson & Schunn, 2000; Clark & Estes, 1996; Corbett &
Anderson, 1995; Paris & Lipson & Wixson, 1983).
COGNITIVE TASK ANALYSIS
37
Research in cognitive psychology shows that procedural knowledge is generally acquired
after declarative knowledge or representations are transitioned from the isolated facts of why
something is or that something is, into the processes of how and when to use that knowledge
(Anderson & Fincham, 1994). Clark & Estes (1996) describe procedural knowledge as
knowledge that is characterized by its unconscious and automated quality. In addition, Clark &
Estes (1996) posit that procedural knowledge is unlike declarative knowledge because it is
difficult to learn, but quick to execute, requiring a vast amount of practice and feedback. And,
unlike declarative knowledge, procedural or production knowledge is difficult to change because
one it is automated, it is unlikely to be revised (Anderson, 1993). Furthermore, procedural
knowledge is production knowledge that combines the factual declarative knowledge that is
learned in specific domains and content areas (Anderson & Schunn, 2000). After the repeated
use of declarative knowledge, research indicates that procedural knowledge is developed, thus
allowing for experts to free working memory (Anderson & Schunn, 2000). In this way,
procedural knowledge is developed as IF/THEN statements that rely on factual information
produced from declarative knowledge (Anderson, 1982). For example experts convert
declarative knowledge, the IF something is that, into procedural knowledge of action and
decision steps, or how or when to use that information (Anderson, 1982). In other words,
procedural knowledge is knowledge that is represented by condition-action processes and rules
of when and how to use knowledge based on factual knowledge gained as expertise develops
(Anderson & Schunn, 2000).
One type, or sub category of procedural knowledge is conditional knowledge.
Conditional knowledge is knowing whether or not to use production or procedural knowledge
(Anderson & Krathwohl, 2001; Paris, Lipson & Wixson, 1983). Paris, Lipson & Wixson (1983)
COGNITIVE TASK ANALYSIS
38
identify conditional knowledge as procedural knowledge that informs experts by providing an
understanding about the circumstances and rational in which actions are based. As described by
Hall et al., (1995), conditional knowledge is the knowledge used by experts that sets forth the
decision of whether or not to move forward with executing or performing a specific task.
Basically, conditional knowledge is unconscious information experts use to determine the value
of a given set, or chunk of declarative knowledge, and the judgments that are used to determine
which processes and procedures to use (Paris, Lipson & Wixson, 1983).
Summarily, all cognitive skills are acquired through repetition and practice. Research
into learning and cognitive architecture distinguishes between declarative, procedural and
conditional knowledge (Anderson, 1982; Paris, Lipson & Wixson, 1983). Each type of
knowledge is required for the execution and completion of complex cognitive tasks. The
acquisition and practice of each knowledge type facilitates competence, and performance
becomes more fluid with exposure and execution of a task. With deliberate practice of domain
specific declarative, procedural and conditional knowledge, expertise develops, and processes
and procedures become highly automatic allowing for the unconscious selection of action and
decision steps required for highly challenging and complex cognitive tasks (Clark & Estes,
1993).
Automaticity
Through repeated performance and deliberate practice of a task, declarative, procedural
and conditional knowledge becomes automated and unconscious in nature, thus automaticity
develops (Feldon, 2007). As automaticity develops, the speed in performing a task increases
while, and at the same time, the amount of active mental effort decreases (Feldon, 2007).
Anderson (1996) identified three stages of automaticity. The first stage, the interpretive or
COGNITIVE TASK ANALYSIS
39
cognitive stage, is the point in which the learner is able to complete a complex task, or close
representation of a task with verbal instructions. One aspect of this stage is that the learner is
accessing self-talk strategies in which they are talking themselves through the performance of the
task (Anderson, 1996). The second stage identified by Anderson (1996) is deemed the
knowledge compilation or associative stage. In this second stage of automaticity, a learner
ascertains the declarative knowledge required to make corrections to procedural errors they
might encounter (Anderson, 1996). At this point in the cognitive architecture, it is posited that
the learner is developing a firmer grasp of the declarative and procedural knowledge structures,
thus reducing and finally eliminating the need for self talk and verbal cuing (Anderson, 1996).
In the final stage of automaticity, classified by Anderson (1996), the autonomous stage,
strengthening or tuning occurs and the learning is reinforced and regulated to the point in which
the task is completely autonomous and the process and procedures are completely automated. In
this stage, the learner is able to perform a procedure efficiently without any verbal cuing, freeing
up space in the working memory (Anderson, 1996). Beyond the three stages of automaticity,
Ericsson, Krampe, and Tesch-Romer (1993) identified a fourth stage of automaticity. According
to the research, the fourth stage of automaticity is reserved for an expert’s performance. During
this phase of automaticity, expertise is developed through repeated, deliberate practice, and the
task performance is improved and adapted through critical performance and feedback. In sum,
automaticity develops as an individual engages in deliberate practice, thus gaining expertise with
declarative knowledge and the ability to apply that knowledge into automated processes,
procedural knowledge, without conscious monitoring (Ericsson, Krampe, & Tesch-Romer, 1993;
Feldon, 2007).
COGNITIVE TASK ANALYSIS
40
Automaticity is considered advantageous, in that the automaticity of procedural
knowledge allows for experts within a given domain to unconsciously perform complex
cognitive tasks and solve novel problems with speed and accuracy (Clark & Elen, 2006). Due to
the limitations in the length and amount of information the working memory can hold,
automaticity is beneficial because it relieves an expert’s overall cognitive load, permitting
greater working memory capacity and the ability to consciously use problem solving strategies if
new information or novel problems arise (Clark, 1999; Kirschner, Sweller & Clark, 2006;
Wheatley & Wegner, 2001). Furthermore, automaticity is advantageous for experts since it
allows for knowledge to move away from the basic, concrete knowledge of declarative and
procedural knowledge into more abstract thoughts and knowledge (Hinds, Patterson & Pfeffer,
2001).
While automaticity is beneficial for lessening the cognitive load of experts, allowing for
greater working memory for complex problem solving, it is disadvantageous for deconstructing,
monitoring, and changing processes and procedural knowledge (Clark & Elen, 2006; Feldon,
2007). In fact, research into pedagogy and curriculum demonstrates that automaticity can
impede an expert’s ability to consciously explain critical information and the fundamental steps
used to complete cognitively complex tasks. As a result of the development of expertise,
procedural knowledge becomes so highly automated that articulation of the discreet steps and
process becomes problematic and experts omit pertinent information, causing knowledge sharing
to become problematic, and moreover, ineffective for novices (Kirschner, Sweller & Clark,
2006). Additionally, research shows that an expert’s instructional processes and procedures are
often initiated without prompting and thus, becomes completely automated (Feldon, 2007).
Moreover, once an expert commences their automated instructional processes and procedures,
COGNITIVE TASK ANALYSIS
41
research shows that they run it through to completion, preventing them from consciously
monitoring how they provide instruction (Feldon, 2007). Feldon’s (2007) research indicates that
when experts are actively engaged in instruction, and are made aware of their omissions due to
automaticity, they continue to make omissions due to the increase load of identifying the
omissions in the working memory. In other words, experts are inhibited from monitoring or
changing their automated instructional processes by the very fact that they are using their
working memory to identify omissions. Again, automated processes are difficult to change and
require large amounts of cognitive load to monitor, modify, and/or eliminate (Clark, 2008).
Consequently, automaticity is a double edge sword, in which subject matter experts can perform
complex cognitive task requiring declarative and procedural knowledge unconsciously, freeing
up working memory to address novel tasks. However, due to the unconscious nature of
automaticity, some instructional practices are resistant to change because it is difficult for experts
to modify, eliminate, or express procedural knowledge using concrete language and examples.
Expertise
Expertise is distinguished by extensive and highly structured knowledge of a specific
domain or content. Experts are able to perform complex tasks reliably and upon demand due to a
mastery of all relevant factors (Ericsson & Lehmann, 1996). As described previously,
individuals with expertise demonstrate highly automated declarative and procedural knowledge,
allowing for effective strategies for problem solving and expanded working memory (Clark,
1999; Kirschner, Sweller & Clark, 2006; Wheatley & Wegner, 2001). As such, automaticity
enables experts to utilize elaborated schemas to organize information effectively for rapid
storage, retrieval and manipulation (Clark, 1999; Kirschner, Sweller & Clark, 2006; Wheatley &
Wegner, 2001). In an adaptation of Hoffman’s (1998) definition of expertise, Chi (2006) defines
COGNITIVE TASK ANALYSIS
42
an expert as an individual who is highly regarded by peers as a distinguished or brilliant
journeyman, with the ability to perform and demonstrate consummate skill and efficiency, using
judgments that are accurate and reliable, even in rare and difficult cases. In further research into
expertise, Chi, Glaser and Farr (1988) categorized experts as having seven primary attributes,
including: (1) expertise in a specific domain; (2) the ability to perceive large meaningful patterns
within their domain; (3) the capacity to quickly and efficiently perform skills and solve problems
related to their content area; (4) a superior short and long term memory; (5) the capability to see
representations in a more principled and deeper level; (6) time spent analyzing problems
qualitatively; and (7) the ability to self-monitor. In effect, research has shown that experts differ
from novices, in that they have more domain knowledge, and are able to categorize declarative
and procedural knowledge, solve complex problems, and create elaborate schemas that allow
them to efficiently retrieve information (Bedard & Chi, 1992). Moreover, research demonstrates
that experts are able to forward think and see beyond simple schemas, view problems differently,
and identify the relevance of cues to perceive meaning from ill-defined problems to match
strategies to problems.
Building Expertise
Expertise, by its nature, is acquired as a result of continuous and deliberate practice in
solving problems within a specific content area or domain. Early research into expertise
assumed that expertise and exceptional performance was a product of innate ability, or
something an individual was born with (Ericsson, Krampe & Tesch-Romer, 1993; Ericsson &
Charness, 1994). In fact, early investigations sought to detect innate characteristics of
individuals to predict or account for advanced performance within a domain (Ericsson, Krampe
& Tesch-Romer, 1993). Current research has changed this theory and identified expertise as a
COGNITIVE TASK ANALYSIS
43
result of an understanding of environmental factors, including a person’s skill, ability, and
experience.
According to Alexander (2003), expertise is developed through a Model of Domain
Learning (MDL). The MDL theory of developing expertise states that there are bridges between
understandings of expertise and educational practice, and moreover, expertise is developed in
academic domains instead of non-academic problem solving that occurs in natural settings
(Alexander, 2003). The MDL distinguishes three components, knowledge, strategic processing,
and interest, as factors in the development of expertise in academic domains. In addition to the
three factors of development, the MDL identifies three stages of domain learning in which
learning and expertise emerge. According to Alexander (2003), the first stage, acclimation, is
the stage in which demands are placed on students as they acclimate to a complex and unfamiliar
domain. The second stage is competence, and in this stage of expertise development, the
individual demonstrates competence and a more cohesive and principled structure of the
foundational body of domain knowledge. Finally, Alexander (2003) defines the third stage of
expertise development is proficiency. The level of proficiency is characterized by the interaction
between all the components, in which the individual’s knowledge base is broad and deep, and the
individual is able to contribute to new knowledge within the domain.
Similar to Alexander’s (2003) MDL and the research into automaticity, Ericsson (2004)
has posited a theory of skill and expertise acquisition that includes multiple phases. In
Ericsson’s (2004) initial phase of learning, novices are attempting to understand a domain
specific activity and concentrating on avoiding mistakes. In the middle phases of learning,
novices have more experience, gross mistakes become rare, and the performance becomes
smoother with less cognitive load required for concentrating on performing at an acceptable
COGNITIVE TASK ANALYSIS
44
level. Finally, performance becomes automated and individuals are able execute skills without
conscious effort. Again, expertise requires experience and deliberate, sustained practice over
time. Specifically, building expertise demands an orderly and purposeful approach whereby an
individual specifically designs aspects of their performance to target and improve (Ericsson,
2004; Ericsson, Krampe & Tesch-Romer, 1993; Ericsson & Charness, 1994).
Consequences of Expertise
As new knowledge becomes automated and unconscious, experts are often able to
completely and accurately recall the declarative and procedural knowledge and skills that
comprise their expertise (Chi, 2006; Ericsson, 2004; Ericsson, Krampe & Tesch-Romer, 1993;
Ericsson & Charness, 1994). Though automaticity is of great benefit to the expert, there are
consequences that negatively impacts their ability to efficaciously provide instruction, leading to
subsequent difficulties for novice learners. Again, automaticity allows experts to perform skills
without conscious though, while at the same time, it makes it difficult for experts to make
intentional modifications during the performance the task. Chi (2006) points out several ways in
which experts are deficient, for instance, experts are limited to knowledge within their domain,
and often fail in their ability to recall details and surface features, relying mostly on context cues
that are difficult to explain. Furthermore, Chi (2006) and Feldon (2007) point out that experts
tend to be inflexible, and have a bias toward functional fixedness, making it difficult for routines
and procedures to be easily modified. More importantly, consequences arise because an expert’s
schema is so highly adapted that it interferes with accurate prediction, judgment, advice, and
recall of problem solving situations. Summarily, an expert’s automaticity impairs their ability to
consciously identify many of the decisions they make, thereby omitting key details and process
information necessary to explain details of complex task and provide instruction for optimal
COGNITIVE TASK ANALYSIS
45
performance (Chi, 2006; Ericsson, 2004; Ericsson, Krampe & Tesch-Romer, 1993; Ericsson &
Charness, 1994; Feldon, 2007).
Expert Omissions
Given the role of automaticity, experts in an instructional role may unintentionally omit
information that learners must master when learning procedural skills. In a study of instruction
and research design in psychology, Feldon (2004) found that automaticity was negatively
correlated with an expert’s ability to accurately self-report action and decision steps that occur
during the performance of a cognitively complex task. In fact, studies have found that experts
omit up to 70% of pertinent domain specific procedural knowledge and strategies required for a
novice to learn and perform a cognitively complex task (Clark, Pugh, Yates, Inaba, Green &
Sullivan, 2011; Feldon, 2004). For instance, Clark et al. studied expert physicians and found
that there was automaticity of skill development as physicians gained expertise through years of
practice and experiences. Furthermore, the study found that the discrete cognitively complex
steps of procedures became blended together, leading expert physicians to omit critical steps
when they described how to perform complex procedures. In addition, the research showed that
most expert physicians were unable to describe the unconscious thought processes behind the
behavioral execution of technical skills. Essentially, the study posited that the omissions of
expert physician made it even more difficult for novices to understand where the expert
physicians made critical action and decision steps. And because of these omissions, novices
were forced to fill in the blanks with their own knowledge, and use less efficient learning
methods, such as trial and error. Due to the high rate of omissions by experts within a domain,
up to 70%, using a Cognitive Task Analysis to study cognitively complex tasks and procedures is
proven to be beneficial (Clark, et al., 2011).
COGNITIVE TASK ANALYSIS
46
Cognitive Task Analysis (CTA)
Cognitive Task Analysis (CTA) is an effective knowledge elicitation technique that has
evolved from traditional task analysis methods, and is utilized in order to elicit and explicate
expert knowledge within a specific domain (Chipman & Shalin, 2000). CTA is an extension of
other traditional task analyses that identifies expert procedural and conditional knowledge
through processes of a variety of observation and interview strategies (Chipman & Shalin, 2000;
Hoffman & Militello, 2008). Indeed, CTA uses techniques that uncover the tacit thought
processes and goal structures that underlie an expert’s observable task performance, in addition
to the overt and cognitive functions that shape an expert’s cognitive functions, which outline the
integrated whole of their performance (Clark & Estes, 1996; Hoffman & Militello, 2008).
Effectually, CTA uses a variety of strategies to capture a clear description of the explicit and
implicit knowledge that experts use to perform complex cognitive tasks. Moreover, the
knowledge elicited from experts by using CTA can be used to teach, train, and assess
performances, as well as designing and developing new expert systems (Chipman & Shalin,
2000; Clark & Estes, 1996; Clark, Feldon, vanMerrinboer, Yates & Early, 2008).
CTA History
Cognitive Task Analysis can be traced back to the 1880s, and the beginning of industrial
psychology (Hoffman & Militello, 2008). The foundation for CTA can be seen throughout the
history of industrial engineering, applied psychology, and human factors. Early practices
focused on behavioral task analysis, whereby individual human performance was observed and
studied to identify precise actions required for the execution of various jobs in manufacturing
fields (Clark & Estes, 1996; Hoffman & Militello, 2008). The information gathered was then
used as a training tool to instruct new and inexperienced workers, as well as improve human
COGNITIVE TASK ANALYSIS
47
performance (Clark & Estes, 1996; Hoffman & Militello, 2008). While components of behavior
task analysis has proven to be beneficial for overt physical tasks, the rapid growth and advances
in technology requires individuals to work in more mentally complex environments, leading the
a need for studies on human cognition and problem solving (Clark & Estes, 1996; Hoffman &
Militello, 2008; Woods & Roth, 1988). The need for a more sophisticated cognitive task
analysis in the workplace has been further heightened by the increasing knowledge and
integration of cognitive factors within business organizations and governments, and more
importantly, the need for business to adapt to the changing structures within a global market
(Clark & Estes, 1996). In order to address the inadequacies of the traditional behavioral task
analysis, as well as the cognitively complex needs of businesses and governments, a knowledge
elicitation technique, identified as CTA was developed.
Cognitive Task Analysis Methodology
Research into task analysis has found over 100 different types of CTA (Cooke, 1994;
Yates, 2007). Regardless of the type of CTA, researchers have identified the main stages
through which a typical CTA would proceed. Specifically, Chipman, Schraagen, and Shalin
(2000) and Clark et al.(2008) have found that a CTA, ideally free from resource restrictions,
would include a series of five discrete steps including: (1) a preliminary phase or preliminary
knowledge collection; (2) the identification of knowledge representations; (3) the application of
knowledge elicitation techniques; (4) a review, verification, analysis and possible modification
of the knowledge elicited from experts; and (5) formatting and using the results from the analysis
as a basis for an expert system or expert cognitive model. Again, while there are over 100
different varieties of CTA, in general, most follow the five-stage process (Cooke, 1994; Yates,
2007).
COGNITIVE TASK ANALYSIS
48
Knowledge elicitation is a key factor in CTA because it is through the process of
extracting the domain specific knowledge underlying an expert’s performance, that protocols and
instructional applications are established. In fact, researchers have developed various
classifications and taxonomies to categorize knowledge elicitation according to specific criteria.
Cooke (1992b) established four categories of knowledge elicitation including observation,
interviews, process tracing, and conceptual methods. Since the current classification schemes
organize CTA methods by process rather than by the desired outcome or application, the
selection of the optimal method for practitioners may be challenging. In further research, Yates
(2007) identified the most frequently used CTA methods and knowledge types are associated
with the representative methods and outcomes, using a product approach verses the existing
process approach. Further, the research suggests that more standardized methods of knowledge
elicitation, with protocol analysis and conceptual methods proved to have greater consistency
than informal methods whereby mostly declarative knowledge is elicited (Yates, 2007). In sum,
CTA relies on the effective use of elicitation and analysis methods. For efficiency and efficacy,
CTA methods need to be classified in terms of desired outcomes rather than the process being
used.
While there are over 100 CTA methodologies, three methods will be reviewed for the
purpose of this study. First, the Critical Decision Method, or CDM, is a CTA method in which a
semi-structured interview technique is used to elicit expert knowledge through cognitive probes
around a non-routine incident (Hoffman, Crandall & Shadbolt, 1998). CDM begins by
questioning an expert’s experience of a critical incident, then proceeding through inquiries about
cues in the environment that led to the expert’s action and decision steps. Further information is
gathered with questions designed to elicit information about specific decision points including:
COGNITIVE TASK ANALYSIS
49
(1) perceptual cues; (2) prior knowledge used; (3) goals; (4) alternatives to the decision; and (5)
assessment factors from other situations. Moreover, research has shown that CDM as a CTA
methodology is supported by the theoretical base of naturalistic decision making, thus making
CDM a valid and reliable method of knowledge elicitation due to its application as a case study.
A second methodology of CTA is the Precursors, Actions, Results, and Interpretation
(PARI) method. The PARI method, first used in the Air Force research program, elicits the
automated cognitive action and decision steps of experts in real work environments (Hall, Gott,
Pokorny, 1995). In a CTA meta-analysis completed by Tofel-Grehl and Feldon (2013), the PARI
method of CTA demonstrated the largest effects for effective elicitation of procedural directions
to use for instructional purposes.
The third CTA method reviewed, and used for the purpose of this study is the concepts,
principles and processes model (CPP). The CPP method of CTA is a combination of the PARI
and CDM method CTA (Clark et al., 2008; Clark 2014; Tofel-Grehl & Feldon, 2014; Yates,
2007). The CPP method is multi- interview technique cycle that elicits the automated knowledge
of subject matter experts. According to Clark et al. 2008 and Clark (2014), CPP is the most
commonly used “evidenced based” CTA method. Regardless of method used for conducting
CTA, it has proven to be effective an effective in knowledge elicitation, as well as instructional
design across a variety of domains and content areas.
Effectiveness of CTA
Cognitive Task Analysis has proven to be an effective method for capturing the explicit,
observable behaviors, as well as the tacit, unobservable behaviors of experts. According to
Hoffman and Militello (2008), CTA is regarded as a necessary component of research into
complex cognitive work because it addresses the issues of research into the interaction of people,
COGNITIVE TASK ANALYSIS
50
technology and work. Through knowledge elicitation and analysis, CTA is able to identify the
explicit declarative knowledge, as well as the tacit, implicit knowledge of an expert. In fact,
research finds that CTA can elicit knowledge about an expert’s domain content, concepts and
principles, schemas, mental models and problem solving skills. Particularly, content and
domains that emphasize technical and functional capabilities are shown to benefit from the use of
CTA due to the fact that CTA is effective at emphasizing the components of the task that are
important to the learner, as well as providing a framework for abstract problem solving and
general principles of knowledge (Means & Gott, 1988). Compared to other strategies, CTA is
more effective at capturing the unconscious, complex cognitive action and decision steps of
domain experts.
In addition to being an effective method for capturing an expert’s implicit knowledge,
research has shown that CTA is more cost effective and efficient than other instructional
methods and models. Through a cost-benefit analysis, Clark et al., (2008) established a positive
impact in the use of CTA. For example, the study found that CTA had a positive impact on
informed instruction versus traditional instructional techniques, and furthermore, by using CTA,
the amount of time used for training was reduced by nearly half (Clark et al., 2008). Further
research has posited that by using a CTA method known as 1i +3r, expenses for training were
reduced by nearly 70% (Flynn, 2012). Summarily, the use of CTA in instruction and training
has been proven to be positively related to cost savings due to reduced training times with
comparable learning outcomes.
Benefits of CTA Instruction and Design
Studies that have applied CTA to capture expert knowledge and deliver instruction have
demonstrated several benefits and useful instructional design strategies as compared to other
COGNITIVE TASK ANALYSIS
51
forms of instruction (Clark et al., 2008; Flynn, 2010). Because all types of knowledge,
declarative, procedural and conditional, are required for novices to learn complex cognitive
tasks, using CTA as a part of an instructional design has been shown to be both, effective and
efficient. Individuals engaged in learning highly complex cognitive tasks are limited by
cognitive capacity (Ayres, & Paas, 2012; Kirschner, Ayres, & Chandler, 2011; Mayer, 2011).
With the limited amount of working memory allowed for learning new content, it is important
that extraneous processing be limited so learners are not inhibited by cognitive overload. And
while declarative knowledge is easily recalled and articulated, procedural and conditional
knowledge, the knowledge required for highly complex tasks, is often omitted from instruction,
resulting in incomplete or inaccurate learning (Hoffman, Coffey, Carnot & Novak, 2002; Means
& Gott, 1988). The use of CTA as part of instructional design is supported by research because
it is proven to be successful in identifying the perceptual differences and procedures experts
leave out during instruction or training (Crandall, Klein & Hoffman, 2006). In fact, the design of
novice instruction places the burden for learning on the expert, so it is essential that they convey
what they know about the domain, including the subtle aspects of the task that important to the
learner, but, though automaticity, have become unconscious and automatic to the expert.
Specifically, studies indicate that direct instruction, using content elicited from experts through
the use of CTA, extends the novice learning by expanding instruction beyond examples and
presenting qualities of a task that are important to the learner (Means & Gott, 1993; Schaafstal,
A., Schaafstal, J.M., & van Berlo, 2000). In other words, using CTA can be used to elicit the
tacit and explicit knowledge required for learning, thus, it is effective in instructional design
because it aligns with what is known about a learner’s cognitive architecture (Kirschner, Ayres,
& Chandler, 2011; Mayer, 2011Clark et al., 2009; Merrill, 2002; Schaafstal, A., Schaafstal, J.M.,
COGNITIVE TASK ANALYSIS
52
& van Berlo, 2000). By including CTA as part of instructional design, research indicates that
effective instruction can be delivered to address the needs of novice learners.
COGNITIVE TASK ANALYSIS
53
CHAPTER THREE: METHODS
The purpose of this study was to capture the automated knowledge of expert palliative
care physicians as they describe how to engage and involved patients and family members in the
process of shared decision-making during the end stages of life. This study used the method of
Cognitive Task Analysis to capture both the explicit observable and the tacit, automated
knowledge of expert palliative care physicians as they engage and interact with patients and
family members in patient-centered communication regarding the diagnoses, treatment goals,
and clinical outcomes at the end stages of life. Given the automaticity of an expert’s procedural
and conditional knowledge in performing complex cognitive tasks, the method of CTA was used
to elicit, capture, and document an expert palliative care physician’s unconscious, tacit
knowledge while engaging with patients and family members in patient-centered communication
and shared decision-making (Clark, et al., 2008).
As such, the CTA method was used and the following research questions guided this study:
1. What are the essential decision and actions steps palliative care physicians recall when
they describe how they involve patients and family members in in-patient shared-decision
making about end-of-life treatment?
2. What percentage critical action and/or decision steps, when compared to a gold standard,
do expert palliative care physicians omit when they describe how they conduct shared
decision-making with patients during end-of-life communication?
Participants
The participants for this study were selected based on their expertise in the field of
palliative care. While expertise is a difficult concept to define, for the purpose of this Cognitive
COGNITIVE TASK ANALYSIS
54
Task Analysis, Subject Matter Experts (SMEs) were defined as physicians in the palliative care
field of medicine and have met the criteria of
• a minimum of five years of consistent and successful on the job professional experience;
• experience in a wide variety of settings that are applicable to palliative care;
• little to no experience as a trainer or instructor of the skills being captured;
• high levels of flexibility, effective verbal skills, and willingness to be interviewed with
the use of an audio recorder; and
• availability to be interviewed using the CTA protocol for at least a 90 minute segment of
time along with a follow up interview lasting between 20 to 30 minutes (Yates & Clark,
2011).
Because palliative care has only been recognized by the ABMS as a subspecialty since 2006,
the years of successful on the job professional experience as a palliative care expert was
modified to include years of experience in other related medical subspecialties congruent with
palliative care.
In order to select the experts, the researcher initially identified expert palliative care
physicians through palliative care, hospital and hospice websites. In addition, the researcher
identified palliative care expert physicians based on recommendations from dissertation
committee members. The researcher sent out emails from the list of identified palliative care
experts. The strategy was marginally successful and the researcher was able to identify two
experts from a large hospital group in Southern California, as well as one expert from a large
university hospital in Southern California. The fourth expert used to review the preliminary gold
standard protocol was identified through a contact of the dissertation chair within a large
children’s hospital system. The recommended participants were qualified as experts based on
COGNITIVE TASK ANALYSIS
55
reputation, years of experience in palliative care and other subspecialties related to end-of-life
care.
Data Collection for Question 1
The first research question asked, “What are the essential decision and actions steps
palliative care physicians recall when they describe how they involve patients and family
members in in-patient shared-decision making about end-of-life treatment?” For the purpose of
this study, the data collection followed the five-stage process of knowledge elicitation as
identified by Clark et al. (2008). The stages for expert knowledge elicitation was conducted in
order and included:
1. Collect preliminary domain specific knowledge.
2. Identify the task specific knowledge representations.
3. Apply the knowledge elicitation technique using semi-structured interviews.
4. Analyze and verify the data collected during the interviews.
5. Format the results into an instructional design or training program.
In this study, the five-stage process was implemented as described in the following sections.
Phase 1: Collect Preliminary Knowledge
Because the researcher is not in the medical field, and has little knowledge of palliative
care, a thorough literature review was conducted to gather preliminary information and gain
familiarity with palliative care, patient-centered communication, and shared decision-making.
Phase 2: Identify Knowledge Types
As part of the process of the literature review, the researcher acquired an understanding
of the differences between declarative, procedural and conditional knowledge types. The
researcher engaged in various structured activities, with the supervision of a senior researcher,
COGNITIVE TASK ANALYSIS
56
whereby the differences of the knowledge types were practiced with other novice researchers. In
addition, the researcher practiced the identification of critical action and decision steps, and this
information, along with a thorough understanding of the knowledge types was used for the
development of the interview protocol. With the knowledge gained, the researcher identified the
declarative, procedural, and conditional knowledge from Phase 1.
Phase 3: Apply knowledge elicitation techniques
Instrumentation. A semi-structured interview protocol based on Clark’s Concepts,
Processes and Principles (CPP) was used to capture the unconscious, tacit knowledge and skills
from the expert palliative care physicians. The semi-structured interview protocol is attached as
Appendix A.
The concepts, processes, and principals (CPP) method of CTA was used as part of the
CTA methodology. This CTA study around physician and patient shared decision-making
during the end stages of life used the CPP method of CTA because it has been shown to be
effective in knowledge elicitation for complex cognitive tasks. The CPP method is a multi-
layered interview technique that begins with the researcher explaining the CTA process to the
interviewee, then asking the participant to identify three to five major steps required to perform
the identified task. The researcher then leads the respondent through a semi-structured interview
that elicits and captures the step-by-step action and decision step for each major step.
In the CPP method, the identification of each action and decision step is deemed essential
information for novice learners. Essentially, each action step begins with a verb, and each
decision step contains two or more alternatives to consider before taking an action. For
example, an action step would be “Pour two cups of flour into a bowl.” An example of decision
steps would be, “IF making a cake is for a boy’s baby shower, THEN make a blue cake with blue
COGNITIVE TASK ANALYSIS
57
frosting; IF making a cake for a girl’s baby shower, THEN make a pink cake with pink frosting;
IF making a gender neutral cake; THEN make a vanilla cake with yellow frosting.”
Specific to the interview with the third SME, the Critical Decision Method (Hoffman,
Crandall, Shadbolt, 1998; CDM) was used in addition to the CPP. The use of CDM in cognitive
task analysis incorporates methods for expert knowledge elicitation that includes semi-structured
structured interviews of participants around a specific incident. In CDM, a SME is guided
through multiple-passes of recalling a particular incident of shared decision-making, with
probing questions to illicit the tacit and unconscious task specific cognitive details related to that
incident.
The CDM became an important method to elicit knowledge during the interview with
SME C, as the SME demonstrated difficulty identifying the discrete action and decision steps
used for shared decision-making due to the expert’s automaticity of procedural and conditional
knowledge. Thus the third SME was guided through a recollection of a specific incident of
shared decision-making, with probing questions about the action and decision steps performed
during the task (Hoffman, Crandall, Shadbolt, 1998).
Interviews. Following the Institutional Review Board (IRB approval from the
University of Southern California, four expert palliative care physicians were asked to participate
in this study to identify how expert palliative care physicians engage with patients in patient-
centered communication and shared decision making in end-of-life decisions. Three of the four
palliative care physicians were asked to voluntarily participate in the CTA semi-structured
interview protocol to identify the automated, tacit procedural and conditional knowledge of
engaging with patients and family members in shared decision-making. A fourth palliative care
COGNITIVE TASK ANALYSIS
58
physician was asked to review the preliminary gold standard protocol developed from the first
three subject matter expert interviews.
To limit confirmation bias, a senior researcher guided the author, a junior researcher, in
the initial expert interview and also provided oversight during subsequent subject matter expert
interviews. Each subject matter expert interview took between 90 to 100 minutes, and was
recorded using an audio recorder with explicit permission from the respondent.
Phase 4: Data analysis
Audio recordings of each interview of the subject matter experts permitted verbatim
transcription. Using an outside transcription service, the interview transcripts were printed out
and coded by the researcher for a comprehensive and concentrated analysis.
Coding. After each interview was transcribed verbatim and printed out as a hard copy, a
coding scheme previously developed by Clark (2006) for the CPP CTA method was used to code
the data. The coding scheme included codes for the subject matter experts’ action and decision
steps, as well as the tools and or decision aids and standards of practice used during patient-
centered communication and shared decision-making. The coding scheme was used for inter-
rater reliability and is included as part of Appendix B.
Inter-rater reliability. To insure the reliability of coding, the first interview transcript
was double coded and compared with another CTA researcher’s codes for inter-rater reliability.
The double coded interview was compared and analyzed, and a standard inter-rater reliability
was calculated as a percentage of correspondence between the two codes. Based on research, an
85% or higher agreement in inter-rater reliability is deemed consistent and reliable between two
coders (Hoffman, Crandall, and Shadbolt, 1998). If the inter-rater reliability is less than 85%
between two coders, then is suggested that the coding scheme be further refined and function-
COGNITIVE TASK ANALYSIS
59
unit categories be developed (Crandall et al., 2006). The results of the inter-rater reliability are
presented in Chapter Four.
Subject matter expert protocol and verification. After the initial interviews with each
subject matter expert, a report was generated from each transcribed interview, and a preliminary
step-by-step protocol was established. Each SME was asked to review and verify the protocol
during a second interview. The second interview allowed for the physician expert and the
researcher to review the step-by-step preliminary protocol, and the expert was asked to make any
corrections, modifications, additions, or deletions of irrelevant or redundant information.
Phase 5: Formatting the results
The results were formatted to align with Research Question One and develop a Gold
Standard Protocol (GSP)
Gold Standard Protocol (GSP). After the verification of the protocol from each of the
three SMEs, a preliminary gold standard (PGSP) was generated from the de-identified
aggregated data. The aggregation of the preliminary gold standard protocol was initiated by
identifying the most comprehensive and clear protocol from one of the three SMEs. Then, each
of the other two SMEs major steps were compared to the most comprehensive protocol, and if
the major steps identified by each SME had the same meaning, then they were attributed to the
SMEs who identified them as major steps in their protocol. If there was a new step not identified
in the most complete protocol, it was then added to the PGSP, and attributed to the SME who
identified it as a major step. Similarly, each action and decision step was analyzed. If the action
and decision steps identified by each SME had the same meaning as the most comprehensive
protocol, it was attributed to each SME who identified it as an action and/or decision step. If an
action and/or decision step was more comprehensive or precise from one of the SMEs, the action
COGNITIVE TASK ANALYSIS
60
and/or decision step was modified and attributed to all SMEs identifying it as a specific action
and/or decision step. Finally, a fourth palliative care physician expert who was selected to
participate in the study, but not part of the initial semi-structured interviews, was asked to review
the aggregated protocol for corrections, modifications, additions, or deletions of irrelevant or
redundant information.. See Appendix C for a description of the procedure for completing the
Gold Standard Protocol.
Figure 1: Visual Representation of CTA 3i + 3r Method of Data Collection
Researcher
Conducts
Semi-‐
structured
Interviews
SME
A
SME
B
SME
C
Individual
Protocol
Individual
Protocol
Individual
Protocol
SME
B
Review
InterviewSME
B
SME
C
Review
InterviewSME
C
SME
A
Review
InterviewSME
A
Preliminary
Gold
Standard
Protocol
(PGSP)
SME
D
Review
of
PGSP
Gold
Standard
Protocol
(GSP)
COGNITIVE TASK ANALYSIS
61
Data Analysis for Question 2
The second research question asked, “What is the percentage of action and/or decision steps,
when compared to a gold standard, do expert palliative care physicians omit when they conduct
shared decision-making with patients during end-of-life communication?”
Spreadsheet Analysis
The data analysis was completed by transferring the identified action and decision steps
of the gold standard protocol into a spreadsheet. The three subject matter expert’s individual
protocols were reviewed and compared with the GSP. If the SME’s protocol included an action
or decision step that was also in the GSP, a “1” was placed into the corresponding action or
decision step cell. If an action or decision step was in the GSP, but was not included in the
individual SME’s protocol, a “0” was placed in the corresponding action or decision step cell.
By completing this analysis, the researcher was able to identify the frequency counts of the
expert’s action and decision steps. Finally, the frequency was converted into percentages to
represent the total number of omissions and agreements between each SME and the GSP.
COGNITIVE TASK ANALYSIS
62
CHAPTER FOUR: RESULTS
This study uses the CTA method to examine the declarative, procedural, and conditional
knowledge, represented by the action and decision steps recalled by expert palliative care
physicians as they describe how to perform the cognitively complex task of engaging with
patients and family members in patient-centered communication and shared decision-making at
the end of life. The results of the data analysis are organized by research question.
Research Questions
Research Question One
Research Question One asked, “What are the essential decision and actions steps
palliative care physicians recall when they describe how they involve patients and family
members in in-patient shared-decision making about end-of-life treatment?
Inter-rater reliability. As described in Chapter Three, inter-rater reliability was
established through two researchers independently coding the initial transcribed interview from
the first subject matter expert, and the tallying the number of coded agreements. The inter-rater
reliability was 100%, and given the high inter-rater reliability, this researcher coded the
remaining two transcribed interview and created protocols for each SME. The results of the
inter-rater reliably tally codes are shown in Appendix B.
Flowchart analysis. SME A’s final protocol was used to create a flowchart of the
individual action and decision steps in patient-centered shared-decision making. The flowchart
was used to ensure the logical progression of each action and decision step. Moreover, the
flowchart was used to check for incomplete decision steps whereby two or more alternatives
were not identified. The flowchart is included as Appendix D.
COGNITIVE TASK ANALYSIS
63
Gold standard protocol. As described in Chapter Thee, the initial gold standard
protocol was developed by reviewing and aggregating the data from the first three SME’s final
protocol. After reviewing each of the expert’s final protocols, it was determined that SME A had
the most complete and comprehensive individual protocol, thus SME A’s protocol became the
foundation for the preliminary gold standard protocol (PGSP). After further review of the
remaining two protocols it was determined that SME B’s protocol was more comprehensive than
SME C’s protocol. The action and decision steps of SME B were then compared to the
foundational protocol from SME A, and if the action and/or decision steps were determined to be
the same, then the action or decision step was associated with both SME A and SME B. On the
other hand, if there was an action or decision step indicated in SME B’s protocol that was not
identified by SME A, the step was added to the foundational protocol from SME A, and the step
was associated with SME B. The same process was completed using the protocol from SME C,
whereby each action and decision step was associated with SME C if it was established to have
the same meaning, and the action and decision steps that were not included in SME A or SME
B’s protocol were added and aligned with SME C. An example of the process for determining
the preliminary gold standard protocol is shown in Figure 2.
COGNITIVE TASK ANALYSIS
64
Figure 2. Example of Aggregating Action and Decision Steps for the Preliminary Gold Standard
Protocol (PGSP). Progression of an Action Step as Data Described By Each SME from Their
Individual Protocols Aggregated to Create an Action Step Found in the GSP
The researcher, along with a senior researcher, met in person with fourth expert, SME D,
to explain the CTA methodology and review the preliminary gold standard protocol. During the
meeting, SME D made further additions and modifications to the preliminary gold standard
protocol. The additions and modifications from SME D were incorporated into the final Gold
Standard Protocol, and the additions and modifications were not attributed to any of the initial
three subject matter experts.
The results for the first research question detailing the essential decision and action steps
palliative care physicians recall when they involve patients and family members in shared
SME A –Action Step:
Ask the patient/SDM what and
how much they know about
their disease/condition and
prognosis. (A)
SME B – Additions (in Bold)
to SME A’s Action Step:
Ask the patient/SDM what
and how much they know
about their disease /condition
and prognosis. Use the
information gathered
during the discussion
(content/tone/level of
vocabulary) to evaluate the
level of knowledge/
understanding of the
stakeholders and present
knowledge accordingly step
2.5.1 (A, B)
SME C – Additions
(underlined) to PGSP (As
Step reads in final GSP):
Ask the patient/SDM what
and how much they know
about their disease /condition
and prognosis. Use the
information gathered
during the discussion
(content/tone/level of
vocabulary) to evaluate the
level of knowledge/
understanding of the
stakeholders and present
knowledge accordingly. IF
the patient uses basic
terminology/vocabulary and
low levels of medical
knowledge, THEN use the
same level of vocabulary and
terminology for the remainder
of the meeting and during
procedure 4 step 2.5.1.1. (A,
B, C)
COGNITIVE TASK ANALYSIS
65
decision-making about end-of-life treatment, is a final gold standard protocol. Overall, there are
five main procedures expert palliative care physicians use when engaging in shared decision-
making with patients and family members during end-of-life conversations. The five main
procedures are:
1. Start a structured notecard with all pertinent information
2. Conduct a preliminary meeting with patient/Surrogate Decision Maker (SDM)
3. Consult with palliative care team to determine meeting time, place, and attendees
4. Conduct the shared decision-making meeting with all stakeholders
5. Document important aspects of meeting
The five main procedures include the main procedure for a shared decision-making
meeting, as well as the preparation for the meeting and documentation following the meeting.
According to the subject matter experts interviewed for this study, the preparation and
documentation are part of the main procedures because they are useful new practitioners of
palliative care.
The disaggregated results of Research Question 1 are described in the following sections.
Recalled action and decision steps.
The purpose of this study was to identify the
unconscious and tacit action and decision steps used by palliative care physicians as they involve
and engage with patients and family members at the end stages of life. In other words, this study
sought to identify the unobservable cognitive decision steps that accompanied the observable
action steps of palliative care physicians during patient centered communication and shared
decision-making. Essentially, these action and decision steps contain the fundamental
declarative, procedural, and conditional knowledge novices need to perform the highly complex
cognitive task of shared decision making during end-of-life discussions.
COGNITIVE TASK ANALYSIS
66
In order to conduct this analysis, each action and decision step in the final gold standard
protocol was recorded in a Microsoft Excel spreadsheet. Each step of the gold standard protocol
was coded with an “A” for action steps or a “D” for decision steps. The initial three palliative
care physicians that participated in the semi-structured interviews were each assigned a letter,
“SME A,” SME B,” or “SME C” for identification purposes. Each SME was credited with the
associated action or decision step in their individual protocol, and were marked with a “1” in the
a column on the spreadsheet to identify when their individual protocol included the action and/or
decision steps of the final gold standard protocol. The total number of actions and decisions for
each SME is indicated at the bottom of the spreadsheet. See Appendix F for the spreadsheet
analysis. Table 1 provides a total of each SME’s action and decision steps.
Table 1
Cumulative Action and Decision Steps Captured for Each SME
________________________________________________________________________
Steps
Action Steps Decision Steps Total Steps
SME A 37 23 60
SME B 33 21 54
SME C 37 22 59
Action and decision steps contributed by each SME. Figure 3 identifies the action and
decision steps recalled by each of the three SMEs. In the preliminary GSP, there were a total of
83 action and decision steps. The action and decision steps that were isolated through the
knowledge elicitation technique of CTA may not be solely attributed to only one SME due to
two or more SMEs identifying the same action or decision step.
COGNITIVE TASK ANALYSIS
67
Figure 3. Number of Action Steps, Decision Steps, and Action and Decision Steps for SME A,
SME B, and SME C Captured through CTA. Total Non-Repeating Action and Decision Steps
from the CTA Process Represented in the Final Gold Standard Protocol. Total steps- 88; action
and decision steps – 47 action steps – 41 decision steps
While the preliminary GSP identified a total of 83 action and decision steps, the review
of the preliminary GSP by a fourth SME added five more action and decision steps, making the
total action and decision steps of the final GSP 88 steps. Although there was a total of 88 action
and decision steps in the final GSP, none of the SMEs were able to recall more than 60 steps.
SME A was able to recall the most action and decision steps at 60 steps, contributing to 42.05%
of the GSP. SME C identified 59 action and decision steps, contributing to 42.05% of the GSP.
SME B recalled the lowest amount of action and decision steps, identifying a total of 54 steps
accounting for 37.5% of the final GSP.
Action and decision steps captured in the follow-up interviews. SME A and SME B
reviewed the preliminary protocol generated from the initial interview. During the protocol
37
33
37
23
21
22
60
54
59
0
10
20
30
40
50
60
70
SME
A
SME
B
SME
C
AcKon
Steps
Decision
Steps
Total
COGNITIVE TASK ANALYSIS
68
review, each SME was asked to add, modify, or combine or delete action and decisions steps
described in the preliminary protocol. When SMEs A and B reviewed their individual protocols
in a follow-up interview, the process resulted in increased action and decision steps. The results
of this analysis are shown in Table 2.
Table 2
Additional Expert Knowledge Captured, in Action and Decision Steps, During Follow-up
Interviews
________________________________________________________________________
Additional Steps Captured
SME Action Decision
A 1 1
B 2 1
C 0 0
D 1 4
Note: SME C had no additional action and decision steps because SME C did not complete a review of their
individual protocol. SME D did not participate in the CTA semi-structured interviews and only reviewed the initial
gold standard protocol for additions, modifications, and deletions after it was reviewed by the other 3 SMEs.
Alignment of SMEs in describing the same action and decision steps. The data
analysis using the spreadsheet was also used to determine the frequency and percentage of action
and decision steps described by each SME. The alignments were categorized as highly aligned,
partially aligned, or slightly aligned, or not aligned. If all three SMEs described the same action
or decision step, then that step was considered highly aligned, and a number “3” was added to an
alignment column. If two SMEs described the same action or decision step, the step was
considered partially aligned, and a number “2” was added to the alignment column, and if only
one SME recalled an action or decision step, it was considered slightly aligned, with a “1” being
COGNITIVE TASK ANALYSIS
69
added to the alignment column. Finally, if SME D, the expert who reviewed the preliminary
gold standard protocol, identified an action or decision step, the step was added to the GSP with
no SME given credit, and it was considered not aligned, and the number “0” was placed in the
alignment column on the spreadsheet. Table 3 shows the results of this analysis. Figure 3
demonstrates a visual representation of Table 3.
Table 3
Number and Percentage of Action and Decision Steps that are Highly Aligned, Partially Aligned,
Slightly Aligned, Not Aligned
_____________________________________________________________________________________
Number Percentage
Highly Aligned 2 2.27%
Partially Aligned 20 22.73%
Slightly Aligned 61 69.32%
Not Aligned 5 5.68%
________________________________________________________________________
COGNITIVE TASK ANALYSIS
70
Figure 4. Percentage of Action and Decision Steps that are Aligned with the Final Gold
Standard Protocol: Highly Aligned 2.27%; Partially Aligned 22.73%; Slightly Aligned 69.32%;
Not Aligned 5.68%.
Ultimately, the SMEs were “highly aligned” for a total of two action and decision steps
or 2.27% of the GSP, and “partially aligned” for a total of 20 steps or 22.73% of the GSP. The
majority of the action and decision steps were “slightly aligned” with 61 steps or 69.32% of the
palliative care physician experts recalling an action or decision step included in the final GSP.
Moreover, the initial three palliative care physician experts that participated in the semi-
structured interview protocol were “not aligned” for five steps, or 5.68% of the total action and
decision steps as identified in the final GSP. The implications of these results are further
discussed in Chapter 5.
Research Question Two
2%
23%
69%
6%
Highly
Aligned
ParKally
Aligned
Slightly
Aligned
Not
Aligned
COGNITIVE TASK ANALYSIS
71
The second research question asked, “What is the percentage of action and/or decision
steps, when compared to a gold standard, do expert palliative care physicians omit when they
conduct shared decision-making with patients during end-of-life communication?”
Total knowledge omissions. In addition to using the Microsoft Excel spreadsheet for the
aggregation and analysis of the percentage of recalled action and decision steps, the spreadsheet
was also used to ascertain the number of action and decision steps omitted by the individual
SMEs when recalling how they involve patients and family members in shared-decision making
about end-of-life treatment. The action and decision steps that were included in the final GSP but
omitted by the SME were marked “0.” Using Microsoft Excel, the data was analyzed and the
frequency and percentage of omissions of action and decision steps as compared to the final GSP
is shown in Table 4, with a graphical representation shown in Figure 3. Table 4 provides a
comparison of action and decision steps omitted by each SME when compared to the gold
standard protocol including the range and standard deviation.
COGNITIVE TASK ANALYSIS
72
Table 4
Total Action and Decision Steps, or Expert Knowledge, Omissions by SME when Compared to
the Gold Standard Protocol
________________________________________________________________________
Steps Omitted
Total Action & Action Decision
Decision Steps Steps Steps
Omitted % Omitted % Omitted %
SME A 51 57.95% 24 51.06% 27 65.85%
SME B 55 62.50% 26 55.32% 29 70.73%
SME C 51 57.95% 25 53.19% 26 63.41%
Mean
Omissions 52.33 59.47% 25 53.19% 27.33 66.67%
Range 4 2 3
SD 2.31 1 1.53
Note. Total non-repeating action and decision steps from the CTA process represented in the final gold standard
protocol: Total steps- 88; action and decision steps – 47 action steps; – 41decision steps.
The average omissions of action and decision steps for palliative care physicians when
recalling how to involve patients and family members in patient-centered communication and
shared decision-making was 52.33 or 59.47% (SD ± 2.31) of the total steps indicated in the final
GSP. And, among the three SMEs, the percentage of expert omissions when describing action
steps averaged 25, or 53.19% (SD ± 1), and the average decision steps omitted by experts when
recalling how to engage with patients and family members in patient-centered communication
was 27.33 or 66.67% (SD ± 1.53). As individuals, the SMEs percentage of action and decision
step omissions, or total expert knowledge omissions, was marginally varied when compared to
the gold standard protocol.
COGNITIVE TASK ANALYSIS
73
Analysis of action and decision step omissions. Figure 3 represents the action and
decision step omissions, or expert knowledge omissions, data for SME A, SME B, and SME C
when compared to the final GSP.
Total SME Knowledge Omissions when Compared to the Gold Standard Protocol
Figure 5. Total non-repeating action and decision steps from the CTA process represented in the
final gold standard protocol: Total steps- 88; action and decision steps – 47 action steps; –
41decision steps.
The next chapter will include an overview of the study, a discussion of the findings,
limitations, implications, and future research.
24
26
25
27
29
26
51
55
51
0
10
20
30
40
50
60
SME
A
SME
B
SME
C
OmiTed
AcKon
Steps
OmiTed
Decision
Steps
Combined
OmiTed
AcKon
and
Decision
Steps
COGNITIVE TASK ANALYSIS
74
CHAPTER FIVE: DISCUSSION
Overview of the Study
The primary purpose of this study was to use a cognitive task analysis to capture the
unconscious, tacit procedural and conditional knowledge of expert palliative care physicians as
they involve patients and family members in in-patient patient-centered communication and
shared decision-making. In addition, this study sought to ascertain the critical knowledge
experts omit when they describe how to perform a complex cognitive task, such as engaging in
shared decision-making.
Although no formal hypothesis was stated, the two research questions regarding cognitive
task analysis being used as a method for knowledge elicitation, and the critical omissions of
information from experts in explaining a complex cognitive tasks guided and informed the study.
Indeed, research studies have shown that experts may omit 70% of the critical knowledge and
information a novice requires to perform a cognitively complex task, thus leaving novices to
learn less efficiently and possibly “fill in the gaps” with erroneous information (Clark, 2008).
One major reason for expert omissions is the fact that as tasks are deliberately practiced
and performed, the declarative, procedural, and conditional knowledge becomes unconscious,
and automaticity develops (Clark & Elen, 2006; Feldon, 2007). Although automaticity is
beneficial for experts because it enables greater cognitive capacity for novel situations and
problem solving, it also becomes a “double-edged sword” for experts in communicating essential
knowledge for novices learning a cognitively complex skill.
Medical school faculty and senior physicians, who are deemed experts in the field of
medicine, are charged with providing training and education to medical school students,
residents, fellows, and practicing physicians. While the expertise of senior physicians and
COGNITIVE TASK ANALYSIS
75
faculty members is advantageous for treating patients, performing medical procedures, and
providing information to medical personnel, they may at a disadvantaged in training novices due
to their automaticity in recalling the procedural and conditional knowledge required for
describing cognitive complex tasks. The use of CTA has been shown to be an effective and
efficient method for knowledge elicitation and training in medicine and education (Hoffman &
Militello, 2009). Thus, this study may add to the basis for increased training and research in
medicine, principally, patient-centered communication and shared decision-making during end-
of-life conversations between patients, family members and physicians.
Process of Conducting Cognitive Task Analysis
Selection of Experts
Crispen (2010) replicated the results of previous research completed by Chao and
Salvendy (1994) which indicated that the amount of subject matter experts necessary to
efficiently and effectively conduct a CTA was between three to five experts. Similarly,
Bartholio (2010) found that a majority of cognitive action and decision steps required for a
cognitive task could be elicited with three to four experts. Essentially, research over time has
illustrated that in order to gain the optimal amount of information, with the elicitation of critical
action and decision steps, the ideal number of experts needed to recall a cognitively complex task
reached saturation at three to five experts. In other words, the studies have indicated that there is
a diminishing marginal utility of recalled procedural and conditional knowledge needed to create
a gold standard protocol as the number of subject matter experts increased past the requisite
number of three to five subject matter experts. Thus, to reverse the 70% rule of subject matter
expert omissions in recalling the critical action and decision steps necessary for instructing a
COGNITIVE TASK ANALYSIS
76
novice in a cognitively complex task, a CTA should ideally identify between three and five
subject matter experts (Clark & Feldon, 2006).
In addition to the research on selecting the optimal amount of subject matter experts
necessary for this study, the criteria for selecting the experts included five key components,
including (a) a minimum of five years of consistent and successful on the job professional
experience; (b) experience in a wide variety of settings that are applicable to palliative care; (c)
little to no experience as a trainer or instructor of the skills being captured; (d) high levels of
flexibility, effective verbal skills, and willingness to be interviewed with the use of an audio
recorder; and (e) availability to be interviewed using the CTA protocol for at least a 90 minute
segment of time along with a follow up interview lasting between 20 to 30 minutes (Yates &
Clark, 2011). As stated previously, palliative care is a relatively newly recognized medical
subspecialty, with the ABMS identifying it as a subspecialty in 2006, therefore the subject matter
experts’ previous expertise in related medical specialties was taken into account as part of their
years of experience. Thus, for the purpose of this study, each of the experts met the criteria
based on current and past years of experience, and successful on the job experience. Of note,
one of the SMEs is on the faculty at a university medical center, however, it was determined that
the physician does not specifically train or provide instruction on the skill being captured as part
of this study.
As a researcher outside the field of medicine, identifying and attaining respondents
proved to be difficult. After numerous emails, and the assistance of committee chair members,
the researcher was able to obtain interviews with four palliative care physicians, who were
subsequently judged to be subject matter experts based on the above criteria.
COGNITIVE TASK ANALYSIS
77
Selection of the Cognitive Task
While the identification of the palliative care physicians was clear-cut based on the five
main criteria recognized in the literature, the identification of expertise for the cognitive task in
this study, patient-centered communication and shared decision-making, was more complex.
Heckman and Kautz (2012) suggest that “hard skills,” those skills that include cognitive ability,
are easily measured with achievement tests, IQ tests, and grades. However, it is theorized that
hard skills may not be as valuable as “soft skills,” such as personality traits and communication
skills, because soft skills are regarded as necessary for success in contemporary society. So,
while hard skills can be easily established based on grades, assessment scores, and degrees
attained from universities, recognizing an individual’s proficiency in soft skills proves to be
more complicated. Indeed, the purpose of this study was to capture and document, though a gold
standard protocol, patient-centered communication and shared decision-making between
patients, their family members and a palliative care physician during end-of-life conversations.
In fact, the identification of expertise for this study, the “soft skills” of communication, eliciting
questions, and demonstrating empathy during end-of-life conversations, was not easily
discernable. And, while there are websites dedicated to rating physicians, the reliability of these
sites are not standardized, nor are the comments and ratings vetted for internal reliability because
the ratings rely on the patient’s perception of a physicians’ soft skill abilities. Consequently,
future studies may need to consider how soft skills are documented and assessed relative to
identifying subject mater expertise in areas of communication and interpersonal skills.
Collection of Data
Data collection for this study was conducted with the use of semi-structured interviews
with the first three SMEs. The initial interviews with the first three SMEs took a little less time
COGNITIVE TASK ANALYSIS
78
than the estimated 90 minutes, with the shortest interview lasting 76 minutes and the longest
interview lasing 88 minutes. Given the time constraints of each physician, the researcher strictly
adhered to the maximum allotted time of 90 minutes for the initial interviews.
The interviews for SME A and SME B followed the semi-structured interview protocol
developed for the study, and each SME identified the main steps or procedures they would
perform while engaging in patient-centered communication and shared decision-making. Then,
when prompted by probing questions, SME A and SME B elaborated on the individual action
and decision steps within each delineated procedure. With SME C, the interview began with the
researcher using the same semi-structured interview protocol, however as the interview
progressed, the SME demonstrated difficulty verbalizing the main steps or procedures used when
engaging patients and family members in patient-centered communication and shared decision-
making. Additionally, SME C had difficulty recalling the tacit action and unconscious decision
steps used during this cognitively complex task. Therefore, the researcher altered the semi-
structured interview protocol, shifting to a Critical Decision Method (CDM) knowledge
elicitation technique, whereby SME C was guided through multiple-passes of recalling a
particular incident of shared decision-making with a specific patient. Then the researcher guided
the expert through a set of probing questions to illicit the tacit and unconscious task specific
cognitive details related to the particular incident with the patient (Hoffman, Crandall, Shadbolt,
1998). One of the possible reason for SME C’s difficulty in recalling the main procedures, as
well a particular action and decision steps, may be due to the process of their automation of
procedural and conditional knowledge (Feldon, 2007). In effect, the expert may have completely
automated the process of patient-centered communication and shared decision making to allow
more mental capacity for novel situations and problem solving. By changing to the use of a
COGNITIVE TASK ANALYSIS
79
CDM, SME C was better able to recall the conscious and unconscious action and decision steps
for shared decision making. Given this fact, it is essential that knowledge analysts have multiple
tools for interviewing, and match the interview technique with the types of knowledge being
sought (Chao & Salvendy, 1994).
While the researcher and physicians adhered to the narrow time frame for the initial
interview, the follow up review interviews took longer than the estimated 20 minutes, with the
each of the SMEs taking between 35 to 48 minutes to review their respective protocol. Though
the SMEs did not make substantial changes to the action and decision steps of their individual
protocols, each SME seemed to be more engaged in reviewing the procedures and examining
each action and decision step for accuracy. This may have been because the SMEs may have
found it easier to look at a hard copy of the protocol, and attend to the precision of the text due to
the reduction of cognitive load (Sweller 1998). In fact, the subject matter experts may have been
able to focus on the protocol because the protocol was similar to a worked example, thus
reducing the load on the SME’s working memory. Further research is needed to identify the
reasons for the attention to precise language and the possible reduction of cognitive load through
the use of a worked example.
The final process of this CTA study and data collection included a review of the
preliminary gold standard protocol by a fourth SME. The review by SME D added an additional
procedure whereby a palliative care physician would conduct a preliminary meeting with the
patient and family members. Aside from the additional main procedure added by the fourth
SME, five additional action and decision steps were added to the gold standard protocol. Further
research needs to be conducted to determine if the added procedure of a preliminary meeting
with patients and family members would be appropriate or feasible in different institutions and
COGNITIVE TASK ANALYSIS
80
settings. Finally, further research is needed to determine if a longer initial interview (3i) may
change the procedures and increase the number of action and decision steps.
Discussion of Findings
There were no formal hypotheses for this research study, however, two main research
questions guided the study.
Research Question 1
The first research question asked, “What are the essential decision and actions steps
palliative care physicians recall when they describe how they involve patients and family
members in shared-decision making about end-of-life treatment?” To answer this question, three
SMEs were involved in the initial semi-structured interview protocol, and one SME participated
by reviewing the preliminary gold standard protocol developed from the compilation of the
initial individual SME protocols.
Action steps versus decision steps. As indicated in Chapter Four: Results, all three
SMEs described more action steps than decision steps. In fact, the action steps recalled by SME
A were 62% of the total steps in the GSP, SME B’s recollection of action steps comprised 64%
of the total steps in the GSP total, and the action steps recollected by SME C made up 59% of the
total steps identified in the GSP. The results from this study correlate with previous research
about expertise and automaticity, in which the deliberate practice of cognitively complex tasks
allows for procedural and conditional knowledge to become unconscious, allowing experts to
have more cognitive capacity for problem solving and novel situations (Clark, 2014; Clark &
Elen, 2006; Feldon, 2007). Moreover, these results support research that suggests that experts
are able to recall more action steps, or physical actions performed, because action steps afford
experts the ability to create mental images (Clark, 2014). In contrast, decisions and subsequent
COGNITIVE TASK ANALYSIS
81
decision steps are not directly observable, thus do not allow experts to form mental models, thus
decision steps more difficult to recall. Indeed, the research suggests that while decision steps
are more difficult for experts to recall, they are critical to the instruction and training for novices
in complex tasks. Consequently, the amount of decision steps versus the amount of action steps
recalled corresponds with other CTA studies, in which experts can more easily recall action steps
than they can recall decision steps. (Clark, 2014; Canillas, 2010; Mutie, 2015)
Action and decision steps during SME reviews. After the initial interviews using the
semi-structured interview protocol, the researcher reviewed, coded, and analyzed the interview
transcript from each expert and developed an individual SME protocol describing the main
procedures, and action and decision steps a palliative care physician would engage in during
patient centered communication and shared decision-making. Once the individual protocol was
developed, the researcher met with the SME for a second, follow up interview. During the
second interview, each SME was asked to review and make any revisions to their individual
protocol. The SMEs, along side the researcher reviewed the protocol and, and in each instance,
the SME made additions and to the protocol. However, unlike other CTA protocol studies, the
SMEs in this study did not make substantial changes to the number of action and decision steps
(Zepeda-McZeal, 2014; Hammitt, 2014; Mutie, 2014). For example, a CTA study involving
informal classroom walk-throughs, the subject matter experts added between three and 43
additional action and decision steps to their initial protocol (Hammitt, 2014). Furthermore, in a
CTA conducted by Mutie (2014), the subject matter experts recalled between 30-49 additional
action and decision steps during the interview protocol review. In contrast, for the current study,
SME A added only one action step and one decision step, totaling an addition of two steps.
Likewise, SME B added two action steps and one decision step for a total of three additional
COGNITIVE TASK ANALYSIS
82
steps. As indicated previously, during the second interviews, much of the time was used for the
clarification of wording within each step.
Expert review of the preliminary gold standard protocol. The review of the
preliminary gold standard protocol was initiated and completed with SME D, who had not
participated in the initial semi-structured interview and subsequent individual protocol
development. At the time of the interview, SME D was given a copy of the PGSP to review.
During the review, SME D was guided through the PGSP and asked to make revisions. SME D
was asked to add, delete, or combine any steps to make the protocol more accurate for a novice
to follow as an instructional tool or job aide. SME D made several changes to the PGSP, adding
a procedure, as well as deleting action and decision steps. In a case such as this, where the final
SME makes significant changes to a PGSP, a decision was required regarding the final GSP.
The researcher had to analyze the data and decide if the first three SMEs were incorrect in their
recollection of the process of shared decision making at the end of life, or if the process of shared
decision-making is matter of different policies at different medical institutions, or if SME D was
an anomaly with regards to how he would engage with patients and families in patient-centered
communication and shared decision making, or if the data gather in SME D’s interview would
add to the steps in the final GSP, and no credit be given to the first three SMEs. After
consultation with other CTA knowledge elicitation experts, the researcher decided to develop a
GSP with the additions of SME D, with no credit given to SME A, SME B, or SME C, thus
increasing the frequency count of expert omissions. Accordingly, in the final GSP, one procedure
and five decision steps were added to reflect a more complete GSP representing the practice
within multiple health care institutions.
COGNITIVE TASK ANALYSIS
83
Alignment of Action and Decision Steps. There was a significant lack of alignment
between the action and decision steps from each SME, with 75% of the steps being not aligned
or slightly aligned to the final gold standard protocol. There were two action steps, inquiring
into the emotional well being of the patient/SDM and asking what the patient/SDM what is
known about their condition that were highly aligned, with all three subject matter experts
identifying them as part of their protocol. There were a total of 20 steps, or 23% of action and
decision steps that were partially aligned to the gold standard protocol, with at least two subject
matter experts identifying them as part of the protocol during their interviews. These steps were
mainly action steps and included the actions taken relating to the preparation of the shared
decision-making meeting, including reviewing the patient’s history and current medical issues,
as well as identifying the key stakeholders, and setting up the most conducive environment for
the meeting. Two other action steps that were partially aligned were the actions steps to inquire
about the goals of the patient and to identify how the patient had lived their life prior to the
serious illness. The action and decision steps that were not aligned because they were attributed
to SME D, included one action step and four decision steps. The one decision step included not
mentioning death if it had not already been discussed, and the action steps included inquiring
about a patient’s advanced directive and the election of a surrogate decision maker, and inviting
the primary care physician and patient’s social worker and/or psychiatrist.
Research Question 2
The second research question asked, “What is the percentage of action and/or decision
steps, when compared to a gold standard, do expert palliative care physicians omit when they
conduct shared decision-making with patients during end-of-life communication?”
COGNITIVE TASK ANALYSIS
84
Expert knowledge omissions. To determine the percentage of omissions for the task of
patient centered communication and shared decision making during end-of-life communication,
each SME’s individual protocol was compared to the final gold standard protocol. After the
data was aggregated and compared to the GSP, this study found that each SME omitted an
average of 53.19% of the action steps, and 66.67% of the decision steps for involving patients
and family members in shared decision-making at the end of life. Again, studies have shown
that experts develop automaticity of procedural and conditional knowledge to allow for more
cognitive load to be given for problem solving and novel situations (Clark & Elen, 2006; Feldon,
2007). This study aligns with other CTA studies because the data indicates that as experts
develop automaticity as result of deliberate practice, they are susceptible to the omission of 70%
of the critical action and decision steps upon recalling a cognitively complex task, such as
involving patients and family members in shared decision-making at the end-of-life (Clark &
Feldon, 2006; Feldon, 2004). Though the cognitive task completed for this study involved soft
skills, it is similar to other CTA studies in the medical field where expert physicians have greater
omissions of decision steps versus action steps, and as research has shown, decision steps are
more difficult to recall due to the fact that an expert may be unable to create mental images of
the actual decision step (Clark, 2014). On a final note, the nature of the task being studied, in
this case, a task that has increased complexity due to the emotional aspects and individual
medical situations, may influence the amount of action and decision steps captured as part of a
CTA. As Hoffman (1987) notes, the length and the complexity of the task both influence the
number of action and decision steps the expert may recall.
COGNITIVE TASK ANALYSIS
85
Limitations
This study and the results of the study were consistent with other CTA studies in
medicine, as well as in other fields of study. In fact, this study supports the 70% rule whereby
experts omit up to 70% of the action and decision steps necessary for effective and efficient
instruction of novices (Clark, 2014). The following sections will discuss the limitations of the
study.
Confirmation Bias
The first limitation of this study is confirmation bias. As stated previously, the data
collected for this study was elicited and analyzed by an individual outside of the health care
profession. The researcher used the process of “bootstrapping,” in which the researcher acquired
the necessary knowledge of palliative care, patient-shared communication, and shared decision-
making through an in depth literature review (Chipman, Schraagen & Shalin, 2000). Although,
the researcher had previous experiences with hospice care, and with the end-of-life of loved
ones, the researcher had no experiences with engaged in patient-centered shared decision-making
with palliative care physicians or other healthcare providers. However, because of the past
experiences with end-of-life care, the researcher made an effort to decrease confirmation bias by
reducing preconceived ideas of how palliative care physicians should involve patients and family
members in patient-centered communication and shared-decision making during end-of-life
conversations. Confirmation bias can become a threat when researchers attempt to insert or edit
their knowledge into the research. Clark (2014) identified confirmation bias in healthcare
whereby CTA analysis with healthcare backgrounds may edit the information from expert
interviews to fit within their own experiences. For this study, the researcher had no previous
experience with palliative care physicians and patient-centered, shared decision-making during
COGNITIVE TASK ANALYSIS
86
end-of-life communication, therefore the expert interview data was new, therefore the researcher
was able to limit the editing of the information to fit past experiences.
Internal Validity
The second limitation for this study was internal validity. As a methodology that
predominately uses interviews for data collection, CTA is a qualitative research method,
therefore internal validity is a limitation to this study. Interval validity limits this study because
there is a question of how closely the palliative care expert physicians’ protocols match the
reality of how they engage patients and family members in patient-centered communication and
shared decision-making at the end-of-life (Merriam, 2009). One way to address the threat to
internal validity is to observe palliative care physicians as they engage with patients and family
members in end-of-life communication using the GSP. To minimize the threat of internal
validity, future studies should be conducted to observe expert’s as they involve patients and
family members in the process of shared decision-making to identify any misalignments in what
the expert indicated in the protocol versus what observable actions are taken.
External Validity
Another limitation of this qualitative study is its external validity. Although the experts
interviewed for this study work with different patient populations, the study is limited by its
small sample size (n=4), thus there may be a threat to the generalizability of the results to larger
populations from a quantitative perspective. However, from a qualitative perspective, Merriam
(2009) notes that the reader and how the topic resonates with the individuals interested in the
study’s topic determine generalizability. By increasing the number of subject mater experts, in
addition to different contexts in which the experts practice, a more complete protocol may be
COGNITIVE TASK ANALYSIS
87
developed. Possible contexts for future research may include in-patient hospital hospice care
settings, home hospice care, and other end-of-life care settings.
A second limitation related to the external validity of the study is that the palliative care
physicians interviewed as part of this study are all located at large hospitals in Southern
California, therefore the unique characteristics of different locations and populations may be an
issue for external validity. Moreover, the study may be limited due to the differences in health
literacy and different cultural views at the end-of-life in diverse racial and ethnic populations
(Enguidanos, Yip, & Wilber, 2005; Smith, McCarthy, Paulk, Balboni, Maciejewski, Block, &
Prigerson, 2008). Future research needs to be completed on larger, more diverse populations to
assess and increase the generalizability of the GSP in shared decision-making in palliative care.
Another limitation with regard to external validity is related to the review of the
preliminary gold standard protocol by SME D. SME D, who was not involved in the initial
semi-structured interviews, completed the review of the preliminary gold standard protocol and
spoke from a position of policy creation and leadership. Unlike the first three subject matter
experts, SME D sees palliative care patients, in addition to acting in a leadership role and
working with policies and procedures related to in patient palliative care programs.
Implications
The goal of using a CTA is to capture the unconscious, tacit declarative, procedural, and
conditional knowledge of experts as they perform cognitively complex tasks (Clark et al., 2008).
The use of CTA for instruction in the medical field has been shown to be effective and efficient
in reducing the 70% omission rate of healthcare experts as they describe how they perform
cognitive tasks (Clark, 2014). Research using CTA in healthcare has also been shown to be cost
effectiveness and reduce error rates among novices performing tasks taught using the CTA
COGNITIVE TASK ANALYSIS
88
methodology. This study supports the use of CTA for the elicitation of an expert unconscious
and unobservable knowledge in the cognitive task of patient-centered communication and shared
decision- making at the end of life.
More importantly, this study has implications for any health care provider and social
service agency that is setting up a palliative care program, or has an existing palliative care
program. The current study can be used to increase the training and development for patient-
centered communication and shared decision-making in palliative care.
Future Research
There are several indications for future research using CTA in palliative care, patient-
centered communication, and shared decision-making. Current research is limited in palliative
care and shared-decision making, and the current study provides preliminary evidence of the
need for national efforts to develop gold standards for shared decision making around goals of
care for all seriously ill patients. The use of CTA in palliative care is limited, with a search
indicating there is one known CTA study on a theoretical model for surrogate decision-making at
end-of-life in adult intensive care units (Dionne-Odom, 2013). As a result of the current study,
future research may consider using the GSP generated from this study in a randomized
experimental design to instruct novice palliative care physicians in patient-centered-
communication and shared decision-making at the end of life. If the a randomized experimental
design using the GSP developed as part of this study, it would be similar to other healthcare CTA
studies where the short term and long term learning benefits of novices would be assessed using
traditional training methods versus using CTA guided training techniques.
Additional research is also indicated for populations with differing palliative care needs.
Current pediatric palliative care research is limited, thus conducting a randomized experimental
COGNITIVE TASK ANALYSIS
89
study in involving patients and family members in patient-centered communication and shared
decision-making using the GSP for pediatric palliative care patients may be considered.
Furthermore, additional research is indicated for the use CTA in shared decision-making in
different contexts. The current study provides for a protocol to be used in in-patient palliative
care settings. Further research is needed to identify if the current GSP is applicable in home
hospice care, out-patient palliative care or other end-of-life settings and contexts. Additional
randomized experimental design studies may also be indicated for other potential palliative care
patients including patients of different racial and ethnic backgrounds, as well as patients in
intensive care units and trauma care units.
Finally, there is increased attention on the Affordable Care Act and on the patient
centered medical home (PCMH) model of care to contain costs and provide more patient-
centered medical practice (Adashi, Geiger & Fine2010; Zajac, Norris, & Keenum, 2014). In
PCMH models of care, the focus is on creating and sustaining the relationship between the
patient and the primary care physician. Moreover, the proactive and continuous collaboration and
coordination between primary care physicians, medical specialists, and patients, supports patient-
centered communication and shared decision-making during all aspects of patient care, including
acute illness and end-of-life care. As the healthcare system moves toward meeting the
requirements of the ACA, and additional research is conducted on PCMHs, CTA research may
be conducted as an effectual way to elicit the unconscious automated knowledge of experts for
use in training novices. In fact, future research may include a randomized experimental study
using the GSP developed in this this study for involving primary care physicians, patients and
family members in PCMH in patient-centered communication and shared decision-making at the
end of life.
COGNITIVE TASK ANALYSIS
90
Conclusion
The purpose of this study was to add to the body of research regarding the benefits of
using CTA as a method for knowledge elicitation and instruction for cognitively complex tasks.
CTA has been shown to be effective in capturing the critical unconscious and automated
knowledge of experts as they perform cognitively complex tasks. Moreover, CTA has been
shown to be efficient in training novices using protocols developed from experts’ tacit
procedural and conditional knowledge. Current training in healthcare typically relies on experts
who are senior physicians and medical school faculty. This study shows that palliative care
experts omitted a total of 59.47% action and decision steps, with 53% of the action steps omitted
and 67% of the decision steps omitted. This study provides further evidence that senior
physicians and medical school faculty omit up to 70% of the critical knowledge novices require
to successfully perform a cognitive task. Consequently, this study adds to the research indicating
that educational systems, specifically medical education involving shared decision-making and
palliative care would benefit from using CTA as a knowledge elicitation and training method.
COGNITIVE TASK ANALYSIS
91
References
Adashi, E. Y., Geiger, H. J., & Fine, M. D. (2010). Health care reform and primary care—the
growing importance of the community health center. New England Journal of
Medicine, 362(22), 2047-2050.
Ahmed, N., Bestall, J. E., Ahmedzai, S. H., Payne, S. A., Clark, D., & Noble, B. (2004).
Systematic review of the problems and issues of accessing specialist palliative care by
patients, careers and health and social care professionals. Palliative Medicine, 18(6), 525-
542.
Alexander, P.A. (2003). The development of expertise: The journey from acclimation to
proficiency. Educational Researcher, 32(8), 10-14.
Anderson, J.R. (1982). Acquisition of cognitive skill. Psychological Review, 89(4), 369-406.
Anderson, J. R. (1993). Problem solving and learning. American Psychologist, 48(1), 35-44.
Anderson, J.R. (1996a). ACT: A simple theory of complex cognition. American Psychologist,
51(4), 355-365.
Anderson, J. R. (1996b). The Architecture of Cognition. Mahwah, NJ: Lawrence Erlbaum
Associates.
Anderson, J.R., & Fincham, J.M. (1994). Acquisition of procedural skills from examples.
Journal of experimental psychology: Learning, memory, and cognition 20(6), 1322-1340.
Anderson, J. R. & Schunn, C.D. (2000). Implications of the ACT-R learning theory: No magic
bullets. In R. Glaser (Ed.), Advances in instructional psychology (Vol. 5). Mahwah,
NJ: Erlbaum.
Anderson, L.W., & Krathwohl (Eds.). (2001). A Taxonomy for learning, teaching, and assessing:
A revision of Bloom's taxonomy of educational objectives. New York, NY: Longman.
COGNITIVE TASK ANALYSIS
92
Ayd, F. J. (1962). The hopeless case: medical and moral considerations. JAMA,181(13), 1099-
1102.
Back, A. L., Arnold, R. M., Baile, W. F., Fryer-Edwards, K. A., Alexander, S. C., Barley, G. E.,
& Tulsky, J. A. (2007). Efficacy of communication skills training for giving bad news
and discussing transitions to palliative care. Archives of Internal Medicine, 167(5), 453-
460.
Bancino, R., & Zevalkink, C. (2007). Soft Skills: The New Curriculum for Hard-Core Technical
Professionals. Techniques: Connecting Education and Careers (J1), 82(5), 20-22.
Bartholio, C. W. (2010). The use of cognitive task analysis to investigate how many experts must
be interviewed to acquire the critical information needed to perform a central venous
catheter placement (Doctoral dissertation). Retrieved from
http://digitallibrary.usc.edu/cdm/ref/collection/p15799coll127/id/385767
Barratt, A. (2008). Evidence based medicine and share decision making: the challenge of getting
both evidence and preferences into health care. Patient Education and Counseling, 73(3),
407-412.
Bedard, J. & Chi, M.T.H. (1992). Expertise. Current Directions in Psychological Science, 1(4),
135-139.
Benini, F., Spizzichino, M., Trapanotto, M., & Ferrante, A. (2008). Pediatric palliative
care. Italian journal of pediatrics, 34(4), 34-4.
Bickel-Swenson, D. (2007). End-of-life training in US medical schools: a systematic literature
review. Journal of palliative medicine, 10(1), 229-235.
Billings, J. A., & Block, S. (1997). Palliative care in undergraduate medical education: status
report and future directions. JAMA, 278(9), 733-738.
COGNITIVE TASK ANALYSIS
93
Broad, J. B., Gott, M., Kim, H., Boyd, M., Chen, H., & Connolly, M. J. (2013). Where do people
die? An international comparison of the percentage of deaths occurring in hospital and
residential aged care settings in 45 populations, using published and available
statistics. International journal of public health, 58(2), 257-267.
Brumley, R., Enguidanos, S., Jamison, P., Seitz, R., Morgenstern, N., Saito, S., McIlwane, J.,
Hillary, K. and Gonzalez, J. (2007). Increase satisfaction with care and lower costs:
Results of a randomized trial of in-home palliative care. J Am Geriatr Soc 55993-1000.
Bunker, J. P. (2001). The role of medical care in contributing to health improvements within
societies. International Journal of Epidemiology, 30(6), 1260-1263.
Lynn, J., Harrold, J., & Schuster, J. L. (2011). Handbook for mortals: guidance for people facing
serious illness. Oxford University Press.
Bunker, J. P., Frazier, H. S., & Mosteller, F. (1994). Improving health: measuring effects of
medical care. The Milbank Quarterly, 225-258.
Canillas, E. N. (2010). The use of cognitive task analysis for identifying the critical information
omitted when experts describe surgical procedures
(Unpublished doctoral dissertation). University of Southern California, Los Angeles.
Carter, B. S., Howenstein, M., Gilmer, M. J., Throop, P., France, D., & Whitlock, J. A. (2004).
Circumstances surrounding the deaths of hospitalized children: opportunities for pediatric
palliative care. Pediatrics, 114(3), e361-e366.
COGNITIVE TASK ANALYSIS
94
Chao, C-J., & Salvendy, G. (1994). Percentage of procedural knowledge acquired as a function
of the number of experts from whom knowledge is acquired for diagnosis, debugging,
and interpretation of tasks. International Journal of Human-Computer Interaction, 6(3),
221-233.
Charles, C., Gafni, A., & Whelan, T. (1997). Shared decision-making in the medical encounter:
what does it mean? Social Science & Medicine, (44)5, 681-692.
Chi, M. T. H. (2006). Two approaches to the study of experts’ characteristics. In K. A.
Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge
handbook of expertise and expert performance (pp. 21–30). New York: Cambridge
University Press.
Chipman, S. F., Schraagen, J. M., & Shalin, V. L. (2000). Introduction to cognitive task
analysis. In J. M. Schraagen, S. F. Chipman, & V. L. Shalin (Eds.), Cognitive task
analysis (pp. 3-23). Mahwah, NJ: Lawrence Erlbaum Associates.
Clark, D. (2007). From margins to centre: A review of the history of palliative care in cancer.
The Lancet oncology, 8(5), 430-438.
Clark, R.E. (1999). Ying and yang: Cognitive motivational processes in multimedia learning
environments. In J. van Merriënboer (ed.), Cognition and multimedia design. Herleen,
Netherlands: Open University Press.
Clark, R.E. (2008). Resistance to change: Unconscious knowledge and the challenge of
unlearning. In D.C. Berliner, & H. Kupermintz (Eds.), Changing institutions,
environments, and people. Mahwah, NJ: Lawrence Erlbaum Associates.
COGNITIVE TASK ANALYSIS
95
Clark, R.E. (2014). Cognitive task analysis for expert-based instruction in healthcare. In
Spector, J.M. Merrill, M.D. Elen, J. & Bishop, M.J. (Eds.), Handbook of research on
educational communications and technology (4
th
ed., pp.541-551). New York, NY:
Springer.
Clark, R., & Elen, J. (2006). When less is more: Research and theory insights about instruction
for complex learning. In J. Elen & R.E. Clark (Eds.), Handling complexity in learning
environments: Theory and research (pp. 283-295). New York, NY: Elsevier.
Clark, R. E., & Estes, F. (1996). Cognitive task analysis. International Journal of Educational
Research, 25(5), 403-417.
Clark, R. E., Feldon, D. F., van Merriënboer, J. J. G., Yates, K. A., & Early, S. (2008). Cognitive
task analysis. In J. M. Spector, M. D. Merrill, J. van Merriënboer, & M. P. Driscoll
(Eds.), Handbook of research on educational communications and technology (3rd ed.,
pp. 577-593). Mahwah, NJ: Lawrence Erlbaum Associates.
Contro, N. A., Larson, J., Scofield, S., Sourkes, B., & Cohen, H. J. (2004). Hospital staff and
family perspectives regarding quality of pediatric palliative care. Pediatrics, 114(5),
1248-1252.
Cooke, N. J. (1994). Varieties of knowledge elicitation techniques. International Journal of
Human-Computer Studies, 41(6), 801-849.
Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of
procedural knowledge. User Modeling and User-Adapted Interaction, 4, 253-278.
Crandall, B., & Getchell-Reiter, K. (1993). Critical decision method: A technique for eliciting
concrete assessment indicators from the intuition of NICU nurses. Advances in Nursing
Science, 16(1), 47-51.
COGNITIVE TASK ANALYSIS
96
Crandall, B., Klein, G., & Hoffman, R. R. (2006). Working minds: A practitioner’s guide to
cognitive task analysis. Cambridge, MA: MIT Press.
Crispen, P. D. (2010). Identifying the point of diminishing marginal utility for cognitive task
analysis surgical subject matter expert interviews (Doctoral dissertation). Available from
ProQuest Dissertations and Theses database. (UMI No. 3403725)
Cutler, D. M. (2007). The lifetime costs and benefits of medical technology. Journal of Health
Economics, 26(6), 1081-1100.
Cutler, D. M., Rosen, A. B., & Vijan, S. (2006). The value of medical spending in the United
States, 1960–2000. New England Journal of Medicine, 355(9), 920-927.
DeHaes, H. (2006). Dilemmas in patient centeredness and shared decision making: a case for
vulnerability. Patient Education and Counseling, 62(3), 291-298.
Delgado‐Guay, M. O., Parsons, H. A., Li, Z., Palmer, L. J., & Bruera, E. (2009). Symptom
distress, interventions, and outcomes of intensive care unit cancer patients referred to a
palliative care consult team. Cancer, 115(2), 437-445.
Duff, R. S., & Campbell, A. G. (1973). Moral and ethical dilemmas in the special-care
nursery. The New England journal of medicine.
Enguidanos, S., Housen, P., Penido, M., Mejia, B. & Miller, J.A. (2014). Family members’
perceptions of inpatient palliative care consult services: a qualitative study. Palliative
Medicine 28(1), 42-28.
Enguidanos, S., Yip, J., & Wilber, K. (2005). Ethnic variation in site of death of older adults
dually eligible for Medicaid and Medicare. Journal of the American Geriatrics
Society, 53(8), 1411-1416.
COGNITIVE TASK ANALYSIS
97
Embry, K. K. (2012). The use of cognitive task analysis to capture expertise for tracheal
extubation training in anesthesiology (Doctoral dissertation). Retrieved from
http://digitallibrary.usc.edu/cdm/ref/collection/p15799coll127/id/678652
Ericsson, K. A. (2004). Invited Address: Deliberate practice and the acquisition and
maintenance of expert performance in medicine and related domains. Academic
Medicine, 79(10), s70-s81.
Ericsson, K.A. & Charness, N. (1994). Expert performance its structure and acquisition.
American Psychologist, 49(8), 725-747.
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the
acquisition of expert performance. Psychological Review, 100(3), 363-406.
Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence of
maximal adaptation to task constraints. Annual Review of Psychology, 47(1), 273-305.
Feldon, D. F. (2004). Inaccuracies in expert self-report: Errors in the description of strategies
for designing psychology experiments (Doctoral dissertation). Available from ProQuest
Dissertations and Theses database. (UMI No. 3145196)
Feldon, D. (2007). Implications of research on expertise for curriculum and pedagogy.
Educational Psychology Review, 19(2), 91-110.
Feldon, D. F., & Clark, R, E. (2006). Instructional implications of cognitive task analysis as a
method for improving the accuracy of experts’ self-report. In G Clareabout & J. Elen
(Eds.), Avoiding simplicity, confronting complexity: Advances in studying and designing
(computer-based) powerful learning environments (pp. 109-116). Rotterdam,
Netherlands: Sense Publishers.
COGNITIVE TASK ANALYSIS
98
Flynn, C. L. (2012). The relative efficiency of two strategies for conducting cognitive task
analysis. (Doctoral dissertation). Retrieved from
http://gradworks.umi.com/35/61/3561771.html
Feudtner, C., Kang, T. I., Hexem, K. R., Friedrichsdorf, S. J., Osenga, K., Siden, H., ... & Wolfe,
J. (2011). Pediatric palliative care patients: a prospective multicenter cohort
study. Pediatrics, 127(6), 1094-1101.
Feudtner, C., Womer, J., Augustin, R., Remke, S., Wolfe, J., Friebert, S., & Weissman, D.
(2013). Pediatric palliative care programs in children’s hospitals: a cross-sectional
national survey. Pediatrics, 132(6), 1063-1070.
Fitzsimons, D. Mullan, D., Wilson, J.S., Conway, B. Corcoran, B., Dempster, M., Gamble, J.,
Stewart, C., Rafferty, S., McMahon, M. MacMahon, J., Mulholland, P., Stockdale, P.,
Chew, E., Hanna, L., Brown, J., Ferguson, G., & Fogarty, D. (2007). The challenge of
patients' unmet palliative care needs in the final stages of chronic illness. Palliative
Medicine, 21: 313-322, doi:10.1177/0269216307077711
Freytag, J. (2012). Barriers that define a genre of shared decision making in palliative care
communication. Journal of Communication in Healthcare, 5(2), 140-146.
Gade, G., Venohr, I., Conner, D., McGrady, K., Beane, J., Richardson, R. H., ... & Penna, R. D.
(2008). Impact of an inpatient palliative care team: a randomized controlled trial. Journal
of palliative medicine, 11(2), 180-190.
Glaser, R., & Chi, M. T. H. (1988). Overview. In M. T. H. Chi, R. Glaser, & M. Farr (Eds.), The
nature of expertise (pp. xv-xxviii). Mahwah, NJ: Lawrence Erlbaum Associates.
COGNITIVE TASK ANALYSIS
99
Gott, S. P., Hall, E. P., Pokorny, R. A., Dibble, E, & Glaser, R. (1993). A naturalistic study of
transfer: Adaptive expertise in technical domains. In D. K. Detterman & R. J. Sternberg
(Eds.), Transfer on trial: Intelligence, cognition, and instruction (pp. 255-288).
Norwood, NJ: Ablex.
Gucev, G. V. (2012). Cognitive task analysis for instruction in single-injection ultrasound
guided-regional anesthesia (Doctoral dissertation). Available from ProQuest
Dissertations and Theses database. (UMI No. 3513770)
Hamilton, B. E., Hoyert, D. L., Martin, J. A., Strobino, D. M., & Guyer, B. (2013). Annual
summary of vital statistics: 2010–2011. Pediatrics, peds-2012.
Hammel, J. F., Sullivan, A. M., Block, S. D., & Twycross, R. (2007). End-of-life and palliative
care education for final-year medical students: a comparison of Britain and the United
States. Journal of Palliative Medicine, 10(6), 1356-1366.
Heckman, J. J., & Kautz, T. (2012). Hard evidence on soft skills. Labour economics, 19(4), 451-
464.
Higginson, I., Finlay, I., Goodwin, D., Hood, K., Edwards, H. & Norman, C. (2003). Do
hospital-based palliative care teams alter end-of-life experiences of patients and their
caregivers? J Pain Symptom Mangem, 25, 150-168.
Himelstein, B. P., Hilden, J. M., Boldt, A. M., & Weissman, D. (2004). Pediatric palliative
care. New England Journal of Medicine, 350(17), 1752-1762.
Hoffman, R. R., Crandall, B. W., & Shadbolt, N. R. (1998). Use of the critical decision method
to elicit expert knowledge: A case study in cognitive task analysis methodology. Human
Factors, 40(2), 254-276.
COGNITIVE TASK ANALYSIS
100
Hoffman, R. & Militello, L. (2009). Perspectives on cognitive task analysis: Historical origins
and modern communities of practice. New York: Psychology Press.
Hoffman, R., & Woods D. D. (2000). Studying cognitive systems in context: Preface to the
special section. Human Factors: The Journal of Human Factors Ergonomics Society, 42
(1), 1-7.
Horowitz, R., Gramling, R., & Quill, T. (2014). Palliative care education in US medical schools.
Medical Education, 48(1), 59-66. doi:10.1111/medu.12292
Hsu, C., Coleman, K., Ross, T. R., Johnson, E., Fishman, P. A., Larson, E. B., ... & Reid, R. J.
(2012). Spreading a patient-centered medical home redesign: a case study. The Journal of
ambulatory care management, 35(2), 99-108.
Hung, W., Ross, J., Boockvar, K., Siu, A. (2011). Recent trends in chronic disease, impairment,
and disability among older adults in the United States. BMC Geriatrics 11(1)47.
doi:10.1186/1471-2318-11-47
Irby, D. M., & Wilkerson, L. (2003). Educational innovations in academic medicine and
environmental trends. Journal of General Internal Medicine, 18(5), 370-376.
Kirschner, P. A, Sweller, J., & Clark, R. E. (2006). Why minimally guided instruction does not
work: An analysis of the failure of constructivist, discovery, problem based, experiential,
and inquiry based teaching. Educational psychologist, 41(2), 75-86. doi:
10.1207/s15326985ep4102_1
Klein, G., Calderwood, R., & MacGregor, D. (1989). Critical decision method for eliciting
knowledge. IEEE Transactions on Systems, Man, and Cybernetics, 19(3), 462-473.
Klick, J. C., & Hauer, J. (2010). Pediatric palliative care. Current problems in pediatric and
adolescent health care, 40(6), 120-151.
COGNITIVE TASK ANALYSIS
101
Kocher, R., Emanuel, E. J., & DeParle, N. A. M. (2010). The Affordable Care Act and the future
of clinical medicine: the opportunities and challenges. Annals of internal
medicine, 153(8), 536-539.
Korones, D. N. (2007). Pediatric palliative care. Pediatrics in Review, 28(8), e46-e56.
Landefeld, C., Winker, M, & Chernof, B. (2009). Clinical care in the aging century-announcing
“Care of the aging patient: From evidence to action.” JAMA, 302(24), 2703-2704.
Kumar, S. P., Goyal, M., Sisodia, V., & Kumar, V. K. (2014). Availability, current issues, and
anticipation training for clinician-patient communication in palliative care: Learning and
doing or learning by doing? Indian journal of palliative care, 20(3), 245.
LeBlanc, A., Kenny, D. A., O'Connor, A. M., & Légaré, F. (2009). Decisional conflict in patients
and their physicians: a dyadic approach to shared decision making. Medical decision
making, 29(1), 61-68.
Lorenz, K., Lynn, J., Dy, S., Shugarman, L., Wilkinson, A., Mularski, R., Morton, S., Hughes,
R., Hilton, L., Maglione, M., Rhodes, S., Rolon, C., Sun, V. & Shekelle, P. (2008).
Evidence for improving palliateive care at the end of live: A systematic review. Annals of
Internal Medicine 148(2), 147-159.
Merrill, M. D. (1994). Instructional Design Theory. Englewood Cliffs, NJ: Educational
Technology Publications.
Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and
Development, 50(3), 43-49.
COGNITIVE TASK ANALYSIS
102
Merrill, M. D. (2006). Hypothesized performance on complex tasks as a function of scaled
instructional strategies. In J. Elen & R. E. Clark (Eds.), Handling complexity in learning
environments: Research and theory (pp. 265-281). Oxford, UK: Elsevier Science
Limited.
Metzger, M., Norton, S. A., Quinn, J. R., & Gramling, R. (2013). Patient and family members'
perceptions of palliative care in heart failure. Heart & Lung: The Journal of Acute and
Critical Care, 42(2), 112-119.
Militello, L. G., & Hoffman, R. R. (2008). The forgotten history of cognitive task analysis.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 52, 383-
387.
Moody, K., Siegel, L., Scharbach, K., Cunningham, L., & Cantor, R. M. (2011). Pediatric
palliative care. Primary Care: Clinics in Office Practice, 38(2), 327-361.
Monterosso, L., & Kristjanson, L. J. (2008). Supportive and palliative care needs of families of
children who die from cancer: An australian study. Palliative Medicine, 22(1), 59-69.
doi:http://dx.doi.org/10.1177/0269216307084608
Mosenthal, A. C., Murphy, P. A., Barker, L. K., Lavery, R., Retano, A., & Livingston, D. H.
(2008). Changing the culture around end-of-life care in the trauma intensive care
unit. Journal of Trauma and Acute Care Surgery, 64(6), 1587-1593.
Murphy S., Xu, J.Q., and Kochanek, K.D. Deaths: Final data for 2010. National vital statistics
reports; vol 61 no 4. Hyattsville, MD: National Center for Health Statistics. 20
Nelson, J. E., Cortez, T. B., Curtis, J. R., Lustbader, D. R., Mosenthal, A. C., Mulkerin, C., &
Project, T. I. I. (2011). Integrating palliative care in the ICU: the nurse in a leading
COGNITIVE TASK ANALYSIS
103
role. Journal of hospice and palliative nursing: JHPN: the official journal of the Hospice
and Palliative Nurses Association, 13(2), 89.
Neugarten, B. L. (1974). Age groups in American society and the rise of the young-old. The
Annals of the American Academy of Political and Social Science, 415(1), 187-198.
O’Mahony, S., McHenry, J., Blank, A. E., Snow, D., Karakas, S. E., Santoro, G., & Kvetan, V.
(2009). Preliminary report of the integration of a palliative care team into an intensive
care unit. Palliative medicine.
Orszag, P. R., & Ellis, P. (2007). The challenge of rising health care costs-a view from the
Congressional Budget Office. New England Journal of Medicine, 357(18), 1793
Oshima Lee, E., & Emanuel, E. J. (2013). Shared decision making to improve care and reduce
costs. New England Journal of Medicine, 368(1), 6-8.
Paris, S.G., Lipson, M.Y., & Wixson (1983). Becoming a strategic reader. Contemporary
Educational Psychology, 8(3), 293-316.
Population Division, DSEA, United Nations, World Population Ageing Executive Summary.
Retrieved from
http://www.un.org/en/development/desa/population/publications/pdf/ageing/WorldPopula
tionAgeingReport2013.pdf Accessed April 3, 2014
Reid, R. J., Fishman, P. A., Yu, O., Ross, T. R., Tufano, J. T., Soman, M. P., & Larson, E. B.
(2009). Patient-centered medical home demonstration: a prospective, quasi-experimental,
before and after evaluation. The American journal of managed care, 15(9), e71-87.
Reyes-Ortiz, C. A., Williams, C., & Westphal, C. (2014). Comparison of Early Versus Late
Palliative Care Consultation in End-of-life Care for the Hospitalized Frail Elderly
Patients. American Journal of Hospice and Palliative Medicine, 1049909114530183.
COGNITIVE TASK ANALYSIS
104
Rocque, G. B., & Cleary, J. F. (2013; 2012). Palliative care reduces morbidity and mortality in
cancer. Nature Reviews.Clinical Oncology, 10(2), 80-89. doi:10.1038/nrclinonc.2012.211
Salmon, P., & Young, B. (2011). Creativity in clinical communication: from communication
skills to skilled communication. Medical education, 45(3), 217-226.
Epstein, R. M., & Hundert, E. M. (2002). Defining and assessing professional
competence. Jama, 287(2), 226-235.
Schaafstal, A., Schraagen, J. M., & Marcel, v. B. (2000). Cognitive task analysis and innovation
of training: The case of structured troubleshooting. Human Factors, 42(1), 75. Retrieved
from http://search.proquest.com/docview/216446753?accountid=14749
Schraagen, J. M., Chipman, S. F., & Shalin, V. L. (2000). Cognitive Task Analysis. Mahwah,
NJ: Lawrence Erlbaum Associates.
Sihra, L., Harris, M., & O’Reardon, C. (2011). Using the Improving Palliative Care in the
Intensive Care Unit (IPAL-ICU) project to promote palliative care consultation. Journal
of pain and symptom management, 42(5), 672-675.
Smith, A. K., McCarthy, E. P., Paulk, E., Balboni, T. A., Maciejewski, P. K., Block, S. D., &
Prigerson, H. G. (2008). Racial and ethnic differences in advance care planning among
patients with cancer: impact of terminal illness acknowledgment, religiousness, and
treatment preferences. Journal of Clinical Oncology, 26(25), 4131-4137.
Smith, T. J., & Hillner, B. E. (2011). Bending the cost curve in cancer care.New England Journal
of Medicine, 364(21), 2060-2065.
Sullivan, M. E., Yates, K. A., Baker, C. J., & Clark, R. E. (2010). Cognitive task analysis and its
role in teaching technical skills. In Tsueda, S., Scott, D., & Jones, D. (Eds.), Textbook of
Simulation, Skills, and Team Training (pp.85-94). Woodbury, CT: Cine-Med.
COGNITIVE TASK ANALYSIS
105
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive
Science, 12(2), 257-285.
Taylor, K. (2009). Paternalism, participation and partnership-the evolution of patient
centeredness in the consultation. Patient Education and Counseling, 74(2), 150-155
Temel, J., Greer, J., Muzikansky, A., Gallagher, E., Dahlin, C., Blinderman, C., Jacobsen, J.,
Pirl, W., Billings, J., & Lynch, T. (2010). Early palliative care for patients with metastatic
non-small-cell lung cancer. New England Journal of Medicine, 363(8), 733-742.
Teno, J. M., Gozalo, P. L., Bynum, J. P., Leland, N. E., Miller, S. C., Morden, N. E., ... & Mor,
V. (2013). Change in end-of-life care for Medicare beneficiaries: site of death, place of
care, and health care transitions in 2000, 2005, and 2009.JAMA, 309(5), 470-477.
Tilden, V. P., & Thompson, S. (2009). Policy Issues in end-of-life care. Journal of Professional
Nursing, 25(6), 363-368
Tirapelle, L. A. (2010). The effect of cognitive task analysis based instruction on surgical skills
expertise and performance. Available from ProQuest Dissertations and Theses database.
(UMI No. 3403766)
Tofel-Grehl, C., & Feldon, D. F. (2013). Cognitive task analysis-based training: A meta-analysis
of studies. Journal of Cognitive Engineering and Decision Making, 7(3), 293-304.
Tolano-Leveque, M. (2010). Using cognitive task analysis to determine the percentage of
critical information that experts omit when describing a surgical procedure (Doctoral
dissertation). Available from ProQuest Dissertations and Theses database. (UMI No.
3418184)
COGNITIVE TASK ANALYSIS
106
Truog, R. D., Campbell, M. L., Curtis, J. R., Haas, C. E., Luce, J. M., Rubenfeld, G. D., &
Kaufman, D. C. (2008). Recommendations for end-of-life care in the intensive care unit:
a consensus statement by the American College of Critical Care Medicine. Critical care
medicine, 36(3), 953-963.
University of Toronto. (2004). Ian Anderson program in end of life care: Pediatric palliative
care. Halifax, NS: McConnell, Y., & Frager, G.
Van Aalst-Cohen, E., Riggs, R., & Byock, I. (2008). Palliative care in medical school curricula:
A survey of United States medical schools. Journal of Palliative Medicine, 11(9). 1200-
1202
van Merriënboer, J. J. G., Clark, R. E., & de Croock, M. B. M. (2002). Blueprints for complex
learning: The 4C/ID-model. Educational Technology, Research and Development, 50(2),
39-64. doi: 0.1007/BF02504993
Wei, J. & Salvendy, G. (2004). The cognitive task analysis methods for job and task design:
review and reappraisal. Behavior & Information Technology, 23(4), 273–299.
Weidner, N. J. (2007). Pediatric palliative care. Current oncology reports, 9(6), 437-439.
Wetle, T., Shield, R., Teno, J., Miller, S. C., & Welch, L. (2005). Family perspectives on end-of-
life care experiences in nursing homes. The Gerontologist, 45(5), 642-650.
Wheatley, T., & Wegner, D. M. (2001). Automaticity of action, psychology of. In N. J. Smelser
& P. B. Baltes (Eds.), International encyclopedia of the social and behavioral sciences
(pp. 991-993). Oxford, UK: Elsevier Science Ltd.
White, D. B., Braddock, C. H., Bereknyei, S., & Curtis, J. R. (2007). Toward shared decision
making at the end-of-life in intensive care units: opportunities for improvement. Archives
of Internal Medicine, 167(5), 461-467.
COGNITIVE TASK ANALYSIS
107
Winker, M. & De Angelis, C. (2010). Caring for the aging populations: Call for papers. JAMA,
303(5), 455-456.
Yates, K. A. (2007). Towards a taxonomy of cognitive task analysis methods: A search for
cognition and task analysis interactions. (Doctoral dissertation). Retrieved from
http://search.proquest.com/docview/304828231?accountid=14749
Yates, K. A., & Feldon, D. F. (2011). Advancing the practice of cognitive task analysis: A call
for taxonomic research. Theoretical Issues in Ergonomics Science, 12(6), 472-495.
Yong, W. C., bin Main, N., Tan, L., Low, J., & Quah, S. C. (2013). Palliative Care for the
Elderly with Colorectal Cancer. In Colorectal Cancer in the Elderly (pp. 183-199).
Springer Berlin Heidelberg.
Yeung, A. S., Jin, P., & Sweller, J. (1998). Cognitive load and learner expertise: Split-attention
and redundancy effects in reading with explanatory notes. Contemporary educational
psychology, 23(1), 1-21.
Zepeda-McZeal, D. (2014). Using cognitive task analysis to capture expert reading instruction
in informational text for students with mild to moderate learning disabilities
(Unpublished doctoral dissertation). University of Southern California, Los Angeles,
CA.
Zajac, P., Norris, M., & Keenum, A. (2014). Patient-centered medical home. Osteopathic Family
Physician, 6(1).
Zernikow, B., Michel, E., Craig, F., & Anderson, B. J. (2009). Pediatric Palliative
Care. Pediatric Drugs, 11(2), 129-151.
Ziv, A., Wolpe, P. R., Small, S. D., & Glick, S. (2003). Simulation-based medical education: an
ethical imperative. Academic Medicine, 78(8), 783-788.
COGNITIVE TASK ANALYSIS
108
Appendix A
Interview Protocol
Cognitive Task Analysis Interview Protocol
Begin the Interview: Meet the Subject Matter Expert (SME) and explain the purpose of the
interview. Ask the SME for permission to record the interview. Explain to the SME the
recording will be only used to ensure that you do not miss any of the information the SME
provides.
Name of task(s):
Shared decision-making in palliative care settings
Performance Objective:
Conduct shared decision-making with patients and/or family members for the comfort of the
patient.
Step 1:
Objective: Capture a complete list of outcomes for shared decision-making between a patient
and/or family member and a palliative care physician.
A. Ask the Subject Matter Expert (SME) to list patient and/or family member outcomes when
these tasks are complete. Ask them to make the list as complete as possible
B. How are the patient and or family members assessed on these outcomes?
Step 2:
Objective: Provide practice exercises that are authentic to the job environment in which the
tasks are performed
A. Ask the SME to list all the contexts in which these tasks are performed (hospital settings)
B. Ask the SME how the tasks would change for each hospital setting (ICU, oncology,
CICU, etc.)
C. Ask the SME how the task would change for each stakeholder (patient, family member,
surrogate decision maker)
Step 3:
Objective: Identify main steps or stages to accomplish the task
D. Ask SME the key steps or stages required to accomplish the task.
E. Ask SME to arrange the list of main steps in the order they are performed, or if there is
no order, from easiest to difficult.
COGNITIVE TASK ANALYSIS
109
Step 4:
Objective: Capture a list of “step by step” actions and decisions for each task
A. Ask the SME to list the sequence of actions and decisions necessary to complete the task
and/or solve the problem
Ask: “Please describe how you accomplish this task step-by-step, so a novice could
perform it.”
For each step the SME gives you, ask yourself, “Is there a decision being made by the
SME here?” If there is a possible decision, ask the SME.
If SME indicates that a decision must be made…
Ask: “Please describe the most common alternatives (up to a maximum of three) that
must be considered to make the decision and the criteria trainees should use to decide
between the alternatives”.
Step 5:
Objective: Identify prior knowledge and information required to perform the task.
A. Ask SME about the prerequisite knowledge and other information required to perform the
task.
1. Ask the SME about Cues and Conditions
Ask: “For this task, what must happen before someone starts the task? What prior task,
permission, order, or other initiating event must happen? Who decides?”
2. Ask the SME about New Concepts and Processes
Ask: “Are there any concepts or terms required of this task that may be new to the
novice?”
Concepts – terms mentioned by the SME that may be new to the novice
Ask for a definition and at least one example
Processes - How something works
If the novice is operating equipment, or working on a team that may or may not
be using equipment, ask the SME to “Please describe how the team and/or the
equipment work - in words that a novice will understand. Processes usually
consist of different phases and within each phase, there are different activities –
think of it as a flow chart” Where does this fit within a process?
Ask: “Must novices know this process to do the task?” “Will they have to use it
to change the task in unexpected ways?”
COGNITIVE TASK ANALYSIS
110
IF the answer is NO, do NOT collect information about the process.
3. Ask the SME about Equipment and Materials
Ask: “What equipment and materials are required to succeed at this task in
routine situations? Where are they located? How are they accessed?
4. Performance Standard
Ask: “How do we know the objective has been met? What are the criteria, such
as time, efficiency, quality indicators (if any)?”
5. Sensory experiences required for task
Ask: “Must novices see, hear, smell, feel, or taste something in order to learn any
part of the task? For example, are there any parts of this task they could not
perform unless they could see something (such patient/family member emotional
status)?”
Step 6:
Objective: Identify routine or simple problems and complex problems that can be solved by
using the procedure.
A. Ask the SME to describe at least one routine problem and 2-3 complex problems that
the novice should be able to solve if they can perform each of the tasks on the list you
just made.
Ask: “Of the task we just discussed, describe at least one routine problem that the novice
should be able to solve IF they learn to perform the task”.
Ask: “Of the task we just discussed, describe two-three routine problems that the novice
should be able to solve IF they learn to perform the task”.
COGNITIVE TASK ANALYSIS
111
Appendix B
Inter-Rater Reliability Code Sheet for SME A
COGNITIVE TASK ANALYSIS
112
Appendix C
Job Aid for Developing a Gold Standard Protocol
Richard Clark and Kenneth Yates (2010, Proprietary)
The goals of this task are to 1) aggregate CTA protocols from multiple experts to create a “gold
standard protocol” and 2) create a “best sequence” for each of the tasks and steps you have
collected and the best description of each step for the design of training.
Trigger: After having completed interviews with all experts and capturing all goals, settings,
triggers, and all action and decision steps from each expert – and after all experts have edited
their own protocol.
Create a gold standard protocol
STEPS Actions and Decisions
1. For each CTA protocol you are aggregating, ensure that the transcript line number is
present for each action and decision step.
a. If the number is not present, add it before going to Step 2.
2. Compare all the SME’s corrected CTA protocols side-by-side and select one protocol
(marked as P1) that meets all the following criteria:
a. The protocol represents the most complete list of action and decision steps.
b. The action and decision steps are written clearly and succinctly.
c. The action and decision steps are the most accurate language and terminology.
3. Rank and mark the remaining CTA protocols as P2, P3, and so forth, according to the
same criteria.
4. Starting with the first step, compare the action and decision steps of P2 with P1 and
revise P1 as follows:
a. IF the step in P2 has the same meaning as the step in P1, THEN add “(P2)” at the
end of the step.
b. IF the step in P2 is a more accurate or complete statement of the step in P1,
THEN revise the step in P1 and add “(P1, P2)” at the end of the step.
c. IF the step in P2 is missing from P1, THEN review the list of steps by adding the
step to P1 and add “(P2N)”* at the end of the step.
5. Repeat Step 4 by comparing P3 with P1, and so forth for each protocol.
6. Repeat Steps 4 and 5 for the remaining components of the CTA report such as triggers,
main procedures, equipment, standards, and concepts to create a “preliminary gold
standard protocol” (PGSP).
7. Verify the PGSP by either:
a. Asking a senior SME, who has not been interviewed for a CTA, to review the
PGSP and note any additions, deletions, revisions, and comments.
b. Asking each participating SME to review the PGSP, and either by hand or using
MS Word Track Changes, note any additions, deletions, revisions, or comments.
i. IF there is disagreement among the SMEs, THEN either
COGNITIVE TASK ANALYSIS
113
1. Attempt to resolve the differences by communicating with the
SMEs, OR
2. Ask a senior SME, who has not been interviewed for a CTA, to
review and resolve the differences.
8. Incorporate the final revisions in the previous Step to create the “gold standard protocol”
(GSP).
COGNITIVE TASK ANALYSIS
114
Appendix D
SME A Individual Protocol Flowchart
Begin
Procedure
1:
Start
structured
notecard
to
record
medical
information
Review
current
and
past
medical
history
Does
the
consultant
have
differing
opinions/
recommendations?
End
Procedure
1:
Structured
Notecard
Contact
consultant
to
obtain
more
information
YES
Review
consultant
notes
Identify
signs
of
tension
(physical,
spiritual,
psychosocial,
personal)
No
COGNITIVE TASK ANALYSIS
115
Determine
meeting
attendees
Begin
Procedure
2:
Consult
with
Palliative
Care
Team
to
determine
meeting
time
and
place
Invite
nurse
to
meeting
YES
NO
End
Procedure
2:
Is
there
a
primary
nurse
that
will
be
carrying
out
orders?
Is
there
a
social
worker
assigned
to
the
patient.
Tell
social
worker
to
set
up
meeting
NO
YES
COGNITIVE TASK ANALYSIS
116
Gather
in
in
predetermined
location
Open
with
an
explanation
of
the
purpose
of
the
meeting
Follow
adapted
SPIKE
model
of
care
S-‐Setting,
P-‐Perception,
I-‐
Invitation,
K-‐Knowledge,
E-‐
Empathy
Setting:
Meet
in
a
quiet
and
controlled
environment
Begin
Procedure
3:
Conduct
the
meeting
Perception:
Ask
the
patient/SDM
what
and
how
much
they
know
about
the
disease
and
prognosis
COGNITIVE TASK ANALYSIS
117
Ask
the
patient/SDM
if
they
want
a
detailed
information
or
general
information
Provide
patient/SDM
with
detailed
information
Details
General
Provide
patient/SDM
with
general
information
Knowledge:
Provide
patient/SDM
with
treatment
choices,
including
risks
and
benefits
Give
space
&
acknowledge
grief
YES
Invitation:
Ask
the
patient/SDM
to
identify
the
level
of
information
they
want
Is
the
patient/SDM
having
difficulty
processing
the
knowledge?
NO
COGNITIVE TASK ANALYSIS
118
Discuss
patient
goals
Summarize:
Provide
patient/SDM
with
simple,
itemized
plan
to
move
forward
Empathize:
Make
a
special
effort
to
ask
the
patient/family
about
their
emotional
well
being,
fears,
and
concerns
End
Procedure
3:
COGNITIVE TASK ANALYSIS
119
Appendix E
Gold Standard Protocol
Task Title: Conduct Shared Decision Making for Palliative Care
Engaging in shared decision-making with a patient/Surrogate Decision Maker (SDM) to identify
and mutually agree upon the next step(s) for a patient’s care considering the patient’s function,
goals, prognosis, risks & benefits of treatments, and personal belief system.
Main Procedures
6. Start a structured notecard with all pertinent information
7. Conduct a preliminary meeting with patient/Surrogate Decision Maker (SDM)
8. Consult with palliative care team to determine meeting time, place, and attendees
9. Conduct the meet with all stakeholders
10. Document important aspects of meeting
Procedure
1. Start a structured notecard to record all pertinent medical information
1.1. Call medical team making the referral
1.1.1. Ask the primary doctor, nurses, specialists currently caring for the patient to
identify all relevant medical, psychosocial, spiritual and functional information
1.1.1.1. Identify physical conflicts/stressors
1.1.1.2. Identify spiritual conflicts/stressors (beliefs that may hinder treatment)
1.1.1.3. Identify personal conflicts/stressors (values or emotions that may conflict
with treatment)
1.1.1.4. Identify psychosocial conflicts/stressors (patient support systems, family
dysfunction, conflicts within family structure-bio or social)
1.1.2. Ask the primary medical team what the patient/family members have/have not
been told
1.1.2.1. Ask primary physician if the possibility of death has been mentioned to
the patient
1.1.2.2. IF the possibility of death has been mentioned to the patient, THEN plan
the mention the possibility of death during the meeting in Procedure 4
1.1.2.3. IF the possibility of death has not been mentioned to the patient, THEN do
not mention death during the meeting in Procedure 4
1.2. Review if patient has an Advance Directive
1.2.1. IF there is an Advance Directive, THEN obtain a copy and review the directives
1.2.1.1. IF a patient has an Advanced Directive, THEN the person on the
Advanced Directive is the SDM
1.2.1.2. IF a patient does not have an Advance Directive, AND has decisional
capacity, THEN during the preliminary meeting (Procedure 2) ask the patient
who they elect to be the SDM
COGNITIVE TASK ANALYSIS
120
1.2.1.3. IF a patient does not have an Advance Directive, AND does not have
decisional capacity, AND has caregivers, THEN ask the caregivers who will
act as SDM
1.2.1.4. IF a patient does not have an Advance Directive, AND does not have
decisional capacity, AND does not have caregivers, THEN a person on
medical team identifies the person they believe would best represent the
patient’s goals and values as the SDM
1.2.1.4.1. IF the medical team identifies a SDM, and it becomes contentious,
THEN enlist the help of a bio-ethicist
1.3. Review of medical record/medical history including all of the notes from care teams
(primary physician, residents, fellows, specialists, consultants, primary nurses) related to
the patient’s diagnosis and prognosis
1.3.1. IF consultants have differing opinions/recommendations THEN follow up with
personal conversations to obtain more information
1.4. Review how well chronic conditions are controlled
1.5. Review co-morbid conditions
1.6. Identify active issues and what brought patient into hospital for current stay and the
patient’s most current course of treatment
1.7. Review most recent lab work and imaging
1.8. Review current medications
1.9. Review past medication history
2. Conduct preliminary meeting with patient/family members
2.1. Use UARE understand, appreciate, reason, appreciate choices, express choices- to assess
the decisional capacity of the patient
2.2. Inquire about the patient’s symptoms
2.3. Inquire about the patient’s family and living circumstances to identify the level of
support for the patient
2.3.1. IF the patient needs more support, THEN set up a meeting with other family
members or support personnel and move to Procedure 3 and 4
2.4. Query patient and family about how this has affected them emotionally and
acknowledge the difficulty of the situation
2.5. Ask the patient/SDM what and how much they know about the disease/condition and
prognosis
2.5.1. Use the information gathered during the discussion (content, tone, level of
vocabulary) to evaluate the level of knowledge/understanding of the stakeholders
and present further information accordingly
2.5.1.1. IF the patient/SDM uses basic terminology/vocabulary and a low level of
medical knowledge, THEN use basic terminology/vocabulary and low levels
of medical information during the remainder of the meeting and during
Procedure 4
2.5.1.2. IF the patient/SDM uses high levels terminology/vocabulary and
demonstrates a high level of medical knowledge, THEN use the same level of
terminology/vocabulary and medical information during the remainder of the
meeting and during Procedure 4
2.5.2. IF the patient has an understanding of their illness and prognosis, THEN continue
with the meeting and go to Procedure 3 and 4
COGNITIVE TASK ANALYSIS
121
2.5.3. IF the patient does not have an understanding of their illness and prognosis,
THEN identify the defining barriers and address them in Procedures 3 and 4
2.6. Restate to the patient their understanding of their illness and prognosis in simple, easy to
understand language
2.7. Ask the patient/SDM if they have thought about what will happen if there are not more
treatment options available
2.7.1. IF the patient has thought about no more treatment options, THEN engage in
shared decision-making about palliative care services
2.7.2. IF the patient has not thought about it, THEN discuss the prospect that not all
illnesses respond successfully to treatment, and discuss possible options
2.8. Ask the patient about their emotional well being, including their worries and fears
2.8.1. IF the patient has an immediate non-medical worry (the meal they ordered is not
going to be delivered correctly), THEN engage patients in a discussion at that level
2.8.2. IF the patient has a medical/functional worry or fear related to their illness and/or
treatment (wanting to know how much time they have left, risks and benefits of
chemotherapy), THEN address their worries and fears in simple, easy to understand
language
2.9. Conduct a spiritual screening by
2.9.1. Asking the patient what has given them strength
2.9.2. Asking the patient if they are willing to share any particular spiritual or religious
beliefs
2.9.2.1. IF the patient shares their religious/spiritual beliefs, THEN ask the patient
how what they are doing spiritually/religiously fits into their illness and
treatment
3. Consult with other palliative care team members to determine who will attend the meeting,
when, and where the meeting will take place
3.1. Determine who needs to attend the meeting
3.1.1. IF the patient has a primary physician, THEN invite the primary physician to the
meeting
3.1.2. IF the primary nurse is going to carry out the treatment, THEN invite the primary
nurse to the meeting
3.1.3. IF there is a social worker, THEN ask the social worker to set up the meeting
3.1.4. IF there is no social worker, THEN any of the team members can serve in that
role
3.2. IF the patient has their own psychologist and/or social worker, THEN invite their
psychologist/social worker to the meeting IF the decision-maker/SDM is unavailable,
THEN provide an appropriate mode of communication OR reschedule the meeting
3.3. IF the patient or SDM requests a non-decisional stakeholder to be present, and that
person is not able to be there in physically, THEN provide phone and/or computer access
3.4. IF a patient is hospitalized and can leave the hospital room, THEN plan to conduct the
meeting in a conference room
3.5. IF a patient is hospitalized in private room, THEN conduct the meeting in the room
3.6. IF a patient is hospitalized in a semi-private room and cannot leave the room, THEN ask
if the other patient can be out of the room
3.7. Establish expectations for how long the meeting will proceed and an estimated length of
time for the meeting
COGNITIVE TASK ANALYSIS
122
4. Conduct shared decision making meeting with all stakeholders
4.1. Gather all stakeholders in a pre-determined controlled location with the least amount of
distractions
4.2. Tell everyone with cell phones & pagers to put them on silent
4.2.1. IF a team member receives a call or page and needs to step out, THEN advise the
member to step out at a time that does not interrupt the meeting
4.3. Position all stakeholders at eye level to eliminate hierarchical environment
4.4. Open meeting with explanation of why the team is meeting
4.5. Proactively and consciously set an empathetic tone commensurate with the level of the
patient/SDMs level of need throughout the meeting
4.6. Discuss goals with the patient/SDM by
4.6.1. Asking the patient/SDM how the patient has lived their life
4.6.2. Asking the patient/SDM what is most important to the patient
4.6.3. Asking the patient/SDM what they are hoping for
4.6.4. Asking the patient/SDM questions in an attempt to determine what would be an
acceptable minimal quality of life
4.7. Ask patient/family to identify the level of information they want to receive about the
diagnosis, prognosis and care
4.7.1.1. IF the patient/family want to know only general information, and the “Big
Picture” THEN provide general, “Big Picture” information
4.7.1.2. IF the patient/family want details, THEN provide information with the
level of detail specified
4.7.1.3. IF the patient does not want to know information, and want their family
members to know, THEN ask the patient why they do not want to know the
information AND address the patient’s fears/reservations
4.7.1.3.1. IF the patient still does not want to know after the
fears/reservations are addressed, and wants their family to know, THEN
meet with the family members privately
4.7.1.4. IF the patient wants to know, and the family members do not want the
patient to know, THEN acknowledge their request, AND try to explore the
fears behind the request, AND explain the ethical duties of truth telling to the
patient AND work with family to come to acceptable compromise (as long as it
doesn’t violate ethical duty)
4.8. Provide patient/SDM with medical information about the medical condition, the
treatment options, and the risks/benefits for each option choice (in complicated cases,
may consider 3 options, one at each end and one in the middle)
4.8.1. IF a patient/SDM needs more time to process the knowledge/information, THEN
go back to Procedure 4.6 (goals)
4.8.2. IF the patient/SDM has an emotional and/or negative reaction to the information,
THEN pause, give space, and acknowledge their emotions and/or grief
4.8.3. IF a patient/SDM is unable to identify a reasonable option, THEN go back to
procedure 4.6 (goals) and use the patient’s goals in the context of the discussion
about options
4.8.4. IF there is silence for processing, THEN wait for questions from the patient/SDM
4.9. Ask the patient/SDM what they understand about the medical information shared to this
point (diagnosis, prognosis, and treatment options)
COGNITIVE TASK ANALYSIS
123
4.9.1. IF the patient/SDM has misconceptions or gaps in knowledge about the medical
information shared (diagnosis, prognosis, or treatment options), THEN correct the
misperceptions and fill in the missing information
4.10. Ask the patient/SDM to acknowledge the information shared during the meeting
4.10.1. IF a treatment option has been chosen, THEN the patient/SDM acknowledges the
treatment option they have chosen
4.10.2. IF a treatment option has not been chosen, THEN the patient/SDM acknowledges
the options and commits to making a decision and meeting with the care provider to
discuss the chosen option at an agreed upon meeting time
4.11. Provide the patient/SDM with a simple and itemized plan identifying the agreed
upon next steps
4.12. Tell patient that you will be back the following day to review the information
shared, and to bring necessary documentation requested by the patient
5. Document important aspects of the meeting, including: who attended the meeting, salient
points of the medical situation, options provided, goals of the patient, and chosen option for
treatment
COGNITIVE TASK ANALYSIS
124
Appendix F
Incremental Coding Spreadsheets
SME
Steps
Alignment
Type
Final
Gold
Standard
Protocol
Analysis
A
B
C
A
D
1,
2,
or
3
1.
Start
a
structured
notecard
to
record
all
pertinent
medical
information
A
1.1.
Call
medical
team
making
the
referral
0
0
1
1
0
1
A
1.1.1.
Ask
the
primary
doctor,
nurses,
specialists
currently
caring
for
the
patient
to
identify
all
relevant
medical,
psychosocial,
spiritual
and
functional
information
0
0
1
1
0
1
A
1.1.1.1.
Identify
physical
conflicts/stressors
1
0
0
1
0
1
A
1.1.1.2.
Identify
spiritual
conflicts/stressors
(beliefs
that
may
hinder
treatment)
1
0
0
1
0
1
A
1.1.1.3.
Identify
personal
conflicts/stressors
(values
or
emotions
that
may
conflict
with
treatment)
1
0
0
1
0
1
A
1.1.1.4.
Identify
psychosocial
conflicts/stressors
(patient
support
systems,
family
dysfunction,
conflicts
within
family
structure-‐bio
or
social)
1
0
0
1
0
1
A
1.1.2.
Ask
the
primary
medical
team
what
the
patient/family
members
have/have
not
been
told
0
0
1
1
0
1
A
1.1.2.1.
Ask
primary
physician
if
the
possibility
of
death
has
been
mentioned
to
the
patient
0
0
1
1
0
1
COGNITIVE TASK ANALYSIS
125
D
1.1.2.2.
IF
the
possibility
of
death
has
been
mentioned
to
the
patient,
THEN
mention
the
possibility
of
death
during
the
meeting
0
0
1
0
1
1
D
1.1.2.3.
IF
the
possibility
of
death
has
not
been
mentioned
to
the
patient,
THEN
do
not
mention
death
during
the
meeting
0
0
0
0
1
0
A
1.2.
Review
if
patient
has
an
Advanced
Directive
0
1
0
1
0
1
D
1.2.1.
IF
there
is
an
Advanced
Directive,
THEN
obtain
a
copy
and
review
the
directives
0
1
0
0
1
1
D
1.2.1.1.
IF
a
patient
has
an
Advanced
Directive,
THEN
the
person
on
the
Advanced
Directive
is
the
SDM
0
0
1
0
1
1
D
1.2.1.2.
IF
a
patient
does
not
have
an
Advanced
Directive,
AND
has
decisional
capacity,
THEN
during
the
preliminary
meeting
(procedure
2)
ask
the
patient
who
they
elect
to
be
the
SDM
0
0
0
0
1
0
D
1.2.1.3.
IF
a
patient
does
not
have
an
Advanced
Directive,
AND
does
not
have
decisional
capacity,
AND
has
caregivers,
THEN
ask
the
caregivers
who
will
act
as
SDM
0
0
0
0
1
0
D
1.2.1.4.
IF
a
patient
does
not
have
an
Advanced
Directive,
AND
does
not
have
decisional
capacity,
AND
does
not
have
caregivers,
THEN
a
person
on
medical
team
0
0
1
0
1
1
COGNITIVE TASK ANALYSIS
126
identifies
the
person
they
believe
would
best
represent
the
patient’s
goals
and
values
as
the
SDM
D
1.2.1.4.1.
IF
the
medical
team
identifies
a
SDM,
and
it
becomes
contentious,
THEN
enlist
the
help
of
a
bio-‐ethicist
0
0
1
0
1
1
A
1.3.
Review
of
medical
record/medical
history
including
all
of
the
notes
from
care
teams
(primary
physician,
residents,
fellows,
specialists,
consultants,
primary
nurses)
related
to
the
patient’s
diagnosis
and
prognosis
1
0
1
1
0
2
D
1.3.1.
IF
consultants
have
differing
opinions/recommendatio
ns
THEN
follow
up
with
personal
conversations
to
obtain
more
information
1
0
0
0
1
1
A
1.4.
Review
how
well
chronic
conditions
are
controlled
1
0
1
1
0
2
A
1.5.
Review
co-‐morbid
conditions
1
0
0
1
0
1
A
1.6.
Identify
active
issues
and
what
brought
patient
into
hospital
for
current
stay
and
the
patient’s
most
current
course
of
treatment
1
0
1
1
0
2
A
1.7.
Review
most
recent
lab
work
and
imaging
1
0
1
1
0
2
A
1.8.
Review
current
medications
1
0
1
1
0
2
A
1.9.
Review
past
medication
history
0
0
1
1
0
1
2.
Conduct
preliminary
meeting
with
patient/family
members
A
2.1.
Use
UARE
understand,
appreciate,
reason,
appreciate
0
1
0
1
0
1
COGNITIVE TASK ANALYSIS
127
choices,
express
choices-‐
to
assess
the
decisional
capacity
of
the
patient
A
2.2.
Inquire
about
the
patient’s
symptoms
0
0
1
1
0
1
A
2.3.
Inquire
about
the
patient’s
family
and
living
circumstances
to
identify
the
level
of
support
for
the
patient
0
0
1
1
0
1
D
2.3.1.
IF
the
patient
needs
more
support,
THEN
set
up
a
meeting
with
other
family
members
or
support
personnel
and
move
to
procedure
3
and
4
0
0
1
0
1
1
A
2.4.
Query
patient
and
family
about
how
this
has
affected
them
emotionally
and
acknowledge
the
difficulty
of
the
situation
1
0
0
1
0
1
A
2.5.
Ask
the
patient/SDM
what
and
how
much
they
know
about
the
disease/condition
and
prognosis
1
1
1
1
0
3
A
2.5.1.
Use
the
information
gathered
during
the
discussion
(content,
tone,
level
of
vocabulary)
to
evaluate
the
level
of
knowledge/understanding
of
the
stakeholders
and
present
further
information
accordingly
0
1
0
1
0
1
D
2.5.1.1.
IF
the
patient/SDM
uses
basic
terminology/vocabulary
and
a
low
level
of
medical
knowledge,
THEN
use
basic
terminology/vocabulary
and
low
levels
of
medical
information
during
the
remainder
of
the
meeting
0
0
1
0
1
1
COGNITIVE TASK ANALYSIS
128
and
during
procedure
4
D
2.5.1.2.
IF
the
patient/SDM
uses
high
levels
terminology/vocabulary
and
demonstrates
a
high
level
of
medical
knowledge,
THEN
use
the
same
level
of
terminology/vocabulary
and
medical
information
during
the
remainder
of
the
meeting
and
during
procedure
4
0
0
1
0
1
1
D
2.5.2.
IF
the
patient
has
an
understanding
of
their
illness
and
prognosis,
THEN
continue
with
the
meeting
and
go
to
procedure
3
and
4
0
0
1
0
1
1
D
2.5.3.
IF
the
patient
does
not
have
an
understanding
of
their
illness
and
prognosis,
THEN
identify
the
defining
barriers
and
address
them
in
procedures
3
and
4
0
0
1
0
1
1
A
2.6.
Restate
to
the
patient
their
understanding
of
their
illness
and
prognosis
in
simple,
easy
to
understand
language
0
0
1
1
0
A
2.7.
Ask
the
patient/SDM
if
they
have
thought
about
what
will
happen
if
there
are
not
more
treatment
options
available
0
0
1
1
0
1
D
2.7.1.
IF
the
patient
has
thought
about
no
more
treatment
options,
THEN
engage
in
shared
decision-‐
making
about
palliative
care
services
0
0
1
0
1
1
COGNITIVE TASK ANALYSIS
129
D
2.7.2.
IF
the
patient
has
not
thought
about
it,
THEN
discuss
the
prospect
that
not
all
illnesses
respond
successfully
to
treatment,
and
discuss
possible
options
0
0
1
0
1
1
A
2.8.
Ask
the
patient
about
their
emotional
well
being,
including
their
worries
and
fears
1
1
1
1
0
3
D
2.8.1.
IF
the
patient
has
an
immediate
non-‐
medical
worry
(the
meal
they
ordered
is
not
going
to
be
delivered
correctly),
THEN
engage
patients
in
a
discussion
at
that
level
0
0
1
0
1
1
D
2.8.2.
IF
the
patient
has
a
medical/functional
worry
or
fear
related
to
their
illness
and/or
treatment
(wanting
to
know
how
much
time
they
have
left,
risks
and
benefits
of
chemotherapy),
THEN
address
their
worries
and
fears
in
simple,
easy
to
understand
language
0
0
1
0
1
1
A
2.9.
Conduct
a
spiritual
screening
by
0
0
1
1
0
1
A
2.9.1.
Asking
the
patient
what
has
given
them
strength
0
0
1
1
0
1
A
2.9.2.
Asking
the
patient
if
they
are
willing
to
share
any
particular
spiritual
or
religious
beliefs
0
0
1
1
0
1
D
2.9.2.1.
IF
the
patient
shares
their
religious/spiritual
beliefs,
THEN
ask
the
patient
how
what
they
are
doing
spiritually/religiously
fits
into
their
illness
and
treatment
0
0
1
0
1
1
COGNITIVE TASK ANALYSIS
130
3.
Consult
with
other
palliative
care
team
members
to
determine
who
will
attend
the
meeting,
when,
and
where
the
meeting
will
take
place
A
3.1.
Determine
who
needs
to
attend
the
meeting
1
1
0
1
0
2
D
3.1.1.
IF
the
patient
has
a
primary
physician,
THEN
invite
the
primary
physician
to
the
meeting
0
0
0
0
1
0
D
3.1.2.
IF
the
primary
nurse
is
going
to
carry
out
the
treatment,
THEN
invite
the
primary
nurse
to
the
meeting
1
0
1
0
1
2
D
3.1.3.
IF
there
is
a
social
worker,
THEN
ask
the
social
worker
to
set
up
the
meeting
1
0
0
0
1
1
D
3.1.4.
IF
there
is
no
social
worker,
THEN
any
of
the
team
members
can
serve
in
that
role
1
0
0
0
1
1
D
3.2.
IF
the
patient
has
their
own
psychologist
and/or
social
worker,
THEN
invite
their
psychologist/social
worker
to
the
meeting
DIF
the
decision-‐maker/SDM
is
unavailable,
THEN
provide
an
appropriate
mode
of
communication
OR
reschedule
the
meeting
0
0
0
0
1
0
D
3.3.
IF
the
patient
or
SDM
requests
a
non-‐decisional
stakeholder
to
be
present,
and
that
person
is
not
able
to
be
there
in
physically,
THEN
provide
phone
and/or
computer
access
1
0
0
0
1
1
D
3.4.
IF
a
patient
is
hospitalized
and
can
leave
the
hospital
room,
THEN
plan
to
conduct
the
meeting
in
a
conference
room
1
0
0
0
1
1
COGNITIVE TASK ANALYSIS
131
D
3.5.
IF
a
patient
is
hospitalized
in
private
room,
THEN
conduct
the
meeting
in
the
room
1
0
0
0
1
1
D
3.6.
IF
a
patient
is
hospitalized
in
a
semi-‐
private
room
and
cannot
leave
the
room,
THEN
ask
if
the
other
patient
can
be
out
of
the
room
1
0
0
0
1
1
A
3.7.
Establish
expectations
for
how
long
the
meeting
will
proceed
and
an
estimated
length
of
time
for
the
meeting
0
1
0
1
0
1
4.
Conduct
shared
decision
making
meeting
with
all
stakeholders
A
4.1.
Gather
all
stakeholders
in
a
pre-‐
determined
controlled
location
with
the
least
amount
of
distractions
1
1
0
1
0
2
A
4.2.
Tell
everyone
with
cell
phones
&
pagers
to
put
them
on
silent
1
1
0
1
0
2
D
4.2.1.
IF
a
team
member
receives
a
call
or
page
and
needs
to
step
out,
THEN
advise
the
member
to
step
out
at
a
time
that
does
not
interrupt
the
meeting
1
0
0
0
1
1
A
4.3.
Position
all
stakeholders
at
eye
level
to
eliminate
hierarchical
environment
1
1
0
1
0
2
A
4.4.
Open
meeting
with
explanation
of
why
the
team
is
meeting
1
0
0
1
0
1
A
4.5.
Proactively
and
consciously
set
an
empathetic
tone
commensurate
with
the
level
of
the
patient/SDMs
level
of
need
throughout
the
meeting
0
1
0
1
0
1
A
4.6.
Discuss
goals
with
the
patient/SDM
by
1
1
0
1
0
2
COGNITIVE TASK ANALYSIS
132
A
4.6.1.
Asking
the
patient/SDM
how
the
patient
has
lived
their
life
1
1
0
1
0
2
A
4.6.2.
Asking
the
patient/SDM
what
is
most
important
to
the
patient
0
1
0
1
0
1
A
4.6.3.
Asking
the
patient/SDM
what
they
are
hoping
for
0
1
1
1
0
2
A
4.6.4.
Asking
the
patient/SDM
questions
in
an
attempt
to
determine
what
would
be
an
acceptable
minimal
quality
of
life
0
1
0
1
0
1
A
4.7.
Ask
patient/family
to
identify
the
level
of
information
they
want
to
receive
about
the
diagnosis,
prognosis
and
care
1
1
0
1
0
2
D
4.7.1.1.
IF
the
patient/family
want
to
know
only
general
information,
and
the
“Big
Picture”
THEN
provide
general,
“Big
Picture”
information
1
1
0
0
1
2
D
4.7.1.2.
IF
the
patient/family
want
details,
THEN
provide
information
with
the
level
of
detail
specified
1
1
0
0
1
2
D
4.7.1.3.
IF
the
patient
does
not
want
to
know
information,
and
want
their
family
members
to
know,
THEN
ask
the
patient
why
they
do
not
want
to
know
the
information
AND
address
the
patient’s
fears/reservations
1
1
0
0
1
2
D
4.7.1.3.1.
IF
the
patient
still
does
not
want
to
know
after
the
fears/reservations
are
1
1
0
0
1
2
COGNITIVE TASK ANALYSIS
133
addressed,
and
wants
their
family
to
know,
THEN
meet
with
the
family
members
privately
D
4.7.1.4.
IF
the
patient
wants
to
know,
and
the
family
members
do
not
want
the
patient
to
know,
THEN
acknowledge
their
request,
AND
try
to
explore
the
fears
behind
the
request,
AND
explain
the
ethical
duties
of
truth
telling
to
the
patient
AND
work
with
family
to
come
to
acceptable
compromise
(as
long
as
it
doesn’t
violate
ethical
duty)
0
1
0
0
1
1
A
4.8.
Provide
patient/SDM
with
medical
information
about
the
medical
condition,
the
treatment
options,
and
the
risks/benefits
for
each
option
choice
(in
complicated
cases,
may
consider
3
options,
one
at
each
end
and
one
in
the
middle)
1
1
0
1
0
2
D
4.8.1.
IF
a
patient/SDM
needs
more
time
to
process
the
knowledge/information,
THEN
go
back
to
procedure
4.6
(goals)
0
1
0
0
1
1
D
4.8.2.
IF
the
patient/SDM
has
an
emotional
and/or
negative
reaction
to
the
information,
THEN
pause,
give
space,
and
acknowledge
their
emotions
and/or
grief
1
0
0
0
1
1
D
4.8.3.
IF
a
patient/SDM
is
unable
to
identify
a
reasonable
option,
THEN
go
back
to
procedure
4.6
0
1
0
0
1
1
COGNITIVE TASK ANALYSIS
134
(goals)
and
use
the
patient’s
goals
in
the
context
of
the
discussion
about
options
D
4.8.4.
IF
there
is
silence
for
processing,
THEN
wait
for
questions
from
the
patient/SDM
0
1
0
0
1
1
A
4.9.
Ask
the
patient/SDM
what
they
understand
about
the
medical
information
shared
to
this
point
(diagnosis,
prognosis,
and
treatment
options)
0
1
1
1
0
2
D
4.9.1.
IF
the
patient/SDM
has
misconceptions
or
gaps
in
knowledge
about
the
medical
information
shared
(diagnosis,
prognosis,
or
treatment
options),
THEN
correct
the
misperceptions
and
fill
in
the
missing
information
0
1
0
0
1
1
A
4.10.
Ask
the
patient/SDM
to
acknowledge
the
information
shared
during
the
meeting
0
1
0
1
0
1
D
4.10.1.
IF
a
treatment
option
has
been
chosen,
THEN
the
patient/SDM
acknowledges
the
treatment
option
they
have
chosen
0
1
0
0
1
1
D
4.10.2.
IF
a
treatment
option
has
not
been
chosen,
THEN
the
patient/SDM
acknowledges
the
options
and
commits
to
making
a
decision
and
meeting
with
the
care
provider
to
discuss
the
chosen
option
at
an
agreed
upon
meeting
time
0
1
0
0
1
1
COGNITIVE TASK ANALYSIS
135
A
4.11.
Provide
the
patient/SDM
with
a
simple
and
itemized
plan
identifying
the
agreed
upon
next
steps
1
0
0
1
0
1
A
4.12.
Tell
patient
that
you
will
be
back
the
following
day
to
review
the
information
shared,
and
to
bring
necessary
documentation
requested
by
the
patient
0
0
1
1
0
1
5.
Document
important
aspects
of
the
meeting,
A
5.
Document
important
aspects
of
the
meeting,
including:
who
attended
the
meeting,
salient
points
of
the
medical
situation,
options
provided,
goals
of
the
patient,
and
chosen
option
for
treatment
1
1
0
1
Total
Action
and
Decision
Steps
37
33
37
88
Action
Steps
23
21
22
47
Decision
Steps
14
12
15
47
41
41
Total
Action
and
Decision
Steps
42.05%
37.50
%
42.05%
Action
Steps
48.94%
44.68
%
46.81%
Decision
Steps
34.15%
29.27
%
36.59%
47
41
Action
and
Decision
Steps
Omitted
51
55
51
Action
Steps
Omitted
24
26
25
Decision
Steps
Omitted
27
29
26
Action
and
Decision
Steps
Omitted
57.95%
62.50
%
57.95%
Action
Steps
Omitted
51.06%
55.32
%
53.19%
COGNITIVE TASK ANALYSIS
136
Decision
Steps
Omitted
65.85%
70.73
%
63.41%
Average
Total
Action
and
Decision
Steps
Captured
Omitt
ed
Action
Steps
46.81%
53.19
%
Decision
Steps
33.33%
66.67
%
Highly
Aligned
2
2.30
%
Partially
Aligned
20
22.99
%
Slightly
Aligned
60
68.97
%
Not
Aligned
5
5.75
%
0
87
100.0
0%
Abstract (if available)
Abstract
The purpose of this study was to apply Cognitive Task Analysis (CTA) methods to capture the tacit and unconscious action and decision steps of palliative care physicians as they engage in shared decision‐making at the end of life. Additionally, the study sought to identify the number and percentage of critical action and decisions steps omitted by experts as they describe how they perform this complex cognitive task. CTA has been shown to be effective at acquiring the automated knowledge of experts as they perform cognitively complex tasks. Shared decision‐making has been shown to have a positive impact on patients and family member’s perceived quality of medical care and sense of contentment at the end of life. Three expert palliative care physicians were and interviewed to capture the action and decision steps they use when engaging with patients and their family members during shared decision‐making in end‐of‐life conversations. The interview data was aggregated into a preliminary gold standard protocol and reviewed by a fourth palliative care physician expert. Overall, the study found there were five main steps or procedures experts completed when engaging in shared decision‐making. The study’s findings indicate that palliative care physician experts omitted 53.19% of action steps and 66.67% of decision steps when compared to a final gold standard protocol, supporting the research on expert omissions. The expert knowledge and skills captured by this CTA may be used for further training in medical schools, residency and fellowship programs as they relate to palliative care, patient‐centered communication, and shared decision‐making.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
The use of cognitive task analysis for the postanesthesia patient care handoff in the intensive care unit
PDF
Using cognitive task analysis to capture how expert anesthesia providers conduct an intraoperative patient care handoff
PDF
Using incremental cognitive task analysis to capture expert instruction in expository writing for secondary students
PDF
Using cognitive task analysis to capture expert instruction in division of fractions
PDF
The use of cognitive task analysis to capture expert instruction in teaching mathematics
PDF
The use of cognitive task analysis to capture expert patient care handoff to the post anesthesia care unit
PDF
Using cognitive task analysis to capture how expert principals conduct informal classroom walk-throughs and provide feedback to teachers
PDF
Using cognitive task analysis to capture expert reading instruction in informational text for students with mild to moderate learning disabilities
PDF
The use of cognitive task analysis to determine surgical expert's awareness of critical decisions required for a surgical procedure
PDF
Towards a taxonomy of cognitive task analysis methods: a search for cognition and task analysis interactions
PDF
Using cognitive task analysis for capturing expert instruction of food safety training for novice employees
PDF
Using individual cognitive task analysis to capture expert writing instruction in expository writing for secondary students
PDF
Using cognitive task analysis to determine the percentage of critical information that experts omit when describing a surgical procedure
PDF
The use of cognitive task analysis for identifying the critical information omitted when experts describe surgical procedures
PDF
Cognitive task analysis for instruction in single-injection ultrasound-guided regional anesthesia
PDF
The effect of cognitive task analysis based instruction on surgical skills expertise and performance
PDF
The use of cognitive task analysis to capture exterptise for tracheal extubation training in anesthesiology
PDF
Employing cognitive task analysis supported instruction to increase medical student and surgical resident performance and self-efficacy
PDF
Eliciting perspectives on palliative care: outpatient visits, advance care planning, and the impact of COVID-19
PDF
Identifying the point of diminishing marginal utility for cognitive task analysis surgical subject matter expert interviews
Asset Metadata
Creator
Larson, Deidre Lee
(author)
Core Title
Using cognitive task analysis to capture palliative care physicians' expertise in in-patient shared decision making
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Education (Leadership)
Publication Date
03/23/2015
Defense Date
02/10/2015
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
cognitive task analysis,OAI-PMH Harvest,palliative care,shared decision making
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Yates, Kenneth A. (
committee chair
), Enguidanos, Susan M. (
committee member
), Wang, Susan (
committee member
)
Creator Email
dllarson@usc.edu,dllarson3.3@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c3-540169
Unique identifier
UC11298313
Identifier
etd-LarsonDeid-3238.pdf (filename),usctheses-c3-540169 (legacy record id)
Legacy Identifier
etd-LarsonDeid-3238.pdf
Dmrecord
540169
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Larson, Deidre Lee
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
cognitive task analysis
palliative care
shared decision making