Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Using individual cognitive task analysis to capture expert writing instruction in expository writing for secondary students
(USC Thesis Other)
Using individual cognitive task analysis to capture expert writing instruction in expository writing for secondary students
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Running head: COGNITIVE TASK ANALYSIS
USING INDIVIDUAL COGNITIVE TASK ANALYSIS TO CAPTURE EXPERT
WRITING INSTRUCTION IN EXPOSITORY WRITING FOR SECONDARY
STUDENTS
by
Milo Jury
________________________________________________________________________
A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2015
Copyright 2015 Milo Jury
COGNITIVE TASK ANALYSIS 2
Dedication
This dissertation is dedicated to my strongest supporters. I could not have done
this without the patience and strength of my family, my children, my parents, and
especially my wife, Annita. You have been beyond patient and supportive during this
journey. Your understanding, your sacrifice, your strength and your insistent reminders of
my academic commitments have resulted in the completion of this dissertation. This
work is as much yours as mine. Thank you from the bottom of my heart.
COGNITIVE TASK ANALYSIS 3
Acknowledgements
Acknowledging all those who helped in some way to support me in this academic
journey seems such an inadequate way to express how much you have meant to this
project. First, I thank the expert school teachers who spent hours struggling to describe
in detail the amazing things they do every day in the classroom to help students be
successful. You are an inspiration to me and to my dedication to our noble profession.
I am forever grateful for the support and direction of Dr. Kenneth Yates. Thank
you for your untiring efforts in moving us towards completion of our dissertations. You
were both patient and insistent as you guided us in our efforts to complete this demanding
task. Your willingness to meet with us at all hours, day or night, is an example of what all
good teachers should aspire to be. I also thank Dr. Paula Carbone and Dr. Anthony
Maddox for your support and your willingness to sit on our dissertation committee. Your
suggestions, to the betterment of our work, have been invaluable.
I cannot forget my research partner and my friend, Dr. Nicolas Lim. We
embarked on this journey many months ago excited for what we were about to begin. I
have had occasion to admire time and again your quest for excellence and your absolute
dedication and focus to your work. Your support inspired me during moments of doubt. I
especially thank your beautiful family for their sacrifice of husband and father as you
worked tirelessly to finish your work and to encourage me to finish mine. I also thank my
colleagues at USC as we worked together in earning our degrees. Dr. Megan McGuiness
and Dr. Thomas Weinmann, you were often the only reason I kept at this!
To my family and long-time friends, thank you. I look forward to being there for
you once again.
COGNITIVE TASK ANALYSIS 4
Table of Contents
List of Tables 7
List of Figures 8
List of Abbreviations 9
Abstract 10
Chapter One: Overview of the Study 11
Statement of the Problem 11
Cognitive Task Analysis 16
Purpose of Study 18
Methodology of the Study 19
Definition of Domain Terms 20
Definition of CTA Terms 22
Organization of the Study 24
Chapter Two: Review of the Literature 25
Expository Writing 25
The Importance of Expository Writing 25
Definition of Expository Writing 27
The Importance of Expository Writing at the Secondary Level 28
Common Core State Standards 28
High Stakes Testing 29
The Current State of Students’ Writing Proficiency 30
Students’ Underachievement in Writing 30
Teachers’ Influence on Student Achievement 31
Teachers Feel Underprepared to Teach Writing 33
Teacher Experiences as Students 33
Preservice Training in Writing Instruction 34
Realities in Teaching and Assessing Writing 35
Subject Matter Expertise 38
The Need for Expertise in Writing Instruction 38
Summary 39
Using Subject Matter Experts to Train Non-experts 40
Knowledge Types 42
Declarative Knowledge 44
Procedural and Conditional Knowledge 45
Automaticity 46
Expertise 48
Characteristics of Experts 48
Building Expertise 50
Consequences of Expertise 52
Expert Omissions 53
Cognitive Task Analysis 54
Definition of CTA 54
CTA History 55
Cognitive Task Analysis Methodology 56
Taxonomies of Knowledge Elicitation Techniques 57
COGNITIVE TASK ANALYSIS 5
Pairing Knowledge Elicitation with Knowledge
Representation/Analysis 58
Effectiveness of CTA 59
Efficiency of CTA 60
Benefits of CTA for Instruction 61
Comparisons of 3i+3r Individual and 1i+3r
Incremental CTA Methods 62
Summary 63
Chapter Three: Methodology 64
The Current Study 64
Participants 65
Data Collection for Question 1 68
Procedure for Data Collection 68
Phase 1: Collect Preliminary Knowledge 69
Phase 2: Identify Knowledge Types 69
Phase 3: Identify Knowledge Elicitation Techniques 69
Instrumentation 69
Interviews 73
Phase 4: Data Analysis 73
Coding 74
Inter-rater Reliability (IRR) 74
SME Protocol and Verification 75
Phase 5: Formatting the Results 74
Gold Standard Protocol (GSP) 75
Data Collection for Question 2 75
Spreadsheet Analysis 75
Data Collection for Question 3 75
Chapter Four: Results 77
Overview of the Results 77
Research Questions 77
Question 1 77
Inter-rater reliability (IRR) 77
Flowchart Analysis 77
Gold Standard Protocol (GSP) 78
Recalled Action and Decision Steps 81
Action and Decision Steps Contributed by Each SME 82
Action and Decision Steps Contributed
Round Two Interviews 84
Alignment of SMEs in describing the same
Action and Decision Steps 84
Question 2 87
Total Knowledge Omissions 87
Question 3 90
Total Time 90
Total Cost 91
Knowledge Elicitation 92
COGNITIVE TASK ANALYSIS 6
Expert Contribution of Action and Decision Steps 93
3i+3r Incremental Method Analysis 95
1i+3r Individual Method Analysis 95
Summary 96
Chapter Five: Discussion 97
Overview of Study 97
Process of Conducting Cognitive Task Analysis 98
Selection of Experts 98
Collection of Data 101
Importance of Context 102
Cognitive Load Theory 104
Automaticity and Conscious Recall 106
Discussion of Findings 106
Question 1 106
Action Steps versus Decision Steps 106
Number of Action and Decision Steps 107
Teacher choice in pedagogical strategy use 109
Question 2 110
Question 3 111
Efficiency 111
Criteria to Measure Effectiveness 112
Task Complexity 114
Limitations 116
Confirmation Bias 116
Internal Validity 117
External Validity 117
Implications 118
Future Research 119
Conclusion 121
References 123
Appendix A: Cognitive Task Analysis Interview Protocol 138
Appendix B: Inter-rater Reliability Code Sheet for SME A 141
Appendix C: Job Aid for Developing a Gold Standard Protocol 142
Appendix D: SME A Protocol Flowchart 144
Appendix E: Gold Standard Protocol: Teaching Expository Writing to
11
th
Grade Students 161
Appendix F: 3i+3r Individual Coding Spreadsheet 188
COGNITIVE TASK ANALYSIS 7
List of Tables
Table 1: SMEs interviewed for both the Current Study and
the Concurrent Study (Lim, 2015) 67
Table 2: Example of Process toward creating main Procedures
in Preliminary Gold Standard Protocol (PGSP) 79
Table 3: Cumulative Action and Decision Steps Captured
for Each SME using the Individual (3i+3r) CTA method 82
Table 4: Cumulative Action and Decision Steps Captured
for Each SME in Round 2 Interviews 84
Table 5: Number and Percentage of Action and Decision
Steps that are Completely Aligned, Highly Aligned,
Partially Aligned, and Slightly Aligned 86
Table 6: Total Action and Decision Steps, or Expert
Knowledge, Omissions by SME when Compared to
the Gold Standard Protocol 88
Table 7; Comparison of Total Time spent doing the
1i+3r Incremental Method and the 3i+3r
Individual Method of CTA 90
Table 8: Comparison of Total Cost doing the 1i+3r
Incremental Method and the 3i+3r Individual
Method of CTA 91
Table 9: Comparison of Overall Action and Decision
Steps from 3i+3r Independent Method and 1i+3r 92
COGNITIVE TASK ANALYSIS 8
List of Figures
Figure 1: 3i+3r Individual CTA Method 70
Figure 2: Number of Decision Steps, Action Steps, and
Action and Decision Steps for SMEs captured through
3i+3r Individual CTA 83
Figure 3: Action and Decision Steps that are Completely Aligned,
Highly Aligned, Partially Aligned, and No Alignment 87
Figure 4: Total SME Knowledge Omissions When
Compared to the Gold Standard Protocol. 89
Figure 5: Total Expert Knowledge Recall for the 1i+3r
Incremental Method and 3i+3r Individual Method 93
COGNITIVE TASK ANALYSIS 9
List of Abbreviations
1i 1i+3r (1 Independent Interview + 3 Reviews)
3i 3i+3r (3 Independent Interviews + 3 Reviews)
BTA Behavioral Task Analysis
CDE California Department of Education
CCSS Common Core State Standards
CTA Cognitive Task Analysis
CDM Critical Decision Method
CPP Concepts, Processes, and Principles
EAP Early Assessment Program
ELA English Language Arts
ELL English Language Learner
ERWC Expository Reading and Writing Course
GSP Gold Standard Protocol
IRR Inter-rater Reliability
NAEP National Assessment of Educational Progress
NCES National Center for Educational Statistics
NCLB No Child Left Behind
PARI Precursor, Action, Result, and Interpretation
PD Professional Development
PGSP Preliminary Gold Standard Protocol
SME Subject Matter Expert
COGNITIVE TASK ANALYSIS 10
Abstract
Cognitive Task Analysis (CTA) is a collection of methods used to elicit the cognitive
processes, unobserved knowledge, and goal structures that make up human behavior.
This study sought to apply CTA methods to elicit the knowledge and skills expert English
teachers use as they teach expository writing to eleventh grade students. Three semi-
structured CTA interviews were held to capture the procedural and declarative
knowledge represented as action and decision steps. The results were coded, analyzed,
and aggregated into a gold standard protocol (GSP) that was then given to a fourth expert
for verification. This study also looked to identify and quantify the percentage and
number of knowledge and skill omissions as experts recall how they deliver expository
writing instruction. The omission data was recorded in a spreadsheet and a frequency
count was used to determine the amount of omitted knowledge and skills by each expert.
These results confirmed prior research that suggests experts may omit up to 70% of
critical information and can be reversed by utilizing 3-4 experts in eliciting expert
knowledge and skills. Finally, this study and a concurrent study (Lim, 2015) compared
the efficiency of two varying methods of CTA, the 3i+3r individual method and the 1i+3r
incremental method (Lim, 2015). These studies operationalized efficiency by determining
which method captures as much, or more, action and decision steps from experts for less
cost and time. The comparison results produced abundant data, but did not provide a clear
answer as to which method is more efficient. The knowledge and skills captured by CTA
may be used to inform and develop pre-service and in-service professional development
training for teachers in performing the task of expository writing instruction.
COGNITIVE TASK ANALYSIS 11
CHAPTER ONE: OVERVIEW OF THE STUDY
Statement of the Problem
The definition of expository writing encompasses various modes or genres of
writing, including description, cause and effect, comparison, sequencing, and problem-
solution organizational patterns (Baker, Brizee, & Angeli, 2013). These patterns of
writing are an important focus in curriculum development and instructional delivery
because of their use in high school, college and the workplace (Kiuhara, Graham, &
Hawken, 2009). While each genre noted above has value as a form of written
communication, this study will limit its investigation to two main purposes of expository
writing: to inform (Boscolo, 1990) and to present an argument with evidence and reasons
to support that argument (Beck, Llosa, & Fredrick, 2013; Chandrasegaran, 2013).
Of all the skills and knowledge gained in school, few are as important or needful
for students to master as the ability to communicate using the written word (MacArthur &
Philippakos, 2010; Taylor & Beach, 1984; Alber-Morgan, Kessler, & Konrad, 2007). The
ability to write as taught in middle school and high school, especially that needed to
master expository writing, is seen as an evolutionary next step in student acquisition of
writing skills (Beck, Llosa, & Fredrick, 2013). The complexity of expository writing is
found in the use of more formal language structure to explain abstract ideas and
consequently presents to students a much more difficult skill to acquire (Graham &
Harris, 2013). But proficient acquisition of expository writing skills by students helps
ensure academic success in college (Addison & McGee, 2010, Graham and Perin, 2007b)
and professional success for workers in the workplace (NAGB, 2010). In the recent past,
state academic standards and assessment programs often placed less emphasis on
COGNITIVE TASK ANALYSIS 12
assessing writing than on reading skills and math skills (ACT, 2012). However, the recent
adoption by many states of Common Core State Standards (CCSS; Council of Chief State
School Officers & National Governors Association, 2010) has increased the importance
of expository writing instruction enough for states to include writing assessments along
with reading and math assessments to gauge student academic performance (Colman,
Pimentel, and Zimba, 2012, Graham & Harris, 2013). With the implementation of the
Common Core State Standards (CCSS; Common Core State Standards Initiative, 2010),
there will be a greater demand that students learn the knowledge, skills, and procedures
to become more effective writers, especially those needed to write expository pieces. This
places a renewed expectation on educators to provide rigorous expository writing
instruction in the high school and middle school classroom.
Expository writing continues to be a foundational skill for student success in
college and the university. Standardized college entrance exams such as Scholastic
Aptitude Test (SAT), the American College Testing exam (ACT), and Early Admissions
Program exam (EAP) administered by the California State University system have
subsections in which prospective students respond to a writing prompt as an assessment
of proficiency in expository writing (Beck, Llosa, & Fredrick, 2013; Read & Landon-
Hays, 2013). Results from these exams have shown that large numbers of high school
students are not proficient in their expository writing skills (Applebee & Langer, 2009),
which indicates a growing number of students who appear to be unprepared for the rigor
of college-level courses (Beck, Llosa, & Fredrick, 2013). There is great need to improve
writing instruction in classrooms across the country. And yet, there are students who do
pass these writing assessments with proficiency and bring these skills to college as
COGNITIVE TASK ANALYSIS 13
freshmen ready to enter higher education (Fanetti, Bushrow, & DeWeese, 2010).
Education researchers (Darling-Hammond, 2000; Darling-Hammond & Baratz-Snowden,
2007) have found that highly qualified teachers who complete a teacher preparation
program are more effective in helping increase student achievement. Effective
preparation in teaching expository writing should help develop expertise among
beginning teachers. Experts in subject matter domains have deep understanding of the
curriculum along with knowledge of a variety of pedagogical strategies to help students
learn. These experts have the experience and knowledge to make decisions about what to
teach and when to teach it to their students (Darling-Hammond & Baratz-Snowden,
2007). If novice teachers could access the knowledge and skills of their experienced
colleagues, schools will begin to make gains in addressing the need for students to be
prepared for the rigors of college writing.
One of the many things writing teachers are tasked with in the classroom is
ensuring students gain proficient narrative and expository writing skills throughout their
time in school. Schools are more successful at teaching students factual writing skills
often found in creative writing or narrative writing because of its close relationship with
familiar informal speech and writing patterns of early elementary students (Berman &
Nir-Sagiv, 2007). Expository writing does not share these traits with narrative writing.
Because of the formality and structure of expository writing instruction (Chandrasegaran
2013), students are less able to use informal language and personal experience when
writing expository texts. Students need support in using more formal language and
specific structures of logical reasoning in their writing. Novice teachers must develop the
knowledge and skills needed to provide this type of instruction. Expert teachers might
COGNITIVE TASK ANALYSIS 14
have the experience and skills to develop a rigorous lesson plan to meet students’ needs
(Smith, 2005), but novice teachers do not have immediate access to these same skills and
knowledge used by experts in the classroom.
This leads to the result that novice teachers are often unsure how to teach
expository writing. In fact, the teaching of writing tends to cause the most anxiety among
teachers of all levels of expertise (Grisham, 2011; Read & Landon-Hays, 2013). One
common method of instruction is teaching students organizational patterns (Baker, Brizee,
& Angeli, 2013). High School teachers are tasked with helping students perform well
enough on writing assessments that lead to passing college entrance exams. Teaching
organizational patterns in the form of step-by-step formulas or templates (Fanetti,
Bushrow, & DeWeese, 2010) are used by teachers to help students structure and write
essays. It is thought that by providing a structured organizer for students to fill in with
their ideas, teachers help to reduce the cognitive load students experience when writing.
The thought is that students will be able to address the subject matter of the writing
prompt without having to worry about structure and organization. The goal here is to help
students garner the highest possible score on standardized writing assessments.
The use of writing structures and formulas appear at first to be intuitive.
McCutchen (2011) details student success at narrative (story) writing because students
are familiar with the structure of this type of writing. Having a structure upon which to
build a piece of writing allows students to concentrate on the ideas, words, and grammar
needed to write coherent pieces. This appears on its face to be the case with expository
writing patterns as well. However, expository writing is not always so clear cut. Writers
use different expository writing structures for different purposes and audiences which
COGNITIVE TASK ANALYSIS 15
causes confusion among students and teachers as to which genres are most applicable and
important to learn (Graham & Harris, 2013; Grossman, Loeb, Cohen, Hammerness,
Wyckoff, Boyd, & Lankford, 2000).
A common structural component of expository writing is the 5-paragraph essay,
with each paragraph beginning with an introductory sentence and ending with a
concluding sentence. In real life, this is not always the case. Expert teachers are aware of
the real-life needs of their student writers. They know the knowledge and skills students
require and know when and how to apply this knowledge. Again, novice teachers would
greatly benefit from this expertise.
Expert and novice teachers alike are often under great amounts of stress to teach
all that is required of them by state standards along with additional and changing state
curricular requirements, district curricular requirements, and various other teaching duties
(Read and Landon-Hays, 2013). It seems impractical to expect teachers to complete the
myriad duties of an educator while asking them to digest new standards and create
lessons to address them all the while as states revamp student assessment tools in
response to Common Core Standards (CCSS). Again, experienced teachers able to access
years of experience along with the expertise brought by time in the classroom make this
change more effectively. Novice teachers are as willing but are less able to do so. Thus,
they often are unable to respond as they would like to the needs of their students (Kiuhara,
Graham, and Hawken, 2007). Accessing the expertise of veteran teachers would go far in
helping novice teachers address this problem.
Students who exhibit proficiency in expository writing benefit greatly from
teachers who are subject matter experts (Read & Landon-Hays, 2013; Grossman et al.
COGNITIVE TASK ANALYSIS 16
2010). It is this teacher expertise that results in increased academic success for students
(Darling-Hammond, 2000; Darling-Hammond & Baratz-Snowden, 2007). Capturing this
expert knowledge and skill to help novice teachers acquire their own expertise more
quickly would help students improve academically (Saphir, 2011). Cognitive Task
Analysis (CTA) is one such tool that can capture the expertise of Subject Matter Experts
(SMEs).
Cognitive Task Analysis
Cognitive Task Analysis (CTA) is an interview, observation, and analysis method
whereby a trained interviewer conducts semi-structured interviews with experts on how
they perform complex mental tasks (Clark, Feldon, van Merriënboer, Yates, & Early,
2008). CTA is used to capture the performance objectives, equipment, conceptual
knowledge, procedural knowledge, and performance standards used by experts to
complete a complex task. A complex task is composed of controlled knowledge and
automated knowledge used to perform a task over a specific length of time (van
Merriënboer, Clark, & De Croock, 2002). The knowledge captured can then serve as a
record of the task performance and provide to novices a tool to help them achieve the
performance goals within any context (Clark, et al., 2008), Often this recording of
knowledge results in the creation of job aids, instructional design, and training programs
(Yates & Feldon, 2011).
Since the early 20
th
century, human performance analysts would perform a
Behavior Task Analysis (BTA) by watching workers and noting the physical and
observable actions done to complete a task. However, the nature of work completion has
evolved into increasingly complex tasks which make clear the need for analysis of not
COGNITIVE TASK ANALYSIS 17
only the observable actions of workers but also the unobservable cognitive processes and
structures involved (Clark & Estes, 1996). Research has found that there are three
knowledge types used by experts to complete their tasks. The first type, declarative or
conceptual knowledge, is the conscious knowledge easily recalled by experts. Declarative
knowledge refers to the conceptual understanding of principles and processes related to
the task (Clark, Pugh, Yates, Inaba, Green, & Sullivan, 2011). For experts, declarative or
conceptual knowledge are schema-based which allow for efficient problem analysis and
accurate recall. In short, declarative knowledge is how-it-works knowledge learned by
experts that help set them apart from novices (Hall, Gott, & Pokorny, 1995). Second,
procedural knowledge is the unconscious knowledge gained through experience and
practice. As experts perform a task, certain procedures become so routine and automatic
that the mental effort to complete them diminishes (Clark, et al., 2008). Experts thus
become unaware of the steps they believe they perform and, when asked, often give
inaccurate descriptions of the steps they complete (Clark & Estes, 2011). Finally, the
third knowledge type is conditional knowledge. A subset or specialized type of
procedural knowledge, conditional knowledge is the when-to-do-it knowledge that tells
an expert when to perform and certain action or to take an alternative path (Hall, et al.,
1995). While procedural knowledge defines the decisions an expert might want to
perform, conditional knowledge helps to decide when this action will proceed.
As the decisions and actions of experts become more automated, experts use less
cognitive load to perform these tasks. The decisions and actions have become easier
through practice and experience, and require less cognitive processing to complete. The
consequence of this, however, is that experts often report inaccurately the unconscious
COGNITIVE TASK ANALYSIS 18
actions and decision steps they make when performing these tasks (Clark, 2014).
Research has shown that experts may omit up to 70% of the critical information novices
need to perform a complex task or solve a difficult problem (Feldon, 2006). This is
because expert knowledge has become so automated they forget to state it or even
remember it when called upon to do so (Ericsson, Krampe, & Tesch-Römer, 1993). This
creates a gap for novice learners, which they cannot successfully fill alone (Clark et. al.
2008). CTA looks to fill these gaps and improve novice learning through the elicitation
of expert knowledge and skills.
Many varieties of CTA exist and differences appear even within similar methods
(Clark et al. 2008). In the current study, two different methods of CTA are being
compared to find which is more efficient.
Purpose of Study
Acquiring expert knowledge in expository writing instruction can help inform
teacher training programs by capturing the knowledge and skills of subject matter experts.
CTA has been found to be a successful method to elicit automated and unobservable
knowledge, decisions, and skills experts use to perform complex tasks (Clark, et al., 2008,
Zepeda-McZeal, 2014). As such, this study seeks to use CTA to capture the knowledge,
decisions, and skills of teachers who are experts in expository writing instruction at the
eleventh grade level. This study also seeks to determine how many action and decision
steps experts omit when describing how they teach expository writing. This study will
use the 3i+3r Individual CTA method (Flynn 2010; Zepeda-McZeal, 2014), to elicit this
information. The action and decision steps collected by this method will be conducted
concurrently with another study (Lim, 2015) to determine which method of CTA, the
COGNITIVE TASK ANALYSIS 19
individual (3i+3r) or the incremental (1i+3r), is more efficient at capturing the automated
knowledge of experts.
The questions this study will attempt to answer are:
1. What are the action and decision steps that expert teachers recall when they
describe how they provide expository writing instruction to their eleventh-
grade students?
2. What percentage of action and/or decision steps, when compared to a gold
standard, do expert teachers omit when they describe how they provide
expository writing instruction to their eleventh-grade students?
3. Which method of CTA, 3i+3r individual or 1i+3r incremental (Lim, 2015), is
more efficient represented by the number of actions and decisions steps and
represented by cost and time?
Methodology of the Study
This study used Cognitive Task Analysis (CTA) as a method to capture and record
the knowledge and skills of eleventh grade English teachers who teach expository writing.
These English teachers work in a high school district in Southern California and had been
identified as subject matter experts (SMEs) by a set of specific criteria. Four of these
experts were randomly selected to participate in this study. Four additional, randomly
selected SMEs were chosen for the concurrent study (Lim, 2015). In both studies, three
SMEs sat for semi-structured interviews to capture their knowledge and skills. The 4
th
SME in each study was chosen to verify the data collected in the form of a protocol.
Specifically, this study is using a CTA method referred to as a 3i+3r individual method
(Flynn, 2010; Zepeda-McZeal, 2014). The concurrent study (Lim, 2015) is looking at the
COGNITIVE TASK ANALYSIS 20
same identical task of expository writing instruction, but using a different elicitation
method referred to as a 1i+3r incremental method. The CTA method followed a five step
process as suggested by Clark et al. (2008):
1. Preliminary phase to build general familiarity frequently called “bootstrapping.”
2. The identification of declarative and procedural knowledge and any hierarchal
relationships in the application of these knowledge types.
3. Knowledge elicitation through semi-structured interviews.
4. Data analysis involving coding, inter-rater reliability, and individual SME
protocol verification.
5. The development of a gold standard protocol that was used to analyze and
determine expert omissions and ultimately for use in the training of novice
teachers.
Definition of Terms
Definition of Domain Terms
Expository essay: a type of argument that asks students to take a position on a
specific topic or issue and support their position with their evidence.
Socratic Questioning: a strategy of asking questions of students to which you
already know the answer. The outcome of this line of questioning is meant to lead
students to a desired conclusion.
Argument: A formal argument emphasizes a line of reasoning that attempts to
prove by logic. When presenting an argument, the goal is to convince an audience of the
rightness of the claims being made using logical reasoning and relevant evidence.
COGNITIVE TASK ANALYSIS 21
Introduction: The first paragraph of the essay. The overall purpose of an
introductory paragraph is to properly contextualize the essay’s topic or issue to help the
reader understand what is being written about and why. The introduction typically
includes a hook and a thesis.
Thesis or the primary claim: Typically a sentence that clearly conveys the
student’s position on the topic or issue of the essay. An assertion based on evidence of
some sort.
Supporting paragraphs or Body paragraphs: The portion of the essay where
students provide evidence in support of their thesis. Evidence can come in many forms,
including but not limited to: a syllogism, numerical data, personal observations, current
and historical events, and fictional and non-fictional literature. All of the following terms
below are commonly used by teachers when teaching their students to write an effective
supporting paragraph:
Topic Sentence: The first sentence of each supporting paragraph. The topic
sentence typically indicates the argument that will be made for that particular paragraph.
Concrete Detail or Evidence: A term that often refers to the specific evidence
students use to support their thesis. A concrete detail should not be debatable. For
example, if a student chooses to use iPhones as his example to make a point about
technology, the iPhone example is the concrete detail because iPhones do exist and have
clear connections as a technological device.
Commentary: The student’s explanation or rationale as to how the concrete detail
supports his thesis. It is opinion-based. Explains how the evidence supports the claim. It
COGNITIVE TASK ANALYSIS 22
is a commonplace rule that people accept as generally true, laws, scientific principles or
studies, and thoughtfully argued definitions.
Closing sentence or Transition sentence: The last sentence of the supporting
paragraph. This sentence attempts to communicate to the reader that the argument
presented in that paragraph has now come to an end. Concurrently, the closing sentence
serves as a transition to the next paragraph.
Conclusion: The final paragraph of the essay. The general expectation is that
students finalize their argument. Students may do so by addressing opposing views,
offering pertinent arguments that were outside the scope of the essay, exploring other
possible solutions or explanations, etc.
Definition of CTA Terms
The following are definitions of terms related to CTA as suggested by Zepeda-
McZeal (2014).
Adaptive expertise: the situation where an expert can rapidly retrieve and apply
appropriate knowledge and skills to problem solving within their domain; to possess the
cognitive ability to evaluate and solve problems (Gott, Glaser, Hall, Dibble, & Pokorny,
1996; Hatano & Inagaki, 2000).
Automaticity: unconscious fluidity of performing a task as a result of repeated
execution or practice; results in automated functioning of that task (Anderson, 1996a;
Ericsson, 2004).
Automated knowledge: knowledge of how to do a task; happens outside of
conscious awareness because of the repetition of the task (Wheatley & Wegner, 2001)
COGNITIVE TASK ANALYSIS 23
Cognitive load: demands of external and internal stimuli placed on the working
memory of learners during information processing (Sweller, 1988; Feldon, 2007a).
Cognitive tasks: tasks that require mental effort and engagement to perform
(Clark & Estes, 1996).
Cognitive task analysis: techniques for knowledge elicitation that capture the
overt and covert knowledge types from experts for use in developing job aids, training
materials, or instruction design (Clark, et al., 2008).
Conditional knowledge: Knowledge type that describes the conditions when a
decision is made; knowledge that facilitates the application of declarative and procedural
knowledge to solve a problem (Hall, et al., 1995).
Declarative knowledge: knowledge type that is accessible in long-term memory
and observable in working memory; knowledge about the why or the what of something
(Clark & Estes, 1996).
Expertise: The level at which an expert has learned or acquired skills and
knowledge sufficient to ensure consistent and superior performance and complex
problem solving in a particular domain; expertise is typically developed after 10 years or
more of repeated engagement or practice in tasks specific to a domain (Anderson, 1982).
Procedural knowledge: unconscious, automated knowledge type that is developed
through instruction or through repeated practice (Clark & Estes, 1996).
Subject matter expert: a person with extensive experience in a domain who is able
to perform tasks rapidly and successfully. Subject matter experts have a solid record of
successful performance at the task being analyzed (Clark, et al., 2008).
COGNITIVE TASK ANALYSIS 24
Organization of the Study
Chapter Two of this study reviews the literature in two parts. The first part looks
at the literature on the specific writing genre of exposition, and especially its impact on
academic success. The second part discusses the literature on CTA as a knowledge
elicitation technique in capturing subject matter expertise. Next, Chapter Three addresses
the methods of this study and how the approach to the study answers the research
questions. Chapter Four examines the results of the study and describes the findings for
each of the research questions. Finally, Chapter Five discusses these findings, their
implication upon expository writing instruction and CTA, limitations of this study, and
implications for future research.
COGNITIVE TASK ANALYSIS 25
CHAPTER TWO: LITERATURE REVIEW
Expository Writing
Much has been said in research literature about the importance of writing (Beck,
Llosa, & Fredrick, 2013; Graham & Perin, 2007a; NAGB, 2010). Writing can persuade
others to action, as well as allowing writers to record their experiences and ideas for
others who are far away in distance and also in time (Graham & Perin, 2007a). Applebee
(1984) describes writing’s permanence of allowing writers to rethink and revise ideas
over time, the explicitness of writing’s ability to capture and hold meaning and ideas, and
the conventions of writing that lead to organizing and thinking through ideas and
experiences. The goal of writing teachers is to prepare their students as proficient
expository writers capable of meeting the expectations of college and the workplace.
The Importance of Expository Writing
Of writing genres, the ability to organize and compose a point of view is seen as a
necessary and often used skill not only in the corporate world, but also in secondary
classrooms and colleges and universities (NCES, 2012; Kiuhara, Graham, & Hawken,
2009; Graham & Perin, 2007b). The National Assessment of Education Progress (NAEP)
asserts that the ability to write effectively under time constraints is critical to the
economic success of the nation (NAGB, 2010). Corporations in almost all industries and
services report that more than 80% of salaried employees have some requirement for
writing within their professional responsibility, a substantial increase from previous
decades. The 2011 NAEP Writing Assessment measures three communicative purposes
common to academic and professional settings: to persuade, to explain, and to convey
experience. Of the three purposes, two of the three (to persuade and to explain) are
hallmarks of expository writing (NAGB, 2010).
COGNITIVE TASK ANALYSIS 26
The importance of expository writing is evident in both middle school, high
school, and in college as well. While many genres of writing instruction are taught to
middle school and high school students, expository writing is arguably thought to be the
most significant for academic success (Graham and Perin, 2007b; Beck, Llosa ,a&
Fredrick, 2013). Graham and Perin (2007b) have found writing assignments in middle
and high schools involve expository tasks, such as reporting, summarizing and analyzing
factual information, and expressing an opinion with the support of evidence. The
percentage of writing assignments of an expository nature increase beginning in upper
elementary school (60%), through middle school (65%) and into high school (75%).
The analytical nature of exposition is thought to be more complex and challenging
than factual genres (Beck, Llosa, & Fredrick, 2013). This assumption has support in
research showing that mastery of global text structure, or the elements making up the
macrostructure of a text, emerge later in higher grade levels for expository text than for
narrative text which appear in earlier grade levels (Berman & Nir-Sagiv, 2007). Children
are exposed at an early age to more narrative writing as an outcome of their experience
with everyday, oral discourse and simple, informal storytelling. This results in primary
grade students being taught narrative and creative writing. Expository writing, on the
other hand, is topic-based and is more suited to the type of writing in academic
disciplines found in higher grade levels. Expository writing relies on academic language
and more formal structures in attempting to explain more abstract thoughts and ideas
(Beck, Llosa, and Fredrick, 2013; Berman & Nir-Sagiv, 2007; Graham & Harris, 2013).
Indeed, expository writing can be seen as more advanced level of literacy than what
students experienced in earlier school years. Graham and Perin (2007b) argue that,
COGNITIVE TASK ANALYSIS 27
because literacy is defined as skills in both reading and writing, a scarcity in writing
proficiency among students should be recognized as an important part of our national
literacy crisis.
Definition of Expository Writing
The term expository writing has wide-ranging meaning when attempting to
determine what types of writing fit within this genre. Boscolo (1990) defines expository
writing as text which attempts to express factual information and theoretical ideas whose
general objective is to inform. Berman and Nir-Sagiv (2007) further added that because
expository texts are topic oriented, they focus on concepts and issues, and articulate the
unfolding of ideas, claims, and arguments in terms of the logical interrelations among
them. Beck, Llosa and Fredrick (2013) find expository writing to have roughly the same
meaning as argumentative writing while Chandrasegaran (2013) suggests that expository
writing is defined as writing that presents and supports a point of view with evidence and
reasons. Given that taking a stance and supporting it are the defining acts in expository
essay writing, the ability to select appropriate meanings to achieve argument support
moves seems crucial to students’ success in expository writing.
It is this focus on argumentation supported by evidence and reason that lead some
researchers (Graham & Harris, 2013; Chandrasegaran; 2013) to suggest a socio-cognitive
approach to writing instruction. The cognitive aspect of this approach describes writing
as a goal-setting, decision-making activity, which leads to explicit instruction that
supports student writers in controlling an array of skills, knowledge, and processes that
include planning, drafting, revising, editing, and publishing text. The social aspect of this
approach refers to the contextual and cultural interaction between students and authors as
COGNITIVE TASK ANALYSIS 28
students deconstruct texts written by one another. It also refers to the social aspect of
writing within classrooms as students engage in explicit and implicit dialogues between
reader and writer (Graham & Harris, 2013). Chandrasegaran (2013) wanted to see if a
pedagogical approach that integrates social and cognitive theoretical views of writing
would enable students to write better expository essays. The results support the argument
for “a visible pedagogy” in writing classrooms that integrates the explicit teaching of the
social practices of a genre with instruction in the implicit, cognitive processes for
performing the genre practices. Such a visible pedagogy would entail articulating, for the
benefit of students, the influence of social goals and cultural contexts in shaping both
genre practices and cognitive processes during writing.
The Importance of Expository Writing at the Secondary Level
Common Core State Standards
While most genres are taught in secondary classrooms, expository writing is the
genre most often used in English classes as well as other disciplines (Pascolo, 1990; Beck,
Llosa, & Fredrick, 2013, Chandrasegaran, 2013). Past standards under No Child Left
Behind (NCLB, 2001) concentrated on mathematics and reading while relegating writing
to a secondary, and often neglected, role in the classroom. Common Core (CCSS) has
reinvigorated the importance of writing such that, like reading and math, it has become a
rigorously tested curricular area. The Common Core State Standards (CCSS) include
writing standards in grades K-12 (Council of Chief State School Officers & National
Governors Association, 2010). Additionally, NCLB addressed standards at the school
level whereas CCSS focuses reform on individual teachers and teacher education
programs as well as school sites (McQuitty, 2012). Graham (2013) and colleagues
COGNITIVE TASK ANALYSIS 29
suggest that to meet writing standards and benchmarks, teachers and schools must place
greater emphasis on learning how to write and how to use expository text, especially
persuasive and informational texts, to promote learning within and across disciplines for
a variety of purposes and audiences.
The Common Core State Standards (Council of Chief State School Officers &
National Governors Association, 2010) place a premium on writing using evidence from
texts to present careful analyses, well-defended claims, and clear information. Students
must be able to answer a range of questions using evidence and inferences drawn from
the text itself. The standards’ focus on evidence-based writing to inform and persuade is a
major change from past practice. Today, the most popular forms of writing in grades K-
12 are based on student experiences and opinions which, while valuable, do not prepare
students for the demands of college and career (Colman, Pimentel, and Zimba, 2012,
Graham & Harris, 2013). Graham and Harris (2013) note that Common Core State
Standards do not tell teachers how to teach, but provide a map for teachers to create the
best lessons and classroom environments so that students to gain the required skills and
knowledge to become proficient expository writers.
High-Stakes Testing
With the implementation of Common Core State Standards (Council of Chief
State School Officers & National Governors Association, 2010) across the nation,
accountability measures are being developed and used by school districts to measure
students proficiency (McQuitty, 2012). Because CCSS looks to ensure college readiness
of students, expository writing prompts manifest themselves more often than other types
of prompts in writing assessments (Beck, Llosa, & Fredrick, 2013). Computer-based
COGNITIVE TASK ANALYSIS 30
writing assessments are being developed by Smarter Balanced Assessment Consortium
(SBAC) and Partnership for Assessment of Readiness for College and Careers (PARCC)
to assess and analyze student composition. Even the SAT and ACT have included a
writing portion to their tests in recent years (Burdick, Swartz, Stenner, Fitzgerald,
Burdick, & Hanlon, 2013). Applebee and Langer (2011) noted that secondary teachers
reported that high stakes external writing tests given in their respective states drove much
(85.7%) of their instruction. This was followed by district exams (63.6%), other exams
such as the SAT and ACT (45.7%), and Advance Placement (AP) and International
Baccalaureate (IB) exams (30.4%). Some teachers reported that they have added a time
limit to writing assessment practice in their classrooms as a result of the time limits used
by some of these tests. Finally, some teachers report that writing instruction has increased
in their classrooms because of the expectations of the Advanced Placement and
International Baccalaureate exams.
The Current State of Students’ Writing Proficiency
Students Underachievement in Writing
Student assessment data show that many of our students are not demonstrating
proficiency in expository writing tasks. The Nation’s Report Card (2012) found that 24%
of students in grades 8 and 12 scored proficient on the National Assessment of
Educational Progress (NAEP) while only 3% of students in grades 8 and 12 scored at an
advanced level. The 2013 Report on College and Career Readiness (College Board, 2013)
reports that 43% of students taking the SAT are prepared for college, a number that has
been stagnant for the past 5 years. Under California’s Early Assessment Program, 28% of
students who took the Early Assessment of Readiness for College English exam
COGNITIVE TASK ANALYSIS 31
(www.eap2012ets.org) were found to be ready for college level English Composition.
2012 English Placement Test (EPT) results show that approximately 70% scored not
proficient on the essay portion. EAP exam results show that of the 384,722 eleventh-
grade students within California who participated in 2013, only 23% (88,486)
demonstrated proficiency in English. Additional data from the CSU released in 2013
showed that only 32% of incoming freshmen demonstrated proficiency on the essay
subtest of the English Placement Test (EPT). Beck, Llosa, and Fredrick (2013) argue that
the prevalence of expository writing used as writing assessments of students at the
secondary level of schooling account for the lower scores. They also noted that lower test
scores takes into account that one-fifth of students assessed come from homes where
English is not the only language spoken. As stated before, exposition can be considered
an advanced level of literacy for which schools have not prepared students as well as they
should (Graham & Perrin, 2007b). Providing excellent education helps to alleviate
societal inequities by providing teachers the best training possible to help all students be
successful at expository writing.
Teachers’ Influence on Student Achievement
Research shows that highly-qualified teachers have proven to be most effective in
positively influencing student achievement (Vandevoort &Berliner, 2004, Darling-
Hammond, 2000; Darling-Hammond & Baratz-Snowden, 2007). Vandevoort and
Berliner (2004) looked at research on the National Board Certified Teacher program.
They looked at research with and without student assessment outcomes as measurement
data and found teachers who went through the process to be nationally certified exhibited
exemplary practices in providing high quality education and an environment of high
COGNITIVE TASK ANALYSIS 32
expectations. A smaller number of research articles using student outcomes as data also
found that NBCT program participants were more effective in increasing student
achievement.
On the other hand, Kane (2008) and colleagues found little difference in the
quality between teachers who participated in state teacher certification programs, those
who had not participated in teacher certification programs, and those who participated in
alternative certification programs. However, they did find differences in the ability of
teachers within their experimental and control groups. These differences among teachers
were positively correlated to increased student achievement. Vandevoort and Berliner
(2004) concede that teachers who attempt NBCT certification may already be effective
before they begin the process. They suggest that perhaps effective teachers might be more
willing to participate in activities that improve their own effectiveness.
Darling- Hammond (Hammond, 2000, Hammond & Baratz-Snowden, 2007) and
colleagues found that highly qualified teachers are more effective in increasing student
achievement gains. More troubling is the finding that students who have several
ineffective teachers consecutively have significantly lower achievement gains than their
peers who have highly effective teachers in consecutive school years. When aggregated
at the state level, teacher quality variables appear to be more strongly related to student
achievement than class size, overall spending levels, or teacher salaries.
Darling-Hammond and Baratz-Snowden (2007) describe the myriad ways that
highly effective teachers use pedagogical strategies, content knowledge, and quality
assessment to determine they provide and measure quality instruction in their classrooms.
COGNITIVE TASK ANALYSIS 33
It is these actions and their judicious use that helps these highly-qualified teachers make
significant increases in student achievement.
Teachers Feel Underprepared to Teach Writing
Even if we assume that every teacher is highly effective, this still does not mean
all teachers are providing the high quality expository writing instruction they would like.
Teachers themselves report they feel underprepared to teach writing in their classrooms.
Read and Landon-Hays (2013) interviewed high school teachers to find the difference
between what teachers know and what they do when giving writing instruction in the
classroom. The obstacles they identified include teacher’s personal experiences and
learning opportunities as students, preservice training in writing instruction, and the
realities of teaching and assessing of daily writing.
Teacher experiences as students. Studies have lamented the poor quality of
writing instruction in high schools and middle schools across the nation for many years
(Kiuhara, Graham, & Hawken, 1991). If one could extrapolate the findings Kiuhara,
Graham, and Hawken (1991) found in most schools throughout the country, the students
at the time of that research (1991) could certainly be the teachers of Read and Landon-
Hays (2013) more recent research. The teachers in Read’s and Landon-Hays’ (2013)
findings report the writing instruction they received in high school was formulaic and
devoid of any application of any formal writing process. They described writing as
simple and undemanding. This corresponds with the findings of Fanetti, Bushrow, and
DeWeese (2010), who interviewed secondary teachers and college instructors and found
complaints of incoming freshmen struggling to write beyond formulaic writing. And yet
COGNITIVE TASK ANALYSIS 34
high school teachers interviewed felt compelled to use formula writing so students could
pass high-stakes writing assessments.
It seems that high school teachers are aware there are more effective strategies,
such as peer editing and teacher modeling. But their own experiences in high school did
not give them exposure to this type of instruction (Read and Landon-Hays, 2013,
Grossman, et al., 2000). Given this lack of exposure, teachers know and use various
pedagogical strategies to teach writing but are unsure what good instruction looks like. In
the stress of daily teaching, teachers often resort to instruction practices with which they
are most familiar.
Preservice training in writing instruction. Teachers interviewed by Read and
Landon-Hays (2013) reported they received what they perceived as insufficient training
in teacher preparation courses. Reports by these teachers were mixed. Some received
very little preparation in writing instruction and assessment. A few teachers report they
received some theoretical preparation, but were given little guidance in how that learning
would be implemented in the classroom. All agree that their varying levels of preparation
in writing instruction did not prepare them for teaching writing in the context of a
classroom. This is corroborated by Kiuhara (2009) and colleagues who surveyed 355
teachers, finding 71% of teachers report little or no preparation and 44% report little in-
service training in writing instruction. Grossman (2000) and colleagues interviewed
teachers new to the classroom who describe conflicting views on how to address writing
in the classroom. Preservice preparation taught these new teachers to encourage creativity
and develop a writer’s workshop model in class. However, school districts often purchase
curriculum which teaches writing in a structured program with teacher-led instruction and
COGNITIVE TASK ANALYSIS 35
little student interaction. This leads to confusion and tension within teachers who feel
compelled to follow district mandates even if they conflict with the teacher’s views of
writing instruction (McQuitty, 2012; Applebee and Langer, 2011).
Kiuhara (2009) and colleagues describe the successful strategy of having students
write across different subjects or domains. But they and others (Fanetti, Bushrow, and
DeWeese, 2010) find that students’ experiences in expository writing tend to come
mostly from English classes, with some assignments from social studies classes, and very
few writing experiences from science or math classes. Kiuhara (2009) and colleagues
conclude this occurs because most training in writing instruction occurs in English
teacher preparation courses and not as much in other content areas. Considering the
varied amounts of training teachers receive in writing instruction, researchers find that
the amount of writing done in class depends on the teachers’ self-efficacy in writing
ability and personal enjoyment of writing itself (Grossman, et al., 2000; Kiuhara, Graham,
& Hawken, 2009; Zumbrunn and Krause, 2012).
Realities in teaching and assessing writing
Writing differs from most subjects in that its use can be found throughout most
middle school and high school classes. Writing is often used to assess student learning in
domain-specific courses as well as in English courses. Additionally, many assignments
given in other subjects have a distinct beginning and end while writing is considered to
be more of an on-going process (Graham & Perin, 2007a). Teachers interviewed for
multiple research articles universally agree that getting students to see writing as a
process and not as meeting some predetermined goal is a most intractable problem (Read
& Landon-Hays, 2013, Grossman, et al., 2000; Kiuhara, Graham, & Hawken, 2009).
COGNITIVE TASK ANALYSIS 36
Most teachers report that they want their students to become familiar with the writing
process, but struggle with limited student attention spans and curricular requirements to
cover multiple writing genres, without adequate practice in successful writing strategies
such as modeling and scaffolding (Grossman, et al., 2000). College instructors bemoan
the fact they must help entering freshmen unlearn the rigid rules and structure high school
teachers push daily in fear of poor performance on standardized tests. Secondary teachers
respond that they feel compelled to teach students formulas and shortcuts in the hopes
they may achieve a high score on assessments rather than to write something meaningful
(Fanetti, Bushrow, & DeWeese, 2010). Grisham & Woolsey (2011) report minimal
improvement in the quality of student writing over the past two decades and that many
college entrants are surprised to find themselves completing remedial courses in writing.
Another problem faced by secondary teachers in their daily work is enough time
to teach writing the way they believe they should (Read & Landon-Hays, 2013,
Grossman, et al., 2000). Teachers generally concur that more time is needed to teach
students individually rather than in a traditional whole-class, transmission model of
teacher-led lecture.
Socio-cultural writing models propose that students should write authentic pieces
that reflect their own interests. They also suggest students create goals for writing
improvement (Chandrasegaran, 2013; Applebee and Langer, 2011; De La Paz and
Graham, 2002). Teachers then offer feedback based on those goals. Students should also
receive peer feedback on their writing (Graham & Perin, 2007a). The reality of district
mandates and state required writing assessment make this quite difficult. Read &
Landon-Hays (2013) noted that one teacher in their study attempted to create a holistic
COGNITIVE TASK ANALYSIS 37
writing program that included teacher feedback while striving to meet district
requirements. This teacher found that many hours of grading were needed beyond the
contract teacher day. Teachers also found that by the time they graded the work days later,
the students had forgotten the assignment.
A related problem to the time crunch felt by secondary teachers is the amount of
students they are required to teach. Many teachers have multiple classes of students with
each class having from 30-40 students (Kiuhara, Graham, & Hawken, 2007; Read &
Landon-Hays, 2013). Some teachers feel they must grade each paper they see to give
students feedback. Some teachers, however, concede they cannot grade everything that
crosses their desk (Grossman, et al., 2000; Alvermann, 2002). Given the time crunch and
the number of students on their attendance sheets, teachers are in a quandary about
providing enough meaningful writing experiences and at the same time being able to
respond to the needs of each student.
It appears that nibbling at the edges of reform will not be enough to make writing
instruction more efficient and manageable by teachers. Whether less-than-effective
writing instruction can be blamed solely on external factors, a more important change
must come from within the teachers themselves through more effective professional
development (PD) that helps novice and expert teachers gain subject matter expertise and
build capacity in expository writing instruction.
COGNITIVE TASK ANALYSIS 38
Subject Matter Expertise
The Need for Expertise in Writing Instruction
Highly effective teachers are considered as such because of their expertise in
subject matter knowledge, knowing how to teach the subject matter to others, and having
knowledge about how children learn, feel, and develop. They are also experts in self-
awareness, social skills, and organizational competence (Smith, 2005; Darling-Hammond
& Baratz-Snowden, 2007). While teachers have little to no control over external obstacles
such as class size or available time with students, they do have control over their
acquisition of the subject matter knowledge they teach. Graham (2013) and colleagues
find that, in order to meet Common Core State Standards writing benchmarks (Council of
Chief State School Officers & National Governors Association, 2010), teachers and
schools must place greater emphasis on learning how to write and how to use expository
text, especially persuasive and informational texts, to promote learning within and across
disciplines for a variety of purposes and audiences. Common Core State Standards
provide teachers with a roadmap on what objectives, topics, and skills they should cover
in the school year. But this map is of limited value if teachers do not possess the
knowledge, skills and tools needed to achieve the outlined objectives. Graham (2013) and
colleagues suggest that writing is a goal directed and self-sustained cognitive activity
requiring management of the writing environment, management of the constraints
imposed by the writing topic, the purposes of the writer(s), and the processes, knowledge,
and skills involved in composition. Writing teachers must possess considerable wisdom
about how to teach writing effectively. If the teachers possess effective tools to teach
writing, they are more likely to achieve the goals of CCSS.
COGNITIVE TASK ANALYSIS 39
The advantage of studying exceptional teachers is that it allows one to examine
what they do in the classroom, enriching our understanding of what effective writing
instruction looks like in these situations. These observations can also be used to provide
teacher preparation and professional development (PD) programs based on this expertise
(Smith, 2005; Saphir, 2011; Graham, Gillespie, & McKeown, 2013). Saphir (2011)
proposes that expertise must be expanded into clear exemplars or standards that educators
can understand at the concrete level and that are tied to performance assessments. Ozer
(1998) argues these standards are the models of actions backed by scientific reflection
that teachers take within the context of the classroom. They are also much more than
automated skills, such that they cannot be easily perceived by laymen or specialists
outside of education. Professional development, based on these actions and discrete
skills and decisions, can then move away from being reactive to individual teacher
evaluation prescriptions or driven by local needs assessments. They can then move
toward a clear, uniform vision of high-expertise practice.
Saphir (2011) adds that proficiency in the knowledge, skills, and pedagogical
practices that make up good teaching would be the highest-leverage path to increase
student achievement. Every effort should be made by teachers and school leaders to make
sure expert practices show up consistently in every classroom.
Summary
Expository writing is a necessary skill for students to learn to help ensure
academic success at school and in the universities, and to help ensure professional
success in the workplace. In this time of increased academic standards and increased
high-stakes, standardized testing, instruction in expository writing has risen in importance.
COGNITIVE TASK ANALYSIS 40
The complex nature of expository writing depends on academic language to support
meaning and structures based on logical reasoning. This results in an academic domain
where students benefit from support from peers and explicit teacher-led instruction. And
while having a qualified teacher in every classroom is the most effective way to improve
student academic performance, not all teachers feel prepared to teach expository writing
effectively. It behooves curriculum planners and teacher trainers to find and share the
knowledge and skills that expert teachers use to teach their students to be successful
expository writers. This knowledge can then be shared with novice teachers so that they
can teach their students more effectively with greater expertise.
Using Subject Matter Experts to Train Non-experts
Experts are often called upon for their knowledge and skills to teach novice
learners, to inform curriculum content and instructional material development, and to
mentor or coach others to perform complex tasks and solve challenging problems. One of
the purposes of education is the traditional view that education is to replicate knowledge
(Jackson, 1985). A historical view of education begins with a system based on the
relationship between a master and apprentice. The job of the apprentice was to imitate the
master. As education become more ubiquitous and egalitarian, the master-apprentice
relationship became one teacher to many students. And yet, the function of the student
imitating the teacher as a process of learning remained (Jackson, 1985). A more modern
view of the purpose of education is to teach children how to learn rather than fill their
heads with what the teacher finds important (Glassman, 2001). Whether learning
knowledge or learning how to gain that knowledge through inquiry, imitation by students
as novices of what the expert teacher knows or does is still the model used in education
COGNITIVE TASK ANALYSIS 41
(Jackson, 1985).
Recent research shows that experts may omit up to 70% of the knowledge and
skills novices need to replicate the performance of experts. Feldon and Clark (2006)
looked at self-reports of subject matter experts (SMEs) and found that SME reports were
prone to omission errors (from 48% to 88%). Omission errors are the failure of these
experts to report a step that was taken without misrepresenting the event’s sequence.
These omission errors were higher than commission errors (0% to 5.7%). Commission
errors are statements made during self-reporting that misrepresent reported events by
stating incorrectly either the order of the steps or the carrying out of steps that did not
occur. The study concluded that self-reports of experts are often inaccurate and
incomplete. Errors of omission and commission can obstruct a novice’s performance
based on the knowledge collected (Feldon & Clark, 2006). Because most of the errors
were omitted by experts as they recount the steps they take to perform a complex task,
novices who receive this incomplete information fill the holes with their own information,
which often contains misconceptions and guesses. Experts omit critical knowledge and
skills because they have automated their skills and knowledge through repeated practice
to such a degree that they become unconscious and difficult to recall.
Cognitive Task Analysis has been shown to be an effective method in capturing
both the conscious controlled knowledge and unconscious, automated knowledge experts
use to perform complex skills and solve difficult problems (Clark, et al., 2008). To
further understand the effectiveness of CTA, the following sections examine knowledge
types, the nature of automaticity, and the characteristics of expertise.
COGNITIVE TASK ANALYSIS 42
Knowledge Types
Knowledge can be classified into distinct types of cognitive processing with
different functions and uses. Merrill (1983) suggested in his first iteration of Component
Design Theory that instructional goals can be classified in two dimensions: content
knowledge and performance goals. The first dimension is content knowledge types,
which can be classified as facts, concepts, principles, and procedures. In a newer version
of CDT (1990), Merrill and colleagues also suggested principles can be referred to as
processes. The second dimension of CDT is performance goals which are exhibited by
the learner. The first version of CDT (Merrill, 1983) classifies performance goals as
remembering, using, and finding, while the newer version of CDT adds sub-goals to each
performance goal type (Merrill, 1990).
The performance goals described by Merrill (1983, 1990) above associated with the
performance of learning can be described in another way. To show that one remembers a
piece of knowledge, the learner must declare or tell that fact for the cognitive process to
be observable. And, to show that one is using (applying) or finding (creating) new
knowledge, the learner must use or apply a learned procedure or process to exhibit
cognitive processing in an application setting.
Merrill (1990) argues that the way knowledge is shared with students changes the
learner’s cognitive structure and results in learned behavior. Instruction must not only
provide the appropriate representation of content to be learned but also assist the learner
in using or applying this representation. CTA looks to provide novices the knowledge
and skills needed to change cognitive structures and allow learners to use their newly
gained learning in application.
COGNITIVE TASK ANALYSIS 43
Schneider and Shiffrin (1977) first described knowledge as two different types of
cognitive processing. They described controlled processing as a temporary act in a
sequence that has not yet been learned. This type of processing is easy to set up, modify,
and use in novel situations. It requires attention and makes liberal use of a learner’s
cognitive load. Controlled processing, often referred to as declarative knowledge (Clark,
et al., 2008), is easily recalled from long-term memory and is consciously observable.
While this declarative knowledge can be taught so that it is remembered, it is not enough
to ensure successful performance. Declarative and procedural knowledge are not the
same and enable different types of performance. One may know facts but not be able to
perform the procedure or know when to execute the function. Similarly, one may have
the ability to perform a function but not able to explain why they are doing it (Anderson,
1982; Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010).
The second type of cognitive processing described by Schneider and Shiffrin (1977)
is automatic processing. While it is triggered by appropriate inputs, it operates
independently of the learner. Automated processing does not require the attention of the
subject and does not tax cognitive load. Automated processing, also referred to as
procedural knowledge, is required for performing a complex skill (Clark, et al., 2008).
Application of procedural knowledge is repeatedly practiced by experts to such a degree
that these skills become automatically learned and difficult to recall, being outside the
conscious long-term memory of the expert.
Declarative and procedural knowledge are acquired as one transitions from novice
to expert (Paris, Lipson, & Wixson, 1983). Declarative knowledge and procedural
COGNITIVE TASK ANALYSIS 44
knowledge will be discussed in detail, as will a form of procedural knowledge known as
conditional knowledge.
Declarative Knowledge
Declarative knowledge is made up of concepts, processes and principles that are
controlled consciously by the learner and can be changed abruptly in the working
memory. Declarative knowledge is factual, goal-independent knowledge (Clark & Estes,
1996; Corbett & Anderson, 1995; Schneider & Shiffrin, 1977). It is the retrievable
information that can answer the “why” and the “what” about facts, concepts, events, and
objects (Anderson & Schunn, 2000). More importantly, declarative knowledge is
characterized by its conscious quality and the speed at which this knowledge can be
learned or modified. Declarative knowledge is most adept at helping learners handle new
and different tasks.
Nearly all new knowledge that comes into the cognitive process is declarative
knowledge (Anderson & Fincham, 1994). It is committed to long-term memory through
practice and elaboration where it is then converted into procedures made up of production
rules. Knowledge is learned at first in a conscious, declarative form. Because of repeated
learning or practice, this declarative knowledge is transformed over time into an
unconscious, automated procedural form (Anderson, 1982). Declarative knowledge, the
knowing why and what of an object or idea, begins the cognitive process that supports the
creation of procedural knowledge, which is the how and when of an object or idea
(Anderson & Krathwohl, 2001).
COGNITIVE TASK ANALYSIS 45
Procedural and Conditional Knowledge
Declarative, procedural, and conditional knowledge are required for completing
complex tasks and are acquired as one transitions from novice to expert. Procedural
knowledge is goal-oriented, facilitation knowledge about “when and how” to perform a
task or solve a problem (Corbett &Anderson, 1995). Procedural knowledge includes
the steps and sequences to be followed when completing a task. This type of
knowledge is also subject-specific as many steps and procedures do not transfer readily
across all domains (Anderson & Krathwohl, 2001). Procedural knowledge is the
knowledge of processes and procedures, which the performance of these can lead to the
acquisition of new declarative or factual knowledge (Merrill, 1983).
Skill acquisition is made up of two stages. First, declarative knowledge is
acquired as facts about a skill. This knowledge is then compiled and interpreted by the
leaner. These facts drive the creation of decisions of how and when to use these steps.
In CTA, the use of “IF/THEN” statements describes these decision steps as learners
decide how and when to complete these steps (Anderson, 1982). As this knowledge is
continuously applied or practiced, it becomes more automated and needs less cognitive
processing to execute. However, once this knowledge becomes automated, it is hard to
change or revise because of its unconscious quality (Anderson, 1993).
Procedural knowledge is not just knowledge of how to complete the steps and
sequences of performing a task. Learners must know when and why to use these steps
and sequences, and which of various steps and sequences to use (Anderson &
Krathwohl, 2001; Ambrose, et al., 2010). This classification is referred to as
COGNITIVE TASK ANALYSIS 46
conditional knowledge, a sub set or type of procedural knowledge. Conditional
knowledge moderates the fact-to-action process (Anderson, 1982).
For educators, it is important to assess student’s prior knowledge in terms of both
declarative and procedural knowledge (Ambrose, et al., 2010). Learners need practice to
strengthen both factual knowledge and production rules. Retention of knowledge is a
function of how effectively information was learned and practiced (Paris, et al., 1983).
With repetition and practice, both declarative and procedural knowledge become stronger
and performance becomes more fluid, rapid and consistent (Corbett & Anderson, 1995).
Automaticity
Through repeated performance and deliberate practice of a task, declarative and
procedural knowledge becomes automated and unconscious in nature, and the speed in
performing the task increases while the amount of active cognitive effort decreases
(Feldon, 2007a; Kirschner, Sweller, & Clark, 2006).
Anderson (1996b) suggests three stages that lead to automaticity of expert knowledge.
The first stage is the interpretive stage or cognitive stage in which a learner is able to
complete a task or a close approximation of the task with initial instructions that are often
verbal. This stage frequently involves the learner talking to oneself when performing the
action. The second stage is the knowledge compilation or associative stage. In this stage
the learner works through the procedure and applies or learns the declarative knowledge
necessary to correct any procedural errors. As errors are corrected, the learner develops
stronger procedural knowledge. The verbal cueing of talking to oneself begins to
decrease and ultimately disappears. The third stage is the strengthening and tuning of
knowledge, or autonomous stage, where the learner performs the procedure automatically
COGNITIVE TASK ANALYSIS 47
without verbal cueing and any changes made to the procedure serve to strengthen the
process and make it more efficient (Wheatley & Wegner, 2001, Anderson, 1996b).
Ericcson (1993) and colleagues identified a fourth stage of automaticity. Reserved
only for expert performance, this stage describes experts who have mastered the majority
of the training provided by their teachers or coaches. At this level, these experts also
begin to add their own innovative and creative contributions to their field of expertise.
With practice, cognitive tasks become fluid and automatic and subject matter
experts are able to deploy strategies to solve problems with ease (Clark, 1999). Expertise
is developed through repetition of task performance during deliberate practice. Task
performance improves when behavior is adapted and performance feedback is received. It
is deliberate practice that is essential for expertise acquisition and continuous
improvement of performance. Expert performance reflects intense training and
preparation and, in most domains, at least 10 years of experience is required to reach this
level of performance (Ericsson, et al., 1993).
Clark and Elen (2006) assert that automation of knowledge is advantageous to expertise
as it sustains the capacity of experts to respond to novel problems with speed, accuracy,
and consistency within a specific domain. Research has suggested that experts are
unaware of the information they use to complete complex tasks because of automaticity.
As experts develop declarative knowledge, it becomes gradually more automated.
Automated processes often initiate without prompting and once they initiate, automated
processes run to completion without being available for conscious monitoring (Feldon,
2007a). Automated processes are resistant to change because of their unconscious nature,
and it takes considerable sustained monitoring of mental processes to modify or eliminate
COGNITIVE TASK ANALYSIS 48
an automated process (Clark, 2008; Wheatley & Wegner, 2001).
Automated knowledge helps to alleviate cognitive overload and/or processes that
can impede the efficiency of working memory. The length and amount of information
that can be retained and processed in working memory is limited. Procedural knowledge
is difficult to articulate because it has become an automated, unconscious action. Critical
information omitted by experts may thwart effective knowledge sharing (Kirshner, et al.,
2006; Wheatley & Wegner, 2001). Feldon (2007a) noted that even when teachers are
made aware of omissions in their automatic teaching processes or are provided with goals
to modify these automated processes, they fail to make changes because their working
memory becomes occupied with the changes needed while their automated processes
begin and run to completion because working memory is occupied. Because experts have
automated procedural knowledge they cannot consciously explain, methods like CTA are
critical to deconstruct this knowledge into its original steps (Clark & Estes, 1996).
Automaticity enables subject matter experts to perform complex tasks requiring
declarative and procedural knowledge with less cognitive processing through repeated
use and practice. This frees up working memory to address novel tasks. However, due to
its unconscious quality of automaticity, procedural knowledge is resistant to change and
difficult to modify, eliminate, or express to others using concrete language and examples.
Expertise
Characteristics of Experts
The characteristics of expertise include extensive and highly structured
knowledge of the domain, effective and often multiple strategies for solving problems
within their domain (Glaser & Chi, 1988), and expanded working memory that utilizes
COGNITIVE TASK ANALYSIS 49
elaborate schemas to organize information effectively for rapid storage, retrieval, and
manipulation. An expert is one who has accomplished this within their specific domain
(Feldon, 2007b).
Chi (2006) defines an expert (adapted from Hoffman, 1998) as a distinguished or brilliant
journeyman highly regarded by peers whose judgments are uncommonly reliable and
accurate. The expert’s performance shows skill and economy of effort and the ability to
deal effectively with rare or “tough” cases. An expert is one who has special skills or
knowledge learned from extensive experience within their specific domain of practice
(Feldon, 2007b; Bedard & Chi, 1992). Ericsson and Lehman (1996) add to this the idea
that expert performers can display their superior performance reliably upon demand. To
achieve this control, they contend expert performers need to master all relevant factors—
including motivation.
One important difference between a novice and expert is knowledge organization.
Experts not only have more knowledge than a novice in a certain domain, but they also
have more advanced knowledge stored cognitively in more developed schema. This
information is organized to allow experts the rapid retrieval of information with minimal
cognitive effort (Feldon, 2007b; Chi, 2006; Glaser & Chi, 1988). Novices rely on literal,
predictable surface information to solve complex tasks, whereas experts rely on concepts,
deeper learning, and mental models not readily apparent. Experts also have increased
incidental memory and memory skills that improve and increase knowledge acquisition
and manipulation (Feldon, 2007a; Ericsson & Lehmann, 1996; Glaser & Chi, 1988).
Experts are able to discriminate between various cues and develop representations that
create meaning to complex problems (Feldon, 2007b; Bedard & Chi, 1992). Experts can
COGNITIVE TASK ANALYSIS 50
engage in forward reasoning processes based on their domain knowledge. They leverage
their highly structured knowledge of relevant concepts and principles within the domain
to generate effective strategies (Feldon, 2007b; Glaser & Chi, 1988). Experts solve
problems deductively by manipulating mental models to identify optimal solutions based
on requirements of task and task constraints. Thus, experts can see and detect features
and solutions that novices cannot (Chi, 2006). It is the goal of CTA to capture this
expertise and provide to novices this opportunity to improve the accuracy and efficiency
of their own knowledge and skills.
Building Expertise
Expertise, by its nature, is acquired as a result of continuous and deliberate practice in
solving problems in a specific domain. Expert knowledge was once thought to be a gift
from the gods (Ericsson & Charness, 1994), and was later thought to be a natural or
inherited trait. However, research in inherited expertise has been unfruitful (Ericsson,
Krampe, & Tesh- Romer, 1993; Ericsson & Charness, 1994). Modern research has shown
thus far that experience and practice are the main components of expertise. Simon and
Chase (1973) found that experts become so after approximately 10 years of deliberate
practice and experience. Expertise attainable in one domain is not easily transferable to
other domains. Thus, an expert’s knowledge and skill are important attributes of the
expertise itself more so than natural ability (Ericsson, et al., 1993).
Ericson (2004a) suggests a theory of skill acquisition by arguing that the primary
goal of a learner is to reach a level of mastery that allows them to perform tasks at an
acceptable level or engage in recreational activities with friends at a proficient level. In
the first phase, the novice tries to understand the activity and concentrates on avoiding
COGNITIVE TASK ANALYSIS 51
mistakes. The second phase finds large mistakes becoming increasingly rare while
performance begins to appear more natural and smooth. The amount of concentration to
perform the task begins to decrease as well. After a limited period of experience and
training, the novice has reached an acceptable level of performance. Over time and with
continued practice, the novice’s ability become more automated as they perform skills
with little apparent effort. Soon the performer has reached a high level of performance,
but often not maximum performance (Ericsson, et al., 1993).
Ericsson (2004) argues further that at this level of expertise, improvement
requires deliberate practice and conscious efforts to circumvent the automatic,
unconscious skills already learned. It is this deliberate practice with attendant monitoring,
planning and analyses of performance that is needed to attain further changes towards
maximum performance (Ericsson & Charness, 1994; Ericsson, et al., 1993). An expert’s
inability to improve is not attributable to lack of talent but more likely to a lack of
deliberate effort, feedback, and planning. It is possible, however, that a lack of
improvement could be attributable to an innate physical barrier that prohibits expertise
(Ericsson & Charness, 1994, Ericsson, et al., 1993).
Alexander (2003) proposed the Model of Domain Learning (MDL). This theory
shows the nature of developed expertise in academic domains rather than defining this
nature from non-academic tasks within the realm of problem solving. The Model of
Domain Learning looks at the path that a novice takes to be considered an expert. There
are three components that play a role in creating expertise in academic domains. They
are knowledge, strategic processing, and interest. These components are thought to
influence one another at each stage but at different rates. These three components work
COGNITIVE TASK ANALYSIS 52
together as individual learner’s progress through three stages of domain learning. The
first stage of MDL is acclimation where the learner becomes acclimatized to a complex
and unfamiliar domain. The second stage is competence where the learner begins to
demonstrate a foundational body of knowledge from within the domain which is
becoming more cohesive and principled in its cognitive structure. The third and final
stage is proficiency where the components have synergistically worked together to move
the learner from competence to expertise. The learner’s knowledge base is not only broad
but deep, and characterized by new knowledge contributions by the expert learner.
By engaging in deliberate practice and problem solving, a novice learner develops
over time (usually 10 years) more efficient schema, knowledge, skills and decision steps.
Consequences of Expertise
As new knowledge becomes automated and unconscious, experts are often unable to
completely and accurately recall the knowledge and skills that comprise their expertise,
thereby negatively impacting instructional efficacy and leading to subsequent difficulties
for learners. Expertise is domain limited. Experts also rely on contextual cues within
their domain and often overlook details or surface features of a complex task (Feldon,
2007b; Chi, 2006).
When pressed, experts are overly confident in their knowledge and ability to share
with others. They often make inaccurate predictions and offer incorrect advice or
judgments. Their knowledge is automated to such a degree that it is ingrained deeply and
difficult to change or modify. Experts, while knowledgeable of multiple approaches to
problem solving, often fall into habitual approaches that are goal-activated and often limit
the available solutions sought to solve problems (Chi, 2006, Feldon, 2007b).
COGNITIVE TASK ANALYSIS 53
Feldon (2007b) found inaccuracies prevalent in explanations of self-reported problem
solving processes by experts. The research found that as skills improved, self-report
errors increased. Feldon found that experts attribute their action to intentional decision-
making processes. Yet, this belief can lead these experts to unintentionally fabricate
reasoned explanations for their behavior. The highly efficient schemas that store and
retrieve data can also interfere with accurate recall of procedural knowledge.
Consequently, the most often employed elements would be the most difficult to articulate
through recall. The automaticity of experts impairs their ability to consciously identify
many of the decisions they make thereby omitting key details and process information
necessary to provide instruction on optimal performance.
Expert Omissions
Experts in an instructional role may unintentionally leave out information that
students must master when learning procedural skills. Feldon (2004) found in a study of
instruction research design that automaticity and self-reporting accuracy were negatively
correlated. Feldon found that 70% of experts were unaware of the strategies used in
practice. Feldon and Clark (2006) found that when experts describe how they perform a
complex task, they unintentionally omit up to 70% of information critical for novices to
learn to successfully perform a procedure (Clark, 2008; Bedard & Chi, 1992). Novices
attempt to fill in the gaps with their own steps developed using trial-and-error methods
prone to mistakes and inefficiencies. As these novices continue to use and practice these
gap-filling steps, they become more difficult to modify and unlearn (Clark, 2008).
Clark (Clark, Pugh, Yates, Inaba, Green, & Sullivan, 2011) and colleagues found in a
study of surgeons that expert surgeons have less time to share expertise with novice
COGNITIVE TASK ANALYSIS 54
surgeons as training in hospitals has moved towards a more simulated environment.
Expert surgeons have gained expertise through practice and experience such that the steps
used by these veterans become blended together (Clark & Elen, 1996). When asked to
describe the procedure, these experts omit specific steps because of the unconscious
nature of the automated processes. Additionally, experts omit essential information and
are unable to identify points in automated procedures where important decisions are made.
Consequently, these errors are not recognized by experts because of their own automated
knowledge (Wheatley & Wegner, 2001). More importantly, these errors are prone to
increase in number and impact during stressful situations (Joslyn & Hunt, 1998, in
Shraagen, 2000). CTA looks to counteract these omissions by capturing expert
knowledge and skills through elicitation and knowledge representation methods.
Cognitive Task Analysis
Definition of CTA
Cognitive Task Analysis has evolved from traditional behavioral task analysis (BTA)
methods, and is utilized to elicit and explain expert knowledge within a specific
domain. CTA uses a variety of interview and observation strategies to capture the
explicit and implicit expert knowledge that experts use to complete complex tasks. CTA
is an outgrowth of traditional BTA. However, the behavioral focus of traditional methods
makes them inadequate to support current demands (Clark & Estes, 1996). Beyond
traditional behavioral task analysis methods, CTA identifies the knowledge, thought
processes, and goal structures that trigger observable task performance, as well as overt
and covert cognitive functions (Chipman, 2000; Clark et al., 2008). CTA yields
information through elicitation techniques that produce knowledge and skills that can be
COGNITIVE TASK ANALYSIS 55
used for creating learning objectives, job descriptions, schemas, hiring criteria, and
performance appraisal systems.
CTA History
The historical foundations of CTA were planted as far back as 1880 and are found
throughout the history of applied psychology and industrial engineering (Hoffman &
Woods, 2000). Militello and Hoffman (2008) state that the foundation of modern CTA is
rooted in Taylor’s (1911) time and motion studies and the work of Frank and Lillian
Gilbreth who studied the cognitive and collaborative elements of work performance to
improve performance (Annett, 2000; Schraagen, Chipman, and Shalin, 2000).
Human factors in the operation of complex machine systems became obvious
areas in need of research before and especially after World War II. The study of cognitive
engineering resulted from advances in technology and computers, as well as
computerization within settings which required the need to understand human behavior in
complex situations. Cognitive engineering also sought to describe how problem-solving
in complex situations can be improved as they evolved from the increased workload,
mental load, and cognitive task load that have resulted from rapid advances in technology
(Annett, 2000; Woods & Roth, 1988).
The term Cognitive Task Analysis came into general use among the education and
technology community in the early 1970s from the research of Gagne (1962) and Glaser
(1976). The complex and cognitive demands of the workplace since the 1980’s have
encouraged the use of CTA as well as fueling the demand for CTA-derived expert
systems and other applications of artificial intelligence (Hoffman & Woods, 2000). The
need for change driven by social, psychological and cognitive factors has resulted in CTA
COGNITIVE TASK ANALYSIS 56
being used to meet the need for improved human performance in work settings and expert
systems (Clark & Estes, 1996). CTA has been used in many studies and is now one of
the most successful elicitation methods of expert knowledge used today.
The advanced study of human cognition ushered in the need to understand human
behavior in complex systems, and a deeper understanding of cognition in human
performance. Interest increased in capturing human expertise which time and motion
studies could not capture and capturing the mental processes and decisions behind expert
performance, and the illumination of declarative and procedural knowledge. However,
the basis of CTA in cognitive theory is not fixed because models of cognition are
somewhat fluid (Annett, 2000). CTA is the advanced task analysis system that can
capture complex cognitive decisions and knowledge, thereby helping to fill the gap that
the outward focus of BTA cannot see.
Cognitive Task Analysis Methodology
A number of researchers have identified the stages through which a typical, ideal
cognitive task analysis would proceed. An ideal model of cognitive task analysis, one that
is not subject to resource restrictions, is typified by a series of distinct steps:
1. A preliminary phase.
2. The identification of knowledge representations.
3. Knowledge elicitation techniques.
4. A review and possible modification of the knowledge elicited to date by
experts.
5. Using the results of the analysis as a basis for an expert system or expert
cognitive model. (Chipman, et al., 2000; Clark, et al., 2008)
COGNITIVE TASK ANALYSIS 57
While over 100 types of cognitive task analysis have been developed, most
varieties follow a five-stage process. Multiple authors have developed taxonomies that
categorize these techniques according to a number of criteria.
Clark (2014) and Clark et al. (2008) suggests that the Concepts, Processes, and
Principles (CPP) is one of the most often used evidence-based CTA methods. CPP is
based on the PARI (Precursors, Actions, Results, and Interpretations) method but adapted
to incorporate Merrill’s (1994, 2002, 2006) recommendations for instructional design.
PARI is a process where experts, working in pairs, look for complex cognitive and
behavioral demands in each of the 4 categories above. The experts think aloud, ask
probing questions, and use diagrams or drawings (Clark, 2014; Hoffman & Militello,
2009; Yates, 2007; Yates & Feldon, 2001, Tofel-Grehl & Feldon, 2013). CPP has been
shown to be an effective way across disciplines to capture expert knowledge.
Taxonomies of Knowledge Elicitation Techniques
Knowledge elicitation is the process of extracting domain specific knowledge that
underlies human performance. Cooke (1999) identified four categories of elicitation. The
first is observations where analysts observe task performance within a domain and
provide a general conceptualization of the domain observed and constraints and issues to
be addressed in future phases. The second category is interviews. This is the most often
used elicitation method using various types of interview techniques such as structured,
unstructured, goal decomposition, teach-back, and PARI (Cooke, 1994). The third
category is process tracing which entails the collection of behavioral events with an
analysis of the resulting protocols. Inferences are made about underlying cognitive
processes and is the most often used to elicit procedural information. The fourth category
COGNITIVE TASK ANALYSIS 58
is conceptual methods. Conceptual methods gather and represent conceptual structures in
the form of domain-related concepts. It is used to gather knowledge to improve interface
designs, to guide development of training programs, and to understand expert-novice
differences. Cook (1999) suggests the defining characteristic of knowledge elicitation is
the collection of information from a human source of knowledge. Techniques used most
often by determine the actions of the knowledge elicitor. Generally, the more formal the
technique used, the less active the role of the knowledge elicitor. The less formal
techniques often require more introspection and verbalization from the expert. Formal
methods require more preparation of elicitation materials. However, formal methods are
more artificial and lack face validity (Cooke, 1994).
Pairing Knowledge Elicitation with Knowledge Representation/Analysis
Since the current classification schemes organize CTA methods by process rather
than the desired outcome or application, practitioners find it difficult to select an optimal
method for their specific purpose. Such taxonomies/typologies may make it difficult for
analysts to choose an appropriate CTA approach, especially when the desired result is a
particular type of knowledge (Yates, 2007). Yates (2007) identified the most frequently
used CTA methods and the knowledge types associated with the respective methods and
outcomes, or a product approach versus a more traditional process approach.
Although data analysis and knowledge representation are considered as two
separate techniques of CTA, they are often linked with elicitation methods. Since both
techniques share common characteristics, data analysis and knowledge representation are
often combined into a single category in a classification scheme. Seeing CTA as a
pairing of knowledge elicitation with an analysis/representation technique may be more
COGNITIVE TASK ANALYSIS 59
effective. Yates (2007) found the most frequently used CTA method pairings include
standardized methods and informal methods. For efficiency and optimal use, CTA
methods need to be classified in terms of desired outcome rather than process. It was also
found that the application of these methods have been associated more with declarative
knowledge than procedural knowledge. The study also found that standardized methods
appear to provide greater consistency in the results obtained than in informal models.
Finally, analysis of interactions among applications, methods, and knowledge types may
be influenced by representation bias. CTA relies on the effective use of both elicitation
and analysis/representation methods to elicit expert knowledge.
Effectiveness of CTA
Cognitive Task Analysis has proven to be an effective method for capturing the
explicit observable behaviors, as well as the implicit, unobservable knowledge of experts.
CTA addresses the issues of research that look at the interactions between people,
technology, and task completion in education and work settings (Crandall, Klein, &
Hoffman, 2006). Data captured from CTA supports effective, efficient training and
instructional activities in complex systems (Hoffman & Militello, 2009). CTA is also
useful to educators to identify the skills, perceptual differences, and procedures that
might be left out of instruction (Crandall, et al., 2006).
Asking experts to list steps or to make observations does not accurately account
for their abstract knowledge. One reason that CTA is an optimal method for capturing
knowledge includes an emphasis on the aspects of the task that are important to the
learner (Crandall, et al., 2006). CTA also assists in the scalability of understanding
abstract knowledge across domains, and provides a framework for problem solving and
COGNITIVE TASK ANALYSIS 60
general principles of knowledge (Means & Gott, 1988). Compared to other strategies,
Cognitive Task Analysis is more effective at capturing the unconscious, complex
cognitive action and decision steps of experts.
Efficiency of CTA
Research has shown instruction using CTA is more cost effective and efficient
than other elicitation models. CTA informed instruction has been found to have stronger
results than regular instruction. As compared to behavior task analysis, CTA-informed
instruction can decrease days spent in training by almost 50% (Clark, et al., 2008; Clark
& Estes, 2006) Studies looking at knowledge assessments found that test takers using
CTA-informed instruction experienced nearly a 50% reduction in time spent in trying to
find a solution to a problem using their new-found knowledge (Schaafstal, et al., 2000).
Flynn (2012) looked at a 3i+3r independent CTA method compared to a 1i+3r
incremental CTA method. The research found the incremental 1i+3r CTA captured more
decision steps than the independent 3i+3r CTA method. Flynn also found the incremental
method took 67% less time and was 70% less expensive to conduct. The use of CTA in
instruction and training has been proven to be positively related to cost savings due to
reduced training times with comparable learning outcomes
Benefits of CTA for Instruction
Studies that have applied Cognitive Task Analysis to capture knowledge and to
create instruction delivery models have uncovered several benefits and useful design
strategies as compared to other forms of instruction. CTA has captured the explicit and
implicit knowledge used by experts for training and for computer systems (Hoffman &
Militello, 2009; Crandall, et al., 2006). CTA has captured expert knowledge, such as
COGNITIVE TASK ANALYSIS 61
critical decision points, judgments, and patterns, which were essential to the training of
firefighters (Crandall, et al., 2006). CTA has identified the precise protocol needed for
troubleshooting within a specific domain (Means & Gott, 1988). CTA elicited the
principles of troubleshooting which could be global learning objectives across many
domains. However, the problems used here as case-examples should be considered within
the scope of the specific domain or context being studied. CTA successfully enables a
more structured and guided instruction as compared to alternative instructional strategies
(Clark, et al., 2010). CTA has proven to be an effective method for eliciting the nuances
in expert knowledge, such as decision points and perspectives, resulting in a variety of
instructional strategies utilizing the outcomes of CTA (Means & Gott, 1988; Crandall et
al., 2006; Hoffman & Militello, 2009).
Studies across a variety of domains have explored the degree to which CTA-
informed instruction has influenced learning outcomes. CTA results in a nearly 30-45%
learning performance increase as compared to instruction that is informed by traditional
observation or BTA. There is evidence that CTA-informed instruction is advantageous
for increasing learning and reducing the number of mistakes made by recently graduated
healthcare students (Clark, 2014). CTA has been shown to be useful in understanding
communication and decision making among physicians. By understanding physician
communication and decision making processes, there is a possibility of preventing harm
to patients (Fackler, et al., 2009). Tofel-Grehl & Feldon (2013) conducted a meta-
analysis of CTA methods and concluded that, despite CTA’s higher front-end
implementation costs, the results from CTA-based training are highly effective compared
to non-CTA based training. They also concluded that the PARI method yielded the
COGNITIVE TASK ANALYSIS 62
largest effects. Thus, CTA has been shown to be effective in capturing expertise and
informing instruction in a wide range of domains, including software development,
military (Tofel-Grehl & Feldon, 2013; Fackler et al., 2009; Flynn, 2010), business sector
(Klein et al., 1989), and medical fields (Clark, 2014).
Comparison of 3i+3r individual and 1i+3r incremental CTA methods.
One of the negative aspects of CTA is the length of time and amount of money
needed in the front end to perform an analysis, and to develop training materials or
software from the knowledge collected. What if the time spent in elicitation could be
reduced without a reduction in the quality or quantity of expert declarative and
procedural knowledge? Flynn (2010) sought to answer this question by completing a
comparative study looking at two versions of a (CTA) system as applied in interviewing
recruitment officers for the US Army. Flynn (2010) described the first version of CTA
used in her study as the 3i+3r individual method and the second version from her study
was described as 1i+3r incremental method. A description of both methods will be given
in Chapter Three. Using these versions of CTA, Flynn (2010) concluded that the 1i+3r
incremental method was better able to capture more decision steps than was the 3i+3r
individual method and at less cost and less time spent in analysis. Zepeda-McZeal (2014)
sought to replicate Flynn’s (2010) analysis of decision step elicitation by analyzing
teaching informational reading comprehension to intermediate grade special education
students. Zepeda-McZeal (2014) also varied her study by investigating which method
was most effective in finding both action and decision steps. Zepeda-McZeal (2014)
concluded that the 3i+3r individual method was more effective at gathering the greatest
amount of action and decision steps. This study and the concurrent study (Lim, 2015)
COGNITIVE TASK ANALYSIS 63
seek to replicate Zepeda-McZeal’s (2014) study to find which method, 3i+3r individual
or 1i+3r incremental, is most effective in gathering the most action and decision steps
from SMEs teaching expository writing.
Summary
CTA is a method for capturing expert knowledge by using interview and
observation techniques to elicit a description of the implicit and explicit knowledge used
by experts to perform complex tasks. When asked to describe how to perform domain-
specific processes or procedures, they unintentionally omit up to 70% of critical
information needed by novices to perform these tasks successfully. CTA has been shown
to be an effective knowledge elicitation method in spite of its costs. The purpose of this
study was to conduct a CTA to capture expert knowledge and skills in the area of
expository writing instruction for eleventh grade students. CTA will be used to capture
the action and decision steps of subject matter experts to develop a Gold Standard
Protocol that can be used to inform instruction for novice teachers. Additionally, this
study compares CTA methods with a concurrent study (Lim, 2015) by looking at the
difference in action and decision steps captured using an individual (3i + 3r) CTA method
and an incremental (1i + 3r) CTA method.
COGNITIVE TASK ANALYSIS 64
CHAPTER THREE: METHODOLOGY
The Current Study
The purpose of this study was to capture the expertise of high school teachers as
they describe how they provide expository writing instruction to eleventh-grade students.
This study used Cognitive Task Analysis methods (CTA; Clark, Feldon, van Merriënboer,
Yates, & Early, 2008) as a knowledge elicitation method to capture the knowledge,
decisions and skills of expert writing teachers in teaching expository essay writing.
Studies have shown that expertise, valuable in informing instruction for novice teachers
and students alike, becomes automated through practice and experience over time such
that experts may omit up to 70% of critical information when asked to recall their actions
and decision steps in performing a task (Feldon, 2007b; Clark, 1999). This study also
examined the efficiency of two varying methods of CTA to capture the same task, the
3i+3r individual CTA method and 1i+3r incremental CTA method (Lim, 2015). This
chapter discusses the research methodology employed in this descriptive study on CTA
and presents details on the study design, participant selection, task instrumentation, data
collection, and data analysis.
The research questions that guided this study are:
1. What are the action and decision steps that expert teachers recall when they
describe how they provide expository writing instruction to their eleventh-
grade students?
2. What percentage of action and/or decision steps, when compared to a gold
standard do expert teachers omit when they describe how they provide
expository writing instruction to their eleventh-grade students?
COGNITIVE TASK ANALYSIS 65
3. Which method of CTA, 3i+3r individual or 1i+3r incremental (Lim, 2015), is
more efficient represented by the number of actions and decisions steps and
represented by cost and time?
Participants
Eight expert English-Language Arts teachers within a Southern California school
district were selected randomly for this study or the concurrent CTA study using the 1i+
3r incremental CTA method (Lim, 2015). Based on previous research indicating the need
for interviews with at least three experts (Clark, et al., 2008; Bartholio, 2010; Chao &
Salvendy, 1994; Crispen, 2010), three of the eight experts were interviewed using the
3i+3r individual CTA method in this study, while the three other random SMEs were
interviewed using the 1i+3r incremental CTA method (Lim, 2015). Each study used one
of the remaining two SMEs to verify the protocols created.
These expert teachers were selected for both studies based on the following
criteria. These teachers must be at the top of their profession and have a minimum of five
years and preferably ten years of consistent and exceptionally successful on-the-job
experience (Ericsson, Krampe, & Tesch-Romer, 1993). This determination of success is
based on reliable, industry standard outcomes that have been or can be validated, and not
merely based on “time on the job” (Ericsson & Charness, 1994). These subject matter
experts (SMEs) were selected based on levels of education (Master’s degree or higher),
their professional development (PD) and training in expository writing instruction (i.e.,
ERWC training), peer recognition as having achieved expertise in expository writing
instruction, and experience in the widest possible variety of settings, problems, specialties
and applications that characterize the range of contexts that students may face when they
COGNITIVE TASK ANALYSIS 66
graduate. Finally, these subject matter experts were selected based on not having
provided teacher instruction to others on the performance of this task within the past year
or more. Yates (2007) suggests that trainers or instructors often describe how they would
train others in task completion rather than how they actually perform the task on the job
or in the classroom.
The Assistant Superintendent of the school district wherein the expert teachers
worked was contacted and provided a brief description of the research study, including a
list of potential SMEs. This Assistant Superintendent approved both the study and the list
of potential SMEs. Further permission was obtained from a fellow Assistant
Superintendent whose duties include approving outside research studies taking place
within the district, from the school board, and from the principals of the school sites
where the SMEs worked. Each prospective SME was then contacted through email
detailing the purpose of the study, study requirements, and an invitation to participate.
The selection of SMEs to either the 3i+3r individual or 1i+3r incremental method
(Lim, 2015) of CTA was made randomly. More specifically, randomness of SMEs
assigned was based on their availability for interviews and the researchers’ timetables.
For example, the data collected from the initial in-depth interview with SME 1 was used
for the 1i+3r incremental method so that the researcher (Lim, 2015) could immediately
begin developing the CTA protocol for the subsequent SME interview. SME 2, SME 3,
and SME 4 were then selected to inform the present study using the 3i+3r individual
method as a result of those SMEs becoming available for interviews shortly after the
interview with SME 1.
COGNITIVE TASK ANALYSIS 67
Table 1
SMEs interviewed for both the Current Study and the Concurrent Study (Lim, 2015)
SME
Years of
Experience
teaching
expository
writing
Level of
education
attained
Eleventh-
grade courses
taught at
varying levels
of difficulty
Hours of
Professional
Development
in Expository
Writing
Was
recommended
by way of the
district, peers, or
both
1
*
17
M.A. in
English
Literature
2
75
YES
2
tt
8
M.F.A in
Creative
Writing
1
40
YES
3
tt
7
M.A in
English
2
20+
YES
4
tt
13
M.A. in
English/Ph.D
Candidate
2
50
YES
5
*
11
M.A. in
English
Literature
4
50
YES
6
*
18
M.A. in
English
Literature
3
80
YES
7
*
20
M.A. in
English
0
**
50
YES
8
tt
20
M.A. in
English
Literature
3
75
YES
Note: All data is de-identified. Each SME is numbered for demonstration purposes only and numbering
does not represent any rank order or selection criteria. SMEs with an asterisk (*) denote the 1i+3r
incremental method, and SMEs with the double cross (tt) denote the 3i+3r individual method of CTA. The
double asterisk (
**
) indicates that SME 7 has taught 20 total years of English, including the twelfth-grade
level and the post-secondary level, but not at the eleventh-grade level.
In other words, the decision to not use SME 2, SME 3, and SME 4 to inform the
1i+3r incremental method (Lim, 2015) was based on the knowledge that the researcher
would need enough time to develop the CTA protocol and confirm it prior to the
interview with SME 2 to review the protocol.
COGNITIVE TASK ANALYSIS 68
Research has shown that conducting CTA with an excess of 4 SMEs reaches a
point of marginal diminishing returns, which occurs when the knowledge acquired from a
SME yields less than 10% in additional knowledge steps (Bartholio, 2010; Chao &
Salvendy, 1994). Research indicates that three to four SMEs are optimal for knowledge
elicitation (Bartholio, 2010; Chao & Salvendy, 1994; Crispen, 2010). Therefore, to
remain consistent with research on the optimal number of experts for knowledge
elicitation, three SMEs were randomly assigned to the 3i+3r independent method, along
with verification from a fourth SME, and three were randomly assigned to the1i+3r
incremental method, along with verification from a fourth SME.
Data Collection for Question 1: What are the action and decision steps that
expert teachers recall when they describe how they provide expository writing instruction
to their eleventh-grade students?
Procedure for data collection. The CTA procedure will follow the five steps of
knowledge elicitation suggested by Clark, et al. (2008), as follows:
1. Collect preliminary information that builds general familiarity with study topic
using document analysis, observation, and informal interviews.
2. Identifying knowledge types used by subject matter experts when performing
the task which requires the researcher to identify declarative and procedural
knowledge and possible hierarchal relationships in the application of identified
knowledge types.
3. Applying the knowledge elicitation methods best suited for the study at hand.
4. Verifying and analyzing the collected data through use of qualitative data
analysis techniques.
COGNITIVE TASK ANALYSIS 69
5. Organize results into a training tool (i.e., professional development or job aid).
Phase 1: Collect preliminary knowledge. This researcher is a public school
teacher at the elementary level and has extensive experience teaching writing at that level.
Preliminary knowledge and explanation was gathered through a review of literature on
the subject was conducted to gather preliminary information on the subject at hand and to
develop familiarity with expository writing instruction. Additionally, information was
gathered throughout the study from a research partner who has several years experience
teaching expository writing at the secondary level.
Phase 2: Identify knowledge types. While reviewing the literature, a thorough
understanding of the characteristics and features of declarative and procedural knowledge
was developed. To help in distinguishing between knowledge types, the researcher
engaged in exercises with fellow researchers under the guidance of a senior researcher to
recognize action and decision steps along with content knowledge types such as concepts,
processes, and principles (Merrill, 1983). These knowledge classifications were used in
the protocol created from data collection.
Phase 3: Identify knowledge elicitation techniques.
Instrumentation. Many different methods exist for conducting a CTA, but only
six are evidenced based and can predict knowledge outcomes when followed correctly
(Yates and Feldon, 20011). Of the six, two CTA methods helped guide the methods
chosen for the present study (Clark, et al., 2008). The first is a semi-structured interview
format that gathers concepts, processes, and principles (CPP). CPP using a multi-stage
interview technique that elicits expert automated and conscious knowledge. The SME is
asked to list the sequence of steps to complete the task and to describe problems an expert
COGNITIVE TASK ANALYSIS 70
should be able to solve after mastery of the task. The list of tasks and subtasks created
become the outline for training development.
Figure 1. 3i+3r Individual Method
Subject matter experts (SME) are
identified and invited for in-depth
semi-structured interviews
SME A
In-depth
Interview
SME B
In-depth
Interview
SME C
In-depth
Interview
Individual
protocol created
Individual
protocol created
Individual
protocol created
2
nd
Interview to
review protocol
2
nd
Interview to
review protocol
2
nd
Interview to
review protocol
3 Individual Protocols compiled into
Preliminary Gold Standard Protocol
Preliminary Gold Standard Protocol
Reviewed by SME D
Gold Standard Protocol
COGNITIVE TASK ANALYSIS 71
The second CTA method that informed this study is the Concepts, Processes, and
Principles (CPP) technique (Clark 2006). CPP is based on the Precursor, Action, Result,
Interpretation (PARI) method, which emphasizes “adaptive expertise” or the ability to
solve novel problems not addressed in formal training programs (Clark and Estes, 1996;
Shraagen, Chipman, and Shalin, 2000). PARI looks for procedural knowledge, as well as
declarative knowledge, used to solve novel problems that occur in the future. Clark’s
(2006) CPP uses PARI steps, but also includes an instructional design component. CPP
suggests holding interviews with at least 3 experts and has them describe the same task.
Interviews are followed by experts engaging in self- and peer-review. Clark (2008) found
that having 3 experts describe the same procedure results in similar strategies reported,
but with varying decisions and analysis strategies the other experts might have missed. In
this study, a semi-structured interview was used to capture the knowledge and skills from
the SMEs using the concepts, processes and principles (CPPs) technique. The semi-
structured interview protocol is attached as Appendix A.
In the present study, semi-structured interviews took place using the 3i+3r
individual method as shown in Figure 1. This study performed an in-depth semi-
structured interview for each of three subject matter experts (3i). From each interview,
the knowledge and skills captured were written into a preliminary protocol outlining the
steps and knowledge the expert uses to teach expository writing. Three preliminary
protocols were created from each interview. Then, the protocol representing the
knowledge and skills of each subject matter expert was presented in a second follow-up
interview with each expert (3r) to provide an opportunity to make corrections,
clarifications, or additions to their protocol. Once this process was completed, the
COGNITIVE TASK ANALYSIS 72
researcher compiled all three protocols into a preliminary gold standard protocol (PGSP).
This PGSP was then presented to a fourth expert, SME D, who verified the elicited
information and suggested missing steps or knowledge if needed. This interview is not as
in-depth as the first three subject matter expert interviews. Changes suggested by SME D
were included into the final gold standard protocol (GSP), which can be considered as a
gold standard of expository writing instruction and can be used to design instruction and
instructional materials (Clark, et al., 2008).
The GSP is developed using the action and decision steps considered as critical
information novices need to perform a complex task. Action steps should begin with a
verb and are statements about what a person should do such as, “Insert car keys into the
ignition and turn to start the car.” Decision steps should contain two or more alternatives
to consider before taking an action, such as “IF the traffic light is yellow and you cannot
maintain current speed through the intersection before the light turns red,
THEN proceed to stop the car; IF the traffic light turns yellow and you can maintain
current speed through the intersection before the light turns red, THEN proceed with
caution.”
As a comparison of the methods and their results, in the concurrent study, Lim
(2015) followed the incremental CTA method by initially completing one in-depth
interview (1i) followed by reviews of the CTA protocol with two more subject matter
experts (3i) randomly chosen and different than the experts used in the current study.
Once the CTA protocol was generated using the data from the initial in-depth interview
with SME A, a follow-up interview took place for SME A to review the protocol with the
researcher in order to provide SME A with the opportunity to make corrections and/or
COGNITIVE TASK ANALYSIS 73
additions. SME B was then asked to review SME A’s protocol and make any possible
corrections and/or improvements. This process of review was repeated with SME C in
order to achieve a Preliminary Gold Standard Protocol (PGSP) of the task. As a final step,
the PGSP was taken to SME D for one final review at which point the Gold Standard
Protocol (GSP) was achieved.
Interviews. Following Institutional Review Board approval, a total of eight
subject matter experts were asked to participate in semi-structured interviews that
followed the methods described above. Because one of the goals of conducting the two
studies concurrently is to determine whether the incremental (1i+3r) approach provides as
much, if not more, useful decision information from the SMEs as the individual (3i+3r)
method in less time and for less cost, both researchers were present for all eight
interviews with the SMEs. This step was included to make certain both researchers,
novices at using CTA, could ensure consistency in the questions asked of each SME and
to keep conformity to the interview method. Completing the interviews together also
contributed to “bootstrapping”, or the time taken to educate the researchers about basic
knowledge of the domain being studied (Hoffman, et al., 1998, Clark, 2008). The
duration of each SME interview varied but averaged approximately 90 minutes. With the
subject matter expert’s approval, each interview was audio recorded.
Phase 4: Data Analysis. Each recorded interview was transcribed. The advantage
of recording and transcribing the SMEs’ interviews is that it enables deep analysis of the
data captured through elicitation of the skills, decisions, and steps provided by each SME.
COGNITIVE TASK ANALYSIS 74
Coding. All of the interviews were transcribed and coded using a coding scheme
developed based on Clark’s (2006) method. It was used to code the data gathered from
the semi-structured interviews. The coding scheme used is included in Appendix B.
Inter-rater reliability (IRR). The first transcript collected was coded by the
researcher and another fellow researcher. The results were discussed and analyzed closely
by both researchers to determine inter-rater reliability. Hoffman (1998) and colleagues
suggest that once there is an 85% or higher IRR agreement, the coding process is
consistent and reliable among different coders. However, if the IRR is less than 85%,
Crandall and colleagues (2006) recommend the coding scheme and function-unit
categories may need further refinement. IRR results are presented in Chapter Four.
SME protocol and verification. After each interview was coded and analyzed, a
protocol was generated. This protocol was formatted as a list of action and decision steps,
including the standards, objectives, reasons, and other information elicited during each
interview. This type of protocol was generated for each SME, who was asked to review it
for any modifications, if necessary. The goal for each SME was to verify the information
as a correct reflection of the knowledge, steps, and decisions of this particular SME
(Clark, et al., 2008).
Phase 5: Formatting the results.
Gold standard protocol (GSP). Once each subject matter expert reviewed and
corrected their individual protocol, all three individual protocols were compiled into one
formatted description of the action and decision steps needed to teach expository writing.
This compilation, known as a Preliminary Gold Standard Protocol (PGSP, Clark, 2014)
contains the knowledge, skills, and steps elicited from each SME and formatted as an
COGNITIVE TASK ANALYSIS 75
instructional guide. The PGSP was then submitted to a fourth SME to verify the contents
and add any missing or incorrect steps. Any final changes suggested by SME D were
included and the result was a Gold Standard Protocol (GSP). Both studies relied on the
job aid format created by Clark and Yates (2010) to develop the GSP, which has been
attached as Appendix C
Data Collection for Question 2: What percentage of action and/or decision steps,
when compared to a gold standard do expert teachers omit when they describe how they
provide expository writing instruction to their eleventh-grade students?
Spreadsheet Analysis. The final stage of data analysis was completed by
transferring the action and decision steps of the final GSP to a spreadsheet. Each
individual SME protocol was reviewed and compared to the GSP. If the individual SME
protocol included the action and decision step then a “1” was placed in the appropriate
cell for that SME. If there was no agreement, or the SME missed this action or decision
step, a “0” was recorded. A frequency count was conducted and the total number of
agreements and omissions between the individual SMEs and the GSP was calculated.
Data Collection for Question 3: Which method of CTA, 3i+3r individual or
1i+3r incremental (Lim, 2015), is more efficient represented by the number of actions
and decisions steps and represented by cost and time?
The following criteria were used to determine which method of CTA is more
efficient. First, the number of action and decision steps captured in the final 3i+3r
independent method in this study were compared to the results of Lim’s (2015) 1i+3r
incremental method to determine which knowledge elicitation method yielded a greater
number of critical action and decision steps. Furthermore, the total amount of time spent
COGNITIVE TASK ANALYSIS 76
to complete the 1i+3r incremental method as opposed to the total amount of time and cost
to do the 3i+3r individual method was calculated. Specifically, the total time was
calculated by adding the amount of time spent by the knowledge analyst to conduct the
CTA interviews with each SME. The total cost was calculated by adding up the
individual costs of transcribing each CTA interview.
COGNITIVE TASK ANALYSIS 77
CHAPTER FOUR: RESULTS
Overview of the Results
This study examines the declarative and procedural knowledge represented as
action steps, decision steps, objectives, standards, and cues of expert expository writing
teachers using an Individual (3i+3r) CTA method to capture this expertise. The results are
organized by research question.
Research Questions
Question 1
What are the action and decision steps that expert teachers recall when they
describe how they provide expository writing instruction to their eleventh-grade
students?
Inter-rater reliability (IRR). IRR was completed by the researcher and a
colleague to safeguard against researcher bias and to ensure consistency. IRR was
measured by counting the number of coded items in agreement between both raters and
dividing the result by the total number of coded items. The transcript from the first SME
interview, labeled SME A, was coded using a shared coding scheme by the researcher
and colleague to identify the objectives, standards, conditions, reasons, and action and
decision steps. The results are shown in Appendix B. The Inter-rater reliability was
established at 98%. Based on this result, the remaining interviews from SME B and SME
C were coded by the researcher and the data collected was used to create an initial
protocol for each subject matter expert.
Flowchart analysis. A flowchart was created from the protocol developed using
data captured from SME A and is attached as Appendix D. The flowchart was analyzed
and reviewed to ensure all action and decision steps flowed in a logical manner.
COGNITIVE TASK ANALYSIS 78
Additionally, decision steps were reviewed to ensure they had appropriate potential
actions. The flowchart revealed some steps at the conclusion of the process had been
neglected. Further analysis provided the actions steps needed to conclude the logical
progression of the flowchart.
Gold standard protocol. All three preliminary protocols were analyzed and
aggregated into a preliminary gold standard protocol (PGSP) for teaching expository
writing to 11
th
grade students using the job aid found in Appendix E. After reviewing
each protocol, it was determined that the protocol from SME A would be the
foundational protocol upon which the gold standard protocol would be created. SME B’s
protocol was then taken and aggregated with SME A’s protocol as each action and
decision step found in SME B’s protocol was compared to SME A’s protocol. If the steps
were identical in meaning, SME A’s step would remain. However, both SME’s were
given credit for the action or decision step. The step would be coded with the “A”
designation. For instance, SME B’s designation is “B1R”, meaning the line came from
SME B (B) and from the 1
st
round interview (1R) held with that SME. If SME B had a
step that did not match SME A, the step was added into the protocol and coded with
credit given to SME B. Once this process had been completed for SME B, the process
began again aggregating SME C’s protocol. If SME C’s action and decision step matched
those of SME A or SME B, they were aggregated, coded, and credit given to SME C. If
the SME C’s protocol contained a step not found in the protocols of SME A and SME B,
the step was added, to the protocol, coded, and credit given to SME C. Table 2 provides
an example of this process.
COGNITIVE TASK ANALYSIS 79
Table 2
Example of Process toward creating main procedures in Preliminary Gold Standard
Protocol (PGSP)
SME A SME B SME C
1. Assess students’
prior knowledge and
plan for student
instruction.
2. Complete Literature
Unit/ Demonstrate
Thinking Process
3. Present topic as a
problem
4. Provide Feedback
and Examples to
students
5. Conduct a Writer’s
Workshop
6. Assess the papers
7. Guide the revision
of the papers.
1. Prepare Unit Plan
2. Assess students’ prior
knowledge and plan for
student instruction.
3. Begin Literature Unit
4. Present topic as a
problem
5. Introduce Prompt/
Prompt Analysis
6. Provide Feedback and
Examples to students
7. Construct Essay Outline
8. Writing Instruction
9. Conduct a Writer’s
Workshop
10. Peer Review and
Revision
1. Prepare to teach.
2. Assess the student’s
prior writing
knowledge
3. Implement
Literature Unit
4. Present topic as a
problem.
5. Introduce Prompt/
Prompt Analysis
6. Provide Feedback
from Diagnostic
Papers
7. Construct essay
outline
8. Provide Writing
Instruction.
9. Conduct Writing
Workshop
10. Assess papers.
11. Hand back papers
and provide
optional
opportunities for
further revision.
12. Take opportunities
to publish student
writing.
Note: The bolded portions of text indicate a new step. A column for SME D was not added because nothing
was added or changed to the main procedures during the interview with SME D.
Once the PGSP was created from the aggregation of all three preliminary
protocols, it was sent to SME D for verification. SME D was asked to review the PGSP
for suggestions of any possible additions, modifications, or deletions to the PGSP, a task
different than that asked of SMEs A, B, and C.
COGNITIVE TASK ANALYSIS 80
The final result and answer to Research Question One is the gold standard
protocol (GSP) found in Appendix E, which has captured the expertise of 4 SMEs and is
the actions and decisions steps expert teachers use to teach expository writing. The GSP
found there are 12 procedures for teaching expository writing. They are as follows:
1. Prepare to teach.
2. Assess the student’s prior writing knowledge.
3. Implement Literature Unit.
4. Present topic as a problem.
5. Introduce Prompt/ Prompt Analysis.
6. Provide Feedback from Diagnostic Papers.
7. Construct essay outline.
8. Provide Writing Instruction.
9. Conduct Writing Workshop.
10. Assess papers.
11. Hand back papers and provide optional opportunities for further revision.
12. Take opportunities to publish student writing.
The gold standard protocol (GSP) created from the compilation of each individual
SME’s preliminary protocol contains the actions and decision steps these SMEs use in
teaching expository writing to their students. Along with the steps described, the GSP
also contains reasons these experts do what they do and also standards that describe what
this action or decision step seeks to do in practice. A novice teacher can follow this
protocol and, with small variation as needed, be able to teach expository writing to their
students. This protocol has applicability in not only 11
th
grade but also in other grade
levels on a high school campus. If applied in other grades, there might need to be some
additional changes implemented to meet grade level and district level requirements.
COGNITIVE TASK ANALYSIS 81
The GSP contains the actions and decisions of these expert teachers in a step-by-
step format. Because of the knowledge submissions of the experts, there are at time
various paths for novice teachers to follow based on what they are attempting to teach in
to their students. For instance, step 9.6 begins a section on various ways to handle peer
review of papers in class. A novice teacher can choose which path fits her needs and
follow the steps in 9.6, 9.7, or in 9.8. By determining the objectives a novice teacher
would like to achieve, the GSP can help this teacher find step-by-step instructions on how
to proceed to having her students complete an expository essay.
Recalled action and decision steps. A spreadsheet was used to compile and
organize data on elicited action and decision steps represented by the final Gold Standard.
The spreadsheet can be found in Appendix F. In the first column, each step of the gold
standard protocol was coded with “A” for action steps and “D” for decision steps. There
are some protocol entries such as standards, reasons, and objectives. These are not steps
and were left out of the spreadsheet. In 4 columns following the actions or decision steps,
data was entered reflecting which SME was responsible for each step. The columns were
labeled “SME A”, “SME B”, “SME C”, and “SME D” for Round 1 interviews. If a step
is attributed to one or more SMEs, the cell is marked with a “1”. If the step is not
attributed to a SME, then a “0” was marked. As an example, step 1.3.1 (line 7) has a “1”
attributed to all three SMEs. However, step 1.3.2 was attributed to SME B only.
Consequently, a “1” is placed in the column for SME B and a “0” is placed in the other 3
columns. The total number of actions and decision steps were tabulated and are noted in
the spreadsheet. Table 3 provides a total of the action steps, decision steps, and both
action and decision steps.
COGNITIVE TASK ANALYSIS 82
Table 3
Cumulative Action and Decision Steps Captured for Each SME using the Individual
(3i+3r) CTA method.
Action Steps
Decision Steps
Total Action and
Decision Steps
SME A
100
28
128
SME B
132
18
150
SME C
123
24
147
SME D
278
52
330
Note: A fourth row was included for this table because SME D provided new action and decision steps
during the interview.
Action and decision steps contributed by each SME. Table 3 reports action and
decision steps reported by each SME. The steps reported in Table 3 are not solely
attributable to one SME only. The actions and decisions steps reported from each SME,
when added together, result in amounts larger than the amount of action and decisions
steps listed in the gold standard protocol. In many cases, multiple SMEs reported the
same action or decision step throughout the CTA process. The SMEs reported a total of
333 action and decision steps. All three experts recalled less than the total number of
action and decision steps elicited with CTA as seen in Figure 2.
The percentages of action and decision steps reported by SMEs A, B, and C as
compared to the total action and decisions steps contained in the gold standard protocol
was between38.44% and 45.05%. None of the SMEs reported more decision steps than
action steps. SME B and SME C reports of action and decision steps were relatively
comparable with a difference of 3.21% in action steps and 11.53% in decision steps.
COGNITIVE TASK ANALYSIS 83
SME A had less action steps reported (5.7 %) than the next SME but reported more
decision steps (7.7%) than the SME B and SME C. The range of percentages of actions
steps is 6.61% and the range of decision steps is 19.23% for SMEs A, B, and C.
Figure 2. Number of Decision Steps, Action Steps, and Action and Decision Steps for
SME A, SME B, SME C, and SME D captured through 3i+3r Individual CTA. Total non-
repeating decision and action steps from CTA process represented in the gold standard
protocol: decisions steps- 52; action steps-281; action and decision steps-333.
SME D was asked to verify the steps contained in the protocol and to add missing
steps if needed. Thus, the amount of action and decisions steps is much more than for the
other SMEs. During verification, SME D agreed with the action and decision steps of the
other SMEs unless a change seemed in order. In these cases, SME D added an action step
or suggested a modification to an existing step. Because of this, SME D was attributed
with 278 action steps in agreement with the other SME’s. SME D was also attributed
with 52 decision steps for a total action and decision step amount of 330 total steps.
28
18
24
52
100
132
123
278
128
150
147
0
0
50
100
150
200
250
300
SME A SME B SME C SME D
Decision Steps
Action Steps
Total Steps
COGNITIVE TASK ANALYSIS 84
Action and decision steps contributed in round two interviews. Once
preliminary protocols had been completed for each SME, a second interview was held to
allow each SME review their data and add, subtract, or modify information. Further
analysis of this data is shown below in Table 4.
Table 4
Cumulative Action and Decision Steps Captured for Each SME in Round 2 Interviews.
Action Steps
Decision Steps
Total Action and
Decision Steps
SME A
0
0
0
SME B
6
0
6
SME C
2
0
2
Note: These actions and decision steps are included in the amount reported in Table 1.
As seen in Table 4, SME B added 6 more action steps to the protocol created after
the first interview. SME C was able to add only 2 more action steps. SME A did not add
any action steps and all three SMEs did not add any additional decision steps. All three
SME’s made minor clarifications to some of the steps contained in their protocols, but
very few additional action and decision steps were added or subtracted.
SME D was not included in analysis of the results of 2
nd
round interviews. SME D
was given the opportunity to verify the preliminary gold standard protocol rather than
participate in the in-depth interviews held with SMEs A, B, and C. This resulted in a
second interview was not being held for SME D.
Alignment of SMEs in describing the same action and decision steps. Analysis
was conducted to determine the number and percentage of action and decision steps
recalled by each SME that were aligned to one another. For each action and decision step
COGNITIVE TASK ANALYSIS 85
reported, each SME was given credit if they submitted a step or matched a step already
submitted. After all three protocols were aggregated as one, a number for each action and
decision step was placed in a spreadsheet column entitled “Alignment.” For each “1”
attributed to a step, the quantity was added and recorded in the column. If only one SME
added a particular step, then a “1” was added in the alignment column. If two SMEs had
provided matching steps, then a “2” was placed in the column. Finally, if all three SMEs
contributed this step, then a “3” was placed in the column. A score of “1” is considered
slightly aligned, a score of “2” is considered partially aligned, and a score of “3” is
complete alignment. Table 5 reports the amount of aligned action and decision steps.
Alignment results show in Table 5 that all four SME’s were completely aligned
on 26 action steps and 5 decision steps. When comparing action and decision steps that
were completely aligned, 83.87% of those steps completely aligned were action steps and
16.13% were decision steps. Three of the SMEs were highly aligned on 30 action steps
and on 8 decision steps, for percentages of 78.95% and 21.05% respectively. When
comparing steps that were partially aligned, only two of the SME’s were aligned on 214
(84.58%) action steps and 29 (15.42%) decision steps at the same time. Finally, eleven
action steps and no decision steps were reported by one SME during the interviews held.
When analyzing alignment of total action and decision steps, all four SMEs were
found to be completely aligned on 9.31% of total action and decision steps as shown in
Figure 3. To be highly aligned, three of the SMEs reported similar action and decision
steps 11.41% of the time during their interviews. At least two SME’s were aligned
75.98% of the time while only one SME reported an action or decision step alone that
none of the others reported 3.30% of the time.
COGNITIVE TASK ANALYSIS 86
Table 5
Number and Percentage of Action and Decision Steps that are Completely Aligned,
Highly Aligned, Partially Aligned, and No Alignment.
Action Steps
Decision Steps
Total Action and
Decision Steps
Complete
Alignment
26
83.37%
5
16.13%
31
9.31%
High
Alignment
30
78.95%
8
21.05%
38
11.41%
Partial
Alignment
Slight
Alignment
214
11
84.58%
100%
39
0
15.42%
0.00%
253
11
75.98%
3.30%
SME D was included in this analysis. Analysis for alignment looks to determine
what actions and decisions steps might be common across all experts interviewed. SMEs
A, B, and C were interviewed separately and their responses were unknown to each other.
Thus, the actions and decision steps reported should not have been influenced by
one another. SME D was given a copy of the preliminary gold standard protocol to
review and was asked to confirm the action and decision steps contained in the protocol
and if there were any changes needed to improve it. By confirming the action and
decision steps in the PGSP, SME D became aligned with the steps reported by the other
three SMEs. SME D did contribute 3 action steps the other SMEs did not contribute.
Consequently, any steps added by SME D must, by default, be a step none of the other
SMEs added during their interviews.
COGNITIVE TASK ANALYSIS 87
Figure 3. Action and Decision Steps that are Completely Aligned, Highly Aligned,
Partially Aligned, and No Alignment.
Question 2
What percentage of action and/or decision steps, when compared to a gold
standard do expert teachers omit when they describe how they provide expository writing
instruction to their eleventh-grade students?
Total knowledge omissions. Analysis was completed to determine the
percentage of action and decision steps omitted by each individual SME when describing
the knowledge and skills used to teach expository writing. When marking action and
decision steps in the spreadsheet, actions that were not contributed by one or more SMEs
were given a “0”. The total number of action and decision steps omitted was added and
divided by the total number of cumulative action and decision steps for all SME’s in the
gold standard protocol. This produced a percentage of knowledge omissions for action
steps, decision steps, and the total of both groups. Table 6 and Figure 4 show the
resulting data.
SME D was included in this data set. Although unable to participate in an in-
depth interview, SME D did contribute to the amount of decision and action steps
9.31%
11.41%
75.98%
3.30%
Complete
Alignment
High
Alignment
Partial
Alignment
Slight
Alignment
COGNITIVE TASK ANALYSIS 88
captured in the gold standard protocol. By agreeing with the action and decision steps
contained in the PGSP, SME D became aligned to those choices of the other three SMEs.
This reduced the amount of omissions by SME D to around 1% or less for action steps
and for total action and decision steps.
Table 6
Total Action and Decision Steps, or Expert Knowledge, Omissions by SME when
Compared to the Gold Standard Protocol
Steps Omitted
Action Steps
Decision Steps
Total Action and
Decision Steps
Omitted % Omitted % Omitted %
SME A
181
64.41
24
46.15
205
61.56
SME B
149
53.02
34
65.38
183
54.95
SME C
158
56.23
28
53.85
186
55.86
SME D
3
1.07
0
0.00
3
0.90
Mean
Omissions
122.75
43.68
21.5
41.35
144.25
43.32
Range
202
34
202
SD
74.96
30.45
77.12
Note: Total non-repeating decision and action steps from CTA process represented in the gold standard
protocol: decisions steps- 52; action steps-281; action and decision steps-333.
All four SMEs omitted an average of 144.25 (SD + 77.12) total action and decision steps,
a percentage of 43.32%, as shown in Table 6. The range among the first 3 SMEs was
only 6.61% or 22 steps. The amount of total steps omitted by the first 3 SMEs was
relatively equal. In analyzing actions steps, all four SMEs had omitted an average of
122.75 action steps (SD + 74.96), with SME B and SME C omitted close to the same
amount with a difference of only 9 action steps. SME A omitted 32 more steps than SME
COGNITIVE TASK ANALYSIS 89
B suggesting that SME B and SME C’s knowledge base or instructional design might be
different than SME A. In terms of decision steps, all four SMEs omitted an average of
21.5 (SD + 30.45) decision steps.
All three SMEs who participated in the in-depth interviews had a difference of 10
steps between the SME with the highest and the SME with the lowest numbers. With
percentages for the first 3 SME’s in the 50%-60% range, this confirms Chao and
Salvendy’s (1994) findings that experts omit up to 70% of recalled knowledge and skills.
Figure 4. Total SME Knowledge Omissions When Compared to the Gold Standard
Protocol. Total non-repeating decision and action steps from CTA process represented in
the gold standard protocol: decisions steps- 52; action steps-281; action and decision
steps-333.
24
34
28
0
181
149
158
3
205
183
186
3
0
50
100
150
200
250
SME A SME B SME C SME D
Omitted Decision Steps
Omitted Action Steps
Omitted Action and Decision
Steps
COGNITIVE TASK ANALYSIS 90
Question 3
Which method of CTA, 1i+3r incremental or the 3i+3r individual, is more
efficient at capturing the expertise of writing experts?
Question three sought to determine overall efficiency and effectiveness of the 3i +
3r individual CTA method compared to the 1i + 3r incremental CTA method (Lim, 2015).
Efficiency in this study is defined by determining which method described in Chapter
Three is more efficient in terms of time and costs to complete the analysis. The following
results present the following comparisons between both methods looking at time, cost,
and expert knowledge elicitation.
Total time. The total amount of time spent by each researcher on this study and
the concurrent study included conducting two rounds of CTA interviews with four SME.
This was done to determine which CTA method is more efficient. Table 7 lists this data
of the amount of hours and minutes spent by each researcher to conduct their particular
method of CTA.
Table 7
Comparison of Total Time spent doing the 1i+3r Incremental Method and the 3i+3r
Individual Method of CTA
Time Spent Conducting SME Interviews
1i+3r Incremental
09h 03min
3i+3r Individual
11h 46min
Difference
02h 43min
COGNITIVE TASK ANALYSIS 91
The results above show that the 1i+3r incremental method is more time efficient
than the 3i+3r individual method. While the interviews using the 1i+3r incremental
method took a total of 9 hours and 3 minutes, the interviews using the 3i+3r individual
method lasted a total of 11 hours and 46 minutes. The results indicate a difference of 2
hours and 43 minutes favoring the 1i+3r incremental method.
Total cost. The costs involved in performing these CTA studies were spent on
transcribing audio recordings of the interviews. Both studies used the same transcription
service to transcribe the 1
st
round interviews with their first three SMEs. The transcription
service charged $1 an minute for each recording. Table 8 lists the total costs as well as
the costs incurred by both research studies.
Table 8
Comparison of Total Cost doing the 1i+3r Incremental Method and the 3i+3r Individual
Method of CTA
First-Round
Interviews
1i+3r Incremental
3i+3r Individual
Difference
SME A/1
$133.00
$106.25
$26.75
SME B/2
$92.00
$205.00
$113.00
SME C/3
$89.00
$163.75
$74.75
Total Costs
$314.00
$475.00
$161.00
Note: The researcher of the 1i+3r CTA method (Lim, 2015) chose to be more thorough and decided to
transcribe and code interviews for SME B and C even though those steps are typically excluded during the
1i+3r incremental method.
As demonstrated, the 1i+3r incremental CTA method was more cost effective
than was the 3i+3r individual CTA method. The total costs to transcribe the audio
recordings for SMEs A, B, and C using the 3i + 3r CTA method was $475.00 while the
costs for the researcher doing the 1i +3r incremental approach spent $314.00 for a
COGNITIVE TASK ANALYSIS 92
difference of $161.00. The 1i + 3r incremental CTA study found its initial cost to
transcribe the audio recording for SME A was $133.00 whereas the researcher using the
3i + 3r method spent $106.25 on the transcript for SME A. However, the 1i+3r study
spent less on the next two interviews ($92.00 and $89.00) than did the researcher using
the 3i+3r individual approach ($205.00 and $163.75).
Knowledge elicitation. The total number of action and decision steps from the
1i+3r incremental CTA method and the 3i+3r individual CTA method were compared to
find out which knowledge elicitation method is more efficient. Table 9 provides a
comparison of total action and decision steps generated from the 1i+3r incremental and
the 3i+3r individual method GSPs.
Table 9
Comparison of Overall Action and Decision Steps from 3i+3r Independent Method and
1i+3r Incremental Method Gold Standard Protocols
Gold Standard
Protocol
Total Action and
Decision Steps
Action Steps
Decision Steps
1i+3r Incremental
210
159
51
3i+3r Independent
333
281
52
Difference
123
122
1
% Difference
45.30%
55.45%
1.94%
Comparing the two knowledge elicitation methods reveals that the 3i+3r
individual method captured a greater total of action and decision steps than the 1i+3r
incremental method as shown in Table 9. Subject matter experts recalled 210 non-
repeating action and decision steps with the 1i+3r incremental method and 333 non-
repeating action and decision steps with the 3i+3r individual method. The 3i+3r method
elicited 123 more action and decision steps with a difference of 45.30%. A closer look
COGNITIVE TASK ANALYSIS 93
reveals that the 3i+3r individual method captured more action steps with 281 steps,
compared to 159 action steps captured by the 1i+3r incremental method. The 3i+3r
individual method captured a total of 122 more action steps, a difference of 55.45%.
However, the number of decision steps was nearly identical between the 1i+3r
incremental and the 3i+3r individual methods. The 3i+3r individual method captured 52
decision steps, while the 1i+3r incremental method captured 51 decision steps. The 3i+3r
individual method captured only 1 more decision step for a difference of 1.94%.
Figure 5. Total Expert Knowledge Recall for the 1i+3r Incremental Method and 3i+3r
Individual Method
Expert contribution of action and decision steps. Figure 5 shows action and
decision steps recalled by each SME for the two knowledge elicitation methods.
No matter the method used, SMEs in both studies recalled more action steps than
decision steps. SMEs interviewed with the 3i+3r individual method recalled a total of 363
action steps and 70 decision steps for a grand total of 433 action and decision steps.
88
141
155
159
100
132
123
278
25
40
46
51
28 18 24
52
0
50
100
150
200
250
300
SME A SME B SME C SME D SME 1 SME 2 SME 3 SME 4
Total Steps Recalled
1i+3r Incremental Method 3i+3r Individual Method
Action Steps Decision Steps
COGNITIVE TASK ANALYSIS 94
Altogether, the SMEs reported a total of 83.83% action steps and 16.17% decision steps
under the 3i+3r individual method. SME 1 recalled a total of 128 steps, of which 100
were action steps (i.e. 78.13%) and 28 were decision steps (i.e. 21.87%), and was able to
recall 56.26% more action than decision steps. SME 2 recalled a total of 150 steps, of
which 132 were action steps (i.e. 88%) and 18 were decision steps (i.e.12%), and was
able to recall 76% more action than decision steps. SME 3 recalled a total of 147 steps, of
which 123 were action steps (i.e. 83.67%) and 24 were decision steps (i.e. 16.33%), and
was able to recall 67.34% more action than decision steps. While verifying the PGSP,
SME 4 added 11 new steps, all of which were action steps.
Likewise, SMEs who were interviewed using the 1i+3r incremental method (Lim,
2015) recalled a total of 543 action steps and 162 decision steps for a grand total of 705
action and decision steps. All of the SMEs interviewed using the 1i+3r incremental
method recalled approximately 3 times more action than decision steps with SMEs
recalling 77.02% of action steps and 22.98% of decision steps. After two rounds of 1i+3r
Incremental CTA (Lim, 2015) interviews, SME A recalled a total of 113 steps, 88 of
which were action steps (i.e. 77.88%) and 25 were decision steps (i.e. 22.12%), and
recalled 55.76% more action than decision steps. SME B not only agreed with the 113
steps of SME A’s protocol, but also added 68 new action and decision steps, for an
incremental contribution of 181 steps. More specifically, SME B contributed 141 action
steps (i.e. 77.90%) and 40 decision steps (i.e. 22.10%), which comes to a 55.80%
difference. After taking the protocol to SME C, she confirmed the 181 steps that SME A
and B contributed and was also able to add 20 new action and decision steps of her own
for a total incremental contribution of 201 steps. More specifically, SME C contributed
COGNITIVE TASK ANALYSIS 95
155 action steps (i.e. 77.11%) and 46 decision steps (i.e. 22.89%), which results in a
54.22% difference. The 1i+3r incremental PGSP (Lim, 2015) that had been created by
SMEs A, B, and C was then taken to a fourth expert, SME D, for verification. While
reviewing the PGSP, SME D not only verified the 201 action and decision steps SMEs A,
B, and C contributed, but also identified some omissions and added 9 new action and
decision steps, for an incremental total of 210 steps. More specifically, SME D was able
to contribute 159 action steps (i.e. 75.71%) and 51 decision steps (i.e. 24.29%), a
difference of 51.42%.
3i+3r individual method analysis. Analysis shows the 3i+3r individual method
captured a total of 281 non-repeating action steps and 52 non-repeating decision steps
from SMEs 1 through 4. Compared to the 3i+3r independent GSP, SME 1 recalled
35.59% of action steps and 53.85% of decision steps. SME 2 recalled 46.98% of action
steps and 34.62% of decision steps, and SME 3 recalled 43.77% of action steps and
46.15% of decision steps. SME 4’s additions contributed 2.85% of action steps.
1i+3r incremental method analysis. After further analysis of the 1i+3r
incremental method, a total of 159 non-repeating action steps and 51 non-repeating
decision steps from SMEs A through D were captured. SME A recalled 55.35% of action
steps and 49.02% of decision steps as compared to the 1i+3r incremental method GSP.
Including the steps in agreement with SME A’s protocol, SME B contributed 88.68% of
action steps and 78.43% of decision steps. Additionally, SME C contributed 97.48% of
action steps and 90.20% of decision steps including the steps in agreement with SME B’s
incremental protocol. SME D contributed 100% of action and decision steps.
COGNITIVE TASK ANALYSIS 96
Summary
The results of the data collected created a gold standard protocol of the action and
decision step to teach expository writing to eleventh grade students. In discussing the
3i+3r individual CTA method, all three SME’s involved in the semi-structured interviews
contributed considerably more action than decision steps. Most of these steps were
captured in the first round of interviews while very few action or decision steps were
captured in the second round of interviews. SME D verified the preliminary Gold
Standard Protocol, agreeing with the majority of action and decision steps while adding
only 8 action steps to the protocol as a whole. Expert omissions percentages for the first 3
SME’s fell in the 50%-60% range, confirming Chao and Salvendy’s (1994) findings that
experts omit up to 70% of recalled knowledge and skills. In comparing alignment
between SMEs, the protocol was partially aligned with two SMEs agreeing to
approximately 78% of the action and decision steps. There was only 11.41% of the action
and decision steps where three SMEs agreed and only 9.31% where all four SMEs agreed
on action and decision steps.
In comparing the 3i+3r individual method and the 1i+3r incremental method (Lim,
2015) for efficiency, the 1i+3r incremental method appears to use less time for interviews
and transcription. It was also less costly to implement than was the 3i+3r individual
method. In terms of effectiveness, the results were inconclusive. The 3i+3r individual
method elicited many more action steps than the 1i+3r incremental method, but only one
more decision step.
COGNITIVE TASK ANALYSIS 97
CHAPTER 5: DISCUSSION
Overview of Study
One of the purposes of this study was to capture the knowledge and skills of
expert eleventh-grade teachers of expository writing in the form of a gold standard
protocol containing the action and decision steps used by these experts when providing
expository writing instruction. Researchers have found that experts are able to perform
complex tasks within their domain of expertise with automaticity. But when these experts
share the knowledge and skills used to conduct complex tasks, they often omit up to 70%
of this critical information needed by novices to replicate these same tasks (Clark, 2014,
Clark, et al., 2008; Clark & Feldon, 2006).
To elicit this information, CTA was the method used because of its ability to
capture, analyze, and organize the explicit and implicit knowledge experts use to perform
complex tasks (Chipman, 2000; Clark, et al., 2008). CTA has been used across various
domains including medicine, computer software design, and the military (Clark, 2014).
The use of CTA in the domain of K-12 education is more recent and the amount of
research is sparse. Thus, the purpose of this study was to elicit the knowledge and skills
of expert expository writing instructors and thereby adding the existing research on the
use of CTA.
The second purpose of the study was to compare two methods of conducting CTA.
Two studies have attempted to answer the question regarding which method, the 3i+3r
individual method and the 1i+3r incremental (Lim, 2015) method is more effective. Flynn
(2012) determined that the 1i+3r incremental method elicited more decision steps in less
time with less cost than did the 3i+3r individual method. Zepeda-McZeal (2014) found
COGNITIVE TASK ANALYSIS 98
the 3i+3r individual method was more effective at capturing the total number of action
and decision steps than did the 1i+3r incremental method but did not address cost or time
savings. This study, along with a concurrent study (Lim, 2015), sought to replicate facets
of the previous studies (Flynn, 2012; Zepeda-McZeal, 2014) with the hope of
determining which of these two methods is more efficient in terms of knowledge
elicitation, cost, and time savings.
A desired outcome of CTA is to capture expert knowledge and skills so they can
be used in instruction design and professional development (PD) training (Tjiam, Schout,
Hendrix, Scherpbier,Witjes, & Van Merriënboer, 2012). Means and Gott (1988) found in
their study that five years of work experience can be condensed into 50 hours of CTA-
based training. Clark (2014) reports that when CTA methods are applied to training,
students learn about 30% more than with other task analysis techniques. When PARI-
type CTA methods are used, gains in learning increase to 45%. This study, then, seeks to
inform future professional development (PD) and pre-service teacher preparation
programs in developing effective training for teachers of expository writing.
The remaining sections discuss the process of conducting CTA with expert
teachers in the context of prior CTA studies, followed by results of the study, study
limitations, implications, and areas for future research.
Process of Conducting Cognitive Task Analysis
Selection of Experts
Chao and Salvendy (1994) determined that knowledge acquisition is best
optimized in terms of cost-benefit when three experts are used. Bartholio (2010) and
Crispen (2010) sought replicate the aforementioned study above and found that three to
COGNITIVE TASK ANALYSIS 99
four experts were optimal in capturing the knowledge necessary for task completion. In
this study and the concurrent study (Lim, 2015), eight expert English-Language Arts high
school teachers from within a school district in Southern California district were
interviewed. Three of the SMEs were randomly selected and interviewed using the 3i+3r
individual method and the 3 were interviewed using the 3i+3r incremental method. Each
method then used a separate fourth SME to verify the gold standard protocol.
The experts were selected using the criteria mentioned in Chapter Three. When
trying to find expert teachers within a public school system, one of the easier criteria to
meet is the requirement that an expert have 5-10 years experience in the classroom. Each
subject matter expert must be at the top of their profession with a minimum of five,
preferably ten, years of consistent and successful on-the-job experience as measured by
objective, industry standard measurements (Ericsson, Krampe, & Tesch-Romer, 1993).
The experts randomly chosen for both the present studies had at a minimum seven years
experience teaching expository writing up to a maximum of twenty years experience.
Another easy criterion to determine expertise is to meet is the level of education. The
SME’s chosen for both this study and the concurrent study (Lim, 2015) all had Master’s
degrees or higher in the field of education or writing. What becomes harder to measure is
the quality of professional development (PD) experienced by the SMEs chosen and by
teachers in general. Did the professional development actually teach these experts what it
purported to teach? And was the professional development actually used in classroom
instruction? Porter (2000) and colleagues found that professional development for
teachers was more likely to be used in the classroom if it concentrated on higher-order
teaching strategies rather than traditional workshop or conference professional
COGNITIVE TASK ANALYSIS 100
development. They also found that professional development (PD) varies from school
year to school year and from school to school. Given that the SMEs used in both this
study and the partner study (Lim, 2015) work at different schools, determining the quality
and consistency of the professional development (PD) experienced by the teachers would
be problematic. The experts chosen began their careers at varying dates and at varying
schools. Plus, a teacher with 10 years or more experience may not remember or have
records of the professional development (PD) attending throughout their employment.
Asking expert teachers to list the professional development (PD) over a career spanning a
decade or two might be problematic to confirm.
The use of objective, industry standard data for measuring teacher performance is
also problematic. Schools have been engaged in some form of standardized testing for
many years (Au, 2011). This emphasis on testing and accountability has its pros and cons
and is a discussion for another venue. However, finding individual data on teachers is
next to impossible in most school districts. Concerns over privacy and validity of test
results have made measuring individual teachers difficult. As such, this study and the
concurrent study were compelled to take the recommendations of peers within the district
and from district administrators.
The Assistant Superintendent of the particular Southern California school district
was contacted and provided descriptions of the research studies and a list of potential
SMEs. The Assistant Superintendent approved the study and the list of potential SMEs,
and sought further approval from a fellow Assistant Superintendent in charge of
approving outside research studies within the district, the school board, and the principals
of the school sites where the SMEs worked. Thus, it was necessary for the researchers in
COGNITIVE TASK ANALYSIS 101
this study and the partner study (Lim, 2015) to base choosing SMEs in expository writing
from not only the years spent teaching and educational attainments, but also from the
recommendations of peers and district supervisors. The lack of verifiable testing data for
individual teachers will be a consistent problem for future CTA analysts as they engage
in knowledge elicitation in the field of public education.
Collection of Data
CTA was conducted in the following five stages: (a) collection of preliminary
knowledge; (b) identification of knowledge representations; (c) application of focused
knowledge elicitation methods; (d) analysis and verification of data collected; and (e)
formatting of results for the intended purpose or application (Clark et al., 2008, Hammitt,
2014; Canillas, 2010; Tolano-Leveque, 2010).
This researcher researched literature in the domain of expository writing and
instruction to gather preliminary knowledge and a sequence of task components along
with the types of knowledge needed to perform the task. A fellow researcher with
experience in teaching expository writing was also a source of information and
preliminary knowledge. CTA was used to investigate these knowledge representations
and any additional skills that surfaced during interviews with subject matter experts.
Data collection ensued using the 3i+3r individual CTA method. This method has
been shown effective in capturing expert knowledge to inform instructional design and
the development of instructional materials (Clark, et al., 2008, Zepeda-McZeal, 2014,
Hammitt, 2014). One initial interview was held with each of three SMEs, SME A, SME
B, and SME C. The interviews lasted from 1 hour 25 minutes to 2 hours 44 minutes. The
interviews occurred at or near the conclusion of the academic year 2013-2014. This led
COGNITIVE TASK ANALYSIS 102
all three experts to consider their actions and decisions over the entire school year as they
taught expository writing. Follow-up interviews were held approximately 3 months later
once preliminary protocols had been created from each interview. These interviews lasted
much less time, from 35 minutes to 1 hour 10 minutes. These interviews were held
shortly after the beginning of the 2014-2015 school year. Each SME was given the
opportunity to review their protocol and make any suggestions or corrections they felt
were needed. As the data showed, very few changes were made to the protocols.
Canillas (2010) suggests that a separate group of SME’s might be used to review
preliminary protocols created from the interviews of an initial group of SMEs. She noted
that her SMEs had difficulty committing the time and sustained mental effort needed to
review their initial protocols during a follow-up interview. Hammitt (2014) found the
opposite effect when interviewing public school principals. His SMEs dedicated over 2
hours of time to both the initial interview and the follow-up interviews with full
commitment to the CTA process.
Importance of Context
In the present study, each SME did not give the same amount of time in the
follow-up interview as that given in the initial interview which occurred 3 months earlier.
Each SME reviewed their own protocol and was asked if anything needed to be reviewed,
removed, or refined. All three SMEs agreed with the protocol with only minor
suggestions. The shorter follow-up interviews could be the result of protocols that
adequately captured the actions and decision steps given in each interview. An alternative
view is that the SME’s were in a different frame of mind since they had begun a new
school year with a new group of students. Their frame of mind was no longer engaged in
COGNITIVE TASK ANALYSIS 103
reviewing the recently past school year but was now forward focused into the current
school year. This could show the importance of context as it influences the knowledge
acquisition and experience of these SMEs. Lave (1993) argues that learning is not
independent of experience. Learning is not problematic but what is learned is “complexly
problematic” (Lave, 1993, pg. 203). Acquisition of knowledge is not simply taking in
knowledge, but assuming all things fit in categories requiring reconceptualization as a
cultural or social product. Lave (1993) asserts that engaging in learning is an act
extending beyond the present situation.
These findings might suggest the mental state of SMEs is influenced by the place
and context where the teacher is in within the place and time of the school year, leading
to the conclusion that the specific time in which interviews are completed and the time
span between interviews may affect knowledge elicitation during the CTA interviews. In
the current study, the knowledge elicitation responses of the SMEs being interviewed
were based on prior experience and professional knowledge. But these responses were
also based on the current school year as it was ending. It is likely that the images these
experts created as they were asked to reflect on their teaching of expository writing was
based on the students they presently had in class coupled with the assignments and
expectations that come at the end of a school year (Feldon, 2007a). These external
contextual situations might have had heavy meaning at that point of the school year.
However, when the second interviews occurred, the teachers were in the midst of
beginning a new school year with new students. The curriculum was at a different point
of progression as were the expectations these teachers had for their new group of students.
COGNITIVE TASK ANALYSIS 104
The contexts, so important in the first interview, were no longer in play. These SMEs
were now in a different context, with different students and different situations.
To alleviate this possible situation, the semi-structured interviews should be
completed as soon as possible to allow teachers to keep the contextual realities of their
current classroom fresh in their minds. Of course, a fruitful line of interviewing might
including performing the first interview close to the beginning of the school year and
holding the second review interview later in the school year to capture new action and
decision steps. This might allow the gold standard protocol to pick up the natural
progression of writing instruction or any other curricular subject as it is taught throughout
a school year.
Cognitive Load Theory
Another alternative explanation for the lack of addition action and decision steps
in the second round of interviews may lie in the cognitive demands of teaching.
Cognitive Load Theory looks at the way learners absorb and structure knowledge for
later recall. An important facet of this theory is our cognitive structure based on working
and long-term memory (Sweller, Ayres, & Kalyuga, 2011). Working memory load is
determined by element interaction as it is influenced by the further interaction of the
contents of our long-term memory and the material being taught. Too many interacting
elements create a heavy working memory load unless schemas or scaffolding are
introduced to help hold this information in long-term memory.
When SMEs are asked to uncover unconscious procedural knowledge, a heavy
cognitive load is being placed upon their working and long-term memory (Sweller, 1988).
Dual process theory suggests that information processing occurs on parallel pathways at
COGNITIVE TASK ANALYSIS 105
the same time (Feldon, 2007a). As SMEs work through the slower conscious processing
of perceptual knowledge, they also process quickly through unconscious thought. When
conflicts occur, performance slows. SMEs then compensate by modifying the quantity of
information to be processed by the use of reasoning strategies. Teachers are unable to
consciously consider all the complexity of problems within the classroom. So they begin
to simplify the situation and act rationally based on that simplified scenario. Feldon
(2007a) suggests that as teachers build expertise, they build elaborate schemas to handle
the complexity of the classroom.
In the present study, the SMEs interviewed had given their elicited knowledge in
the form of action and decision steps during the first interview. When presented with the
protocol, the SMEs may have used the protocol’s steps and procedures as a form of
elaborate schema, thereby reducing their cognitive load as they reviewed their individual
protocols. Because protocols are based on the responses of the SME who reported the
declarative and procedural knowledge contained therein, experts may automatically use
familiar words and phrases as cues that trigger prior knowledge and allow the SME to
find agreement with the protocol’s contents. During the second round of interviews, SME
A, B, and C all made similar affirmative comments like “yep”, and “yes”, and “I
remember saying that,” as they reviewed their protocols. Future research may focus on
changing the phrasing of the protocol to reduce this triggering of prior knowledge.
Automaticity and Conscious Recall
In each initial interview in the 3i+3r individual method, each SME began by
identifying the main procedures of expository writing instruction. Once completed, each
SME was prompted to provide detailed action and decision steps within each main
COGNITIVE TASK ANALYSIS 106
procedure. During the interviews, each SME recalled different iterations of expository
writing instruction as they thought about the actions and decisions made during different
times of the school year. While proceeding through the interview, each SME noted they
had neglected a procedure or action that resulted in the change of the main procedures. It
seems that the deep and probing questions indicative of CTA help experts bring their
unconscious, automated knowledge to a more conscious level.
Discussion of Findings
No formal hypotheses were developed for this research study. The study, however,
was guided by 3 research questions.
Question 1
What are the action and decision steps that expert teachers recall when they
describe how they provide expository writing instruction to their eleventh-grade
students?
Four SMEs were interviewed using the 3i+3r individual method to capture the
total action and decision steps recalled to provide expository writing instruction to
eleventh-grade students. This method elicited a total of 333 action and decision steps, of
which 281 were action steps and 52 were decision steps.
Action steps versus decision steps. The results of this study and the concurrent
study (Lim, 2013) elicited far more action than decision steps. Flynn (2010) argued that
the 1i+3r incremental method was more conducive to eliciting more decision steps than
was the 3i+3r method. The concurrent study (Lim, 2013) does not confirm this finding.
Research on expertise suggests that experts who have developed high levels of
automaticity are able to recall action steps much more easily than automated decision
COGNITIVE TASK ANALYSIS 107
steps based on procedural knowledge (Clark, 2014, Clark & Estes, 1996, Ericsson, 2004).
Clark (2014) found that healthcare experts could recall more action steps because they
place themselves “in the moment,” allowing them to recall more action steps than
decision steps. The difference between action steps and decision steps of the SMEs
interviewed for this study bear this research out.
One possible reason decision steps were tough to elicit by is the nature of the task
being investigated. Expository writing is a multi-faceted task with multiple steps and a
plethora of various strategies that can be used to teach this important skill. As the
interviews in both studies proceeded, it appeared that the SMEs applied great efforts to
cognitively process the unconscious steps they perform as they teach this subject.
However, once the SMEs shared the main procedures they used to teach, they were then
asked to add the sub-tasks subsumed under each main procedure and the decisions that
accompany them. By this time, the interviews were well over an hour and the SMEs
appeared to be losing focus on the subject at hand. Soon, as Feldon (2007a) noted, the
SMEs began to simplify their answers and provide less decisions steps and more action
steps. The vast nature of expository writing may have overtaxed the cognitive load of the
SMEs, thereby reducing focus and motivation on the task at hand. It would be prudent in
future students to break up the CTA analysis of such large learning tasks, such as
expository writing, into more manageable chunks.
Number of action and decision steps. The concurrent study (Lim, 2015) found
210 action and decision steps compared to 333 in the present study. What might account
for the difference in total action and decision steps? Lim (2015) and Zepeda-McZeal
(2014) suggest one reason is that the experts interviewed in the 1i+3r incremental method
COGNITIVE TASK ANALYSIS 108
were influenced unconsciously by the protocol they were asked to review. In the 1i+3r
incremental method SME A was asked through a semi-structured interview to report the
knowledge and skills used to teach expository writing. Once this protocol was created by
the knowledge analyst and verified again by SME A, the protocol was given to SME B
and SME C for addition discussion and verification. This non-conscious, cognitive
process where memory is prompted by external cues is called priming (Tulving &
Schacter, 1990). Lim (2015) reports that, as an example, SME B in the 1i+3r incremental
method added 68 more action and decision steps to SME A’s protocol, and SME C added
only 20 more action and decision steps to SME B’s protocol. Lim (2015) suggests a
possible reason for this is SME C accepted the assumption that the protocol was correct
because she had been told that two other SMEs had already approved it to this point.
Lim (2015) also suggests that SME B and SME C in his study could have been
influenced by groupthink. Irving Janis, in Aronson (2003) defines groupthink as “the
mode of thinking that persons engage in when concurrence seeking becomes so dominant
in a cohesive in-group that it tends to override realistic appraisal of alternative courses of
action” (p. 15). Under the pressure of conforming within a group, individuals begin to
doubt their own reservations and do not contribute dissenting opinions (Aronson, 2003).
One weakness with this possibility is that while SMEs in the 1i+3r incremental CTA
method were aware that other experts had reviewed the protocol created from the
previous expert, they had little or no direct opportunity to pressure one another into a
conforming state of thought.
In the present 3i+3r individual study, none of the SMEs had knowledge of one
another through this study and were not given any of the interview material from the
COGNITIVE TASK ANALYSIS 109
other experts. SME B did mention during the initial interview the name of SME C as an
example of a teacher using a learning strategy SME B was interested in trying out within
the classroom. This lends credence to the peer recognition as one criterion when selecting
experts for CTA study (Clark, 2014, Clark, et al., 2008, Hammitt, 2014).
Teacher choice in the use of pedagogy strategies. Another possible reason for the
larger amount of actions steps elicited in the 3i+3r individual method is the inclusion by
the SMEs of various teaching strategies used by each expert in their classroom.
Spreadsheet analysis showed that there was on 9.31% complete alignment between all
SMEs and 11.41% partial alignment. Upon closer examination of the gold standard
protocols, the 3i+3r individual method elicited more actions tied to discrete pedagogical
strategies often used by one expert alone and not the other two experts. CTA looks to
capture the knowledge and skills of experts in their domain. In the field of education,
there are often a multitude of strategies that teachers acquire and use to great effect in the
classroom. This begs the question of whether teachers should be compelled to adopt
common strategies within schools or among grade levels. Does this ensure student
success in acquiring academic knowledge? The 3i+3r individual method suggest this
might not be the case. Galbraith & Rijlaarsdam (1999) suggest that different pedagogical
strategies may apply differently to students and teachers alike. They suggest that the
choice of strategy is not as important as teaching writing as a process with goals set for
specific communicative purposes (Chandrasegaran, 2013). Lampert (1985) argues the
assumption that there are often conflicting problems and concerns within one class. The
teacher is tasked with solving these pedagogical problems for which there are many
different solutions that may well be right. However, the teacher cannot base all
COGNITIVE TASK ANALYSIS 110
pedagogical choices upon theory but upon choosing to act upon a particular choice and
living with the consequences. Thus, it is the context of the class and the participants,
students and teacher alike, that determine what strategies might be used. This leads to the
conclusion that the main procedures noted in the gold standard protocol should offer
teachers a viable blueprint that, if followed, will help a novice teacher teach more
effectively. The strategies contained therein may be looked upon as suggested activities.
The different teaching strategies used by one or more SMEs contained in this protocol
alone suggest this course of action has led to success in their classrooms.
Question 2
What percentage of action and/or decision steps, when compared to a gold
standard do expert teachers omit when they describe how they provide expository writing
instruction to their eleventh-grade students?
To answer this research question, the gold standard protocol was compared to
each SME’s individual protocol to determine omissions of expert knowledge derived
from omission of actions and decisions steps for the task of expository writing instruction.
Research shows that when experts report the knowledge and skills used to perform a
complex task, they unintentionally omit up to 70% of critical information novices need to
perform a task successfully (Clark & Feldon, 2006, Feldon, 2004). The average omission
SMEs A, B, and C was 57.46%. This confirms the 70% omission rule. The average action
step omissions for SMEs A, B, and C were 57.89% and the average decision step
omissions for all SMEs were 55.13%.
In the concurrent study (Lim, 2015), total omission steps for SME A were 71
action steps and 26 decision steps for a total of 97 steps. Because the protocol created
COGNITIVE TASK ANALYSIS 111
from the expert elicitation of SME A, SME B omitted 11 action steps and 18 decision
steps for a total of 29 steps. SME C omitted 4 action steps and 5 decision steps for a total
of 9 steps. Finally, SME D verified the preliminary gold standard protocol and omitted
zero action or decision steps.
Looking at SME A in the 1i+3r incremental method, omissions totaled 46.19%
compared to the 1i+3r incremental GSP. While the average omissions in the 3i+3r
individual method was 144 action and decision steps, the average percentage was 43.32%
which is roughly equivalent to the omissions of SME A in the 1i+3r incremental method.
As SME B and SME C reviewed the 1i+3r incremental protocol created from SME A’s
interviews, their own knowledge elicitation becomes focused on the protocol and with
what happens in their own classrooms concurrently. In the 3i+3r individual method, there
was no instance where the SMEs were influenced by the action and decision steps of
other SMEs. The SME’s in the 3i+3r individual method had little opportunity to consider
actions and decisions independent of what happens in their own classrooms. It appears
possible that more action and decision steps are elicited with the 3i+3r individual method
at the expense of more steps omitted by each SME.
Question 3
Which method of CTA, 1i+3r incremental or the 3i+3r individual, is more
efficient at capturing the instructional expertise of SMEs?
Efficiency. The final purpose of this study was to replicate facets of Flynn’s
(2012) and Zepeda-McZeal’s (2014) dissertations to determine which CTA method, the
3i+3r individual or the 1i+3r incremental, is more efficient. Flynn (2012) found in her
research that the 1i+3r incremental method was more efficient that the 3i+3r individual
COGNITIVE TASK ANALYSIS 112
method because of its capacity to capture more decision steps in a shorter amount of time
and for less cost. Zepeda-McZeal (2014) found that the 3i+3r individual method proved
more effective at capturing both critical action and decision steps than the 1i+3r
incremental method. The results from this study and a concurrent study (Lim, 2015)
compared actions and decision steps and found the results inconclusive as to which is
more efficient. However, the results from both current studies warrant further analysis
and discussion.
Criteria to measure effectiveness. Flynn (2012) suggests a list of criteria to
measure the effectiveness of the results of a CTA. The first is a cost-effectiveness
measurement which looks at various CTA methods to determine which one gets a quality
job completed in the most cost effective way. Another criterion is to measure the amount
of resources used to complete a CTA. Hoffman (1987) suggests several qualitative and
quantitative measurements to determine if a CTA method outcome, such as interviews or
protocols, is effective. Hoffman (1987) also suggests method efficiency as one
measurement used to quantify the effectiveness of a CTA interview is to count the total
number of propositions captured in a unit of time. He uses propositions per minute as his
metric. These propositions include both declarative and procedural knowledge. Flynn
(2012) also considered this metric as a method to measure effectiveness.
Using Flynn’s (2012) cost criterion, costs were analyzed by comparing the
amount of money spent on transcribing the interviews for each SME. The transcribing for
the 3i+3r individual method totaled $475.00 while the cost of transcribing the 1i+3r
incremental method was $314.00. In this case, the 1i+3r incremental method was slightly
more effective in terms of money saved. Using Flynn’s (2012) time criterion, the only
COGNITIVE TASK ANALYSIS 113
quantifiable time available for analysis was the time spent in interviewing each SME. The
time spent interviewing SMEs for the 3i+3r individual method was 11 hours 46 minutes.
The time spent on interviewing SMEs for the 1i+3r incremental method was 9 hours 3
minutes for a savings of 2 hours 43 minutes. Again, the 1i+3r incremental method proves
to be more time efficient as well. A critique of using money and time to measure CTA
effectiveness is the possible decrease in quality when seeking to trim costs and time.
Using Hoffman’s (1987) method efficiency metric of counting propositions in a
specified unit of time as a third criterion, Flynn (2012) counted only decision steps in her
study. McZeal (2014) counted both action and decisions steps which is replicated here.
Neither Flynn (2012) nor Zepeda-McZeal (2014) figured the amount of propositions per
minute. In the present study, the 1i+3r incremental method captured 210 action and
decision steps total in 9 hours 3 minutes (543 minutes) of interview time for a average
of .38 propositions per minute, while the 3i+3r individual method captured 333 action
and decisions steps in 11 hours 46 minutes (706 minutes) of interview time for an
average of .47 propositions per minute. This comparison results in 3i+3r individual
having 123 more total action and decision steps than the 1i+3r incremental method, and
more per minute of interview time with a difference of .09 propositions (declarative or
procedural knowledge) per minute.
A possible reason for the large number of action and decision steps in the 3i+3r
individual method as compared to the 1i+3r incremental method could be the nature of
the task being investigated. Hoffman (1987) suggests several quantitative comparative
measurements to perform analysis of a task. Task brevity, task flexibility, task simplicity
and method efficiency all relate to the elicitation of knowledge from experts by the
COGNITIVE TASK ANALYSIS 114
knowledge analyst. This elicitation is influenced by the task being studied. If the task is
familiar and straightforward, analysis should be brief and simple. However, if there are
times where a task requires flexibility because the nature of the task changes as variations
occur and different sets of materials become variables in expert decision-making.
Task complexity. The studies of Flynn (2012), Zepeda-McZeal (2014), and the
current and concurrent study (Lim, 2015) looked at different types of tasks. Flynn (2012)
investigated the task of recruitment interviews by the Army. The nature of the task is
straightforward and lasts at most an hour or two to complete. Zepeda-McZeal (2014)
analyzed informational reading instruction for intermediate grade-level students with
mild to moderate learning disabilities. This task would take a number of weeks to
complete and would include a group of students with various needs and abilities. The task
studied in this study and concurrent study (Lim, 2015) is analyzing expository writing
instruction to eleventh-grade students. Much like Zepeda-McZeal, this task is much
longer in duration and contains many variables based on topic and classroom make-up.
Hoffman (1987) argues that straightforward CTA investigations with tasks of shorter
duration should result in more action and decision steps elicited than would be the case in
CTAs analyzing tasks of longer duration and more complexity.
The duration and complexity of the task may also result in SMEs experiencing
greater cognitive overload (Sweller, 1988). In studying problem-solving using a means-
end analysis, Sweller (1988) noted that this analysis method may impose a heavy
cognitive load. It was also noted that the cognitive mechanisms required for problem
solving and for schema acquisition appear to be distinct. This suggests that cognitive load
used to problem solve may not assist in schema acquisition. Thus, expertise is limited if
COGNITIVE TASK ANALYSIS 115
its development is based on problem-solving as a developmental tool. In the current study,
experts are asked to report the actions and decision steps they use to teach expository
writing. As they reflect back on their practice, think about the things they do, and create
mental images of their performance in the classroom, Sweller’s (1988) research suggests
that experts would find it increasingly easier to share their conscious action steps but
much more difficult to share their unconscious actions steps. The cognitive load required
to bring these procedural steps to light might tax the expert’s cognitive load and result in
less information gathered using CTA analysis.
In this study of the 3i+3r individual method, SME’s were asked to go through an
in-depth interview with the knowledge analyst. The interviews lasted on average over 2
hours with the longest lasting 2 hours 42 minutes. Each expert was asked to share the
main procedures they use to teach expository writing. Once this step was completed, the
interview questions went into greater detail looking at each main step for the actions and
decision steps that make up the sub-steps of each main procedure. These experts were
asked to delve into detail on their knowledge and skills.
On the other hand, the experts who were part of the 1i+3r incremental method
fared differently. SME A was engaged in an in-depth interview much like those experts
in the 3i+3r individual study. But SME B and SME C were not subjected to these same
in-depth interviews. The interviews in the 1i+3r incremental method averaged 1 hour 23
minutes with the longest interview with SME A lasting 2 hours 12 minutes. Comparing
the amount of decision steps in both protocols showed that 35 of the 52 decision steps in
the 3i+3r individual method were captured within the first four main procedures, while 37
of the 51 decision steps for the 1i+3r incremental method were elicited within the last
COGNITIVE TASK ANALYSIS 116
five main procedures of the protocol. These results suggest that the 3i+3r individual
method where experts are asked to recall both action and decision steps may create a
sense of cognitive overload for the SMEs. This may explain why decision steps diminish
as the CTA interview moves through the main procedures. The opposite appears to be
true with the 1i+3r incremental CTA method. The experts who participated in this
method may have experienced less cognitive overload which frees up working memory to
attend to recall of procedural knowledge. Fisk and Taylor (1991) described the idea the
rational people conserve cognitive energy as a way to use ones cognitive abilities more
efficiently. Cognitive misers create cognitive heuristics or shortcuts by ignoring
information to reduce cognitive load, or overuse familiar information to keep from
having to continue searching for more, or even accept a lesser alternative because it is
close to good enough. In order to ease cognitive load, the experts participating in the
3i+3r individual method may have engaged in cognitive conservation and recalled less
decisions than was possible.
Limitations
The present study produced both consistent and inconsistent results with existing
CTA research studies regarding expert knowledge captured as action and decision steps,
expert knowledge omissions, and the relative efficiency of the 1i+3r incremental and
3i+3r individual methods in capturing expertise. The following sections talk about the
limitations of this study, such as confirmation bias, and internal and external validity.
Confirmation Bias
The researcher of this study has 17 years experience in elementary education.
This experience does not include teaching expository writing at the secondary level. This
COGNITIVE TASK ANALYSIS 117
reduces the tendency of researchers to align the knowledge and skills elicited from the
experts in this and the concurrent study (Lim, 2015) to their own experiences (Clark,
2014). The lack of experience and prior knowledge on the part of the researcher in this
domain resulted in the need for extensive bootstrapping (Schraagen, et al., 2000). This
bootstrapping resulted in the efforts of the researcher to avoid placing any preexisting
experiences or expectations upon the collected data.
Internal Validity
Observations of the experts’ knowledge and skills while performing this task
would have ensured the internal validity of the captured data as a result of this study and
the gold standard protocol created. Observations of the SMEs did not occur as part of this
study or the concurrent study (Lim, 2015), and therefore the results cannot be validated at
this time. Although this study did not include validating the GSP by observation, this
would be an appropriate study for future research.
External Validity
External validity for the current study would require that the results be
generalizable enough to transfer to other settings containing similar domain
characteristics. The sample size of four SMEs, all of whom work within the same
Southern California school district, limits the external validity or generalizability of the
current study. Future CTA research studies in the same domain might include a larger
number of experts and from multiple locations to improve external validity.
However, the nature of the present study can lend itself as a case study focusing
on one particular task. Merriam (2009) argues that a case study such as this can add to the
knowledge base, such as on expository writing, thus bringing about understanding that
COGNITIVE TASK ANALYSIS 118
can possibly improve practice. While external validity may be limited, the practical
application of the material discussed in the study can allow the reader to transfer
particular lessons to similar situations. It will be up to the reader and practitioners of
expository writing instruction to determine the usefulness and applicability of this study.
Implications
Experts are often tasked with using their knowledge and skills to teach novice
learners, to help create curriculum and instructional materials, and to mentor or coach
these novice learners to problem-solve or perform complex tasks (Jackson, 1985). This
becomes problematic when these experts have automated their knowledge and often omit
up to 70% of critical information when mentoring or teaching novice learners. CTA has
shown to be effective in capturing this automated knowledge and skills that are often
unavailable for instant recall across a variety of domains. This study substantiates the use
of CTA to capture the knowledge and skills of SMEs in completing complex tasks such
as teaching expository writing at the eleventh-grade level.
Research has shown further that training and instruction informed by CTA studies
can improve performance proficiency by increasing human accuracy and adaptability
while also providing long-term cost savings (Clark, et al., 2008). CTA training has been
shown to increase learning gains 50% with reduced training time and costs (Clark, 2011).
Velmahos (2004) and colleagues determined that surgical residents may complete tasks
25% faster, learn 40% more information, and do so with up to 50% less mistakes. The
implication is clear; CTA has the potential to inform teacher training and instruction to
produce highly trained and skilled teachers in less time and with less cost.
COGNITIVE TASK ANALYSIS 119
Writing research has found that students struggle to successfully write expository
essays (Applebee & Langer, 2009; Beck, et al., 2013; Kiuhara, et al., 2009). Additionally,
researchers agree that highly-qualified teachers have a greater positive influence on
student achievement than class sizes, additional financial resources, or teacher salaries
(Darling-Hammond, 2000; Hanushek, 1992; McQuitty, 2012). The current study suggests
that putting CTA-based instruction at the use of classroom teachers so they can replicate
expertise may improve instruction and thus improve student achievement in writing
expository essays.
Additionally, Common Core State Standards (CCSS) call for students to write
more expository text and more writing in general in other subject areas including social
studies, science, and math. Content area teachers other than English teachers will need
future PD to improve instruction for their students with writing. To meet these
expectations, CTA-based instruction may be the solution to providing in-service teachers
and pre-service teachers, especially those in subject areas outside of English Language
Arts with effective writing instruction developed from experts in the field.
Future Research
Further research in to the identification of experts, especially in the education
field, may be necessary. For instance, this study used several criteria to help identify
expert teachers. One criterion was consistent and successful on-the-job experience as
measured by student achievement in expository writing. In this case, “success” was
based on reliable, industry standard outcomes that can be validated, such as standardized
test scores. However, a relevant concern, and one that this study does not address, was
the multitude of variables that might influence the results of student achievement other
COGNITIVE TASK ANALYSIS 120
than the teacher. One assumption is that students enter classrooms each year with a
variety of skill levels, content knowledge, motivation level, access to resources, and
psychosocial support, just to name a few. With all these varied factors influencing student
achievement and students writing achievement in particular, identifying an expert teacher
using “success” as part of the criteria becomes much more complex.
It is also unknown whether CTA-guided instruction has been used to inform pre-
service and in-service teacher training programs to date. A search for literature on this
subject did not reveal any studies. Therefore, future research could involve a randomized
experimental design study implementing pre-service and in-service teacher training,
particularly for novice teachers, using CTA-guided instruction and traditional
instructional methods to compare learning gains on expository writing instruction similar
to the tasks outlined in this study. Conducting longitudinal research will also inform this
body of research in determining short- and long-term learning gains in expository writing.
When considering the improvement of knowledge elicitation methods, future
research is needed to compare the relative effectiveness of the 3i+3r individual and the
1i+3r incremental methods in eliciting critical declarative and procedural knowledge for
complex task performance in K-12 school settings. Further research could be conducted
in the domain of expository writing instruction to determine which of the two
methodologies is most appropriate for complex learning tasks. The 1i+3r incremental
method may be yet another methodology added to the stock of CTA methods. Additional
research is needed to determine effectiveness in capturing complete and accurate
knowledge representations of complex tasks in a more effectively than the 3i+3r method.
COGNITIVE TASK ANALYSIS 121
Future research may also delve deeper into the ground covered here. Using one of
the main procedures noted in the protocol, research could determine more clearly and
concisely the total actions and decision steps. By focusing on one procedure at a time,
future knowledge analysts can increase the amount of declarative and procedural
knowledge captured while reducing any possible cognitive overload on the part of the
SMEs used in the study (Sweller, 1988).Future research may also analyze the optimal
length of time needed to perform a complex task for a CTA study. As previously
mentioned, the nature of the task can greatly influence the outcomes of doing CTA
(Hoffman, 1987). If the goal is to maximize the educational benefits of instructional
content generated by way of CTA, perhaps certain complex tasks are more suitable to
doing a CTA based on their relative time frame.
Conclusion
The purpose of this study was to add to the existing body of CTA research. CTA
has demonstrated effectiveness in capturing the declarative and procedural knowledge
necessary for novice learners to attain information and skills needed to perform complex
tasks in a variety of domains. Only recently have researchers attempted to apply CTA
within the field of education to determine its effectiveness in this particular domain. This
study had three purposes. First, it sought to describe the benefits of CTA for capturing
complete descriptions of the action and decision steps experts use when providing
expository writing instruction at the eleventh-grade level. Second, this study identified
the omissions made by experts when recalling the action and decision steps used to teach
expository writing. Finally, this research sought to determine which of the two common
CTA methods, the 3i+3r individual and the 1i+3r incremental, is more effective at
COGNITIVE TASK ANALYSIS 122
capturing action and decision steps in a shorter period of time and at a lower cost. The
GSP developed as a result of this study has identified both the observable and
unobservable behavior a novice teacher needs to replicate expert expository writing
instruction. This expertise captured within the GSP can function as a foundation for the
instructional design in a teacher preparation program for pre-service teachers and for the
Professional Development of in-service teachers teaching at the eleventh grade level. By
replicating the expertise to inform future professional training, CTA-based instruction
such as this research study can lead to significant improvement gains in student writing
achievement.
COGNITIVE TASK ANALYSIS 123
References
Addison, J., & McGee, S. J. (2010). Writing in high school/writing in college: Research
trends and future directions. College Composition and Communication, 147-179.
Alber-Morgan, S. R., Hessler, T., & Konrad, M. (2007). Teaching writing for
keeps. Education and Treatment of Children, 30(3), 107-128.
Alexander, P.A. (2003). The development of expertise: The journey from acclimation to
proficiency. Educational Researcher, 32(8), 10-14.
Alvermann, D. E. (2002). Effective literacy instruction for adolescents. Journal of
literacy research, 34(2), 189-208
Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K. (2010).
How learning works: Seven research-based principles for smart teaching. John
Wiley & Sons.
Anderson, J. R. (1996a). ACT: A simple theory of complex cognition. American
Psychologist, 51(4), 355-365.
Anderson, J. R. (1996b). The Architecture of Cognition. Mahwah, NJ: Lawrence
Erlbaum Associates.
Anderson, J. R. (1993). Problem solving and learning. American Psychologist, 48(1),
25-44
Anderson, J.R. (1982). Acquisition of cognitive skill. Psychological Review, 89(4), 369-
406
Anderson, J.R., & Fincham, J.M. (1994). Acquisition of procedural skills from examples.
Journal of experimental psychology: Learning, memory, and cognition 20(6),
1322-1340.
COGNITIVE TASK ANALYSIS 124
Anderson, J.R., & Schunn, C.D. (2000). In R. Glaser (Ed.), Advances in instructional
psychology (5). Mahwah, NJ: Erlbaum
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E.,
Pintrich, P. R., & Wittrock, M. C. (2001). Taxonomy for learning, teaching, and
assessing: A revision of Bloom's taxonomy of educational objectives, abridged
edition. White Plains, NY: Longman.
Annett, J. (2000). Theoretical and pragmatic influences on task analysis methods. In J.M.
Schraagen, S.F. Chipman & V.L. Shalin (Eds.), Cognitive Task Analysis (pp. 24-
36). Mahwah, NJ: Lawrence Erlbaum Associates.
Applebee, A. N. 1984. Writing and reasoning. Review of Educational Research, 54:
577–596.
Applebee, A. N., & Langer, J. A. (2011). " EJ" Extra: A Snapshot of Writing Instruction
in Middle Schools and High Schools. English Journal, 14-27.
Applebee, A. N., & Langer, J. A. (2009). EJ Extra: What Is Happening in the Teaching of
Writing?. English Journal, 18-28.
Aronson, E. (2003). The social animal. Macmillan.
Au, W. (2011). Teaching under the new Taylorism: high‐stakes testing and the
standardization of the 21st century curriculum. Journal of Curriculum Studies,
43(1), 25-45.
Baker, J., Brizee, A., & Angeli, E. (2013, March 11) Expository essays. Retrieved from
https://owl.english.purdue.edu/owl/resource/685/02/
Bartholio, C. W. (2010). The use of cognitive task analysis to investigate how many
COGNITIVE TASK ANALYSIS 125
experts must be interviewed to acquire the critical information needed to perform
a central venous catheter placement (Doctoral dissertation, University of
Southern California).
Beck, S. W., Llosa, L., & Fredrick, T. (2013). The challenges of writing exposition:
Lessons from a study of ELL and non-ELL high school students. Reading &
Writing Quarterly, 29(4), 358-380.
Bedard, J. & Chi, M.T.H. (1992). Expertise. Current Directions in Psychological
Science, 1(4), 135-139
Berman, R. A., & Nir-Sagiv, B. (2007). Comparing narrative and expository text
construction across adolescence: A developmental paradox. Discourse processes,
43(2), 79-120.
Boscolo, P. (1990). The construction of expository text. First Language, 10(30), 217-230.
Burdick, H., Swartz, C. W., Stenner, A. J., Fitzgerald, J., Burdick, D., & Hanlon, S. T.
(2013). Measuring students’ writing ability on a computer-analytic developmental
scale: An exploratory validity study. Literacy Research and Instruction, 52(4),
255-280.
Canillas, E. N. (2010). The Use of Cognitive Task Analysis for Identifying the Critical
Information Omitted when Experts Describe Surgical Procedures(Doctoral
dissertation, University of Southern California).
Chandrasegaran, A. (2013). The effect of a socio-cognitive approach to teaching writing
on stance support moves and topicality in students’ expository essays. Linguistics
and Education, 24(2), 101-111.
Chi, M. T. H. (2006). Two approaches to the study of experts’ characteristics. In K. A.
COGNITIVE TASK ANALYSIS 126
Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge
handbook of expertise and expert performance (pp. 21–30). New York:
Cambridge University Press.
Chipman, S.F. (2000). Introduction to cognitive task analysis. In J.M. Schraagen, S.F.
Chipman & V.L. Shalin (Eds.), Cognitive Task Analysis (pp. 24-36). Mahwah,
NJ: Lawrence Erlbaum Associates
Chao, C. J., & Salvendy, G. (1994). Percentage of procedural knowledge acquired as a
function of the number of experts from whom knowledge is acquired for
diagnosis, debugging, and interpretation tasks. International Journal of
Human ‐Computer Interaction, 6(3), 221-233.
Clark, R. (2014). Cognitive task analysis for expert-based instruction in healthcare. In
Handbook of research on educational communications and technology (pp. 541-
551). Springer New York.
Clark, R. E. (2006). Training Aid for Cognitive Task Analysis, Technical report produced
under contract ICT 53-0821-0137-W911NF-04-D-0005 from the Institute for
Creative technologies to the Center for Cognitive Technology, University of
Southern California.
Clark, R., & Elen, J. (2006). When less is more: Research and theory insights about
instruction for complex learning. In J. Elen & R.E. Clark (Eds.), Handling
complexity in learning environments: Theory and research (pp. 283-295). New
York: Elsevier.
Clark, R.E. (2008). Resistance to change: Unconscious knowledge and the challenge of
unlearning. In D.C. Berliner, & H. Kupermintz (Eds.), Changing institutions,
COGNITIVE TASK ANALYSIS 127
environments, and people. Mahwah, NJ: Lawrence Erlbaum Associates.
Clark, R.E. (1999). Ying and yang: Cognitive motivational processes in multimedia
learning environments. In J. van Merriënboer (ed.) Cognition and multimedia
design. Herleen, Netherlands: Open University Press.
Clark, R. E. and Estes, F. (1996) Cognitive task analysis, International Journal of
Educational Research. 25(5). 403-417.
Clark, R. E., Feldon, D., vanMerrienboer, J., Yates, K, and Early, S. (2008). Cognitive
Task Analysis. In Spector, J. M., Merrill, M. D., van Merriënboer, J. J. G., &
Driscoll, M. P. (Eds.) Handbook of research on educational communications and
technology (3
rd
ed.). Mahwah, NJ: Lawrence Erlbaum Associates. 577-593.
Clark, R. E., Pugh, C. M., Yates, K. A., Inaba, K., Green, D. J., & Sullivan, M. E. (2011) The
use of cognitive task analysis to improve instructional descriptions of
procedures. Journal for Surgical Research.
Clark, R. E., Yates, K., Early, S., & Moulton, K. (2010). An analysis of the failure of
electronic media and discovery-based learning: Evidence for the performance
benefits of guided training methods. In K. H. Silber & R. Foshay, (Eds.).
Handbook of training and improving workplace performance, Volume 1
Instructional design and training delivery (pp. 263-297). New York: John Wiley
and Sons
Coleman, D., Pimentel, S., & Zimba, J. (2012). Three core shifts to deliver on the
promise of the common core standards. State Education Standard, 12(2), 9-12.
College Board. (2013). SAT report on college and career readiness. New York: NY.
COGNITIVE TASK ANALYSIS 128
Cooke, N. J. (1999). Knowledge elicitation. In F. T. Durso (Ed.), Handbook of Applied
Cognition (pp. 479-509). New York: Wiley.
Cooke, N. J. (1994). Varieties of knowledge elicitation techniques. International Journal
of Human-Computer Studies, 41, 801-849.
Corbett, A. T., Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of
procedural knowledge. User Modeling and User-Adapted Interaction, 4, 253-278.
Council of Chief State School Officers, & National Governors Association. (2010).
Common core state standards for English language arts & literacy in
history/social studies, science, and technical subjects.
Crandall, B., Klein, G., & Hoffman, R. R. (2006). Working minds: A practitioner’s guide
to cognitive task analysis. Cambridge, MA: MIT Press.
Crispen, P. D. (2010). Identifying the point of diminishing marginal utility for cognitive
task analysis surgical subject matter expert interviews (Doctoral dissertation,
University of Southern California).
Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of
state policy evidence. Education Policy Analysis Archives, 8(1).
Darling-Hammond, L., & Baratz-Snowden, J. (2007). A good teacher in every
classroom: Preparing the highly qualified teachers our children deserve.
Educational Horizons, 85(2), 111-132.
De La Paz, S., & Graham, S. (2002). Explicitly teaching strategies, skills, and
knowledge: writing instruction in middle school classrooms. Journal of
Educational Psychology, 94(4), 687.
Ericsson, K. A. (2004). Invited Address: Deliberate practice and the acquisition and
COGNITIVE TASK ANALYSIS 129
maintenance of expert performance in medicine and related domains. Academic
Medicine, 79(10), s70-s81.
Ericsson, K.A. & Charness, N. (1994). Expert performance its structure and acquisition.
American Psychologist, 49(8), 725-747.
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate
practice in the acquisition of expert performance. Psychological Review, 100(3),
363-406.
Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance:
Evidence of maximal adaptation to task constraints. Annual Review of Psychology,
47(1), 273-305.
Fackler, J. C., Watts, C., Grome, A., Miller, T., Crandall, B., & Pronovost, P. (2009).
Critical care physician cognitive task analysis: an exploratory study. Critical Care,
13(2), R33.
Fanetti, S., Bushrow, K. M., & DeWeese, D. L. (2010). Closing the gap between high
school writing instruction and college writing expectations. English Journal, 77-
83.
Feldon, D. F. (2007a). Cognitive load and classroom teaching: The double-edged sword
of automaticity. Educational Psychologist, 42(3), 123-137.
Feldon, D. (2007b). Implications of research on expertise for curriculum and pedagogy.
Educational Psychology Review, 19(2), 91-110.
Feldon, D. F., & Clark, R. E. (2006). Instructional implications of cognitive task analysis
as a method for improving the accuracy of experts’ self-report.
Feldon, D. F. (2004). Inaccuracies in expert self-report: Errors in the description of
COGNITIVE TASK ANALYSIS 130
strategies for designing psychology experiments (Doctoral dissertation, University
of Southern California).
Fiske, S. T., & Taylor, S. E. (1991). Social cognition .
Flynn, C. L. (2012). The relative efficiency of two strategies for conducting cognitive
task analysis (Doctoral dissertation, UNIVERSITY OF SOUTH CAROLINA).
Gagne, R. M. (1962). The acquisition of knowledge. Psychological review, 69(4), 355.
Galbraith, D., & Rijlaarsdam, G. (1999). Effective strategies for the teaching and learning
of writing. Learning and instruction, 9(2), 93-108.
Glaser, E. M. (1976). Putting knowledge to use: A distillation of the literature regarding
knowledge transfer and change. Los Angeles: Human Interaction Research
Institute.
Glaser, R., & Chi, M. T. H. (1988). Overview. In M. T. H. Chi, R. Glaser, & M. Farr
(Eds.), The Nature of Expertise (p. xv-xxviii). Mahwah, NJ: Lawrence Erlbaum
Associates.
Glassman, M. (2001). Dewey and Vygotsky: Society, experience, and inquiry in
educational practice. Educational Researcher, 30(4), 3-14.
Graham, S., & Harris, K. R. (2013). Common core state standards, writing, and
students with LD: Recommendations. Learning Disabilities Research & Practice,
28(1), 28-37.
Graham, S., Gillespie, A., & McKeown, D. (2013). Writing: Importance, development,
and instruction. Reading and Writing: An Interdisciplinary Journal, 26(1), 1-15.
Graham, S., & Perin, D. (2007a). Writing next: Effective strategies to improve writing
of adolescents in middle and high schools. A Report to Carnegie Corporation of
COGNITIVE TASK ANALYSIS 131
New York. Alliance for Excellent Education.
Graham, S., & Perin, D. (2007b). A meta-analysis of writing instruction for adolescent
students. Journal of educational psychology, 99(3), 445.
Grisham, D. L., & Wolsey, T. D. (2011). Writing instruction for teacher candidates:
Strengthening a weak curricular area. Literacy Research and Instruction, 50(4),
348-364.
Gott, S. P., Glaser, R., Hall, E. P., Dibble, E., & Pokorny, R. A. (1996). A naturalistic
study of transfer: Adaptive expertise in technical domains (pp. 258-288). Ablex.
Grossman, P., Loeb, S., Cohen, J., Hammerness, K., Wyckoff, J., Boyd, D., & Lankford,
H. (2010). Measure for measure: The relationship between measures of
instructional practice in middle school English language arts and teachers’
value-added scores (No.16015). National Bureau of Economic Research
Hall, E. M., Gott, S. P., & Pokorny, R. A. (1995). A procedural guide to cognitive task
analysis: The PARI methodology. Brooks Air Force Base, TX: Manpower and
Personnel Division, U.S. Air Force.
Hammitt, C. S. (2014). Using cognitive task analysis to capture how expert principals
conduct informal classroom walk-throughs and provide feedback to teachers
(Doctoral dissertation, University of Southern California).
Hatano, G., & Inagaki, K. (2000). Domain-specific constraints of conceptual
development. International Journal of Behavioral Development, 24(3), 267-275.
Hoffman, R. R. (1987). The problem of extracting the knowledge of experts from the
perspective of experimental psychology. AI magazine, 8(2), 53.
Hoffman, R. R., Crandall, B., & Shadbolt, N. (1998). A case study in cognitive task
COGNITIVE TASK ANALYSIS 132
analysis methodology: The Critical Decision Method for the elicitation of expert
knowledge. Human Factors, 40(2), 254-276.
Hoffman, R. & Militello, L. (2009). Perspectives on cognitive task analysis: Historical
origins and modern communities of practice. New York: Psychology Press.
Hoffman R., Woods D. (2000). Studying cognitive systems in context: Preface to the
special section. Hoffman, R. R., & Woods, D. D. (2000). Studying cognitive
systems in context: Preface to the special section. Human Factors, 42 (1), 1-7
Jackson, P. W. (1985). Private lessons in public schools: Remarks on the limits of
adaptive instruction. In M. C. Wang & H. J. Walberg (Eds.), Adapting instruction
to individual differences (pp. 66–81). Berkeley, CA: McCutchan.
Kane, T. J., Rockoff, J. E., & Staiger, D. O. (2008). What does certification tell us about
teacher effectiveness? Evidence from New York City. Economics of Education
Review, 27(6), 615-631.
Kirschner, P. A, Sweller, J., & Clark, R. E. (2006). Why minimally guided instruction
does not work: An analysis of the failure of constructivist, discovery, problem
based, experiential, and inquiry based teaching. Educational psychologist, 41(2),
75-86.
Kiuhara, S. A., Graham, S., & Hawken, L. S. (2009). Teaching writing to high school
students: A national survey. Journal of Educational Psychology, 101(1), 136.
Klein, G. A., Calderwood, R., & Macgregor, D. (1989). Critical decision method for
eliciting knowledge. Systems, Man and Cybernetics, IEEE Transactions on, 19(3),
462-472.
Lampert, M. (1985). How do teachers manage to teach? Perspectives on problems in
COGNITIVE TASK ANALYSIS 133
practice. Harvard Educational Review, 55(2), 178-195.
Lave, J., Chaiklin, S., & Lave, J. (1993). The practice of learning. Contemporary
Theories of Learning, 200.
Lim, N. (2015). Using incremental cognitive task analysis to capture expert writing
instruction in expository writing for secondary students (Unpublished doctoral
dissertation). University of Southern California, Los Angeles, CA.
MacArthur, C. A., & Philippakos, Z. (2010). Instruction in a Strategy for Compare
Contrast Writing. Exceptional Children, 76(4), 438-456.
McCutchen, D. (2011). From novice to expert: Implications of language skills and
writing-relevant knowledge for memory during the development of writing
skill. Journal of Writing Research, 3(1), 51-68.
Means, B., & Gott, S. (1988). Cognitive task analysis as a basis for tutor development:
Articulating abstract knowledge representations. In J. Psotka, L. D. Massey, & S.
A. Mutter (Eds.), Intelligent Tutoring Systems: Lessons Learned (pp. 35-58). Hills.
Merriam, S. B. (2009). Qualitative Research. A guide to Design and Implementation
(Revised and Expanded from Qualitative Research and Case Study Applications
in Education).
Merrill, M. D. (2006). Hypothesized performance on complex tasks as a function of
scaled instructional strategies. In J. Elen & R. E. Clark (Eds.), Handling
complexity in learning environments: Research and theory (pp. 265-281). Oxford,
UK: Elsevier Science Limited.
Merrill, M. D. (2002). First principles of instruction. Educational Technology Research
and Development, 50(3), 43-49.
COGNITIVE TASK ANALYSIS 134
Merrill, M. D. (1994). Instructional Design Theory. Englewood Cliffs, NJ: Educational
Technology Publications.
Merrill, M.D. (1983). Component display theory. In C. Reigeluth (ed.), Instructional
design theories and models. Hillsdale, NJ: Erlbaum Associates.
Merrill, M. D., Li, Z., & Jones, M. K. (1990). Second generation instructional design
(ID2). Educational Technology, 30(2), 7-14.
McQuitty, V. (2012). Emerging possibilities: A complex account of learning to teach
writing. Research in the Teaching of English, 46(4), 358-389
Militello, L. G., & Hoffman, R. R. (2008). The forgotten history of cognitive task
analysis. Proceedings of the Human Factors and Ergonomics Society Annual
Meeting, 52, 383- 387.
National Assessment Governing Board (September, 2010). Writing framework for the
2011 national assessment of educational progress. U.S. Government Printing
Office, Washington, DC.
National Center for Education Statistics . (2012). The nation's report card: Writing 2011
(NCES 2012–470). Washington , DC : Institute of Education Sciences, U.S.
Department of Education .
No Child Left Behind (NCLB) Act of 2001, 20 U.S.C.A. § 6301 et seq. (West 2003).
Ozer, F. (1998). Standards in teacher training. European Education, 30 (2), 20–25.
Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader.
Contemporary educational psychology, 8(3), 293-316.
Porter, A. C., Garet, M. S., Desimone, L., Yoon, K. S., & Birman, B. F. (2000). Does
COGNITIVE TASK ANALYSIS 135
professional development change teaching practice? Results from a three-year
study.
Read, S., & Landon-Hays, M. M. (2013). The knowing/doing gap: Challenges of
effective writing instruction in high school. Teaching/Writing: The Journal of
Writing Teacher Education, 2(2), 3.
Saphier, J. (2011). Outcomes: Coaching, teaching standards, and feedback mark the
teacher’s road to mastery. Journal of Staff Development, 32(4), 58-62.
Schaafstal, A., Schraagen, J. M., & van Berl, M. (2000). Cognitive task analysis and
innovation of training: The case of structured troubleshooting.Human Factors:
The Journal of the Human Factors and Ergonomics Society, 42(1), 75-86.
Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information
processing: I. Detection, search, and attention. Psychological review, 84(1), 1.
Schraagen, J. M., Chipman, S. F., & Shalin, V. L. (Eds.). (2000). Cognitive task analysis.
Psychology Press
Simon, H. A., & Chase, W. G. (1973). Skill in chess: Experiments with chess-playing
tasks and computer simulation of skilled performance throw light on some human
perceptual and memory processes. American scientist, 394-403.
Smith, K. (2005). Teacher educators’ expertise: What do novice teachers and teacher
educators say? Teaching and Teacher Education, 21(2), 177-192.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive
science, 12(2), 257-285.
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory in perspective. In
Cognitive Load Theory (pp. 237-242). Springer New York.
COGNITIVE TASK ANALYSIS 136
Taylor, B. M., & Beach, R. W. (1984). The effects of text structure instruction on middle
grade students' comprehension and production of expository text. Reading
Research Quarterly, 134-146.
Tjiam, I. M., Schout, B. M., Hendrikx, A. J., Scherpbier, A. J., Witjes, J. A., & Van
Merriënboer, J. J. (2012). Designing simulator-based training: An approach
integrating cognitive task analysis and four-component instructional
design. Medical teacher, 34(10), e698-e707.
Tofel-Grehl, C., & Feldon, D. F. (2013). Cognitive task analysis–based training: A
meta-analysis of studies. Journal of Cognitive Engineering and Decision Making,
7(3), 293-304.
Tolano-Leveque, M. (2010). Using cognitive task analysis to determine the percentage of
critical information that experts omit when describing a surgical procedure.
University of Southern California.
Tulving, E., & Schacter, D. L. (1990). Priming and human memory systems.
Science, 247(4940), 301-306.
van Merriënboer, J. J., Clark, R. E., & De Croock, M. B. (2002). Blueprints for complex
learning: The 4C/ID-model. Educational Technology Research and Development,
50(2), 39-61
Vandevoort, L. G., & Berliner, D. C. (2004). National board certified teachers and
their students' achievement. Education policy analysis archives, 12(46), n46.
Velmahos, G. C., Toutouzas, K. G., Sillin, L. F., Chan, L., Clark, R. E., Theodorou, D., &
Maupin, F. (2004). Cognitive task analysis for teaching technical skills in an
COGNITIVE TASK ANALYSIS 137
inanimate surgical skills laboratory. The American Journal of Surgery, 187(1),
114-119.
Wheatley, T. and Wegner, D. M. (2001). Automaticity of action, psychology of. In
International Encyclopedia of the Social and Behavioral Sciences, edited by N. J.
Smelser and P. B. Baltes, pp. 991–993. Oxford: Elsevier
Wheatley, T. P., Wegner, D. M., Smelser, N. J., & Baltes, P. B. (2001). Automaticity in
action.
Woods, D. D., & Roth, E. M. (1988). Cognitive engineering: Human problem solving
with tools. Human Factors: the Journal of the Human Factors and Ergonomics
Society, 30(4), 415-430.
Yates (2007). Towards A Taxonomy of Cognitive Task Analysis Methods: A Search for
Cognition and Task Analysis Interactions. (Doctoral dissertation).
Yates, K. A., & Feldon, D. F. (2011). Advancing the practice of cognitive task analysis: a
call for taxonomic research. Theoretical Issues in Ergonomics Science, 12(6),
472-495.
Zumbrunn, S., & Krause, K. (2012). Conversations with leaders: Principles of effective
writing instruction. The Reading Teacher, 65(5), 346-353.
Zepeda-McZeal, D. (2014). Using cognitive task analysis to capture expert reading
instruction in informational text for students with mild to moderate learning
disabilities (Doctoral dissertation, UNIVERSITY OF SOUTHERN
CALIFORNIA)
COGNITIVE TASK ANALYSIS 138
Appendix A
Cognitive Task Analysis Interview Protocol
Begin the Interview: Meet the Subject Matter Expert (SME) and explain the
purpose of the interview. Ask the SME for permission to record the interview.
Explain to the SME the recording will be only used to ensure that you do not miss
any of the information the SME provides.
Name of task(s): Teaching expository writing
Performance Objective:
Ask: “What is the objective of teaching expository writing? What action verb should be used?”
Step 1:
Objective: Capture a complete list of outcomes for expository writing instruction.
A. Ask the Subject Matter Expert (SME) to list outcomes when these tasks are complete.
Ask them to make the list as complete as possible.
B. Ask SME how the outcomes are assessed.
Step 2:
Objective: Provide practice exercises that are authentic to the task of teaching
expository writing.
A. Ask the SME to list all the tasks that are performed during expository writing
instruction.
B. Ask the SME how the tasks would change when teaching expository writing among
various student skill levels.
Step 3:
Objective: Identify main steps or stages to accomplish the task
A. Ask SME the key steps or stages required to accomplish the task.
B. Ask SME to arrange the list of main steps in the order they are performed, or if there is
no order, from easiest to difficult.
Step 4:
Objective: Capture a list of “step by step” actions and decisions for each task
A. Ask the SME to list the sequence of actions and decisions necessary to complete the
task and/or solve the problem
COGNITIVE TASK ANALYSIS 139
Ask: “Please describe how you accomplish this task step-by-step, so a first year teacher
could perform it.”
For each step the SME gives you, ask yourself, “Is there a decision being made by the
SME here?” If there is a possible decision, ask the SME.
If SME indicates that a decision must be made…
Ask: “Please describe the most common alternatives (up to a maximum of three) that
must be considered to make the decision and the criteria first year teachers should use
to decide between the alternatives”.
Step 5:
Objective: Identify prior knowledge and information required to perform the task.
A. Ask SME about the prerequisite knowledge and other information required to perform
the task.
1. Ask the SME about Cues and Conditions
Ask: “For this task, what must happen before someone starts the task? What prior
task, permission, order, or other initiating event must happen? Who decides?”
2. Ask the SME about New Concepts and Processes
Ask: “Are there any concepts or terms required of this task that may be new to the
first year teacher?”
Concepts – terms mentioned by the SME that may be new to the first
year teacher
Ask for a definition and at least one example
Processes - How something works
If the first year teacher is operating equipment, or working on a team that
may or may not be using equipment, ask the SME to “Please describe how the
team and/or the equipment work - in words that a first year teacher will
understand. Processes usually consist of different phases and within each
phase, there are different activities – think of it as a flow chart”
Ask: “Must first year teachers know this process to do the task?” “Will they
have to use it to change the task in unexpected ways?”
IF the answer is NO, do NOT collect information about the process.
3. Ask the SME about Equipment and Materials
Ask: “What equipment and materials are required to succeed at this task in
routine situations? Where are they located? How are they accessed?
4. Performance Standard
COGNITIVE TASK ANALYSIS 140
Ask: “How do we know the objective has been met? What are the criteria, such
as time, efficiency, quality indicators (if any)?”
5. Sensory experiences required for task
Ask: “Must first year teachers see, hear, smell, feel, or taste something in order
to learn any part of the task? For example, are there any parts of this task they
could not perform unless they could smell something?”
Step 6:
Objective: Identify problems that can be solved by using the procedure.
A. Ask the SME to describe at least one routine problem that the first year teacher
should be able to solve if they can perform each of the tasks on the list you just
made.
Ask: “Of the task we just discussed, describe at least one routine problem that the first
year teacher should be able to solve IF they learn to perform the task”.
COGNITIVE TASK ANALYSIS 141
Appendix B
Inter-rater Reliability Code Sheet for SME A
COGNITIVE TASK ANALYSIS 142
Appendix C
Job Aid for Developing a Gold Standard
Richard Clark and Kenneth Yates (2010, Proprietary)
The goals of this task are to 1) aggregate CTA protocols from multiple experts to create a
“gold standard” protocol and 2) create a “best sequence” for each of the tasks and steps
you have collected and the best description of each step for the design of training.
Trigger: After having completed interviews with all experts and capturing all goals,
settings, triggers and all action and decision steps from each expert – and after all experts
have edited their own protocol.
Create a gold standard protocol
STEPS Actions and Decisions
1. For each CTA protocol you are aggregating, ensure that the transcript line number
is present for each action and decision step.
a. If the number is not present, add it before going to Step 2.
2. Compare all the SME’s corrected CTA protocols side-by-side and select one
protocol (marked as P1) that meets all the following criteria:
a. The protocol represents the most complete list of action and decision steps.
b. The action and decisions steps are written clearly and succinctly.
c. The action and decision steps use the most accurate language and
terminology.
3. Rank and mark the remaining CTA protocols as P2, P3, and so forth, according to
the same criteria.
4. Starting with the first step, compare the action and decision steps of P2 with P1
and revised P1 as follows:
a. IF the step in P2 has the same meaning as the step in P1, then add “(P2)”
at the end of the step.
b. IF the step in P2 is a more accurate or complete statement of the step in P1,
THEN revise the step in P1 and add “(P1, P2)” at the end of the step.
c. IF the step in P2 is missing from P1, THEN revise the list of steps by
adding the step to P1 and add “(P2N)”
at the end of the step.
5. Repeat Step 4 by comparing P3 with P1, and so forth for each protocol,
6. Repeat Steps 4 and 5 for the remaining components of the CTA report, such as
triggers, main procedures, equipment, standards, and concepts to create a “Draft
Gold Standard” protocol (DGS).”
7. Verify the DGS protocol by either:
a. Asking a senior SME, who has not been interviewed for a CTA, to review
the DGS and note any additions, deletions, revisions, and comments.
b. Asking each participating SME to review the DGS, and either by hand or
using MS Word Track Changes, note any additions, deletions, revisions,
or comments.
COGNITIVE TASK ANALYSIS 143
i. IF there is disagreement among the SMEs, THEN either:
1. Attempt to resolve the differences by communicating with
the SMEs, OR
2. Ask a senior SME, who has not been interviewed for a
CTA, to review and resolve the differences.
8. Incorporate the final revisions in the previous Step to create the “Gold Standard”
protocol.
COGNITIVE TASK ANALYSIS 144
Appendix D
SME A Protocol Flowchart
Begin Procedure 1:
Assess student prior
knowledge and plan for
student instruction.
Have students write to
a prompt without
guidance for one class
period.
Review papers for
grammatical errors.
Consult list of areas of
needed improvement.
Decide which can best be
addressed in upcoming
literature unit.
Procedure 1
continued on
page 132
Collect papers and
identify areas of
needed
improvement for
future instruction.
If less than 70% of
papers show an area of
needed improvement,
then consider revisiting
these topics as time
allows throughout year.
If more than 70% of
papers show an area of
needed improvement,
then identify and list this
area for future instruction.
COGNITIVE TASK ANALYSIS 145
If lit unit has unfamiliar
background information,
archaic language, or
needs scaffolding for
student comprehension,
then choose few topics to
cover during writing
instruction.
Move into literature unit
as prescribed by grade
level or site pacing
guide.
Procedure 1
continued from
page 131
If lit unit has more familiar
background information,
modern language, or needs
less teacher scaffolding for
comprehension, then
choose more topics to
cover during writing
instruction.
Pick 2-3 areas of
needed
improvement to
cover during
literature unit.
Prepare examples of
student writing for
instruction at the
conclusion of lit unit.
End Procedure 1
COGNITIVE TASK ANALYSIS 146
Begin Procedure 2:
Complete literature
unit/ Demonstrate
thinking process
Provide students a copy of
this analysis essay to show
how literature is
incorporated into a
written argument.
Use critical literary
analysis as content for
literature unit.
Discuss with students how literary
analysis essay incorporates
literature into written argument.
Explain the elements that make
this piece a good writing example.
Find a critical literary
analysis essay of the type
of literature being taught.
Move into
reading of
literature.
End Procedure 2
COGNITIVE TASK ANALYSIS 147
Prepare a writing
question or prompt to
which students will
address the unit content.
Reverse engineer this prompt
by walking students through the
thought process that resulted in
prompt. Prepare to show
students questions and
authentic thinking that led to
creation of prompt.
Procedure 3
continued on
page 135
Tell student that an essay is a
physical manifestation of a thought.
Convey to students that writing an
essay is a demonstration of a
problem through writing and the
solution that comes from thinking.
Begin Procedure 3:
Present topic as a
problem to solve.
Use Socratic Questioning (SQ) to
lead students to ideas, thoughts,
and values applicable to answering
a question or choosing a side of an
argument.
COGNITIVE TASK ANALYSIS 148
Procedure 3
continued from
page 134
Use SQ and brainstorming
to come up with central idea
for paper.
Determine when to begin
Socratic Questioning.
If you are teaching lit
units in the beginning
of the school year,
then engage in SQ and
brainstorming after
completing lit unit.
If you are teaching lit
units in the middle of
the school year, then
engage in SQ and
brainstorming during
the lit unit.
If you are teaching lit
units in the 2
nd
semester
of the school year OR
during the research unit,
then engage in SQ and
brainstorming at the
beginning of the lit unit.
Engage in brainstorming for
about ½ a class period.
Demonstrate to students that
this thinking leads to a question
that can be responded to through
writing an expository piece.
End Procedure 3
COGNITIVE TASK ANALYSIS 149
Present to students examples
from diagnostic writing piece
that uncovered areas of
weakness.
Let students know that they
will begin writing papers the
next day and should keep
these suggestions and
approaches in mind as they
write.
Show to students written
examples of areas of needed
improvement.
Provide suggestions and
approaches on how
students can fix these areas
of concern.
Use discussion to show
students they can improve
future writing pieces.
Procedure 4
continued on
page 137
Begin Procedure 4:
Provide feedback from
diagnostic papers.
COGNITIVE TASK ANALYSIS 150
Procedure 4
continued from
page 136
Remind students that a thesis
statement is the main argument and
has three parts:
The paper needs a clear
designation of what it is the
student is going to talk about.
The paper is making a clear
judgment about something.
The paper has chosen a side to
support and that it is
meaningful in some way.
If students are in AP class,
then these three
components are given
more direct emphasis.
If students are in a college
prep (CP) class, then the
instructor attempts to
guide students toward
this goal.
End Procedure 4
COGNITIVE TASK ANALYSIS 151
If you determine class
needs additional
support, then develop
writing topic as a
problem together.
Note: research project is done only
once in the year for CP students and 2-
3 times a year for AP students. This is
because of limited research resources
on high school campuses.
Introduce research assignment later
in school year.
Engage in SQ and brainstorming to
develop ideas and thoughts that lead
to presenting a writing topic as a
problem to address.
Procedure 5
continued on
page 139
Begin Procedure 5:
Research Project
Require students
to come up with
their own writing
topic based on SQ
and subsequent
discussion.
If you determine class
is capable, then
instruct students to
come up with their
own individual writing
topic.
COGNITIVE TASK ANALYSIS 152
Instruct and model annotation and
note taking as students pull
supporting facts from larger texts
and critical literary analysis papers.
Require students to perform
research to find how research
questions are phrased and how
research questions are answered.
Tell students they are now required
to do their own brainstorming and
thinking process as explained and
exemplified throughout school year.
Procedure 5
continued from
page 138
Remind students that MLA format will
be part of final grade for research
project. It will be a separate grade
apart from the content writing.
End Procedure 5
COGNITIVE TASK ANALYSIS 153
Give students time to write.
Give time based on assignment
length (2-7 days).
Circulate during this time and
provide help and individualized
instruction as needed.
Procedure 6
continued on
page 141
Begin Procedure 6:
Conduct Writer’s
Workshop
Provide support by checking for
thesis statements.
Provide support by checking
first body paragraphs.
Provide support (concrete
details) by helping students find
support from text or from
secondary sources.
COGNITIVE TASK ANALYSIS 154
Yes No
Provide support by helping students
identify and narrow their central idea or
argument.
Provide support by helping students
check how all the elements noted above
are incorporated into the text.
Procedure 6
continued from
page 140
If students are
comfortable with
the direction of
their paper and
have proven to
be capable
writers, then
leave them alone
to write
If students lack
confidence, or
have shown the
need for more
support, then
have these
students bring
every completed
paragraph to you
for review.
Refrain from providing concrete guidelines
on the amount of quotes or support needed
by students in writing their arguments. Tell
them they need to collect and use enough to
support their points.
Are students
comfortable
with
direction of
their paper?
During Writer’s Workshop, allow
students to support one another and
provide peer reflection.
Procedure 6
continued on
page 142
COGNITIVE TASK ANALYSIS 155
Use Writer’s Workshop as a
time to customize feedback to
individual students during their
time of actual writing.
Your feedback might include suggestions to
improve argumentation, to improve evidence, to
improve analysis, to improve transitions
between sentences and paragraphs, to improve a
more sophisticated inclusion of quotations, and
to help clarify what students are trying to say.
At the conclusion of
Writer’s Workshop, give
students and additional 3
days to polish and type
papers before due date.
Procedure 6
continued from
page 141
If students are in
an AP class, then
all papers
completed in 2
nd
semester are
timed in-class
writing in
preparation for the
upcoming AP test.
Remind students papers
must be typed in MLA format
(taught in previous grades).
Review MLA format issues
with using citations.
End Procedure 6
COGNITIVE TASK ANALYSIS 156
Collect student papers for
teacher evaluation.
Put off grading for a few days
to allow teacher to be more
objective in evaluation.
Remind yourself what areas
of needed improvement were
to be evaluated in this paper.
Procedure 7
continued on
page 144
Begin Procedure 7:
Assess the papers.
Remind yourself that evaluation
is based on whether student
writers supported their
arguments, wrote arguments
that made sense, and wrote
arguments that are justifiable.
COGNITIVE TASK ANALYSIS 157
Consider during evaluation the
differences between individual
classes in a particular year, the age
group of students, and the
personalities and approaches of
students individually and as a group.
Grade student work holistically.
Be prepared to accept arguments
that might be wrong as long and
they have support or are
justifiable.
Procedure
7continued on
page 145
Procedure 7
continued from
page 143
Remind yourself that evaluation
is based on whether student
writers supported their
arguments, wrote arguments
that made sense, and wrote
arguments that are justifiable.
Grade student work in a holistic
way. Be prepared to accept
arguments that might be wrong
as long and they have support or
are justifiable.
COGNITIVE TASK ANALYSIS 158
Yes
Procedure 7
continued from
page 144
Consider the distance each writer
has grown and moved towards an
acceptable level of proficiency.
Consider letting grades reflect effort for
students who have improved their skill
dramatically. These grades will not affect
external evaluation but can be used to
boost the confidence of struggling writers.
Assess grammar
and punctuation.
Do these areas
affect what
students’
arguments?
If grammar and
punctuation do
not affect
students’
arguments, then
they are not
reflected in final
grade of paper.
If grammar and
punctuation
make the paper
unclear, then
they will be
reflected in final
grade of paper.
No
End Procedure 7
COGNITIVE TASK ANALYSIS 159
Begin Procedure 8:
Guide revision of
papers.
Hand back
graded work to
students.
IF students wrote
a paper that is
not acceptable,
then mark the
grade as ‘R”
meaning a
revision is
required. This
will remain until
the revision is
turned in
If students wrote
an acceptable
paper but are
unhappy with
their grades, then
provide
opportunity to
revise paper.
Allow for multiple revisions as
needed or as time requires. This
step is up to students to complete.
When revising with students,
look for problems with
argumentation and support.
When revising with students,
look for arguments that have
too few quotations or citations.
Procedure 8
continued on
page 147
COGNITIVE TASK ANALYSIS 160
Procedure 8
continued from
page 146
When revising with students,
look for arguments that have
too few quotations or citations.
When revising with students,
look for papers that have not
done a good job of connecting
quotations to the argument.
Review revised papers and
adjust grade if appropriate.
Hand back graded revisions to
students.
End Procedure 8
COGNITIVE TASK ANALYSIS 161
Appendix E
Gold Standard Protocol
Teaching Expository Writing to 11
th
-12
th
grade students
These steps will guide 11
th
and 12
th
grade teachers in teaching expository writing
to students, preparing them for college entrance exams and college level writing.
Task: To teach 11
th
and 12
th
grade students how to write a well-reasoned, well-supported,
well-argued physical manifestation of students thought.
Objective: At the end of each unit, students will create an expository paper that
meets the standards described in the rubric or checklist.
Main Procedures:
1. Prepare to teach. (A1R, B1R, C1R)
2. Assess the student’s prior writing knowledge. (A1R)
3. Implement Literature Unit (A1R, B1R, C1R)
4. Present topic as a problem. (A1R361-363)
5. Introduce Prompt/ Prompt Analysis (A1R, B1R)
6. Provide Feedback from Diagnostic Papers (A1R)
7. Construct essay outline (P2N, 41R)
8. Provide Writing Instruction (A1R, B1R, C1R, 41R)
9. Conduct Writing Workshop (A1R, B1R, C1R)
10. Assess papers. (A1R, B1R, C1R)
11. Hand back papers and provide optional opportunities for further revision. (A1R,
B1R, C1R, 41R)
12. Take opportunities to publish student writing (3PN, 41R)
1. Procedure 1. Prepare to Teach. (A1R, B1R, C1R)
1.1. Develop lesson objectives based on district expectations and state academic
standards. (41R)(4N)
1.2. Choose the text (i.e., novels, short stories, article, etc.) you will use to teach
expository writing subject (A1R406) (C1R406) (P31R)
1.3. IF you choose to use a novel as your reading text, THEN follow steps 1.3.1 to
1.1.6.2.
1.3.1. Choose a novel to study. Follow parameters in novel selection set by
district requirements or by curriculum requirements (i.e., Advanced
Placement, International Baccalaureate, etc.) (A1R685) (B1R685,C1R422)
1.3.1.1. STANDARD: You selection of novels is often dictated by district
requirements. Your choices may be limited. (41R)(4N)
1.3.2. Find thematic connections between your various choices of novels.
(B1R688)
1.3.2.1. Consider teaching literature within a specific context (historical,
psychological, etc.). (B1R691) (P2N)
1.3.3. Think about selecting a text that interests you as a teacher and that you
might not have taught before. (B1R875) (P2N) (41R)
COGNITIVE TASK ANALYSIS 162
1.3.3.1. REASON: It is your passion as a teacher of literature that will
excite students and create interest in the novel. (B1R780) (P2N)
1.3.4. IF you are teaching a novel that you have already read, THEN review
annotations and notes from last year to see if they remain applicable to your
current students’ abilities and background knowledge. (B1R865) (P2N)
1.3.5. IF you are teaching a new novel for the year, THEN strive to annotate and
analyze novel ahead of students. Stay ahead of students’ reading progress.
(B1R877) (P2N)
1.3.6. Select ideas, themes, and issues within the novel that you would like to
cover with students. (B1R805-807) (P2N)
1.3.7. Select contemporary ideas, themes, and issues that connect current issues
to the novel’s historical context. (41R) (4N)
1.3.7.1. List them on inside of the cover of your novel. (B1R1094) (P2N)
(41R)
1.3.8. Assign color-coding to each feature and apply to list on the cover of your
novel. (B1R1089) (P2N)
1.3.8.1. During your annotating of the novel, place matching color-coded
sticky notes on pages within the novel where these ideas, themes, and
issues are addressed. (B1R1088-1092) (P2N)
1.3.8.2. REASON: This will help you identify for students particular
quotes or literary features that address particular prompts chosen for
this novel. (B1R1098) (P2N)
1.4. IF you choose to use a critical literary analysis as your reading text, THEN
follow steps 1.2.1 to 1.2.6.
1.4.1. Find piece of writing that is a critical literary analysis of the type of
literature taught. (A1R182-183) (P1)
1.4.1.1. STANDARD: These types of analysis are college level text
analyses of the novels read in class. (A1R182)(P1)
1.4.2. Search for critical literary analysis example from a suitable research
database or MLA Bibliography Database. (A2R183) Examples of critical
literary analysis might also be available from classroom textbooks.
(A1R185) (P1)
1.4.3. Choose an example of critical literary analysis that is approachable to a
college senior, based on teacher judgment. (A1R184) (P1)
1.4.4. Choose an example of critical literary analysis that is not too technical.
(A1R184) (P1)
1.4.5. Choose an example of critical literary analysis that is an academic analysis
of a portion of the text taught during literature unit. (A1R185) (P1)
1.5. Within the first few weeks of the school year, plan to teach different writing
modes (i.e. description, compare/contrast, narrative, or expository) and the
appropriate context for their use. (C1R844-845) (P3N)
1.5.1.1. REASON: Having knowledge of different modes of writing allows
students to vary their style that best conveys their thinking and address
topics in creative ways. (C1R836-847) (P3N)
COGNITIVE TASK ANALYSIS 163
1.6. IF you choose to teach argumentative writing, rhetorical devices, particular
writing skills, or are want to provide students a choice of topics, THEN follow
steps 1.5.1 to 1.3.11.1.
1.6.1. IF you are having students write an argumentative paper using a topical
current event to stimulate making connections and choosing a side to argue,
THEN choose articles that argue both sides of an argument. (C1R428-449)
(P3N)
1.6.1.1. STANDARD: Teach argumentative writing closer to the halfway
point of the school year. (31R865-866) (P3N)
1.6.2. IF you need to teach rhetorical features, style structures, or particular skills
that need to introduced or reviewed, THEN find and choose articles that
exemplify these features. (31R428-449) (P3N)
1.6.3. IF writing assignment is student choice, THEN allow students the freedom
to write on any topic they choose. (A1R834) You can give student choice
writing assignments anytime during the school year. (C1R834) (P3N)
1.6.3.1. STANDARD: Remember the goal is to teach students to be
metacognitive in their approach to writing by teaching themselves how
to write. (C1R474-476) (P3N) (41R)
1.7. Collect or develop prompts for students to analyze the literary text assigned.
(41R)(4N)
1.7.1. If students are reading literary text, THEN also assign a contemporary
review of the novel. (P3N)
1.7.1.1. Choose or create prompts that ask students to explain or justify
why they agree or disagree with the contemporary review using the
novel for support. (C1R414-417) (P3N)
1.7.2. Choose or create prompts using one or more of the following criteria.
(41R)(4N)
1.7.2.1. Find or develop writing questions or prompts based on ideas,
themes, and issues to be covered during reading of text. (A1R266-279,
B1R802-804, C1R693-694) (P1, P2, P3)
1.7.2.2. Choose or create prompts that address character development.
(B1R802) (P2N)
1.7.2.3. Choose or create prompts that address setting. (B1R1081) (P2N)
1.7.2.4. Choose or create prompts that address the historical context from
which the literary piece is based. (B1R333) (P2N)
1.7.2.5. Choose or create prompts that address the historical context of
when the literary piece was written and published. This context may
also address current historical experience as compared to the literary
piece. (B1R333) (P2N)
1.7.2.6. Choose or create prompts that address literary features. (B1R803)
(P2N)
1.7.2.7. Choose or create prompts that address psychological trauma.
(B1R804) (P2N)
1.7.2.8. Choose or create prompts that address universal feelings (i.e.,
betrayal, courage, compassion, etc.) (B1R554) (P2N)
COGNITIVE TASK ANALYSIS 164
1.7.3. Collect prompts from Internet sources or college course sources.
(A1R238) If needed, reverse engineer prompts to make them work for your
particular class. (AIR420, B1R820) (P1) (P2)
1.7.4. Prepare to walk students through the thought process that resulted in the
prompt. (A1R420-449)
1.7.5. Allow students, with your prior approval (41R), to come up with their own
prompts that address the text. (B1R847) (P2N)
1.7.6. Prepare to provide instruction in prompt analysis using the prompts
collected to give students practice in breaking apart writing prompts (See
Procedure 5). (B1R240) (P2N)
1.7.7. IF you have a College Prep (CP) class, THEN prepare to provide
scaffolding and direct instruction throughout unit to support students as
progress through literature unit and writing assignment. (C1R1523) (P3N)
(41R)
1.7.8. IF you have an Advanced Placement (AP) class, THEN consider
providing less scaffolding and allow students more latitude in making their
own writing choices. (C1R1500-1502) (P3N)
1.8. Create a rubric. (A1R, B1R, C1R)
1.8.1. Consider what rubric will be used to assess writing assignments.
(A1R172-173) (P3N)
1.8.2. Use lesson objectives to determine what gets assessed. (C1R171)(P3N).
1.8.2.1. IF you are assessing your students’ thinking, THEN create a
prompt that assess students’ metacognitive processing used in
developing their writing. (A1R172-173) (P3N) (41R)
1.8.2.2. IF you want to assess students’ argumentation based on discussion
held in class, THEN build rubric with students. (See steps 3.8)
(31R692) (P3N)
1.8.3. Prepare to give rubric to students before writing begins. (C1R698, 708)
(P3N) (41R)
2. Procedure 2. Assess the student’s prior writing knowledge. (A1R)
2.1. By the third day of school, have students write to a prompt without guidance for
one class period. (A1R140) (P1)
2.1.1. STANDARD: IF students are in AP (advanced placement) class, they
have read two novels assigned as summer reading. Have students respond in
writing to one prompt that can be applied to any one of the novels read.
(A1R122-126) The reading level of these texts is grade level appropriate.
(A2R126)(P1)
2.1.2. STANDARD: IF students are in CP (college prep) classes, have students
respond to one of 2-3 prompts based on readings from the prior school year.
(A1R126-132) These texts are one grade level below their current grade
level. CP student begin to read grade level texts within the first two weeks of
school. (A2R132) (P1)
2.2. Collect papers and identify areas needing writing instruction as determined by
teacher experience (based on academic level and age level of students) that will
be addressed in future instruction. (A1R141, 144-148) (P1)
COGNITIVE TASK ANALYSIS 165
2.2.1. IF more than 70% of papers show particular area(s) needing improvement,
THEN identify and list each area of needed improvement for future lessons.
(A1R144-146) (P1)
2.2.2. IF less than 70% of papers have a particular area needing improvement,
THEN consider revisiting these topics as time allows throughout year.
(A1R144-146) (P1)
2.3. STANDARD: The following are often areas of concern based on experience:
(A1R171) (P1)
2.3.1. Writing that does not answer question or prompt completely (A1R162)
(P1)
2.3.2. Author chooses no side to argue. (A1R163) (P1)
2.3.3. Writing that does not have support from text. (A1R165) (P1)
2.3.4. Students retell plot without analysis (41R)(4N)
2.4. Review papers for grammatical errors (162-166) (A1R166) (P1)
2.4.1. IF a student’s paper shows adequate knowledge of grammar, THEN paper
will be handed back to student without comment regarding
grammar.(A1R162) (P1)
2.4.2. IF an individual student’s paper shows an overabundance of grammar
mistakes, THEN teacher provides grammar instruction to individual students
either through comments on paper or with a short conference when papers
are handed back to students. (A1R163-165) (P1)
2.4.3. If a preponderance of student writing show a need for a particular
grammar topic to be reviewed, THEN explicitly review grammar topic in
whole class instruction. (A1R163) (P1)
2.5. Consult list of areas of student writing that need improvement. Decide which can
be best addressed in upcoming literature unit. (A1R201-202) (P1)
2.5.1. Choose 2-3 areas to cover during each writing instruction to teach within
chosen literature unit. (A1R201) (P1) (41R)
2.5.2. IF the literature text has background information that is unfamiliar to
students, archaic language, and/or needs teacher scaffolding for
comprehension, THEN choose fewer areas of needed improvement to cover
during subsequent writing instruction. (A1R200-201) (P1)
2.5.3. IF literature unit has more familiar background information, modern
languages, and/or needs less explanation for comprehension, THEN cover
more areas of needed writing improvement during subsequent writing
instruction. (A1R202-203) (P1)
2.6. Gather examples of student writing that exemplify areas of needed improvement
and mentor texts that exemplify quality writing to use during writing instruction
(see step 8.4). (A1R170-171) (P1) (41R)
2.7. Move into first literature unit as prescribed by grade level or site pacing.
(A1R182) (P1)
3. Implement Literature Unit. (A1R, B1R, C1R)
3.1. Assign text (article, blog, novel, etc.) for students to begin reading. (A1R182,
B1R1188, C1R 193)
3.2. Define for students that the term “text” may refer to any written work. (C1R193-
194) (P3N)
COGNITIVE TASK ANALYSIS 166
3.2.1. Instruct students to use text to gather ideas throughout reading of text.
(C1R199) Students may gather ideas from:
3.2.1.1. Ideas or text read by student in their daily readings. (C1R192)
(P3N)
3.2.1.2. Ideas or thoughts from conversations during the day. (C1R193)
(P3N)
3.2.1.3. Things or human interactions observed by the student. (C1R193)
(P3N)
3.2.1.4. Clarify ideas that they have collected. (41R)(4N)
3.2.2. Guide students through class discussion on making connections between
these gathered thoughts and ideas and to what they are reading. (C1R195)
(P3N)
3.2.2.1. Discuss with students that connections between what they have
gathered and those within the assigned text can be made in the
following ways (C1R201-202) (P3N)
3.2.2.2. Connections of multiple ideas within the text itself. (C1R196-197)
(P3N)
3.2.2.3. Connections to other texts read. (C1R197) (P3N)
3.2.2.4. Connection to things seen, heard, or read outside of class.
(C1R198) (P3N)
3.2.3. Discuss with students how these connected ideas can create new ideas.
(C1R201) Model this by taking your own thoughts, ideas, and observations
and connect them to the readings assigned or to what you might be reading
personally at the time. (C1R198) (P3N)
3.3. IF you are using a novel as your reading text, THEN provide students a copy of
novel. (B1R717) (P2N) Encourage students to procure their own personal copy.
(41R)(4N)
3.3.1. REASON: Students with personal copies of texts will be able to annotate
right into the novel. Students with school copies will need to annotate in
notebook. (Lit log) (41R)(4N)
3.3.2. Before beginning to read the novel, pass out to students a list of prompts
from which they can choose to write. (B1R197) Tell students, “Start
thinking about which prompts you might want to write to and start collecting
your evidence now.” (B1R200-202) (P2N)
3.3.3. Pass out rubric at this time that will be used to assess the final paper unless
you are creating one with students (see step 3.8). The rubric created with
students will be developed before writing instruction begins. (B1R1056)
(P2N)
3.3.4. Assign reading to occur outside of class. You may read sections of the
novel in class to model Naked Reading (see step 3.1.16) or to support class
discussions. (B1R1188) (P2N)
3.3.5. Think about reading the novel whole-class. (B1R1190) (P2N)
3.3.5.1. REASON: Experience shows that many students are not reading
outside of class. (B1R1192) (P2N)
3.3.5.2. STANDARD: Having students reading in class gives them more
guided experience in reading complex texts. (B1R1195) (P2N) Time
COGNITIVE TASK ANALYSIS 167
spent reading in class reduces the amount of novels read throughout the
school year. (B1R1193) (P2N)
3.4. IF you are using a critical analysis of a literary work as your reading text, THEN
provide students a copy. (A1R175-177, B1R1313, C1R414-417) (P1, P2, P3)
3.4.1. Tell students that critical essays provide someone else’s words and
arguments to help support what you will be saying about the novel.
(B1R1401) (P2N)
3.4.2. STANDARD: Having students use critical essays requires more
scaffolding and support from the teacher. The academic writing is complex
and hard for students to understand. (B1R1381) (P2N)
3.5. As you read, remember to constantly assess student knowledge of the
subject/topics/themes being discussed. Assess students explicitly if they
understand what is being discussed. (B1R271) (P2N)
3.5.1. Use quizzes, objective tests, short answer responses, or on-demand writing
assignments. (41R)(4N)
3.5.1.1. STANDARD: Assessment can also be made informally during
classroom discussions. (B1R271) (P2N)
3.5.1.2. STANDARD: Formative Assessment can also be made by
assigning a prompt through a school-wide electronic discussion board.
(B1R90) (P2N)
3.5.2. Have students respond to a prompt writing their response in the form of a
body paragraph. (see step 8.11) Students submit a paragraph and then are
required to respond to at least two other student submissions. (B1R91) (P2N)
3.5.3. Look for evidence through monitoring discussion and discussion board
submissions that students are thinking deeply about the novel. (P2N)
3.5.4. Use class discussion, tests, quizzes. short answer response, and on-demand
writing assignments to help students develop thinking in the following ways:
(41R)(4N)
3.5.4.1. Help students develop deeper thinking in the lives of the
characters; (B1R328) (P2N)
3.5.4.2. Help students develop deeper thinking in the emotions of the
characters; (B1R328) (P2N)
3.5.4.3. Help students develop deeper thinking in the psychological
development of the characters. (B1R329) (P2N)
3.5.4.4. Help students develop deeper thinking in Contextual analysis.
(B2R329) (P2N)
3.5.4.5. Help students develop deeper thinking in the influence of setting
on characters. (B2R329) (P2N)
3.5.4.6. Help students see that they must connect the characters with
today’s world or the story’s context or their writing becomes superficial.
(B1R334) (P2N)
3.5.5. Discuss with students the big ideas identified during reading of the novel.
(B1R221) (P2N)
3.5.5.1. STANDARD: Remember that you as a teacher may have read this
novel multiple times while most students have not before entering your
classroom. Slow down for students. (B1R934) (P2N)
COGNITIVE TASK ANALYSIS 168
3.5.6. Be ready for diversions in class conversations as students take discussion
in different directions. (B1R222) (P2N)
3.6. Model annotation of literary text with students. (B1R903)
3.6.1. Model Naked Reading as an annotation strategy with students. Take an
unmarked page with no annotations and model Naked Reading. Naked
Reading is showing to students how you, as a reader, read a page for the first
time. (B1R892-897) (P2N)
3.6.1.1. Think out loud to students as you hold discussion between yourself
and the text about what inspires you within the page being read.
(B1R893, 901) (P2N)
3.6.1.2. STANDARD: This metacognitive act of reflective reading must be
modeled often for students. (B1R910) (P2N)
3.6.1.3. REASON: Annotation trains students to be deeper readers.
Students begin to develop a system to keep track of their information
while reading. This strategy can also transfer to other college subjects.
(B2R374-375) (P2N)
3.6.2. Instruct students to annotate create a color code list inside cover of their
novels. (B2R375) (P2N)
3.6.2.1. Students annotate their novels and the critical essays using sticky
notes and color coding. (B1R374-375) (P2N)
3.6.2.2. Provide to students or have them bring sticky notes and
highlighters to color code their annotations. (41R)(4N)
3.6.2.2.1. STANDARD: Students with school copies of texts must
record annotations in notebook and not in texts. (B1R718) (P2N)
3.6.3. Work with students during reading to identify those concrete details or
literary features of the story’s themes (pre-identified by you during planning
phase) that will be useful in answering the prompts assigned or chosen by
students. (B1R1168) (P2N)
3.6.3.1. IF a prompt addresses literary features within the novel, THEN
students will identify and highlight instances where these features
manifest themselves in the novel. (B1R1132) (P2N)
3.6.3.2. IF a prompt addresses a theme or motif within the novel, THEN
students will identify concrete details that exemplify this theme.
(B1R1130) (P2N)
3.7. IF you are using a critical literary essay as your text, THEN do the following:
3.7.1. Discuss with students how critical literary analysis piece incorporates
literature into written argument. (A1R176) (P1)
3.7.2. Give instruction on the elements that make this literary analysis piece a
good writing example. (A1R246) (P1)
3.7.3. Direct students to keep literary analysis piece as a job aid for future
writing assignments throughout year. (A1R191-192) (P1)
3.7.4. Work with students to find the following from the critical essays assigned:
(P2N)
3.7.4.1. Find evidence supporting writer’s claims. (B1R1367) (P2N)
3.7.4.2. Find counterarguments from essays to give students perspectives
they might not have considered yet. (B1R1404) (P2N)
COGNITIVE TASK ANALYSIS 169
3.7.4.3. Ask students to consider if the critic has said something that the
student can use to support their own thesis. (B1R1405) (P2N)
3.7.5. Have students color code what the literary critic said about the text as it
relates to the prompt. (B1R1314) (P2N) (41R)
3.7.6. As an alternative annotation strategy, teach students how to annotate using
SOAPSTone. (P3N)
3.7.6.1. STANDARD: SOAPSTone is an acronym for the following: (P3N)
3.7.6.1.1. Speaker- persona of the writer “How do they want to come
across to audience? (C2R955)(P3N)
3.7.6.1.2. Occasion-what prompted the piece? (C2R955)(P3N)
3.7.6.1.3. Audience-for whom was it written? (C2R955)(P3N)
3.7.6.1.4. Purpose – what is the main point or idea? (C2R955)(P3N)
3.7.6.1.5. Subject-what are they talking about? (C2R955)(P3N)
3.7.6.1.6. Tone-speaker’s attitude towards the audience and/ or
subject (C1R955-960) (P3N)
3.7.7. Teach students in the beginning of the school year how to use
SOAPSTone. (C1R941) (P3N)
3.7.8. Use visual aids such as photography or artwork to introduce SOAPSTone
to students. (C1R962) (P3N)
3.7.8.1. Provide examples of photographs or art for students to analyze.
Discuss with students the choices the photographer makes while
applying SOAPSTone as an analytic framework. (C1R963) (P3N)
3.7.9. Once this has been practiced, guide students in applying SOAPSTone at
the textual level. (C1R966) (P3N)
3.7.9.1. STANDARD: Tell students that anything and everything is making
an argument and that argument is not an afterthought. Students can
observe anything and find within it an argument about the object or
person. (C1R977) (P3N)
3.8. IF students are writing an argumentative paper responding to arguments in
opinion pieces, THEN do the following: (P3N) (41R)
3.8.1. Ask students to find and read opinion pieces with different takes on the
same subject. (C1R871-872, 883) (P3N)
3.8.1.1. STANDARD: Choose articles are to from major newspapers (LA
Times, NY Times, etc.) or from a syndicated columnists. (C1R874-
875) (P3N)
3.8.1.2. REASON: These articles have been through an editing process.
Writers for these publications tend to write more polished pieces.
(C1R877-880) (P3N)
3.8.2. Assign students to write on that subject by taking a side using those
sources to support their argument. (C1R388-392) (P3N)
3.8.3. Tell students to find articles that support counterclaims or counter
arguments to the stand they have chosen. (C1R896-897) (P3N)
3.9. If you want to assess student writers’ knowledge of argumentation, THEN build
a rubric with students using the following questions to stimulate discussion (see
procedure 3.2.3): (31R) (3N)
COGNITIVE TASK ANALYSIS 170
3.9.1. While building the rubric, ask students, “What did we just learn about
argumentative writing?” (31R)
3.9.2. Ask students, “What shall we expect in this paper?” (31R)
3.9.3. Ask students, “What should that look like?”(C1R693-696) (P3N)
3.10. IF students are reading articles as examples of rhetorical devices, THEN
work with the class to identify and analyze the rhetorical device. (P3N)
3.10.1. Have students analyze how that rhetorical device contributes to or creates
the author’s purpose. (C1R1001) (P3N)
3.10.2. Have students analyze how that rhetorical device helps connect the article
to the audience. (C1R1001) (P3N)
3.10.3. Have students analyze how that rhetorical device connects readers to the
pathos (emotional appeal), ethos (ethical appeal), or logos (logical appeal) of
the article and its effect on the text. (B2R456) (P3N)
3.11. IF students are struggling with a particular writing skill (i.e., transitions,
theme, narrative, etc.), THEN have students read articles that contain examples
of this skill and analyze its use and effect in the text. (B2R456-458) (P3N)
3.12. Consider blogging (see procedure 12) for students to use as an additional
writing exercise (P3N)
3.12.1.1. REASON: Required blogging forces students to practice often in
taking a blank page to connect ideas and bring them into a new and
interesting shaping of ideas. (C1R1414-1415) (P3N)
3.12.2. Instruct students to keep a notebook of themes and ideas they learn from
reading various texts. (31R)(3N)
3.12.2.1. STANDARD: Students use this notebook throughout the year to
collect ideas and themes (4N) for future writing pieces. (C1R468)
3.13. Teach students to discover their own needs within their writing by asking
the following questions. (C1R488) (P3N)
3.13.1. Teach them to ask, “What are you learning from other writers?” (C1R494)
(P3N)
3.13.2. Teach them to ask, “What do I notice that the writer does in the article
(book, novel, etc.)?” (C1R469) (P3N)
3.13.3. Teach them to ask, “How does the writer accomplish this? (C1R469)
(P3N)
3.13.4. Teach them to ask, “What can I take from this article? What can I learn
from the author?” (C1R470) (P3N)
3.13.5. Teach them to ask, “What can you do in your writing? Did the writer
conform to standard writing conventions, or did he/she break the rules? Does
it work in the article?” (C2R496) (P3N)
3.13.6. Teach them to ask, “What did the writer craft in his/her article that
surprised you?” (C2R496) (P3N)
3.13.7. Teach them to ask, “What does the writer want you to know?” (4N)
4. Present topic as a problem. (A1R361-363)
4.1. Tell students that an essay is a physical manifestation of a thought. (A1R26)
Convey to students that writing an essay is a demonstration of a problem through
writing and the solution comes from thinking. (A1R278-279, C1R143) (P1) (P3)
COGNITIVE TASK ANALYSIS 171
4.2. Instruct students that they can learn to write better by what they read. (C1R498)
(3PN)
4.3. Use Socratic Questioning (SQ) to lead students to ideas, thoughts, and values
applicable to coming up with a answering a question or choosing a side in an
argument. (A1R266-267) (P1)
4.3.1. STANDARD: Socratic Questioning is a strategy of asking questions of
students to which you already know the answer. The outcome of this line of
questioning is meant to lead students to a desired conclusion. (A2R267) (P1)
4.4. Determine when to begin Socratic Questioning (SQ) and developing a question
to address in an essay. Decide whether this brainstorm discussion comes before,
during, or after the literature unit. (A1R282-299, A1R334-347) (P1)
4.4.1. STANDARD: Use of SQ and brainstorming at varying points within
literature units provides a gradual release of responsibility for authentic
thinking from teacher to student. You as teacher provide more frontloading
and scaffolding, and direct instruction in the beginning of the year, and
slowly have students engage in their own thinking and brainstorming as the
year progresses. (A1R349-352) (P1)
4.5. IF you are teaching literature units in the beginning of the school year, THEN
engage in SQ and brainstorming after completing a literature unit. (A1R284) (P1)
4.5.1. STANDARD: Providing SQ and brainstorming after reading but before
writing the essay provides a feedback loop where student work is submitted,
evaluated, and returned with suggestions for improvement. (A1R286) (P1)
4.6. IF you are teaching literature units in the middle of the school year, THEN
engage in SQ and brainstorming during the literature unit with less frontloading
and scaffolding. (A1R282) (P1)
4.6.1. STANDARD: You are still working on areas of needed improvement (see
Procedure 1) but instruction is more focused on problematization of the
literature content. (A1R288-289) (P1)
4.7. IF you are teaching literature units around springtime (2
nd
semester), THEN
engage in SQ and brainstorming at the beginning of the literature unit. (A1R282)
(P1)
4.8. IF you are teaching literature units around springtime (2
nd
semester), THEN
engage in SQ and brainstorming at the beginning of the literature unit. (A1R282)
(P1)
4.8.1. Engage in Socratic Questioning by asking students, “What values might
be important in the culture within the literary text?” (A1R269) (P1)
4.8.2. Ask students, “How is this apparent to the student?” (A1R271) (P1)
4.8.3. Ask students, “How is this defended?” (A1R272P1)
4.8.4. Ask students, “How do you find examples in the text?” (A1R272) (P1)
4.8.5. Ask students, “How do you find counterexamples?” (A1R274) (P1)
4.8.6. Ask students, “Why would this be a good writing technique?” (A1R274)
(P1)
4.8.7. Ask students, “How does this strengthen your argument?” (A1R276) (P1)
4.9. Engage in this brainstorming discussion on text just read for about ½ half a class
period. (A1R276) (P1)
COGNITIVE TASK ANALYSIS 172
4.10. Use SQ and brainstorming to come up with a central idea. (A1R620) A
central idea is a short 2-3 sentence description of what the student author is
trying to say.
4.10.1. Consider using the term central idea rather than thesis statement,
especially for struggling writers. (A1R630) (P1)
4.10.2. STANDARD: Students often see the term thesis statement as restrictive
because it is one sentence with a period. (A1R621-622) Using central idea
allows students to explain more without worrying whether their thoughts fit
within a formal sentence. You can then help student narrow the central idea
into a thesis statement. (A1R620-630) (P1)
4.11. Demonstrate to students that brainstorming and Socratic Questioning are
ways of thinking that lead to a question that can be responded to through writing
an expository piece. (A1R277) (P1) (C1R148) (P3)
4.12. Take ideas developed during brainstorming and get them down on paper.
(C1R150-151) (P3N)
4.12.1. Instruct students to engage in Zero Drafting. Zero Drafting is a strategy of
allowing students to write without thought of editing or limitations. Students
are allowed to write anything-answer prompt, begin a novel, write a journal
entry, write poetry-to stimulate ideas and encourage writing. (C1R210-227)
(P3N)
4.12.1.1. Model Zero Drafting with students and discuss your choices in
your own writing. (C1R1084-1087) (P3N)
5. Procedure 5. Introduce Prompt/ Prompt Analysis. (A1R, B1R)
5.1. Demonstrate to students the questions and authentic thinking that led to the
creation of prompts. (A1R420-449) (P1)
5.2. Discuss with students that writing prompts can be used to set up the structure of
an essay. (B1R232) (P2N)
5.2.1. STANDARD: This activity may take 30-60 minutes. (B1R229) (P2N)
5.3. Discuss with class each prompt choice for the novel to be read. (B1R392) (P2N)
5.3.1. Show students that writing prompts can be broken down so that they can
help in developing the paper to be written. (B1R)(P2N)
5.3.2. Analyze the writing prompt to see if it can show how the thesis should
look. (B1R233) The thesis or central idea is the question from the prompt
that students need to answer. (B1R1027) (P2N)
5.3.3. Analyze the writing prompt to see if it identifies the topics of the body
paragraphs. (B1R234) These are the arguable thesis statements derived from
the main thesis of the paper. (B1R1030) (P2N)
5.3.4. Analyze the writing prompt to see if it can identify what must be included
in the conclusion. (B1R235) (P2N)
5.3.4.1. STANDARD: Not every prompt can be dissected into these
various segments of a paper. Teacher experience and classroom
discussion will help illustrate when this is the case. (41R)(4N)
5.3.4.2. REASON: High school seniors need this instruction to prepare
them for complex prompts, critical thinking, and organization skills
found at the university level. (B1R237, 393-396) (P2N)
COGNITIVE TASK ANALYSIS 173
5.4. Have students begin color coding the writing prompt of their choice. (B1R1004)
(P2N)
5.4.1. STANDARD: This color coding is different than that used to find concrete
details or literary features in text for answering prompts. (B1R1004) (P2N)
5.5. Use different colors for each of the following steps. (B1R1005) (P2N)
5.5.1. STANDARD: Color-coding helps students stay organized when analyzing
their prompts. (B1R1111) (P2N)
5.5.2. Have students highlight the verbs within prompt. (B1R1007) (P2N)
5.5.3. Have students highlight the writing task. (B1R1007) (P2N)
5.5.4. Have students highlight the textual situation presented in the prompt.
Textual situations are often included by prompt writers to help the students
think critically and provide focus about the prompt. (B2R1007) (P2N)
5.5.5. Determine together as a class if the prompt they have chosen sets up the
scene to be discussed in the paper. (B1R1013) (P2N)
5.5.6. Have students highlight the parts that make an arguable thesis statement.
These drive the formation of the body paragraphs. (B1R1029) (P2N)
5.5.7. Have students highlight within prompt what is going into the conclusion.
(B1R1045) (P2N) Each prompt may or may not provide this information.
(41R)(4N)
6. Procedure 6. Provide Feedback from Diagnostic Papers. (A1R)
6.1. Prepare to present to students examples from diagnostic writing piece that shows
areas needing improvement (from step 2.6). (A1R247) (P1)
6.2. Show examples from diagnostic papers (see Procedure 2) of areas that need
improvement. (A1R172, 243) (P1)
6.2.1. Provide suggestions and approaches on how students can fix these areas of
concern. Tell students, “These are examples from your papers, and here are
some ways you can fix them. (A1R243) (P1)
6.2.1.1. Consider providing students an editing checklist and have them
assess student work as a whole class assignment. (41R)(4N)
6.3. Use discussion to show students they can improve future writing pieces.
(A1R344) (P1)
6.4. Let students know that they will begin writing their papers the next day and
should keep these suggestions and approaches in mind as they write. (A1R231)
(P1)
6.5. Remind students that a thesis statement is the main argument and has 3 parts.
(A1R689)(P1)
6.5.1. Instruct students that their paper needs a clear designation of what it is the
student writer is going to talk about. (A1R690)(P1)
6.5.2. Instruct students that their paper is making a clear judgment about
something (A1R691)(P1)
6.5.3. Instruct students that their paper has chosen a side to support and that it
needs to be meaningful in some way. (A1R691)(P1)
6.6. IF students are in AP class, THEN these three parts are given more direct
emphasis. (A1R686) (P1)
6.7. IF students are in a CP class, THEN the instructor attempts to guide students
towards this goal. (A1R687) (P1)
COGNITIVE TASK ANALYSIS 174
6.7.1. NOTE: This description (See steps 6.5-6.5.3) is what is communicated to
students. The description of a thesis statement in 8.11.3-8.11.3.3 is more for
novice teachers. (A1R593)(P1)
7. Procedure 7. Construct Essay Outline (P2N, 41R)
7.1. Students begin to outline their essay. (B1R362)(41R)
7.1.1. REASON: Students will take the AP test which is a timed assessment.
Outlines help students organize their ideas quickly (B1R372)(P2N)
(C1R543)(P3)
7.2. Consider allowing students to get together in groups of 3-4 who are working on
the same prompt to work collaboratively. (B1R359-360) (P2N) They support one
another as they find concrete details and literary features to address their prompt.
(B1R363) (P2N) (C1R542)(P3)
7.3. Have students take the results from prompt analysis and identify the body
paragraphs. (B1R366) (P2N)
7.4. Have students find the concrete details they will use in each body paragraph and
insert into outline. (B1R367, 406) (P2N)
7.5. Have students write into the outline what they are going to say about the concrete
details (commentary). (B1R369) (P2N)
8. Procedure 8. Provide Writing Instruction. (A1R, B1R, C1R, 41R)
8.1. Introduce rhetorical features, style structures, or skills that you have decided to
cover for this writing piece (see step 1.3.4). (C1R695)(P3N) critical analysis in
essays
8.2. Remind students what is expected in each paper based on instruction given and
the rubric used to assess. (B1R1056)(P2N) (C1R707)(P3)
8.2.1. Tell students that papers will be assessed according to their response to the
prompt along with discussion of articles, novels, or other texts which have
been notated in student notebooks (see step 1.10). (C1R693-694) (P3N)
8.3. Review prompts with students so they know those requirements that are needed
for a good paper. (C1R707) (P3N)
8.4. Show students models of exemplary essays or mentor texts (see step 2.7).
Discuss with students each exemplary essay and how the claims are addressed.
(B1R425, C1R1209) (P2N) (P3)
8.4.1. STANDARD: Always collect exemplary essays from student work to
show to future classes. (B1R425) (P2N)
8.4.2. Have students identify what the writer has done well and think about how
they as writers begin to move towards that goal. (C1R1178)(P3N)
8.4.3. Have students begin writing everything they want to say to the topic being
made within the assigned text. (C1R513) (P3N)
8.5. Ask students to look back and decide if they have said what they wanted to say.
(C1R515) (P3N)
8.6. Begin to write body paragraphs. (P1) (P2) (P3)
8.6.1. Pull class together for whole-group discussion on creating arguable topic
sentences using the concrete details previously found. They are also referred
to as thesis statements. (B1R521) (P2N)
8.6.1.1. STANDARD: An arguable topic sentence or thesis statement is:
8.6.1.1.1. An introduction to the body paragraph. (B1R1521) (P2N)
COGNITIVE TASK ANALYSIS 175
8.6.1.1.2. Specific to the text but does not contain a concrete detail or
quotes. (B1R1523) (P2N)
8.6.1.1.3. Something that can be argued about. (B1R1530) (P2N)
8.6.1.1.4. An argument that needs to be proved by concrete evidence
in the text. (B1R1536) (P2N)
8.6.1.2. STANDARD: The arguable topic sentence at this point need only
be 3-4 words long as inserted into outline. (B1R415)(P2N)
8.6.2. Have discussion with the class about what the arguable topic sentences say
and what the writer is going to say about them. (B1R416-417) (P2N)
8.6.2.1. STANDARD: These decisions made by the student writer become
the arguable topic sentences for each body paragraph. (B1R420-421)
(P2N) put this in topic statement central statement move this!
Procedure 2 or 4
8.6.3. Provide support to students by checking for a thesis statement. (A1R467,
B1R1521)(P1)
8.6.3.1. STANDARD:A thesis statement should include: (P1)
8.6.3.2. A topic-facts and description of the story and/or element of the
story under discussion. (A1R597-599) (P1)
8.6.3.3. An assertion- assertions answer what the element of the story is
doing within the text. It might be contributing or undermining the plot,
the theme, or characterization. (A1R600-602) (P1) Assertions may also
be referred to as claims. (P2N)
8.6.3.4. Significance- this answers why the element is important, or how it
connects to the life of the student or other texts. (A1R602-603) (P1)
8.7. Remind students that each body paragraph should begin with a topic sentence,
followed by concrete details that support the topic sentence, and a concluding
sentence to wrap up what has just been said and point to the next paragraph in the
essay. (A1R583, B1R462,C2R466)(P1) (P2) (P3)
8.7.1. Provide support by helping them find examples, support from text, or
support from secondary sources (if applicable). (A1R469, 558-559) Students
may know the term “support from text” as concrete details, or details that
can be found directly within a text. (A1R583, B1R462) (P1) (P2)
8.7.2. Support students by helping them identify or narrow their central idea or
argument. (A1R470) (P1)
8.7.3. Check how all the elements noted above are incorporated into the text.
(A1R559-560) (P1)
8.7.4. Instruct students that they may choose the structure of their body
paragraphs. (B1R466) (P2N)
8.7.5. Tell students that they may deviate from this format as they see the need to
add explanation or to set up a scene from the novel. (B2R467) (P2N)
8.7.6. Tell students to make their structure flow in a natural way that makes
sense to them. (B1R456) (P2N)
8.7.7. Refrain from providing concrete guidelines in the amount of quotes or
support needed by students in writing an argument. Tell them they have the
responsibility to use enough textual support to your point. (A1R561-563,
B1R1430) (P1) (P2)
COGNITIVE TASK ANALYSIS 176
8.7.8. Tell students to review thesis statements for rewriting or revision as they
complete body paragraphs. (B1R2298)(P2N)
8.7.9. STANDARD: Students can always rewrite claims to tighten them up and
ensure they fit the argument made. (B2R2298)(P2N)(41R)
8.7.10. REASON: students often find the body paragraph necessitates a change in
the in the claim. (B1R506-509) (P2N)
8.8. Begin to write conclusion. (B1R539)(P2N)
8.8.1. Tell students that their prompt will often tell them what type of conclusion
to write. (B1R542) (P2N)
8.8.1.1. IF the prompt has a question that asks students to reflect or gives
students a task to do within their writing, THEN this question, once
restated, becomes the basis for the conclusion. (B1R1678-1682) (P2N)
8.8.1.2. IF the prompt does give a task to do or is the task is more open-
ended, THEN the prompt has not indicated what the conclusion should
be. This would then require a more formal conclusion which entails a
restating of the introduction. (B1R1682-1684) The Common Core
writing rubric can help guide instruction a more formal conclusion.
(B1R1703-1704) (P2N)
8.8.1.3. EXAMPLE: The prompt might ask students to explain how their
thesis impacts the world. The students know that will be the direction
their conclusion should take. (B1R541) (P2N)
8.8.2. Encourage students to avoid the phrase, “In conclusion.” (B1R547) (P2N)
8.8.3. Instruct students that a formal conclusion is the reverse of the introduction
but is not an exact copy. The conclusion is a restatement of what is said in
the essay, but with a more reflective look at the significance of the thesis.
(B1R1700-1702) (P2N)
8.8.4. In writing a more formal conclusion, tell students to begin with a
restatement of their thesis in a differently from what was written in the
introduction. (B1R566, 1734) (P2N)
8.8.4.1. Instruct students to avoid giving advice in the final sentence of the
conclusion. (B1R1749) (P2N)
8.8.5. Tell students to restate the points or claims made in the body paragraphs.
(1734) (P2N)
8.8.5.1. Tell students that the last statement of the conclusion is a larger
statement that is more universal and addresses the theme of the paper.
(B1R567-568, 1734) (P2N)
8.9. Begin to write introductory paragraph as the last step in completing draft.
(B1R1904)(P2N) (C1R537-538) (P3)
8.9.1.1. REASON: It is easier to craft an introductory paragraph when the
rest of the paper is already completed. The student now knows what the
paper is about. (B1R1905) (P2N)
8.9.2. Tell students that an introductory paragraph may start with a broad general
thematic statement sentence that has to do with the topic or theme at hand.
(B2R550-551, 1896) This is common but not required. Students may choose
to move their general thematic statement further into the paragraph.
(B1R1898) (P2N) (C1R538)(P3)
COGNITIVE TASK ANALYSIS 177
8.9.3. Tell students that the first sentence could also be a theme statement.
Theme statements do not mention the novel at all. They are universal
statements that pertain to people. (B1R552-554) (P2N)
8.9.4. Tell students that the next sentence is narrowed down to the particular text,
novel, or characters being discussed. (B1R554)(P2N)
8.9.5. Tell students that the next sentence is the thesis, or the main argument of
the writing piece. (B1R557) (P2N)
8.9.6. Instruct students that in writing an introductory paragraph, the writing
starts with a broad, universal statement which narrows to the thesis which is
exactly what the paper is going to talk about. (B1R556-558) (P2N)
8.10. Provide additional instruction to class on revising and improving papers
before engaging in Writers Workshop. (41R)(4N)
8.11. Instruct students on the use of linkages and transitions between sentences
and paragraphs. (B1R1944)
8.11.1. Tell students that transition words are used to link sentences or ideas
within paragraphs. (A1R) (B1R1961)
8.12. Tell students that linkages connect paragraphs to each other. Linkages
occur when a writer takes a few words from the last sentence of a paragraph and
uses those same words in the first sentence of the next paragraph. (B1R1944)
8.13. Consider providing additional instruction in the use of SOAPSTone (see
procedure 2) as a strategy for students to review their own writing. (41R)(4N)
8.13.1. Tell students that, as writers, they should consider SOAPSTone as criteria
for their own writing. (C1R1004) (P3N)
8.13.1.1. STANDARD: SOAPSTone can be used throughout the school year
by students in analyzing everything they write as well as what they read
(see step 3.5.2). (C1R999) (P3N)
8.13.2. Instruct students that the SOAPSTone criteria guide the decisions they
make in an essay or text (C1R1005) (P3N) SOAPSTone can help guide
writers in their efforts to convey meaning or improve writing voice.
(C1R847-848)
8.13.2.1. STANDARD: Voice refers to the writer’s choice of diction and the
consideration of tone a writer gives to the intended audience. These
choices may include: (P3N)
8.13.3. Use writing devices (i.e. narrative, anecdote, etc.) to soften or illustrate a
point when providing commentary in their writing. (C2R847) (P3N)
8.13.3.1. Use comparisons that help to define or describe. (C2R847) (P3N)
8.13.3.2. Concentrate on the flow and rhythm of the writing as it is read by
the reader. (C2R847) (P3N)
8.13.4. Guide students in analyzing their writing to make their sentences less
complex or obtuse. (C1R1538) Work with students to do the following:
(P3N)
8.13.4.1. REASON: students often strive to make their writing sound
intelligent through the use of length and complexity. They might also
attempt to hide their misunderstandings of content or writing
knowledge by obfuscating their sentences. Encourage student writers to
say what they mean. (C2R)(P3N)
COGNITIVE TASK ANALYSIS 178
8.13.5. Review verb usage by having students think about the ratio of passive
verbs to active verbs without affecting meaning. (C2R)(P3N)
8.13.6. Have students check the ratio of descriptive nouns to abstract nouns.
(C2R)(P3N)
8.13.6.1. STANDARD: A descriptive noun is a common noun with which a
person can experiment with our senses. Abstract nouns are those that
refer to something with which a person cannot interact (e.g., love, hate,
pride, peace, etc.) (C2R)(P3N)
8.13.6.2. STANDARD: A good ratio to consider is 4-5 descriptive nouns to
1 abstract noun. (C2R)(P3N)
8.13.7. Review the number and variety of words in each sentence.
(C1R1548)(P3N)
8.13.8. Instruct students to consider the rhythm within the structure of their
paragraph. Discuss with students if longer, passive sentences convey the
same message as shorter, more direct sentences. (C1R1548) (P3N)
8.13.9. Remind students often that sentence length and word choice are the
prerogative of the writer. (C1R1550)(P3N)
8.14. STANDARD: Encourage students that their writing should say what they
want it to say. (C1R1533)(P3N)
9. Procedure 9. Conduct Writing Workshop (P1) (P2) (P3)
9.1. Give students time to write. Give time based on assignment length (2-7 days).
(A1R456, 464-465) (P1)(B1R1416)(P2)
9.1.1. STANDARD: Allow students time to write drafts but shorten writer’s
workshop time if students begin to complete papers earlier than anticipated.
(41R)(4N)
9.1.2. IF a student is in an AP class, THEN all papers completed during 2
nd
semester are timed in-class writing in preparation for the upcoming AP test.
(A1R549-552) (P1)
9.1.3. STANDARD: AP students might be given only one day to complete essay
in preparation for AP test later in school year. (41R)(4N)
9.2. Circulate during Writer’s Workshop and provide oral feedback and
individualized instruction as needed. (A1R468, B1R481, C1R316-317) (P1) (P2)
(P3)
9.2.1. Gauge student understanding of expectations. Be prepared to provide
additional instruction to correct misunderstandings or to provide more
practice with a particular skill. (C1R713) (P3N)
9.2.2. IF misunderstandings or need for additional instruction limit their ability
write a quality paper, THEN stop and address the additional topic before
moving on in the writing process. (C1R714)(P3N)
9.2.2.1. REASON: Teach this additional instruction immediately if its
absence will hinder students from completing their writing piece.
(C1R714)(P3N)
9.3. IF misunderstandings or need for additional instruction do not limit the students’
ability to write a quality paper, THEN cover this additional topic sometime
during the draft writing. (C1R715)(P3N)
9.3.1. Check first body paragraphs during Writers Workshop. (A1R468) (P1)
COGNITIVE TASK ANALYSIS 179
9.4. IF students are comfortable with the direction of their drafts and have shown you
they are capable writers (based on past assignments), THEN leave students alone
to write. (A1R474) Spot-check writing as you circulate if possible. (A1R512,
B1R1416, C1R585) (P1) (P3)
9.5. IF students lack confidence or have shown they need more support, THEN these
students are required to bring every completed paragraph to you for review.
(A1R476, 509) (P1) (C1R286-289)(P3)
9.6. Have students engage in peer review during writer’s workshop. (41R) (4N)
9.6.1. Allow students during writing workshop to support one another or provide
peer reflection during this part of assignment. You may follow an informal,
semiformal, or formal peer review process.
9.6.2. STANDARD: In order for peer review to be successful, Students need to
approach peers they know will be critical of their writing and of whom they
are willing to accept criticism. (A1R541-542, B1R2156 ) (P1) (P2)
9.6.3. STANDARD: This pairing does not naturally happen very often.
(A2R541)(P1)
9.7. IF you want students to engage in some peer review of papers, THEN follow the
semiformal peer review process in steps 9.7.1 to 9.7.3.5. (B1R2151)(P2N)
9.7.1. Have students peer review each other’s drafts. (A1R580) Students may be
grouped together by the teacher or by student choice. (B1R2153)(P2N)
(C1R238)(P3) This process will occur twice during each writing unit.
(C1R241,325)(P3N)
9.7.2. Consider pairing up weak writers with strong writers who would be
willing to work with another student in providing support. (B1R2156) This
happens as you figure out the skills and personality characteristics of each
student. (A1R541-542, B1R2156) (P1)(P2)
9.7.2.1. STANDARD: Trust is important in making collaborative pairings.
Sharing writing can be frightening for students. Choose student
partnerships wisely (B1R2168-2174) (P2N)
9.7.3. During peer review, have students review the following as peer editors.
(P2N)
9.7.3.1. Have students determine if writer has answered the prompt;
(B2R)(P2N)
9.7.3.2. Have students determine if writer has written effective transitions
between sentences within paragraphs; (B2R)(P2N)
9.7.3.3. Have students determine if writer has written effective transitions
between paragraphs; (B2R)(P2N)
9.7.3.4. Have students determine if writer has written an effective
conclusion that wraps up the paper; (B2R)(P2N)
9.7.3.5. Have students determine if writer has written a paper that makes
sense. (B2R)(P2N)
9.8. IF you feel students find value and increased editing skill from peer review,
THEN provide a formal peer review process as outline in steps 9.8.1 to 9.8.11.
(C1R646) (P1)
9.8.1. Have students to get in groups of 3-4 people of their choice. (C1R323)
(P3N)
COGNITIVE TASK ANALYSIS 180
9.8.2. IF the class is a CP class, THEN provide peer editors with a checklist (see
end of protocol) of what to look for in their colleague’s papers. (C1R1518)
(P3N)
9.8.3. IF the class is an AP class, THEN have peer editors use the following
strategy to provide peer revision- Bless, Press, Address (C1R1517).
(C1R635)
9.8.4. Instruct students do the following: (P3N)
9.8.4.1. Bless-talk about what is working in the paper. (C1R635)(P3N)
9.8.4.2. Press-press author to improve where there is need for improvement.
(C1R635)(P3N)
9.8.4.3. Address-help address author’s questions about their paper.
(C1R635)(P3N)
9.8.5. Model the review strategy Bless, Press, and Address with class, showing
what peer editors should be doing to writer’s papers during revision process.
(C1R1180-1182) (P3N)
9.8.5.1. REASON: This activity helps to train student editors and helps
norm the process of revision. (C1R1182) (P3N)
9.8.6. Student writers have first draft reviewed by peers of their choice. (C1R318,
325) (P3N)
9.8.6.1. STANDARD: This process should allow groups to get through 3-4
people within an hour (1-2 class periods). (C1R652-654) Consider
mixing groups at various times throughout school year. (C1R648)
(P3N)
9.8.7. Ask students to bring copies of 1
st
draft for each member of peer group, if
possible or practical. (C1R628-629) (P3N) (41R)
9.8.8. Instruct students seeking peer support to bring questions that address
issues within their own writing (paragraph or paper). (C1R630-631) (P3N)
9.8.9. Instruct peer editors to read paper. Editors may give revisions as they read
or annotate and finish notes on paper itself. (C1R632-633) (P3N)
9.8.9.1. As an alternative step, have one member of the peer editing group
read paper while writer and remaining peer editors listen. (C2R)(P3N)
9.8.9.1.1. REASON: Having a peer editor read the paper gives the
writer an opportunity to listen to the paper from another reader’s
voice. The writer must listen to the paper and any unclear or
confusing writing. They must also hear any grammatical or
punctuation errors. (C2R)(P3N)
9.8.10. Instruct student authors to sit and listen to peer editors without comment.
The writer does not speak at all except through their writing. (C1R641-643)
(P3N)
9.8.11. Give students a copy of the rubric and have the students assess their
colleagues’ papers. (B1R2085-2086) The International Baccalaureate and
Common Core rubrics are less vague than the Advanced Placement rubrics.
(B1R2094) Consider using the Common Core Informative Rubric to assess
expository pieces in a literature class. (B1R2131) (P2N)
9.9. As the teacher, use Peer Review time to provide feedback to individual students
if they seek it or if you feel individual students need it based on past writing
COGNITIVE TASK ANALYSIS 181
assignments. (A1R480-481, B1R1416, C1R667-670) (P1) (P2) (P3) You may
also join peer editing groups as needed. (C1R316-317) (P3N)
9.10. Have students begin to write 2
nd
drafts based on review, comments, and
suggestions from peers and from you as the teacher. (41R)(4N)
9.10.1. Allow students to conference with you during writing of 2
nd
draft with
questions about papers or ideas (C1R677)(P3N)
9.10.1.1. REASON: While the workload is great, it is easier to guide
students towards making efforts to improve writing while they are in
the process of actually writing. This also makes final evaluation easier
since you have seen the papers and worked with students steering them
towards an improved product. (A1R488-490) (P1) (C1R686)(P3)
9.10.2. Work with students during individual conferences to address the
following:
9.10.2.1. Provide guidance in possible directions to improve argumentation.
(A1R486) (P1)
9.10.2.2. Provide feedback in suggesting improvements to evidence.
(A1R486) (P1)
9.10.2.3. Provide feedback in improving transition between sentences and
paragraphs- connecting ideas that blend one thought into another (see
step 9.12). (A1R882)(P1) (B1R535)(P2)
9.10.2.4. Provide customized feedback in suggesting improvements to
analysis. (A1R487) (P1)
9.10.2.4.1. STANDARD: Students may know the term commentary,
which is the explanation of how a concrete detail explains or
deepens your thesis. (A1R650-651) (P1)
9.10.2.5. Provide feedback in helping students clarify what he/she is trying
to say. (A1R487) (P1)
9.10.2.6. Provide customized feedback in developing a more sophisticated
inclusion of quotations into students’ texts. (A1R884)(P1)
9.10.2.7. Provide customized feedback in helping students take fewer words
and incorporate them into own sentences rather than dropping a quote
into drafts out of context. (A1R887) (P1)
9.10.2.8. Writing summary rather than analysis. (C1R1190)(P3N)
9.10.2.8.1. STANDARD: While these tend to appear each school year,
address them based on the work of the students within each class.
(C1R1189-1194) (P3N)
9.11. Consider reviewing drafts as a whole class by assigning students to turn in
drafts in their current form (1
st
or 2
nd
revision).
9.11.1. Select 6 random papers and share with whole class. (C1R1177-1178)
(P3N)
9.11.2. Discuss areas of concern that exhibit themselves during drafting and
revision. (C1R1190-1191) (P3N)
9.12. IF your students are writing an argumentative paper, encourage students to
revise their own drafts by reminding them of the modes of writing they have
learned so far. (31R) (P3N)
COGNITIVE TASK ANALYSIS 182
9.13. Hold second round of peer revision to review second draft following any
peer review model noted above. (C1R241, 325) (P3N)
9.14. At the conclusion of writers workshop, give students 3 additional days to
type and polish papers before final due date. (A1R547-549, B1R582) (P1) (P2)
9.15. Remind students paper must be typed in MLA format. Review MLA
format issues with citations. Provide help to students who need extra support.
(A1R664, B1R1972) (P1)(P2)
9.15.1. NOTE: MLA format has been taught in previous grades so this instruction
is not a focus in 12
th
grade. (A1R664-667) (P1)
9.16. IF possible, have students submit papers to an online grammar check
program such as TurnItIn.com. (B1R594)(P2N)
9.16.1. REASON: This program helps correct grammar. This requirement puts the
responsibility for correcting grammar issues on the student. (B1R594) (P2N)
9.17. Analyze completed papers with an online program to detect plagiarism.
(41R)(4N)
10. Procedure 10. Assess papers. (A1R, B1R, C1R)
10.1. Collect student papers for teacher evaluation. (A1R708)(P1)
10.2. Put off grading for a few days. (A1R709) (P1)
10.2.1. REASON: This allows instructor to be more objective in evaluation.
(A1R711) (P1)
10.3. Remind yourself what areas of needed improvement and content based on
instruction given will be evaluated during grading using the rubric provided or
the rubric created with class in step 3.8. Provide very detailed and excessive
feedback. (A1R714)(P1) (B1R583) (P2N)
10.4. Consider giving audio feedback to student writing at the if possible within
your school’s assessment software. (B1R2270) (P2N)
10.4.1. STANDARD: Provide more detailed and excessive feedback in the
beginning of the year. This will take much time, but this helps to resolve
student writing problems which results in the need for less feedback as the
year progresses. (B1R602-606) The amount and quality of feedback sets the
tone for the rest of the year. Students know you take writing seriously and
read their writing critically. (B2R606)(P2N)
10.5. Base evaluation on whether student writers supported their thesis, their
arguments make sense, and are justifiable. (A1R718-719) (P1)
10.5.1. Consider the personalities and approaches of students individually and as a
class group. (A1R716) Ask yourself, “How did this particular class approach
the text being analyzed?” (P1)
10.6. Consider the differences between classes in a particular year. (A1R714)
(P1)
10.7. Base evaluation on the age group of students. (A1R715) (P1)
10.8. Grade student work in a holistic way. (A1R743) Be prepared to accept
arguments that might be wrong as long as they have support and are justifiable.
(A1R741-742) (P1)
10.9. Consider the distance each writer has grown and moved towards an
acceptable level of proficiency. (A1R728-729, A1R750) (P1)
COGNITIVE TASK ANALYSIS 183
10.9.1. STANDARD: Students are never informed of this grade scheme and it
applies to very few students each year (2-3 students and is usually applied to
students with IEP’s or 504’s) (A1R763-771)The inflated grade does not
change external evaluation of lower performing students but is used as a
boost in confidence which reflects their improvement and hard work.
(A1R754-757) (P1)
10.10. Assess grammar. (A1R785)(P1)
10.10.1. IF the grammar and punctuation do not affect what the student
wanted to say, THEN it is not reflected in the student’s grade. (A1R785)(P1)
10.10.2. IF grammar and punctuation makes a paper unclear, THEN it will
be reflected in the student’s grade. (A1R785-788) (P1)
10.10.3. STANDARD: Grammar and punctuation should clarify what the
writer is trying to say. (A1R786)(P1)
11. Procedure 11. Hand back papers and provide optional opportunities for further
revision. (A1R, B1R, C1R, 41R)
11.1. Return back graded work to students. (A1R813, B1R1478)(P1) (P2)
11.1.1. IF students wrote an acceptable paper but are unhappy with their grades,
THEN they have the opportunity to revise their paper. They must meet with
you before they begin to revise. (A1R814-816) (P1)
11.1.2. IF students wrote a paper that does not meet proficiency based on the
rubric, THEN mark the grade as ‘R’, meaning a revision is required.
(A1R818) An ‘R’ means the paper has no point value. This will remain the
case until a revision is turned in. (A1R819-825) (P1)
11.1.3. REASON: This helps incentivize students to actually attempt a revision
rather than settle for a poor grade. (A1R824) (P1)
11.2. Consider allowing for multiple revisions as needed and as time requires.
This is up to students and your schedule as a teacher. (A1R852) Experience,
however, shows that students do not revise more than one time. (A1R849) (P1)
11.3. Offer to conference with students who want to revise their graded paper to
provide additional targeted practice to improve writing. (C1R288, C1R301-302)
(3PN)
11.4. During conferences, look for the following common problem areas. (P1)
11.4.1. Look for problems in student writing such as problems with the
argumentation and support: (A1R859)(P1)
11.4.2. Look for problems in student writing such as papers having too few
quotations or citations. (A1R859)(P1)
11.4.3. Look for problems in student writing such as writers not doing a good job
connecting quotations to the argument. (A1R861) (P1)
11.4.4. Ask students to be ready to discuss where they struggled with their writing.
(C1R254)(3PN)
11.5. IF time allows, consider returning papers to students for a reflective
revision. (C1R254-256) (3PN)
11.5.1. Ask students to reflect on the top 5 things they need to work on as targeted
practice to improve paper. (C1R263) (3PN)
11.5.2. Assign students to submit multiple versions of an improved writing area of
concern. (C1R265-266) (3PN)
COGNITIVE TASK ANALYSIS 184
11.6. Tell students to resubmit reflective revisions and revised paper for a
revised final grade. (C1R301, C1R306-308) (3PN)
12. Procedure 12. Publish student writing. (3PN, 41R)
12.1. Encourage students to share their writing through blogging and
submissions of writing pieces to various publishers. (C1R) (4N)
12.2. Have students consider the sense of audience when publishing writing.
(C1R746) Model for students how and why writing pieces might be published.
(C1R1338) (3PN)
12.3. Tell students that this is where they decide whether to use all the feedback
from peer review and conferencing or not. Establish with students that this is
their paper and they have the final choice of what is included when they decide to
publish. (C1R1266-1276) (3PN)
12.4. Assign students to blog regularly. Have students make connections
between articles and texts, and between what they are learning in class and
something experienced outside the classroom. (C1R374)
12.4.1. Encourage students to write a blog post and share post on Twitter.
(C1R757-759) (3PN)
12.4.2. STANDARD: Publishing gives students an authentic reason to engage in
writing.
12.5. Instruct students on tagging blogs to help writing to be found on Internet.
(C1R767) (3PN) Internet tags help search engines find Blogs or other writings
more easily based on search parameters.
12.5.1. Instruct students to find main ideas and key points from their text and turn
them into tags. (C1R770) (3PN)
12.6. Encourage students to participate in Network Reading. Network Reading
is where writers find an audience or cohorts of people on line (i.e., Twitter,
WordPress) who might be interested in their writing topics. Fellow writers read,
respond, and share one another’s articles on their blogs. (C1R771)(3PN)
12.6.1. REASON: This gives students an audience outside of school for their
writing. (C1R771-772) (3PN)
12.7. Encourage students to read and comment on the blogs of other writers.
Encourage them to join into the conversation of writing pieces and share with
each other through individual blogs. (C1R774-778) (3PN)
12.7.1. Monitor this Internet activity through your participation in Blogging and
making connections with other authors including your students. (C1R)
12.7.2. REASON: The more student writers engage on other writers, the
more other writers return the favor and engage student writing online.
Students participate in blogger communities to find and get followers, to
share good writing, and to observe the writing craft of other bloggers.
(C1R1292)(3PN)
12.8. Encourage students to publish by offering extra credit for getting any
writing piece published anywhere outside the classroom. This can include Blogs,
newspapers, or magazines (C1R1292)(3PN) (41R)
12.9. Require students to establish a twitter account and to set up a personal
blog. (C1R1293) (3PN)
COGNITIVE TASK ANALYSIS 185
Expository Writing
Editing Checklist
1) Does the introduction draw you into / make you want to continue reading the
essay? Why? If not, how might the author improve the intro?
2) How clear is the author's argument/claim/thesis? Scale 5 - 1; Highest: Very clear,
Lowest: Not clear at all
3) At which point did you feel most interested by this piece? When least? Explain.
4)Does the logic of this paper's argument ever fall flat? Where? What might be done
to correct this?
5) How smoothly does this paper integrate examples/evidence into its own
argument? Does it clearly illustrate connections between the evidence it cites and the
ideas they support, or does it merely assume them? Explain.
6) Can the conclusion of this paper be convincingly drawn from the thesis and the
argument made in the body of the paper? Why or why not?
7) Scan and spot the paper for the any of the following problems: colloquialisms,
informality, clichés, and wordiness. Give a few examples of these problems, if they
exist.
8) How thorough is the author's use of MLA formatting (name, pagination, source
citations, and works cited)?
Scale 5 - 1 / Highest: Very thorough, Lowest: MLA format missing
9) AP Essay Scoring: 9-8 Effective, 7-6 Adequate, 5, 4 Inadequate, 3-2 Little Success
1) Does the introduction draw you into / make you want to continue reading the
essay? Why? If not, how might the author improve the intro?
2) How clear is the author's argument/claim/thesis? Scale 5 - 1; Highest: Very clear,
Lowest: Not clear at all
3) At which point did you feel most interested by this piece? When least? Explain.
4)Does the logic of this paper's argument ever fall flat? Where? What might be done
to correct this?
5) How smoothly does this paper integrate examples/evidence into its own
argument? Does it clearly illustrate connections between the evidence it cites and the
ideas they support, or does it merely assume them? Explain.
COGNITIVE TASK ANALYSIS 186
6) Can the conclusion of this paper be convincingly drawn from the thesis and the
argument made in the body of the paper? Why or why not?
7) Scan and spot the paper for the any of the following problems: colloquialisms,
informality, clichés, and wordiness. Give a few examples of these problems, if they
exist.
8) How thorough is the author's use of MLA formatting (name, pagination, source
citations, and works cited)?
_______ Scale 5 - 1 / Highest: Very thorough, Lowest: MLA format missing
9) AP Essay Scoring: 9-8 Effective, 7-6 Adequate, 5, 4 Inadequate, 3-2 Little Success
COGNITIVE TASK ANALYSIS 187
Argumentative Peer Revision Checklist
1. Write one question per paragraph that you’d like your reader to answer (in the margin)
2. Exchange essay w/ another student (out of the 3 students today, TWO must be
different from last week)
3. Check for:
claim—is there an argument
evidence/data/reasons
counter argument—can you suggest others?
structure
grammar, sp. mechanics
ANSWER THEIR MARGIN QUESTIONS
4. 10 minutes / 2 min warning / Finish early? Re-read. Take another close look.
5. 3x
6. Review and reflect on responses.
Were your questions answered? Do you agree w/ them?
What revisions are needed? Why?
What did you learn from reading the other essays about your own?
COGNITIVE TASK ANALYSIS 188
Appendix F
Coding Spreadsheet
Spreadsheet follows this page
COGNITIVE TASK ANALYSIS 189
SME Ste
ps
Alignme
nt
SME (ROUND
2)
Step Typ
e
Final Incremental Gold
Standard Protocol Procedures
SME A SME B SME C SME D A D 1, 2, 3,
or 4
A2
R
B2
R
C2
R
D2
R
Procedure 1. Prepare to Teach. (A1R, B1R, C1R) 35 13
1 A 1.1. Develop lesson objectives
based on district expectations
and state academic standards.
(41R)(4N)
0 0 0 1 1 0 0 0 0
2 A 1.2. Choose the text (i.e.,
novels, short stories, article,
etc.) you will use to teach
expository writing subject
(A1R406) (C1R406) (P31R)
1 0 1 1 3 0 0 0 0
3 D 1.3. IF you choose to use a
novel as your reading text,
THEN follow steps 1.3.1 to
1.1.6.2. (B1R,)
0 1 0 1 2 0 0 0 0
4 A 1.3.1. Choose a novel to
study. Follow parameters in
novel selection set by district
requirements or by curriculum
requirements (i.e., Advanced
Placement, International
Baccalaureate, etc.) (A1R685)
(B1R685,C1R422, 41R)
1 1 1 1 4 0 0 0 0
5 A 1.3.2. Find thematic
connections between your
various choices of novels.
(B1R688)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 190
6 A 1.3.2.1. Consider teaching
literature within a specific
context (historical,
psychological, etc.). (B1R691)
(P2N)
0 1 0 1 2 0 0 0 0
7 A 1.3.3. Think about selecting a
text that interests you as a
teacher and that you might not
have taught before. (B1R875)
(P2N) (41R)
0 1 0 1 2 0 0 0 0
8 D 1.3.4. IF you are teaching a
novel that you have already
read, THEN review annotations
and notes from last year to see if
they remain applicable to your
current students’ abilities and
background knowledge.
(B1R865) (P2N)
0 1 0 1 2 0 0 0 0
9 D 1.3.5. IF you are teaching a
new novel for the year, THEN
strive to annotate and analyze
novel ahead of students. Stay
ahead of students’ reading
progress. (B1R877) (P2N)
0 1 0 1 2 0 0 0 0
10 A 1.3.6. Select ideas, themes,
and issues within the novel that
you would like to cover with
students. (B1R805-807) (P2N)
0 1 0 1 2 0 0 0 0
11 A 1.3.7.1. List them on
inside of the cover of your
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 191
novel. (B1R1094) (P2N) (D1R)
12 A 1.3.8. Assign color-coding to
each feature and apply to list on
the cover of your novel.
(B1R1089) (P2N)
0 1 0 1 2 0 0 0 0
13 A 1.3.8.1. During your annotating
of the novel, place matching
color-coded sticky notes on
pages within the novel where
these ideas, themes, and issues
are addressed. (B1R1088-1092)
(P2N)
0 1 0 1 2 0 0 0 0
14 D 1.4. IF you choose to use a
critical literary analysis as your
reading text, THEN follow steps
1.2.1 to 1.2.6. (A1R, B1R,
D1R)
1 1 0 1 3 0 0 0 0
15 A 1.4.1. Find piece of writing
that is a critical literary analysis
of the type of literature taught.
(A1R182-183) (P1)
1 0 0 1 2 0 0 0 0
16 A 1.4.2. Search for critical
literary analysis example from a
suitable research database or
MLA Bibliography Database.
(A1R183, A2R) Examples of
critical literary analysis might
also be available from
classroom textbooks. (A1R185)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 192
(P1)
17 A 1.4.3. Choose an example of
critical literary analysis that is
approachable to a college
senior, based on teacher
judgment. (A1R184) (P1)
1 0 0 1 2 0 0 0 0
18 A 1.4.4. Choose an example of
critical literary analysis that is
not too technical. (A1R184)
(P1)
1 0 0 1 2 0 0 0 0
19 A 1.4.5. Choose an example of
critical literary analysis that is
an academic analysis of a
portion of the text taught during
literature unit. (A1R185) (P1)
1 0 0 1 2 0 0 0 0
20 A 1.5. Within the first few weeks
of the school year, plan to teach
different writing modes (i.e.
description, compare/contrast,
narrative, or expository) and the
appropriate context for their
use. (C1R844-845) (P3N)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 193
21 D 1.6. IF you choose to teach
argumentative writing,
rhetorical devices, particular
writing skills, or are want to
provide students a choice of
topics, THEN follow steps 1.5.1
to 1.3.11.1. (C1R)
0 0 1 1 2 0 0 0 0
22 D 1.6.1. IF you are having
students write an argumentative
paper using a topical current
event to stimulate making
connections and choosing a side
to argue, THEN choose articles
that argue both sides of an
argument. (C1R428-449) (P3N)
0 0 1 1 2 0 0 0 0
23 D 1.6.2. IF you need to teach
rhetorical features, style
structures, or particular skills
that need to introduced or
reviewed, THEN find and
choose articles that exemplify
these features. (31R428-449)
(P3N)
0 0 1 1 2 0 0 0 0
24 D 1.6.3. IF writing assignment
is student choice, THEN allow
students the freedom to write on
any topic they choose.
(A1R834) You can give student
choice writing assignments
anytime during the school year.
(C1R834) (P3N)
1 0 1 1 3 0 0 0 0
COGNITIVE TASK ANALYSIS 194
25 A 1.7. Collect or develop prompts
for students to analyze the
literary text assigned. (A1R,
B1R, D1R) (P1, P2, P3)
1 1 0 1 3 0 0 0 0
26 D 1.7.1. IF students are reading
literary text, THEN also assign
a contemporary review of the
novel. (A1R, B1R, C1R, D1R)
(P1, P2, P3)
1 1 1 1 4 0 0 0 0
27 A 1.7.1.1. Choose or create
prompts that ask students to
explain or justify why they
agree or disagree with the
contemporary review using the
novel for support. (A1R, B1R,
C1R414-417) (P1)
1 1 1 1 4 0 0 0 0
28 A 1.7.2. Choose or create
prompts using one or more of
the following criteria. (A1R,
B1R, C1R, D1R)(P1, P2, P3)
1 1 1 1 4 0 0 0 0
29 A 1.7.2.1. Find or develop
writing questions or prompts
based on ideas, themes, and
issues to be covered during
reading of text. (A1R266-279,
B1R802-804, C1R693-694,
D1R) (P1, P2, P3, 4)
1 1 1 1 4 0 0 0 0
30 A 1.7.2.2 Choose or create
prompts that address character
development. (B1R802) (P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 195
31 A 1.7.2.3. Choose or create
prompts that address setting.
(B1R1081) (P2N)
0 1 0 1 2 0 0 0 0
32 A 1.7.2.4. Choose or create
prompts that address the
historical context from which
the literary piece is based.
(B1R333, D1R) (P2N)
0 1 0 1 2 0 0 0 0
33 A 1.7.2.5. Choose or create
prompts that address the
historical context of when the
literary piece was written and
published. This context may
also address current historical
experience as compared to the
literary piece. (B1R333, D1R)
(P2N)
0 1 0 1 2 0 0 0 0
34 A 1.7.2.6. Choose or create
prompts that address literary
features. (B1R803) (P2N)
0 1 0 1 2 0 0 0 0
35 A 1.7.2.7. Choose or create
prompts that address
psychological trauma.
(B1R804) (P2N)
0 1 0 1 2 0 0 0 0
36 A 1.7.2.8. Choose or create
prompts that address universal
feelings (i.e., betrayal, courage,
compassion, etc.) (B1R554)
(P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 196
37 A 1.7.3. Collect prompts from
Internet sources or college
course sources. (A1R238) If
needed, reverse engineer
prompts to make them work for
your particular class. (AIR420,
B1R820) (P1) (P2)
1 1 0 1 3 0 0 0 0
38 A 1.7.4. Prepare to walk
students through the thought
process that resulted in the
prompt. (A1R420-449)
1 0 0 1 2 0 0 0 0
39 A 1.7.5. Allow students, with
your prior approval (D1R), to
come up with their own prompts
that address the text. (B1R847)
(P2N)
0 1 0 1 2 0 0 0 0
40 A 1.7.6. Prepare to provide
instruction in prompt analysis
using the prompts collected to
give students practice in
breaking apart writing prompts
(See Procedure 5). (B1R240)
(P2N)
0 1 0 1 2 0 0 0 0
41 D 1.7.7. IF you have a College
Prep (CP) class, THEN prepare
to provide scaffolding and
direct instruction throughout
unit to support students as
progress through literature unit
and writing assignment. (A1R,
1 0 1 1 3 0 0 0 0
COGNITIVE TASK ANALYSIS 197
C1R1523) (P1, P3) (D1R)
42 D 1.7.8. IF you have an
Advanced Placement (AP)
class, THEN consider providing
less scaffolding and allow
students more latitude in
making their own writing
choices. (A1R, C1R1500-1502)
(P1, P3)
1 0 1 1 3 0 0 0 0
43 A 1.8. Create a rubric. (A1R,
B1R, C1R, D1R)
1 1 1 1 4 0 0 0 0
44 A 1.8.1. Consider what rubric
will be used to assess writing
assignments. (A1R,
B1R,C1R172-173) (P1, P2, P3)
1 1 1 1 4 0 0 0 0
45 A 1.8.2. Use lesson objectives
to determine what gets assessed.
(C1R171, D1R)(P3N)
0 0 1 1 2 0 0 0 0
46 D 1.8.2.1. IF you are
assessing your students’
thinking, THEN create a prompt
that assess students’
metacognitive processing used
in developing their writing.
(A1R172-173) (P3N) (41R)
1 0 1 1 3 0 0 0 0
COGNITIVE TASK ANALYSIS 198
47 D 1.8.2.2. IF you want to
assess students’ argumentation
based on discussion held in
class, THEN build rubric with
students. (See steps 3.8)
(31R692) (P3N)
0 0 1 1 2 0 0 0 0
48 A 1.8.3. Prepare to give rubric to
students before writing begins.
(B1R, C1R698, 708, D1R)
(P2N, P3) (D1R)
0 1 1 1 3 0 0 0 0
2. Procedure 2. Assess the student’s prior writing knowledge.
(A1R)
7 7
1 A 2.1. By the third day of school,
have students write to a prompt
without guidance for one class
period. (A1R140) (P1)
1 0 0 1 2 0 0 0 0
2 A 2.2. Collect papers and identify
areas needing writing
instruction as determined by
teacher experience (based on
academic level and age level of
students) that will be addressed
in future instruction. (A1R141,
144-148) (P1)
1 0 0 1 2 0 0 0 0
3 D 2.2.1. IF more than 70% of
papers show particular area(s)
needing improvement, THEN
identify and list each area of
needed improvement for future
lessons. (A1R144-146) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 199
4 D 12.2.2. IF less than 70% of
papers have a particular area
needing improvement, THEN
consider revisiting these topics
as time allows throughout year.
(A1R144-146) (P1)
1 0 0 1 2 0 0 0 0
5 A 2.4. Review papers for
grammatical errors (162-166)
(A1R166, B1R, C1R, D1R)
(P1)
1 1 1 1 4 0 0 0 0
6 D 2.4.1. IF a student’s paper
shows adequate knowledge of
grammar, THEN paper will be
handed back to student without
comment regarding
grammar.(A1R162) (P1)
1 0 0 1 2 0 0 0 0
7 D 2.4.2. IF an individual
student’s paper shows an
overabundance of grammar
mistakes, THEN teacher
provides grammar instruction to
individual students either
through comments on paper or
with a short conference when
papers are handed back to
students. (A1R163-165) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 200
8 D 2.4.3. IF a preponderance of
student writing show a need for
a particular grammar topic to be
reviewed, THEN explicitly
review grammar topic in whole
class instruction. (A1R163) (P1)
1 0 0 1 2 0 0 0 0
9 A 2.5. Consult list of areas of
student writing that need
improvement. Decide which can
be best addressed in upcoming
literature unit. (A1R201-202)
(P1)
1 0 0 1 2 0 0 0 0
10 A 2.5.1. Choose 2-3 areas to
cover during each writing
instruction to teach within
chosen literature unit. (A1R201)
(P1) (41RN)
1 0 0 1 2 0 0 0 0
11 D 2.5.2. IF the literature text
has background information that
is unfamiliar to students, archaic
language, and/or needs teacher
scaffolding for comprehension,
THEN choose fewer areas of
needed improvement to cover
during subsequent writing
instruction. (A1R200-201) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 201
12 D 2.5.3. IF literature unit has
more familiar background
information, modern languages,
and/or needs less explanation
for comprehension, THEN
cover more areas of needed
writing improvement during
subsequent writing instruction.
(A1R202-203) (P1)
1 0 0 1 2 0 0 0 0
13 A 2.6. Gather examples of student
writing that exemplify areas of
needed improvement and
mentor texts that exemplify
quality writing to use during
writing instruction (see step
8.4). (A1R170-171, B1R, C1R,
D1R) (P1, P2, P3)
1 1 1 1 4 0 0 0 0
14 A 2.7. Move into first literature
unit as prescribed by grade level
or site pacing. (A1R182, B1R,
C1R, D1R) (P1, P2, P3)
1 1 1 1 4 0 0 0 0
3. Procedure 3. Implement Literature Unit. (A1R, B1R,
C1R)
62 9
1 A 3.1. Assign text (article, blog,
novel, etc.) for students to begin
reading. (A1R182, B1R1188,
C1R 193, D1R)
1 1 1 1 4 0 0 0 0
2 A 3.2. Define for students that the
term “text” may refer to any
written work. (C1R193-194)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 202
(P3N)
3 A 3.2.1. Instruct students to use
text to gather ideas throughout
reading of text. (A1R, B1R,
C1R199)
1 1 1 1 4 0 0 0 0
4 A 3.2.2. Guide students through
class discussion on making
connections between these
gathered thoughts and ideas and
to what they are reading.
(C1R195) (P3N)
0 0 1 1 2 0 0 0 0
5 A 3.2.2.1. Discuss with students
that connections between what
they have gathered and those
within the assigned text can be
made in the following ways
(C1R201-202) (P3N)
0 0 1 1 2 0 0 0 0
6 A 3.2.3. Discuss with students
how these connected ideas can
create new ideas. (C1R201)
Model this by taking your own
thoughts, ideas, and
observations and connect them
to the readings assigned or to
what you might be reading
personally at the time.
(C1R198) (P3N)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 203
7 D 3.3. IF you are using a novel as
your reading text, THEN
provide students a copy of
novel. (B1R717) (P2N)
Encourage students to procure
their own personal copy.
(41R)(4N)
0 1 0 1 2 0 0 0 0
8 A 3.3.2. Before beginning to read
the novel, pass out to students a
list of prompts from which they
can choose to write. (B1R197)
Tell students, “Start thinking
about which prompts you might
want to write to and start
collecting your evidence now.”
(B1R200-202) (P2N)
0 1 0 1 2 0 0 0 0
9 A 3.3.3. Pass out rubric at this
time that will be used to assess
the final paper unless you are
creating one with students (see
step 3.8). The rubric created
with students will be developed
before writing instruction
begins. (B1R1056, C1R, D1R)
(P2N)
0 1 1 1 3 0 0 0 0
10 A 3.3.5. Think about reading the
novel whole-class. (B1R1190)
(P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 204
11 D 3.4. IF you are using a critical
analysis of a literary work as
your reading text, THEN
provide students a copy.
(A1R175-177, B1R1313,
C1R414-417) (P1, P2, P3)
1 1 1 1 4 0 0 0 0
12 A 3.4.1. Tell students that critical
essays provide someone else’s
words and arguments to help
support what you will be saying
about the novel. (B1R1401)
(P2N)
0 1 0 1 2 0 0 0 0
13 A 3.5. As you read, remember to
constantly assess student
knowledge of the
subject/topics/themes being
discussed. Assess students
explicitly if they understand
what is being discussed. (A1R,
B1R271) (P1, P2)
1 1 0 1 3 0 0 0 0
14 A 3.5.1. Use quizzes, objective
tests, short answer responses, or
on-demand writing assignments.
(41R)(4N)
0 0 0 1 1 0 0 0 0
15 A 3.5.2. Have students respond to
a prompt writing their response
in the form of a body paragraph.
(see step 8.11) Students submit
a paragraph and then are
required to respond to at least
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 205
two other student submissions.
(B1R91) (P2N)
16 A 3.5.3. Look for evidence
through monitoring discussion
and discussion board
submissions that students are
thinking deeply about the novel.
B1R, D1R) (P2N)
0 1 0 1 2 0 0 0 0
17 A 3.5.4. Use class discussion,
tests, quizzes. short answer
response, and on-demand
writing assignments to help
students develop thinking in the
following ways: (B1R, 41R)(P2,
4)
0 1 0 1 2 0 0 0 0
18 A 3.5.4.1. Help students develop
deeper thinking in the lives of
the characters; (B1R328) (P2N)
0 1 0 1 2 0 0 0 0
19 A 3.5.4.2. Help students develop
deeper thinking in the emotions
of the characters; (B1R328)
(P2N)
0 1 0 1 2 0 0 0 0
20 A 3.5.4.3. Help students develop
deeper thinking in the
psychological development of
the characters. (B1R329) (P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 206
21 A 3.5.4.4. Help students develop
deeper thinking in Contextual
analysis. (B2R329) (P2N)
0 1 0 1 2 0 0 0 0
22 A 3.5.4.5. Help students develop
deeper thinking in the influence
of setting on characters.
(B2R329) (P2N)
0 1 0 1 2 0 0 0 0
23 A 3.5.4.6. Help students see that
they must connect the
characters with today’s world or
the story’s context or their
writing becomes superficial.
(B1R334) (P2N)
0 1 0 1 2 0 0 0 0
24 A 3.5.5. Discuss with students the
big ideas identified during
reading of the novel. (A1R,
B1R221, D1R) (P1, P2, 4)
1 1 0 1 3 0 0 0 0
25 A 3.5.6. Be ready for diversions in
class conversations as students
take discussion in different
directions. (B1R222) (P2N)
0 1 0 1 2 0 0 0 0
26 A 3.6. Model annotation of
literary text with students.
(A1R, B1R903, C1R)
1 1 1 1 4 0 0 0 0
27 A 3.6.1. Model Naked Reading as
an annotation strategy with
students. Take an unmarked
page with no annotations and
model Naked Reading. Naked
Reading is showing to students
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 207
how you, as a reader, read a
page for the first time.
(B1R892-897) (P2N)
28 A 3.6.1.1. Think out loud to
students as you hold discussion
between yourself and the text
about what inspires you within
the page being read. (B1R893,
901, C1R) (P2N, P3)
0 1 1 1 3 0 0 0 0
29 A 3.6.2. Instruct students to
annotate create a color code list
inside cover of their novels.
(B2R375) (P2N)
0 1 0 1 2 0 0 0 0
30 A 3.6.2.1. Students annotate their
novels and the critical essays
using sticky notes and color
coding. (B1R374-375) (P2N)
0 1 0 1 2 0 0 0 0
31 A 3.6.2.2. Provide to students or
have them bring sticky notes
and highlighters to color code
their annotations. (D1R)(4N)
0 0 0 1 1 0 0 0 0
32 A 3.6.3. Work with students
during reading to identify those
concrete details or literary
features of the story’s themes
(pre-identified by you during
planning phase) that will be
useful in answering the prompts
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 208
assigned or chosen by students.
(B1R1168) (P2N)
33 D 3.6.3.1. IF a prompt addresses
literary features within the
novel, THEN students will
identify and highlight instances
where these features manifest
themselves in the novel.
(B1R1132) (P2N)
0 1 0 1 2 0 0 0 0
34 D 3.6.3.2. IF a prompt addresses a
theme or motif within the novel,
THEN students will identify
concrete details that exemplify
this theme. (B1R1130) (P2N)
0 1 0 1 2 0 0 0 0
35 D 3.7. IF you are using a critical
literary essay as your text,
THEN do the following: (A1R,
B1R, D1R)
1 1 0 1 3 0 0 0 0
36 A 3.7.1. Discuss with students
how critical literary analysis
piece incorporates literature into
written argument. (A1R176)
(P1)
1 0 0 1 2 0 0 0 0
37 A 3.7.2. Give instruction on the
elements that make this literary
analysis piece a good writing
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 209
example. (A1R246) (P1)
38 A 3.7.3. Direct students to keep
literary analysis piece as a job
aid for future writing
assignments throughout year.
(A1R191-192) (P1)
1 0 0 1 2 0 0 0 0
39 A 3.7.4. Work with students to
find the following from the
critical essays assigned: (A1R,
B1R, D1R)
1 1 0 1 3 0 0 0 0
40 A 3.7.4.1. Find evidence
supporting writer’s claims.
(A1R, B1R1367, D1R) (P1, P2)
1 1 0 1 3 0 0 0 0
41 A 3.7.4.2. Find counterarguments
from essays to give students
perspectives they might not
have considered yet. (A1R,
B1R1404) (P1, P2)
1 1 0 1 3 0 0 0 0
42 A 3.7.4.3. Ask students to
consider if the critic has said
something that the student can
use to support their own thesis.
(B1R1405) (P2N)
0 1 0 1 2 0 0 0 0
43 A 3.7.5. Have students color code
what the literary critic said
about the text as it relates to the
prompt. (B1R1314) (P2N)
(41R)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 210
44 A 3.7.6. As an alternative
annotation strategy, teach
students how to annotate using
SOAPSTone. (P3N)
0 0 1 1 2 0 0 0 0
45 A 3.7.7 Teach students in the
beginning of the school year
how to use SOAPSTone.
(C1R941) (P3N)
0 0 1 1 2 0 0 0 0
46 A 3.7.8. Use visual aids such as
photography or artwork to
introduce SOAPSTone to
students. (C1R962) (P3N)
0 0 1 1 2 0 0 0 0
47 A 3.7.8.1. Provide examples of
photographs or art for students
to analyze. Discuss with
students the choices the
photographer makes while
applying SOAPSTone as an
analytic framework. (C1R963)
(P3N)
0 0 1 1 2 0 0 0 0
48 A 3.7.9. Once this has been
practiced, guide students in
applying SOAPSTone at the
textual level. (C1R966) (P3N)
0 0 1 1 2 0 0 0 0
49 D 3.8. IF students are writing an
argumentative paper responding
to arguments in opinion pieces,
THEN do the following: (P3N)
(41R)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 211
50 A 3.8.1. Ask students to find and
read opinion pieces with
different takes on the same
subject. (C1R871-872, 883)
(P3N)
0 0 1 1 2 0 0 0 0
51 A 3.8.2. Assign students to write
on that subject by taking a side
using those sources to support
their argument. (C1R388-392)
(P3N)
0 0 1 1 2 0 0 0 0
52 A 3.8.3. Tell students to find
articles that support
counterclaims or counter
arguments to the stand they
have chosen. (C1R896-897)
(P3N)
0 0 1 1 2 0 0 0 0
53 D 3.9. IF you want to assess
student writers’ knowledge of
argumentation, THEN build a
rubric with students using the
following questions to stimulate
discussion (see procedure
3.2.3): (31R) (3N)
0 0 1 1 2 0 0 0 0
54 A 3.9.1. While building the rubric,
ask students, “What did we just
learn about argumentative
writing?” (31R)
0 0 1 1 2 0 0 0 0
55 A 3.9.2. Ask students, “What shall
we expect in this paper?” (31R)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 212
56 A 3.9.3. Ask students, “What
should that look
like?”(C1R693-696) (P3N)
0 0 1 1 2 0 0 0 0
57 D 3.10. IF students are reading
articles as examples of
rhetorical devices, THEN work
with the class to identify and
analyze the rhetorical device.
(P3N)
0 0 1 1 2 0 0 0 0
58 A 3.10.1. Have students analyze
how that rhetorical device
contributes to or creates the
author’s purpose. (C1R1001)
(P3N)
0 0 1 1 2 0 0 0 0
59 A 3.10.2. Have students analyze
how that rhetorical device helps
connect the article to the
audience. (C1R1001) (P3N)
0 0 1 1 2 0 0 0 0
60 A 3.10.3. Have students analyze
how that rhetorical device
connects readers to the pathos
(emotional appeal), ethos
(ethical appeal), or logos
(logical appeal) of the article
and its effect on the text.
(C1R456, C2R) (P3N)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 213
61 D 3.11. IF students are struggling
with a particular writing skill
(i.e., transitions, theme,
narrative, etc.), THEN have
students read articles that
contain examples of this skill
and analyze its use and effect in
the text. (C1R456-458, C2R)
(P3N)
0 0 1 1 2 0 0 0 0
62 A 3.12. Consider blogging (see
procedure 12) for students to
use as an additional writing
exercise (P3N)
0 0 1 1 2 0 0 0 0
63 A 3.12.2. Instruct students to keep
a notebook of themes and ideas
they learn from reading various
texts. (31R)(3N)
0 0 1 1 2 0 0 0 0
64 A 3.13. Teach students to discover
their own needs within their
writing by asking the following
questions. (C1R488) (P3N)
0 0 1 1 2 0 0 0 0
65 A 3.13.1. Teach them to ask,
“What are you learning from
other writers?” (C1R494) (P3N)
0 0 1 1 2 0 0 0 0
66 A 3.13.2. Teach them to ask,
“What do I notice that the writer
does in the article (book, novel,
etc.)?” (C1R469) (P3N)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 214
67 A 3.13.3. Teach them to ask,
“How does the writer
accomplish this? (C1R469)
(P3N)
0 0 1 1 2 0 0 0 0
68 A 3.13.4. Teach them to ask,
“What can I take from this
article? What can I learn from
the author?” (C1R470) (P3N)
0 0 1 1 2 0 0 0 0
69 A 3.13.5. Teach them to ask,
“What can you do in your
writing? Did the writer conform
to standard writing conventions,
or did he/she break the rules?
Does it work in the article?”
(C1R496, C2R) (P3N)
0 0 1 1 2 0 0 0 0
70 A 3.13.6. Teach them to ask,
“What did the writer craft in
his/her article that surprised
you?” (C1R496, C2R) (P3N)
0 0 1 1 2 0 0 0 0
71 A 3.13.7. Teach them to ask
themselves, “What does the
writer want you to know?” (4N)
0 0 0 1 1 0 0 0 0
Procedure 4. Present topic as a problem.
(A1R361-363)
18 4
1 A 4.1. Tell students that an essay
is a physical manifestation of a
thought. (A1R26) Convey to
students that writing an essay is
a demonstration of a problem
through writing and the solution
1 0 1 1 3 0 0 0 0
COGNITIVE TASK ANALYSIS 215
comes from thinking. (A1R278-
279, C1R143) (P1) (P3)
2 A 4.2. Instruct students that they
can learn to write better by what
they read. (C1R498) (3PN)
0 0 1 1 2 0 0 0 0
3 A 4.3 Use Socratic Questioning
(SQ) to lead students to ideas,
thoughts, and values applicable
to coming up with a answering
a question or choosing a side in
an argument. (A1R266-267)
(P1)
1 0 0 1 2 0 0 0 0
4 A 4.4. Determine when to begin
Socratic Questioning (SQ) and
developing a question to
address in an essay. Decide
whether this brainstorm
discussion comes before,
during, or after the literature
unit. (A1R282-299, A1R334-
347) (P1)
1 0 0 1 2 0 0 0 0
5 D 4.5. IF you are teaching
literature units in the beginning
of the school year, THEN
engage in SQ and brainstorming
after completing a literature
unit. (A1R284) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 216
6 D 4.6. IF you are teaching
literature units in the middle of
the school year, THEN engage
in SQ and brainstorming during
the literature unit with less
frontloading and scaffolding.
(A1R282) (P1)
1 0 0 1 2 0 0 0 0
7 D 4.7. IF you are teaching
literature units around
springtime (2nd semester),
THEN engage in SQ and
brainstorming at the beginning
of the literature unit. (A1R282)
(P1)
1 0 0 1 2 0 0 0 0
8 D 4.8. IF you are teaching
literature units around
springtime (2nd semester),
THEN engage in SQ and
brainstorming at the beginning
of the literature unit. (A1R282)
(P1)
1 0 0 1 2 0 0 0 0
9 A 4.8.1. Engage in Socratic
Questioning by asking students,
“What values might be
important in the culture within
the literary text?” (A1R269)
(P1)
1 0 0 1 2 0 0 0 0
10 A 4.8.2. Ask students, “How is
this apparent to the student?”
(A1R271) (P1
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 217
11 A 4.8.3. Ask students, “How is
this defended?” (A1R272P1)
1 0 0 1 2 0 0 0 0
12 A 4.8.4. Ask students, “How do
you find examples in the text?”
(A1R272) (P1)
1 0 0 1 2 0 0 0 0
13 A 4.8.5. Ask students, “How do
you find counterexamples?”
(A1R274) (P1)
1 0 0 1 2 0 0 0 0
14 A 4.8.6. Ask students, “Why
would this be a good writing
technique?” (A1R274) (P1)
1 0 0 1 2 0 0 0 0
15 A 4.8.7. Ask students, “How does
this strengthen your argument?”
(A1R276) (P1)
1 0 0 1 2 0 0 0 0
16 A 4.9. Engage in this
brainstorming discussion on text
just read for about ½ half a class
period. (A1R276) (P1)
1 0 0 1 2 0 0 0 0
17 A 4.10. Use SQ and brainstorming
to come up with a central idea.
(A1R620, D1R) A central idea
is a short 2-3 sentence
description of what the student
author is trying to say. (P1)
1 0 0 1 2 0 0 0 0
18 A 4.10.1. Consider using the term
central idea rather than thesis
statement, especially for
struggling writers. (A1R630,
D1R) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 218
19 A 4.11. Demonstrate to students
that brainstorming and Socratic
Questioning are ways of
thinking that lead to a question
that can be responded to
through writing an expository
piece. (A1R277) (P1) (C1R148)
(P3)
1 0 1 1 3 0 0 0 0
20 A 4.12. Take ideas developed
during brainstorming and get
them down on paper. (C1R150-
151) (P3N)
0 0 1 1 2 0 0 0 0
21 A 4.12.1. Instruct students to
engage in Zero Drafting. Zero
Drafting is a strategy of
allowing students to write
without thought of editing or
limitations. Students are
allowed to write anything-
answer prompt, begin a novel,
write a journal entry, write
poetry-to stimulate ideas and
encourage writing.
0 0 1 1 2 0 0 0 0
22 A 4.12.1.1. Model Zero Drafting
with students and discuss your
choices in your own writing.
(C1R1084-1087) (P3N)
0 0 1 1 2 0 0 0 0
5. Procedure 5. Introduce Prompt/ Prompt Analysis. (A1R,
B1R)
15 0
COGNITIVE TASK ANALYSIS 219
1 A 5.1. Demonstrate to students the
questions and authentic thinking
that led to the creation of
prompts. (A1R420-449) (P1)
1 0 0 1 2 0 0 0 0
2 A 5.2. Discuss with students that
writing prompts can be used to
set up the structure of an essay.
(B1R232) (P2N)
0 1 0 1 2 0 0 0 0
3 A 5.3. Discuss with class each
prompt choice for the novel to
be read. (B1R392) (P2N)
0 1 0 1 2 0 0 0 0
4 A 5.3.1. Show students that
writing prompts can be broken
down so that they can help in
developing the paper to be
written. (B1R)(P2N)
0 1 0 1 2 0 0 0 0
5 A 5.3.2. Analyze the writing
prompt to see if it can show
how the thesis should look.
(B1R233) The thesis or central
idea is the question from the
prompt that students need to
answer. (B1R1027) (P2N)
0 1 0 1 2 0 0 0 0
6 A 5.3.3. Analyze the writing
prompt to see if it identifies the
topics of the body paragraphs.
(B1R234) These are the
arguable thesis statements
derived from the main thesis of
the paper. (B1R1030) (P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 220
7 A 5.3.4. Analyze the writing
prompt to see if it can identify
what must be included in the
conclusion. (B1R235) (P2N)
0 1 0 1 2 0 0 0 0
8 A 5.4. Have students begin color
coding the writing prompt of
their choice. (B1R1004) (P2N)
0 1 0 1 2 0 0 0 0
9 A 5.5. Use different colors for
each of the following steps.
(B1R1005) (P2N)
0 1 0 1 2 0 0 0 0
10 A 5.5.2. Have students highlight
the verbs within prompt.
(B1R1007) (P2N)
0 1 0 1 2 0 0 0 0
11 A 5.5.3. Have students highlight
the writing task. (B1R1007)
(P2N)
0 1 0 1 2 0 0 0 0
12 A 5.5.4. Have students highlight
the textual situation presented in
the prompt. Textual situations
are often included by prompt
writers to help the students
think critically and provide
focus about the prompt.
(B1R1007, B2R) (P2N)
0 1 0 1 2 0 0 0 0
13 A 5.5.5. Determine together as a
class if the prompt they have
chosen sets up the scene to be
discussed in the paper.
(B1R1013) (P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 221
14 A 5.5.6. Have students highlight
the parts that make an arguable
thesis statement. These drive the
formation of the body
paragraphs. (B1R1029) (P2N)
0 1 0 1 2 0 0 0 0
15 A 5.5.7. Have students highlight
within prompt what is going
into the conclusion. (B1R1045)
(P2N) Each prompt may or may
not provide this information.
(41R)(4N)
0 1 0 1 2 0 0 0 0
6. Procedure 6. Provide Feedback from Diagnostic Papers.
(A1R)
10 2
1 A 6.1. Prepare to present to
students examples from
diagnostic writing piece that
shows areas needing
improvement (from step 2.6).
(A1R247) (P1)
1 0 0 1 2 0 0 0 0
2 A 6.2. Show examples from
diagnostic papers (see
Procedure 2) of areas that need
improvement. (A1R172, 243)
(P1)
1 0 0 1 2 0 0 0 0
3 A 6.2.1. Provide suggestions and
approaches on how students can
fix these areas of concern. Tell
students, “These are examples
from your papers, and here are
some ways you can fix them.
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 222
(A1R243) (P1)
4 A 6.2.1.1. Consider providing
students an editing checklist and
have them assess student work
as a whole class assignment.
(D1R)(4N
0 0 0 1 1 0 0 0 0
5 A 6.3. Use discussion to show
students they can improve
future writing pieces. (A1R344)
(P1)
1 0 0 1 2 0 0 0 0
6 A 6.4. Let students know that they
will begin writing their papers
the next day and should keep
these suggestions and
approaches in mind as they
write. (A1R231) (P1)
1 0 0 1 2 0 0 0 0
7 A 6.5. Remind students that a
thesis statement is the main
argument and has 3 parts.
(A1R689)(P1)
1 0 0 1 2 0 0 0 0
8 A 6.5.1. Instruct students that their
paper needs a clear designation
of what it is the student writer is
going to talk about.
(A1R690)(P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 223
9 A 6.5.2. Instruct students that their
paper is making a clear
judgment about something
(A1R691)(P1)
1 0 0 1 2 0 0 0 0
10 A 6.5.3. Instruct students that their
paper has chosen a side to
support and that it needs to be
meaningful in some way.
(A1R691)(P1)
1 0 0 1 2 0 0 0 0
11 D 6.6. IF students are in AP class,
THEN these three parts are
given more direct emphasis.
(A1R686) (P1)
1 0 0 1 2 0 0 0 0
12 D 6.7. IF students are in a CP
class, THEN the instructor
attempts to guide students
towards this goal. (A1R687)
(P1)
1 0 0 1 2 0 0 0 0
7. Procedure 7. Construct Essay Outline (P2N,
41R)
5 0
1 A 7.1. Students begin to outline
their essay. (B1R362)(41R)
0 1 0 1 2 0 0 0 0
2 A 7.2. Consider allowing students
to get together in groups of 3-4
who are working on the same
prompt to work collaboratively.
(B1R359-360) (P2N) They
support one another as they find
concrete details and literary
features to address their prompt.
0 1 1 1 3 0 0 0 0
COGNITIVE TASK ANALYSIS 224
(B1R363) (P2N) (C1R542)(P3)
3 A 7.3. Have students take the
results from prompt analysis
and identify the body
paragraphs. (B1R366) (P2N)
0 1 0 1 2 0 0 0 0
4 A 7.4. Have students find the
concrete details they will use in
each body paragraph and insert
into outline. (B1R367, 406)
(P2N)
0 1 0 1 2 0 0 0 0
5 A 7.5. Have students write into the
outline what they are going to
say about the concrete details
(commentary). (B1R369) (P2N)
0 1 0 1 2 0 0 0 0
8. Procedure 8. Provide Writing Instruction. (A1R, B1R,
C1R, 41R)
49 2
1 A 8.1. Introduce rhetorical
features, style structures, or
skills that you have decided to
cover for this writing piece (see
step 1.3.4). (C1R695)(P3N)
critical analysis in essays
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 225
2 A 8.2. Remind students what is
expected in each paper based on
instruction given and the rubric
used to assess. (B1R1056)(P2N)
(C1R707)(P3)
0 1 1 1 3 0 0 0 0
3 A 8.2.1. Tell students that papers
will be assessed according to
their response to the prompt
along with discussion of
articles, novels, or other texts
which have been notated in
student notebooks (see step
1.10). (C1R693-694) (P3N)
0 0 1 1 2 0 0 0 0
4 A 8.3. Review prompts with
students so they know those
requirements that are needed for
a good paper. (B1R, C1R707)
(P2, P3)
0 1 1 1 3 0 0 0 0
5 A 8.4. Show students models of
exemplary essays or mentor
texts (see step 2.7). Discuss
with students each exemplary
essay and how the claims are
addressed. (B1R425, C1R1209)
(P2N) (P3)
0 1 1 1 3 0 0 0 0
6 A 8.4.2. Have students identify
what the writer has done well
and think about how they as
writers begin to move towards
that goal. (C1R1178)(P3N)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 226
7 A 8.4.3. Have students begin
writing everything they want to
say to the topic being made
within the assigned text.
(C1R513) (P3N)
0 0 1 1 2 0 0 0 0
8 A 8.5. Ask students to look back
and decide if they have said
what they wanted to say.
(C1R515) (P3N)
0 0 1 1 2 0 0 0 0
9 A 8.6. Begin to write body
paragraphs. (P1) (P2) (P3) (4)
1 1 1 1 4 0 0 0 0
10 A 8.6.1. Pull class together for
whole-group discussion on
creating arguable topic
sentences using the concrete
details previously found. They
are also referred to as thesis
statements. (B1R521) (P2N)
0 1 0 1 2 0 0 0 0
11 A 8.6.2. Have discussion with the
class about what the arguable
topic sentences say and what the
writer is going to say about
them. (B1R416-417) (P2N)
0 1 0 1 2 0 0 0 0
12 A 8.6.3. Provide support to
students by checking for a thesis
statement. (A1R467,
B1R1521)(P1, P2)
1 1 0 1 3 0 0 0 0
COGNITIVE TASK ANALYSIS 227
13 A 8.7. Remind students that each
body paragraph should begin
with a topic sentence, followed
by concrete details that support
the topic sentence, and a
concluding sentence to wrap up
what has just been said and
point to the next paragraph in
the essay. (A1R583,
B1R462,C2R466)(P1) (P2) (P3)
1 1 1 1 4 0 0 0 0
14 A 8.7.1. Provide support by
helping them find examples,
support from text, or support
from secondary sources (if
applicable). (A1R469, 558-559)
Students may know the term
“support from text” as concrete
details, or details that can be
found directly within a text.
(A1R583, B1R462) (P1) (P2)
1 1 0 1 3 0 0 0 0
15 A 8.7.2. Support students by
helping them identify or narrow
their central idea or argument.
(A1R470) (P1)
1 0 0 1 2 0 0 0 0
16 A 8.7.3. Check how all the
elements noted above are
incorporated into the text.
(A1R559-560) (P1)
1 0 0 1 2 0 0 0 0
17 A 8.7.4. Instruct students that they
may choose the structure of
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 228
their body paragraphs.
(B1R466) (P2N)
18 A 8.7.5. Tell students that they
may deviate from this format as
they see the need to add
explanation or to set up a scene
from the novel. (B1R467, B2R)
(P2N)
0 1 0 1 2 0 1 0 0
19 A 8.7.6. Tell students to make
their structure flow in a natural
way that makes sense to them.
(B1R456) (P2N)
0 1 0 1 2 0 0 0 0
20 A 8.7.7. Refrain from providing
concrete guidelines in the
amount of quotes or support
needed by students in writing an
argument. Tell them they have
the responsibility to use enough
textual support to your point.
(A1R561-563, B1R1430) (P1)
(P2)
1 1 0 1 3 0 0 0 0
21 A 8.7.8. Tell students to review
thesis statements for rewriting
or revision as they complete
body paragraphs.
(B1R2298)(P2N)
0 1 0 1 2 0 0 0 0
22 A 8.8. Begin to write conclusion.
(B1R539)(P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 229
23 A 8.8.1. Tell students that their
prompt will often tell them what
type of conclusion to write.
(B1R542) (P2N)
0 1 0 1 2 0 0 0 0
24 D 8.8.1.1. IF the prompt has a
question that asks students to
reflect or gives students a task
to do within their writing,
THEN this question, once
restated, becomes the basis for
the conclusion. (B1R1678-
1682) (P2N)
0 1 0 1 2 0 0 0 0
25 D 8.8.1.2. IF the prompt does give
a task to do or is the task is
more open-ended, THEN the
prompt has not indicated what
the conclusion should be. This
would then require a more
formal conclusion which entails
a restating of the introduction.
(B1R1682-1684) The Common
Core writing rubric can help
guide instruction a more formal
conclusion. (B1R1703-1704)
(P2N)
0 1 0 1 2 0 0 0 0
26 A 8.8.2. Encourage students to
avoid the phrase, “In
conclusion.” (B1R547) (P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 230
27 A 8.8.3. Instruct students that a
formal conclusion is the reverse
of the introduction but is not an
exact copy. The conclusion is a
restatement of what is said in
the essay, but with a more
reflective look at the
significance of the thesis.
(B1R1700-1702) (P2N)
0 1 0 1 2 0 0 0 0
28 A 8.8.4. In writing a more formal
conclusion, tell students to
begin with a restatement of their
thesis in a differently from what
was written in the introduction.
(B1R566, 1734) (P2N)
0 1 0 1 2 0 0 0 0
29 A 8.8.5. Tell students to restate the
points or claims made in the
body paragraphs. (B1R1734)
(P2N)
0 1 0 1 2 0 0 0 0
30 A 8.8.5.1. Tell students that the
last statement of the conclusion
is a larger statement that is more
universal and addresses the
theme of the paper. (B1R567-
568, 1734) (P2N)
0 1 0 1 2 0 0 0 0
31 A 8.9. Begin to write introductory
paragraph as the last step in
completing draft.
(B1R1904)(P2N) (C1R537-538)
(P2N, P3)
0 1 1 1 3 0 0 0 0
COGNITIVE TASK ANALYSIS 231
32 A 8.9.2. Tell students that an
introductory paragraph may
start with a broad general
thematic statement sentence that
has to do with the topic or
theme at hand. (B1R550-551,
1896, B2R) This is common but
not required. Students may
choose to move their general
thematic statement further into
the paragraph. (B1R1898)
(P2N) (C1R538)(P3)
0 1 1 1 3 0 0 0 0
33 A 8.9.3. Tell students that the first
sentence could also be a theme
statement. Theme statements do
not mention the novel at all.
They are universal statements
that pertain to people. (B1R552-
554) (P2N)
0 1 0 1 2 0 0 0 0
34 A 8.9.4. Tell students that the next
sentence is narrowed down to
the particular text, novel, or
characters being discussed.
(B1R554)(P2N)
0 1 0 1 2 0 0 0 0
35 A 8.9.5. Tell students that the next
sentence is the thesis, or the
main argument of the writing
piece. (B1R557) (P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 232
36 A 8.9.6. Instruct students that in
writing an introductory
paragraph, the writing starts
with a broad, universal
statement which narrows to the
thesis which is exactly what the
paper is going to talk about.
(B1R556-558) (P2N)
0 1 0 1 2 0 0 0 0
37 A 8.10. Provide additional
instruction to class on revising
and improving papers before
engaging in Writers Workshop.
(41R)(4N)
0 0 0 1 1 0 0 0 0
38 A 8.11. Instruct students on the
use of linkages and transitions
between sentences and
paragraphs. (B1R1944)
0 1 0 1 2 0 0 0 0
39 A 8.11.1. Tell students that
transition words are used to link
sentences or ideas within
paragraphs. (A1R) (B1R1961)
1 1 0 1 3 0 0 0 0
40 A 8.12. Tell students that linkages
connect paragraphs to each
other. Linkages occur when a
writer takes a few words from
the last sentence of a paragraph
and uses those same words in
the first sentence of the next
paragraph. (B1R1944)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 233
41 A 8.13. Consider providing
additional instruction in the use
of SOAPSTone (see procedure
2) as a strategy for students to
review their own writing.
(C1R)(P3N)
0 0 1 1 2 0 0 0 0
42 A 8.13.1. Tell students that, as
writers, they should consider
SOAPSTone as criteria for their
own writing. (C1R1004) (P3N)
0 0 1 1 2 0 0 0 0
43 A 8.13.2. Instruct students that the
SOAPSTone criteria guide the
decisions they make in an essay
or text (C1R1005) (P3N)
SOAPSTone can help guide
writers in their efforts to convey
meaning or improve writing
voice. (C1R847-848)
0 0 1 1 2 0 0 0 0
44 A 8.13.3. Use writing devices (i.e.
narrative, anecdote, etc.) to
soften or illustrate a point when
providing commentary in their
writing. (C2R847) (P3N)
0 0 1 1 2 0 0 0 0
45 A 8.13.3.1. Use comparisons that
help to define or describe.
(C2R847) (P3N)
0 0 1 1 2 0 0 0 0
46 A 8.13.3.2. Concentrate on the
flow and rhythm of the writing
as it is read by the reader.
(C2R847) (P3N)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 234
47 A 8.13.4. Guide students in
analyzing their writing to make
their sentences less complex or
obtuse. (C1R1538) Work with
students to do the following:
(P3N
0 0 1 1 2 0 0 0 0
48 A 8.13.5. Review verb usage by
having students think about the
ratio of passive verbs to active
verbs without affecting
meaning. (C2R)(P3N)
0 0 1 1 2 0 0 1 0
49 A 8.13.6. Have students check the
ratio of descriptive nouns to
abstract nouns. (C2R)(P3N)
0 0 1 1 2 0 0 0 0
50 A 8.13.7. Review the number and
variety of words in each
sentence. (C1R1548)(P3N)
0 0 1 1 2 0 0 0 0
51 A 8.13.8. Instruct students to
consider the rhythm within the
structure of their paragraph.
Discuss with students if longer,
passive sentences convey the
same message as shorter, more
direct sentences. (C1R1548)
(P3N)
0 0 1 1 2 0 0 0 0
Procedure 9. Conduct Writing Workshop (P1)
(P2) (P3)
45 11
1 A 9.1. Give students time to write.
Give time based on assignment
length (2-7 days). (A1R456,
1 1 1 1 4 0 0 0 0
COGNITIVE TASK ANALYSIS 235
464-465, B1R1416, C1R)(P1,
P2, P3)
2 D 9.1.2. IF a student is in an AP
class, THEN all papers
completed during 2nd semester
are timed in-class writing in
preparation for the upcoming
AP test. (A1R549-552, B1R,
C1R, D1R) (P1, P2, P3)
1 1 1 1 4 0 0 0 0
3 A 9.2. Circulate during Writer’s
Workshop and provide oral
feedback and individualized
instruction as needed. (A1R468,
B1R481, C1R316-317) (P1)
(P2) (P3)
1 1 1 1 4 0 0 0 0
4 A 9.2.1. Gauge student
understanding of expectations.
Be prepared to provide
additional instruction to correct
misunderstandings or to provide
more practice with a particular
skill. (C1R713) (P3N)
0 0 1 1 2 0 0 0 0
5 D 9.2.2. IF misunderstandings or
need for additional instruction
limit their ability write a quality
paper, THEN stop and address
the additional topic before
moving on in the writing
process. (C1R714)(P3N)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 236
6 D 9.3. IF misunderstandings or
need for additional instruction
do not limit the students’ ability
to write a quality paper, THEN
cover this additional topic
sometime during the draft
writing. (C1R715)(P3N)
0 0 1 1 2 0 0 0 0
7 A 9.3.1. Check first body
paragraphs during Writers
Workshop. (A1R468) (P1)
1 0 0 1 2 0 0 0 0
8 D 9.4. IF students are comfortable
with the direction of their drafts
and have shown you they are
capable writers (based on past
assignments), THEN leave
students alone to write.
(A1R474) Spot-check writing as
you circulate if possible.
(A1R512, B1R1416, C1R585)
(P1) (P3)
1 1 1 1 4 0 0 0 0
9 D 9.5. IF students lack confidence
or have shown they need more
support, THEN these students
are required to bring every
completed paragraph to you for
review. (A1R476, 509) (P1)
(C1R286-289)(P3)
1 0 1 1 3 0 0 0 0
10 A 9.6. Have students engage in
peer review during writer’s
workshop. (A1R, B1R, C1R)
1 1 1 1 4 0 0 0 0
COGNITIVE TASK ANALYSIS 237
(P1, P2, P3)
11 A 9.6.1. Allow students during
writing workshop to support
one another or provide peer
reflection during this part of
assignment. You may follow an
informal, semiformal, or formal
peer review process. (A1R) (P1)
1 0 0 1 2 0 0 0 0
12 D 9.7. IF you want students to
engage in some peer review of
papers, THEN follow the
semiformal peer review process
in steps 9.7.1 to 9.7.4.2.
(B1R2151)(P2N
0 1 0 1 2 0 0 0 0
13 A 9.7.1. Have students peer
review each other’s drafts.
(A1R580) Students may be
grouped together by the teacher
or by student choice.
(B1R2153)(P2N) (C1R238)(P3)
This process will occur twice
during each writing unit.
(C1R241,325)(P3N)
0 1 1 1 3 0 0 0 0
COGNITIVE TASK ANALYSIS 238
14 A 9.7.2. Consider pairing up weak
writers with strong writers who
would be willing to work with
another student in providing
support. (B1R2156) This
happens as you figure out the
skills and personality
characteristics of each student.
(A1R541-542, B1R2156)
(P1)(P2)
1 1 0 1 3 0 0 0 0
15 A 9.7.3. During peer review, have
students review the following as
peer editors. (P2N)
0 1 0 1 2 0 0 0 0
16 A 9.7.3.1. Have students
determine if writer has
answered the prompt;
(B12)(P2N)
0 1 0 1 2 0 1 0 0
17 A 9.7.3.2. Have students
determine if writer has written
effective transitions between
sentences within paragraphs;
(B2R)(P2N)
0 1 0 1 2 0 1 0 0
18 A 9.7.3.3. Have students
determine if writer has written
effective transitions between
paragraphs; (B2R)(P2N)
0 1 0 1 2 0 1 0 0
19 A 9.7.3.4. Have students
determine if writer has written
an effective conclusion that
wraps up the paper;
0 1 0 1 2 0 1 0 0
COGNITIVE TASK ANALYSIS 239
(B2R)(P2N)
20 A 9.7.3.5. Have students
determine if writer has written a
paper that makes sense.
(B2R)(P2N)
0 1 0 1 2 0 1 0 0
21 D 9.8. IF you feel students find
value and increased editing skill
from peer review, THEN
provide a formal peer review
process as outline in steps 9.8.1
to 9.8.11. (C1R646) (P3)
0 0 1 1 2 0 0 0 0
22 D 9.8.1. IF the class is a CP class,
THEN provide peer editors with
a checklist of what to look for in
their colleague’s papers.
(C1R1518) (P3N)
0 0 1 1 2 0 0 0 0
23 D 9.8.2. IF the class is an AP
class, THEN have peer editors
use the following strategy to
provide peer revision- Bless,
Press, Address (C1R1517).
(C1R635)
0 0 1 1 2 0 0 0 0
24 A 9.8.3. Instruct students do the
following: (P3N)
0 0 1 1 2 0 0 0 0
25 A 9.8.3.1. Bless-talk about what is
working in the paper.
(C1R635)(P3N)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 240
26 A 9.8.3.2. Press-press author to
improve where there is need for
improvement. (C1R635)(P3N)
0 0 1 1 2 0 0 0 0
27 A 9.8.3.3. Address-help address
author’s questions about their
paper. (C1R635)(P3N)
0 0 1 1 2 0 0 0 0
28 A 9.8.4. Model the review strategy
Bless, Press, and Address with
class, showing what peer editors
should be doing to writer’s
papers during revision process.
(C1R1180-1182) (P3N)
0 0 1 1 2 0 0 0 0
29 A 9.8.5. Student writers have first
draft reviewed by peers of their
choice. (C1R318, 325) (P3N)
0 0 1 1 2 0 0 0 0
30 A 9.8.6. Ask students to bring
copies of 1st draft for each
member of peer group, if
possible or practical. (C1R628-
629) (P3N) (D1R)
0 0 1 1 2 0 0 0 0
31 A 9.8.7. Instruct students seeking
peer support to bring questions
that address issues within their
own writing (paragraph or
paper). (C1R630-631) (P3N)
0 0 1 1 2 0 0 0 0
32 A 9.8.8. Instruct peer editors to
read paper. Editors may give
revisions as they read or
annotate and finish notes on
paper itself. (C1R632-633)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 241
(P3N)
33 A 9.8.8.1. As an alternative step,
have one member of the peer
editing group read paper while
writer and remaining peer
editors listen. (C2R)(P3N)
0 0 1 1 2 0 0 1 0
34 A 9.8.9. Instruct student authors to
sit and listen to peer editors
without comment. The writer
does not speak at all except
through their writing. (C1R641-
643) (P3N)
0 0 1 1 2 0 0 0 0
35 A 9.8.10. Give students a copy of
the rubric and have the students
assess their colleagues’ papers.
(B1R2085-2086) The
International Baccalaureate and
Common Core rubrics are less
vague than the Advanced
Placement rubrics. (B1R2094)
Consider using the Common
Core Informative Rubric to
assess expository pieces in a
literature class. (B1R2131)
(P2N)
0 1 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 242
36 A 9.9. As the teacher, use Peer
Review time to provide
feedback to individual students
if they seek it or if you feel
individual students need it
based on past writing
assignments. (A1R480-481,
B1R1416, C1R667-670) (P1)
(P2) (P3) You may also join
peer editing groups as needed.
(C1R316-317) (P3N)
0 0 1 1 2 0 0 0 0
37 A 9.10. Have students begin to
write 2nd drafts based on
review, comments, and
suggestions from peers and
from you as the teacher.
(D1R)(4N)
0 0 0 1 1 0 0 0 0
38 A 9.10.1. Allow students to
conference with you during
writing of 2nd draft with
questions about papers or ideas
(C1R677)(P3N)
0 0 1 1 2 0 0 0 0
39 A 9.10.2. Work with students
during individual conferences to
address the following: (A1R,
B1R, C1R, D1R)
1 1 1 1 4 0 0 0 0
40 A 9.10.2.1. Provide guidance in
possible directions to improve
argumentation. (A1R486) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 243
41 A 9.10.2.2. Provide feedback in
suggesting improvements to
evidence. (A1R486) (P1)
1 0 0 1 2 0 0 0 0
42 A 9.10.2.3. Provide feedback in
improving transition between
sentences and paragraphs-
connecting ideas that blend one
thought into another (see step
9.12). (A1R882)(P1)
(B1R535)(P2)
1 1 0 1 3 0 0 0 0
43 A 9.10.2.4. Provide customized
feedback in suggesting
improvements to analysis.
(A1R487) (P1)
1 0 0 1 2 0 0 0 0
44 A 9.10.2.5. Provide feedback in
helping students clarify what
he/she is trying to say.
(A1R487) (P1)
1 0 0 1 2 0 0 0 0
45 A 9.10.2.6. Provide customized
feedback in developing a more
sophisticated inclusion of
quotations into students’ texts.
(A1R884)(P1)
1 0 0 1 2 0 0 0 0
46 A 9.10.2.7. Provide customized
feedback in helping students
take fewer words and
incorporate them into own
sentences rather than dropping a
quote into drafts out of context.
(A1R887) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 244
47 A 9.10.2.8. Writing summary
rather than analysis.
(C1R1190)(P3N)
0 0 1 1 2 0 0 0 0
48 A 9.11. Consider reviewing drafts
as a whole class by assigning
students to turn in drafts in their
current form (1st or 2nd
revision). (C1R) (P3)
0 0 1 1 2 0 0 0 0
49 A 9.11.1. Select 6 random papers
and share with whole class.
(C1R1177-1178) (P3N)
0 0 1 1 2 0 0 0 0
50 A 9.11.2. Discuss areas of concern
that exhibit themselves during
drafting and revision.
(C1R1190-1191) (P3N)
0 0 1 1 2 0 0 0 0
51 D 9.12. IF your students are
writing an argumentative paper,
encourage students to revise
their own drafts by reminding
them of the modes of writing
they have learned so far. (31R)
(P3N)
0 0 1 1 2 0 0 0 0
52 A 9.13. Hold second round of peer
revision to review second draft
following any peer review
model noted above. (C1R241,
325) (P3N)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 245
53 A 9.14. At the conclusion of
writers workshop, give students
3 additional days to type and
polish papers before final due
date. (A1R547-549, B1R582)
(P1) (P2)
1 1 0 1 3 0 0 0 0
54 A 9.15. Remind students paper
must be typed in MLA format.
Review MLA format issues
with citations. Provide help to
students who need extra
support. (A1R664, B1R1972)
(P1)(P2)
1 1 0 1 3 0 0 0 0
55 D 9.16. IF possible, have students
submit papers to an online
grammar check program such as
TurnItIn.com. (B1R594)(P2N)
0 1 0 1 2 0 0 0 0
56 A 9.17. Analyze completed papers
with an online program to detect
plagiarism. (D1R)(4N)
0 0 0 1 1 0 0 0 0
Procedure 10. Assess papers. (A1R, B1R, C1R) 11 2
1 A 10.1. Collect student papers for
teacher evaluation. (A1R708,
B1R, C1R, D1R)(P1, P2, P3)
1 1 1 1 4 0 0 0 0
2 A 10.2. Put off grading for a few
days. (A1R709) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 246
3 A 10.3. Remind yourself what
areas of needed improvement
and content based on instruction
given will be evaluated during
grading using the rubric
provided or the rubric created
with class in step 3.8. Provide
very detailed and excessive
feedback. (A1R714)(P1)
(B1R583) (P2N)
1 1 0 1 3 0 0 0 0
4 A 10.4. Consider giving audio
feedback to student writing at
the if possible within your
school’s assessment software.
(B1R2270) (P2N)
0 1 0 1 2 0 0 0 0
5 A 10.5. Base evaluation on
whether student writers
supported their thesis, their
arguments make sense, and are
justifiable. (A1R718-719, B1R,
C1R) (P1, P2, P3)
1 1 1 1 4 0 0 0 0
6 A 10.5.1. Consider the
personalities and approaches of
students individually and as a
class group. (A1R716) Ask
yourself, “How did this
particular class approach the
text being analyzed?” (P1)
1 0 0 1 2 0 0 0 0
7 A 10.6. Consider the differences
between classes in a particular
1 0 0 0 1 0 0 0 0
COGNITIVE TASK ANALYSIS 247
year. (A1R714) (P1)
8 A 10.7. Base evaluation on the age
group of students. (A1R715)
(P1)
1 0 0 0 1 0 0 0 0
9 A 10.8. Grade student work in a
holistic way. (A1R743) Be
prepared to accept arguments
that might be wrong as long as
they have support and are
justifiable. (A1R741-742, B1R,
C1R) (P1, P2, P3)
1 1 1 1 4 0 0 0 0
10 A 10.9. Consider the distance each
writer has grown and moved
towards an acceptable level of
proficiency. (A1R728-729,
A1R750, D1R) (P1)
1 0 0 1 2 0 0 0 0
11 A 10.10. Assess grammar.
(A1R785)(P1)
1 0 0 1 2 0 0 0 0
12 D 10.10.1. IF the grammar and
punctuation do not affect what
the student wanted to say,
THEN it is not reflected in the
student’s grade. (A1R785,
B1R)(P1, P2)
1 1 0 1 3 0 0 0 0
13 D 10.10.2. IF grammar and
punctuation makes a paper
unclear, THEN it will be
reflected in the student’s grade.
(A1R785-788) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 248
Procedure 11. Hand back papers and provide optional opportunities
for further revision. (A1R, B1R, C1R, 41R)
12 2
1 A 11.1. Return back graded work
to students. (A1R813,
B1R1478, C1R, D1R)(P1) (P2)
(P3) (Pr)
1 1 1 1 4 0 0 0 0
2 D 11.1.1. IF students wrote an
acceptable paper but are
unhappy with their grades,
THEN they have the
opportunity to revise their
paper. They must meet with you
before they begin to revise.
(A1R814-816, B1R, C1R) (P1,
P2, P3)
1 1 1 1 4 0 0 0 0
3 D 11.1.2. IF students wrote a
paper that does not meet
proficiency based on the rubric,
THEN mark the grade as ‘R’,
meaning a revision is required.
(A1R818) An ‘R’ means the
paper has no point value. This
will remain the case until a
revision is turned in. (A1R819-
825) (P1)
1 0 0 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 249
4 A 11.2. Consider allowing for
multiple revisions as needed
and as time requires. This is up
to students and your schedule as
a teacher. (A1R852)
Experience, however, shows
that students do not revise more
than one time. (A1R849) (P1)
1 0 0 0 1 0 0 0 0
5 A 11.3. Offer to students who
want to revise their graded
paper the opportunity to meet
with you to provide additional
targeted practice to improve
writing. (A1R, B1R, C1R288,
C1R301-302) (P1, P2, P3)
1 1 1 1 4 0 0 0 0
6 A 11.4. During these conferences,
look for the following common
problem areas. (P1, P2, P3)
1 1 1 1 4 0 0 0 0
7 A 11.4.1. Look for problems in
student writing such as
problems with the
argumentation and support:
(A1R859, B1R)(P1, P2)
1 1 0 1 3 0 0 0 0
8 A 11.4.2. Look for problems in
student writing such as papers
having too few quotations or
citations. (A1R859, B1R,
C1R)(P1, P2, P3)
1 1 1 1 4 0 0 0 0
COGNITIVE TASK ANALYSIS 250
9 A 11.4.3. Look for problems in
student writing such as writers
not doing a good job connecting
quotations to the argument.
(A1R861) (P1)
1 0 0 1 2 0 0 0 0
10 A 11.4.4. Ask students to be ready
to discuss where they struggled
with their writing.
(C1R254)(3PN)
0 0 1 1 2 0 0 0 0
11 A 11.5. IF time allows, consider
returning papers to students for
a reflective revision. (C1R254-
256) (3PN)
0 0 1 1 2 0 0 0 0
12 A 11.5.1. Ask students to reflect
on the top 5 things they need to
work on as targeted practice to
improve paper. (C1R263) (3PN)
0 0 1 1 2 0 0 0 0
13 A 11.5.2. Assign students to
submit multiple versions of an
improved writing area of
concern. (C1R265-266) (3PN)
0 0 1 1 2 0 0 0 0
14 A 11.6. Tell students to resubmit
reflective revisions and revised
paper for a revised final grade.
(A1R, B1R, C1R301, C1R306-
308, D1R) (P1, P2, P3)
1 1 1 1 4 0 0 0 0
Procedure 12. Publish student writing. (3PN,
D1R)
12 0
COGNITIVE TASK ANALYSIS 251
1 A 12.1. Encourage students to
share their writing through
blogging and submissions of
writing pieces to various
publishers. (C1R, D1R)
0 0 1 1 2 0 0 0 0
2 A 12.2. Have students consider the
sense of audience when
publishing writing. (C1R746)
Model for students how and
why writing pieces might be
published. (C1R1338) (3PN)
0 0 1 1 2 0 0 0 0
3 A 12.3. Tell students that this is
where they decide whether to
use all the feedback from peer
review and conferencing or not.
Establish with students that this
is their paper and they have the
final choice of what is included
when they decide to publish.
(C1R1266-1276) (3PN)
0 0 1 1 2 0 0 0 0
4 A 12.4. Assign students to blog
regularly. Have students make
connections between articles
and texts, and between what
they are learning in class and
something experienced outside
the classroom. (C1R374)
0 0 1 1 2 0 0 0 0
5 A 12.4.1. Encourage students to
write a blog post and share post
on Twitter. (C1R757-759)
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 252
(3PN)
6 A 12.5. Instruct students on
tagging blogs to help writing to
be found on Internet. (C1R767)
(3PN) Internet tags help search
engines find Blogs or other
writings more easily based on
search parameters.
0 0 1 1 2 0 0 0 0
7 A 12.5.1. Instruct students to find
main ideas and key points from
their text and turn them into
tags. (C1R770) (3PN)
0 0 1 1 2 0 0 0 0
8 A 12.6. Encourage students to
participate in Network Reading.
Network Reading is where
writers find an audience or
cohorts of people on line (i.e.,
Twitter, WordPress) who might
be interested in their writing
topics. Fellow writers read,
respond, and share one
another’s articles on their blogs.
(C1R771)(3PN)
0 0 1 1 2 0 0 0 0
9 A 12.7. Encourage students to
read and comment on the blogs
of other writers. Encourage
them to join into the
conversation of writing pieces
and share with each other
0 0 1 1 2 0 0 0 0
COGNITIVE TASK ANALYSIS 253
through individual blogs.
(C1R774-778) (3PN)
10 A 12.7.1. Monitor this Internet
activity through your
participation in Blogging and
making connections with other
authors including your students.
(C1R, D1R) (P3N)
0 0 1 1 2 0 0 0 0
11 A 12.8. Encourage students to
publish by offering extra credit
for getting any writing piece
published anywhere outside the
classroom. This can include
Blogs, newspapers, or
magazines (C1R1292)(3PN)
(D1R)
0 0 1 1 2 0 0 0 0
12 A 12.9. Require students to
establish a twitter account and
to set up a personal blog.
(C1R1293) (3PN)
0 0 1 1 2 0 0 0 0
333 Total Action and Decision Steps 128 150 147 330 28
1
52 0 6 2 0
281 Action Steps 100 132 123 278 0 6 2 0
52 Decision Steps 28 18 24 52 0 0 0 0
Total Action and Decision Steps 38.44% 45.05% 44.14% 99.10
%
COGNITIVE TASK ANALYSIS 254
Action Steps 35.59% 46.98% 43.77% 98.93
%
Decision Steps 53.85% 34.62% 46.15% 100.00
%
Action and Decision Steps
Omitted
205 183 186 3 14
4
77
Action Steps Omitted 181 149 158 3 12
3
75
Decision Steps Omitted 24 34 28 0 22 30
Action and Decision Steps
Omitted
61.56% 54.95% 55.86% 0.90%
Action Steps Omitted 64.41% 53.02% 56.23% 1.07%
Decision Steps Omitted 46.15% 65.38% 53.85% 0.00%
Total Mean/Average Capture
d
Omitte
d
Total Action and Decision Steps 56.68% 43.32%
Action Steps 56.32% 43.68%
Decision Steps 58.65% 41.35%
Complete Alignment (Total
Action and Decision Steps)
31 9.31%
High Alignment (Total Action
and Decision Steps)
38 11.41%
Partial Alignment (Total Action
and Decision Steps)
253 75.98%
Slight Alignment (Total Action
and Decision Steps)
11 3.30%
COGNITIVE TASK ANALYSIS 255
333
Complete Alignment (Action
Steps)
26 83.87%
Complete Alignment (Decision
Steps)
5 16.13%
High Alignment (Action Steps) 30 78.95%
High Alignment (Decision
Steps)
8 21.05%
Partial Alignment (Action
Steps)
214 84.58%
Partial Alignment (Decision
Steps)
39 15.42%
Slight Alignment (Action Steps) 11 100.00
%
Slight Alignment (Decision
Steps)
0 0.00%
Abstract (if available)
Abstract
Cognitive Task Analysis (CTA) is a collection of methods used to elicit the cognitive processes, unobserved knowledge, and goal structures that make up human behavior. This study sought to apply CTA methods to elicit the knowledge and skills expert English teachers use as they teach expository writing to eleventh grade students. Three semi‐structured CTA interviews were held to capture the procedural and declarative knowledge represented as action and decision steps. The results were coded, analyzed, and aggregated into a gold standard protocol (GSP) that was then given to a fourth expert for verification. This study also looked to identify and quantify the percentage and number of knowledge and skill omissions as experts recall how they deliver expository writing instruction. The omission data was recorded in a spreadsheet and a frequency count was used to determine the amount of omitted knowledge and skills by each expert. These results confirmed prior research that suggests experts may omit up to 70% of critical information and can be reversed by utilizing 3-4 experts in eliciting expert knowledge and skills. Finally, this study and a concurrent study (Lim, 2015) compared the efficiency of two varying methods of CTA, the 3i+3r individual method and the 1i+3r incremental method (Lim, 2015). These studies operationalized efficiency by determining which method captures as much, or more, action and decision steps from experts for less cost and time. The comparison results produced abundant data, but did not provide a clear answer as to which method is more efficient. The knowledge and skills captured by CTA may be used to inform and develop pre‐service and in‐service professional development training for teachers in performing the task of expository writing instruction.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Using incremental cognitive task analysis to capture expert instruction in expository writing for secondary students
PDF
Using cognitive task analysis to capture expert reading instruction in informational text for students with mild to moderate learning disabilities
PDF
Using cognitive task analysis to capture expert instruction in division of fractions
PDF
Towards a taxonomy of cognitive task analysis methods: a search for cognition and task analysis interactions
PDF
The use of cognitive task analysis to capture expert instruction in teaching mathematics
PDF
The effect of cognitive task analysis based instruction on surgical skills expertise and performance
PDF
Using cognitive task analysis for capturing expert instruction of food safety training for novice employees
PDF
Using cognitive task analysis to determine the percentage of critical information that experts omit when describing a surgical procedure
PDF
The use of cognitive task analysis to capture exterptise for tracheal extubation training in anesthesiology
PDF
Using cognitive task analysis to capture how expert principals conduct informal classroom walk-throughs and provide feedback to teachers
PDF
Cognitive task analysis for instruction in single-injection ultrasound-guided regional anesthesia
PDF
The use of cognitive task analysis for the postanesthesia patient care handoff in the intensive care unit
PDF
The use of cognitive task analysis for identifying the critical information omitted when experts describe surgical procedures
PDF
Employing cognitive task analysis supported instruction to increase medical student and surgical resident performance and self-efficacy
PDF
The use of cognitive task analysis to capture expert patient care handoff to the post anesthesia care unit
PDF
Using cognitive task analysis to capture palliative care physicians' expertise in in-patient shared decision making
PDF
The use of cognitive task analysis to determine surgical expert's awareness of critical decisions required for a surgical procedure
PDF
Using cognitive task analysis to capture how expert anesthesia providers conduct an intraoperative patient care handoff
PDF
Identifying the point of diminishing marginal utility for cognitive task analysis surgical subject matter expert interviews
PDF
The use of cognitive task analysis to investigate how many experts must be interviewed to acquire the critical information needed to perform a central venous catheter placement
Asset Metadata
Creator
Jury, Milo A.
(author)
Core Title
Using individual cognitive task analysis to capture expert writing instruction in expository writing for secondary students
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Education (Leadership)
Publication Date
03/09/2015
Defense Date
02/19/2015
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
cognitive task analysis,CTA,expertise,expository writing,High School,high school writing,knowledge elicitation,OAI-PMH Harvest,secondary students,teaching strategies,writing instruction,writing pedagogy
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Yates, Kenneth A. (
committee chair
), Carbone, Paula M. (
committee member
), Maddox, Anthony B. (
committee member
)
Creator Email
milojury@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c3-539594
Unique identifier
UC11298657
Identifier
etd-JuryMiloA-3223.pdf (filename),usctheses-c3-539594 (legacy record id)
Legacy Identifier
etd-JuryMiloA-3223.pdf
Dmrecord
539594
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Jury, Milo A.
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
cognitive task analysis
CTA
expertise
expository writing
high school writing
knowledge elicitation
secondary students
teaching strategies
writing instruction
writing pedagogy