University of Southern California Dissertations and Theses

Districtwide instructional improvement: A case study of a high school in the Los Coyotes High School District

(USC Thesis Other)

Districtwide instructional improvement: A case study of a high school in the Los Coyotes High School District

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content DISTRICTWIDE INSTRUCTIONAL IMPROVEMENT: A CASE STUDY OF A
HIGH SCHOOL IN THE LOS COYOTES HIGH SCHOOL DISTRICT
by
Sylvia Martinez Kaufman
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2005
Copyright 2005 Sylvia Martinez Kaufman
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
UMI Number: 3180469
Copyright 2005 by
Kaufman, Sylvia Martinez
All rights reserved.
INFORMATION TO USERS
The quality of this reproduction is dependent upon the quality of the copy
submitted. Broken or indistinct print, colored or poor quality illustrations and
photographs, print bleed-through, substandard margins, and improper
alignment can adversely affect reproduction.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
®
UMI
UMI Microform 3180469
Copyright 2005 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
DEDICATION
This dissertation is dedicated to my husband, Russell David Kaufman. Your
patience, words of encouragement, and sacrifices have been greatly appreciated and
acknowledged. Words can never express how much your support has meant to me
during this doctorate journey. Cheers to our marriage and family!
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
iii
ACKNOWLEDGMENTS
I would like to acknowledge and thank my family, friends, and colleagues
who have been a part of my life during these past few years (3/5/02 - 5/13/05) as I
worked on obtaining my Educational Doctorate (Ed.D.). Your words of
encouragement and support helped me to move forward when the thought of
finishing seemed impossible. I am truly fortunate to have all of you in my life.
This dissertation is not only dedicated to my husband but I need to
acknowledge him as well. Since I began this doctorate program many things have
occurred that has brought us closer together and changed the way we view the
precious things life has to offer. Life has so much more in store for us and I look
forward to living the experiences with you. You have been like a coach that has
guided me from the sidelines during this dissertation period. You encouraged me
with words of wisdom when I felt lost, you recommended that I take breaks from
working when you saw I was too tired to continue, you lectured me when I wanted
to quit (I really did not care for that day), and you waited to cheer me on as I
reached and crossed the finish line. Once again, thank you for all the different roles
you took to help me achieve my goal of earning a doctorate degree.
To my son, Troy Russell Martinez Kaufman, who made me smile everyday
as he learned to hold his bottle, roll over, sit, crawl, stand, and take his first steps.
He inspired me to finish this dissertation so I could spend more time with him and
his future siblings.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
iv
To my mom, Maria V. Martinez, and dad, Juan C. Martinez, who have
always been thrilled and proud of any and all educational opportunities I have taken
on. This makes all three of your children graduates from USC. Mom and Pop,
thank you for your “Support.” Also, I am grateful to my mother-in-law, Helen
Jeanne Kaufman, who always treated me as her daughter from the first day we met
and was so proud of me when I started the program. Unfortunately, she did not
have an opportunity to share in the celebration of the completion of this program,
but 1 know she is smiling down upon me now.
To my brother, Juan “Jay” Martinez, who started the USC tradition and
made a dramatic improvement in his life by choosing education as a foundation. To
my sister, Lorena Martinez, who is also a USC alumni and has put constant
pressure on me by sharing with her friends that I was done with my doctorate when
1 had just started the program. Thanks for the pressure, sister!
To my two cousins who are like bothers to me, Daniel and David Mendoza.
I hope to have more time to spend with you and your families now that I am done.
Daniel you are an inspiration to all single parents — your dedication to your four
children is heart warming. I am proud of you.
To my friends I have set aside the past three years. Get ready, 1 am back and
I have so much catching up to do. Please get me back into the social loop and
update me on your lives.
I want to thank the Fullerton Joint Union High School Board of Trustees
(Marilyn Buchi, Bill Dunton, Robert Hathaway, Barbara Kilponen, and Dr. Robert
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
V
Singer), Dr. Michael Escalante, and Dr. George Giokaris for establishing a
partnership with the University of Southern California, School of Education which
provided an opportunity to earn an Educational Doctorate from a prestigious
university. Thank you for the use of the Education Center for classes every
Thursday evening which made it extremely convenient for full-time working
individuals as myself. Dr. Giokaris, thank you for handling all of the tedious
administration task requirements, working as our advocate, and serving as our
communication link between the university and our cohort group.
Dissertation committee members Dr. David Marsh, Dr. Richard Brown, and
Dr. Gregory Bowman thank you for your guidance and support as you challenged
me to reach a level of knowledge I was unaware I was capable of achieving. Your
thought provoking questions and synthesis expectations provided me with
enhanced analytical skills I will continue to use in the future.
Carlye Marousek Olsen, you were the best thing that happened to me in the
doctorate program. I really want to thank you for the memorable Thursday
evenings in class and all the hours spent on e-mails monitoring our work progress.
Communicating to you by e-mail every night became habit-forming as we vented
our frustrations and cheered on our accomplishments. You are a great friend,
motivator, and inspiration to me. Thank you!
To my twenty-one cohort colleagues who met weekly in our quest to obtain
a doctorate. I enjoyed your company and each of your unique personalities that
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
VI
contributed to the memorable moments in our classes. I will miss all of you very
much and our dinners, too.
I want to thank the faculty and staff at Fullerton Union High School who
helped proof my class assignments from various courses over the past three years:
Kara Bluntach, Linda Kay Gluhak, Anita Leech, and Connee Valle.
I also want to thank the following individuals who helped proof my
dissertation: Marrily Denton who was always very thorough and honest in proofing
my course papers and entire dissertation; Cathy Gach who has been a mentor and
friend; Dr. Patricia Howell, another mentor, provided me with time to work on my
dissertation, and who always had motivating words of encouragement to keep me
going; and Suzanne Fenton, Christina Zubko, Helen Rodriguez, Frank Tocco who
helped proof sections of the dissertation.
I especially want to thank all my babysitters who watched Troy while I tried
to finish this doctorate program: David Kaufman and Cheryle Dutcher, Juan
Martinez and Maria Martinez, Lorena Martinez and Miguel Juarez, Elizabeth
Rodriguez, and Connee Valle.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
v ii
TABLE OF CONTENTS
Dedication............................................................................................................................ ii
Acknowledgements........................................................................................................... iii
List of Tables......................................................................................................................xi
List of Figures....................................................................................................................xii
Abstract............................................................................................................................. xiii
CHAPTER ONE: INTRODUCTION............................................................................... 1
Statement of the Problem.....................................................................................15
Purpose of the Study.............................................................................................17
Importance of the Study ......................................................................................18
Limitations.............................................................................................................20
Delimitations........................................................................................................ 20
Definitions of Terms.............................................................................................21
Organization of the Study................................................................................... 25
CHAPTER TWO: REVIEW OF THE LITERATURE................................................. 27
The Status of Student Performance.................................................................... 27
Historical Trends and Patterns............................................................... 27
Factors that Affect Student Performance...........................................................32
Demographic Factors..............................................................................32
Educational Factors................................................................................. 33
Instruction Improves Learning .............................................................. 36
Efforts to Improve Instruction.............................................................................37
“Good Instruction” .................................................................................. 37
“Good Instruction” Changes Over Time .............................................. 38
Varied Teaching Strategies Evolved......................................................41
Standards-Based Reform and State Assessments............................................. 44
Historical Background.............................................................................44
Key Elements of Standards-Based Reform........................................... 47
Standards-Based Reform, Shifted Views, and Context........................ 52
Role of School Districts........................................................................................56
Why Do Districts Matter Now?.............................................................. 57
Key Elements in District Design............................................................ 57
Factors that Influence District D esign...................................................59
Strategies Used for Change ................................................................... 59
District/School Connection ................................................................... 61
Reform Impact..........................................................................................62
Exemplary Districts..............................................................................................64
Edmonton School District.......................................................................64
New York Community School District #2............................................ 66
San Diego City School District.............................................................. 69
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
v iii
Summary................................................................................................................72
CHAPTER THREE: RESEARCH METHODOLOGY............................................... 74
Sample................................................................................................................... 76
Selected District Profile...........................................................................81
Selected School........................................................................................84
School Participants ................................................................................. 85
Instrumentation..................................................................................................... 87
Frameworks for Instrument Design........................................................93
Data Collection Instruments................................................................... 95
Data Collection................................................................................................... 100
Data Analysis.......................................................................................................106
Summary..............................................................................................................110
CHAPTER FOUR: FINDINGS, ANALYSIS, AND DISCUSSION........................ 111
Introduction.........................................................................................................I l l
Research Question #1: What was the district design for improving
teaching and learning?........................................................................................112
District Background............................................................................... 112
Creation of the District Design............................................................. 116
Components (Strategies) of the District Design..................................116
Communication of the District D esign................................................117
Training Provided at the D istrict......................................................... 119
Teachers’ Perceptions of the District Design ..................................... 120
Assessment of the District D esign.......................................................121
School Level Survey Items 1-8............................................................ 122
Research Question #2: What school level efforts facilitated the
implementation of the district design?............................................................. 124
R oles.......................................................................................................125
Monitoring District Design....................................................................127
Implementation Timeline......................................................................128
Teacher Trainings.................................................................................. 129
Support Activities................................................................................. 129
Things that Worked............................................................................... 131
Areas to Improve ...................................................................................133
School Level Survey Items 9-21.......................................................... 135
Research Question #3: To what extent had the district design been
implemented at the school and classroom levels?........................................... 138
School Level Survey Items 22-33........................................................ 140
Innovation Configuration ............................ 141
Teacher Questionnaire (Stages of Concern)........................................ 146
Scoring the Teacher Questionnaire .....................................................146
Individual Raw Score ...........................................................................150
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
ix
Individual Mean Responses...................................................................153
Frequency Compared to Days of Training......................................... 156
Research Question #4: How effective were the district design and
implementation strategies at the school and the classroom levels?................ 159
Teachers’ Response and Attitude toward District D esign................. 160
School Assessment of District Design ................................................162
Was Professional Development Helpful?........................................... 163
Profile of Implementation.....................................................................164
Positive Implementation Strategies .....................................................167
Negative Implementation Strategies.....................................................169
School Level Survey Items 34-44 .......................................................170
Discussion...........................................................................................................173
Los Coyotes District D esign.................................................................173
Russellville’s Efforts to Implement District Design ..........................180
Extent of Implementation of District Design at Russellville.............192
Russellville’s Effectiveness of Implementation.................................200
Summary.............................................................................................................208
CHAPTER FIVE: SUMMARY, CONCLUSIONS, AND IMPLICATIONS.......... 210
Summary of Background................................................................................... 210
Purpose of the Study..........................................................................................212
Methodology....................................................................................................... 212
Sample.................................................................................................... 213
Data Collection and Analysis .............................................................. 214
Summary of Findings ....................................................................................... 216
Research Question One ........................................................................217
Research Question T w o........................................................................219
Research Question Three...................................................................... 221
Research Question Four ...................................................................... 223
Conclusions ....................................................................................................... 225
Best Practices..........................................................................................225
Areas to Improve .................................................................................. 227
Implications and Recommendations................................................................. 230
Board of Trustees and District Administrators....................................230
Site Administrators................................................................................230
Teachers.................................................................................................231
Recommendations for Future Research...........................................................231
REFERENCES................................................................................................................234
APPENDICES.................................................................................................................257
A Case Study Guide ....................................................................................... 259
B Data Collection Schedule............................................................................262
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
X
C Conceptual Framework A........................................................................... 265
D Conceptual Framework B ......................................................................... 266
E Conceptual Framework C ........................................................................... 267
F Conceptual Framework D............................................................ 268
G Principal Interview Guide...........................................................................269
H Leadership Team Member Interview Guide..............................................271
I Lead Teacher Interview Guide...................................................................272
J Teacher Interview Guide............................................................................ 273
K Innovation Configuration ......................................................................... 274
L Innovation Configuration - Teacher ....................................................... 276
M Innovation configuration - K ey.................................................................278
N School Level Survey...................................................................................279
O Teacher Questionnaire (Stage of Concern) ..............................................285
P Document Review G uide........................................................................... 288
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
x i
LIST OF TABLES
1. District and School Selection Criteria............................................................... 77
2. Teacher Questionnaire Internal Reliability Coefficients.....................................89
3. School Level Survey Reliability Coefficient for Field Test Scales....................90
4. The Relationship of Data Collection Instruments to Research Question..........92
5. Timeline for Classroom Instruction that Works................................................ 115
6. Summative School Level Survey Results by Percentage for
Research Question 1...........................................................................................123
7. Summative School Level Survey Results by Percentage for
Research Question 2 ...........................................................................................136
8. Summative School Level Survey Results by Percentage for
Research Question 3 ...........................................................................................139
9. Summary of Innovation Configuration Teacher S elf-Report...........................143
10. Stages of Concern about the Innovation.............................................................147
11. Statements on Stages of Concern Questionnaire Arranged by Stage.............. 148
12. Item Numbers and Associated Stage of Concern..............................................149
13. Teachers’ Highest Level of Concern Expressed on the Stages of Concern
Questionnaire..................................................................................................... 152
14. Distribution of Strongest Teacher Concerns..................................................... 155
15. Summative School Level Survey Results by Percentage for
Research Question 4 ...........................................................................................172
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
x ii
LIST OF FIGURES
1. Graphic Representation of the Percentage of Teacher Responses on
Extent of Implementation on Classroom Instruction that Works................ 145
2. Profile of Each Participant’s Five Item Raw Scale Score............................... 151
3. Frequency of Teachers’ Highest Stage of Concern.......................................... 156
4. Frequency of Peak and Second Highest Stage of Concern for Teachers in
Relation to Days of Training.............................................................................. 157
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
x iii
ABSTRACT
The purpose of this case study is to understand the role of districts in
enhancing good teaching and learning through systemic change, and how it is
linked to districtwide instructional improvement efforts. The scope of this study
focused on one high school in Southern California that implemented a districtwide
reform effort. The four research questions that guided the study are as follows:
1. What was the district design for improving teaching and learning?
2. What school level efforts facilitated the implementation of the district
design?
3. To what extent had the district design been implemented at the school
and classroom levels?
4. How effective were the district design and implementation strategies at
the school and classroom levels?
This cross-analysis method of study consisted of triangulated qualitative
and quantitative data from interviews and surveys which found that the use of a
top-down approach to implement a research-based instructional model was
influenced by the actions of the principal, communication, role clarification, and
the type of professional development provided. The monitoring and assessing of
the district design was limited by the type of data collected and the lack of training
for administrators. Five key best practices were found: district recognition when
change is required; instructional strategies that were feasible for teachers to
implement; effective principals; trainer-of-trainers model; and, empowerment of
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
x iv
teachers. Five areas of improvement were identified to help sustain the change:
effective communication; defined roles; administrator training; meaningful data;
and limited training time. Implications for the board of trustees, district
administrators, site administrators, and teachers implementing instructional reform
included a: focused vision; use of timely and accurate data; training for
administrators; and clarification of roles. Recommendations for future research
included: studying the impact of the instructional improvement in districts that
provided administrator training; understanding subjective attributes of leadership
and its potential impact to influence teaching and learning; and, a case study of a
district that used a bottom-up approach to implement instructional reform.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
1
CHAPTER ONE
INTRODUCTION
In the past fifty years, American education has been repeatedly criticized
and become the main topic of discussion at every level of society. The initial panic
became evident during the 1950s when the launch of Sputnik left Americans
concerned that they were falling behind other countries in science and technology.
People were increasingly critical of teachers and the schools that trained them. At
the end of the 1960’s, policymakers began focusing on the success of all students
instead of the highly-gifted or those identified through special education. In the
1970’s, competency testing results helped local educators and state officials detect
educational deficiencies, and led inquiry for the improvement of student academic
performance. These scores were not required to be publicized; therefore,
comparisons were not possible (Elmore, 1997).
The publication of A Nation at Risk in 1983 drew the attention of
Americans and caused many to question the quality of education provided to U.S.
youth. The statistics presented made even the most common individual in society
question and lose faith in American education (Maroney, 1998). Since this study
was completed in 1983, the Center for Education Reform (1998) estimated that
more than 10 million Americans reached the twelfth grade without achieving basic
reading skills; 20 million were unable to perform basic arithmetic calculations; and
25 million were uneducated in the basic essentials of American history.
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
2
In 1995, an extensive investigation was conducted in the content areas of
math and science (National Center for Education Statistics [NCES], 1999).
Approximately 50 countries participated in the Third International Mathematics
and Science Study (TIMSS) that included three age-group categories. The results
showed that fourth-grade students performed well, eighth-grade students were
average, and twelfth-grade students were below the international average as well as
lowest among the nations that participated in the TIMSS study. The study also
discovered that students in American schools received a less rigorous curriculum.
For example, in mathematics the instruction focused on procedural solutions of a
problem rather than on understanding the mathematical concept (NCES, 1999;
TIMSS International Study Center, 1997).
Four years later a repeat study, TIMMS-R, was conducted that allowed the
comparison of the relative performance of a cohort group of American students.
The eighth graders performed about average in both content areas, just as they had
scored four years earlier as fourth graders. When assessment results were compared
to eighth graders from both 1995 and 1999, there were not any changes in the
eighth grade mathematics or science scores (NCES 2000a, 2001).
According to an examination of the trends in the results of the National
Assessment of Educational Progress (NAEP) over the past 20 years, achievement
gaps in reading and mathematics among ethnicities had decreased, but the
difference had not been found to be significant (NCES, 2000b). Hanushek (2003)
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
3
suggested that differences in the quality between schools affected how students
learn and that creative policies were needed to positively affect student outcomes.
Multiple factors have been related to poor student achievement. The first
area that caused concern was demographic variables that could be addressed by
schools: i.e., socio-economic status (SES), ethnicity, parent education, and English
language proficiency. The second area of concern related to instructional factors
that were influenced by schools and teachers. Darling-Hammond (1999) asserted
that the most significant factor in determining the quality of instruction was the
teacher. Marzano, Pickering, and Pollock (2001) concurred that an individual
teacher could have a powerful effect on students, even if the school did not.
Williams (2003) suggested the same message by stating that an individual teacher
had a significant and crucial influence on the effectiveness of student performance.
Through trial and error, studies have tried to find instructional designs that
improved student learning in the classroom. According to Darling-Hammond
(2003), this nation has not adequately invested in teachers and their capacity to
teach all students. When students were provided with good teaching through
modeling, practice opportunities, and other teaching strategies, it had a direct
positive influence on student performance. Studies have found that quality
instruction improved student performance (Darling-Hammond, 1999; Hirsch,
Koppich, & Knapp, 2001).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
The federal government has played a role in the educational process.
Unfortunately, the politicians who made legislative decisions on education have
had limited educational background knowledge and have failed to use credible
research to make decisions (Kilzer, 2001; Vinovskis, 1996). In an effort to reduce
achievement gaps, centralized control attempts were made to serve the needs of
minority students because a decentralized system did not serve them adequately
(Oakes & Wells, 1996). This centralization resulted in additional federal assistance
for supplemental resources to aid Title I services to reduce the student achievement
gap. However, no models were supplied for spending the funds effectively. Even
though attempts were made, legislators who created educational policies failed to
listen to educators.
Despite additional resources, teachers were not encouraged to provide any
input with site-level decisions. This resulted in a professional development gap, as
one-day lecture workshops failed to address teacher and student needs (Joyce &
Showers, 2002). Shanker (1997) and Joyce (1990) suggested that even though
professional development was widespread, its effectiveness and impact upon
classroom instruction was disappointing to both teachers and districts. Since
educators were not seen as partners in the curriculum effort, sustainability of
reforms was not established because teachers did not take ownership or buy-in to
the reform efforts. Not only was there a lack of teacher input but attempts to
incorporate the instructional strategies or designs were not followed-up with the
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
5
necessary training. Butler (1992) suggested that, to make a definite difference in
student achievement, the quality of actual learning experiences of teachers needed
to be changed.
Historically, there have been many models of instructional strategies that
attempted to improve classroom instruction: i.e., Mager’s incorporation of
instructional objectives into a lesson, Madeline Hunter’s contributions of the five-
step lesson plan, and Bloom’s Taxonomy of higher level thinking (Woolfolk,
2001). Many of these reforms focused on input by educators and not on results of
student performance. These strategies demonstrated the variety of improvement
efforts that were attempted, but the reforms did not consistently improve student
performance, overall nor were they sustained over time (Scherer, 1996).
Teachers went through numerous reforms during their careers and often
skeptical of any new instructional strategies or reform efforts that attempted to
improve student performance (Brennen, 2001). Reeves (1998, p.48) stated he had
been told by teachers: “This too shall pass—one more reform effort that won’t be
here a year from now.” Reforms were seen as an attempt to question or undermine
teachers’ instructional practices. This resentment and distrust by teachers to these
reform efforts were exacerbated by federal and state legislation that discouraged
teachers from providing input. Educators were seen as requiring training, rather
than as partners in the curriculum reform effort (Consortium for Policy Research
Education [CPRE], 1993).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
6
In 2001, Congress enacted the No Child Left Behind Act (NCLB) after
states ignored the requirements set forth by the Improving America’s School Act,
the reauthorization of the Elementary and Secondary Education Act (ESEA) that
required states to close the achievement gap between students of varying
demographics. Based on increased accountability, the NCLB legislation placed
pressure on every state to demonstrate that it was making “Adequate Yearly
Progress” (AYP) toward the goal of having all students performing at the proficient
level (Ananda, 2003; Elmore, 2003).
The national goal of student proficiency pressured schools and districts to
increase student performance in all content areas tested on state-mandated
examinations. The coordination and alignment of curriculum became a priority that
led states to establish standards for the main core subjects (English, math, science,
and social science). States went through a lengthy process putting committees
together to write standards and holding public hearings before the state approved
them. This reform movement not only focused on developing content' and
performance standards, but also on aligning assessment and accountability with the
standards (Elmore, 1997; Tucker & Codding, 1997).
Standards-based instruction guaranteed all students the same rigor and
quality of instruction regardless of what school they attend (Reeves, 1998, 2001).
Though students were not necessarily learning the same thing at the same time,
they learned the same material over the course of the year. This focus on standards
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
7
necessitated a shift in curriculum and instruction that otherwise would not have
occurred (Elmore, 1997). Prior to the standards movement, teachers had total
freedom within their classrooms (Tucker & Codding, 1998). Teachers who
previously taught in isolation began to engage in a rigorous examination of their
own teaching practices through collaboration (St. John, 1999). Even though many
teachers continued to exercise academic freedom to teach as they saw fit, they were
now expected to teach a curriculum that was aligned with the standards rather than
teach a favorite lesson that was unrelated to the standards (Elmore, 1997).
Administrators monitored the implementation of standards-based instruction by
classroom observations.
Standards-based reform was facilitated by the influential publication of A
Nation at Risk and the National Education Goals (Marzano & Kendall, 1998).
Standards-based assessments measured how well students learned the material that
teachers were supposed to be teaching. Before the emphasis on standardized
accountability, students were tested to determine if they had learned what the
teachers’ lectured, but the content was not necessarily standardized from teacher to
teacher. The standards-based movement and accountability assisted in bringing this
change (Elmore, 2002).
The standards-based accountability system included incentives,
interventions, rewards, and consequences for results that differed from past
accountability systems (EdSource, 2000). Although federal legislation from NCLB
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
8
mandated annual assessment of student performance, not all state assessments were
aligned with the same standards. Each state varied in how the progress of schools
and districts were measured (Ananda, 2003; Elmore, 2003). States generally
focused on one of three measures: (1) reaching an absolute target; (2) making
annual growth; or, (3) narrowing the achievement gap (Duffy & Goertz, 2001).
Proficiency was not comparable across the states as each state used different
assessments aligned with different standards and different cut scores for each
performance level.
Schools had varied responses to the incentives and sanctions brought about
by accountability systems. Of importance were the particular differences in the
responses of low-performing and high-performing schools. Low-performing
schools used data to focus on specific students or programs to increase test results.
High-performing schools focused on using the data to improve the entire student
population. These schools incentives for the actions of high-performing schools
were praise and rewards; whereas, low-performing schools goal was to get off
probation (Diamond & Spillane, 2002).
The Public Schools Accountability Act (PSAA) of 1999, established a
system in which the state of California held schools accountable for student
performance under the belief that the entire school environment influenced student
success. The PSAA consisted of three components: (1) Academic Performance
Index (API); (2) Immediate Intervention/Underperforming Schools Program
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
9
(II/USP); and, (3) the Governor’s Performance Award (GPA) program which
ultimately was unfunded due to state budget cuts (California Department of
Education, 2002). In a study of PSAA, it was found that the accountability system
focused the attention of schools and districts on improved performance for under
performing students who had been ignored in past reform efforts (American
Institute for Research, 2003).
Standards-based accountability systems were not linear and required
collaborative efforts from the schools and district to meet the national and state
expectations (Elmore, 1997). Expectations to increase student achievement from
both federal and state accountability systems brought pressure on schools and
districts to use effective instructional strategies that could be monitored and
assessed. To ensure that all children learned, the focus was shifted toward the
students. In the past decade, research supported a constructivist ideal that
intelligence could be learned. This complex process of learning suggested that
intelligence constantly changed as the learner interacted with his/her surroundings
(Asmul, Wilhehnsen, & Meistad, 1998; Mayer, 1998; Resnick, 1999). This
breakthrough in student learning opened the door as to what schools could do to
increase student achievement.
To support students’ growth in intelligence, teachers needed to create
lessons with instructional strategies that provided students with meaningful
connections to previously learned knowledge (Mayer, 1998). Since intelligence was
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
1 0
teachable, students had to take some responsibility for their own learning by
making it a habit to recognize when they were not learning and to try different
learning strategies (Resnick, 1999). Teachers needed to be reform-minded and
flexible, as well as willing to create classroom environments with varied student
experiences (Garcia, 1999; Marzano, 2003; Resnick & Hall, 1998). This included
using clear academic standards, having high student expectations, and using visible
rubrics. These changes in the classroom required a shift in focus from teaching to
learning. This reform effort was linked directly to student learning results driven by
research-based instruction and data (Resnick, 1999).
Improvement efforts in the past were geared towards changing the structure
of schools to improve student achievement. However, findings showed that these
changes, i.e., class-size reduction, had little impact on student learning. More recent
improvement efforts required teachers to utilize research-based pedagogy (Reeves,
1998). Many organizations created research-based frameworks and principles that
guided schools in classroom-based reform. Development of these frameworks was
dependent upon which strategies fit the needs of the school, the demographics of
the students, and the buy-in of the teachers. By moving reform to the classroom, it
introduced research-based learning frameworks and focused on student learning.
In the era of improving instruction with the standards-based reform,
attempts were made to limit teacher isolation and move towards on-going
collaboration, as outlined by professional learning communities (DuFour, 1997).
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
11
Teachers, just like students, needed constructivist approaches to help them grow
and learn as professionals to become experts in their field. Their teaching practices
needed to be research-based and supported by professional development to be
sustainable. By teachers reflecting on their performance, reevaluating their lessons,
discussing student outcomes with colleagues, and analyzing students’ work,
teachers became researchers themselves in order to improve student performance.
Professional development needed to be well communicated, serve the needs
of teachers, and be continuous. The teaching pedagogy focus shifted from lectures
and drills to the constructivist strategies that drew on a student’s prior knowledge to
build new knowledge. Butler (1992) stated, “There is virtually no question that
effective staff development programs do change teacher practice” (p. 14). Thus,
effective professional development needed to be on-going, focused, and utilize
collaborative professional development teams. According to DuFour (1997),
comprehensive reform could not positively impact student performance until the
schools exhibited the habits of a professional learning community where teachers
used collaboration as a way to analyze students’ work and monitored performance.
In the past, districts were not perceived as instructional change agents that
played a significant role in instruction (McLaughlin & Talbert, 2002). Many times
district positions were filled by people who were perceived by personnel as
unqualified (Tucker & Codding, 1998). Districts were viewed as a place to handle
administrative paperwork and record keeping. With the standards-based reform
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
12
movement, districts needed to act as change agents to monitor and ensure that
policy implementation reached not only school administrators: but also the
classrooms. A model district should contribute a vision, provide instructional
support through professional development, support principals at their school sites,
and most importantly, build capacity simultaneously throughout the system
(Hightower, Knapp, Marsh, & McLaughlin, 2002; Togneri, 2003).
Districts created a vision with goals, priorities, and performance targets
through systemic planning. One method for ensuring a common districtwide vision
was to hold school site principals accountable for sharing the district vision with
schools, teachers, students, and parents (Elmore & Bumey, 1997b). If a principal
did not respond accordingly, the district replaced the principal with an individual
who shared the same district vision and expectations.
Districts also created a common vision for instructional improvement by
offering training to teachers and used professional development as a management
strategy. District training sessions served as a way to communicate state policies
and district expectations to teachers. Through time, effort, and persistence the
training modified and enhanced the culture of teaching and learning (Hightower,
2002; Stein & D’Amico, 2002).
Districts trained, supported, and provided administrators at the school sites
with the necessary tools and skills to be instructional leaders who could move their
school sites toward improved student performance. Some district training consisted
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
13
of increasing the administrators’ awareness and understanding of the reform
movement, transmitting expectations of administrators as leaders, providing
training in data analysis, and providing guidance in the creation of schoolwide
improvement goals. The district support and guidance of principals ensured that
schools shared a uniform district vision (Hightower, 2002; Massell & Goertz, 2002;
Stein & D’Amico, 2002).
Districts increased the capacity of administrators and teachers through
professional development and empowered them to become effective instructional
leaders. Release time was provided for teachers to participate in professional
development to ensure they received adequate training and were supported. District
training attempted to create a uniform culture of beliefs and expectations which
communicated the district’s vision to the individual school sites (Hightower, 2002;
Massell & Goertz, 2002; Stein & D’Amico, 2002). Overall, the district’s
organizational efforts and culture affected teachers’ and administrators’ attitudes
and commitment (Hightower et al., 2002).
Some districts used broad strategies to provide instructional leadership to
administrators and teachers. District-driven strategies such as: aligning the
curriculum to the standards, developing common assessments, and training in data
analysis built connecting relationships between districts and schools, thereby
increasing their capacity. These same districts, in their efforts to play a key role in
improving student performance, followed common methodologies: i.e., using
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
14
systemic approaches, implementing and sustaining a collaborative school culture,
sharing the same vision with schools, and monitoring student learning to increase
school improvement (Regional Educational Laboratory Network, 2000). Other
strategies used by districts included changing the physical setting of the school by
improving the facilities or entering into contracts with external agencies that
developed and promoted the reform design (Marsh, Kerr, Ikemoto, & Darilek,
2004).
Not all educators believed in the benefit of having strong district oversight
in the areas mentioned above, they believed that strategies would be more
meaningful if they originated at the school site (Burke, 1996). When schools took
this responsibility, they had to develop a clear defined mission and establish
specific goals to remain focused. The schools also needed a well communicated
decision-making process in which all stakeholders at the site knew what was
expected of them and how changes would be communicated, as well as
implemented, to ensure staff commitment (Marsh, 1995).
Centralization at the district and school site autonomy had strengths and
weaknesses, therefore, some districts and schools chose to incorporate a
combination. The best framework depended on the capacity of the district and
school sites to improve student performance (Marsh, 2000; Togneri, 2003).
Case studies of districts that succeeded in reform efforts to change
instructional practices and improve student achievement provided evidence of the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
15
need for districts to take time, use patience, and have commitment (Togneri, 2003).
As the specific cases discovered, districts were faced with many challenges to
overcome when reform efforts were implemented. The process took years to fully
implement, the resources were limited, and the districts needed to develop methods
to disseminate reform information to the schools. The role that districts took was
not just in the areas of curriculum, instruction, and assessment. They also created
incentives to motivate staff, planned and implemented professional development,
increased equity, and most importantly, increased capacity (Fuhrman, 1994).
Edmonton School District, District #2 in New York City, and San Diego
City Schools were examples of successful districts that stepped-up to the challenge
and succeeded (Hightower et al., 2002; Marsh, 1995; Stein & D’Amico, 2002;
Tucker & Codding, 1998). To ensure an effective implementation model, these
districts worked on building capacity through professional development and
professional learning to achieve quality instruction. Exemplary districts took the
necessary time to be effective by assessing their needs and resources, planning
efforts, and using national reforms to improve student performance.
Statement of the Problem
In the past few years, educational reform has warranted district
administrators to reorganize their districts’ structure and purpose. Exemplary
districts developed an instructional vision, assessed school needs, and placed
teachers and teacher learning as its core strategy (Hightower et al., 2002). The
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
16
capacity of a district and school may create limitations, but considering these areas
of weakness ahead of time may reduce instructional implementation barriers
(Massell & Goertz, 2002). Focusing on research-based designs and frameworks
could lead to improved student performance as stated by Marzano et al. (2001),
“We are at the beginning of a new era in education, one in which research will
provide explicit guidance for the classroom teacher” (p. 10). Districts implementing
reform need district models that incorporate the combination of elements and
strategies to achieve student improvement and sustainability.
Developing teacher buy-in and internal commitment requires
communicating to a school staff assessment results with timely and accurate data.
Implementing change in any organization causes the affected members to
experience a personal level of stress and concern. Effective schools require
strategies to be in place and implement the necessary interventions that will address
teachers’ concerns (Marsh & Jordan-Marsh, 1985). Teachers need to be
empowered, take ownership of the design, and be motivated to work through
implementation barriers. Fullan (2001) stated, “Effective leaders make people feel
that even the most difficult problems can be tackled productively” (p.7). Districts
need to know the critical steps that are necessary before, during, and after the
reform to ensure support from within the district, schools, and community.
Implementation of a program requires the commitment of teachers (Marsh,
2002). Instructional design attempts to reach the desired outcomes through
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
17
sustainability of the program. Often staff development workshops have been one-
day attempts to change teachers teaching strategies and failed to obtain
sustainability (American Institute for Research, 1999; CPRE, 1995; McLaughlin &
Marsh, 1978). To increase the use of new strategies by teachers requires on-going
training, support, and follow-up from the school and district (Huberman & Miles,
1984). Districts and schools need to know how to plan and design reform
implementations in phases, throughout several years, while simultaneously
increasing capacity.
Providing on-going monitoring procedures and assessing the degree of
implementation, using various indicators, provides meaningful information to
determine if progress has met the school, teacher, and student needs. Schmoker
(1999) suggested that if something needs to improve, the focus must be on short
term result, and feedback to provide information on how well a school is attaining
its short-term as well as long-term goals. Districts need to be provided with tools
and resources that are available to determine the impact a program or reform has
made on the district, schools, or students.
Purpose of the Study
The purpose of this case study is to understand the role of districts in
enhancing good teaching and learning and how it is linked to districtwide
instructional improvement efforts. It will explore district designs to improve
teaching and learning, along with the manner in which school efforts facilitated the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8
implementation process, followed by an investigation into the extent of the
implementation and the effectiveness of the district design.
The research study questions are:
1. What was the district design for improving teaching and learning?
2. What school level efforts facilitated the implementation of the
district design?
3. To what extent had the district design been implemented at the
school and classroom levels?
4. How effective were the district design and implementation
strategies at the school and classroom levels?
Importance of the Study
The findings of this study will have relevance to professional organizations,
district administrators, site administrators, and teachers.
Professional organizations provide support and guidance to schools and
districts, assisting them in their efforts to improve instruction. The availability of
current research and practices is essential in a time of high-stakes accountability.
The resulting changes require professional organizations to provide their audiences
with examples of research-based procedures and strategies used by districts to
implement and sustain reform.
District office administrators provide guidance to schools that are
implementing instructional improvements. They will be able to use this study as a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
19
model to guide district efforts, and tailor lessons learned to meet their own
individual district and school goals that support quality teaching and learning.
Districts can anticipate and attempt to reduce implementation barriers; and/or use
the strategies utilized by the district that was studied to ensure sustainability.
Site administrators are a critical connection between the district and the
school. As such, it is important that they understand their role in connecting the
district’s vision and mission to reform efforts and strategies. Administrators need to
understand the importance of empowering staff and communicating to teachers
their significant impact and role within the organization.
Classroom teachers are the true implementers of the reform effort. As they
target instructional improvement efforts, it is important that they fully understand
the district-led reform efforts in order to provide equitable instruction to all district
students. This study will help teachers understand the significant role they have in
influencing change and sustaining district reform efforts.
Professional organizations, district office administrators, site administrators,
and teachers will benefit from this study because it provides a detailed description
of one district-led effort to improve instruction. This study will also contribute to
research that clarifies the role of districts in leading instructional reform; and will
provide a strategic design for implementing systemic change to benefit districts,
schools, teachers, and students.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0
Limitations
The data for this study was collected from one secondary 9-12 high school in
Southern California. This limitation restricts making generalizations of the findings
for all schools and districts. The researcher was confined to interviewing leadership
team members and lead teachers recommended by the site principal. The data analysis
and findings may be subject to researcher bias and interpretation.
Delimitations
This is a qualitative and quantitative case study of one high school.
Purposeful sampling was used in selecting the district and school. Generalizability
is not the goal of this qualitative study and any applicability will be how the reader
interprets and applies case similarities to their own situation due to depth of
coverage in this study.
The school was selected based on the following criteria:
1. District served between 10,000 and 60,000 students.
2. District ethnicity and socioeconomic status was diverse.
3. District demonstrated an intentional effort to improve teaching and
learning.
4. School met Academic Performance Index (API) two of the last three
years.
5. Principal at the school site had been there more than one year.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1
Thirteen interviews were conducted and eighty-nine selected certificated
staff was provided an opportunity to volunteer to complete two questionnaires. The
interviewees consisted of one high school principal, two leadership team members,
four lead teachers, and six classroom teachers.
Definition of Terms
For the purpose of this study, the following terms are operationally defined
as specified below:
Academic performance index (API): California’s numerical indicator of
student achievement, used as a basis for a comparative ranking of schools
statewide.
Accountability: A designed effort or system that holds districts, schools
and/or students responsible for student performance. Accountability systems
typically consist of assessments, public reporting of results, and rewards or
sanctions based upon student performance over time (Elmore, 2002).
Assessment: A measurement of a student’s particular skill or knowledge
that may be written, oral, or performance in nature. Standardized assessments that
are designed to measure specific skills and knowledge are administered and scored
in exactly the same way for all students.
Behaviorism: The learning theory that describes learning as a response to an
environmental stimulus. Behaviorism advocates that children learn through a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2
change in behavior, so the use of rewards and punishment result in the
establishment of good behaviors or the extinction of bad behaviors.
Benchmark: An articulated expectation of student performance at specific
grades, ages, or developmental levels.
Capacity: The ability to flexibly respond to external demands in order to
translate high standards into effective instruction and strong student performance
that is comprised of both qualitative and quantitative factors residing within
structures, processes, and relationships (Massell, 1998).
Conceptual framework: A consistent and comprehensive integration of
research literature, theories, and other pertinent information that is the basis for the
analysis of findings within the study.
Constructivism: Learning theory asserting that children construct new
information themselves based on a foundation of preexisting beliefs. Within this
process, their ideas become more complex and they need to verify new information
in a social context. This position advocates that students need to create their own
knowledge, and it can not be transferred to them through listening to lectures or
engaging in rote practice.
Content standards: As the foundation of a standards-based system, content
standards describe what content knowledge and skills students must master
(American Federation of Teachers, 2001).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3
Data-driven decision making: The process of making decisions about
curriculum and instruction based on the analysis of classroom data and
standardized test data. Data-driven decision making uses data regarding function,
quantity and quality of inputs, and how students learn to suggest educational
solutions (Massell, 2000).
Design: A plan that is intended to affect change which may be districtwide,
specific to a certain level or population, or specific to individual schools.
Equity: Educational impartiality that ensures all students receive fair
treatment and have access to the services they need in order to receive a high-
quality education.
Elaboration: The extent of specific directions given for the enactment of a
design to improve instruction (Cohen & Lowenberg-Ball, 2000).
Implementation: The translating of an idea into action in order to
accomplish the specified goal.
Innovation: An effort, strategy, or plan whose goal is to improve instruction
by changing what currently exists.
Instructional improvement: A change in the structures or opportunities that
enables quality teaching that results in improved learning (Gilbert, Hightower,
Husbands, Marsh, McLaughlin, Talbert, & Young, 2002).
Instructional leadership: An influence that guides the activities that impart
knowledge or skills to students.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4
Performance Standards'. The level of performance that students are
expected to demonstrate in relation to the content standards; such as, basic,
proficient, or advanced levels of performance (Hambleton, 1999).
Professional development: Opportunities for staff to develop new
knowledge and skills that will improve their teaching ability. Also termed “staff
development” in some literature.
Reform: A change effort that is undertaken to improve the educational
system.
Sanctions: The consequences imposed for not meeting expected
performance outcomes in some accountability systems.
Stakeholder: Any person with interest in the operation and outcomes of a
specific educational system, including administrators, teachers, parents, students,
and community members.
Standards-based reform: The change to an educational system that utilizes
subject-matter benchmarks to measure student achievement, assessments aligned
with standards to measure student performance, and accountability systems that
provide rewards or sanctions to district, schools, and students based on student
performance. Full implementation of all three components—standards, assessment,
and accountability—is also termed standards-based accountability.
Systemic reform: Change that occurs in all aspects and levels of the
educational process; and impacts all stakeholders with implications for all
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5
components, including policies, curriculum, assessment, professional development,
instruction, and budgeting. The goal of systemic reform is more clarity, coherence,
and economy throughout the system (Schmoker, 2003).
Teaching and learning: Based on the premise that effective instruction
results in strong student performance; improving teaching and learning refers to the
demonstration of improved instruction, even in the absence of precise student
outcome measurements.
Organization of the Study
Chapter One of the study presents the introduction to the study, the
statement of the problem, the purpose of the study, the research questions to be
answered, importance of the study, limitations, delimitations, and the operational
definitions of terms. Chapter Two is a review of relevant literature. It addresses the
status of student performance, factors that affect student performance, efforts to
improve instruction, standards-based reform and state assessments, role of school
districts, and examples of exemplary districts. Chapter Three presents the research
methodology used in the study, including the researcher’s reasons for interest in the
study and relevant background; the sample used in the study including the selection
process, rationale, and instrumentation; the data collection procedures; and methods
used to analyze the data. Chapter Four presents the findings of the study, followed
by analysis and discussion on each of the research questions. Chapter Five presents
the summary of background, purpose of study, methodology, summary of findings,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6
conclusions, implications for practice, as well as recommendations for future
research.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 7
CHAPTER TWO
REVIEW OF THE LITERATURE
The Status of Student Performance
The status of student performance in the United States has been a growing
area of concern for the public over the past few decades. This increasing
educational concern led elected officials and educators to believe educational
reform was needed (Elmore, 1997). Prior reform attempts failed to increase student
performance, close the achievement gap, and end inequality among student
subgroups in the American educational system (SIG, 2004). Hanushek (2003, p. 1)
stated that “America’s failure to address these concerns has led to substantial losses
for individuals and society as a whole.”
The Center for Education Reform (1998) found that since 1983 more than
10 million Americans reached the twelfth grade without reading at basic level and
more than 20 million students reached their senior year unable to do basic
mathematics. The effect of low student academic performance was also felt in the
workforce as students graduated without the necessary skills required by
employers. Murane & Levy (1998) found that nearly half of American students left
high school without these requisite skills.
Historical Trends and Patterns
In 1964, President Johnson declared a “war on poverty” that triggered the
Civil Rights Act request for a nationwide survey on educational opportunities in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 8
American public schools. This study, known as The Coleman Report, concluded
that students’ background characteristics strongly affected their performance and
level of academic success (Coleman et al., 1966; Marzano, 2003). This left
educators to question their roles.
A Nation at Risk. In 1983, the National Commission on Excellence in
Education released a publication, A Nation at Risk, which publicized the declining
trends in student performance in American schools. This report increased the
educational quality concerns, focusing the public’s attention on the crisis of low
student expectations and weak instructional practices (Elmore, 1997; Maroney,
1998). Triggering a nationwide panic about the American educational system,
people were afraid that the United States was losing economic and political status
in the international arena (Amrein & Berlinear, 2002; Dorn, 1998).
The Third International Mathematics and Science Study, hi 1994, the
International Evaluation of Educational Achievement (IEA) conducted its largest
and most ambitious study called The Third International Mathematics and Science
Study, or TIMMS (TIMSS International Study Center, 1997). This study
investigated mathematics and science education for three student populations:
fourth, eighth, and twelfth graders (Elmore, 1997; NCES, 1999). It also analyzed
curriculum documents, surveys from students and teachers, video tapes of class
instruction, student assessment results from over 20 countries; and focused on the
particular student populations and areas of interest (American Educational
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 9
Research Association [AERA], 2004; NCES, 1999). These international
comparisons in mathematics and science performance showed that U.S. fourth
graders did relatively well in science and about average in mathematics. The U.S.
eighth graders did slightly better than average in science and slightly below average
in mathematics (Elmore, 1997; NCES, 2000a). The most disappointing were the
U.S. twelfth grade results. They scored near the bottom, placing nineteenth out of
21 nations in mathematics and sixteenth out of 21 nations in science (Center for
Education Reform, 1998; NCES, 1999, 2000a). This study provided new and
valuable information about the relationship between instructional practice and
student performance. The study sent a message to the American people that science
and mathematics education needed to be more rigorous, and reinforced their fears
about the poor quality of education in American schools (Elmore, 1997).
In 1999, TIMSS conducted a repeat study (TIMMS-R) in which American
eighth graders ranked seventeenth among the 38 countries that participated in the
study (Troen & Boles, 2003; NCES, 2001). People continued to worry that U.S.
students were academically behind their international peers in nations like Japan,
the Netherlands, and France (Hanushek, 2003). TIMSS and TIMMS-R both made
international and national headlines, and motivated state legislators to rapidly enact
statutes that governed schools to reflect new rigorous standards (Dorn, 1998; Klein,
1996; Mid-Continent Research for Education and Learning, 2001).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3 0
National Assessment o f Educational Progress. The National Assessment of
Educational Progress (NAEP) was often referred to as the “Nation’s Report Card”
because it reliably measured the achievement of American students (NCES, 2000b,
2003). NAEP results from 1970 through 1990 indicated that the average scores in
mathematics of thirteen year-olds increased by approximately six percentile points,
whereas, that of seventeen year-olds increased by roughly four percentile points
(Coley, 2003; NCES, 1992). The scores for reading were similar, showing only a
very slight gain over the twenty-year period.
More disturbing findings surfaced when NAEP student performance results
were disaggregated by subgroups. The educational gaps between advantaged and
disadvantaged students were high, handicapping poor children in their pursuit of
higher education, good jobs, and a better life (Center for Education Reform, 1998).
The gaps in school achievement among racial and ethnic groups and between
students from poor and non-poor families were well documented. These gaps were
consistently large and persisted over time (Barton, 2003; NCES 2003).
Evidence supporting the existence of the achievement gap was found by the
Center for Education Reform (1998), which stated that 13% of all African
American students between the ages of 16 and 24 were not in school and had not
earned a diploma in 1996. Another disappointing discovery showed that 17% of
first-generation Hispanics had dropped out of high school (Center for Education
Reform, 1998).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
31
NAEP scores were used to compare the performance of a group of students
from one year to several years later. In 1996, the math skills of fourth and eighth
graders were nearly a year ahead of their scores from 1990, with African American
and Latino students also making significant gains. The same study showed that
eighth graders made steady progress in reading and fourth graders were about the
same level they were in 1990 (NCES, 2003). NAEP results from 1971 through
1990 consistently showed the same four-year grade level gap between majority and
poorer minority students (NCES, 2003).
NAEP eighth grade and high school reading scores of minority ethnic
groups from 1971 through 2003 showed little growth. According to Haycock,
Jerald, and Huang (2001), minority students showed improvement over the decade,
however, the gaps that separated them from white students remained significantly
wide so that twelfth grade minority students were about four years behind white
students. A thirteen year-old white student read at grade level, while an African
American or Latino student would reach that reading level four grade levels later
(Haycock et al., 2001; SIG, 2004). These findings provided additional evidence of
the need to reform American education (Elmore, 1997). Although policymakers
frequently devoted special attention to the education of disadvantaged students,
most states have not been successful in reducing the achievement gap between
white and minority students (Barton, 2001; Hanushek, 1997).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3 2
Factors that Affect Student Performance
Many factors affected student performance. Identifying the most critical
factors was the key to improving student performance. These factors have been
grouped into two categories: (1) student demographics, i.e., socio-economic status,
ethnicity, language proficiency and parent education level; and, (2) school-related
educational and instructional factors, i.e., preschool, class scheduling, class size,
teacher professional development and instructional practices, and teacher quality
(Cohen & Hill, 1998).
Demographic Factors
When examining demographic factors, the socio-economic status (SES) of
students was related to large variations in achievement that were often
systematically linked to poverty. Low-income children have consistently been
shown to be “at risk” for performing below national average on a variety of
indicators related to school success, especially academic indicators (Huston, 1992;
Lee, Brooks-Gunn, & Schnur, 1998; Ramey & Ramey, 1998). In a San Diego
study, tenth grade students, who attended the least affluent school, read at the same
level as fifth grade students in a more affluent school (Bett, Rice, & Zau, 2003).
Comparisons of student performance showed significant differences
between whites and other ethnicities. According to Whitmire (1997), most of the
disparity developed between the womb and a child’s first day of school. For
example, he found that kindergarten children recognize between 4,000 and 12,000
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3 3
vocabulary words. He also found that far more African American children were on
the lower end of vocabulary recognition than their counterparts on the Iowa Test of
Basic Skills. In math classes, whites gain 2.1 years and minorities gain 1.5 years
between their sophomore and senior years in high school (Whitmire, 1997).
Language proficiency limited students’ exposure to higher-level thinking
skills. While a student was acquiring a new language, the majority of the time was
spent developing their basic language skills (Pardon, 1992). Au (2002) suggested
improving Title I students’ skills by improving reading instruction. It was found
that bilingual students were taught fewer and different reading strategies than
English students (Pardon, 1992).
The level of parent education was also related to student performance and
influenced the educational opportunities outside of school (Darling-Hammond,
1999). Poorer students may not have access to books at home or an adult who
models reading habits.
Educational Factors
Preschool. Student educational and instructional factors also influenced
student performance. Preschool attendance has been cited as an influential factor in
student performance. The Abecedearian Project was a carefully controlled scientific
study of the potential benefits of early childhood education for poor children. Since
1971, project director Ramey examined the relationship of quality preschool on
later academic achievement and student outcomes (Ramey, Burchinal, Campbell,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3 4
Miller-Johnson, & Pungello, 2001). The Abecedearian Project asserted that
children who participated in quality preschool had higher academic test scores and
achievement in mathematics and reading from elementary school through
adulthood. These students also completed more years of schooling, and were more
likely to attend a four-year college (Ramey et al., 2001; Ramey, Burchinal,
Campbell, Miller-Johnson, & Pungello, 2002). Ramey and his colleagues also
found that a high-quality preschool could increase a child’s IQ by seven to ten
points (Ramey & Ramey, 1998). Hirsch (1996) found that preschools routinely
taught basic cognitive skills, giving those children an advantage at the start of
school.
Scheduling. Class scheduling was also a factor that affected student
performance. There were over 200 standards and 3,000 benchmarks that needed to
be adequately covered within approximately 9,042 hours of instruction from
kindergarten through twelfth grade (Marzano & Kendall, 1998). At least 15,465
hours would be needed to adequately cover the identified standards and
benchmarks, requiring as long as 22 years of schooling within the current structure,
or a 71% increase in schooling (Marzano & Kendall, 1998). An analysis of the
TIMSS study concurred with these findings, stating that American schools tried to
cover too much content when compared to other countries (Hiebert & Stigler,
1997). Time in class affected student performance, but only to the degree that it
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3 5
was devoted to appropriate instruction (Aronson, Zimmerman, & Carlos, 1996;
Marzano & Kendal, 1998).
Class size. Class size was supported by some and disputed by others to be a
factor that affected student performance. Class sizes appeared to be the reason why
Texas minority children outscored California minority students (Finn, Harman, &
McRobbie, 1998). In San Diego, researchers also found that class size influenced
English language learners in reading achievement in elementary grades; however,
they found no evidence that class size mattered in middle and high school (Bett et
al., 2003). In contrast, in a study of California schools for Pacific Research
Institute, Izumi and Cox (2003) found no conclusive evidence linking class-size
reduction to improved student performance at any grade level.
Teachers. Many researchers have argued that the quality and effectiveness
of teachers outweighed the demographic and other school-related factors mentioned
above (Darling-Hammond, 1999; Rosenshine & Stevens, 1986; Rowan, Correnti, &
Miller, 2002). In a review of teacher quality and student achievement, Darling-
Hammond (1999) found that teacher quality variables appeared to be more strongly
related to student achievement than other variables such as class size, spending, and
salaries. Based on a large-scale study of elementary schools, researchers found that
student performance was positively affected by various characteristics of teachers:
i.e., quality teacher training, content knowledge, use of teaching strategies, and
patterns of context coverage (Rowan et al., 2002). Qualitative and quantitative
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3 6
analyses suggested that increased investments in the quality of teachers were
positively related to improvements in student performance (Darling-Hammond,
1999).
Instruction Improves Learning
Although a San Diego study noted higher achievement in more affluent
schools, the researchers also found that demographics were not the only difference
between schools. Teachers in the more affluent schools had more than double the
years of teaching experience, were twice as likely to hold a master’s degree, and
10% more likely to hold a full credential than their counterpart teachers at lower
SES schools (Bett et al., 2003). When reviewing the relationship between student
achievement and teacher qualifications in different states, Darling-Hammond (1999)
found that the connection between teacher qualifications and student achievement
persisted even when student poverty and lack of other school resources were taken into
account.
Classroom instruction was a key ingredient in improving learning. Teachers
needed to be able to see new methods from a student’s perspective in order to improve
learning (Darling-Hammond & McLaughlin, 1995). An individual teacher could have
a powerful effect on students, even if the school did not (Marzano et ah, 2001). Since
evidence suggested that knowledgeable, skilled, and better-qualified teachers might
make a difference in student learning; instruction needed to be well-organized to
ensure increased student performance (Darling-Hammond, 1999). In 1990, the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3 7
Northwest Regional Laboratory released a list of instructional practices that were
associated with improvements in student performance. It showed that instruction
was a key ingredient in improving learning when it provided a clear and focused
pre-planned curriculum, with matching learning goals and objectives, and organized
lessons that provided teachers and students the opportunity to know the desired
outcomes (DuFour & Eaker, 1992,1998). Research showed that direct instructional
methods that were clear, sequenced, and provided feedback could bring positive results
if used more often in a collaborative manner (Schmoker, 1999; Wang, Haertel, &
Walberg, 1993; Wiggins & McTighe, 1998).
Efforts to Improve Instruction
Throughout the years, educational views on the methods to improve
instruction and their underlying theories of learning shifted because of updated
research, student performance results, and policy reform efforts. With the
implementation of standards-based reform in the late 1990s, the view and context of
instructional improvement shifted once again.
“Good Instruction ”
It was often difficult to pinpoint what was meant by good instruction.
Instructional improvement efforts attempted to improve student performance,
however, were unable to demonstrate sustainable results. In the past, methods
selected to improve the instructional system had little to do with students’ learning,
but focused primarily on teachers’ instructional practices (Kilzer, 2001).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3 8
Policymakers proposed changes that did not directly alter the educational system,
but instead merely modified the parameters, not reaching the core issues (Kilzer,
2001). Good instruction needed to focus on teaching each student a challenging
standards-based curriculum, using a variety of instructional strategies and
metacognitive skills, with multiple opportunities for students to practice (Resnick
& Hall, 2000).
“Good Instruction ” Changes Over Time
Views of good instruction have shifted over time. Before 1900, good
instruction meant knowing what was right from wrong; therefore, the curriculum
was designed to develop morals and values in students through tales (Wilburg,
1995). As criticisms of teaching practices and poor learning became a concern,
scientific methods developed by Frederick Taylor (1911/1998) were used to
improve teaching. Later, Thorndike (1913) suggested that if subject-matter material
was organized and reinforced, it would improve learning. Skinner (1954)
developed programmed learning as a way to instruct and reinforce skills in students
before moving them to the next level of learning. Skinner’s behaviorism was one of
the major learning theories that formed the basis for instructional strategies. It
simplified learning by focusing on objectively observable behaviors, positive and
negative reinforcement, and repetitive actions (Cohen, 2001; Reyhner, 2003). At
the time, good instruction was teacher-directed followed by guided rote practice
with rewards or sanctions. Behaviorism simplified learning, but it did not explain
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3 9
the learning process in the absence of reinforcement. It also failed to consider
mental activities, social interaction, and higher-order learning skills (Cohen, 2001;
Mayer, 1998; Reyhner, 2003).
Mager (1962,1997) added to educational theory and practice by identifying
instructional objectives that explained the task and objectives of a lesson as a way
for teachers and students to know what students would be learning. Research also
showed that intelligence could be acquired if identifying objectives became a habit
within an individual’s mind. In 1999, Resnick stated that “one’s intelligence is the
sum of one’s habits of mind” (p.2). For educators, this meant that metacognition
strategies and students’ self-regulatory skills could be taught and learned (Resnick,
1999; Resnick & Hall, 2000). These findings made it possible to help students
develop learning-oriented goals to produce high-level academic achievement
(Resnick, 1999). This began the era of cognitivism that focused on the learner as an
active participant in the teaching-learning process (Asmul et al., 1998).
The transition from behavioral to cognitive psychology provided educators
with a new way to view learning and knowledge as abstract representations in the
minds of students. Good instruction built on students’ prior knowledge and
increased students’ capacity to learn. Along with the new view of learning came
many different strategies designed to improve instruction. Gagne (1985) stipulated
that there were various types or levels of learning, each of which required different
strategies to facilitate students in the learning process. Hunter (1969) developed a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4 0
five-step lesson plan to assist teachers in organizing and planning instruction
toward an eventual outcome. She later expanded this lesson plan design to seven
elements (Hunter, 1984) that provided an even clearer guide for teachers’
instruction. Mastery learning (Bloom, Hastings, & Madaus, 1971) set forth
strategies to improve student learning by using tutors, small groups, peer tutoring,
and audiovisual materials. This instructional design demonstrated that slow learners
could learn if given the opportunity. In an effort to guide students to higher levels
of intellectual learning, Bloom’s taxonomy identified six levels ranging from
lower- to higher-level thinking that provided teachers with techniques for
increasing student’s critical thinking skills (Bloom, 1989; Brennen, 2001).
Constructivism was the learning theory that related the quality of learning to
the quality of the learning environment (Asmul et al., 1998; Reyhner, 2003).
According to this theory, good instruction emphasized the importance for teachers
to provide an environment that encouraged stimulating and challenging learning
opportunities in critical thinking and analysis. Constructivist strategies built upon
students’ prior knowledge and were tailored around their responses, not the
standardized curriculum (Ertmer & Newby, 1993; Maschke, 1999).
The combination of cognitive and constructivist theories supported school
improvement efforts that allowed all students to reach high standards of
achievement by combining knowledge and effort (Resnick, 1999; Resnick & Hall,
1998). To apply this combination of theories required significant changes in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
41
classroom practice and in the way schools and districts functioned (Resnick & Hall,
1998).
Educational instructional practices and models constantly shifted, leaving
behind the traditional behaviorist models that led to superficial learning through the
use of lectures, memorization of facts, and reinforcement (Costa & Garmston,
2002). According to Joyce & Showers (2002), it was increasingly important for
teachers to know and be able to apply different learning theories in their classrooms
in order to provide good instruction.
Varied Teaching Strategies Evolved
Throughout the years, research-based instructional strategies were
developed as tools to improve the academic performance of all students. No single
strategy worked well in all situations, so a variety of approaches needed to be used
to provide good instruction (Marzano et al., 2001). To increase comprehension for
students, cognitive reading strategies contributed to the development of
instructional programs. Pardon (1992) provided examples of strategies that were
beneficial to students in lower grades, lower ability, and within certain populations
and ethnicities. Pardon’s strategic approaches consisted of teacher-directed
activities and included modeling, practice, reciprocal teaching, cooperative
learning, student and teacher dialogue, summarization, self-questioning, clarifying,
predicting, and question-answer relationships. Research on instructional strategies
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4 2
demonstrated that when knowledge about a particular topic was meaningful and
well-connected, new information was more readily acquired (Marzano, 2003).
Rosenshine (1996) discussed three instructional strategies that helped
students increase their knowledge through a process of developing, enlarging,
expanding, and refining students’ knowledge structures. The teaching of cognitive
strategies required support structures or scaffolding by the teacher to help student’s
bridge the gap between current abilities and the goal (Brophy, 1999; Rosenshine,
1996). Brophy (1999) stated that teachers typically concentrated their lesson plans’
on the content on what they intended to cover and the steps involved in the
activities their students will carry out, without connecting the intended outcome of
the instruction. He also suggested that teachers ask themselves what, how, when,
and why questions prior to teaching a lesson.
In 2002, Cotton contributed a list of five research-based effective
educational strategies that increased student performance. The five attributes
included: (1) careful orientation to lessons; (2) clear and focused instruction; (3)
effective questioning techniques; (4) feedback and reinforcement; and, (5) review
and reteaching as needed. The University of Virginia (2002) developed a chart of
the best research-based instructional strategies which included concept
development, direct instruction, cooperative learning, Suchman’s inquiry model,
problem-based learning, and classroom discussion model.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4 3
To improve student performance, teachers must be well-trained and updated
on the latest teaching strategies and practices. Staff development programs were
systematic attempts to change teachers’ classroom practices, their beliefs and
attitudes, and student learning outcomes (Guskey, 1986). Joyce & Showers (1980)
stated that almost all teachers could acquire new skills by learning different
teaching strategies. Butler (1992) concurred, stating that “there is virtually no
question that effective staff development programs do change teacher practice”
(p. 14). Yet, the existence of effective staff development programs was rare.
California researchers took a critical look at the existing systems of
professional development in the state and found a poor and incoherent state of
affairs (St. John, 1999). While most teachers received some professional
development each year, it tended to be delivered in one-day workshops that often
neglected relevant topics and was not related to the teacher’s classroom activities
(American Institutes for Research, 1999; CPRE, 1995; McLaughlin & Marsh,
1978). The general feeling has been that most staff development programs have not
benefited teachers or students (Corcoran, 1995; Joyce & Showers, 2002;
McLaughlin & Marsh, 1978).
Stigler & Hiebert (1999) concluded, after studying Japanese teachers’
continuous professional development, that teachers and students learn best in
collaborative settings. To improve student performance, educators needed to accept
that professional learning was a step in the right direction. Professional learning
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4 4
communities brought teachers together in collaborative teams to share teaching
practices, provide support, and reduce teacher isolation. The main purpose was to
work collaboratively with a shared vision, mission, values, and goals that
established a commitment to continuous improvement. Teams had access to
relevant information and used results to guide instruction (DuFour & Eaker, 1992,
1998).
Standards-Based Reform and State Assessments
Standards-based reforms (SBR) dramatically changed the roles and
responsibilities of all individuals involved. Several historical events led to the
evolution of the SBR movement. Three elements not only shaped the efforts to
improve student performance, but also became the foundation of SBR: (1) content
standards; (2) assessment; and, (3) accountability.
Historical Background
The publication of A Nation at Risk in 1983 created public demand to
change the industrial-aged model of schooling (Reigeluth, 2004; Wagner, 1993).
The public was concerned about falling behind other countries educationally and
economically (Loveless & Ravitch, 2000). The National Commission on Excellence
recommended reforming the expectations for teaching, teacher education, and
education standards in an attempt to respond to demands for change (LaRue, 1996;
Scherer, 1996).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4 5
Examination, guidance, and accountability played a prominent role in the
history of reform in the United States (Mazzeo, 2001). Examination of students in
education increased during the 1950s when the public became concerned that it was
falling behind other countries in the areas of science and technology, especially
after the launching of Sputnik (Marzano, 2003). The following two decades
focused on providing guidance to the educational system by bringing in a variety of
reform efforts. At the end of the 1960s, policymakers implemented assessments
that focused on improving education for all students. Previously, assessments were
used to identify gifted students, a highly visible item on state and national policy
agendas (Elmore, 1997).
The 1970s brought competency tests into the educational forum as a way to
help local educators and state officials detect educational problems in order to
improve academic performance for all students (Bracey, 1995; Elmore, 1997;
Heubert & Hauser, 1999). According to Cohen (1996), the national focus on “back-
to-basics” and assessment began the era of performance-based accountability for
schools and students. The disappointing performance of students on the TIMMS
and NAEP provided states with sufficient reason to put an end to minimum
competency testing and to begin the high-stakes testing movement. To raise the
nation’s standards of achievement drastically, the focus was on an educational
reform designed to reverse the educational recession and move toward more
rigorous standards and accountability (Supovitz, 2004; Walker, 2000).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4 6
New reforms responded to the demand for change (Elmore, 1997). States
began to implement assessments as a way to monitor whether or not important subject
matter was taught, to increase student and teacher motivation, and as a tool to inform
parents and the community of student performance (Mazzeo, 2001). During this
period, attempts centered on greater flexibility and less regulation for schools and
districts in exchange for tangible student performance results (Elmore, 1997). In the
late 1980s and into the 1990s states began shifting their focus from educational inputs
to student outcomes. State and local boards began implementing policies that
attempted to directly affect student achievement levels, as recommended in the
publication o f A Nation at Risk (Ammar, Bifulco, Duncombe, & Wright, 2000; Hurst,
Tan, Meek, & Sellers, 2003; Klein et al., 1996).
In 1989, President George Bush and 50 governors met in Virginia at an
Education Summit to draft national educational goals (Elmore, 1997). This reform
effort continued as President Clinton’s administration took office in 1992 by
creating the Goals 2000 initiative (Austin, 1996; Sizer, 1992). During the 1994
congressional election, previous advocates of national standards now supported
state standards and local control (Dom, 1998; Elmore, 1997). Further controversy
was added when national history standards were vetoed by the Senate and
English/language arts standards were held up to public ridicule (Elmore, 1997).
Despite this controversy, SBR continued to gain momentum.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4 7
The federal government enacted Improving America’s School Act which
reauthorized the Elementary and Secondary Education Act (ESEA), requiring state
content standards in reading and mathematics, and assessment programs for states that
received funding (Hurst et al., 2003; NCES, 2003). According to Pipho (1999),
standards, assessment, and accountability were among the most visible legacies
created to improve education as suggested by A Nation at Risk. Cohen and
Steinberg (2002, p. 1) stated, “This standards-based reform movement of the
1990’s represented the first significant statehouse-led effort to drive change in the
classroom.”
Key Elements o f Standards-Based Reform
Significant support existed for SBR because of the belief that accountability
systems played a key role in closing the achievement gap that historically existed
between the academic performance of children from white middle-class families
and the academic performance of children of color and children from low-income
homes (Skrla, Scheurich, & Johnson, 2001). With standards, students were not
competing with other students; instead, their performance was measured against
clearly stated standards (Tucker & Codding, 1998). If they did not meet the standards,
students were offered additional support and educational opportunities to help them
meet the standards (Reeves, 2001). The use of data led to improved teaching
strategies, better planning, and enhanced student performance (Reeves, 1998, 2001).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4 8
Standards-based reform consisted of three key elements: (1) content
standards; (2) assessment; and, (3) accountability. The assumptions were that the
standards would influence what teachers taught and what students ultimately
learned (Ananda, 2003). With standards-based reform, the responsibility for student
achievement: was shared among districts, schools, and students (Reeves, 1998).
Content standards. Academic content standards provided a blueprint for
what students were expected to know and be able to do (Hurst et al., 2003; Reeves,
1998). Content standards paved the way for additional support efforts, including
quality curriculum frameworks, coherent instructional materials, assessments tied
to the standards, intensive teacher preparation, and professional development
guided by related standards for teaching (Cross, 1998; Darling-Hammond, 2003;
Elmore, 1997). The alignment of all instructional components within the system
increased the success and sustainability of the standards-based reform efforts
(Cross, 1998; Frey, 2000).
Criticisms of the ambiguity of content standards promoted the development
of performance standards by states and localities (Elmore, 1997). Performance
standards demonstrated how well students needed to perform in order to be
considered proficient in a particular subject area (Hurst et al., 2003). To provide
specific expectations at critical intervals, benchmarks were established (Reeves,
1998). The problem with performance standards was that every state measured
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4 9
proficiency differently. Each state had different assessments, different standards,
and different scores for each performance level (Duffy & Goertz, 2001).
Assessments. In making assessments a high-stake operation, policymakers
borrowed principles from the business-sector models and attached incentives and
sanctions based on test performance (Amerein & Berlinear, 2002). Annual
statewide assessments measured student progress toward attaining the goals defined
by content and performance standards (Hurst et al., 2003). The high stakes attached
to annual student results created an increasing emphasis on external assessment and
accountability (Diamond & Spillane, 2002). Critics of these annual high-stakes
assessments were skeptical of gains noted on the state tests. They attributed
improved scores to the school’s exclusion of poor performing students and
simplified questions on the test (Ananda, 2003; Duffy & Goertz, 2001; Elmore,
2003).
Accountability. Despite placing considerable pressure on schools and
districts to improve outcomes, external performance-based accountability systems
did not mandate specific changes to instructional practice, school organization, or
educational inputs. Accountability policies identified desired outcomes through
standards and performance targets, then provided incentives to focus and motivate
staff efforts. Consequently, this policy design relied on schools and districts to
make the changes necessary for improved performance (Supovitz, 2004).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5 0
State accountability systems had common elements such as assessments,
standards, performance reporting, and consequences for performance. States
measured progress toward meeting the standards by using one of three approaches:
(1) meeting an absolute target; (2) making relative annual growth; or, (3) narrowing
the achievement gap (Duffy & Goertz, 2001). Policymakers believed that schools,
teachers, and students would improve only if they were held accountable (Diamond
& Spillane, 2002).
In California, the Public Schools Accountability Act (PSAA) of 1999
established a system that held schools accountable for demonstrating academic
progress of their students. The PSAA consisted of three components: (1) Academic
Performance Index (API); (2) Immediate Intervention/Underperforming Schools
Program (II/USP); and, (3) the Governor’s Performance Award (GPA) program.
The API was a composite scale that ranged from a low o f200 to a high of 1000.
The score was calculated from individual test scores on annual assessments that
comprised California’s Standardized Test and Reporting (STAR) program
(California Department of Education, 2002). The annual school growth target was
five percent of the distance between the schools current baseline and the expected
target of 800. The schools received two statewide rankings: (1) an absolute measure
based on scores; and, (2) a relative measure in relation to schools with similar
characteristics (American Institutes for Research, 2003; Boese, Burkhardt,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
51
Carstens, Devine, Gaffney, & Just, 2000; Rogosa, 2000; Technical Design Group,
2000).
The API calculations changed over time with the inclusion of different
assessments of varied weights: i.e., the Stanford Achievement Test (SAT-9), the
California Achievement Test version six (CAT-6), the California High School Exit
Exam (CAHSEE), and the California Alternate Performance Assessment (CAPA).
California eventually planned to include graduation and attendance rates in the
calculation (Frey, 2000).
To support schools in realizing continual progress toward meeting the state
target of 800, II/USP provided funds to low-performing schools to create and
implement an action plan for school improvement (PACE, 2003). A reward system
was created in the Governor’s Performance Award (GPA) which was discontinued
due to budget constraints. The impact of PSAA made educators focus on student
achievement and low-performing schools (American Institutes for Research, 2003).
In an effort to improve instruction, accountability was also monitored at the
federal level. When states ignored the federal requirements established by America’s
School Act and achievement gaps continued to increase among student groups,
Congress enacted the No Child Left Behind (NCLB) Act in 2001 (Brown, Hara, &
Shepardson, 1998; Edwards & Frey, 2003; Massell, 1998; Reigeluth, 2004). The
enactment of NCLB mandated that every state design accountability and support
systems that met particular federal requirements, Adequate Yearly Progress,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5 2
(AYP), and ensured that the accountability movement would expand to every state
in the nation (Supovitz, 2004).
NCLB provisions required that an increasing percentage of students
perform at the proficient level or higher on state reading and math assessments each
year, so that by 2014 all students scored at the proficient level or higher (Ananda,
2003; California Department of Education, 2002; Elmore, 2003). This goal applied
to all students in public schools (Edwards & Frey, 2003; Hall, Wiener, & Carey,
2003; Stein, 2003).
The main goals of the federal AYP requirement and California’s
accountability system were the same: (1) to improve the academic performance of
all students; and, (2) to comparably improve the academic achievement of student
subgroups (Edwards & Frey, 2003; Hall et al., 2003). Assessment and
accountability drove many other elements of the educational delivery system
including instructional design, classroom technique, resource allocation,
administrative practice, and central office decision making (Reeves, 1998).
Standards-Based Reform, Shifted Views, and Context
SBR shifted the way instructional improvement was viewed within the
classroom. Academic standards led to curriculum reform. Curriculum, instruction,
and assessment changed the classroom focus in order to provide equal opportunities
for all students and ensure they had the support necessary to meet the standards
(Pipho, 1999; Reeves, 1998).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5 3
Curriculum design changed to emphasize what was learned and focused on
identifying students’ needs. Expectations were clarified and students were informed
about what they should know and be able to demonstrate by the end of a lesson.
Students were provided with multiple resources and all students were taught the
basics as well as higher-level thinking skills (Rutherford, 1999; Tucker & Codding,
1998).
Instruction was leamer-centered and organized for results. A variety of
teaching strategies were employed that were specific to the subject matter.
Instruction focused on enrichment opportunities that required active learning and
provided reteaching when students had difficulty grasping the concept. Also,
instructional strategies differentiated within student groups to meet unique student
needs (Reeves, 2001; Rutherford, 1999; Tucker & Codding, 1998).
Rubrics were used to clearly inform students of what was being assessed
and how the assessments would be scored. Rubrics also clarified what needed to be
learned by the student and defined the quality expectations for their performance
(Reeves, 2001; Tucker & Codding, 1998). Multiple opportunities were given to
each student to accommodate the different speeds at which they gained proficiency.
Performance-based assessments were aligned with standards and integrated with
instruction to provide students and teachers with a coherent system that reinforced
learning (Darling-Hammond, 2003). Although grades were determined by final
performances, the focus was not solely on the product, but the process as well.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5 4
Assessment feedback included diagnosing and prescribing ways to improve
performance (American Federation of Teachers, 2001; Rutherford, 1999).
Good news. The good news was that SBR encouraged educators to try new
ways to deliver instruction in an effort to improve teaching and learning. Research-
based strategies improved teaching and learning; learning communities encouraged
collaboration; and the use of data identified areas of strengths and weaknesses,
provided feedback, and served as a guide for improving instruction (Darling-
Hammond, 2003; Marzano & Kendall, 1998; Reeves, 2001; Tucker & Codding,
1998).
Research-based strategies were utilized to support practice (Marzano et. al.,
2001). In the era of high-stakes accountability every decision was critical, and
evidence was required to support the rationale of the decisions. DuFour & Eaker
(1992) stated that “research findings can provide a frame of reference for thinking
about characteristics consistently associated with effective schooling” (p.24).
Including research in decision making expanded the role of a teacher to that of an
analyst as well as teacher-practitioner (Richardson, 1994; Stigler & Hiebert, 1999;
Wayne, 1997).
Challenges. Critics argued that the implementation of standards-based
instruction and high-stakes monitoring through state assessments narrowed the
focus of instruction and seriously limited the breath and depth of instruction
(Amrein & Berliner, 2002). Teachers complained that there were too many
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5 5
standards to be covered in such a short amount of time. They felt that they were
required to move so quickly through the material that they could not always check
for understanding as they could in the past (Marzano & Kendall, 1998). They
argued that streamlining the standards would allow more in-depth coverage and
would restore many of the critical-thinking discussions that made the curriculum
interesting and relevant to students. Others saw the benefit of evening the playing
field so that all students could at least receive all of the basics, allowing for a more
seamless transition from class to class and teacher to teacher (Reeves, 2001).
Tucker and Codding (1998) stated:
Our society is ceaselessly sorting out those who are well educated and
trained from those who are not. The greatest favor we can do for low-
achieving students is to make implicit standards explicit, so they will know,
often for the first time, what they have to do to succeed, (p.ix)
Many did not believe that tests alone were a true measure of how a student
or a school was performing. They claimed that there was not any consideration
given to individual motivation, or lack of motivation, for students to perform well
on tests that they perceived to be of little value to them personally (Marzano, 2003;
Tucker & Codding, 1998).
Powerful methods of improving school performance included analyzing
student work, student performance levels, and benchmarks. Tucker and Codding
(1998) suggested that it was necessary to evaluate results in order to determine
what worked, what did not work, and what needed to be done next. Effective
school reform efforts were linked to students’ learning results. To achieve the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5 6
desired outcomes, educators questioned their core teaching and learning processes,
methods, and results. They then used the information gathered from the analysis of
student results to drive continuous improvement (Goldberg & Cole, 2002).
Schmoker (1999) provided additional insight into the importance of analyzing data,
“Not focusing on patterns is unfortunate, because the real power of data emerges
when they enable us to see and address patterns of instructional program strengths
or weaknesses, thus multiplying the number of individual students we can help”
(p.43).
Role of School Districts
The role of the school district in improving teaching and learning was one of
research, support, and facilitation. District staff members communicated state policies
to practitioners, along with providing professional development and financial
resources: i.e., staff development time, substitute teacher time, and instructional
materials that supported the reform effort. Districts also provided a focus. By working
with site administrators and teacher-leaders, specific instructional strategies were
identified as focal points upon which all schools concentrated, thus providing a
standard structure for staff development. Tyack (2002, p. 22) stated, “What counts in
school reform, finally, is what happens in classrooms, and this depends heavily on
what happens in the school districts.” Each district existed within its own political
environment by having unique goals, strategies, leadership interests, and capacity to
improve instruction (Marsh et al., 2004).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5 7
Why Do Districts Matter Now?
The role of the district was of vital importance in improving teaching and
learning. The high-stakes testing, accountability, and reporting of results were not only
expected by the public, but mandated by law. These components have created pressure
on districts to perform new roles (Marsh et al., 2004). Districts had to reevaluate their
roles as intermediaries between the state and classroom and sharpen their focus on
teaching and learning. The most significant piece has been the threat of sanctions
against districts and their schools if they failed to demonstrate improvement. This
placed greater demands on central office administrators to provide teachers and
administrators with the skills, knowledge, and resources they needed in order to help
all students meet high academic standards (Marsh et al., 2004). The way districts
ensured that schools were addressing students’ needs was by objectively monitoring
and facilitating the process.
Key Elements in District Design
A district design needed to have a conceptual framework that included certain
elements to improve teaching and learning. The first element was vision. Successful
districts such as District #2 in New York, New Haven, and San Diego all had vision
statements that viewed teachers, teacher learning, and instructional goals at the core of
their renewal strategy (Hightower et al., 2002). A second element consisted of aligning
the curriculum to state and district standards. For ownership to be realized by
stakeholders, Reeves (1998) stated that each local district had the responsibility to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5 8
develop its own standards that were at least as challenging as the state standards. A
third element was related to the effect of the district’s organizational structure on
schools (Goldring and Hallinger, 1992). The type of district organizational structure,
decentralized or centralized, did not appear to matter as long as it was not fragmented
(Marsh, 2002). The fourth element was the use of high-quality research in making
decisions (Togneri, 2003). Lastly, the fifth element was communication within the
district and dissemination of information to school sites. Districts needed clear
channels of communication to establish expectations for central office staff and schools
(McLaughlin & Talbert, 2002). Trust building and commitment were also critical, as
Marsh (2002) stated:
Districts that mobilize human, social, and physical capital appear to be able to
enact and sustain state and local reform goals and policies. These forms of
capital represent capacity beyond fiscal and staff resources, including
normative aspects of district culture and values, as well as the relationship,
networks, and trust among individuals, (p.38)
It was also important for districts to provide structured professional
development that offered cutting-edge, research-based practices related to what
teachers were doing and provide schools with strategies to improve their practice
(Grossman, Thompson, & Valencia, 2002; Marsh, 2002). The last important element
of a district’s design was to have a database accountability system in place. Districts
needed to collect and examine data in order to set goals and identify improvement
plans throughout the system, to effectively make assessment and accountability a
priority (Marsh, 2002).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5 9
Factors that Influence District Design
Several factors influenced the development and nature of a district’s design.
The capacity of a district influenced the district’s ability to interpret, implement, and
sustain state policies and programs. The district’s capacity was influenced by human,
social, and physical capital (Marsh, 2000,2002). The size of a district influenced the
degree of implementation of instructional improvements according to the number of
connections to external sources of information (Marsh, 2002). Trained and
knowledgeable leadership also affected the design. Marsh (2002) stated:
How leaders’ understand the reform affects the degree and focus of support
for realizing reforms. Various things shaped leaders’ understandings: their
sources of information (e.g., conferences, state test), local context (e.g.,
attitudes about state government), and personal beliefs and experiences (e.g.,
in reading), (p. 31)
Vertical and horizontal aspects of the district’s organization and governance
structure influenced the district’s design in connection with the level of collaboration
with other departments within the district office and the school sites (Marsh, 2002).
The political climate and culture of an organization also affected the acceptance of
new policies and programs (Marsh, 2002). Additionally, the nature and perceived
importance of the policy influenced the effectiveness of a district’s design to improve
instruction (Marsh, 2002).
Strategies Used fo r Change
Districts used a variety of strategies to implement major changes. These
consisted of: arranging professional development for principals and teachers,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6 0
empowering schools, decentralizing organizations, building capacity, or using
externally developed reform designs.
When a district focused on redefining principals’ roles and responsibilities,
they often provided training or professional development to give greater knowledge
about high-quality instruction and improve improvement of their skills as instructional
leaders. This often resulted in altered interactions between principals and teachers
because they were centered on instruction with a common language goal for the reform
efforts (Marsh et al., 2004). In District #2, New York, professional development was
used as a management strategy for teachers and principals when mathematics reform
was implemented. Teachers were taught to share and reflect on lessons while principals
were taught what to look for in a lesson (Stein, 2003).
David (1990) suggested empowerment of schools was a successful strategy. In
this study, schools were delegated functions directly related to teaching and learning,
while districts retained control of district-wide functions: i.e., transportation, food
services, and communication. Elmore and Burney (1999) found that District #2
empowered their schools by first setting clear expectations, then decentralized
responsibility to the school sites. The level of decentralization was related to the
outcomes produced by the school. The schools became the vehicles of major change
in the district.
In a study of over 22 districts, Massell and Goertz (2002) found that some
districts focused on building instructional capacity through professional development
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
61
by aligning curriculum to the standards and using data to monitor results. Other
districts increased instructional capacity by focusing on increasing professional
knowledge and skills. Workshops utilized in the districts centered professional
development activities either on generic pedagogical strategies or on enhancing
instruction in specific school subjects. The different types of activities depended
primarily on the district size and relative amount of instructional or financial resources
available within and outside the district.
Another strategy used by districts was to encourage or mandate schools to use
externally developed reform designs. According to Datnow (2000), approximately
6,500 schools throughout the nation implemented an external prepackaged curricular
and instructional reform design to improve student achievement.
District/School Connection
The connection between the district and schools affected the implementation
of district-led reforms. Implementing a district reform was often overwhelming for a
district and frustrating for a school site. School staff questioned the purpose of the
reform and its affect on their personal and professional lives. To increase awareness of
changes, some principals served as catalysts between the district and their school. A
principal’s ability to lead had a dynamic impact on the implementation and
sustainability of the reform (Reeves, 1998).
A critical component in the change process was good communication. Dufour
(1991, p. 43) discussed the role communication plays:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6 2
Communication can serve as the link between organizational cause and the
individual’s commitment to it, and it is the principal’s skill in communicating
which forges that link over time. People within the school are entitled to know
why they are being asked to do certain things and how they relate to the big
picture.
Principals’ actions taken to instigate reform can also serve to establish
effective communication with staff members. Good communication necessitated that
principals used the district’s shared common language with staff to familiarize them
with valuable ideas, and allowed them to provide feedback on districtwide initiatives
and strategies (Marsh et al., 2004).
An additional way for districts to connect with schools was for district staff to
work directly with teachers at school sites. By providing professional development,
substitutes, on-going staff development, resources, data analysis, and feedback
opportunities, teachers were empowered to be a part of the change process while taking
ownership of the reforms (DuFour, 1991; Massell & Goertz, 2002).
Reform Impact
Studying the impact of district efforts within classrooms at a school site
required monitoring strategies and the necessary tools to collect pertinent information.
DuFour (1991, p. 62) provided a number of strategies for collecting data in order to
study the impact of a program, such as: “classroom observations, student performance
on criterion-referenced and norm-referenced tests, questionnaires and interviews,
official records (attendance, grade distribution, disciplinary referrals), teacher logs,
and classroom artifacts (lesson plans, instructional materials, unit tests, etc.).”
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6 3
Hall, Wallace, and Dossett (1973) developed the Stages of Concern Model
(cited in Marsh and Jordan-Marsh, 1985). This model offered useful information for
schools and districts because it addressed the seven stages individuals experienced as a
result of organizational change (Marsh & Jordan-Marsh, 1985). To facilitate the use of
the model by educators, researchers offered suggestions for interventions at the various
levels of concern, thus making it practical for educators (Hord, Rutherford, Huling-
Austin, and Hall, 1987). This model revealed how a teacher or administrator felt about
the district reform by having them complete a 36-item questionnaire or by conducting
one-minute interviews.
Another tool that explored the extent to which schools or classrooms were
implementing a reform was an innovation configuration interview (Hord et al., 1987;
Holloway, 2003). Teacher representatives from a school site were briefly interviewed
to elicit self-assessment data. Questions were asked to determine the extent to which an
instructional reform was being implemented in their classrooms. Critical or related
strategies were evaluated as fully implemented, partially implemented, or just getting
started. Using the innovation configuration to collect this self-assessment data provided
information about how the reform was impacting the school and the classroom
(Champion, 2003).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6 4
Exemplary Districts
Edmonton School District
Context. In 1973, Mike Strembitsky was appointed as superintendent of
Edmonton School District and served in this position for over twenty years. Before
that time, the district controlled the budget. Schools did not participate in the
budgeting process. The allocation of school funds was the district finance
department’s decision for the year. Strembitsky disagreed with this policy and
believed that decisions should be determined by the individual who was responsible
for student results (Tucker & Codding, 1998). The finance department opposed the
allocation of any financial responsibility to school sites, convincing the school
board members that empowering school sites with the finances was detrimental to
the district. The mid-1970s marked the beginning of a restructuring period. The
superintendent’s management style facilitated the development of confidence and
trust with district personnel (Marsh, 1995). As a result, he established a core vision
and culture that enabled changes in allocation of resources.
Design. The district’s restructuring approach was based on a set of principles
that described what students should leam, how success in the district should be
conceptualized and assessed, and how the restructuring should be organized. Each
person in the district focused either on providing support services or student learning
results, not both.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6 5
The district’s role was to collaborate with the board to establish purposes
and priorities; to create results and indicators in holding schools accountable; to
provide service to schools in the areas of purchasing, maintenance, and curriculum;
to manage decentralization; to measure results; and to manage the implementation
of the restructuring arrangements (Marsh, 1995).
Schools were held accountable for achieving district-defined results.
School principals were given authority to design a program and to allocate the
necessary resources in order achieve the expected district defined results. Since
each individual was held accountable for how the results contributed to the overall
goals, communication became a key component at the school site. There were
many professional development opportunities for teachers, and the principal
appointed teaching associates to assist with training.
Implementation strategies. Although the implementation strategies consisted
of having the curriculum centralized at the district, a decentralized approach for
instruction was focused at the school sites. Hence, this empowerment shifted the
authority, the program design, and the funding resources to the sites in order to
support the school’s efforts to improve student results. The district established clear
and powerful goals for education, along with guiding principles for the
management processes within the district. The school board ceased their micro
management of the reform process and the operation of schools in order to support
the decentralization efforts. The district’s approach to supporting and guiding the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6 6
principals was to hire seven associate superintendents who each supervised 25
schools and assisted school site principals in establishing performance targets,
discussing data, identifying targets, and creating annual school plans (Marsh,
1995).
Lessons learned. The stability of the district leadership team was an
intricate component that allowed the district to function under the same vision and
sustain the reform effort over a twenty-year period. Defining responsibilities at the
district level and understanding individuals’ roles in the process built
communication and trust, not only in the district, but also in the community.
Empowering schools with the responsibility of overseeing their own school budget
provided schools with the opportunity to allocate the necessary resources required
to improve student performance results. This structural change was made possible
due to the culture of trust established by Strembitsky. Improved student results
helped to sustain the reform for over 20 years.
New York Community School District #2
Context. After a decade of the failure of reform efforts to positively impact
student achievement, Anthony Alvarado became superintendent in New York
City’s District #2. The prior reform focused on subject content and assessment
standards, but the expected outcomes were not realized. The district had diverse
racial, ethnic, and cultural backgrounds. The district ranked sixteenth out of 32 on
citywide tests of reading and mathematics. Alvarado came in with a clear vision for
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6 7
the district and made instructional improvement his first priority (Elmore &
Burney, 1997a, 1997b, 1998, 1999). The vision according to Hightower et al.
(2002), was that “The central office projects a consistent and unwavering
instructional vision for the district that places teachers and teacher learning at the
center of the district’s renewal strategy” (p.5).
Design. By focusing on the classroom, District #2 was committed to
improving instruction and learning. The reform consisted of a systemwide plan for
professional development that included accountability and focused on content.
Professional development was integrated into classroom practice and maintained a
single focus through the use of a districtwide literacy program, Balanced Literacy.
This program gave teachers a framework that supported efforts to teach literacy and
also served as an evaluation tool. This tool was used for observations, and it
encouraged conversations about the teaching practices within the district.
Professional development for teachers centered on curriculum and teaching
practices in the academic content areas. Teachers were expected to become familiar
with the theory and rationale of the program in order to teach literacy according to
the goals and purposes of the Balanced Literacy Program (Stein & D ’Amico,
2002).
Implementation strategies. District #2 used professional development as a
resource to empower teachers and to build administrator leadership capacity.
Having a combination of diverse individuals working together toward common
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6 8
goals supported the district’s vision and it provided the learning community with
coherence. A key component of the professional development was to assess the
extent to which teachers used the adopted districtwide curricula and approaches of
the literacy program, which required principals to play a strong role in the process.
Administrators attended the same training as teachers to learn what to look for
when they entered classrooms and to improve their personal instructional
leadership skills (Elmore & Burney, 1997a). The district treated all schools as
learning laboratories. Classroom observations by the school principal and district
leaders were used to provide teachers with feedback on their teaching practices and
to determine what additional resources or training they required to improve their
teaching. After each classroom visit, the district leaders and principal discussed the
instruction observed and what was the next step for that particular teacher.
Individuals requiring additional training were partnered with teachers who had
more experience, either for a one-day observation or for a three-week internship
with an expert teacher (Stein & D’Amico, 2002).
Lessons learned. This district served as an example that districts can be
purposeful and effective in addressing teaching quality under difficult conditions
(Hightower et al., 2002). Changes in instruction occurred when teachers received
focused support based on individual improvement needs (Stein & D’Amico, 2002).
The district had a strong culture of shared values related to instructional
improvement that changed instructional practice and improved test scores. An
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6 9
instructional framework provided teachers with guidance on what they must know
and be able to do within their classrooms (Stein & D’Amico, 2002).
San Diego City School District
Context. In the mid-1990s, the San Diego School community had a poor
perception and low confidence in its local schools. They were frustrated with
achievement gaps in student performance and the low-skilled graduates that were
entering the workforce. The community’s resentment grew stronger when a
teachers’ strike resulted in uncooperative actions from the district. Resources,
information, and capacity were not fairly distributed within the district, hence,
equity issues were created within the district and schools. The district’s supervisory
staff worked in isolation and rarely worked collaboratively with other departments.
Test scores revealed continued performance gaps within subgroups based on racial,
ethnic, and geographical parts of the community. The Chamber of Commerce,
along with other business leaders, focused on changing the district’s leadership.
They elected school board members who would appoint a new superintendent and
reform the district. In 1998, a divided school board appointed an attorney, Alan
Bersin. Bersin proceeded to appoint Anthony Alvarado as chancellor of instruction
and co-leader of reform. Alvarado assisted Bersin in the reform effort (Darling-
Hammond, Hightower, Husbands, LaFors, & Young, 2002; Hightower, 2002).
Design. Alvarado believed the best way to improve student learning was to
concentrate on teachers’ practices and to focus district decision making on
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 0
instructional needs. His theory of action recognized that student learning increased
when interaction between students and teachers improved. Bersin and Alvarado
structured the district organization towards teaching and learning, while focusing
on instructional strategies throughout the district. Bersin was responsible for
political, organizational, and business aspects of the district and Alvarado managed
all the instructional components. Overall, Bersin defined his role as supporter for
what Alvarado was trying to accomplish instructionally. They both disapproved of
the districts’ status quo and implemented goals that were focused, coherent, and
dealt with instructional issues not operational. They also created a structure that
provided individuals with the capacity to do their jobs. They believed that the
district needed to make decisions using data. In addition, the district needed to treat
employees as individual learners who contributed to the learning of others and the
larger organization. In 1998, Bersin and Alvarado focused on changing both the
central office and the schools in order to create an organizational system that
operated around instruction by investing strategically in teachers’ work because
they strongly felt that instruction would increase student knowledge and
achievement (Hightower, 2002).
Implementation strategies. Prior to implementing the reform, the district
conducted research to find the best practices that supported teaching and learning.
The district’s system was driven by standards, focusing on building the profession
and tailoring the reform efforts to specific school contexts. The district began the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
71
reform by first focusing on the lowest performing schools and students. The reform
was implemented in two phases. First, an infrastructure was designed to support
professional development for principals, as principals were seen as important
change agents. Seven principals were trained by the University of Pittsburgh’s
Learning Research and Development Center (LRDC). They became District-Wide
Instructional Leaders (IL) and each led twenty-five learning communities within
the district. Principals were trained by their peers on ways to support teacher and
student learning. To hold principals accountable for knowledge acquired, IL
conducted two visits per semester to the classroom with the principals to verify that
the reform had been implemented in the classrooms. They provided immediate
written feedback to the principals on what needed to be improved by the next visit.
The second initiative focused on restructuring the central office in order to provide
a more effective and efficient way to support efforts that improved teaching in
schools (Hightower, 2002). Each central office employee was required to know
how their individual efforts supported teaching and learning in the classroom.
Lessons learned. San Diego had a vision and conceptual framework that
focused on instruction. They began by identifying the instructional needs of the
district, not the resources that were available. The resources and structures were
secondary and revolved around improving teaching and learning. Assessing
implementation by visiting classrooms, discussing the findings, and setting up
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 2
follow-up plans provided district leaders and principals an opportunity to
dynamically interact and shape the change process (Hightower, 2002).
Summary
Societal demands for increasing student performance were and still are
national topic of concern. Results indicate education is moving towards ensuring all
students are achieving academic excellence and achievement gaps are diminishing
among subgroups of students. States are holding districts and schools accountable
to increase student achievement by mandating annual high-stake testing to measure
individual student achievement. The sanctions established for districts and schools
have caused districts to re-evaluate its role, purpose, organizational structure, and
cultural beliefs.
Change was necessary to meet the demands created by communities, local
agencies, county, state, and national legislation. Districts evaluated their schools’
needs and the resources available to determine how to increase the capacity of its
teachers and administrators so they were better prepared to meet the challenges in
improving teaching and learning for students. Having a focus, creating goals,
communicating policies to practitioners, and providing professional development
were ways that district’s used to improve instruction. These reform efforts were
difficult to sustain unless key elements were supported by factors that influenced
the districts reform efforts. Some districts hired consultants, others used packaged
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 3
models, or used its own personnel within the district and schools to provide
guidance.
The reform path taken by districts varied from district to district, but in the
end, what separated exemplary districts from the rest was that they demonstrated
improvement in student performance at each of their schools, and the reform effort
was sustained despite the barriers and challenges encountered by the district. The
rapport established between districts and schools was vital in meeting the intended
outcomes of the reform. Both entities needed to know and understand the district
design, work collaboratively in ensuring that school level efforts are effective, aim
toward fully implementing the district design in a timely manner, and have the
capabilities to measure its effectiveness and evaluate its success.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 4
CHAPTER THREE
RESEARCH METHODOLOGY
This chapter describes the research methodology used in the study, along
with a description and rationale for the sample, sample size and population. Data
collection techniques, instrumentation tools, and data analysis are included. A case
study approach was used as the most appropriate way of addressing the research
questions through a qualitative tradition of inquiry (Creswell, 1998). The purpose
of this case study was to understand the role of one district’s efforts to enhance
good teaching and learning. This chapter describes the steps and procedures taken
by an exemplary district to implement an instructional reform design to improve
teaching and learning in every classroom. The adequacy of the district’s design was
evaluated in terms of school level efforts, the extent of implementation, and its
effectiveness.
Four research questions address the purpose of this study:
1. What was the district design for improving teaching and learning?
2. What school level efforts facilitated the implementation of the district
design?
3. To what extent had the district design been implemented at the school
and classroom levels?
4. How effective were the design and implementation strategies at the
school and classroom levels?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 5
“Case studies provide depth, detail, and individual meaning” (Patton, 2002,
p. 16). Creswell (1998) gave insight as to how and why a case study is chosen:
Choose a case study to examine a “case,” bounded in time or place, and
looks for contextual material about the setting of the “case.” Gather
extensive material from multiple sources of information to provide an in-
depth picture of the “case.” (p.40)
The focus was to develop an in-depth analysis of a single case with a
narrative format to present the findings. This study was aimed at: (1) describing
and understanding the district as the unit of analysis, (2) determining how a school
understood, interpreted, and to what extent implemented the district’s design, and
(3) discovering how the effectiveness of the design was perceived.
An analytical case study was conducted using both qualitative and
quantitative methodologies to measure the extent of implementation and
effectiveness of the district’s design. The use of qualitative research was the
preferred methodology as the “researcher builds a complex, holistic picture,
analyzes words, and reports detailed views of informants, and conducts the study in
a natural setting”(Creswell, 1998, p. 15). Qualitative inquiries find meaning in what
administrators and teachers experience and feel at their school as a result of
implementation of the district’s design. “The themes, patterns, understandings, and
insights that emerge from fieldwork and subsequent analysis are the fruit of
qualitative inquiry” (Patton, 2002, p.5). The interviews provided qualitative
insights into the personal experiences and perspectives of the principal, leadership
team members, lead teachers, and classroom teachers at Russellville High School;
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 6
and were narrowed into a small number of core themes. The study will enable any
future readers to gain an in-depth understanding of the best practices and areas of
concern revealed by the participants who implemented the districtwide instructional
improvement.
Sample
The design strategy used purposeful sampling that provided information-
rich insight and in-depth understanding of research gathered through the chosen
sample (Patton, 2002). Through the use of purposeful sampling, one high school
district was selected for the study with the objective of gaining a comprehensive
understanding of its role in instructional improvement efforts.
The selection criteria for the district included a student population between
ten thousand and sixty thousand students. The researcher sought to study a district
comprised of students from diverse backgrounds and socioeconomic status. The
selection criteria for the school that participated in the study must include evidence
that the school met its Academic Performance Index (API) for two of the last three
years. Additionally, the researcher sought to study a school that maintained stable
leadership, in which the principal had been at the site for at least two years. Table 1
shows the district and school criteria required.
The district selected had the characteristics that met the following criteria:
1. The district served over 16,000 students.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 7
2. The district was comprised of a diverse student population in regard to
ethnicity and socioeconomic status (SES).
3. The district demonstrated an intentional effort to improve teaching and
learning within the past two years.
Table 1 .District and School Selection Criteria
District School
Student population between 10,000
and 60,000
Met API two years of last three years.
Diverse Student Population
(SES/Ethnicity)
Stability of leadership for at least two
years.
Districtwide effort to improve teaching
& learning for at least two years
The school selected for this study had the following characteristics that met
the criteria:
1. The school met its API the last three years.
2. The principal was at the school site for five years.
Southern California districts that met the criteria were identified by a cohort
group of doctoral students as exemplary districts that provided leadership in
implementing reform designs that focused on instruction. The potential districts
were sent written invitations to participate in the study in writing by the Associate
Dean of Academic Programs from the University of Southern California. The
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 8
districts’ letters included statements that provided an overview of the study, names
of research team members, the methodology, and possible benefits to the district
and school for their participation. Los Coyotes High School District accepted the
invitation and welcomed five doctoral students to research its practices. Two of the
doctoral students focused their individual studies on the district change process, the
factors that influenced good instruction, and the impact of the systemic efforts to
improve teaching and learning at schools and classroom levels. The other three
doctoral students, including this researcher, studied the district’s instructional
improvement efforts at individual school sites. The participants in the study
included the high school principal, leadership team members, lead teachers, and
classroom teachers. Participants in the study were informed of their rights as
research subjects, reminded that their participation was voluntary, and reassured of
their confidentiality and anonymity.
Purposeful sampling and stratified purposeful sampling were utilized in
selecting participants for this study. Purposeful sampling was used in selecting the
principal, two leadership team members, and four lead teachers who participated in
formal interviews. According to Patton (2002), purposeful sampling is aimed at
gaining insight about a phenomenon. The principal was considered to be the link
between the district and the school in implementing the design. Therefore, the
principal was the first participant used to gather information. The second group of
participants were the leadership team members, assistant principals, who were
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 9
directly involved in implementation of the district design at the school. The third
group of individuals that participated the in purposeful sampling were the lead
teachers who consisted of department chairpersons or members of the school staff
curriculum team, and were chosen based on their level of involvement in
implementing the district’s design. The school principal assisted in the selection of
leadership team members and lead teachers to participate in the formal interviews.
Stratified purposeful sampling was utilized in selecting the six classroom
teachers who participated in the interviews and self-report. According to Huberman
and Miles (1994), stratified purposeful sampling illustrates subgroups and
facilitates comparisons (cited in Creswell, 1998). Within the high school one
teacher was randomly selected from each of the core content areas, special
education, and an elective area because those were the disciplines the district
targeted for trainings according to the assistant superintendent of education and
documentation at Los Coyotes High School District. At Russellville High School
this resulted in one participant from each of the following discipline departments:
English, math, social science, science, special education, and foreign language.
Classroom teacher participants were also asked to self-report the extent of his/her
implementation of the districts’ design strategies using the Innovation
Configuration as the tool. The purpose of interviewing these teachers was to gain
insight into their perceptions of the district design, school level efforts used to
implement the district design, extent of implementation and its effectiveness.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8 0
Two populations were identified as participants in the two surveys
administered. The first population included selected certificated personnel at the
school who included the principal, leadership team members, lead teachers, and
classroom teachers. This selected population of certificated personnel at
Russellville High School was given the opportunity to complete the forty-four item
School Level Survey. The second population included only teachers who were
teaching in the classroom and other selected certificated staff at Russellville High
School who was offered the opportunity to complete the thirty-six item Teacher
Questionnaire (Stages of Concern). This resulted in classroom teachers and selected
certificated staff who completed both surveys: School Level Survey and Teacher
Questionnaire.
Several structured interviews were conducted with the principal, two
leadership team members, four lead teachers, and six classroom teachers to ensure
in-depth data collection from each participant. One interview was scheduled with
each participant, with the exception of the high school principal who participated in
two interviews. The principal’s interview took one hour and the other interviews
took approximately thirty-five minutes. All the interviews took place at Russellville
High School dining school hours with permission from the district and school
principal. The interviews were conducted based on the Case Study Guide which
included: principal interview guide, leadership team member interview guide, lead
teacher interview guide, teacher interview guide, Conceptual Framework A (CFA),
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
81
Conceptual Framework B (CFB), Conceptual Framework C (CFC), and Conceptual
Framework D (CFD).
Selected District Profile
Los Coyotes High School District is located in Southern California and
covers an area of 55 square miles. At the time of this study, the district served
16,195 students in six comprehensive high schools, one continuation high school,
and one alternative education high school. More than 85 percent of the district’s
students were from seven component junior high and middle schools, which
represented four different elementary school districts. The remainder of student’s,
were interdistrict transfers enrolled in specialized curriculum programs such as
magnet and International Baccalaureate (IB) programs. District employees had
worked together as a team to meet the needs of all students, which resulted in a
significant increase in student enrollment and a decrease in the number of students
enrolled in remedial courses.
The ethnic diversity of students in the district continued to increase during
the period studied. There were over 40 languages spoken and 38 countries
represented. Hispanic students represented 47 percent of the district student
population followed by white students at 28 percent, Asian students at 17 percent,
African American students at 2 percent, and the remainder of the other students
made up 6 percent. The concentration of ethnic students was not evenly dispersed
throughout the district. For example, Russellville High School had a Hispanic
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8 2
population that represented 62 percent of the school student body, while Hills High
School had a Hispanic population of only 12 percent. In contrast, the Asian
population at Hills High School was 56 percent: while Russellville High School
had an Asian population of only 6.7 percent.
Students entered the Los Coyotes High School District with educational
backgrounds ranging from less than one year of formal schooling to students who
were nearly prepared for college; and, in fact, some students came to the United
States with the intent of attending a university.
The district was comprised of a socioeconomically diverse student
population, including 6% of the students receiving free or reduced-pay lunches.
The district measured the poverty level of the district students with California
Work Opportunity and Responsibility to Kids (CalWorks) data. The number of
students identified using CalWorks was 559. The majority of these students, 240,
attended three high schools, and these schools were targeted for assistance as Title I
schools.
An analysis of test data included APIs, California Standards Tests (CST),
and California Achievement Test Sixth Edition (CAT/6) scores across the district,
and showed that reading and mathematics were an area that needed improvement.
All of the schools reported reading as an area for improvement in their Single
School Plan for Student Achievement. Mathematics was also identified as an area
of concern because Algebra I is a required course for all students. The Title I
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
83
schools were most concerned about their students meeting the new, higher
standards required by the state and passing the California High School Exit
Examination (CAHSEE). Disaggregated data indicated two groups of students with
achievement needs: English Language Learners (ELL) and special education
students. The Hispanic subgroup had the lowest scores that needed to be addressed,
both cultural and academic factors contributed to the lower scores.
High quality professional development was the foremost priority of Los
Coyotes High School District. Eighty-nine percent of the teachers were fully
certificated. In addition to using professional development funds to fully qualify
100 percent of the district’s teachers, the funds were used for implementing content
standards and improving instructional skills with an emphasis on reading,
mathematics, and ELL instruction. Professional development funds were also used
to support district efforts to reduce student drug and tobacco use and improve the
safety of the schools.
Los Coyotes High School District was a district with a strong teaching staff
who worked in an environment in which the student demographics were constantly
changing; the demands of colleges and employers with whom district students
interacted were increasingly complex; and the State and Federal academic
accountability requirements were escalating.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8 4
Selected School
Russellville High School was one of six comprehensive high schools in the
Los Coyotes High School District. The October 2003 CBEDS reported an
enrollment of 1,963 students in grades 9 through 12. The school was proud of its
diverse student population with 23 primary languages, other than English, spoken
at the school. The ethnic distribution was 62 percent Hispanic, 16 percent white,
6.7 percent Asian or Pacific Islander, 6 percent African American, 4.5 percent
Filipino and 4.8 percent other. The backgrounds of the community reflected middle
to lower socioeconomic status. As a comprehensive high school, Russellville
offered a wide range of subjects and programs which met their students’ varying
needs and interests. All students took the core curriculum to meet graduation
requirements which prepared them for the challenges of life beyond high school;
whether that represented college or the work world. Special programs existed for
students with identified special needs (special education, Title 1, and/or ELD). In
2003, 30.5 percent of the students were identified as Limited-English Proficient
(LEP) and 15.2 percent were identified as Fluent-English Proficient (FEP). The
LEP and FEP population totaled 45.7 percent, and were served through three levels
of English Language Development (ELD) instruction and Specially Designed
Academic Instruction in English (SDAIE) in the subject areas of English,
mathematics, science, social science, art, and drama.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
85
School personnel included administrators, teachers, counselors, a school
psychologist, a speech and hearing specialist, classroom instructional aides, and
classified support staff. The school was staffed with 89 certificated teachers
including five administrators. Permanent and probationary teachers taught in their
fields of expertise. Bachelor’s degrees or a bachelor’s degree plus thirty units of
upper-division course work were earned by 39 staff members. In addition, twelve
teachers had earned a master’s degree, while another 33 held a master’s degree plus
thirty units. One staff member at Russellville High School had earned a doctorate
degree. In keeping with the district’s commitment to professional development, 26
staff members received training in SDAIE techniques, 39 were Cross-cultural
Language & Academic Development (CLAD) certified, and one was Bilingual
Cross-cultural Language & Academic Development (BCLAD) certified.
School Participants
Principal The first interview was conducted with the school principal of
Russellville High School. This initial meeting provided the opportunity to obtain a
list of leadership team members and lead teachers who were involved in the schools
implementation efforts of the district design, as well as a list of all teachers by
department. The principal had the position for five years. She had previously
served as assistant principal at two other schools within the district. She had been
the principal since the beginning of the district’s instructional reform design.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8 6
Leadership Team Members. Two assistant principals, who were directly
involved in organizing staff development at the school site, participated in the study
through structured interviews. They were also provided an opportunity to complete
the School Level Survey.
Lead Teachers. Four lead teachers participated in the study by way of the
researcher conducting one-on-one interviews with each of them. The principal
selected these teaches for the study because they were directly involved in the
school implementation of the district’s design to improve teaching and learning.
The lead teachers were provided the opportunity to also complete both the School
Level Survey and Teacher Questionnaire.
Classroom Teachers. Using stratified purposeful sampling, the following
were chosen to participate in structured interviews: one teacher from each core
discipline area, one foreign language teacher, and one special education teacher
were chosen by using stratified purposeful sampling to participate in the structured
interviews. At the conclusion of each of the interviews participants were asked to
complete a self-report on the extent of implementation of the district’s design
through the use of an Innovation Configuration chart.
All eighty-four selected certificated teachers currently teaching students at
the school site, including the lead teachers involved in the structured interviews,
were asked to complete both the School Level Survey and Teacher Questionnaire
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8 7
(Stages of Concern). The principal and two leadership team members were asked to
complete the School Level Survey only.
Instrumentation
The instrumentation for this study was developed by 16 members of a
cross-analysis research study team. Team members were Educational Doctoral
(Ed.D.) candidates at the University of Southern California’s School of Education
and met during the summer of 2004 in a thematic dissertation seminar, EDUC 790,
led by David Marsh, Ph.D., Associate Dean of Academic Programs. The research
study team divided into two eight-person teams: one team conducted a study on the
instructional improvement efforts from the perspective of the district; and the other
group focused on the instructional improvement efforts of one school. The eight-
member school team focused on school-led instructional improvement,
collaborative determination of the purpose of the study, and the research questions.
After several class meetings, discussions, and review of current literature, the team
eventually reached consensus on the appropriate data collection instruments and
conceptual frameworks to use for each of the four research questions that focused
on the purpose of this study. The instruments were field tested with a sample of
twenty-four teachers from one school district and refined in the summer prior to
data collection. The two instruments field tested were the Teacher Questionnaire
(Stages of Concern) and the School Level Survey.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8 8
The Teacher Questionnaire (Stages of Concern) was administered as a field
test to determine if the questions on the survey were written clearly and understood
by the participants. Three references (Hall and Hord, 2001; Hord, Rutherford,
Austin, & Hall, 1987, 1998) state that the questionnaire is a set of thirty-five
questions that requires adding only the elements that fit the particular district
design and no other changes in the statements are necessary in the items or the
wording. To verify participants were consistent in their responses, one question was
asked twice which increased the survey items to thirty-six questions. The Stages of
Concern questionnaire is a diagnostic tool for assessing where participants are in
relation to the adopted innovation (Hall & Hord, 2001). It is part of the Concems-
Based Adoption Model (CBAM) created by Hall, Wallace, and Dossett in 1973.
This questionnaire was originally designed with five hundred forty-four potential
questions that covered concerns an individual may experience at particular stages.
From this list, one hundred ninety-five questions were piloted in 1974 to determine
the item correlation of the questionnaire which resulted in a factor analysis
suggesting that over 60% of the common variance among the scales corresponded
to the factor scales. The questionnaire was then narrowed to a 35-item
questionnaire and tested for reliability over a two year cross-sectional longitudinal
study. The findings suggested that each item response correlated highly with
responses to other items measuring the same stage which indicated high internal
reliability, and the alpha coefficients proved internal consistency for each of the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8 9
seven stages (Hall et al., 1998). Several studies have used this tool to determine
individual concerns on an innovation; and a fair amount of evidence proves its
validity, reliability, and consistency. Table 2 displays the coefficients computed in
each of the seven stages from the 1974 results.
Table 2. Teacher Questionnaire (Stages o f Concern) Internal Reliability
Coefficients (Hall et al., 1998)
Stage 0 1 2 3 4 5 6
Alphas .64 .78 .83 .75 .76 .82 .71
The field test School Level Survey consisted of forty-five items. Questions
1-9 focused on perceptions of district design; questions 10-22 related to school
level efforts; 23-34 covered the extent of implementation; and 35-45 asked about
effectiveness of the district design. The initial concern was that the researcher did
not know the consistency of the scale responses with respect to each other. The data
was inputted into an Excel spreadsheet and coded based on the twenty-four
participant responses. Three doctoral students, including the researcher, met with
Dr. Richard Brown in September 2004, and he conducted a syntax analysis based
on the information data provided on the spreadsheet. The spreadsheet consisted of
forty-five columns of variable items and 24 rows corresponding to the participants.
At a second meeting, October 2004, the students met with Dr. Brown again
to discuss the analysis calculations using a Statistical Package for the Social
Sciences (SPSS) to determine if items 1-9,10-22, 23-34, and 34-45 formed a scale;
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9 0
and if the scale was reliable as well as consistent. The data suggested that the
sample questions and four scales had a high reliability, except for one question in
the first scale. The first scale contained items 2-9 and had an alpha coefficient of
.77 which was acceptable. Item 1 was not included in the reliability analysis and
was used as a filter question to determine if participants were aware of the district
design. This question was kept out of the scale because some of the field test
participants showed a lack of awareness about the innovation and their results
skewed the findings. The results indicated a stronger reliability scale of internal
consistency if item six was eliminated. By removing this item, the alpha coefficient
increased to .82 because there was no relationship on how well participants
endorsed that item versus the other items. Table 3 shows the reliability coefficient
of the field test and the results after deleting item six.
Table 3. School Level Survey Reliability Coefficient (Alpha) fo r Field Test Scales.
Scale
Field Test
Reliability Coefficients
Removing Item 6
New Reliability Coefficients
Items 2 - 9 .77 .82
Items 10 - 22 .93 .93
Items 23 - 34 .97 .97
Items 3 5 -4 5 .96 .96
On the second scale, school level efforts, items 10-22 had a high reliability
scale demonstrating a very high internal consistency measure of .93. The lowest
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
91
items were 14 (alpha .37) and 17 (alpha .45) but they were not worth removing
based on the alpha coefficient.
On the third scale, extent of implementation, items 23-34 had a high
reliability also. The internal consistency measure was .97. Only item 23 (alpha .45)
had the lowest alpha, but, once again, it was not enough to have a significant
difference on the scale.
The fourth scale, effectiveness of design, demonstrated a high reliability of
.96; comparable to scales two and three. Item 44 had the lowest correlation score
because it dealt with finding out whether the strategies had helped students, but the
researcher felt it was important to leave in the scale.
Overall, the field test data suggested that the scales were reliable and had an
acceptable reasonable internal level of consistency within each of the four item
scales. With the deletion of item six, the School Level Survey was reduced to forty-
four items. The alpha coefficients calculated for each of the forty-five variables
suggested that the scales were measurable and measured what the researcher
believed they were supposed to measure.
A matrix demonstrating the relationship of data collection instruments to
the research questions addressed are marked with an “X” in Table 4.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9 2
Table 4: The Relationship o f Data Collection Instruments to Research Questions
Data
Collection
Instruments
RQ1:
District
Design
RQ2:
School
Level
Effort
RQ3:
Extent of
Implementation
RQ4:
Effectiveness of
Implementation
C ase Study Guide Components
Principal
Interview
Guide
X X X
Leadership
Team
Member
Interview
Guide
X X X
Lead Teacher
Interview
Guide
X X X
Teacher
Interview
Guide
X X X X
Innovation
Configuration
(self-report)
X
School Level
Survey
X X X X
Teacher
Questionnaire
(SoC)
X
Document
Review
Guide
X X X X
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9 3
Frameworks for Instrument Design
Four conceptual frameworks within the Case Study Guide provided the
parameters for the data collection in this study. Conceptual Framework A (CFA;
see Appendix C), Conceptual Framework B (CFB; see Appendix D), Conceptual
Framework C (CFC; see Appendix E), Conceptual Framework D (CFD; see
Appendix F), and the Case Study Guide (see Appendix A) were developed to
describe and examine the district design and implementation of district-led reform
efforts at the school and classroom level. The conceptual frameworks and data
collection instruments are described as follows.
Conceptual Framework A. The first research question asked, “What was the
district design for improving teaching and learning?” The CFA was separated into
five sections (see Appendix C): 1) The district’s design for improving teaching and
learning; 2) The district’s change strategy; 3) The communication strategies used to
inform staff, parents, students, and the public about the district design; 4) The
district’s standards-based instruction; and 5) The intended outcomes of the
instructional improvement. Four instruments served to gather the necessary
information to answer this question. The instruments used were the Principal
Interview Guide, Leadership Team Member Interview Guide, Lead Teacher
Interview Guide, Teacher Interview Guide, School Level Survey, and Document
Review Guide.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9 4
Conceptual Framework B. The second research question asked, “What
school level efforts facilitated the implementation of the district design?” The CFB
was separated into five sections (see Appendix D): 1) Site level plan for
implementing the district’s design; 2) Site level resources needed to effectively
implement the district design; 3) How the district design was communicated to staff
members; 4) Type of professional development provided; and 5) Support offered.
The instruments used were the Principal Interview Guide, Leadership Team
Member Interview Guide, Lead Teacher Interview Guide, Teacher Interview
Guide, School Level Survey, and Document Review Guide.
Conceptual Framework C. The third research question asked, “To what
extent had the district design been implemented at the school and classroom
levels?” The CFC was separated into nine sections (see Appendix E): 1) How the
design and strategies were monitored; 2) How administrators obtained feedback
from all departments; 3) The data generated; 4) Information on perceived amount
of change throughout the school; 5) Explanations of why the district design cannot
be fully implemented; 6) The process used to modify the strategies; 7) How
capacity promoted or inhibited the extent of implementation; 8) How the reform
affected equity throughout the system; and 9) How the learning culture changed as
a result of the design. The instruments used to address the CFC questions were the
Teacher Interview Guide, Innovation Configuration, School Level Survey, Teacher
Questionnaire, and Document Review Guide.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9 5
Conceptual Framework D. The fourth research question asked, “How
effective were the design and implementation strategies at the school and classroom
levels?” The CFD was separated into four sections (see Appendix F): 1)
Stakeholders’ perspective on the impact of the district design effort; 2) District
design cost effectiveness; 3) District design time efficiency; and 4) District design
meeting intended expectations. The instruments used were the Principal Interview
Guide, Leadership Team Member Interview Guide, Lead Teacher Interview Guide,
Teacher Interview Guide, School Level Survey, and Document Review Guide.
Data Collection Instruments
The eight data collection instruments were created after the conceptual
frameworks were developed and they supported the four research questions. All the
instruments were incorporated into a Case Study Guide that served to organize and
collect the qualitative and quantitative data.
Case Study Guide. The purpose of the Case Study Guide was to provide a
guide for the researcher and others to replicate the study. This guide contained: 1)
A data collection chart; 2) A narrative describing each instrument; 3) A description
of the four data collection phases; and 4) Four conceptual frameworks. Interview
guides for the principal, leadership team members, lead teachers, and classroom
teachers were included. The case study guide also contained the Innovation
Configuration, School Level Survey, Teacher Questionnaire (Stages of Concern),
and Document Review Guide tools that were necessary and critical in obtaining
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9 6
data. The Case Study Guide was written and aligned with the conceptual
frameworks to provide guidance for the study in answering the four research
questions. This guide helped the researcher organize the data collected and
established parameters in obtaining the data to report in Chapter 4. All of the
guiding questions were not asked during the structured interviews and served as a
guide to ensure that the relevant and critical information was collected as indicated
in the data collection chart.
Instruments 1-4: Interview Guides. Structured interviews were conducted
with the school principal, leadership team members, lead teachers, and classroom
teachers. During each interview the researcher obtained individual participant
background information that included job title, years of experience, and years at the
school. The researcher used the corresponding Interview Guides to ensure that the
interview questions were aligned with the research questions. All interviews were
conducted according to the model developed in EDPA 612, Qualitative Methods in
Educational Research.
Instrument 5: Innovation Configuration. Based on the Concems-Based
Adoption Model (CBAM), the Innovation Configuration provided a rubric to
measure implementation of the reform (Hall & Hord, 2001; Holloway, 2003; Hord
et al., 1987, 1998; Roy & Hord, 2004). It focused on knowledge, skills, and
behavioral aspects of the individual’s involvement with change (Hall et al., 1998).
This instrument was used with six classroom teacher participants at the conclusion
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9 7
of each qualitative teacher interview with all six teachers (see Appendix K). The
headers that described the level of implementation were removed for the
participants; and each component level of the implementation were mixed to avoid
a situation where the participants would always choose a particular level and not
read the descriptions that would describe their actual implementation level (see
Appendix L). The researcher asked each of the teacher participants to self-report
his/her individual extent of implementation of the district design on the Innovation
Configuration. This chart was essentially an implementation rubric specific to the
district design that outlined three levels of implementation: fully implemented,
partially implemented, and just getting started. The Innovation Configuration
enabled the researcher to estimate the extent of implementation on each of the nine
district design strategies.
Instrument 6: School Level Survey. Selected certificated personnel including
the principal, assistant principals, dean, lead teachers, and classroom teachers were
asked to complete the School Level Survey. This survey consisted of a brief
introduction, directions for completing the survey, an assurance of confidentiality
and anonymity, eleven demographic questions, and 44 items which took
approximately eleven minutes to complete. The School Level Survey was intended
to measure the degree of awareness about the district’s design for improving
teaching and learning. The questionnaire was grouped in the following way:
Questions 1-8 related to the district design; Questions 9-21 covered school level
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9 8
efforts; Questions 22-33 focused on the extent of implementation of the district
design; and Questions 34-44 were tailored around the effectiveness of the district
design and implementation strategies. The statements were presented on a five-
point Likert scale (ranged from 0 to 4) allowing selected certificated participants
the opportunity to circle their level of agreement within each statement (see
Appendix N). The level of agreement for each point on the scale was: 0) Don’t
Know; 1) Strongly Disagree; 2) Somewhat Disagree; 3) Somewhat Agree; and 4)
Strongly Agree. The survey questions focused on participants’ perceptions about
the district’s design and implementation strategies, school level efforts in
implementing the design, the extent of implementation, and the effectiveness of the
implementation.
Instrument 8: Teacher Questionnaire (Stages o f Concern). Classroom
teachers were asked to complete a second survey called the Teacher Questionnaire.
As mentioned previously, the Stages of Concern questionnaire is the result of three
and one-half years of research and development, including an extensive study of
individuals involved in “change” in both schools and universities. It is also a
dimension of the CBAM, a model developed at the Texas Research and
Development Center to conceptualize and facilitate educational change (Dirksen &
Tharp, 1997; Hall, George, and Rutherford, 1978,1998; Hall and Hord, 2001; Hall et
al., 1973; Hord et al., 1987,1998). The Stages of Concern Questionnaire (SoCQ)
was developed to assess seven hypothesized Stages of Concern when implementing
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9 9
an innovation (Hall et al., 1998). The survey was a 36 item questionnaire based on
the Stages of Concern Model that took approximately eight minutes to complete.
The model offered usefiil information for the school and the district because it
addressed the seven stages individuals experience as a result of organizational change
(Marsh & Jordan-Marsh, 1985). To facilitate the use of the model by educators Hord
et al. (1987), offered suggestions for interventions at various stages of concern, thus
making it practical for educators. The Stages of Concern are: 0) Awareness, 1)
Informational, 2) Personal, 3) Management, 4) Consequence, 5) Collaboration, and 6)
Refocusing.
Hall et al. (1998), provided the following explanation of each stage. The
awareness stage showed that participants had little or no concern about the innovation.
In the informational stage, the individual had a general awareness but wanted to know
more about the innovation. In the personal stage the individual was seeking to find out
how the innovation would affect him/her. In the management stage, the individual
was beginning to ask questions about whether there was a better way to be more
efficient due to the innovation. The focus tended to be on the processes and tasks. An
individual in the consequence stage was wondering how his/her use of the innovation
was impacting and affecting students. In the collaboration stage the individual was
concerned about relating what he/she was doing with what other instructors were
doing. These individuals tended to focus on coordination and cooperation with others.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
100
The refocusing stage occurred when an individual had ideas about something that
would work better than the current innovation in place.
The model was used to determine how a teacher felt about the district reform
and their level of implementation. The responses were on an eight-point Likert scale
that ranged from 0 to 7. The level of agreement on the scale was: 0) Irrelevant; 1)
Not true of me; 2-4) Somewhat true of me; and 5-7) Very true of me (see Appendix
O). This questionnaire was intended to show teachers’ knowledge about the district
design and their comfort level with implementing the district design.
Instrument 9: Document Review Guide. The Document Review Guide was
used to align district documents to the research questions developed in the study.
The guide provided written evidence of the district design, school level efforts,
extent of implementation and effectiveness of district design (see Appendix P). It
was also used as an organizational tool which maintained that the researcher focus
on collecting only relevant information to the research questions.
Data Collection
The school data for this study was collected in four phases over a two-
month period during the months of November and December 2004. According to
Yin (2003) “The use of multiple sources of evidence in case studies allows an
investigator to address a broader range of historical, attitudinal, and behavioral
issues” (p. 98). After each phase of data collection, except for the surveys,
participants reviewed drafts of the data collected by electronic mail to confirm that
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
101
their words and actions were accurate. Findings were based on multiple instruments
and sources that allowed for data and methodological triangulation to ensure
credibility (Yin, 2003).
As previously stated, districts that met the criteria were identified, provided
an overview of the study, and invited to participate in the study. Districts that
accepted the invitation were studied for their instructional improvement efforts at
individual school sites. The Internal Review Board (IRB) application packet was
completed and submitted during the summer of 2004, and the research team was
granted IRB clearance in November 2004.
In the fall of 2004, after clearance from IRB, the three doctoral students
studying Los Coyotes High School District met to discuss and train each other on
the proper use of each instrument and its purpose. This training entailed discussions
of the data collection schedule, proper use of each instrumentation tool, lessons
learned from the field test conducted in the summer, conceptual frameworks
including a review to confirm parameters that were met, and data collection
procedures by each researcher to assure alignment with the Case Study Guide as
well as with one another.
The first data collection phase took place off site, and consisted of
reviewing documents and web sites prior to conducting interviews. The researcher
retrieved and analyzed pertinent information about the district such as district and
school student demographic data (SES status, ethnicity, and language proficiency)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
102
and staff data (number of fully credentialed teachers). This information enabled the
researcher to obtain a holistic view of the context being studied. In order to have a
comprehensive understanding of the district’s design for improving teaching and
learning, the researcher studied official district documents provided by the
superintendent and assistant superintendent of education that described the district
design efforts. At this meeting the researcher verified that the instrument
Innovation Configuration was an accurate depiction of the strategies within the
district design. Also at this meeting confirmation was provided on the school that
would be studied, Russellville High School. The Document Review Guide was
utilized to ensure that the information obtained addressed the research questions.
The researcher also met bi-weekly with the two individuals who were studying the
district design to share findings which led to a better understanding of the district
design.
After the initial document analysis and the first meeting with the two
doctorate colleagues researching the district, this researcher contacted the principal
of the school to determine the logistics for conducting interviews and administering
surveys during school hours.
The second phase of data collection occurred at Russellville High School
where the first round of interviews was conducted over a five-day period. At the
first meeting, the researcher provided the principal with copies and reviewed all of
the instrumentation used for the study and interviewed her using the principal
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
103
interview guide. After the one hour interview, the researcher asked the principal to
identify two leadership team members (assistant principals) and four lead teachers
(department chairperson) who could be interviewed based on their role of
implementing the district design. In preparation to conduct the interviews, the
researcher studied Conceptual Frameworks A, B, and D. The researcher then
conducted six interviews that included the participants selected by the principal.
Purposeful sampling was used to ensure that the participants were directly involved
in the school level implementation of the district’s design for improving teaching
and learning. During the interviews the researcher used the Principal Interview
Guide, the Leadership Team Member Guide and the Lead Teacher Guide, which
outlined key areas of focus that were directly aligned with the research questions.
The third phase also occurred at the school site and took approximately one
hour to distribute the surveys. The School Level Survey took teachers eleven
minutes to complete and consisted of 44 questions. The researcher worked directly
with the principal to clarify the logistics of administering the surveys. The Teacher
Questionnaire (Stages of Concern) consisted of 36 items and was completed in
eight minutes. The principal and leadership team members completed the School
Level Survey. Selected certificated staff received both surveys along with a cover
letter that briefly described the study and outlined directions for completing and
returning the surveys in the enclosed postage-paid envelope. The researcher
addressed the school staff during a faculty meeting by introducing herself and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
104
described the purpose of the study. The selected certificated staff was informed that
the surveys would be coded numerically to send reminder notices if the researcher
did not obtain a 70% response rate. They were informed that the surveys were
voluntary and that their responses would be confidential and anonymous. The
potential participants were provided with an IRB information sheet if they chose to
read more about the study or had any questions or concerns. The researcher
distributed the surveys through staff mailboxes, and participants were given a
three-day window to complete and return the surveys. The collection procedure
included a sixty-cent self-addressed stamped envelope with the researcher’s home
address.
The fourth phase took place on campus over a two-day period to conduct
interviews. The researcher conducted a second round of interviews which included
the principal and six classroom teachers. The teacher participants who were
interviewed were stratified randomly selected within the predetermined criteria
range to ensure a representative sample. A representative had to be chosen from
each of the four core subject areas in addition to one foreign language teacher and
one special education teacher. The researcher coordinated with the principal to
develop the logistics for the second round of interviews that took place during the
school day. In preparation for the teacher interviews, the researcher used
Conceptual Framework A, B, C, and D to focus the questions on the district design,
school level efforts, extent of implementation, and effectiveness of implementation
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
105
and design. During the teacher interviews the participants were informed that the
interview was voluntary and that they could refuse to answer any question or
withdraw from the study without any penalty. After each participant was informed
of their rights as a research subject, they each gave verbal consent to participate in
the study. Each participant was also provided with an IRB informational sheet and
told that their data would be destroyed within two years after the study was
completed. At the conclusion of the interview, the researcher asked each teacher to
self-report his/her extent of implementation using the Innovation Configuration.
The researcher also conducted a second interview with the school principal
to ask any questions that were not addressed during the first three phases of the data
collection. In this final phase, the principal was provided an opportunity to share
any final comments.
A standardized open-ended interview was conducted to ensure that exact
instruments used in the study could be replicated by other researchers, the variation
with interviewers would be minimized, interviews stayed focused for time
efficiency and allowed for ease of the analysis (Patton, 2002). The interviews took
place at the school site, and were conducted during school hours that were
convenient for the participants and with the principal’s permission. The purpose of
the study and the interview guide was shared with the participants prior to asking
the open-ended questions related to the study. Corresponding interview guides were
used to ask probing questions of the participants to ensure that the research
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 06
questions were addressed and answered. The participants were encouraged to share
any closing remarks or additional comments related to the district’s design, school
level efforts, extent of implementation, or effectiveness of the district design. At the
conclusion of the interviews, the interviewees were once again reassured of their
confidentiality and anonymity based on their information provided in the study.
The researcher took notes during the interview and clarified at the conclusion of the
interview the notes with the participants. Participants were given permission to
obtain copies of the researcher’s notes to verify accuracy.
Data Analysis
The purpose of this study was to understand one school’s role in enhancing
good teaching and learning. It described the steps and procedures an exemplary
district had taken to implement an instructional district design for improving
teaching and learning which emanated from the district to the school and classroom
levels. The adequacy of the district’s design was evaluated in terms of its school
level efforts, extent of implementation, and effectiveness of the district design at
one high school. The four research questions addressed the purpose of this study
and served as a guide for data collection and analysis. The data was grouped and
codified into a database for comparison and ease of retrieval.
The 14 interviews conducted were coded according to one of the four
research questions. The data from the researchers notes were reviewed to find
common themes or patterns that compared with each participant’s response to the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
107
questions related to the district design, school level efforts, extent of
implementation, and effectiveness of the district design.
An Innovation Configuration chart was used as a tool with six classroom
teachers to determine their extent of implementation with the district design. The
researcher asked the six participants to self-report his/her individual extent of
implementation of the district’s design. Each participant circled one of the three
levels for each element of the district’s design on the Innovation Configuration
chart (see Appendix L). This chart was essentially an implementation rubric
specific to the district’s design which outlined three levels of implementation: fully
implemented, partially implemented, or was just getting started. Each strategy of
the district design was calculated as a percent based on the number of teachers who
circled their current level of implementation. This enabled the researcher to
determine the percent of teachers who had fully implemented, partially
implemented, or were just getting started in using the district’s design.
The document review guide was used to align district and school documents
to the research questions developed in the study and to provide written evidence of
the district design, school level efforts, extent of implementation, and effectiveness
of the district. It also served as an organizational tool that enabled the researcher to
remain focused on the research questions and maintain documents in an organized
fashion (see Appendix P). The researcher reviewed and compared documents to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
108
understand the relationship among data sources as it was collected and sorted
according to the research question it addressed.
Although this was a case study based on a qualitative approach, it used both
qualitative and quantitative methodologies in collecting and analyzing data. The
case study primarily relied on the data obtained through the qualitative
methodologies such as the 14 interviews, the Innovation Configuration, and the
Document Review Guide which provided an in-depth picture of the reform design
and school level implementation efforts. However, due to the fact that the third and
fourth research questions were seeking to determine the extent of implementation
and the effectiveness of the design, a quantitative approach was also required.
The Teacher Questionnaire (Stages of Concern) and the School Level Survey
allowed the researcher to make inferences about the participants on the basis of the
data collected from the quantitative survey data. The survey data was compiled and
analyzed by using various statistical measures.
The responses from the Teacher Questionnaire (Stages of Concern) were
analyzed using a spreadsheet. The spreadsheet was formatted prior to inputting the
data to ensure that the proper groups of questions were placed into their
corresponding stage of concern. Participants responded to each statement on the
questionnaire by circling a number from zero to seven based on their personal
feelings, with zero being the lowest stage of concern and seven representing the
highest stage of concern (see Appendix O). The total overall raw score for each
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
109
stage of concern was determined by taking the sum of the raw scores for each of the
seven stages. In-depth analysis of the data was calculated by determining the
number of teachers whose peak scores, or highest raw score, was in each of the
seven Stages of Concern. The frequency distribution of the number of teachers with
peak scores in each stage, as well as the frequency distribution of the number of
teachers in each Stage of Concern based on each teachers’ second highest score,
provided a more detailed profile of the sample population. Demographic
information collected on the School Level Survey was matched to the Teacher
Questionnaire. This facilitated further analysis of the data, such as the examination
of teachers’ stage of concern in relationship to the number of days of training.
The School Level Survey was analyzed also on a spreadsheet by tallying the
responses in each of the four scaled areas to discover the perceptions or patterns of
the respondents at the school with regard to the district’s design. As mentioned
previously, this questionnaire was grouped in the following way: questions 1-8
related to the district’s design; questions 9-21 related to the school level efforts;
questions 22-33 focused on the extent of implementation of the district’s design;
and questions 34-44 were tailored around the effectiveness of the district’s design
and implementation strategies. A reliability analysis was calculated to determine
the internal consistency of each scale using alpha coefficients. The respondents’
results were placed in a summative chart as a percent to facilitate comparisons and
reporting.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
110
In an effort to increase internal validity, methodological triangulation and
data triangulation were used by combining both qualitative and quantitative data.
Triangulation ensured that the findings were supported by using multiple sources
which confirmed and validated the results.
Summary
This chapter discussed the research methodology used for this study. This
discussion included a description of the sample and population, data collection
instruments, data collection process, and the data analysis. Procedures for this study
included receiving permission from the district superintendent to conduct a study in
the district, receiving permission to conduct the study from the high school principal,
conducting interviews, distributing and collecting questionnaires, and gathering
documents. Data findings, analysis, and interpretations of each research question are
presented in the next chapter.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
I l l
CHAPTER FOUR
FINDINGS, ANALYSIS, AND DISCUSSION
Introduction
In this chapter, the findings of the study are presented and discussed. The
findings are explained in terms of the four research questions and accompanying
case study guide which included four conceptual frameworks, interview guides,
and instrumentations. The first section of the chapter provides a description of the
district design at Los Coyotes High School District in order to answer the question:
What was the district design for improving teaching and learning? The second
section of this chapter reviews the findings related to the procedures and process
used by the school to implement the district design in order to answer the question:
What school level efforts facilitated the implementation of the district design? The
third section of this chapter reviews the use of the district design in order to answer
the question: To what extent had the district design been implemented at the school
and classroom levels? The fourth section of this chapter reviews perceptions of the
principal, leadership team members, lead teachers, and classroom teachers toward
the district design in order to answer the question: How effective were the district
design and implementation strategies at the school and classroom levels? A
discussion involving analysis and reflection of the research findings concludes the
chapter.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
112
Research Question #1:
What was the district design for improving teaching and learning?
To answer this question, various instruments in the Case Study Guide were
used to facilitate the data collection. The following instruments were used and aligned
with Conceptual Framework A (CFA) to assist in the data collection phases: The
Principal Interview Guide; Leadership Team Member Interview Guide; Lead Teacher
Interview Guide; Teacher Interview Guide; School Level Survey; and Document
Review Guide. An informal meeting was held with the superintendent and assistant
superintendent of Los Coyotes High School District to confirm the researcher had an
accurate depiction of the district design components on the Innovation Configuration
chart. In addition, information was provided by two researchers who studied the
district design at the district level and notes of interviews with district administrators
of Los Coyotes High School District. The qualitative and quantitative data gathered to
address the first research question was organized and presented as follows: district
background, creation of the district design, strategies of the district design,
communication of the district design, training provided at the district, teachers’
perceptions of the district design, assessment of the district design, and concluded
with the respondents’ results to the School Level Survey for Items 1-8.
District Background
The current superintendent shared that the board of trustees established seven
goals each year as part of the district’s mission statement. Under each goal several
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
113
objectives were used to assess how well things functioned in the district and if annual
goals were met. One of the goals was to ensure that all students had educational
programs with sufficient depth, breadth, and quality which prepared them after
graduation for post-secondary education or a career. Each year the board of trustees
adopted priorities. The first priority listed was instructional excellence because the
board had a strong belief that they could do a better job of influencing how teachers
taught rather than the amount students learned. Los Coyotes High School District’s
professional development plan defined instructional excellence as “Assuring
excellence in classroom instruction by every teacher in every school, classroom, and
during each period.”
During the spring of 2002, the administrators at the district tried to decide
what role the district would take to improve instruction and the type of support it
needed to communicate to schools as well as teachers its efforts to achieve
instructional excellence. A survey was sent to all certificated staff members
throughout the district and requested their input regarding areas of interest for
professional development in the 2002-2003 school year. A total of 225 certificated
staff members responded from varying discipline areas including agriculture,
business, English, fine arts, foreign language, guidance, library, mathematics,
physical education, science, social science, special education, and vocational
education. The results suggested that certificated staff members throughout the
district needed and wanted effective instructional strategies, as reported in a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
114
Weekly Board Letter (WBL). After obtaining information on the needs requested
by certificated staff members, the prior superintendent worked with his
administrative cabinet members made up of three assistant superintendents from
Education Services, Human Resources, Business Services, and the school
principals to identify specific activities or programs that would meet the needs of
certificated staff members and that simultaneously addressed achieving the district
priority of instructional excellence. In the summer of 2002, the superintendent
recommended that the schools focus on research-based strategies from a book
written in 2001 by Marzano, Pickering, and Pollack called Classroom Instruction
that Works (CITW). The nine instructional strategies in this book became the
components (strategies) of the district design. Initially, the strategies were
presented to the seven principals as an option if they didn’t have a better plan in
place to improve instruction. Since this was a top-down approach, the principals
were informed that professional development activities were going to be designed
to implement the nine instructional strategies that would improve instruction and
meet the needs of diverse learners.
During the months of October, November, and December, 2002, the top-
down district design was implemented and each school participated in workshops at
the district office that centered on the CITW theme. Each school participated in two
days of workshops that covered the use of three of the nine instructional strategies
over a three-year period of classroom instruction (See Table 5). The schools were
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
115
encouraged to bring thirty teachers which included five English teachers, five math
teachers, five science teachers, five social science teachers, five special education
teachers, and five elective-area teachers. The teachers learned about the strategies,
practiced the strategies, developed a department implementation plan; and as a
group, they brainstormed ways to implement the strategies schoolwide. These
workshops were organized by the assistant superintendent of education, and the
presenters varied from three to four teachers from different schools within the
district who were selected by their principals as exceptional instructors in the
classroom.
Table 5: Timeline fo r Classroom Instruction that Works
Year Session Strategy
2002
1st Session
Similarities and Differences
Nonlinguistic Representations
2n d Session Summarizing and Note Taking
2003
1st Session
Reinforcing Effort and Providing Recognition
Setting Objectives and Providing Feedback
2n d Session Homework and Practice
2004
1st Session
Generating and Testing Hypotheses
Cues, Questions, and Advance Organizers
2n d Session Cooperative Learning
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
116
Creation o f the District Design
The prior superintendent at Los Coyotes High School District had the idea
to use CITW as the district design to focus on instructional excellence. The
implementation used a top-down approach and was guided by a book. The assistant
superintendent of education was instructed to implement a plan using the CITW
book and to include training workshops for teachers. The principal, leadership team
members, lead teachers, and classroom teachers who were interviewed all
concurred that they were not involved in the creation of the district design plan.
Components (Strategies) o f the District Design
The strategies of the district design consisted of nine research-based
instructional strategies which were implemented over a three-year period (see Table
5). In 2002, the first year of implementation, thirty teachers from Russellville High
School attended the district workshop which covered two strategies: Similarities
and Differences and Nonlinguistic Representations. One month later, thirty teachers
returned to the second session. According to the principal, all content area
department chairpersons were required to attend and they were asked to choose
four other members from within their department to accompany them to the
workshop. At the second session, only the strategy Summarizing and Note Taking
was covered at the district training in 2002. In 2003, the second year of
implementation, two strategies were covered in the first session: Reinforcing Effort
and Providing Recognition and Setting Objectives and Providing Feedback. The
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
117
department chairperson was once again required to attend both sessions that year
and was able to rotate different teachers to attend the workshops. The second
session of that year took place one month later and covered the sixth strategy,
Homework and Practice. In 2004, the third and final year, the first session covered
Generating and Testing Hypotheses and Cues, Questions, and Advance Organizer
strategies. The second session for that year covered the ninth strategy, Cooperative
Learning which was completed in December 2004.
Communication o f the District Design
The principal of Russellville High School was informed about the district
design in the summer of 2002 at an administrative cabinet meeting. She was told
that the purpose of the design was to implement the best practices to improve
student success at each school throughout the district. Leadership team members
stated that they were informed by their principal of the district design and that it
was going to be implemented in the fall of 2002 through a series of workshops over
a three-year period. One CITW book was provided to each school and the
leadership team members had the option to either read the book or glance through it
to become familiar with the strategies. Shortly after receiving the book they were
provided with an overview of the district design at the annual management retreat
in August 2002. The assistant superintendent of education presented an overview of
the district design to all the administrators and provided them with a three-year
timeframe listing the strategies to be covered. District personnel expected the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
118
principals to attend all the CITW trainings to show their support for the district
design. Initially, school administrative teams thought they had flexibility on how to
approach CITW, but it became clear within the first three months of
implementation that schools were expected to follow the district design plan.
Some lead teachers stated that they were informed about the district design
by an invitation from the assistant superintendent of education to present the
training sessions on CITW to six schools. If they accepted, the assistant
superintendent explained the strategies they would be presenting and allowed them
to design the workshop sessions. Lead teachers and classroom teachers who were
not presenters were informed about the district design through their monthly school
leadership council meetings, department meetings, informal conversations with the
principal, or by a note in their mailbox that requested their presence at the next
workshop if they were able and willing to attend. The majority of teachers stated
that they felt they knew the strategies, but were not aware that CITW was the
district design. Teachers who did not attend district training sessions knew the
strategies were important because the strategies were a focus at their school staff
development and department meetings, but they also stated they did not know that
CITW was the district design. One teacher stated that even though she was very
involved at her school, she did not realize the strategies were part of the district
design. She came to the conclusion that if she had just recently found out about the
district design, she was sure that other teachers were also unaware. Overall, the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
119
majority of teachers stated that they were familiar with the nine strategies but did
not have knowledge or a comprehensive understanding of the district design.
Training Provided at the District
The training for the principal, leadership team members, lead teachers, and
teachers was the same except for the presenters who had to learn the strategies on
their own. The individuals who attended the district workshop sessions received
extensive in-depth training by teachers within the district. The presenters were
provided with informal objectives and did not receive training. They were given
release time to meet with their presentation partners to organize and plan the
workshops collaboratively. The expectations from the assistant superintendent were
to discuss the research that supported the specific strategy, define the strategies,
develop or model lessons using the strategies, provide hands-on activities, and
address a variety of disciplines. The teachers had to leave the workshops with
something in their hands they could use in their classrooms the next day. The
district chose teachers to be as presenters who were capable of doing the workshops
on their own and required limited assistance. The presenters prepared by working
together; they outlined the chapters covering the strategy, highlighted key points,
focused presentations toward specific strategy, and created ways to present the
material without lecturing.
The teachers who did not attend the district trainings learned the strategies
from their peers within their departments who had attended the district trainings or
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 2 0
at a school staff development day. The majority of the staff development meetings
at Russellville High School focused on introducing, explaining, modeling, and
reinforcing the strategies recently covered at the district training sessions. For the
first year, the school paid two lead teachers, who were district presenters, from
Title I funds. They were asked to share the three strategies with the entire teaching
staff by providing mini-lessons similar to the district presentations at a school staff
development. For the second year, six strategies were presented by six teachers in
different classrooms. The teachers at the school had thirty-minute periods to visit
each classroom and learn about the strategies and the activities that supported these
strategies. Monthly department meetings provided opportunities for dialogue and to
reinforce the strategies throughout the school year.
Teachers ’ Perceptions o f the District Design
The principal, leadership team members, lead teachers, and classroom
teachers perceived the district design as an effective instructional research-based
teaching tool that provided strategies teachers could use to help their students
improve and grow academically. They felt the district design provided the
structure, facility, trainers, resources, and release time for teachers to be trained. “It
has been wonderful the way the district has involved each school and the teachers,”
stated a leadership team member. The thirty teachers at each training session were
empowered to become creative instructors and to train other teachers throughout
their department on the strategies. One teacher stated that the first year of training
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
121
was more applicable to what she had done in her classroom and that subsequent
years became less useful. Two other teachers stated that districtwide they were not
clear as to what was the district design, but that schoolwide there was an interest in
improving Standardized Testing and Reporting (STAR) test scores in every content
area. These teachers felt that by using CITW their school was trying to get teachers
to think differently about the strategies they used to present lessons instead of using
teacher-directed lessons.
Teachers perceived the district design as affecting their teaching practice in
a positive way. They felt it trained them to reflect on their teaching and it added
additional tools to improve instruction. The strategies included in the district design
were not new to teachers but it refined and improved things teachers were already
doing in the classrooms. One teacher stated that it changed the way he delivered
instruction to students; and the use of some strategies resulted in a positive change
in his students’ understanding of the material and ability to retain the content
information.
Assessment o f the District Design
The principal and leadership team members believed the best way to assess
CITW was to compare annual state testing results or by using common assessments
in content areas. Since this school had demonstrated Academic Performance Index
(API) growth every year, administrators hoped it was relevant to the CITW changes
in teaching methodologies. The principal and leadership team members monitored
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
122
the district design by visiting every classroom and using a checklist that listed the
nine CITW strategies. This checklist listed each classroom number along with the
teacher’s name. The administrator, who visited the teacher during that week,
checked off the strategy observed, initialed, dated, and submitted a summative
checklist to the district office on a monthly basis. Also, at the end of each school
year, the principal distributed a survey to all staff members to find out which
strategies were most commonly used by teachers throughout the school year and
the results were reported by electronic mail to all staff at the school.
School Level Survey Items 1-8
The School Level Survey was completed by selected certificated staff
members who included the principal, three leadership team members, the dean, lead
teachers, and classroom teachers. Sixty-six respondents compiled the results in
Table 6 which provided a summative percentage of eight items that were aligned to
answer research question one, “What was the district design for improving teaching
and learning?” The respondent’s data was tested for reliability to determine if the
scale that contained items 2-8 was consistent. The results indicated that for the
seven items the alpha coefficient was .90 which signified a high level of internal
consistency among the respondents. Item 1 served as a filter question to determine
if participants were aware of the district design and was not included or calculated
into the reliability analysis. The first four items suggested that 90% or higher of the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
123
Table 6:
Summative School Level Survey Results by Percentage for Research Question 1
(N=66)
5
o
a}
£
a )
<0
C D
0
t_
CD
C O
C O
03
0
L_
O )
<
0
0
i_
a>
RQ1 What was the District Design for Improving
c
b
a
•4— >
C O
<
>*
Teaching and Learning?
c
o
a
a>
c
P
< 0
-C
5
03
C
0
E
o>
c
8
-4— '
C O
o
C O
C O
I i
I am aware o f the Classroom Instruction that
1 Works strategies for improving teaching and
learning.
6% 2% 2% 24% 66%
i
2
1
The district supports standard-based instruction. 6% 2% 2% 8% 82%
I
3
I
The Classroom Instruction that Works strategies
are centered on improving student learning.
4% 0% 3% 11% 82%
I 1
The Classroom Instruction that Works strategies
4 call for the use of multiple measures to assess
student performance.
6% 0% 3% 17% 74%
1 I
The Classroom Instruction that Works strategies
5 include ways to identify acceptable levels of
student performance.
12% 0% 5% 45% 38%
I
6
1
The district believes that all students can meet
high standards.
8% 2% 8% 38% 44%
1 .... 1
The Classroom Instruction that Works strategies
7 foster a collaborative approach to improving
student performance.
8% 2% 8% 38% 44%
1
8
1
Leaders in the district want everyone to use the
Classroom Instruction that Works strategies.
14% 2% 2% 14% 68%
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
12 4
respondents agreed that they were aware of the district design, that the district
supported standard-based instruction, that CITW centered on improving student
learning, and that CITW provided multiple measures to assess student performance.
Items 5-8 showed that approximately 83% of the respondents agreed that the
district design provided ways to identify acceptable levels of student performance,
that the district believed all students could meet high standards, that collaboration
was a focus, and that leaders in the district wanted everyone to use the district
design, CITW. Items 5 and 8 had approximately 13% of the respondents who stated
they did not know if they disagreed or agreed that CITW included ways to identify
acceptable levels of student performance and acknowledged that leaders in the
district wanted teachers to use the strategies from CITW.
Research Question # 2:
What school level efforts facilitated the implementation of the district design?
To answer this question various instruments in the Case Study Guide were
used to facilitate the data collection. The following instruments were used and aligned
with Conceptual Framework B (CFB) to assist in the data collection phases: The
Principal Interview Guide; Leadership Team Member Interview Guide; Lead Teacher
Interview Guide; Teacher Interview Guide; School Level Survey; and Document
Review Guide. The qualitative and quantitative data gathered to address the second
research question was organized and presented as follows: description of the role of
each participant, information on how the district design was monitored at the school,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
125
the implementation timeline, the training provided to teachers at the school, support
activities, areas that were effective, areas that needed improvement, and concluded
with respondents’ results to the School Level Survey for items 9-21.
Roles
The principal perceived her role as the one who had to share the information
she had acquired about the district design, CITW, at the administrative cabinet
meetings with her school staff. She presented the information to her school
leadership council (administrators, counselors, and department chairpersons) and
informed them that the district was supporting professional development that
aligned with their school goals and objectives to use teaching practices that helped
improve student success. She attended every district workshop all three years and
worked with the assistant superintendent of education to develop a plan to
implement, monitor, and sustain CITW schoolwide.
The leadership team members had several roles in the implementation of
CITW at the school site. They were seen as organizers, supporters, and evaluators.
The principal demonstrated her support for CITW by making it a main area of
focus on staff development days held at the school. The leadership team members
were required to set money aside from the school budget to pay for up to five homs
of teacher release time so that teacher presenters could prepare for the staff
development presentations. Most of the materials needed were already used in the
classroom by presenters, so they were not creating anything new, they just needed
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
126
time to put the presentation together. Administrators were also responsible for
finding the facilities to hold these meetings and organizing the day’s activities to
assist the presenters as well as support them. Administrators were also required to
visit every classroom at least once a week, “Every Classroom Every Week,” to
informally monitor which teachers used any of the nine strategies during
instruction.
The lead teachers perceived their role as the link between school
administrators and their colleagues, the classroom teachers. They made sure that
their department members had the necessary materials and support to implement
CITW. At department meetings, lead teachers included time to share lesson plans
and dialogue about activities teachers were using that were aligned with CITW.
The lead teachers were provided with release time by the principal if they chose to
visit, observe, or provide guidance to teachers within their department. The
majority of lead teachers stated they used this opportunity to act as a mentor and
dialogue about CITW and other things related to professional growth. They also
selected and encouraged their department members to attend the trainings at the
district on CITW. According to one lead teacher, “It is a combination of these
things that is allowing for sustainability of CITW within our department.”
The teacher’s role was to attend the trainings if they were invited and
become a resource and expert on the strategies at their school. Some teachers chose
to teach individuals only, others chose to teach all their department members, and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 2 7
the remainder became schoolwide presenters at staff development. Each department
decided how they wanted to implement the strategies which usually depended on
their department dynamics. Once everyone within the department was trained,
some teachers took it upon themselves to teach the students about the strategies and
make them aware of the strategies used to teach the instructional lesson.
Monitoring District Design
The philosophy from the board of trustees was that administrators at each
school had to visit every classroom every week. The principal and leadership team
members monitored CITW by visiting every classroom at least once a week and
documented the strategies they observed in teachers’ instructional lessons. They
had informal dialogues with teachers by complimenting them when they observed a
strategy used in a lesson. Administrators also used teachers’ written evaluations as
a formal way to dialogue about their use of CITW.
Lead teachers stated there was no formal process in place to monitor CITW,
only the informal weekly visits by administrators. Some lead teachers (math and
science) chose to conduct their own observations by visiting department members
at least once a month and offering suggestions or compliments as a way to monitor
the implementation of CITW within their departments. Others monitored the
implementation by dialoguing at their monthly department meetings or on
Professional Learning Community (PLC) days. One lead teacher stated that if he
were directed by administration to conduct any formal monitoring, he did not think
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
128
there would be any resentment among colleagues due to his positive rapport with
his department.
Some teachers stated they were not aware if anyone monitored CITW,
while the majority stated that administrators had a checklist and posted findings
every month by electronic mail. One teacher stated that she hoped teachers would
monitor themselves and use a variety of learning techniques to meaningfully
connect with their diverse learners. A few other teachers recalled an end-of-the-
year survey sent by their principal that asked what strategies they were using,
although no one could recall if results were shared or posted. One teacher stated
that her math department chairperson was continuously observing teachers in the
department, providing feedback, and assisting them in lesson planning aligned to
CITW. Other department teachers stated they did not receive feedback from anyone
on CITW strategies they used. Another teacher stated, “Nobody has ever pointed
out that the strategies I am using come directly from CITW.” One teacher stated
since she did not have the opportunity to observe other teachers, she monitored her
colleagues by looking at the “Share Board,” located in the teachers lounge, which
displayed other teachers lessons that incorporated CITW strategies.
Implementation Timeline
All nine strategies were implemented over a three-year period. Every year,
since the fall o f2002, three strategies were released in two sessions. After two
sessions had been completed in one year, the principal reinforced the strategies
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
129
schoolwide by having teachers present mini-lessons on the strategies that had been
covered at the district training. Since all nine strategies were implemented by the end
of fall 2004, the district was in the process of creating a plan to continue the
implementation of the strategies and maintain sustainability with current teachers and
new teachers hired in the years to come.
Teacher Trainings
The trainings for some teachers were offered at the district, but teachers
received on-going training throughout the year from a combination of ways. They
either received training individually, at department meetings, or at staff development
days at the school. The district training sessions had prepared teachers from within the
district as the presenters instead of outside consultants. The presenters reviewed the
research that supported the strategy, modeled the strategy for each content area, and
concluded with the participants working collaboratively with their principal. The
purpose was to create a plan to implement the strategies in every classroom and
conduct trainings for the remaining teachers at their school who did not attend the
district training.
Support Activities
The principal provided support by attending every district training session
to ensure she was knowledgeable on each strategy. She used school site money to
provide teachers with release time to work collaboratively to develop lesson plans
that included CITW strategies. She designed the Single School Plan to revolve
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
130
around CITW strategies and articulated to her staff that it was the main component
for all of the programs offered at Russellville High School.
Leadership team members reinforced the strategies at staff development
meetings, leadership meetings, morning teacher meetings, and department
meetings. They provided necessary release time for the development of lesson
plans and assumed teachers would incorporate CITW during this time; but teachers
were not instructed that it was a requirement.
Lead teachers provided support to teachers within their departments during
PLC time and at monthly department meetings. During these meetings there were
opportunities to share strategies, to share workshop information, to share
instructional materials, and lesson plans; in addition they had discussions on
incorporating different strategies into lessons to add variety to their instruction.
One lead teacher commented, “We certainly share strategies at the discipline level
to avoid teaching the same old way.”
Teachers indicated that they were supported through their department
meetings. Only one teacher stated that the only time she was provided with support
activities was at the annual district workshops, but the rest of the year she did not
feel supported. The remainder of the teachers stated that they shared lessons within
their departments related to CITW, were offered training on staff development
days, and were provided release time from their principal to share within their
department.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
131
Things that Worked
The principal thought the district design was clearly articulated. She liked
the fact that there were several strategies that could work in every lesson and that
teachers could use any one of the nine strategies into their curriculum. Teachers
were not expected to use all nine strategies, but she did want them to use the
strategies that teachers knew were successful in their content area. Teachers liked
the idea that they could apply the strategies that worked best for them in their
content area and did not feel it was dictatorial. Teachers embraced the strategy of
their choice and made it work. The principal also stated that she believed her school
needs were met by using the district design. The district support was above and
beyond the financial and personnel resources she could have provided for her
teachers. The framework of the professional development plan that was offered at
the district aligned to what Russellville High School had set out to achieve— to use
best practiced research-based strategies schoolwide to address their growing
student population of diverse learners. In an effort to support the district design, she
believed that once all nine strategies were implemented it was her role as principal
to support and build a plan to sustain the strategies at her school.
Classroom instruction changed and was very different than it was a few
years ago at Russellville High School. Leadership team members saw departments
work in collaboration to use the instructional strategies consistently. This district
design provided teachers with a resource to look for ideas on how to engage
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 32
students in learning. One administrator stated, “I think teachers responded well to
having research-based examples they can implement immediately.” The examples
that were provided in the book were easily adapted to any class and were not
subject specific which made the strategies attainable for every classroom teacher.
Lead teachers liked the fact that the strategies were research-based and if
used correctly and consistently by a trained teacher, made them valuable in
supporting improved students’ performance. These strategies helped teachers to
think carefully about how they structured their instructional lessons to meet the
needs of diverse learners. Lead teachers thought that all of their colleagues were in
agreement to use any of the strategies for improving instruction because they
wanted to use what worked best for their students. These strategies were things
teachers had already been doing and were comfortable with them. Teachers just
needed to refine the tools and build on them with specific examples. They also
believed it was a great idea to have teachers teach other teachers (trainer-of-trainers
model) because the training did not come from an outside source. According to
lead teachers, the principal’s support and encouragement of departments to utilize
the opportunity for release time to meet and discuss CITW greatly facilitated the
implementation.
Teachers were also in agreement that in-house teacher presenters were
successful and it created buy-in by teachers to diversify their instruction. The
CITW strategies raised awareness that teaching one methodology did not fit all kids
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
133
and that teachers needed to implement other strategies that were proven to be
reliable and valid. One teacher commented, “It’s a good plan and very effective in
terms of it being schoolwide and allowing everybody at the school time to share.”
Other teachers stated they were seeing students thinking a lot more through the
material because of the strategies they used in the classroom. Some teachers felt
their students achieved greater success in their writing and that these strategies
provided students with ways to organize information. One teacher stated that these
strategies forced her to use more manipulatives to address her diverse learners who
were not achieving success in mathematics. Overall, teachers felt that these
strategies changed the way they presented lessons from standing at the board and
writing information to becoming more interactive with their students. The most
common response by teachers about what’s working was that the strategies
reinforced every subject, so students were exposed to them in every content area.
Areas to Improve
The principal said that the only thing she thought could be improved was
that the district design was missing the valuable piece of time. Teachers needed
additional time to collaborate, design lessons, and implement the strategies from
CITW.
Leadership team members believed their biggest problem with the district
design was trying to get all the teachers to buy-in. The teacher presenters and
participants tended to be really engaged and excited about moving forward and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
134
sharing the strategies. The marginal teachers were a challenge to get them to
participate. The strategies needed to be better presented to teachers who taught
honors and Advanced Placement (AP) students because they did not see a benefit in
using the strategies for their students. Administrators felt these teachers needed
more specific examples that applied to their curriculum and the students they
taught. A teacher supported this by saying that there was a conflict in using good
instructional strategies with high stakes testing, such as with AP tests. As he got
closer to the test date, good strategies were abandoned for content. Another area
that could be improved was to provide the administrators with training on the
strategies either before or along with teachers. One administrator stated he was
provided the information at a ten minute presentation, given a book to read, and no
further discussion or follow-up was provided. He felt he lacked training and could
not be a resource for teachers at school.
Lead teachers suggested that more discussions had to revolve around the
resources available; and a connection had to be made to other issues related to
education such as literacy and writing across the curriculum. Instead of looking at
the nine strategies in isolation, the next step would be to conceptualize the
strategies. One teacher stated she wanted the nine strategies to be tied into the
California Standards for the Teaching Profession (CSTP) during the workshops.
The majority of teachers felt the five hour training for each strategy was not
sufficient to be expected to share the information with their colleagues who did not
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
135
attend the training. They did not feel that they were prepared enough because they
did not get an opportunity to experiment with the strategies with their own students
nor were they taught the skills to be an effective mentor. One teacher stated that she
felt that teachers who could not attend the district trainings were only given
handouts which did not provide enough information to understand the material to
the degree she understood it. The majority of participants in this study shared this
same concern and stated the flaw was that the district design was not equally
important to the people who attended the district training compared to those
teachers who did not attend.
School Level Survey Items 9-21
The School Level Survey results displayed in Table 7 provide a summative
percentage of thirteen items that were aligned to answer research question 2, “What
school level efforts facilitated the implementation of the district design?” The
respondents’ data in this scale was tested for reliability to determine whether the
scale that contained items 9-21 was consistent. The results indicated that for the
thirteen items where the alpha coefficient was .90, there was a high level of internal
consistency among the respondents. The findings in this scale were grouped into
items that had similar percentage responses to facilitate the organization of
reporting. Items 9, 10, and 21 showed that over 79% of the respondents agreed that
the school supported the implementation of CITW, frequent CITW professional
development sessions were offered, and that the school’s vision was aligned with
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 36
Table 7:
Summative School Level Survey Results by Percentage fo r Research Question 2
(N=66)
RQ2 What School Level Efforts Facilitated the
Implementation of the District Design?
Don't Know
Strongly Disagree
Somewhat Disagree
Somewhat Agree
Strongly Agree
g The school is supportive in implementing the
Classroom Instruction that Works strategies.
5% 3% 5% 20% 67%
The school offers frequent professional development
10 to raise knowledge of the Classroom Instruction that
Works strategies.
3% 3% 14% 39% 41%
j j Systematic efforts to implement the Classroom
Instruction that Works strategies were communicated.
7% 3% 15% 39% 36%
^2 Teacher training and assistance were provided and
conducted in a timely manner.
5% 6% 12% 32% 45%
I have attended professional development training on
13 the Classroom Instruction that Works strategies in the
past six months.
3% 16% 3% 14% 64%
^ I know a great deal about the Classroom Instruction
that Works strategies.
3% 9% 11% 44% 33%
Teachers have opportunities to provide input on how
15 to implement the Classroom Instruction that Works
strategies at my school.
8% 6% 12% 42% 32%
I would like to modify the way my school uses the
16 Classroom Instruction that Works strategies based on
the experiences of my students.
26% 9% 11% 40% 14%
Teachers are involved in the change process and
17 development/selection of suitable materials to support
Classroom Instruction that Works strategies.
14% 12% 14% 45% 15%
jg I am developing mastery of the Classroom
Instruction that Works strategies.
7% 8% 15% 50% 20%
School leadership support was a key element in
19 assisting this site in the implementation of the
Classroom Instruction that Works strategies.
14% 9% 9% 30% 38%
Financial, staff, and material resources are allocated
20 to facilitate the implementation of the Classroom
Instruction that Works strategies.
22% 11% 17% 26% 24%
My school’s vision, mission, and goals are aligned
with the Classroom Instruction that Works strategies.
12% 5% 3% 44% 36%
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 3 7
CITW strategies. In item 10, 17% of the respondents disagreed that the school
offered frequent professional development; and in item 21, 12% did not know if the
schools’ vision aligned with CITW. In items 11-15, approximately 76% of the
respondents indicated that; systemic efforts were used to implement and
communicate CITW; teachers received training within the past six months in a
timely manner; teachers felt they knew a great deal about CITW; and teachers were
given the opportunity to provide input on implementing CITW. Regarding these
same items, approximately 19% of the respondents indicated that they disagreed
with these statements and the remaining 5% indicated they did not know if they
disagreed or agreed. In items 17-19, 60-70% of the respondents agreed that teachers
were involved in the change process, they have developed mastery of CITW, and
that school leadership support has been a key element in implementing CITW at the
school. About 14% of the respondents indicated in both items 17 and 19 that they
did not know, and an average of 22% disagreed with items 17-19. Two items in this
scale received mixed responses. In item 16, respondents were asked if they would
like to modify the way their school used CITW. The majority, 55%, stated they
agreed with this statement, 20% disagreed, and 26% did not know. The second item
that received a variety of responses was item 20 which stated that financial, staff,
and material resources were allocated to facilitate the implementation of CITW.
Half of the respondents stated they agreed, while the other half, 28%, disagreed and
22% did not know.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
138
Research Question #3: To what extent had the district design
been implemented at the school and classroom levels?
To answer this question several instruments in the Case Study Guide were
used to facilitate the data collection. The following instruments were used and
aligned with Conceptual Framework C (CFC) to assist in the data collection
phases: Teacher Interview Guide, Innovation Configuration, School Level Survey,
Teacher Questionnaire (Stages of Concern), and Document Review Guide. The
quantitative data gathered to address the third research question was organized and
presented as follows: School Level Survey for items 22-33, teacher self-report that
reported the percentage of the implementation level using the Innovation
Configuration, and the Teacher Questionnaire (Stages of Concern) that assessed
selected certificated staff members’ stage of concern about the implementation of
the district design. The Teacher Questionnaire section provided more in-depth
information and used graphic organizers which displayed the individual’s raw score
profile, teacher’s highest stage of concern expressed on questionnaire, distribution
of strongest teacher’s concern by number of peak scores and percent of peaks,
highest mean score for each stage, frequency of teachers’ highest concern, and
concluded with the frequency peak and second highest stage of teacher’s concern
regarding number of days of training.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
139
Table 8:
Summative School Level Survey Results by Percentage for Research Question 3
(N=66)
RQ3 To What Extent Had the District Design Been
Implemented at the School and Classroom
Levels?
Don't Know
Strongly Disagree
Somewhat Disagree
Somewhat Agree
Strongly Agree
22
All teachers are committed to the implementation
of the Classroom Instruction that Works strategies.
21% 17% 32% 24% 6%
23
The Classroom Instruction that Works strategies
are consistently used to improve student learning.
12% 8% 30% 33% 17%
24
The Classroom Instruction that Works strategies
are used to help students who perform below grade
level.
11% 6% 12% 38% 33%
25
The Classroom Instruction that Works strategies
assist students to meet state performance
standards.
11% 3% 6% 42% 38%
26
The Classroom Instruction that Works strategies
are used by other departments in my school.
18% 3% 3% 41% 35%
27
I connect the Classroom Instruction that Works
strategies to the curriculum.
8% 2% 15% 33% 42%
1
28
I connect and use the Classroom Instruction that
Works strategies to teach my students the
standards.
6% 10% 15% 27% 42%
29
I use the Classroom Instruction that Works
strategies to help students understand textbook
content.
15% 10% 13% 27% 35%
30
Students are aware of and use the Classroom
Instruction that Works strategies to improve their
academic achievement level.
20% 6% 18% 39% 17%
31
I use the Classroom Instruction that Works
strategies to improve my instruction.
5% 6% 6% 31% 52%
32
I am comfortable using the Classroom Instruction
that Works strategies in my classroom.
6% 3% 10% 36% 45%
33
Using the Classroom Instruction that Works
strategies has improved my teaching.
6% 6% 17% 30% 41%
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 40
School Level Survey Items 22-33
The School Level Survey showed sixty-six respondent results compiled into
Table 8 which provided a summative percentage of twelve items that were aligned
to answer research question 3, “To what extent had the district design been
implemented at the school and classroom levels?” The respondents’ data was tested
for reliability to determine if the scale that contained items 22-33 was consistent.
The results indicated that for the twelve items the alpha coefficient was .92
showing a high level of internal consistency was reported among the respondents.
Items 25, 31, and 32 showed that approximately 80% of the respondents agreed that
CITW assisted students to meet state performance standards and improved their
instruction; plus the respondents felt comfortable using the strategies. Four items,
24, 26, 27, and 33, ranged from 70-76% where respondents agreed that CITW was
used to help students who performed below grade level, that the strategies were
used by other departments, that the respondents connected and used CITW with the
curriculum, and that CITW had improved their teaching. Items 24, 27, and 33 had
almost 20% of the respondents disagree with the statements and the remaining did
not know if they disagreed or agreed. In item 26, 18% of the respondents marked
that they did not know if CITW strategies were used in other departments.
Respondents stated that 62% agreed that they used CITW to help students
understand textbook content, 23% disagreed, and the remaining 15% did not know.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
141
The majority of respondents, 56%, in item 30 indicated that students were aware
and used CITW to improve their academic achievement, 24% disagreed with this
statement, and 20% did not know. Only half of the respondents agreed that CITW
strategies were consistently used to improve student learning, 38% disagreed, and
12% did not know if they disagreed or agreed with this statement. Less than 50%
stated that they adhered to CITW to teach their students the standards, 25%
disagreed, and 6% did not know. The majority of respondents disagreed, 49%, with
item 22 which stated all teachers were committed to the implementation of CITW,
33% agreed, and 21% did not know if they disagreed or agreed with this statement.
Innovation Configuration
The Innovation Configuration was created at the University of Texas
Research and Development Center in Austin, where a number of powerful and
practical tools were developed to measure school change. Innovation Configuration
was a checklist tool used to monitor and gauge implementations using teacher’s
self-assessment reporting (Hall et al., 1998).
The Innovation Configuration self-assessment was completed by six
participant classroom teachers who were chosen by stratified random sampling and
accepted to take part in the formal interviews. Each participant marked their extent
of implementation within each of the nine strategies that were in the district design.
The first strategy, Identifying Similarities and Differences showed that 50% of the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
142
participants had fully implemented the strategy, 50% had partially, and 0% were
just getting started.
The second strategy, Summarizing and Note Taking, indicated that 67% of
the participants had fully implemented the strategy, 33% had partially, and 0%
were just getting started.
The third strategy, Reinforcing Effort and Providing Recognition,
concluded that 67% of the participants had fully implemented the strategy, 16.5%
had partially, and 16.5% were just getting started.
The fourth strategy, Homework and Practice, showed evidence that 83% of
the participants had fully implemented the strategy, 17% had partially, and 0%
were just getting started.
The fifth strategy, Nonlinguistic Representation, showed that 100% of the
participants had fully implemented the strategy, 0% had partially, and 0% were just
getting started.
The sixth strategy, Cooperative Learning, indicated that 33% of the
participants had fully implemented the strategy, 67% had partially, and 0% were
just getting started.
The seventh strategy, Setting Objectives and Providing Feedback, revealed
that 50% of the participants had fully implemented the strategy, 50% had partially,
and 0% were just getting started.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
143
Table 9. Summary o f Innovation Configuration Teacher Self-Report (N=6)
Strategies
Fully Implemented
Partially
Implemented
Just Getting
Started
# % # % # %
Identifying Similarities
& Differences
3 50% 3 50% 0 0%
Summarizing & Note
Taking
4 67% 2 33% 0 0%
Reinforcing Effort &
Providing Recognition
4 67% 1 16.5% 1 16.5%
Homework & Practice 5 83% 1 17% 0 0%
Nonlinguistic
Representations
6 100% 0 0% 0 0%
Cooperative Learning 2 33% 4 67% 0 0%
Setting Objectives &
Providing Feedback
3 50% 3 50% 0 0%
Generating & Testing
Hypotheses
0 0% 3 50% 3 50%
Cues, Questions, &
Advance Organizers
4 67% 2 33% 0 0%
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
14 4
The eighth strategy, Generating and Testing Hypotheses, exhibited that 0%
of the participants had fully implemented the strategy, 50% had partially, and 50%
were just getting started.
The ninth strategy, Cues, Questions, and Advance Organizers, concluded
that 67% of the participants had fully implemented the strategy, 33% had partially,
and 0% were just getting started. Table 9 provides a summary of the Innovation
Configuration of the teachers’ self-report, and Figure 1 provides a graphic
representation of the percentage of teachers regarding their extent of
implementation on each of the nine strategies in the district design of CITW.
The data suggested that Identifying Similarities and Differences were split
evenly between fully and partially implemented. Generating and Testing
Hypotheses was the strategy that half of the teachers marked as just getting started
and the other half as partially implemented. The strategy Nonlinguistic
Representations was the strategy that all six participants marked as fully
implemented. Homework and Practice resulted in the second highest indication that
the respondents had fully implemented the strategy in the classroom. Both Cues,
Questions, & Advance Organizers along with Summarizing and Note Taking had
the same percent who indicated that they had fully implemented the district design
and the same percent indicated they had partially implemented the district design in
their classroom.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
145
Figure 1: Graphic Representation o f the Percentage o f Teacher Responses on
Extent o f Implementation on Classroom Instruction That Works (N=6)
C ues, Q uestions, &
A d v an ce O rganizers
33% :
G enerating & Testing
H ypotheses
Setting Objectives &
P-oviding F eedback
50%
67% C ooperative Learning
Nonlinguistic
R epresentations
17% Homew ork & Practice
Reinforcing Effort &
FVoviding Recognition
16.5% 16.5%
Summarizing & Note
Taking
33%
Identifying Similarities
D ifferences
50% :
33%
50%
67%
67%
6 7 %
83%
100%
50% 50%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
□ Just Getting Started H Partially Implemented ■ Fully Implemented
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 46
Teacher Questionnaire (Stages o f Concern)
The Teacher Questionnaire (Stages of Concern) was administered to the
teachers at Russellville High School in order to assess their stage of concern about
the implementation of the district’s design, CITW. The Stages of Concern (SoC) is
a model of the complex process of change as it occurs during the adoption of
innovations by individuals within formal organizations (Hall et al., 1998). The SoC
questionnaire was used as a diagnostic tool for assessing where the individual
participants of the school were in relation to the district design. A description of the
seven stages is presented in Table 10.
Scoring the Teacher Questionnaire
On the Teacher Questionnaire, respondents indicated the degree to which
each concern was true of them by marking a number next to each statement on a 0
to 7 scale. High numbers indicated a high stage of concern, low numbers indicated
a low stage of concern, and 0 indicated they had a very low stage of concern or they
perceived the question to be irrelevant (Hall et al., 1998). As previously mentioned
in Chapter 3, each item response correlated highly with responses to other items
measuring the same stage which indicated high internal reliability and the alpha
coefficients proved internal consistency for each of the seven stages (Hall et al.,
1998). This researcher also verified participants were consistent in their responses
by asking one question twice, item 11 and 36. The respondents responses on the
Teacher Questionnaire showed that 47% of the respondents answered both of these
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
147
Table 10: Stages o f Concern about the Innovation (Hall et al., 1998, p.7)
0 Awareness Little concern about or involvement with the innovation is
indicated.
1 Informational
2 Personal
3 Management
A general awareness of the innovation and interest in learning
more detail about it is indicated. The person seems to be not
worried about herself/himself in relation to the innovation.
She/he is interested in substantive aspects of the innovation in
a selfless manner such as general characteristics, effects, and
requirements for use.
Individual is uncertain about the demands of the innovation,
her/his inadequacy to meet those demands, and her/his role
with the innovation. This includes analysis of his/her role in
relation to the reward structure of the organization, decision
making, and consideration of potential conflicts with existing
structures or personal commitment. Financial or status
implications of the program for self and colleagues may also
be reflected.
Attention is focused on the processes and tasks of using the
innovation and the best use of information and resources.
Issues related to efficiency, organizing, managing, scheduling,
and time demands are utmost.
4 Consequence
5 Collaboration
6 Refocusing
Attention focuses on impact of the innovation on students in
her/his immediate sphere of influence. The focus is on
relevance of the innovation for students, evaluation of student
outcomes including performance and competencies, and
changes needed to increase student outcomes.
The focus is on coordination and cooperation with others
regarding use of the innovation.
The focus is on exploration of more universal benefits from
the innovation including the possibility of major changes or
replacement with a more powerful alternative. Individual has
definite ideas about alternatives to the proposed or existing
form of the innovation.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
148
Table 11: Statements on Stages o f Concern Questionnaire Arranged by Stage
Item
#
Statement
Stage 0 - Awareness
3 I don’t know what the CITW strategies are.
12 I am not concerned about the CITW strategies.
21 I am completely occupied with other things besides the CITW strategies.
23
Although I am not familiar with the CITW strategies, I am concerned
about aspects of the CITW strategies.
30 At this time, I am not interested in learning about the CITW strategies.
Stage 1 - Informational
6 I have a very limited knowledge of the CITW strategies.
14 I would like to discuss the possibility of using the CITW strategies.
15 I would like to know what resources are available to support the CITW.
26
I would like to know what the use of the CITW strategies will require in
the immediate future.
35
I would like to know how the CITW strategies are better than what we
have had in the past.
Stage 2 - Personal
7
I would like to know how the use of the CITW strategies affects my
classroom, my position at my school, and my professional status.
13
I would like to know who will make the decisions on how I use the CITW
strategies.
17
I would like to know how my teaching skills are supposed to change with
the use of the CITW strategies.
28
I would like to have more information on time and energy commitments
required by the CITW strategies.
33
I would like to know how my role as a teacher will change when I am
using the CITW strategies.
Stage 3 - Management
4
I am concerned about not having enough time to organize myself each day
in relation to using the CITW strategies.
8
I am concerned about conflict between my interests and my
responsibilities when using the CITW strategies.
16
I am concerned about my inability to manage all that the CITW strategies
require.
25
I am concerned about time spent working with nonacademic problems
related to the CITW strategies.
34
Coordination of tasks and people in relation to using the CITW strategies is
taking too much of my time.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
149
Table 11: Statements on Stages o f Concern Questionnaire (continued)
Stage 4 - Consequence
1
I am concerned about students’ attitudes towards the use of the nine
strategies from CITW.
11 I am concerned about how the CITW strategies affect students.
19
I am concerned about evaluating my impact on students in relation to the
CITW strategies.
24 I would like to excite my students about their part in the CITW strategies.
32 I would like to use feedback from students to change the CITW strategies.
36 I am concerned about how the CITW strategies affect students.
Stage 5 - Collaboration
5 I would like to help other faculty in their use of the CITW strategies.
10
I would like to develop working relationships with both our faculty and
outside faculty who use the CITW strategies.
18
I would like to familiarize other departments or persons with the progress
of using the CITW strategies.
27
I would like to coordinate my effort with others to maximize the effects of
the CITW strategies.
29
I would like to know what other faculty are doing as they use the CITW
strategies.
Stage 6 - Refocusing
2
I now know of some other approaches that might work better than the
CITW strategies.
9 I am concerned about revising my lesson plans to use the CITW strategies.
20 I would like to revise the CITW strategies instructional approach.
22
I would like to modify the school’s use of the CITW strategies based on
the experiences of my students.
31
I would like to determine how to supplement, enhance, or replace the
CITW strategies.
Table 12: Item Numbers and Associated Stage o f Concern (Hall et al., 1998, p.26)
Item # SoC Item # SoC Item # SoC Item # SoC
1 4 10 5 19 4 28 2
2 6 11 4 20 6 29 5
3 0 12 0 21 0 30 0
4 3 13 2 22 6 31 6
5 5 14 1 23 0 32 4
6 1 15 1 24 4 33 2
7 2 16 3 25 3 34 3
8 3 17 2 26 1 35 1
9 6 18 5 27 2 36 4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 5 0
items the same and 53% had different responses. Further analysis demonstrated that
28% of the 53% had scored the item within the same scale but at a different level of
degree. So, in actuality 62% of the respondents responded within the same Likert
scale range and 38% chose a level of degree outside each of the scales for items 11
and 36. Each of the seven SoC was represented by five statements. The “raw score”
for each scale was the sum of the responses to the five statements on that scale.
Table 11 shows the item numbers and statements arranged by SoC and Table 12
shows the item numbers and SoC associated with that item.
Individual Raw Score
Each respondent’s raw scale score was calculated by taking the sum of
his/her five statements aligned to the specific stage. Since each question had a 0-7
Likert scale on the questionnaire and five statements within each stage, there was
only a possibility of obtaining a maximum raw scale score total of 35 for each of
the stages. Figure 2 showed that the sixty respondents’ raw scores were widespread
in terms of their stage of concern, which indicated they were concerned about
different things, and uniformity did not exist. Stages 1 and 5 appeared to be higher
than the rest of the stages. Although both of these stages appeared higher, other
respondents were still quite varied within these levels. For example in stage 5, it
was predominately high but there were a lot of other respondents that were lower
on stage 5. Overall, nothing stood out as a predominant stage since there was a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
151
wide distribution at each of these points with some respondents high and low at any
one of these stages.
Figure 2: Profile o f Each Participant’ s Five Item Raw Scale Score Total (N=60)
40
T “ CM co CO CO
C D C D C D C D C D C D
0 3 0 3 0 3 0 3 0 3 0 3
C O C O C O C O C O C O
CO CO C O CO CO CO
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
152
Table 13: Teachers ’ Highest Level o f Concern Expressed on the Stages o f Concern
Questionnaire (N=60)
Stage 0
Awareness
Stage 1
Inform ational
Stage 2
Personal
Stage 3
M anagem ent
Stage 4
C onsequence
Stage 5
C ollaboration
Stage 6
Refocusing
1 1.2 2.8 2 2.8 4 3 3.2
2 2.4 4.4 1.8 3.6 3.8 4.4 4
3 5.6 7 6.6 4.4 2.2 4.4 2.6
4 2.2 1 1 1 3.4 7 1
5 2.2 4.2 3.4 2.4 2.6 6.2 1.4
6 3.4 6 5 3.8 4.2 3.6 4.6
7 3 1.2 1 1 1 1.2 1
8 2.2 3.6
_
2.4 5 5.6 2
9 .6 1 1.8 2.2 2.8 3.2 5.4
10 2.6 5.6 4.6 4.4 4 4.6 4
1 1 .6 ^ 3.6' .8 2.2 3.6 I 4.6 .6
12 3 2.6 4 .8 2.4 1.8 1.4
13 1 2.8 2.8 1.6 3 4.2 1.2
14 1.2
__
2.6 3.2 2.6 ' 4 2.8
15 1.8 3 3.6 1.8 .6 2.2 .6
16 2 1.8 3 1 3.6 5 2.2
17 2.6 1.6 1.2 3.4 2.6 3.8 1.8
18 4.8 3.4 1.4 2.8 1.6 r 1 4.2
19 1.4 4 3.8 4 3.8 3.2 2.4
20 1 3.2 n 3 2 2.6 3.2 1
2 1 .8 1.4 .6 2 3.2 2.8 1
22 2 2.6 3.2 2.4 3.6 6 . 5.2
23 4.6 2 1 .6 .6 1.8 1.6
24 2.4 4.8 4.6 1.4 2.4 3.8 1.8
25 1.6 4 3.4 3 3.6 3 2.8
26 .8 3 4.2 1.4 3 1.8 2.6
27 .2 2.8 4.2 0 0 0 0
28 1 2 2 1 6.6 4.8 3.8
29 1.6 3.8 4.2 2.8 3.2 3.4 2.8
30 6 3.4 2.4 .8 1 1 1
31 .8 1.4 .6 2 3.2 2.8 1
32 2.6 1.6 1.2 3.4 2.6 3.8 1.8
33 1 4.2 1 3.4 2.8 2.2 1.6
34 3.4 2.8 1 1 1 1.6 1
35 1.4 4.2 6.4 5.8 4.4 3.4 3.6
36 1.2 3.2 2.6 2 3.2 1.4 2.6
37 4 3.4 3.8 4.8 4 2.2 4.2
38 3 2.2 .6 2.4 4.6 .6 2
39 1.8 1.6 .8 1 1.6 2 .8
40 1 3.6 2.6 1.6 4.4 3 2
41 2.2 4.6 5.2 3.8 2.8 3.6 5
42 3 4.8 4.8 ■ 8 3.4 4.4 .2
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
153
Table 13: Teachers ’ Highest Level o f Concern Expressed on the Stages o f Concern
Questionnaire (Continued)
Stage 0
Awareness
Stage 1
Inform ational
Stage 2
Personal
Stage 3
M anagem ent
Stage 4
C onsequence
Stage 5
C ollaboration
Stage 6
R efocusing
43 3.4 2 3.4 4.2 2.8 3.2 1.4
44 2.4 2.6 2.2 .4 .2 .6 .2
45 2.4 2.4 2.8 2.8 4 2.2 2.4
46 1 .6 .6 1 .6 .6 1
47 1.6 1.4 1.6 1 1.8 1 1.2
48 2 4.8 5.8 4.8 5 4.6 3.6
49 1.4 4 3.8 4 3.8 3.2 2.4
50 2.2 2.2 2.2 1.4 2.2 2 1.4
5 1 .8 1.4 3 2.6 3 2.6 2.8
52 1.8 2.2 1.2 1 1.6 4.4 1.6
53 2 1.8 3 1 3.6 5 2.2
54 2.4 2.6 2 1.6 2 1.4 1.6
55 1 3.4 2.2 2 5.2 5.4 4.2
56 3.2 4.6 4.8 3 3.8 5.2 3.2
57 2 4.2 5.2 5.8 4.4 3.8 5.4
58 1.4 2.8 3.4 2.4 3.2 3.2 3
59 2 4 4.6 1 2.8 6.2 1.8
60 3.2 3.8 3.2 1 1.4 3.2 2.4
Individual Mean Responses
The mean raw score for each individual respondent, selected certificated
staff, were charted in Table 13. As mentioned previously, the Likert scale ranged
from 0-7. The mean response for that particular stage was computed for each of the
sixty respondents. Looking at respondent 1 at stage 0, it was noted that all five of
his/her statements totaled a raw score of 6 which was then averaged. The results
suggested that this respondent’s score of 1.2 indicated a low degree of concern or
intensity in stage 0, whereas in stage 4 he/she showed a higher degree of concern
with a score of 4. The chart provided an individual profile for each respondent, and
the highlighted scores indicated the highest stage scores for each individual.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 54
The stages that most frequently received the highest concern across stages 0
through 6 were stage 1, informational, and stage 5, collaboration. In the interviews
with the six teacher participants, they shared the same areas of concern as teachers
were split between those wanting more training and those who were trying to figure
out how to share what they knew about CITW with members within their
department.
The peak, highest, scores were summarized in Table 14. The respondents’
distribution indicated their peak stage was as follows: 9% percent were in the
awareness stage, 24% were in the informational stage, 15% were in the personal
stage, 9% were in the management stage, 14% were in the consequence stage, 25%
were in the collaboration stage, and 4% were in the refocusing stage. The
collaboration stage had the highest percent with 25% of the respondents indicating
that they were focusing on coordinating and cooperating with others regarding the
use of the district design. Very close behind was the informational stage with 24%
of the respondents indicating they had a general awareness and were interested in
learning more about the district design. The lowest peak was stage 6, the refocusing
stage, with 4% of the respondents stating that their focus was on exploring the
benefits with the district design, taking into account the possibilities of major
changes. The second lowest peaks were stage 0, the awareness stage, and stage 3,
the management stage; where 8% of the respondents considered these stages as
their peak area of concern. The awareness stage was not an area of concern
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
155
throughout the interviews; and it appeared everyone was at least aware of the nine
strategies. Although the management stage focused on issues related to efficiency,
organizing, managing, scheduling, and time demands, the researcher found it
surprising that this stage was low, since in every interview the respondents
indicated that there was a lack of time provided for classroom teachers to
implement the strategies after the trainings were provided.
The peak scores were graphed to display the frequency distributions. The
shape of the frequency resulted in a symmetric distribution with a bimodal
distribution, two relative peaks (Figure 3).
Table 14: Distribution o f Strongest Teacher Concerns (N=60)
Stage of Concern
# of
Peak
Scores
%of
Peaks
Teachers' Highest Mean Score for Each Stage of Concern
0 Awareness 6 9% 1 2.2 3 3.4 4.6 4.8
1 Informational 17 24% 2.2 2.6(2 )3.2ra 3.4 3.8 4® 4.2 4.4 4.8(2 > 5.6 6 7
2 Personal 11 15% 2.2 3.4 3.6 4(2) 4.20) 4.8 5.2 5.8 6.4
3 Management 6 9% 1 4(2) 4.2 4.8 5.8
4 Consequence 10 14%
1.8 2.2 3.2(3) 4( 2 ) 4 4 4 6 6.6
5 Collaboration 18 25%
2 3.2 3.8(2) 4 4.2 4.4(2 ) 4.6 5(2 ) 5.2 5.4 5.6 6 6.2(2 ) 7
6 Refocusing 3 4% 1 2.8 5.4
Note: Number of Peak Scores totals more than 60 as some teachers had multiple peak scores
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 56
Figure 3: Frequency o f Teachers ’ Highest Stage o f Concern (N = 60)
20
£
® 15 -
. c
O
0 3
0 3
H
M —
O
10 -
C D
- O
e
z
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 Stage 6 Stage 0
Frequency Compared to Days o f Training
The frequency peak and the second highest stage of concern were plotted
against the number of days of training received by the respondents. The
demographic information was obtained from the School Level Survey which
helped to determine if a relationship existed between the stage of concern and the
number of trainings attended. Figure 4 displayed five graphs with frequency of
peak and second highest stage of concern for teachers in relation to days of
training. Some respondents had multiple peaks which caused the graphs to have
more number of responses than the number of respondents. The first graph showed
that thirteen teachers stated they had more than five days of training, and their
results showed that the peak occurred in stage 2. Their second highest peaks were
stages 0, 1, and 6. The second graph showed twenty-two teachers received three to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
157
Figure 4: Frequency o f Peak and Second Highest Stage o f Concern fo r Teachers in
Relation to Days o f Training
Teachers with More than 5 Days o f Training (N = 13)
0
- C
o
T O
0
0
X I
E
3
— 0
Stage 0 Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 Stage 6
^H^™ »Peak 1 2 3 2 2 2 0
• 2nd
3 3 2 1 2 1 3
Teachers with 3-5 Days o f Training (N = 22)
10 «
0
X
o
0
0
I —
0
8
6 ^
4
2
§ 0
z
Stage 0 Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 Stage 6
“ ■ ^ ^ “ Peak 3 9 6 1 3 6 0
3 4 5 3 3 6 4
Teachers with 1-2 Days o f Training (N = 18)
8 n
5 2
0
sz
o
0 3
0
0
. O
E
4 -
2 -
0
Stage 0 Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 Stage 6
^ ^ ^ “ Peak 1 3 3 0 4 8 1
™«»-»:<-"2nd 1 6 2 0 6 5 2
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
158
Figure 4: Frequency o f Peak and Second Highest Stage o f Concern fo r Teachers in
Relation to Days o f Training (continued)
Teachers with Less than 1 Day o f Training (N = 2)
3 n
S2
< u
O
( 0
0
2 -
0
n
£
D
z
0 ^
Stage 0
T ----
Stage 1 Stage 2 Stage 3 Stage 4
------ * -----
Stage 5 Stage 6
^ ^ ^ “ Peak 0 1 0 0 0 1 0
■ 2nd 0 0 1 1 2 0 0
Teachers with 0 Days o f Training (N = 5)
o
x :
o
C O
0
0
. O
£ * / ■
0
Stage 0
................
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
-------------
Stage 6
■ ■ Peak 1 1 0 1 2 0 1
«».««•».....2nd 1 0 3 2 0 1 0
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 59
five days of training, and their peak was in stage 1 and second highest peak was
stage 5. The third graph had eighteen teachers who stated they had attended one to
two trainings, demonstrating their peak at stage 5. Their second highest peaks were
stage 1 and 4. The fourth graph had two teachers state they had received less than
one day of training, and they had bimodal peaks in stages 1 and 5. Their second
highest peak was in stage 4. The final graph had five teachers state they had zero
training and their peak was in stage 4. Their second highest peak was in stage 2.
The stages of concern in relation to days of training did not show any correlation.
Research Question #4: How effective were the district design and implementation
strategies at the school and the classroom levels?
To answer the last research question the Case Study Guide was once again
used to facilitate the collection of data. The following instruments were used and
aligned with Conceptual Framework D (CFD) to assist in the data collection phases:
The Principal Interview Guide, Leadership Team Member Interview Guide, Lead
Teacher Interview Guide, Teacher Interview Guide, School Level Survey, and
Document Review Guide. The qualitative and quantitative data gathered to address
the fourth research question was organized and presented as follows: teachers’
response and attitude toward district design, school assessment of district design,
helpfulness of professional development, profile of implementation, positive
implementation strategies, negative implementation strategies, and concluded with
responses to the School Level Survey for items 34-44.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 6 0
Teachers ’ Response and Attitude toward District Design
The principal thought initially that teachers perceived the district design as
another vehicle used to dictate to schools what and how to teach, which created
resistance. However, after the first district training, teachers changed their mind and
found the strategies to be relevant instructional tools.
The leadership team members stated that teachers of Russellville High
School first heard about the district design in the fall of 2002, and their response
was positive and favorable. Teachers accepted the district design because they felt
that teachers teaching teachers (trainer-of-trainers model) was an effective way to
run a workshop. These trainers were able to offer experiences they had encountered
in the classroom as they applied the strategies of the district design, and shared
strengths as well as weaknesses in the presentations. After the first district training
session, teachers explained to their colleagues that the trainings were not facilitated
by outside consultants that exerted their opinions, but instead, were done by their
own district teachers who presented the information in a collegial manner. Teachers
perceived the trainings as useful, helpful, and effective. They felt comfortable with
the fact that the strategies presented were already used by teachers and validated by
research.
Lead teachers agreed that the teachers’ responses to the district design were
positive and well-received among the staff at their school because it was viewed as
practical. Initially, lead teachers were unaware of CITW as a specific plan other
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
161
than it was going to take place over three years, three strategies taught per year, and
nine strategies in total. Three of the lead teachers at this school had been invited to
be presenters and knew for a fact that the feedback from the actual workshops was
positive. The presenters handed out evaluation sheets after the workshop
presentations, and the participants wrote many compliments. Some of the
comments on the evaluation sheets were that they enjoyed having teachers
throughout the district provide the training, the strategies were easy to implement,
they liked the specific examples provided at the training, and they liked the way
lessons were modeled. One lead teacher shared that she was excited after the
trainings to get back to her classroom and experiment with the strategies modeled
at the district workshop.
The teachers interviewed were receptive and stated that they thought that
CITW was a good idea because it helped them to be effective teachers. One teacher
stated that her initial perception was that district trainings were just another
obligation, but when she attended the workshop she was very pleased that it was
run the way she wanted a classroom to be run instead of the usual workshops where
presenters stood up and talked at the teachers instead of with them. The majority of
teachers stated that the CITW workshops were timed appropriately for teachers to
absorb the strategies and then adapt them to help students learn. The best part for
one teacher was the sharing of ideas during trainings, since she was new to the
teaching profession and loved learning other teaching techniques.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
162
The principal, leadership team members, lead teachers, and teachers
perceived veteran teachers and new teachers attitudes to be positive, accepting, and
enthusiastic after the first training session held at the district. They viewed the
design as having value to individual teachers and schoolwide. They viewed the
strategies within the district design to be very effective and useful tools to help
improve instructional practice.
School Assessment o f District Design
The principal believed teachers assessed the success of the plan through
student performance related to projects, quizzes, and tests. She shared an example
from one teacher who did everything in his power to get his students to understand
information he was trying to teach. He attended one district training session on
CITW and implemented the strategy Nonlinguistic Representations and noticed
results in his students’ work immediately.
Leadership team members observed teachers sharing and reflecting on the
strategies at monthly department meetings. One release day was used by teachers to
work on lessons collaboratively within their departments, but there was no formal
way to assess the success of the strategies.
Lead teachers indicated they assessed the success of the strategies by how
students were improving on quizzes, tests, projects, and homework. One lead
teacher explained that students in their Freshman Academy were taking notes more
effectively after the strategies were implemented into the program. On the other
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
163
hand, another lead teacher did not believe teachers did much to assess the success
of the plan. He believed they either used the strategies in their class or did not.
Another lead teacher could not conclude whether her colleagues were aware that
these strategies were necessarily changing the culture of the school and district.
Lead teachers agreed with administrators that a missing element in the district
design was the lack of a formal evaluation system in place.
Was Professional Development Helpful?
The principal thought that professional development was helpful to her
school because she observed instructional lessons changing. As she conducted her
weekly visits into classrooms, she observed the strategies implemented which
demonstrated the training sessions were effective. The strategies were a constant
topic of discussion during leadership council and in electronic mail. It was kept in
the forefront of teachers’ minds.
Teachers were satisfied with the professional development that was offered
to them and believed it was helpful in improving their instruction. They enjoyed the
fact that they were able to meet with other curriculum areas to work collaboratively
in implementing the strategies across all discipline areas. The only negative aspect
that all teachers agreed on was that the workshops required them to get substitutes
for their classes and took them away from their students. Teachers who did not
attend the district trainings had an opportunity to learn the strategies at staff
development meetings at the school. These training at the school were facilitated by
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
164
teachers who attended the district trainings and used an overhead projector to share
ideas for lessons. One teacher stated that the staff development at school was
enlightening because she received information from different departments on how
they used the strategies to teach diverse learners such as special education and
English Language Learner (ELL) students.
Profile o f Implementation
The principal saw implementation in the classrooms on her weekly
classroom visitations. She noticed that teachers preferred to use the strategy of their
choice, which tended to be visual organizers and collaboration among colleagues,
more often than higher level thinking strategies. It appeared every classroom used
Nonlinguistic Representations but she rarely recalled seeing the strategy Generating
& Testing Hypotheses which was considered a higher thinking strategy. Overall,
she believed about sixty percent of her teachers embraced the CITW strategies and
one hundred percent used them at one time or another throughout the year.
The leadership team members conducted weekly visits, “Every Classroom
Every Week” and looked for strategies from the district design to be used in the
instructional lessons. They observed that classrooms which had implemented the
district design displayed visual or auditory examples. A variety of evidence was
demonstrated that was not random but intentional because teachers posted a daily
agenda on the board. Students were constantly engaged in the lessons as they
physically took notes, filled out diagrams, and self-assessed themselves using
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
165
rubrics. The majority of classrooms had structured notes or some sort of note taking
process which demonstrated that the instruction was not solely auditory learning.
There was definite teacher-student interaction in some form or another. If on a
particular day an administrator conducted a visit and they did not hear or observe a
CITW strategy implemented in the lesson, the classroom walls had evidence of
strategies used in previous lessons. The strategies most often seen were Note
Taking strategies, Nonlinguistic Representations (graphic organizers), Cooperative
Learning, and Reinforcement of Homework and Practice.
Lead teachers stated that the implementation in the classroom truly
depended on the teacher. For example a science teacher focused on the strategies
Note Taking during instruction and Cooperative Learning on laboratory days. In an
English class, Summarization and Nonlinguistic Representations were the main
strategies teachers had implemented and modeled in lessons. One teacher stated
that the nine strategies were part of their general practice, and it was nice to know
that research supported them. The math department had a lesson plan with
strategies from CITW incorporated into every topic from Algebra 1 classes to
lower. Every lesson had a scripted example of student work and specific strategies
to use. This department also created a common calendar throughout the department
that described the scope and sequence of material that had to be covered and the
CITW strategy to use with the instructional lesson. A social science lead teacher
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 66
stated that he saw the instructional lessons differently than what he recalled five
years ago due to the implementation of CITW.
Teachers perceived the implementation of the district design by using
strategies not only in the main lesson but before the lesson was presented and
during closure to ensure students increased their ability to retain information. The
lessons began with warm-up activities, visuals to introduce the lessons,
expectations modeled by the teachers, and opportunities for students to practice.
During these phases, strategies were used to assist students with understanding the
structure of the lesson and why it was applicable to them. One teacher gave an
example of a social science class that covered material on the different branches in
congress. She saw her colleague use comparison diagrams rather than reading it
from a book as he had done in the past. Another teacher stated she did a lot of
summarizing and paraphrasing because it helped her students learn vocabulary, and
the visual component allowed more of her students to engage in the lessons. In
science, students used diagrams, comparison charts, note taking, and graphic
organizers during instruction. A math teacher stated that she saw graphic
organizers, collaborative groups, and manipulatives used throughout her
department. Another teacher said that there was evidence that CITW was
implemented schoolwide because it was the main focus of their school action plan,
and the strategies were used to improve reading, writing, and math across all
disciplines at Russellville High School.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
167
Positive Implementation Strategies
The principal stated that the implementation of the district design allowed
for uniform training for all district schools in the use of the best practiced research
strategies which support instruction and improve student success. The common
districtwide professional development plan was clearly articulated and created a
support plan that helped schools address strategies that engaged students as active
learners. This principal stated that her school had a high population of ELL
students and that the standard “stand and deliver” methodologies were not
successful in the past or present. The new strategies provided hands-on
manipulative opportunities and created active learners that could be responsible for
their learning. The principal shared that the most positive thing about CITW was
that teachers could choose the strategies they believed fit best with their lessons and
curriculum. She believed that the research-based strategies had a direct correlation
in their improved longitudinal state test scores. She stated that she was fortunate to
have had three of her school’s teachers as district presenters because it empowered
them to become expert trainers. She believed part of the success was that the
presenters made the strategies practical for teachers. Overall, this district design
allowed for a constant dialogue in the staff lounge, department meetings, leadership
meetings, and on staff development days. These strategies allowed her staff to look
at teaching in different ways and to increase department collaboration.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
168
Leadership team members stated that the district design established a
foundation of high expectations in every classroom and outcomes resulted in
improved state test scores. This design involved all teachers and provided them
with effective tools to work with students who may not be ready or willing to leam.
They felt that CITW also built a stronger collaboration among departments to
dialogue and create lessons that incorporated instructional strategies.
Teachers shared their positive perspectives resulting from the
implementation of CITW. They liked that their staff development focused on the
strategies and that their principal was committed to sharing the strategies with the
entire staff. Teachers had the opportunity to obtain training on CITW through a
series of mini-lessons, like the district trainings, to make sure all of the staff was
exposed to specific strategies and examples. The implementation provided
opportunities for all teachers in every discipline area to practice and dialogue as a
school, in a department, or among individual colleagues on the improvement of
instructional practices. For example, homework was always given to students; but
remembering the importance of it and the way to do it effectively became a
priority. One teacher stated that throughout her eleven years of teaching experience
she has come to realize that these strategies were the key to success. It made
teachers throughout the campus realize that a good teacher identified what worked
and what was needed to be done to have a successful classroom using the strategies
as tools. Teachers appreciated getting a lot of new information and working with
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
169
people from different departments, which in the past did not happen. By the end of
year three, the majority of teachers at the school came to the realization that these
strategies were a standard part of their school’s teaching curriculum. It took over
three years to establish a moderate increase in buy-in.
Negative Implementation Strategies
The principal felt that the district design was clearly articulated, but thought
it was not clearly implemented at the schools. The plan was missing the element of
time, which would allow teachers to implement the strategies. Teachers attended
the trainings at the district, but were not provided with enough time to implement
the strategies, and there was a lack of follow-up training. Another thing she
observed was that her teachers were more often using the visual organizer strategies
over the higher level thinking strategies.
The leadership team members felt there was no formal assessment that
made teachers accountable for implementing the strategies. They also felt that as
administrators they needed to receive formal CITW training to be a resource and
asset for teachers.
Teachers stated that in the first year of implementation they did not know
what the plan was or what the workshops were going to be about. This created
anxiety and frustration for some teachers who were asked to attend the trainings
with limited knowledge about the district design. At these staff development
trainings, one teacher recalled a group of veteran teachers complaining about how
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
170
they had been working with these strategies for over 20 years and that they did not
need to be listening to the presentation. This teacher believed that the training at the
district was a lot more in-depth than the training at the school which limited the
teachers from obtaining a true sense of its purpose and intent. Other teachers stated
that they did not like leaving the classroom for an entire day for professional
development. They felt their time was constricted due to all of the state testing
requirements and the extensive content they were obligated to cover. They
preferred after-school trainings or summer workshops. Some teachers also felt that
they needed to be provided with a more structured plan on how to teach their
colleagues, other teachers, on campus and how CITW could be formally monitored
to provide data and evidence of its effectiveness to convince the teachers who
resisted buy-in. Another teacher stated that by year three the remaining strategies
were more difficult to implement and that she wanted an opportunity to meet with
other schools by content area to work collaboratively to develop lesson plans
aligned to the standards using CITW.
School Level Survey Items 34-44
The School Level Survey was displayed in Table 15 which provided a
summative percentage of the items that were aligned to answer research question
#4, “How effective were the district design and implementation strategies at the
school and classroom levels? The respondents’ data was tested for reliability to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
171
determine if the scale that contained items 34-44 was consistent. The results
indicated that for the eleven items where the alpha coefficient was .93, there was a
high level of internal consistency among the respondents. Item 34 had 82% and
item 39 had 83% of the respondents agree that the CITW strategies were effective
in improving student learning and that on-going support was provided at the school.
Approximately 12% did not know and under 8% disagreed. Item 43 addressed
CITW as having helped students to be successful which resulted in 70% of the
respondents who agreed with this statement, 16% disagreed, and 14% did not
know. Items 37 and 38 averaged 67% of the respondents who agreed that CITW
was not expensive to implement and that it was presented in a timely manner. In
item 37, 18% did not know if the strategies were expensive to implement and in
item 38, 25% disagreed that CITW was presented to teachers in a timely manner. In
three items, which included items 35, 36, and 40, only a slight majority agreed,
53%, with the statements that their instructional delivery had changed, that CITW
increased student motivation toward learning, and that on-going support was
provided at their school. Item 35 had 42% of the respondents disagree that their
instructional delivery had changed due to using CITW. Items 36 and 40 each had
28% and 29%, respectively, disagree that CITW increased student motivation
toward learning and that on-going support was provided at their school. The
remaining 20% and 15% did not know if they disagreed or agreed with the
statements. The item that received the least agreed responses on the scale was item
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
172
Table 15:
Summative School Level Survey Results by Percentage for Research Question 4
(N=66)
CD
£
o >
CD
CD
L .
0 3
(0
CD
CD
L .
CD
CD
RQ4 How Effective Were the District Design and
Implementation Strategies at the School and
Classroom Levels?
D on't K nov
CO
CO
b
> >
O )
c
o
< / >
b
ro
. c
5
CD
c
<
CD
- C
CD
E
$
> %
o >
c
2
C O
O
C O
C O
34
The Classroom Instruction that Works strategies
are effective in improving student learning.
12% 0% 5% 39% 44%
35
My instructional delivery has changed by using the
Classroom Instruction that Works strategies.
6% 16% 26% 31% 21%
36
The Classroom Instruction that Works strategies
have increased student motivation toward learning.
20% 8% 20% 46% 6%
37
The Classroom Instruction that Works strategies
are not expensive to implement.
18% 5% 9% 20% 48%
38
The Classroom Instruction that Works strategies
were presented to teachers in a timely manner.
10% 8% 17% 30% 35%
39
Training provided to teachers on the Classroom
Instruction that Works strategies was useful.
10% 0% 8% 47% 35%
40
On-going support is provided at my school on the
Classroom Instruction that Works strategies.
15% 11% 18% 27% 29%
41
Feedback is provided to teachers on their use of the
Classroom Instruction that Works strategies.
20% 21% 18% 30% 11%
42
The Classroom Instruction that Works strategies
are worth keeping.
9% 0% 3% 27% 61%
43
The Classroom Instruction that Works strategies
have helped students to be successful in my class.
14% 0% 16% 40% 30%
44
The Classroom Instruction that Works strategies
are feasible in improving teaching and learning.
9% 0% 2% 41% 48%
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
173
41 with only 41%. The percentage of respondents that disagreed with the statement
that feedback was provided to teachers on their use of CITW was 39%, and the
remaining 20% did not know if they disagreed or agreed.
Discussion
Los Coyotes District Design
The political climate and culture of an organization affects the acceptance
of new policies and programs (Marsh, 2002). The status of student performance in
the United States has been a growing area of concern for the public over the past
few decades, which led elected officials and educators to believe educational
reform was needed (Elmore, 1997). The board of trustees and superintendent of
Los Coyotes School District recognized the need for reform and took professional
development down a path that had not been part of the district culture. The board of
trustees realized that federal mandates in NCLB, state accountability requirements,
and local community pressures were only going to become more strenuous as high
stakes testing became a priority. The board of trustees provided the vision to
achieve instructional excellence, the superintendent introduced Classroom
Instruction that Works (CITW) as the instructional model, and the assistant
superintendent of education created the plan to implement CITW throughout the
district.
Many researchers have argued that the quality and effectiveness of teachers
outweighed the demographic and other school-related factors (Darling-Hammond,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 7 4
1999; Rosenshine & Stevens, 1986; Rowan, Correnti, & Miller, 2002). Researchers
found that student performance was positively affected by the characteristics of
teachers: i.e., quality teacher training, content knowledge, use of teaching
strategies, and patterns of context coverage (Rowan et al., 2002). Qualitative and
quantitative analysis suggested that increased investments in the quality of teachers
were positively related to improvements in students’ performance (Darling-
Hammond, 1999). Successful districts such as District #2 in New York, New
Haven, and San Diego all had vision statements that viewed teachers, teacher
learning, and instructional goals at the core of their renewal strategy (Hightower et
al., 2002) as did Los Coyotes School District. In the past, this district acted as a
support resource while schools chose individual reform efforts. But, when districts
were also held accountable to meet state performance levels, Los Coyotes High
School District had to change its role to enhance the quality of teaching and
learning. Vertical and horizontal aspects of a district’s organization and governance
structure influence the district’s design in connection with the level of collaboration
with the district office and the school sites (Marsh, 2002). Thus, the top-down
approach was the most logical and timely way of introducing the district design
according to the former superintendent. Within a few months the district’s top-
down approach created awareness and aligned all of its comprehensive schools
with research-based instructional strategies to improve teaching and learning.
According to Goldring and Hallinger (1992) the effect of the district’s
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
175
organizational structure on schools to improve teaching and learning was an
important element to consider when implementing reform. Although, the type of
district organizational structure, decentralized or centralized, did not appear to
matter as long as it was not fragmented (Marsh, 2002).
The assistant superintendent took the initiative to conduct a survey on
certificated staff needs during the spring 2002, which proved to be timely since the
findings showed that teachers were seeking training in instructional strategies. The
major task at hand was to determine the most effective way to present, implement,
and sustain CITW in schools, in a district where its schools had a strong culture of
working independently on staff development. According to Reeves (1998) the
connection between the district and schools affects the implementation of district-
led reforms and can be overwhelming for a district and frustrating for a school site
as school staff questions the purpose of the reform and its affect on their personal
and professional lives. The assistant superintendent’s plan was to work with the
principals to obtain names of teachers at their schools who were perceived as good
classroom teachers, respected by their colleagues, and exhibited the potential to be
excellent presenters. This gave the principals an opportunity to showcase their top
teachers districtwide.
Tucker and Codding (1998) stated, “You need to take the strategies you
have selected and turn them into an action plan that specifies what is going to
happen, who is going to do what, how they will do it, how much it will cost, and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 76
when it will be done” (p. 165). The nine strategies from CITW were divided into
three implementation phases and devised to ensure that the order of the strategies
introduced created initial buy-in with teachers who attended the district trainings.
Although, the technique was initially perceived as a positive district strategy, the
principal at Russellville High School began to experience negative effects. The
staff embraced the lower level strategies such as Nonlinguistic Representations
(graphic organizers) into their curriculum, but rarely used higher level thinking
strategies such as Generating & Testing Hypotheses. This was frustrating for the
principal who was trying to achieve greater academic success for students.
The decision to have in-house trainers, a trainer-of-trainers model, was an
extremely successful strategy employed by the district. Research showed that direct
instructional methods that were clear, sequenced, and provided feedback could
bring positive results if used more often in a collaborative manner (Schmoker,
1999; Wang, Haertel, & Walberg, 1993; Wiggins & McTighe, 1998). The
participants in the study expressed their appreciation in interviews and on district
evaluation forms that they liked the presenters because they were actual teachers
who presented clear instructional methods, had used the strategies with students,
served as peer resources, and modeled examples which were relevant to teachers.
The presenters at the trainings created awareness, demonstrated tools, and provided
resources that showed that there were other ways to teach students. District #2,
New York, used the trainer-of-trainers model where staff developers coached other
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
M l
teachers and principals on pedagogical techniques aligned with the reform initiative
and research-based strategies to achieve instructional reform which proved to be
successful by raising students reading scores (Hightower et al., 2002). Participants
in the Russellville High School study agreed that the workshops trained them to
reflect on their teaching and gave them additional tools to improve the instructional
delivery of lessons, which was the intent of the district design by the former and
current superintendents. The various instrumentations used to collect the data
showed that the intent of CITW was evident at Russellville High School.
The top-down approach required extensive communication with all
stakeholders. The principal and teachers who attended the district trainings served
as the link between the district and the school. The majority of participants in the
study who had attended district trainings indicated they knew about CITW but did
not know that it was the district design. They stated they were aware of CITW
because of the district trainings and knew CITW was a priority because it was
presented at school staff development days, leadership meetings, and department
meetings. Dufour (1991) discussed that communication can serve as the link
between organizational cause and the individual’s commitment to it and people
within the school are entitled to know why they are being asked to do certain things
and their role in the process. The School Level Survey substantiated the interviews,
and results demonstrated that the vast majority of respondents indicated they were
aware of CITW but they were not aware that district leaders wanted teachers to use
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
178
CITW. These two items showed that teachers were aware of CITW but did not
have knowledge about the connection of CITW as the district design. The findings
suggested that teachers were aware of the importance of CITW due to the
principal’s emphasis on the strategies, her support in the classroom, and her priority
to have CITW training at the school’s staff development. Teachers lacked the
necessary background information to understand the relationship between the
district trainings and the districtwide professional development plan, the purpose of
the improvement efforts, and the impact of the reform on stakeholders.
Consistent attendance at the district trainings gave participants an initial
introduction to the strategies. The training of a particular strategy provided teachers
with the opportunity to practice with colleagues. Stigler & Hiebert (1999)
concluded, after studying Japanese teachers’ continuous professional development,
that teachers and students learn best in collaborative settings. At the district
trainings they were able to design lessons together that would implement the
strategies in their classrooms, and then shared their experiences at the next session.
This structure was effective for the teachers who attended district trainings. The
lead teachers at Russellville High School obtained the most success because they
were required to attend all CITW trainings at the district. Problems resulted for the
regular classroom teachers because they rotated their attendance at the district
trainings and many times teachers would miss the first session’s information, which
made them unprepared to share their success or challenges on implementing the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
179
prior session’s strategies. The district design lacked a structured method of
ensuring consistent attendance at the district trainings. This was attributed to
allowing the schools to determine who attended the district trainings. All the
participants who attended the trainings stated that working with teachers from a
variety of discipline areas at the trainings helped them to grow professionally, to
build a stronger rapport with one another, and to collaborate in determining how to
help students achieve greater success in all subject areas. The district trainings
enabled the participants to practice the strategies in their own classroom. Formal
training for the leadership team members was lacking, diminishing their potential
to be an asset at the school for their teachers.
The district design lacked a formal assessment that was consistent and
reliable. School administrators were required to submit monthly summative reports
that reported the percentage of teachers using CITW. The principal attended all the
CITW trainings at the district but all three of her leadership team members had
either limited or no training, despite the fact that they were also responsible for
conducting weekly assessments. As such, the data collected by the district over the
past three years was questioned as to its reliability. In District #2, New York,
professional development was used as a management strategy for teachers and
principals. Teachers were taught to share and reflect on lessons while principals
were taught what to look for in a lesson (Stein, 2003).
The lack of formal training for the leadership team members at Los Coyotes
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8 0
School District may also explain the lack of feedback to teachers by administrators.
These administrators were not in a position to provide feedback if they were not
trained on what to look for or prepared to provide guidance to teachers on the
strategies. Marsh (2002) stated that the districts capacity is influenced by factors
such as trained and knowledgeable leadership, which contributes to its ability to
handle change. The annual surveys, distributed by the principal at the end of each
school year, determined which strategies were used by her staff and provided more
reliable data to analyze the percentage of teachers using the nine strategies.
Russellville’ s Efforts to Implement District Design
This section provides a discussion on the roles of district office and school
personnel, procedures taken to monitor the district design, timeline used, training of
teachers at the school, support activities for teachers at the school, things that
worked while implementing the district design, and the areas that could be
improved. Data from a variety of instruments were triangulated to provide evidence
for the findings that led to this discussion.
Individual’s roles need to be clearly communicated when using a top-down
approach. Marsh et al. (2004), stated that greater demands on district administrators
are placed to provide teachers and school administrators with skills, knowledge and
resources needed in order to help all students meet high academic standards.
District personnel need to ensure that the schools understand and know what
everyone’s role will be during the reform effort in order to facilitate
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
181
implementation and move towards sustainability. The principal at Russellville High
School clearly understood her role as the link between the district and her school.
Principals’ actions taken to instigate reform can serve to establish effective
communication with staff members. Some principals used the district’s shared
common language with staff to familiarize them with valuable ideas, and allowed
them to provide feedback on districtwide initiatives and strategies (Marsh et al.,
2004). The principal at Russellville High School made it her priority to discuss
CITW in all formal meetings conducted at school. She focused staff development
on the strategies for three consecutive years and promoted CITW as the schoolwide
umbrella that drove all instructional programs at the school. The leadership team
members, lead teachers, and classroom teachers were all aware of CITW as the
focus at their school. On the School Level Survey a high majority of respondents
indicated that their school vision, mission statement, and goals were aligned with
CITW. Also on this survey approximately the same percentage indicated that their
school was supportive in implementing CITW. In the interviews teachers stated
that CITW was part of their daily instructional routine in the classroom which
aligned to the survey results. They also stated in the interviews that it was their
responsibility to use CITW strategies in their daily classroom instruction. The
leadership team members felt their role was less significant than the principal’s
role, possibly because it was not clearly articulated. They thought their role would
have been enhanced if they had obtained some formal training on CITW strategies.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8 2
The lead teachers were aware of their role with CITW due to the six formal
district trainings they were required to attend. These lead teachers were empowered
to be a part of the change process while taking ownership of the reforms (DuFour,
1991; Massell & Goertz, 2002). They served as mentors and provided dialogue
opportunities at department meetings to share experiences using CITW. They had
the training and knowledge, which allowed them to know what was expected and in
turn provided the guidance for members within their department.
The most important group in the implementation process was the classroom
teachers. Their role had to be clearly communicated since they were the ones who
ultimately implemented the district design. Their buy-in would determine the
sustainability of the reform effort. Teachers that attended the district trainings felt
empowered in comparison to teachers who only received training at the school level.
Higher levels of enthusiasm were seen in teachers who had attended district trainings.
This may be attributed to the extensive depth and coverage of the strategies at the
five-hour district training in comparison to thirty-minute trainings at the school.
The teachers, who attended the district trainings, however were sometimes
confused as to their role. McLaughlin & Talbert (2002) stated that clear
communication and dissemination of information to school sites was an element
districts had to consider to establish expectations for central office staff and
schools. Most district-trained teachers were surprised to find out they had to teach
their colleagues at school about the strategies. They thought they were just
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8 3
supposed to implement the strategies in their own classrooms, and someone else
would train other teachers at the school. They were not aware of the trainer-of-
trainers model and their role in the district design, which caused some teachers to
feel anxious because they did not feel they had the capacity to effectively fill that
role.
Monitoring the district design was done in a very cautious manner to avoid
creating tension and uneasiness among individuals throughout the district. Studying
the impact of district efforts within classrooms at a school site required monitoring
strategies and the necessary tools to collect pertinent information that included
classroom observations, student performance on criterion and norm-referenced
tests, and classroom artifacts (DuFour, 1991). The thought of implementing a
formal and rigid accountability system compelled administrators to work with
California Teachers Association (CTA) union representatives to ensure that
teachers’ contract rights were not infringed. The principal worked collaboratively
with union representatives at the school to ensure that they fully understood the
checklist form that was used in monitoring CITW. This form, which was shared
and explained at a leadership meeting, stated that the schoolwide percentages of
each strategy used were going to be reported. Simultaneously, the board requested
quantitative data from all the schools, which provided information on the extent of
CITW implementation in the classrooms. The board wanted to know if the $85,000
spent per year on training CITW strategies was effectively being used to improve
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8 4
student instruction. Marsh et al. (2004), stated the way districts ensured that schools
were addressing students’ needs was by objectively monitoring and facilitating the
process. Since “Every Classroom Every Week” had been implemented two years
prior, the principal and leadership team members were required to add an additional
observation to their visitations element in order to check which of the nine
strategies teachers were incorporating into their lessons.
The principal made an effort to communicate to the staff via electronic mail
the CITW monthly report that was submitted to the district. The participants
indicated their desire for a formal accountability system that would measure the
extent of implementation in order to provide individual feedback. It was interesting
to note that this was something that teachers did not want three years prior, at the
initial phase of CITW. Lead teachers in their interviews did not oppose increasing
their responsibilities, e.g., formally monitoring their departments on CITW use.
The perceptions and attitudes of teachers at this school changed over the three year
period and moved towards wanting an accountability system which provided
classroom teachers with CITW data that could direct their instruction.
Russellville High School followed the same implementation timeline as the
district. School staff development days focused on the strategies, which the district
trainings had covered during the school year. The timeline was clearly
communicated to the staff at Russellville that it would be a three-year commitment
and that a sustainability plan would be created at the end of year three. In the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8 5
School Level Survey, most respondents agreed that the trainings were timely in
their quest to improve student achievement and a small percentage disagreed.
This school had an edge over other schools in the district since three of the
four core lead teachers were also district trainers. These teachers believed in the
district design and worked with their department teachers to create awareness,
provide information, and communicate updates on the status of CITW. The
principal relied heavily on these trainers at staff development days to train the
entire staff on campus because she perceived classroom instruction to be the critical
component in improving student achievement. An individual teacher could have a
powerful effect on students, even if the school did not (Marzano et al., 2001). Since
evidence suggested that knowledgeable, skilled, and better-qualified teachers might
make a difference in student learning, instruction needed to be well-organized to
ensure increased student performance (Darling-Hammond, 1999). The school
trainings were mini-lessons of the district trainings, and followed the same
organized structure and format except the content was abridged for lack of time.
The staff was divided into groups who visited six classrooms where they were
trained on individual strategies. The concept was powerful, and the majority of
teachers enjoyed the presentations because they felt the presenters provided them
with tools and modeled the strategies in practical ways that the teachers could
replicate in their own classrooms. Most of the teachers felt that the thirty-minute
sessions for each strategy were too short. They felt they needed more time to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8 6
discuss and practice the strategies. The implementation at the school also required
follow-up trainings.
In the School Level Survey, a high percentage of respondents believed the
school offered frequent professional development to raise awareness of CITW,
which allowed them to know about CITW. However, one-fifth of respondents
disagreed with this assertion, which may be attributed to the limited number of
teachers who had attended district trainings for each strategy.
Almost one-quarter of the respondents stated they were in need for more
follow-up trainings in order to develop mastery of the CITW strategies. Monthly
leadership council meeting agendas reflected discussion on CITW, whereby, lead
teachers were reminded to discuss CITW in their monthly department meetings;
however, some teachers did not see this as enough support.
Interestingly, one-fifth of the respondents indicated they had not received
professional development trainings on CITW in the past six months. At the time the
survey was conducted, two sessions had been completed at the district and school
documentation records showed that every certificated staff member at Russellville
High School had received training in August at staff development training. Also,
the district provided records to show that all new teachers hired into the district
during the current school year received a one-day training session on six of the
CITW strategies. This finding could be interpreted in two different ways. First, the
participants did not take their responses on the survey seriously. Another possibility
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8 7
is that the participants only considered training in CITW as that provided at the
district, ignoring the site-based staff development that focused on CITW.
Documentation showed that every certificated staff member at Russellville High
School had been exposed to CITW trainings to some extent within the past six
months.
According to Stein and D’Amico (2002), changes in instruction occurred
when teachers received focused support based on individual improvement needs.
Teachers received support on CITW within their departments. Lead teachers made
an effort to provide dialogue opportunities to share lesson plans at department
meetings. A majority of respondents indicated that school leadership support
should be an element in the implementation of CITW. These responses may
suggest that teachers felt more support within their departments than with
administrators. Administrators rarely provided feedback to teachers, whereas, lead
teachers had on-going dialogues on CITW. Schmoker (1999) stated that time is
arguably a school’s most precious and scarce commodity, and that improvement
requires time for planning, training, and constructive dialogue. In the interviews,
teachers and administrators wanted more time or follow-up workshops by content
area, which would provide time to share and create lessons on a particular strategy.
Although, the principal allocated one release day for departments from Title 1
funding, it was not sufficient. Lead teachers stated they used the majority of the day
to work on common assessments, while administrators had hoped that teachers
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8 8
would use this time to align CITW with their curriculum. The bottom line was that
the time allocated to implement CITW was not sufficient and priorities varied by
department, as demonstrated by the way they spent their release day when there
was not any structure or expectations stated. Schmoker (1999) believed that
educators should systematically exploit every opportunity for improvement by
looking at available times for teachers to learn and refine the most effective
methods collaboratively (p.76). Several staff members commented that they would
have perceived feedback from administrators as a way to support their efforts in
implementing CITW, but stated that rarely occurred. Even though monthly
electronic mails were sent out to all teachers regarding the percent using CITW,
teachers preferred to have individual feedback to support their implementation
efforts in the classroom. Clear goals require feedback about progress and
institutionally as a whole, people continue to ignore the need for feedback
(Schmoker, 1999). Reeves (2000) stated adults find a process meaningful if the
feedback is meaningful and timely. In the context of educational accountability,
teachers and leaders need feedback for continuous improvement.
Several efforts at the school supported the successful implementation of
CITW. The following three factors helped to make CITW effectual at Russellville
High School: teachers’ acceptance of CITW, teachers’ reflections on lessons, and
departmental collaboration. First, the teachers responded well to CITW schoolwide
and believed it was a good plan to have their colleagues as district trainers and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 8 9
school presenters. It was important for districts to provide structured professional
development that offered cutting-edge, research-based practices related to what
teachers were doing and provide schools with strategies to improve their practice
(Grossman et al., 2002). They felt their colleagues understood the strategies and
provided examples of lessons that were applicable to a variety of disciplines. The
trainer-of-trainers model portrayed CITW as a powerful concept. The staff
embraced CITW because the strategies were reliable research-based strategies that
were easy to use and addressed all student learners; although, some participants did
question the benefit of using CITW with students of higher academic achievement.
Second, the CITW strategies created an awareness which helped teachers to
plan more cautiously and to reflect on how they would structure their lessons to
ensure that all students, including special education and ELL students, were
engaged in learning and have acquired the skills to retain knowledge. According to
Joyce & Showers (2002), it was increasingly important for teachers to know and be
able to apply different theories in their classrooms in order to provide good
instruction. The variety of manipulatives modeled to teachers reminded them that
one teaching methodology, e.g., lecturing at the board, did not fit all student
learners. The strategies incorporated what teachers already knew and were doing,
plus provided them with an opportunity to refine or build upon their teaching skills.
No single strategy worked well in all situations, so a variety of approaches needed
to be used to provide good instruction (Marzano et al., 2001). The positive outcome
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 9 0
in student performance was the evidence that substantiated the strategies’ success
and gave teachers a desire to continue using CITW.
Third, the departments at Russellville High School began to share not only a
common language in their subject area but also a common language related to the
strategies for presenting lessons. Teachers must have a certain level of knowledge
about one’s subject area, but perhaps more important, it also involves pedagogical
knowledge of how best to teach that subject-matter content (Marzano, 2003).
CITW provided department members with a resource and an opportunity to share
lesson plans, experiences, and troubleshooting solutions with colleagues. The
departments worked well as a team prior to CITW because of their beginning
attempts in implementing PLC, but CITW provided an additional bond that
connected the departments with a uniform set of instructional practices.
The majority of respondents agreed that they had opportunities to provide
input on the implementation of CITW. At the end of each session at the district
trainings the teachers worked with the principal collaboratively to develop a plan to
implement CITW schoolwide.
The results of CITW could not be correlated to the increase in the school’s
API scores due to the lack of a formal and reliable accountability system; although,
administrators, lead teachers, and teachers believed CITW had some influence on
the school meeting and exceeding API growth targets for the past three years.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
191
School level implementation efforts could improve in five areas. First,
school leadership team members needed formal training on CITW in order to serve
as a resource for teachers at school. They conducted formal yearly evaluations on
teachers and reported on weekly CITW teacher use; however, they either had
limited or no training. Even though some strategies were self-explanatory, a review
of what to look for could guide them in supporting teachers and in providing
feedback on their instruction. A second area for improvement was that CITW needs
to be tied to literacy and writing across the curriculum, as well as connected to the
California Standards for the Teaching Profession. Teachers thought the nine
strategies were taught in isolation and did not allow them to see the big picture on
how everything related. Third, teachers at this school wanted a model on how to
transfer the district trainings to the school. Teachers who attended district trainings
wanted to ensure that the requisite information was shared with their colleagues at
the school site. They felt that teachers who did not attend the district trainings did
not have the same enthusiasm and buy-in. One teacher recommended offering
district training on CITW to teachers who had not attended the district trainings.
Fourth, the teachers purported that the one-day district or school training was not
sufficient. Butler (1992) stated, “there is virtually no question that effective staff
development programs do change teacher practice” (p.14). Teachers needed to
reinforce the training by getting together and creating lessons or discussing how
they could implement the strategies in their classroom and throughout their
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 9 2
department. Follow-up training sessions were also missing. Teachers needed
opportunities to share specific lesson plans with others in content area and build a
bank of districtwide lessons aligned with CITW. Most schools and districts do not
provide opportunities for teachers to field test the strategies presented during staff
development days (Marzano, 2003). While most teachers received some
professional development each year, it tended to be delivered in one-day
workshops that often neglected relevant topics and was not related to the teachers’
classroom activities (American Institutes for Research, 1999; CPRE, 1995;
McLaughlin & Marsh, 1978). The general feeling has been that most staff
development programs have not benefited teachers or students (Corcoran, 1995;
Joyce & Showers, 2002; McLaughlin & Marsh, 1978). The fifth area was to
improve communication with teachers about the district design, its purpose, and the
resources available. Almost half of the teachers did not know or disagreed that
financial, staff, and material resources were allocated to facilitate the
implementation of CITW. Keeping administrators, lead teachers, and teachers
informed about the district design could increase sustainability as they became
empowered to improve teaching and learning.
Extent o f Implementation o f District Design at Russellville
This research question will be discussed using quantifiable data collected
from the School Level Survey, Innovation Configuration, Teacher Questionnaire
(Stages of Concern), and Document Review. These four instruments provided an
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 9 3
in-depth analysis as to the individual participant’s awareness of the district design,
each teacher’s stage of concern, and the level of implementation at the school. Each
of these instruments will also serve as a guide in suggesting the areas of future
professional development trainings to meet teachers’ needs at Russellville High
School.
The School Level Survey, items 22-33, provided percentage results on the
extent of implementation and the findings collaborated with the majority of
interview responses. Teachers responded more positively to implementing the
strategies on a personal level than when they were asked about their colleagues’
level of use. Almost half of the teachers did not believe their colleagues had bought
into CITW or that they were committed to the implementation of the district
design, although a significant majority responded that they personally felt
comfortable with using CITW consistently to improve student learning. Teachers
showed a lack of awareness or understanding of their colleagues’ use of CITW.
Individuals felt they were the only ones using CITW and that their colleagues did
not share the same level of enthusiasm or commitment as they did. By providing
teachers with opportunities to dialogue and share what they are doing in the
classroom, teachers’ perceptions on the use of CITW schoolwide could improve.
Each individual teacher and the group as a whole were making an effort to
implement CITW according to the interviews and individuals surveys. The only
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 9 4
way they were going to know that their colleagues were doing the same thing was
to share and provide on-going dialogue.
The SoC was used to determine what the participants were concerned about
at various times during the implementation of CITW. The individual raw score
results that were profiled in Figure 2 did not show any uniformity. Respondents’
SoC were plotted all over the graph. From the profile it appeared stage 5,
collaboration, and stage 1, informational, had the highest peaks. According to Flail
et al. (1998), multiple peaks in SoC are not common. Each of the sixty teachers’
highest SoC were placed in Table 14 to provide a summarization on the distribution
of the highest and second highest peak. Analyzing these two peaks provided an
insight as to the SoC which were the most intense and it also provided a general
description of where the participants were with understanding the district design.
The results showed that 25% of the respondents had stage 5, Collaboration, as their
highest peak. Stage 1, Informational, had 24% of the respondents at this stage as
their highest peak.
At stage 5, collaboration, the teachers were very concerned about working
with colleagues and coordinating the implementation of CITW. This SoC profile
substantiates the three areas of concern lead teachers and the classroom teachers,
who attended the district trainings, shared during the interview. First, they felt that
they were very enthusiastic about the strategies and were concerned the same
enthusiasm was not experienced by all teachers at Russellville High School.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 9 5
Second, they had received over five hours of district training for each strategy and
their colleagues only received thirty minutes at the school-based staff development
training. The last concern was that teachers, who attended the district trainings,
wanted a model to follow to ensure that they provided their colleagues at school
with the most pertinent and accurate information on each strategy. They were
concerned they did not know what or how to share this information with their
colleagues.
The second highest peak was stage 1, informational. In this stage teachers
were concerned about what CITW was all about and how much work it would take
to use. The teachers basically wanted more general information about what CITW
was, what it would do for them, and what would they have to do to incorporate it.
This stage also supported the interview findings because not all the teachers
received the same amount of training. The thirty teachers who attended the district
trainings received in-depth and extensive training for each strategy in comparison
to the remaining fifty-four. The teachers that attended the district trainings were
rotated and the lead teachers, as well as the principal, attended all trainings. Also,
this school had added 12% more teachers within the past year due to increased
enrollment. Even though these new teachers were aware of CITW due to the
overview provided at the district on new teacher orientation day and staff
development at the school, they had not received any of the in-depth trainings at the
district.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 9 6
The results, which showed stage 5 as the highest peak and stage 1 as the
second highest, indicated overall that the teachers at Russellville High School had
concerns about looking for ideas from others and reflected more on the desire to
learn from what others knew and were doing, rather than a concern for
collaboration (Hall et al., 1998).
The stage that had the lowest percent, 4%, of peaks by respondents was
stage 6, refocusing. According to Hall et al. (1998), a low tailing-off at the end of
the graph suggested that the teachers did not have other ideas that would be
potentially competitive with CITW. This relative stage of concern suggested that
respondents did not have any ideas that competed with the implementation
of CITW.
Future professional development trainings should include ways to address
the highest stage of concern by teachers at Russellville High School. In relation to
the CITW, the findings showed that the kind of assistance that would be helpful to
both groups of individuals should be different. Hord et al. (1987), offered
suggestions for interventions to reduce their concern at each stage. Interventions for
stage 5 include providing these teachers with opportunities to develop the skills
necessary to work collaboratively; helping them establish reasonable expectations
and guidelines for the collaborative effort; having them provide technical assistance
to other teachers who need assistance; and encouraging the collaborators. For stage
1, interventions include providing clear and accurate information about CITW;
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 9 7
using a variety of ways to share information; having teachers who have used CITW
visit these teachers; helping teachers see how the innovation relates to their current
practice; and having enthusiastic presenters.
The principal stated in her interview that about 60% of the teachers
embraced CITW and 100% used the strategies at one time or another. The
principal’s perception of teacher commitment was about 9% lower than what the
survey results showed. The Innovation Configuration helped create a visual of what
the superintendent and assistant superintendent of education had in mind for the
professional development plan at Los Coyotes High School District. The
Innovation Configuration described what the district design was and what it looked
like when teachers indicated that they were just getting started, had partially
implemented, or had fully implemented the CITW strategies. Champion (2003)
stated the Innovation Configuration provided information about how the reform
was impacting the school and the classroom. The Innovation Configuration was
verified by the superintendent and assistant superintendent of education to confirm
that the proper strategies were incorporated and that the extent of implementation
was properly adjusted to meet their expectations. The Innovation Configuration
served as a valuable tool to provide a depiction of where the school was in the
implementation process on each of the nine strategies.
The most commonly used strategy throughout the school was an imagery
mode called Nonlinguistic Representation. This was a graphic organizer that used
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 9 8
symbols and arrows to represent relationships. The Innovation Configuration
teachers’ self-assessment resulted in 100% of participants indicating that they had
fully implemented this strategy into their instruction. No one indicated that they
had fully implemented the strategy Generating and Testing Hypotheses, however,
half indicated they had partially implemented this strategy and the other half was
just getting started. The differences in response to both of the strategies discussed
may be due to the length of time since they had been presented at the district
trainings and at Russellville’s staff development. Nonlinguistic Representations
was the first strategy on which teachers received training three years ago; whereas,
Generating and Testing Hypotheses was introduced within the past two months this
study was conducted. The principal concluded that the differential use of the
strategies was unrelated to the amount of time since the training, but relative to the
level of thinking skills inherent in each strategy. In her perception, Nonlinguistic
Representations were used by the majority of teachers more often because it
required lower-level thinking skills; whereas, Generating and Testing Hypotheses
was used less often by teachers because it required a higher-level thinking skill.
The findings in the Innovation Configuration could provide the school with
a guide as to the current progress in implementing the district design and further
that implementation by identifying teachers’ needs to be addressed at future
professional development trainings. From the results it appears that two strategies
should be an area of focus. The first strategy, Reinforcing Effort & Providing
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 9 9
Recognition, and the second strategy, Generating & Testing Hypotheses, showed
teachers in the “just getting started” phase. Since the design was to train teachers on
three strategies each year, it would be advisable to cover these two strategies plus
the strategy Cooperative Learning, which had only about one-third indicate full
implementation. The following year, the strategies Identifying Similarities &
Differences and Setting Objectives & Providing Feedback should be the area of
focus since half of the respondents indicated they were only partially implemented.
The continued annual use of Innovation Configuration could help the school to
determine how close teachers’ actions were to ideal implementation.
The principal stated that the students who came to her school were fairly-
low performers with the majority of students reading below grade level. A high
percentage of respondents believed that the CITW strategies were used to help
students who performed below grade level. A classroom teacher stated in her
interview that she used CITW and noticed some of her low performing students’
writing had improved. Because such a high percentage of respondents indicated
that CITW helped their low performing students, some teachers might have
deduced that CITW did not benefit or contribute to the academic achievement of
high performing students in honors or advanced placement classes. In addition,
teachers agreed that CITW helped students to meet state performance standards;
however, less than half of the teachers stated that they connected CITW to the
standards in their instruction. More teachers were aligning CITW to the curriculum
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0 0
than to the state standards. This could be attributed to the fact that Los Coyotes
High School District had aligned its curriculum within the past five years to the
state standards and referred to these documents as the district standards according
to district records. So, in actuality, the state standards were the district standards
written in user-friendly terminology.
Russellville’ s Effectiveness o f Implementation
To determine the effectiveness of CITW at the school, participants’
perceptions from the interviews, surveys, and documentation in the following areas
were analyzed. The primary areas were teachers’ attitudes toward the district
design, assessment of CITW, professional development, the level of
implementation at the district and school, and both positive and negative
implementation strategies.
Teachers’ responded very positively toward the reform. Their attitude was
receptive because they felt that CITW strategies were relevant, effective, and
offered value to their personal instruction and for all students schoolwide. Tucker
and Codding (1998) indicated that teachers implementing America’s Choice
Schools bought into their district design when they witnessed real student progress,
saw their students doing things they had never thought they could do, and
understood the district design. The majority of respondents indicated that CITW
helped students to be successful in their classrooms and a higher percentage of
participants felt that the strategies were worth keeping. Teachers especially liked
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0 1
the fact that the strategies showed results, were research-based tools, and were not
new to them; making them feel comfortable with implementing them in their
classrooms. A significant number of respondents perceived CITW as feasible in
improving teaching and learning.
Teachers really enjoyed the trainers sharing their classroom experiences,
offering advice, and modeling lessons. This made teachers feel that the strategies
were easy to implement and would increase their efficacy in the classroom. Guskey
(1986) observed that staff development programs must be systematic attempts to
change teachers’ classroom practices, their beliefs and attitudes, and student
learning outcomes. The irony is that even though nearly 83% of the respondents
indicated that CITW strategies were effective in improving student learning, only
about half of those respondents indicated that their personal instruction had
changed by using CITW. This finding may be attributed to the fact that the
strategies were not new to them and that they were already using these strategies
prior to the implementation of CITW.
The school did not use a formal evaluation system of CITW to obtain
information and assess the effectiveness of the reform. Marsh (2002) suggested an
important element of a district’s design was to have a database accountability
system in place to collect and examine data in order to set goals and identify
improvement plans. Administrators had their weekly classroom visits and used a
checklist to document observed strategies used in teachers’ instructional lessons.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0 2
Lead teachers and teachers stated that their only way of assessing CITW was
through informal conversations with colleagues at department meetings. One
teacher stated that the “Share Board” in the staff lounge provided her with the
opportunity to see written lesson plans by colleagues who used CITW strategies
and were accompanied with their students’ work. Another teacher stated he did not
believe his colleagues assessed CITW, rather they either used the strategies or not.
All of the participants interviewed stated that the most pertinent assessment was the
school’s API scores. They believed that CITW contributed to increased API scores;
however, the lack of a reliable and consistent assessment system prevented the
determination of the actual extent of influence that CITW had on the increased
scores.
The principal stated that the professional development offered at the district
was helpful and useful to her staff. Schmoker (1999) recommended that staff
development time must be devoted exclusively to proven or research-based
methods that ensure student success and concluded by stating that, “this kind of
hard-headed approach was long overdue” (p.75). The school’s staff development
offered mini-lessons that replicated the district training, which were also helpful.
Teachers found the district presenters and district trainings to be extremely helpful.
They liked the way material was presented and that the specific examples provided
were applicable to all content areas. The two most commonly mentioned areas that
proved to be helpful to teachers were: teachers teaching teachers (trainer-of-trainers
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0 3
model), and the ability to apply the strategies to any content area. Teachers made
the same positive comments about the school trainings, except for the fact that the
time spent at the school level was not sufficient for each strategy. Teachers at the
school trainings wanted longer and more in-depth training sessions that provided
them with more examples on how they could use CITW in their classrooms. The
thirty-minute sessions did not compare to the five hours of training at the district.
The School Level Survey showed that the majority of respondents felt that the
school provided on-going support for CITW, although, 29% disagreed. The 29%
disagreement level may have been attributed to the short trainings offered at the
school in comparison to the district. Initially, there appeared to be a relationship
between the frequency results of SoC and the number of trainings attended by
respondents. By using the demographic data from the School Level Survey and the
interviews, further analysis sought to find evidence of why certain stages were more
or less intense; however, upon closer scrutiny, there was not a wide variation in
respondents’ stages of concern (Figure 4). This may be due to the way respondents
interpreted the question on the School Level Survey related to the number of trainings
they attended. Some respondents may have only counted the number of district
trainings attended, whereas other respondents may have counted the number of both
district trainings and school trainings attended. Another factor that may have
attributed to the lack of variation was that the sample size for some of the different
groups was small and might not necessarily be a true depiction of a valid profile.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0 4
Participants provided a very general response when they were asked to
describe what full implementation looked like in the classrooms. They named the
strategies and described specific activities used during a lesson. Most said that
lessons looked different than a few years ago and that instruction was no longer just
auditory. It was difficult to gain a consistent and true picture of what full
implementation looked like. Since a structured implementation process was not
articulated, every participant had a different concept of what implementation
actually meant. The findings supported Roy & Hord’s (2004) assertion that the
common problems faced by educators was being unable to answer the question,
“What does implementation look like?” The school participants did not have a
rubric that specifically explained what full implementation looked like for CITW.
The principal also supported this finding by stating that she believed CITW was
clearly articulated at her school, but that she did not think it was as clearly
implemented.
The lack of structured guidance at the school may be attributed to the
cultural practices throughout the district according to a district administrator.
Schools in the district have always been given a large degree of autonomy due to
their unique demographic needs, community expectations, or both. According to
Tucker and Codding (1998) school autonomy creates a widespread feeling of
having greater control leading to greater effectiveness (p.229). The district did not
mandate how the schools should attain their educational goals, but provided
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0 5
tangible opportunities through staff development. From this finding it appears that
the district, schools, and teachers would greatly benefit from a rubric such as the
Innovation Configuration to improve communication on what implementation
looks like. This would provide a measure of the level of implementation in each
school and classroom, and serve as a guide in planning professional development
that meets the needs of teachers. Attaining full implementation truly depends on the
teachers.
Positive implementation strategies could be seen at two locations: the
district and the school. The district provided the initial trainings to implement
CITW uniformly throughout all schools in the district. A district increased
instructional capacity by focusing on increasing professional knowledge and skills
(Massell and Goertz, 2002). Teachers stated in interviews and on district training
evaluation forms that they liked the concept of using the trainer-of-trainers model.
Having the school district orchestrate the training and having the best teachers train
coworkers was a brilliant model of teamwork. One of the participants stated in the
interview that secondary teachers were more open to trainers among their own
ranks than they were to accepting outside consultants. Allowing the teachers at the
trainings to collaborate with coworkers and letting them share their knowledge
proved to be very powerful. Teachers enjoyed the fact that the presentations were
engaging with applicable techniques to use in the classroom. The presenters
provided techniques that could be applied to all content areas and modeled specific
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0 6
strategies, allowing teachers to implement them immediately in their classroom.
Teachers liked the hands-on activities that allowed them to not only work with
colleagues from within their departments but outside of their departments as well.
Teachers rarely had an opportunity to work collaboratively and discuss instruction
with other content area teachers within their school. One teacher that attended the
district trainings wrote on the evaluation sheet, “The trainings presented effective
ideas that are practical, logical, and reasonable.”
Reeves (1998) stated that to increase awareness of changes, some principals
served as catalysts between the district and their school. He also suggested that a
principal’s ability to lead had a dynamic impact on the implementation and
sustainability of the reform. The school conducted some positive implementation
strategies. First, the dedication of the principal to have CITW at the forefront by
communicating to staff about CITW at every formal and informal meeting she
attended established a foundation of high expectations in every classroom. Second,
the principal expected all core department chairpersons to attend district trainings.
This created buy-in from the teachers who were the most influential leaders within
departments. Due to the trainings and reminders at monthly leadership council
meetings, department chairs made it a priority to include opportunities to dialogue
about CITW in their monthly department meetings, which resulted in their
department members looking at teaching in a different way. Some teachers
commented that CITW was now part of their curriculum. Third, the principal had
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0 7
CITW drive staff development days. Teachers received mini-lessons on the
strategies offered at the district trainings. This kept the staff informed and involved
with the implementation of CITW. Fourth, the presenters at the school were also
the same presenters that facilitated the district trainings. This school was very
fortunate to have had three of its core department chairpersons (lead teachers) as
districtwide trainers. This created even more of a buy-in for the 42 teachers that
worked under the leadership of those department leaders. The fifth strategy was
that the principal worked with the administrators at the district to ensure that she
created a plan to effectively implement CITW at her school.
There were six areas that could have been improved during the
implementation of CITW. The first area was that leadership team members needed
training on CITW in order to serve as instructional leaders. Second, teachers did
not like to leave their classes for an entire day in order to attend district trainings.
Some teachers recommended holding the trainings after school or during the
summertime to minimize their absence from the classroom. Third, a formal
measurable and reliable evaluation system needed to be in place. The checklist was
perceived as informal and did not provide departments or individual teachers with
any pertinent information that would help them modify or improve their
instruction. The fourth strategy was that feedback needed to be consistent and
provided to teachers either in writing or verbally to encourage efforts in
implementing CITW into their instruction. Teachers felt that district expectations
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 0 8
had to be clear, as they were not aware that CITW was the district plan. The
superintendent supported this finding by stating that CITW was “really good stuff’
and more time should have been taken in building buy-in during the adoption
process. The district should have spent three to four months working with
principals, assistant principals, teachers, and curriculum committees to find out
what they thought about bringing in CITW, how to implement it, and if they had
any better ideas. Using this approach could have resulted in a more effective
implementation with a combination of top-down and bottom-up. This led into the
fifth area of improvement, which was developing a structure or format for teachers
on how to present the strategies from the district to the school. Teachers who
attended the trainings felt personally comfortable with using the strategies in their
classroom, but did not feel they had adequate training to train other teachers at their
school or within their departments. The sixth, and most common area of
improvement that was mentioned on every district evaluation sheet, was that
teachers needed a follow-up day to work on implementing the strategies within
their content areas.
Summary
This chapter presented the findings, an analysis, and provided an
interpretation of data for this study. The data collected to present the findings were
aligned with the conceptual frameworks and used the instrumentation in the case
study guide to gather the necessary qualitative and quantitative data to answer each
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
of the four research questions. The discussion included a description of the district
design, district decisions, and intended outcomes. It also provided insight into the
school level efforts, the extent of implementation, and perceptions of the
effectiveness of implementation strategies used to improve teaching and learning.
A summary of the background, methodology, research findings, conclusions,
implications, and recommendations for future research are presented in
Chapter 5.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1 0
CHAPTER FIVE
SUMMARY, CONCLUSIONS, AND IMPLICATIONS
Summary of Background
In the past few years, educational reform has warranted district
administrators to reorganize their districts’ structure and purpose. Exemplary
districts developed a vision for instruction, assessed school needs, and placed
teachers and teacher learning as its core strategy (Hightower et al., 2002). The
capacity of a district and schools may create limitations, but consideration of these
areas of potential weakness may reduce instructional implementation barriers in
the future (Massell & Goertz, 2002). Focusing on research-based designs and
frameworks can lead to improved student performance as stated by Marzano et al.,
2001, “We are at the beginning of a new era in education, one in which research
will provide explicit guidance for the classroom teacher” (p. 10). Districts
implementing reform needed models that incorporated the combination of
elements and strategies necessary to achieve student improvement and sustain the
reform effort.
Developing teacher buy-in and internal commitment to a reform strategy
required the communication of assessment results in a timely manner with accurate
data to the school staff. Teachers needed to be empowered, take ownership of the
design, and be motivated to work through implementation barriers. Fullan (2001)
stated, “Effective leaders make people feel that even the most difficult problems
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1 1
can be tackled productively” (p.7). Implementing change in any organization
causes the affected members to experience a personal level of stress and concern.
Effective schools required strategies to be in place and implemented the necessary
steps to address teachers’ concerns, often identifying those concerns by guiding
them through a Stages of Concern Model (Marsh & Jordan-Marsh, 1985). In order
to sustain instructional improvement efforts, districts needed to know the steps that
were necessary before, during, and after the reform to ensure support from the
district, schools, and community.
Implementation of a program required the commitment of teachers (Marsh,
2002). Often staff development workshops were one-day attempts to change
teachers’ instructional strategies and were not sustained over time (American
Institute for Research, 1999; CPRE, 1995; McLaughlin & Marsh, 1978). To
increase the use of new strategies by teachers required on-going training, support,
and follow-up from the school and the district (Huberman & Miles, 1984). Districts
and schools needed to implement the design in phases over several years, while
simultaneously increasing both the capacity of the district and the schools.
Providing on-going monitoring and assessing the degree of implementation
using various indicators provided meaningful information that indicated whether
there was progress in meeting the needs of the school, teacher, and student.
Schmoker (1999) suggested that if improvement was required, the focus needed to
be on short-term results, using feedback to provide information on how well a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1 2
school was attaining both short-term and long-term goals. Districts needed to
utilize instruments and tools to determine the impact a program or reform had on
districts, schools, and students.
Purpose of the Study
The purpose of this case study was to understand the role of districts in
enhancing good teaching and learning, and how it was linked to districtwide
instructional improvement efforts. It explored the district design to improve
teaching and learning, along with the manner in which schools’ efforts facilitated
the implementation process, followed by an investigation into the extent of the
implementation, and the effectiveness of the district design.
The research study questions were:
1. What was the district design for improving teaching and learning?
2. What school level efforts facilitated the implementation of the district
design?
3. To what extent had the district design been implemented at the school
and classroom levels?
4. How effective were the district design and implementation strategies at
the school and classroom levels?
Methodology
A case study approach was used as the most appropriate way of addressing
the research questions through a qualitative tradition of inquiry to understand the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1 3
role of one district’s efforts to enhance good teaching and learning. This study
aimed at describing and understanding the district as the unit of analysis to
determine how a school understood, interpreted, and implemented the district’s
design; as well as the extent of implementation and the effectiveness of the
perceived design.
Interviews provided qualitative insights into the personal experiences and
perspectives of the participants which were narrowed into a small number of core
themes. One of the research questions was seeking to determine the extent of
implementation and required using a quantitative approach. The instrumentation
used should enable the readers to gain an in-depth understanding of the issues and
concerns revealed by participants who implemented the districtwide instructional
improvement. In an effort to increase internal validity, methodological triangulation
and data triangulation were used by combining both qualitative and quantitative
data. Triangulation ensured that the findings were supported by using multiple
sources which confirmed and validated the results.
Sample
Through the use of purposeful sampling, one high school district was
selected for study to gain a comprehensive understanding of its role in
implementing a districtwide instructional improvement effort. The selection criteria
for the district included a student population between 10,000 and 60,000 students,
comprised of students of diverse ethnicity and socioeconomic status. The selection
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1 4
criteria for the school that participated in the study included evidence that the
school had met its API for two of the last three years and had evidenced stable
leadership with the same principal at the site for at least two years. Purposeful
sampling and stratified purposeful sampling were utilized in selecting participants
for this study.
Data Collection and Analysis
The instrumentation for this study was developed by doctoral candidates at
the University of Southern California’s School of Education during the summer of
2004. The school level data for this study were collected in four phases over a two-
month period during the months of November and December 2004.
A conceptual framework was developed for each of the four research
questions (described in detail under the specific research question in the next
section). Based upon these frameworks, eight instruments were developed. All the
instruments were incorporated into a Case Study Guide (Appendix A) that served
to organize the collected qualitative and quantitative data.
1. The Case Study Guide contained a data collection chart, provided a
narrative that described each instrument, included a description of the
four data collection phases, described four conceptual frameworks, and
included the interview guides. The Case Study Guide also included
instrumentation tools that were necessary and critical in obtaining data:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1 5
the Innovation Configuration, the School Level Survey, the Teacher
Questionnaire (Stages of Concern), and the Document Review Guide.
2. Four Interview Guides were used to ensure that the interview questions
for the principal, leadership team members, lead teachers, and teachers
were aligned with the research questions. The interview data were coded
according to one of the four research questions and the researcher notes
were reviewed to find common themes or patterns.
3. The Innovation Configuration was used as a rubric to measure the extent
of implementation of the district design. Each teacher participant self-
reported his/her individual level of implementation for each of the nine
instructional strategies.
4. The School Level Survey was used with volunteer selected certificated
personnel: principal, leadership team members, lead teachers, and
classroom teachers. It included demographic questions and 44 items
intended to measure the degree of awareness about the district’s design
for improving teaching and learning. The statements were presented on
a five-point Likert scale, allowing participants the opportunity to circle
their level of agreement within each statement. The respondent’s results
were placed in a summative chart as a percent of the total responses to
facilitate analysis and reporting.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5. The Teacher Questionnaire (Stages of Concern) was used by volunteer
classroom teachers. The survey was a 36 item questionnaire based on
the Stages of Concern Model and offered useful information that
addressed the seven stages individuals experience as a result of
organizational change: (0) Awareness; (1) Informational; (2) Personal; (3)
Management; (4) Consequence; (5) Collaboration; and, (6) Refocusing.
The results of this questionnaire demonstrated teachers’ knowledge
about the district design and their comfort level with implementing the
reform.
6. The Document Review Guide was used to organize, sort, and align
district documents according to each research question to ensure that
only relevant information was collected.
The instrumentation and the conceptual framework used for the collection
and analysis of data, as well as the presentation of selected findings, are presented
in relation to the four research questions in the next section.
Summary of Findings
The findings and conclusions for each of the four research questions were
revealed through analysis of the data collected in this study. Selected findings, by
research question, were reviewed below.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1 7
Research Question One
The first research question asked, “What was the district design for
improving teaching and learning?” Data collection and analysis were guided by
Conceptual Framework A (Appendix C) which was separated into five sections:
(1) district’s design; (2) district’s change strategy; (3) communication strategies;
(4) district’s standards-based instruction; and, (5) intended outcomes. Interviews,
the School Level Survey, and district documents were the primary sources of data
for this question.
The board of trustees and district realized that the district’s role in
instructional excellence had to be modified due to the national, state, and local
accountability expectations. The data from this question showed that the key areas
of the district design were the use of research-based strategies presented at district
trainings, communication using a top-down approach, and the assessment of the
implementation and use of CITW.
The prior superintendent envisioned using research-based strategies from a
book entitled, “Classroom Instruction that Works” (CITW) as the basis of the
district design that would have all schools use the nine strategies as tools to reflect
on direct instruction. A top-down approach was used to adopt and implement the
strategies over a three-year period with a focus on three strategies per year. An area
of concern with this approach was communication. The principal was key in
determining the initial staff buy-in at schools. The principal at Russellville High
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1 8
School empowered her lead teachers to assist in the implementation of CITW, and
served as the connection between district administrators and the classroom
teachers. Even with the extra support, a lack of communication still existed among
the majority of teachers as they were unaware that the strategies they were
implementing in their classrooms were strategies of the district design.
The three years of district trainings resulted in a change of the district
culture. In the past, the schools created, organized, and facilitated their own staff
development. The model of trainer-of-trainers was used for the trainings, with
effective district teachers as presenters. Two workshops were held in the fall of
each school year that addressed core subject areas and included discussions on the
research, definitions of the strategies, examples of lessons, and an opportunity to
practice the strategy. Some administrators attended the same training, but without
any supplemental training in the assessment of CITW.
The assessment of the district design was monitored in every classroom at
least once a week by the principal or a leadership team member. A weekly checklist
was used to determine the percentage of teachers’ schoolwide who used each
strategy. The quantifiable data reported to the district on a monthly basis did not
provide useful information on ways to improve the implementation of CITW at the
school for principals, leadership team members, lead teachers, or teachers.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1 9
Research Question Two
The second research asked, “What school level efforts facilitated the
implementation of the district design?” Data collection and analysis were guided by
Conceptual Framework B (Appendix D) which had five sections: (1) site level plan
for implementing the district’s design; (2) site level resources needed; (3)
communication of district design to staff members; (4) type of professional
development provided; and, (5) school support offered. Interviews, the School
Level Survey, and district documents were the primary sources of data for this
question. The data from this question showed that the key areas were the roles of
individuals at the school, the monitoring of implementation, and the school
trainings provided.
The principal understood and accepted her role as the link between the
district and the staff in conveying the district design. Creating awareness,
communicating, supporting, and focusing on the implementation of CITW was her
priority. The leadership team members, lead teachers, and classroom teachers
interviewed were all aware of CITW as the school’s focus and viewed it as part of
their daily instructional routine. Leadership team members felt their role was less
significant than the principal due to their limited or lack of training. Lead teachers
thought their role was less important than the principal perceived their role to be.
The lead teachers were aware of their role to serve as mentors and instructional
leaders due the three consecutive years of formal district trainings they were
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 0
required to attend. The training that the principal and lead teachers received
provided them with the knowledge and confidence to serve as mentors for teachers
at the school. An area of concern for the teachers that attended the district trainings
was the lack of a model on how to implement the strategies and how to provide
school-based trainings. Since classroom teachers were the ones actually
implementing CITW, their role had to be clearly communicated in order to obtain
their buy-in and sustain the reform.
The school was required by the district to provide monthly quantifiable
evidence that CITW had been implemented and was used in classrooms. Two years
prior to implementing CITW, the principal or a leadership team member visited
every classroom at least once a week as a way to be visible on campus. When
CITW was implemented, the administrators began to walk into each classroom and
not only observe but also monitor the strategies they observed during the
instructional lesson with the use of a checklist. This made some teachers begin to
feel uncomfortable with the administrator’s presence, and union representatives
became involved. The school principal then communicated the purpose of the
checklist, and worked collaboratively with union representatives to ensure that
teachers felt comfortable in their working environment. The principal made an
effort to share and communicate the CITW monthly report with all staff. Three
years later, the participants in the study revealed that they wanted a formal CITW
monitoring system which would provide them with individual data. Lead teachers
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 1
also agreed to increase their responsibilities by formally monitoring their
departments for the use of CITW.
Russellville High School had an advantage over other schools in the district.
Three core lead teachers were trainers at the district CITW trainings. They were
empowered and grew professionally as instructional leaders. They motivated and
guided members within their departments to implement the strategies from CITW.
The lead teachers provided department members with resources and an opportunity
to share lesson plans, experiences, or troubleshooting with colleagues. Through
presentations modeled after the district trainings, lead teachers disseminated their
knowledge and acquired skills not only within their department but to all other
departments. The trainer-of-trainers model proved to be very effective and credible
as the presenters offered their expertise from the classroom. The staff also
embraced CITW because the strategies were reliable research-based strategies that
practically addressed the needs of all student learners. However, teachers were
concerned about having only thirty minutes spent on each CITW strategy at the site
trainings. They felt that the school training sessions needed to be longer, more in-
depth, to have examples modeled, and include follow-up trainings.
Research Question Three
The third research question asked, “To what extent had the district design
been implemented at the school and classroom levels?” Data collection and
analysis were guided by Conceptual Framework C (Appendix E) which had nine
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 2
sections: (1) monitoring of district design; (2) feedback obtained by administrators;
(3) data generated; (4) information on perceived amount of change throughout the
school; (5) explanations of why district design may not be fully implemented; (6)
process used to modify the strategies; (7) role of capacity in promoting or inhibiting
extent of implementation; (8) reform affects on equity throughout the system; and
(9) change in culture due to district design. Teacher interviews, the School Level
Survey, the Teacher Questionnaire (Stages of Concern), and district documents
were the primary sources of data for this question.
The majority of respondents personally felt comfortable using CITW and
believed it improved their teaching; although, they did not think that all teachers
were committed to using CITW. An analysis of the implementation and use of the
nine strategies demonstrated that the most commonly used strategies at the school
were the lower-level thinking strategies, i.e., Nonlinguistic Representations;
whereas, the higher-level thinking strategies were not fully implemented in the
classrooms. Two of the higher-level thinking strategies (i.e., Reinforcing Effort and
Providing Recognition and Generating and Testing Hypotheses) were only at the
beginning stages of implementation and needed considerable more attention.
Determining teachers’ stage of concern was necessary to analyze the
training required to move the school toward full implementation of each strategy.
Most of the teachers were either in the most common stage of concern,
collaboration, or the second most common stage of concern, informational. Both of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 3
these stages of concern could be attributed to the inconsistent trainings for teachers.
Some received in-depth district and school training, while the remainder only
received thirty-minute mini-lessons at the school site. Overall, teachers’ primary
concerns were related to the desire to learn from what others knew and were doing,
rather than a need for collaboration.
Research Question Four
The fourth research question asked, “How effective were the design and
implementation strategies at the school and classroom levels?” Data collection and
analysis were guided by Conceptual Framework D (Appendix F) which had four
sections: (1) participants perspective on impact of district design effort; (2) cost
effectiveness; (3) time efficiency; and, (4) realizing intended expectations.
Interviews, the School Level Survey, and district documents were the primary
sources of data for this question.
Teachers’ attitudes toward CITW were very positive. They felt the
strategies were effective and relevant to their personal instruction. Teachers liked
that the strategies were research-based and were not new to teachers, which made
them feel comfortable implementing them in their classrooms. The data from this
question showed that the key areas that demonstrated effectiveness of the district
design were the evaluation system, professional development, and interpretation of
fully implemented.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 4
A formal, measurable and reliable evaluation system provides information
for the district, the school, and the classroom teacher. Administrators conducted
weekly visits and used a checklist to report teachers’ use of CITW to the district.
Lead teachers and teachers stated that their feedback was limited and their only
way of assessing CITW was through informal conversations with colleagues at
department meetings. The formal evaluation system used to monitor and assess the
implementation and use of CITW did not provide useful information for teachers.
This data was compiled into monthly reports by the principals and submitted to the
district. Due to an earlier agreement with the union, feedback to teachers was
limited; thus, the only way teachers had to assess CITW was through informal
conversations with colleagues at department meetings.
The uniform professional development at the district for all the schools was
more effective than the school trainings. The collaborative district trainings
provided teachers with an opportunity to share knowledge with teachers from other
content areas. Both the district and school trainings used the trainer-of-trainers
model which proved to be successful and powerful. Russellville High School used
the district’s training model and replicated the structure at the school into mini
lessons. The area of concern for the school trainings was that due to time
constraints, the school coverage of each strategy was not as in-depth as the district
training sessions. In addition, there was a lack of follow-up training at the school
within content areas.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 5
There were varying interpretations of what CITW was perceived to look
like at each stage of implementation, as well as a lack of a clear understanding of
what full implementation entailed. Improving communication and providing a
measurable way to determine the implementation level in each school and
classroom would help with planning viable professional development which would
meet the needs of teachers. Attaining full implementation truly depends on the
proficiency of the classroom teacher.
Conclusions
Several core themes emerged from the analysis and triangulation of the
qualitative and quantitative data collected. The analysis of the findings suggested
that some areas played a key role in the success of the district design. Also, some
potential areas needing improvement emerged that could hinder sustainability of
the district design if interventions were not taken. The following were “Best
Practices” and areas to improve.
Best Practices
The five “best practice” conclusions that emerged were:
1. A board of trustees and superintendent who recognize when change is
needed can take the necessary intervention steps to protect its staff,
students, and community. The high-stakes accountability system made
the district aware that there was a district need to focus on student
learning and instructional improvement. The districtwide uniform
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 6
research-based strategies were implemented in an attempt to improve
instruction and narrow the achievement gap among district students.
2. Implementing instructional strategies that are perceived by teachers as
practical can increase acceptance and sustainability. The teachers were
already using many of the strategies before the implementation of the
CITW, yet the discovery that these strategies were research-based and
applicable to any subject increased their support of the reform effort.
The modeling of these research-based strategies in the trainings caused
the teachers to reflect on instructional lessons and their students learning
needs.
3. Principals who are instructionally focused and support the district
design play a central role in implementation. The principal at this school
made it a priority to communicate CITW to the staff at every
opportunity. She embedded CITW into the annual schoolwide plan,
included CITW on every meeting agenda, and devoted three years of
staff development to CITW. Her dedication to the district design was
evident to all leadership team members, lead teachers, and classroom
teachers, making them all aware of the school’s focus on CITW.
4. The district and school demonstrated their respect for the expertise of
teachers by using the trainer-of-trainers model to train teachers on
CITW. Using district teachers as the expert presenters at the trainings
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 7
effectively engaged other teachers in learning and increased their
ownership of the reform. The participants trusted the reliability and
applicability of the information presented, not only because it was
research-based but because it was presented by people they knew and
trusted.
5. A school that empowers others as instructional leaders reduces the
barriers to implementation and increases the probability of
sustainability. According to Schmoker (2001), “Change has a much
better chance of going forward when principals team up with teachers
who help to translate and negotiate new practices with the faculty”
(p.l 16). While required to attend all the district trainings, the lead
teachers were empowered as instructional leaders and developed the
skills to become mentors to the teachers in their department. The lead
teachers implemented and monitored CITW throughout their department
which distributed leadership and increased the school’s capacity to
assist teachers in improving instruction.
Areas to Improve
The five areas needing improvement in the implementation of CITW at the
district level and the school level are:
1. Organizations going through change require constant feedback and open
lines of communication. Districts that implement a top-down approach
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 8
need to specifically focus on ensuring that expectations are clearly
communicated throughout the entire organization especially to
individuals that will be expected to demonstrate results. Los Coyotes
High School District’s instructional plan was communicated to the
principals and some lead teachers, but it was not communicated
effectively to the classroom teachers. Although the majority of
classroom teachers were aware of CITW, they were unaware that it was
the district design and were unable to define what full implementation
of CITW looked like.
2. Roles need to be clearly defined to ensure that all organizational
members clearly understand what is expected of them and others during
the change process. “Every work group needs a structure of task roles so
that members understand who is going to do what” (Bolman & Deal,
1997, p. 152). At Russellville High School, the principal understood her
role as the link between the district and school. The lead teachers
assumed their role after being empowered at the district trainings, yet
the leadership team members and classroom teachers were unclear as to
their role in the implementation of CITW.
3. School site administrators require on-going training to serve as effective
instructional leaders. The principal received the same level of training as
the classroom teachers which did not include training on monitoring or
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 2 9
assessing the implementation of the district design. Leadership team
members received limited or no training on CITW, which truly hindered
their ability to provide feedback to teachers on their use of the
instructional strategies.
4. Assessment of the district design needs to provide accountability data
that is meaningful to all stakeholders. The monthly data collected by site
administrators only provided a quantifiable percentage of teachers using
each of the strategies; however, this data did not provide useful
information for teachers that would assist them in modifying their
instruction; i.e., effective use of the strategies. More attention needs to
be directed to the quality of the instructional strategies used, not just the
mere quantity.
5. “Professional development should be determined by an analysis of the
skills and knowledge the staff will need to carry out the school’s plan”
(Tucker & Codding, 1998, p.319). Condensing five-hour district
trainings into thirty minutes of training for each strategy at the site did
not provide the time required to address the needs of the staff,
particularly in relation to the effective use of the more complex
strategies.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 0
Implications and Recommendations
The findings and conclusions of this study have led to the following
implications for those implementing instructional reform:
Board o f Trustees and District Administrators
1. School board members need to have a clearly defined, well
communicated, and focused vision that is assimilated throughout the
organization. The district goals and priorities should revolve around
current national and state expectations for districts, schools, and
students.
2. Acknowledgement and acceptance of the district’s strengths and
identified needs is crucial in making informative decisions that will
have a positive impact on the community at large. District
administrators must assess the needs of the district and schools with
accurate data in order to implement and sustain an effective systemic
design that is supportive of its instructional focus.
Site Administrators
1. Site administrators must demonstrate confidence, enthusiasm, and
technical knowledge to support the successful implementation of their
reform efforts. Additionally, they must recognize the fact that in order
to promote reform efforts, certificated, classified, student body, and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 1
community at large must be empowered to assume responsibility and
ownership to affect positive and lasting results.
2. All site administrators need to develop the expertise in the elements of
the reform effort in order to effectively provide guidance and support to
the classroom level.
Teachers
1. Lead teachers must understand their vital role in ensuring the integrity
of the district design be implemented accordingly. As mentors and
colleagues, they are not only the trusted messenger of the district design,
but they also serve as the expert link between school site administration,
teachers, and parents.
2. Teachers’ level of implementation determines the sustainability of the
district design. Additionally they are required to familiarize themselves
with the district design and establish an intrinsic desire to further their
professional growth and personal development.
Recommendations for Future Research
Based on the findings of this study, the following recommendations are
made regarding future research in this area:
1. Questions developed for a survey must be clear and concise in an effort
to glean specific information relative to the design.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 2
2. In order to ascertain the sustainability of this reform effort, it is
recommended that a follow-up research design be implemented to
determine the degree to which elements of the reform need be modified
or reevaluated systemically.
3. It would be advisable to determine the effectiveness of the
implementation of a district design and its relationship to funding.
Districts should provide equitable level of support so that all
participants could acquire uniform professional development training to
implement the district design.
4. It would be of interest to study a district that provided uniform training
to all school site administrators on their district design and determine
how the professional development affected the extent of implementation
and reform effectiveness at schools and the classroom levels.
5. A district’s decision-making approach in adopting and implementing a
district design may affect the outcomes. It would be advisable to study a
district that used a “bottom-up approach” to ascertain the efficacy and
long-term effectiveness as a means for systemic change. It is
recommended that further research be conducted to focus on a
comparative analysis of this design to determine its overall reliability
and validity as an effective and successful means for systemic change in
comparison to a “top-down approach.”
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 3
6. The implementation of the “top-down approach” in implementing
systemic change is dependent upon different levels of support. It would
be of value to implement a qualitative analysis designed to investigate
the intrinsic attributes of the site administrator in their efforts to support
this methodology and determine how various dimensions of personality
of the educational leader would facilitate successful implementation of
the “top-down approach.”
7. Effective communication models may affect the accuracy and efficiency
of information transmitted within an organization. Future research of
districts implementing instructional improvement efforts can serve as
models on how the efficiency of its communication system within the
infrastructure of the organization influences its implementation and its
impact in assuring successful systemic change.
8. The value of such research cannot be underestimated when attempting
to ascertain an efficient design to implement systems change. In our
continued efforts at assuring a positive relationship between
organizational change and our constantly changing culture, it is of the
utmost significance that we continually monitor, analyze, re-evaluate
and modify our efforts at implementation of systems operations in order
to obtain long range sustainability and successful survival of the
organization.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 4
REFERENCES
American Educational Research Association (2004, April 14). Curriculum
measurement methodologies: New developments from the Third
International Mathematics and Science Study (TIMMS) [Electronic
Version]. Proposal for Division D: Measurement and Research
Methodology at the annual meeting of American Education Research
Association, San Diego, CA. Retrieved April 26, 2004, from
http://convention.allacademic.com/aera2004/AERA papers/
AERA sess 1041 11384a.PDF
American Federation of Teachers (2001). Making standards matter 2001
[Electronic Version]. Retrieved July 3, 2004, from
http.7/65.110.81.56/pubs-reports/downloads/teachers/msm2001 .pdf
American Institutes for Research (1999). Designing effective professional
development: Lessons from the Eisenhower Program. Washington, DC:
US Department of Education.
American Institutes for Research (2003, September). California’s Public
Schools Accountability Act (PSAA): Evaluation findings and
implications [Electronic Version]. Executive Brief for Birman et. al.,
Evaluation study o f the Immediate Intervention/Underperforming
Schools Program and the High Achieving/Improving Schools Program
o f the Public Schools Accountability Act o f 1999 . Retrieved November
1, 2003, from www.edsource.org/pub fre.cfm
Ammar, S., Bifulco, Duncombe, W. & Wright, R. (2000). Identifying low-
performance public schools [Electronic Version]. Studies in
Educational Evaluation, 26, 259. Retrieved January 11, 2004, from
www-cpr.maxwell.svr.edu/efap/publications/identifving%201ow-
performance%20public%20schools.pdf
Amrein, A. & Berliner, D. (2002, March 28). High-stakes testing, uncertainty,
and student learning [Electronic Version]. Education Policy Analysis
Archives, 70(18). Retrieved February 6, 2003 from
http://epaa.asu.edu/epaa/vl0nl8/
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 5
Ananda, S. (2003). Rethinking issues o f alignment under no child left behind.
San Francisco, CA: West Ed.
Aronson, J., Zimmerman, J. & Carlos, L. (1996). Improving student
achievement by extending school: is it just a matter o f time? [Electronic
version]. San Francisco, CA: WestEd. Retrieved May 21,2002, from
http://web.wested.org/online pubs/timeandleaming/TAL PV.html
Asmul, S., Wilhehnsen, S., & Meistad, O. (1998). Psychological theories; A
brief survey o f the changing views o f learning. Retrieved June 9, 2004,
from http://www.uib.no/Peonle/sinia/CSCL/web struktur-4.htm
Au, K. (1992). Balanced literacy instruction: Addressing issues o f equity. HA:
University of Hawaii.
Austin, T. (1996). Goals 2000: The Clinton administration education program.
In D. Schugurensky (Ed.), History o f education: Selected moments o f
the 20th century [Electronic Version]. Retrieved June 21, 2004, from
www.nd.edu/~rbarger/www7/goals200.html
Barton, P. (2001). Raising achievement and reducing gaps: Reporting progress
toward goals for academic achievement. Washington, D.C.: National
Education Goals Panel.
Barton, P. (2003, October). Parsing the achievement gap: Baselines for
tracking progress [Electronic Version]. ETSReport. Retrieved
November 1, 2003, from www.ets.org/research/pic/parsing.pdf
Betts, J., Rice, L. & Zau, A. (2003). Determinants o f student achievement:
New evidence from San Diego [Electronic Version]. PPIC Research
Brief. Retrieved September 30, 2003, from
www.ppic.org/content/pubs/RB 803JBRB.pdf
Bloom B. (Ed.) (1989) Taxonomy o f educational objectives. Handbook I:
cognitive domain. NY: McKay.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 6
Bloom, B., Hastings, J. & Madaus, G. (1971). Handbook o f formative and
summative education o f student learning. New York: McGraw-Hill.
Boese, L., Burkhardt, R. Carstens, L., Devine, L., Gaffney, T., & Just, A.
(2000) Immediate Intervention/Underperforming Schools Program
(II/USP): How low performing schools in California are facing the
challenge o f improving student achievement. Sacramento: California
Department of Education: Division of Policy and Evaluation.
Bolman, L and Deal, T. (2001). Leading with soul. San Francisco, CA: Jossey-
Bass, 2001.
Bracey, G.W. (1995). Variance happens: Get over it! Technos, 4(3), 22-29.
Brennan, A. (2001). A comprehensive paper on staff development [Electronic
Version]. Articles & Resources On Educational Administration &
Supervision. Retrieved May 26, 2004, from
www.soencouragement.org/comprehensive-paper-on-staff-
development.htm
Brophy, J. (1999). Teaching. Educational practices series-1 [Electronic
Version]. Brussels, Belgium: International Academy of Education.
Retrieved June 5, 2003, from
www.ibe.unesco.org/Intemational/Publications/EducationalPractices/Ed
ucationalPracticesSeriesPdf/pracO 1 e.pdf
Brown, C., Hara, S., & Shepardson, B. (1998). What every educator should
know...about the changing social policy landscape and efforts to ensure
student success [Electronic version]. Council o f Chief State Officers,
Publication No. 15. Retrieved March 10, 2002, from
www.temple.edu/LSS/PubSeries98-15 .PDF
Burke, J. (1996, November). Restructuring in the classroom. Educational
Leadership, 54, 88-89.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 7
Butler, J. (1992). Staff Development'. Close-Up #12 [Electronic
Version].School Improvement Research Series (SIRS). Northwest
Regional Educational Laboratory. Retrieved May 25, 2004, from
www.nwrel.org/scnd/ sirs/6/cu 12 .html
California Department of Education (2002). The Academic Performance Index
(API): A six-year plan fo r development (2001-2006). A paper presented
to the State Board of Education April 25,2002. Sacramento: CDE,
Accountability Branch, Policy and Evaluation Division.
Center for Education Reform (1998). A nation still at risk: An education
manifesto. Washington D.C.: Author.
Champion, R. (2003, Spring). Taking measure. National Staff Development
Council, 24(2).
Cohen, D. (1996). Standards-based school reform: Policy, practice, and
performance. In H. Ladd (Ed.), Holding schools accountable (pp. 99-
127). Washington, DC: Brookings Institute.
Cohen, E. (2001). Know what kids think, know what kids want. Retrieved June
12, 2004, from http://www.funderstanding.com/vygotskv.cfm
Cohen, D. & Hill, H. (1998). Instructional policy and classroom performance:
The mathematics reform in California [Electronic Version]. CPRE
Research Report Series, RR-39. Retrieved November 1, 2003, from
www.cnre.org/Publications/rr39.pdf
Cohen, D. & Lowenberg-Ball, D (2000, April). Instructional innovation:
Reconsidering the story [Electronic Version]. CPRE Study of
Instructional Improvement Working Paper. Retrieved July 3, 2004,
from www.sii.soe.umich.edu/documents/ InstructionalInnovation.pdf.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 8
Cohen, M. & Steinberg, A. (2002, March 13). Rigor and relevance: Can policy
keep pace with changing practice in our high schools? Education Week,
21(26)., Retrieved November 1,2003, from www.edweek.org/ew/vol-
21 /26thiswk.htm
Coleman, J., Campbell, E., Hobson, C., McPartland, J., Mood, A., Weinfield,
F., & York, R. (1966). Equality o f educational opportunity. Washington,
DC: U.S. Government Printing Office.
Coley, R. (2003). Growth in school revisited: Achievement gains from the
fourth to the eighth grade [Electronic Version]. ETS Policy Information
Report. Retrieved May 9, 2004, from
www.ets.org/research/pic/growth2.pdf
Consortium for Policy Research Education. (1993, October). Developing
content standards: Creating a process for change. Consortium for Policy
Research Education (Ed.), CPRE Policy Briefs. Retrieved May 29,
2004, from http://www.cpre.org/publications
Corcoran, T. (1995). Helping teachers teach well: Transforming professional
development (CPRE Policy Brief). New Brunswick, NJ: Consortium for
Policy Research in Education. Retrieved November 11, 2003, from
http://www.cpre.org/Publications/rb 16.pdf
Costa, A. & Garmston, R. (2002). Cognitive coaching: A foundation for
renaissance schools [Electronic Version]. Norwood, MA: Christopher
Gordon Publishers. Retrieved May 25, 2004, from www.edu-
books.com/Cognitive Coaching A Foundation for Renaissance Scho
of
Cotton, K. (2002). The schooling practices that matter most. Association for
Supervision and Curriculum Development. Alexandria, VA.
CPRE (1995). Helping teachers teach well: Transforming professional
development [Electronic version]. Retrieved August 24, 2002, from
www.ed. gov/pubs/CPRE/t61 /t61 d.html.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 3 9
Creswell, J. (1998). Qualitative inquiry and research design: Choosing among
five traditions. Thousand Oaks, CA: Sage.
Cross , C. (1998, October 21). The standards wars: Some lessons learned.
Education Week, 18(8), 48.
Darling-Hammond, L. & McLaughlin, M. (1995, April). Policies that support
professional development in an era of reform [Electronic Version]. Phi
Delta Kappan, 76(8), 597-604. Retrieved April 26, 2004, from
http://www.middleweb.com/PDPolicv.html
Darling-Hammond, L. (1999, December). Teacher quality and student
achievement: A review o f state policy evidence [Electronic Version].
Center for the Study of Teaching and Policy Working Paper #R-99-l.
Retrieved May 9, 2004, from http://dents.washington.edu/ctpmail/
PDFs/LDH 1999.pdf
Darling-Hammond, L. (2003, February). Standards and assessment: Where we
are and what we need [Electronic Version]. Teachers College Record.
Retrieved November 5, 2003, from www.tcrecord.org/Content.asp?
ContentID=l 1109
Darling-Hammond, L., Hightower, A., Husbands. J., LaFors, J. & Young, V.
(2002, April 4). Building instructional quality: Inside-out, bottom-up,
and to-down perspectives on San Diego’ s school reform [Electronic
Version]. Paper presented at the annual meeting of the American
Educational Research Association in New Orleans, LA. Retrieved
January 11, 2004, from
http://depts.washington.edu/ctpmail/PDFs/InProgress/SDCS-Reform-
AERAdraft.pdf
Datnow, A. (2000). Power Politics in the Adoption of School Reform Models.
Educational Evaluation and Policy Analysis, 22(4), 357-374.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 0
David, J. (1990). Restructuring in progress: Lessons from pioneering districts.
In R.F. Elmore & associates (Eds.), Restructuring school: The next
generation o f education reform (pp.209-250). San Francisco: Jossey-
Bass.
Diamond, J. & Spillane, J. (2002). High stakes accountability in urban
elementary schools: Challenging or reproducing inequality?
Northwestern University Institute for Policy Research Working Paper
WP-02-22. Retrieved February 12, 2003, from
www.northwestem.edu/ipr/publications/papers/2002/WP-02-22.pdf
Dom, S. (1998, January). The political legacy of school accountability systems
[Electronic version]. Education Policy Analysis Archives, 5(1).
Retrieved March 10, 2002, from http://epaa.asu.edu/epaa/v6n 1 .html
Dossett, W., Hall, G. & Wallace, R., Jr. (1973). A developmental
conceptualization o f the adoption process within educational
institutions. Austin, TX: Research and Development Center for Teacher
Education, University of Texas.
Duffy, M. & Goertz, M. (2001, May). Assessment and accountability across the
50 states [Electronic Version]. CPRE Policy Briefs RB-33. Retrieved
April 13, 2004, from www.cpre.org/Publications/rb33.pdf
DuFour, R. (1991). The principal as staff developer. National Educational
Service. Bloomington, Indiana.
DuFour, R. (1997, Winter). Moving toward the school as a learning community
[Electronic Version]. Journal o f Staff Development, 18(1). Retrieved
April 13, 2004, from
www.nsdc.org/librarv/publications/isd/dufourl 81 .cfm
DuFour, R., & Eaker, R. (1992). Creating the new American school: A
principal’ s guide to school improvement. Bloomington, Indiana:
National Educational Service.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 1
DuFour, R., & Eaker, R. (1998). Professional learning communities at work:
Best practices fo r enhancing student achievement. Alexandria, VA:
Association for Supervision and Curriculum Development.
EdSource, ed. (2000, May). National accountability movement offers lessons
for California [Electronic version]. EdSource. Retrieved July 20,2002,
from www.edsource.org.
Edwards, B. & Frey, S. (2003, May). Show me the data [Electronic Version].
EdSource Report Retrieved November 1, 2003, from
www.edsource.org/pub abs 2003forum.cfm
Elmore, R. (1997, Fall). The politics of education reform. Issues in Science and
Technology, 14, 41-49.
Elmore, R. (2002). Bridging the gap between standards and achievement: The
imperative fo r professional development in education [Electronic
Version]. Albert Shanker Institute. Retrieved November 1, 2003, from
www.shankerinstitute.org/Downloads/Bridging Gap.pdf
Elmore, R. (2003). Knowing the right thing to do: School improvement and
performance-based accountability [Electronic Version]. CPRE: NGA
Center for Best Practices. Retrieved November 1, 2003, from
www.nga.org/cda/files/0803KNQWING.PDF
Elmore, R.F., & Burney, D. (1997a). Investing in teacher learning: Staff
development and instructional improvement in Community School
District #2, NYC. National College for School Leadership. Retrieved
July 30, 2004, from
www.teacherscollege.edu/nctaf/publications/Elmore.pdf
Elmore, R., & Burney, D. (1997b). School variation and systemic instructional
improvement in Community School District #2, New York City (1997).
University of Pittsburgh, HPLC Project, Learning Research and
Development Center.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 2
Elmore, R., & Burney, D. (1998). Continuous improvement in Community
District #2, New York City. Pennsylvania: University of Pittsburgh.
Elmore, R.F., & Burney, D. (1999). Investing in teacher learning: Staff
development and instructional improvement. In L. Darling-Hammond
& G. Sykes (Eds.), Teaching as the learning profession: Handbook o f
policy and practice (pp. 263-291). San Francisco: Jossey-Bass.
Ertmer, P. A., & Newby, T. J. (1993). Behaviorism, Cognitivism,
Constructivism: Comparing critical features from an Instructional
Design Perspective. Performance Improvement Quarterly, 6(4), 50-72.
Finn, J., Harman, P. & McRobbie (1998, August). Class size reduction:
Lessons learned from experience. WestEd Policy Brief #23. San
Francisco, CA: WestEd.
Frey, S. (2000). California Public School Accountability Act (PSAA)
evaluation: Findings and implication. American Institute for Research.
Sacramento: California Department of Education.
Fuhrman, S. (1994, September). Challenges in systemic education reform
[Electronic Version], CPRE Policy Briefs, 14. Retrieved November 1,
2003, from www.cpre.org/Publications/rb 14.pdf
Fullan, M. (2001). Leading in a culture o f change. San Francisco, CA: Jossey-
Bass Publishers.
Gagne, R. (1985). The conditions o f learning (4th ed.). New York: Holt,
Rinehart & Winston.
Garcia, E. (1999). Reforming education and its cultures. The American
Behavioral Scientist, 42(6), 10072-11091.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 3
Gilbert, S., Hightower, A., Husbands, J., Marsh, J., McLaughlin, M., Talbert, J.
& Young, V. (2002, April). Districts as change agents: levers for
system-wide instructional improvement [Electronic Version].
Presentation at the Annual Meeting of the American Educational
Research Association in New Orleans, LA. Retrieved April 13, 2004,
from http://depts.washington.edu/ctpmail/PDFs/InProgress/AERA-
DistrictsAsChange.pdf.
Goldberg, J. & Cole, B. (2002). Quality management in education: Building
excellence and equity in student performance. The Quality Management
Journal, 9(4), 8-22.
Goldring, E. & Hallinger, P. (1992). District control contexts and school
organizational processes. Paper presented meeting of the American
Educational Research Association, San Francisco, CA.
Grossman, P., Thompson, C. and Valencia, S. (2002). Focusing the concerns of
new teachers: the district as teacher educator. In A. Hightower, M.
Knapp, J. Marsh & M. McLaughlin (Eds.), School Districts and
Instructional Renewal (pp. 129-142). New York, NY: Teachers College
Press
Guskey, T. (1986). Staff development and the process of teacher change.
Educational Researcher, 15(5), 5-11.
Hall, G. (1978). The study o f teachers' concerns and consequent implications
fo r staff development. Austin, TX: Research and Development Center
for Teacher Education, The University of Texas.
Hall, G., George, A., & Rutherford, W. (1978). Stages o f Concern about the
innovation: The concept, verification, and implications. Austin, TX:
Southwest Educational Development Laboratory.
Hall, G., George, A., & Rutherford, W. (1998). Measuring stages o f concern
about the innovation: A manual fo r use o f the SoC
Questionnaire. Austin, TX: Southwest Educational Development
Laboratory.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 4
Hall, G. & Hord, S. (2001). Implementing Change: Patterns, principles, and
potholes. Boston: Allyn and Bacon.
Hall, G., Hord, S., Huling-Austin, L. & Rutherford, W. (1998). Taking Charge
o f Change, 3rd edition. Austin, TX: Southwest Educational
Development Laboratory.
Hall, D., Wiener, R., & Carey, K. (2003). What new “ AYP” information tells us
about schools, states, and public education [Electronic Version].
Washington, DC: The Education Trust. Retrieved November 17, 2003,
from www2.edtrust.org/NR/rdonlyres/4B9BF8DE-987A-40630B750-
6D67607E7205/0/NewAYP.t>df
Hambleton, R. (1999, October 15). Setting performance standards on
assessment and criteria fo r evaluating the process [Electronic Version].
Presentation at the Edward F. Reidy Interactive Lecture Series at
Providence, RI. Retrieved May 9, 2004, from
www.nciea.org/publications/ SetStandards Hambleton99.pdf
Hanushek, E. (1997, May). Applying performance incentives to schools for
disadvantaged populations [Electronic Version]. Education and Urban
Society. Retrieved November 1, 2003, from www.umich.edu/
~ps vcours/5 61 /incenti v.pdf
Hanushek, E. (2003, Spring). Lost opportunity [Electronic Version]. Hoover
Institution’s Education Next. Retrieved November 27, 2003, from
www.educationnext.org/20032/
Haycock, K. & Jerald, C. Huang, S. (2001). Closing the gap: Done in a decade.
The Education Trust 5(2).
Heubert, J. & Hauser, R. (Eds.) (1999). High stakes: Testing fo r tracking,
promotion, and graduation. Washington, DC: National Academy Press
[On-line]. Available: http://www.nap.edu/html/highstakes
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 5
Hiebert, J. & Stigler, J. (1997, September). Understanding and improving
classroom mathematics instruction: An overview of the TIMSS video
study [Electronic Version]. Phi Delta Kappan, 14-21. Retrieved July 3,
2004, from http://lsc-net.terc.edu/do.cfm/paper/8037/show/use set-
intl focus.
Hightower, A. (2002). San Diego’s boom: Systemic instructional change in the
central office and schools. In A. Hightower & M. Knapp & J. Marsh &
M. McLaughlin (Eds.), School districts and instructional renewal (pp.
76-93). New York, NY: Teachers College Press.
Hightower, A., Knapp, M., Marsh, J. & McLaughlin, M. (2002). School
districts and instructional renewal. New York, NY: Teachers College
Press.
Hirsch, E. (1996). The schools we need and why we don’ t have them. NY:
Doubleday.
Hirsch, E., Knapp, M. & Koppich, J. (2001, February). Revisiting what states
are doing to improve the quality o f teaching: an update on patterns and
trends [Electronic Version]. CTP Working Paper #W-01-1. Retrieved
April 13, 2004, from http://depts.Washington.edu/ctpmail/PDFs/States-
HKK-02-2001.pdf
Hord, S. & Roy, P. (2004, Spring). Innovation Configurations chart a
measured course toward change [Electronic Version]. Retrieved
December 3, 2004, from www.nsdc.org/librarv/publications/
isd/rov252.cfm. Journal o f Staff Development, 25(2).
Hord, S. & Rutherford, W. & Huling-Austin, L. & Hall, G. (1987). Taking
charge o f change. Alexandria, Va.: ASCD Press.
Huberman, M. & Miles, M. (1984). Innovation up close: How school
improvement works. New York: Plenum, Selected Pages.
Huberman, M. & Miles, M. (1994). Qualitative data analysis: An expanded
sourcebook. Thousand Oaks, CA: Sage Publications.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 6
Hunter, M. (1969). Improved instruction. Carthage, IL: Corwin Press.
Hunter, M. (1984). Knowing, teaching, and supervising. In P. Hosford (Ed.),
Using what we know about teaching (pp. 169-192). Alexandria, VA:
Association for Supervision and Curriculum development.
Hurst, D., Tan., A., Meek, A., & Sellers, J. (2003, July). Overview and
inventory o f state education reforms: 1990 to 2000 [Electronic
Version]. Washington, DC: NCES. Retrieved November 1, 2003, from
http://nces.ed.gov/pubsearch/pubsinfo.asp7pubid~2Q03020
Huston, A. (Ed.). (1992). Children in poverty. NY: Cambridge University
Press.
Izumi, L., & Cox, M. (2003, August). California education report
card: Index o f leading education indicators, Third edition [Electronic
Version]. San Francisco: Pacific Research Institute. Retrieved
September 30, 2003, from www.pacificresearch.org/pub/sab/educat/
03 ed index/EducationIndex03 hifi.pdf
Joyce, B. (Ed.). (1990). Thoughts on staff development [The 1990 ASCD
Yearbook]. In Changing School Culture through Staff Development.
Virginia: Association for Supervision of Curriculum Development.
Joyce, B., & Showers, B. (1980, February). Improving inservice training: the
message of research. Educational Leadership, 37(5), 379-385.
Joyce, B. & Showers, B. (2002). Student achievement through staff
development (3rd ed.). Alexandria, VA: Association for Supervision and
Curriculum Development.
Kilzer, K. (2001). I know how to improve the performance o f the school system
[Electronic Version]. Retrieved on May 25, 2004, from
www.mindspring.com/’kkilzer/Schools21 DecO 1 .htm
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 7
Klein, S., Medrich, E. & Perex-Fereiro, V. (1996, October). Fitting the Pieces:
Education reform that works [Electronic Version], Retrieved November
1, 2003, from
www.ncela.gwu.edu/iasconferences/archives/1997/institutes/change/fitti
ngL
LaRue, A. (1996, August). The changing face o f teacher tenure [Electronic
Version]. American Federation of Teachers’ Report. Retrieved July 3,
2004, from www.aft.org/research/reports/tenure/Laruep.htm
Lee, V., Brooks-Gunn, J. & Shnur, E. (1988). Does Head Start work? A 1-year
follow-up comparison of disadvantaged children attending Head Start,
no preschool, and other preschool programs. Developmental Psychology,
24, 210-222.
Loveless, T. & Ravitch, D. (2000). Broken promises: What the federal
government can do to improve public education. Brookings Review,
18(2), 18
Mager, R. (1962). Preparing instructional objectives. Belmont CA: Fearon
Publishers.
Mager, R. (1997). Preparing instructional objectives (3rd ed). Atlanta, GA:
Center for Effective Performance.
Maroney, O. (1998, September). Standards, tracking and the reform of our
public schools [Electronic Version]. Intercultural Development
Research Association Newsletter, 3. Retrieved November 1, 2003, from
www.idra.org/Newslttr/1998/Sep/Art3
Marsh, D. (1995). Restructuring fo r results: High performance management in
the Edmonton public school. Washington, DC: National Center on
Education and the Economy.
Marsh, D & Jordan-Marsh, M. (1985, April). Addressing teachers ’personal
concerns in staff development efforts. Paper presented at the annual
meeting of the American Educational Research Association, Chicago.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 8
Marsh, J. (2000, September). Connecting districts to the policy dialogue: a
review o f literature on the relationship o f districts with states, schools,
and communities [Electronic Version]. CTP Working Paper #W-00-l.
Retrieved April 13, 2004, from http://depts.Washington.edu/ctpmail/
PDFs/District Lit.pdf
Marsh, J. (2002). How districts relate to states, schools, and communities: A
review of emerging literature. In A. Hightower & M. Knapp & J. Marsh
& M. McLaughlin (Eds.), School districts and instructional renewal
(pp. 25-40). New York, NY: Teachers College Press
Marsh, J., Kerr, K., Ikemoto, G., & Darilek, H. (2004, April). The role o f an
intermediary organization in district instructional improvement: Early
experiences and lessons about the Institute for Learning. Working paper
presented at the annual meeting of the American Education Research
Association. San Diego, California.
Marzano, R. (2003). What works in schools: translating research into action.
Alexandria, VA: Association for Supervision and Curriculum
Development.
Marzano, R. & Kendall, J. (1998). A wash in a sea o f standards. Aurora, CO:
Mid-continent Research for Education and Learning.
Marzano, R., Pickering, D., & Pollock, J. (2001). Classroom instruction that
works: Research-based strategies fo r increasing student achievement.
Alexandria, VA: Association for Supervision and Curriculum
Development.
Maschke, K. L. (1999). Cognitivism. Retrieved June 19, 2004, from
www.gsu.edu/~mstswh/courses/paners/newpage9.htm
Massell, D. (1998). State strategies for building capacity in education: Progress
and continuing challenges [Electronic Version]. CPRE Research Report
Series, RR-041. Retrieved January 11, 2004, from
www.cpre.org/Publications/rr41 .pdf
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 4 9
Massell, D. (2000, September). The district role in building capacity: Four
strategies [Electronic Version]. Consortium for Policy Research in
Education: Policy Briefs, RB
Massell, D. & Goertz, M. (2002). District strategies for building instructional
capacity. In A. Hightower & M. Knapp & J. Marsh & M. McLaughlin
(Eds.), School districts and instructional renewal (pp. 43-60). New
York, NY: Teachers College Press.
Mayer, R. (1998). Cognitive theory for education: What teachers need to know.
In N. Lambert & B. McCombs (Eds.), How students learn: Reforming
schools through learner-centered education. Washington, D.C.:
American Psychological Association.
Mazzeo, C. (2001, June). Frameworks of state: Assessment policy in historical
perspective [Electronic Version]. Teachers College Record, 103(3),
367. Retrieved October 29, 2002, from http://newfirstsearch.oclc.org/
McLaughlin, M. & Marsh, D. (1978). Staff development and school change.
Teachers College Record, 80(1), 69-94.
McLaughlin, M. & Talbert, J. (2002). Reforming districts. In A. Hightower &
M. Knapp & J. Marsh & M. McLaughlin (Eds.), School districts and
instructional renewal (pp. 173-192). New York, NY: Teachers College
Press
Mid-continent Research for Education and Learning (2001). Leadership for
school improvement [Electronic version]. Aurora, CO: Author.
Retrieved May 28, 2002, from www.mcrel. Org
Mumane & Levy, F. (1998, March). Standards, information, and the demand
for student achievement [Electronic Version]. FRBNYEconomic Policy
Review. Retrieved July 28, 2002, from http://ideas.renec.Org/a/fip/
fednep/vl 998imarp 117-124nv.4no. 1 .html
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5 0
National Center for Educational Statistics (1992). International mathematics
and science assessments: What have we learned? [Electronic Version].
U. S. Department of Education NCES 1992-011. Retrieved July 3,
2004, from http://nces.ed.gov/pubs92/web/92011 .asp.
National Center for Educational Statistics (1999). Highlights from TIMSS:
Overview and key findings across grade levels [Electronic Version].
Report NCES 1999-081 from the Office of Educational Research and
Improvement, U.S. Department of Education. Retrieved July 3, 2004,
from http ://nces.ed.gov/pubs99/1999081 .pdf.
National Center for Educational Statistics (2000a). Highlights from the Third
International Mathematics and Science Study [Electronic Version]. U.
S. Department of Education NCES 2001-027. Retrieved July 3, 2004,
from http://nces.ed.gov/pubs2000/ 2000094.pdf..
National Center for Educational Statistics (2000b). NAEP 1999 trends in
academic progress: Three decades o f student performance [Electronic
Version]. U.S. Department of Education Report NCES 2000-469.
Retrieved July 3,2004, from http://nces.ed.gov/nationsreportcard/
pdf/mainl 999/2000469.pdf.
National Center for Educational Statistics (2001). Highlights from the Third
International Mathematics and Science Study—Repeat [Electronic
Version]. U.S. Department of Education Report NCES 2001-027.
Retrieved July 3, 2004, from http ://nces.ed. gov/pubs2001/2001027.pdf.
National Center for Education Statistics, U. S. Department of Education. (2003,
November). The Nation’ s report card: Mathematics highlights 2003._
Washington D.C.: U.S. Government Printing Office.
Northwest Regional educational Laboratory. (1990). Onward to excellence:
Making schools more effective. Portland, Oregon: Author.
Oakes, J. & Wells, A. (1996). Potential pitfalls of systemic reform: Early
lessons for research on detracking [Special issue]. Sociology o f
Education, 69, 135-143.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5 1
PACE (2003) California Public School Accountability Act (PSA A): Evaluation
findings and implications. American Institute for Research. Sacramento,
CA: California Department of Education.
Pardon, Y. (1992). The effect of strategy instruction on bilingual student’s
cognitive strategy use in reading. Bilingual Research Journal, 16, 3-4.
Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.).
Thousand Oaks, CA: Sage.
Pipho, C. (1999, March). High-stakes accountability [Electronic Version], Phi
Delta Kappan, 80(1), 485. Retrieved November 1, 2003, from
www.umi ch. edu/~ps vcours/5 61/ account7 .htm
Ramey, C., Burchinal, M., Campbell, F., Miller-Johnson, S., Pungello, E..
(2001). The development of cognitive and academic abilities: Growth
curves from an early childhood educational experiment. Developmental
Psychology, 37, 231-242.
Ramey, C., Burchinal, M., Campbell, F., Miller-Johnson, S., Pungello, E.
(2002). Early childhood education: young adult outcomes from the
Abecedarian Project. Applied Developmental Science 2002; 6(l):42-57.
Ramey, S. & Ramey, C. (1998). The transition to school: Concepts, practices,
and needed research. American Institutes for Research: Palo Alto, CA.
Reeves, D. (1996-1998) Making standards work: How to implement standards-
based assessments in the classroom, school, and district. Denver, CO:
Center for Performance Assessment.
Reeves, D. (2001) 101 questions & answers about standards, assessment, and
accountability. Denver, CO: Advanced Learning Press.
Regional Educational Laboratory Network. (2000, November). Implementing
education reform: Strategies used by states, districts, and schools.
Aurora, CO: Mid-continent Research for Education and Learning.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5 2
Reigeluth, C. (2004, April 13). Theoretical foundations fo r systemic
transformation ofK-12 education [Electronic Version], Paper presented
at the annual meeting of the American Educational Research
Association Conference, San Diego, CA. Retrieved April 26,2004,
from http://convention.allacademic.com/aera20Q4/AERA papers/
AERA sess 3017 10715a.PDF
Resnick, L. B. (1999). Making America smarter. Education Week Century
Series, 75(40), 38-40.
Resnick, L. B., & Hall, M. W. (1998). Learning organizations for sustainable
education reform. Daedalus, 127, 89-118.
Resnick, L. B., & Hall, M. W. (2000, February). Principles o f learning fo r
effort-based education. Retrieved June 6, 2004, from
http://www.curriculum.dpskl2.org/pol prin leam.html
Reyhner, J. (2003, January 29, 2003). The reading wars: Phonics versus whole
language. Retrieved June 20, 2004, from http://i an.ucc.nau.edu/
~iar/Reading Wars.html
Richardson, V. (1994). Conducting research on practice. Educational
Researcher, 23(5), 5-10.
Rogosa, D. (2000) Interpretive notes fo r the academic performance index.
Sacramento, CA: California Department of Education.
Rosenshine, B. (1996). Advances in research on instruction. J. Lloyd, E.
Kameanui, and D. Chard. (Eds.), Issues in Educating students with
disabilities. Manwah, NJ: Lawrence Erlbaum.
Rosenshine, B. & Stevens, R. (1986). Teaching functions. In M. Wittrock (Ed.)
Handbook o f Research on Teaching, 3rd Edition. New York,
Macmillan.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5 3
Rowan, B., & Correnti, R. & Miller R. (2002, November/December). What
large-scale, survey research tells us about teacher effects on students
achievement: Insights from the Prospects Study of elementary schools
[Electronic Version]. Teachers College Record, 104, 1525.
Rutherford, P. (1999). Instruction fo r all students. Alexandria, VA: Just ASK.
Scherer, M. (1996). A Nation at Risk: The imperative for educational reform,
1983. In D. Schugurensky (Ed.), History o f education: Selected
moments o f the 20th century [Electronic Version]. Retrieved June 21,
2004, from www.nd.edu/~rbarger/www7/nationrs.html
Schmoker, M. (1999). Results: The key to continuous school improvement.
Alexandria, VA: Association for Supervision and Curriculum
Development.
Schmoker, M. (2003). Tipping point: From feckless reform to substantive
instructional improvement [Electronic Version] .Phi Delta Kappan
85(6). Retrieved July 3, 2004, from http://www.pdkintl.org/
kappan/k0402sch.htm
Shanker, A. (1997). Where we stand: ninety two hours. American Educator,
2/(1), 33-34.
SIG, AERA Cultural-Historical Research Interactive Symposium: (2004, April
15). Dismantling the achievement gap: Some issues fo r policy and
socio-cultural researchers [Electronic Version]. Proposal for paper
presented at American Educational Research Association Conference,
San Diego, CA.
Sizer, T. (1992). School reform: what’ s missing? World monitor: Sirs
Researcher. [Electronic Version] Retrieved May 29, 2004,
http://webmail.aol.com/msgview.adp?folder=su5ctlg=9197656
Skinner, B.F. (1954). The science of learning and the art of teaching. Harvard
Educational Review, 24, 86-97.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5 4
Skrla, L., Scheurich, J., & Johnson, J. (2001, May). Toward a new consensus
on high academic achievement for all children [Electronic Version],
Education and Urban Society, 33(3), 221. Retrieved July 28, 2002, from
www.bol.ucla.edu/~mzavala/standards reform link.htm
St. John, M. (1999). The nature o f teacher leadership: Lessons learned from
the California Subject Matter Projects [Electronic Version]. Inverness
Research Associates. Retrieved May 25, 2004, from http://lsc-
net.terc.edu/do.cfm/paper/8123/show/page-5/use set-ldrshp
Stein, M.K., (2003, Winter). Leadership content knowledge. Educational
Evaluation and Policy Analysis, 25(4), 423-448.
Stein, M.K. & D’Amico, L. (2002). The district as a professional learning
laboratory. In A. Hightower & M. Knapp & J. Marsh & M. McLaughlin
(Eds.), School districts and instructional renewal (pp. 61-75). New
York, NY: Teachers College Press.
Stigler, J.W., & Hiebert, J. (1999). The teaching gap: Best ideas from the
world's teachers fo r improving education in the classroom. New York:
Free Press.
Supovitz, J. (chair). (2004, April 13). States demand outcomes: Do high
schools and districts respond? [Electronic Version]. Paper presented at
the annual meeting of the American Educational Research Association
Conference, San Diego, CA. Retrieved April 26, 2004, from
http://convention.allacademic.com/aera20Q4/AERA papers/AERA sess
1013 10370a.PDF
Taylor, F. (1911/1998). The principles o f scientific management. NY: Dover.
Technical Design Group (2000). Construction o f California’ s 1999 school
characteristics index and similar schools rank. Sacramento, CA:
California Department of Education.
Thorndike, E. (1913). Educational psychology: The psychology o f learning.
New York: Teachers College Press.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5 5
TIMSS International Study Center (1997). TIMSS highlights from the primary
grades. Boston, MA: Boston College.
Togneri, W. (2003, March). Beyond islands o f excellence: What districts can
do to improve instructional and achievement in all schools [Electronic
Version]. Learning First Alliance Paper. Retrieved May 6, 2004,
from www.leamingfirst.org/lfa-web/rp?na=doc&docId=62
Troen, V. & Boles, K. (2003. Who’ s teaching your children? Why the teacher
crisis is worse than you think and what can be done about it. New
Haven, CT: Yale University Press. Retrieved may 26, 2004 from
http://64.233.167.104/search?q=cache:b5AZStCW3cli:www.vale.edu/v
un/pdf/097417 fron
Tucker, M.S., & Codding, J.B. (1998). Standards for our schools: How to set
them and reach them. San Francisco: Jossey-Bass Publishers.
Tyack, D. (2002). Forgotten players: How local school districts shaped
American education. In A. Hightower & M. Knapp & J. Marsh & M.
McLaughlin (Eds.), School districts and instructional renewal (pp. 9-
24). New York, NY: Teachers College Press
University of Virginia (2002). The justification and procedures for
various instructional models [Electronic Version]. Retrieved November
1,2003, from www.people.virginia.edu/~ehw2d/
instructionalmodels.html
Vinovskis, M.A. (1996). The changing role of the federal government in
educational research and statistics. History o f Education Quarterly,
36(2), 111-128. Retrieved May 30, 2004, from http://links.istor.org/sici
Wagner, T. (1993, September). Systemic change: Rethinking the purpose of
school. Educational Leadership, 24.
Walker, S. (2000). High-stakes testing: Too much? Too soon? [Electronic
version]. State Education Leader, 75(1). Retrieved March 10, 2002,
fromwww.ecs.org/clearinghouse/16/23/1623.htm
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5 6
Wang, M., Haertel, G., & Walberg, H. (1993, December/January). What helps
students learn? Educational Leadership, 51(4), 74-80.
Wayne, J. (1997). The unavoidable intervention of educational research: A
framework for reconsidering researcher-practitioner cooperation.
Educational Researcher, 26(1), 13-22.
Whitmire, R. (1997). The education gap: Why minority test scores lag. USA
Today.
Wiburg, K. (1995). An historical perspective on instructional design: Is it
time to exchange Skinner’s teaching machine for Dewey’s toolbox
[Electronic Version]. In proceedings of The First International
Conference on Computer Support fo r Collaborative Learning (pp.
385-391). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Retrieved
June 1, 2004, from
http://portal.acm.org/citation.cfrn?id=222863&imp=cit&coll=portal&dl
=acm
Wiggins, G. & McTighe, J. (1998). Understanding by design. Columbus, Ohio:
Merrill Prentice Hall.
Williams, B. (2003), Closing the Achievement Gap. Alexandria, VA: ASCD
Woolfolk, A. (2001). Educational psychology (8th ed.). Needlam Heights, MA:
Allyn and Bacon.
Yin, R. (2003). Case study research: Design and methods (3rd ed.). Thousand
Oaks, CA: Sage Publications.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDICES
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5 8
School-Level Case Study Guide
I. Case Study Guide Overview
II. Data Collection Chart
III. Instrumentation Tools
1. Principal Interview Guide
2. Leadership Team Member Interview Guide
3. Lead Teacher Interview Guide
4. Teacher Interview Guide
5. Innovation Configuration
6. School Level Survey
7. Teacher Questionnaire
8. Document Review Guide
Data collection schedule
Phase I Document Review Guide [initial and on-going]
Phase II Interviews (principal, leadership team members, and lead teachers)
Phase III School Level Survey and Teacher Questionnaire
Phase IV Interviews (principal and teachers)
Innovation Configuration
V. Conceptual Framework
A. The district’s design for improving teaching and learning.
B. School level efforts to facilitate the implementation of the district’s
design.
C. Extent of implementation of the district’s design at the school and
classroom level.
D. Effectiveness of the district’s design at the school and classroom level.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 5 9
Appendix A
Case Study Guide
The purpose of the Case Study Guide is to provide general and specific
directions to the researchers for collecting data according to the methodologies
presented in Chapter 3 and for presenting data as part of the findings in Chapter 4.
The case study guide is in accordance with the conceptual frameworks of this study,
which describes the examination of the design and implementation of district-led
reform efforts at the site and classroom levels.
When collecting data, the conceptual frameworks and the guiding questions
found therein should serve as the focus both when conducting formal interviews
and while administering surveys at the site and classroom level. It is not needed to
ask each of the guiding questions during any given interview, but rather use the
guiding questions and conceptual frameworks to be certain of getting the pertinent
information as specified in the data collection chart.
This case study should be included as a major part of Chapter 3 of the
dissertation. The headings and questions from the conceptual frameworks may
serve as a guide for data collection and presentation of findings.
Instrumentation Tools
Staff Interviews
Formal interviews will be conducted with the site principal, leadership team
members, lead teachers, and classroom teachers. For each interview, the researcher
will obtain relevant background information about the participant. The research
will include the participant’s title, years of experience, and years at current site.
Researchers will use a corresponding Interview Guide to ensure that the interview
questions are aligned with the research questions. Interviews will be conducted
according to the model as developed in EDPA 612, Qualitative Methods in
Educational Research.
Innovation Configuration
At the close of each teacher interview, the researcher will ask the participant
to self-report his/her individual extent of implementation of the district’s design on
a chart called an Innovation Configuration. The Innovation Configuration is
essentially an implementation rubric specific to the district’s design which outlines
three levels of implementation; fully implemented, partially implemented, and just
getting started. The Innovation Configuration will enable the researcher to
determine the extent of classroom implementation of each element of the district’s
design.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6 0
Surveys
Selected certificated personnel will be asked to complete the School Level
Survey. The School Level Survey includes a brief introduction, directions for
completing the survey, an assurance of anonymity, eleven demographic data
questions and forty-four survey questions. The School Level Survey is intended to
measure the degree of awareness about the district design for improving teaching
and learning. The survey questions focus on participants’ perceptions about the
district’s design & implementation strategies, school level efforts in implementing
the design, the extent of implementation, and the effectiveness of the
implementation.
Teachers will be asked to complete a second survey called the Teacher
Questionnaire. The Teacher Questionnaire is a thirty-six item questionnaire based
on the Stages of Concern. This questionnaire is intended to show teachers’
knowledge about the district design and comfort level with implementing reform
strategies. The Stages of Concern, Teacher Questionnaire, will also reveal
teachers’ extent of implementation.
Document Review Guide
The Document Review Guide will be used to align district documents to the
research questions developed in the study, addressing design, factors supporting
design, extent of implementation and effectiveness of design and implementation.
The Document Review Guide is essentially an organization tool, which will enable
the researcher to remain focused on the research questions and maintain documents
in an organized fashion.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6 1
Data Collection Chart
Data Collection
Instruments
R Q 1:
District Design
RQ2:
School Level
Effort
R Q 3:
Extent of
Implementation
RQ4:
Effectiveness of
Implementation
C ase Study Guide Components
Principal
Interview Guide ;
X X X
Leadership Team
Member
Interview Guide :
X X X
Lead Teacher
Interview Guide
X X X
Teacher
Interview Guide
X X X X
Innovation
Configuration
(self-report)
X
School Level
Survey
X X X X
Teacher
Questionnaire
(SOC)
X
Document
Review Guide
X X X X
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6 2
Appendix B
Data Collection Schedule
Phase I: Document Analysis [Initial & Ongoing]
Location: Off-site [internet research & information retrieved from district level
group]
Timeline: Before the first day of interviews and ongoing throughout each
phase of data collection.
The researcher will begin by retrieving and analyzing pertinent information
about the district such as district and school student demographic data (SES status,
ethnicity, language proficiency) and staff data (number of fully credentialed
teachers). This will enable the researcher to obtain a holistic view of the context
being studied. In order to have a comprehensive understanding of the district’s
design for improving teaching and learning, the researcher will study official
district documents describing reform efforts. The Document Review Guide will be
utilized to ensure the information obtained addresses the research questions.
After the initial document analysis, the researcher will contact the principal
of the school to determine the logistics for conducting interviews and administering
surveys. The researcher will also provide the principal with copies of all of the
instruments being used for the study. At this time the researcher will also ask the
principal to identify two leadership team members and four lead teachers who can
be interviewed by the researcher.
Phase IP . Conducting the First Round of Interviews
Location: On-site
Timeline: Approximately 7 Hours (1 -2 days at the school site)
The researcher will conduct approximately seven one-hour interviews:
1 Principal
2 Leadership Team Members (assistant principals, if applicable)
4 Lead Teachers (instructional coaches or department chairpersons)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6 3
The researcher will begin by interviewing the principal. The researcher will
then conduct six more interviews (2 - leadership team members & 4 - lead
teachers). The number of interviews may change based on the size and
configuration of the school being studied. The participants of this round of
interviews will be purposefully selected by the principal. Purposeful sampling will
be used to ensure that the participants are directly involved with the site-level
implementation of the district’s design for improving teaching and learning. In
preparing to conduct the interviews, the researcher will study Conceptual
Frameworks A, B, & D. During the interviews the researcher will use the Principal
Interview Guide, the Leadership Team Member Guide and the Lead Teacher Guide,
which outline key areas of focus directly aligned with the research questions.
Phase III: Administering Surveys
Location: On-site
Timeline: Distribution and collection - 1 hour.
The researcher will administer two surveys:
1. School Level Survey - 44 items (administered to selected certificated
staff)
2. Teacher Questionnaire - 36 items (administered to classroom teachers
only)
The researcher will distribute the surveys through staff mailboxes. Selected
certificated personnel will receive the appropriate survey(s) along with a cover
letter, briefly describing the study and outlining directions for completing and
returning the surveys, and an envelope for returning the survey(s). The researcher
intends to address the school staff, during a faculty meeting in which the researcher
will introduce himself or herself, describe the study, assure confidentiality and
anonymity, and outline the directions for completing and returning the surveys.
Participants will be given a three-day window in which to complete and return the
surveys.
Survey collection procedure may include either a self addressed stamped
envelope to be mailed to the researcher or a collection box in the main office.
Survey envelopes will be pre-numbered to ensure accurate record of the response
rate while maintaining anonymity. After the survey deadline, the researcher will
determine which surveys were not returned and send out friendly reminders. The
researcher will repeat this process until he or she obtains at least a 70% response
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6 4
rate. The researcher will work directly with the principal or administrative designee
to further develop the logistics of administering the surveys.
Phase IV: Interviews (Principal & teachers) & Innovation Configuration
Location: On-site
Timeline: Approximately 7 Hours (1 -2 days at the school site)
The researcher will conduct seven more interviews:
1 Principal
6 Teachers
The researcher will begin this second round of interviews by interviewing
six classroom teachers. The participants of the teacher interviews will be randomly
selected within the predetermined criteria range to ensure a representative sample.
The interviews will take place during the school day. The researcher will also work
with the principal or administrative designee to develop the logistics for the 2n d
round of interviews. In preparing for the teacher interviews, the researcher will use
Conceptual Frameworks A,B, C, and D. During the teacher interviews the
researcher will use the Teacher Interview Guide to focus the discussion on all four
research questions. At the conclusion of the interview, the researcher will ask each
teacher to self-report his/her extent of implementation on the Innovation
Configuration.
The researcher will also conduct a second interview of the school principal
to ask any questions that weren’t addressed during the first three stages of the data
collection. The principal will also be given an opportunity to provide any final
thoughts.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6 5
Appendix C
Conceptual Framework A
The district’s design for improving teaching and learning
• What is the district’s design for improving teaching and learning?
o What elements are included in the design?
o What is the fundamental pedagogy of the design?
o Does the design include strategies and programs to be used?
o How detailed or prescriptive is the design?
o Does the design take into account subgroup accountability and
closing the achievement gap?
• What is the district’s change strategy?
o incremental or major changes
o anchoring reform design to traditional structure or restructuring
district office
o top down approach - collaborative approach
• How was the design communicated to the staff, parents, students, and
the public?
o How much detail of the plan was communicated?
o How were the changed policies and strategies actually
disseminated?
• How does the district define standards-based instruction?
o How does the district define quality instruction?
o How do district policies coherently support this vision of quality
teaching?
• What are the intended outcomes of the instructional improvement?
o What benchmarks demonstrate realization of these outcomes?
o What mechanisms are built into the design to provide feedback
on progress towards goals?
Instruments Aligned to Conceptual Framework A:
Principal Interview Guide - Leadership Team Member Interview Guide - Lead
Teacher Interview Guide - Teacher Interview Guide - School Level Survey -
Document Review Guide
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6 6
Appendix D
Conceptual Framework B
School level efforts to facilitate the implementation of the district’s design
• What was the site level plan for implementing the district’s design?
o What choices were made at the district level?
o What decisions were the schools allowed to make?
o Who was involved in implementing the reform efforts (one person
or team approach)
o How was the staff involved?
o What was the level of community and parent involvement?
• What site level resources were needed to effectively implement the
reform design?
o Human resources / Fiscal resources / Physical resources
• How was the reform design communicated to staff members?
o Were structures developed to facilitate communication of
information?
■ Formal or informal?
■ Horizontally or Vertically?
■ Site-based or District-level?
o How did teachers learn about the reform?
■ How was the importance of the reform communicated? By
whom?
■ Did the district foster a shared understanding of goals?
• What kind of professional development did the staff receive?
o How has professional development impacted the design?
■ Content? Opportunities? Delivery? Concepts of Teacher as
Learner?
o Who is involved with providing professional development?
■ Is professional development general or content specific?
■ How is professional development funded?
• What kind of support is offered to staff members?
o What type of support do you provide for experienced and new
teachers?
o How was data used to signal capacity issues?
Instruments Aligned to Conceptual Framework B:
Principal Interview Guide - Leadership Team Member Interview Guide - Lead
Teacher Interview Guide - Teacher Interview Guide - School Level Survey -
Document Review Guide
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
267
Appendix E
Conceptual Framework C
Extent of implementation of the district’s design
• How are the design and strategies monitored for extent of
implementation?
• How does the site leadership team obtain feedback from all
departments?
• What data are generated?
o Where are the data stored?
o What is done with the data?
■ Is this information reported to the district?
■ Are the results communicated to the teachers? How?
■ Is the information disaggregated?
o What decisions are made based upon this information?
• How much of the change has permeated throughout the school?
o Is it in every classroom at the level? At the desired level?
• What are some of the explanations for why some teachers may not be
fully implementing design?
o How would you get them to change?
o What kinds of support, information, or professional development
do teachers still need to fully implement the design?
• Have the strategies been modified? What process was/will be used?
• How did fiscal, human, and physical resources promote or inhibit the
extent of implementation?
o How does the capacity of teachers, schools, and the organization
compare to before the reform?
• How has the reform affected equity throughout the system?
o Between schools & among student groups?
• How has the learning culture changed as a result of the design?
Instruments Aligned to Conceptual Framework C:
Teacher Interview Guide - Innovation Configuration - School Level Survey -
Teacher Questionnaire - Document Review Guide
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6 8
Appendix F
Conceptual Framework D
Effectiveness of Innovation
• What was the stakeholders’ perspective about impact of the
improvement effort (school level, classroom level, student level)?
o Site Leadership Team
o Teacher
o Parents & Community Members
• Was the design cost effective?
o How will the reform be sustained over time?
• Was the design time efficient?
o Was the roll out of expectations realistic?
• Did the impact rise to the intentions of the design?
o What promoted or inhibited the realization of the intended
outcomes?
Instruments Aligned to Conceptual Framework D:
Principal Interview Guide - Leadership Team Member Interview Guide -
Lead Teacher Interview Guide - Teacher Interview Guide - School Level Survey -
Document Review Guide
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 6 9
Appendix G
Principal Interview Guide
1. Principal background
• years of experience
• years at the district/site
• educational background
• recent training
2. School Data
• School configuration
• Student population
• Demographics
• API/AYP
• Special programs
3. Principal’s view of district involvement
(RQ 1. What is the district design for improving teaching/learning?)
• Describe the district plan.
• How was the district plan developed?
• Who was involved in creating the district plan? (what was the role
of the superintendent and school board? Were site level people
involved?)
• Were forces inside the district such as the board, the superintendent,
or another person or group the initiator of the plan?
• What was their motivation for creating the plan?
• What are the components of the district plan?
• From your perspective what are the plans’ strong points?
• From your perspective what do you think the plan is missing?
• How was the plan communicated to the school site administrators?
• What training accompanied the plan for school leaders/teacher
leaders?
• What district resources were allocated to support the plan?
• Who funded the development of the plan?
• Who funded the implementation of the plan?
• Would your plan have been different if you had more funds?
• Were individual school budgets affected by the decisions made by
the developers of the plan?
• Did the sites have a say in that?
• Who controls the budgeting and funding of the plan?
• How does the district assess the success of the plan?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 7 0
4. Specific administrator activities at the school site related to student
performance
(RQ 2. What school level efforts facilitate the implementation of the district
design? )
• What was your role in the development of the district plan?
• How did you implement the district plan at the site level?
• What was the timetable?
• Did the timetable follow the original plan?
• Who monitored the ongoing implementation of the plan?
• Describe the evaluation component of the plan?
• What measures were used to evaluate success or failure?
• Describe how the program was monitored at the site level?
• How did you train teachers?
• What was the training timeline?
• Who paid for the training?
• How did you assess the success of the plan?
5. Principal’s perception of teachers involvement in the district design
(RQ 3. To what extent has the district design and implementation strategies
at the site/district levels been effective?)
• What was the teacher’s response to the communication of the
district plan?
• What was the teacher’s attitude toward the plan?
• How did you know that the teachers’ understood the plan?
• How did teachers assess the success of the plan?
• What did the implementation look like in the classroom?
• What evidence did that teacher give to you that the plan is either
working or not working?
• To what extent were teachers actually using it? What do you see in
the classroom?
• How much time did the plan take?
• Does the teacher evaluation system currently in place support the
implementation of the district design?
• Do you feel your school needs have been met by the plan?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 7 1
Appendix H
Leadership Team Member Interview Guide
1. Leadership Tearn Member background
• Job description/role of the leadership team member
• years of experience
• years at the district/site
• professional background
• recent training
2. Perceptions of district involvement
• Describe the district plan
• Were you involved in the creation of the district plan?
• What are the elements of the district plan?
• How was the plan communicated to the lead teachers and staff?
• What training accompanied the plan for school leaders?
• How is the success of the plan assessed?
3. Specific Leadership Team Member activities at the school site related to
student performance
• What is your role in the implementation of the district plan at the
site level?
• Describe how the program is monitored at the site level?
• What type of supporting activities do you engage in with the
teachers?
• What is the training/implementation timeline?
• What about the plan do you think is working?
• What about the plan do you think needs work?
• How do you assess the success of the plan?
4. Leadership Team Member perception of teachers involvement in the district
design
• What was the teacher’s response to the communication of the
district plan?
• What was the teacher’s attitude toward the plan?
• How did teachers assess the success of the plan?
• What did the implementation look like in the classroom?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 7 2
Appendix I
Lead Teacher Interview Guide
1. Lead Teacher background
• Job description/role of the lead teacher
• Years of experience
• Years at the district/site
• Professional background
• Recent training
2. Perceptions of district involvement
• Describe the district plan
• Were you involved in the creation of the district plan?
• What are the elements of the district plan?
• How was the plan communicated to the lead teachers?
• What training accompanied the plan for school teacher leaders?
• How is the success of the plan assessed?
3. Specific lead teacher activities at the school site related to student
performance
• What is your role in the implementation of the district plan at the
site level?
• Describe how the program is monitored at the site level?
• What type of supporting activities do you engage in with the
teachers?
• What is the training/implementation timeline?
• What about the plan do you think is working?
• What about the plan do you think needs work?
• How do you assess the success of the plan?
4. Lead Teacher’s perception of teachers involvement in the district design
• What was the teacher’s response to the communication of the
district plan?
• What is the teacher’s attitude toward the plan?
• How do teachers assess the success of the plan?
• What does the implementation look like in the classroom?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 7 3
Appendix J
Teacher Interview Guide
1. Teacher background
• Years of experience
• Years at the district/site
• Professional background
• Recent training
2. Perceptions of district involvement
• What did you know about the district plan for improving teaching
and learning and how does the plan affect what you do in your
classroom?
• Was someone from your school, or you, involved in its creation?
• What are the elements of the district plan?
• How was the plan communicated to the teachers?
• What training accompanied the plan for schools and teachers?
• How was the success of the plan assessed?
3. Specific teacher activities at the school site related to student performance
• What was your role in the implementation of the district plan at the
site level?
• Describe how the program is monitored at the site level?
• What type of activities do you engage in with the administrator or
lead teachers?
• How often did you have training or participate in activities that
support you in the implementation?
• What about the plan do you think is working?
• What about the plan do you think needs work?
• How did you assess the success of the plan?
4. Teacher’s perception of teachers involvement in the district design
• What was your response to the communication of the district plan?
• Are most employee’s familiar with the district plan?
• How did you feel about the plan?
• Was the professional development offered helpful? Examples of
positive or negative aspects of the professional development.
• How did you assess the success of the plan?
• Can you give me examples that highlight either the positive or
negative examples that occurred during the implementation of plan?
• What does the implementation look like in the classroom?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 7 4
Appendix K
Innovation Configuration
Teacher Implementation of Classroom Instruction that Works Reform
Components Fully Implemented
Partially
Implemented
Just Getting
Started
Identifying
Similarities &
Differences
Students are regularly engaged
in tasks that involve
comparisons, classifications,
metaphors, and analogies.
These tasks, on different
occasions, are directed by both
teacher and students.
Students are occasionally
engaged in activities that
require identification of
similarities and differences
as teacher periodically
incorporates their use in
lessons.
Teacher is just
beginning to explore
the use of similarities
and differences as an
instructional strategy.
Summarizing and
Note-taking
Teacher regularly facilitates
students’ use of summarizing
techniques, including but not
limited to, rule-based, summary
frames, & reciprocal teaching
strategies. Students also
engage in note-taking
strategies, i.e., using teacher-
prepared notes, note-taking
formats, & combination
strategies.
Teacher occasionally
encourages the use of
summarizing techniques by
students. Some students
occasionally make use of
some note-taking
strategies.
Teacher is just learning
about summarizing
techniques and there is
little evidence of note-
taking strategies on the
part of students that is
encouraged by the
teacher.
Reinforcing Effort
& Providing
Recognition
Teacher regularly educates
students about effort and
continuously tracks student
effort and achievement.
Teacher also provides
personalized recognition;
makes use of pause, prompt,
and praise strategies; & uses
concrete symbols or tokens of
recognition on a recurring basis.
Teacher occasionally
informs students about their
effort and provides some
tracking of student effort
and achievement. Teacher
also periodically recognizes
and praises students.
Teacher is just
becoming aware of
how to inform students
regarding effort and
how to track their effort
and achievement.
Teacher infrequently
engages in recognition
and/or praise of
students.
Homework &
Practice
Teacher clearly communicates
an established homework policy
with parents, articulating the
purpose and outcome of the
assignments. Teacher uses a
variety of approaches to provide
feedback to students. Practice
assignments focus on specific
elements of complex skills &
students are provided with
planning time to increase their
conceptual understanding.
Teacher has a clear
homework policy that is
only occasionally
communicated to parents.
The frequency and quality
of feedback to students is
not necessarily consistent.
Practice of complex skills
occurs when time allows.
Teacher either does
not have a clearly
established homework
policy or does not
communicate policy to
parents. Feedback to
students is minimal.
Practice of skills is not
planned to reinforce
complex skills and/or
does not occur in
class.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 7 5
Components Fully Implemented
Partially
Implemented
Just Getting
Started
Nonlinguistic
Representation
Teacher frequently uses graphic
organizers, i.e., structuring
common patterns, making
physical models, generating
mental pictures, drawing
pictures, & engaging in
kinesthetic activity.
Teacher occasionally uses
of graphic organizers or
non-linguistic
representations.
There is little evidence
of the use of graphic
organizers or non
linguistic
representations in the
classroom.
Cooperative
Learning
In the process of regularly
incorporating cooperative
learning strategies into their
lessons, the teacher uses a
variety of criteria for grouping
students; creating informal,
formal and base groups;
managing group size; and/or
combining cooperative learning
with other classroom structures.
Teacher occasionally
incorporates cooperative
learning activities into
lessons that incorporate
strategies to make the
efforts particularly
successful.
The use of cooperative
learning techniques
and strategies are at
the beginning stages or
non-existent.
Setting
Objectives &
Providing
Feedback
Teacher has specific but flexible
goals and/or uses contracts with
students. Teacher also uses
criterion-referenced feedback
for specific types of knowledge
and skills or facilitates student-
led feedback.
Teacher occasionally
establishes goals or
contracts for students.
Feedback strategies are
used periodically to inform
students.
The establishment of
goals and the use of
feedback strategies are
minimally used.
Generating &
Testing
Hypotheses
Teacher frequently uses a
variety of structured tasks to
guide students through
generating and testing
hypotheses, making sure all
students can explain their
hypotheses and conclusions.
Teacher occasionally
makes use of activities that
engage students in
generating and testing
hypotheses.
Teacher is in the
beginning stages of
using strategies that
incorporate the
generation and testing
of hypotheses.
Cues, Questions &
Advanced
Organizers
Teacher regularly uses
straightforward, explicit cues,
and uses questions that require
analysis or that elicit inferences
on a recurring basis. Teacher
also uses advance graphic
organizers for expository and
narrative texts, as well as for
skimming.
Teacher periodically makes
use of cues and questions
during their lessons.
Teacher occasionally uses
advance graphic
organizers.
Teacher seldom uses
cues and questions in
their lessons. There is
little evidence of the
use of advance graphic
organizers.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
276
Appendix L
Innovation Configuration:
Teacher Implementation of Classroom Instruction that Works Reform
Identifying
Similarities &
Differences
Students are regularly
engaged in tasks that
involve comparisons,
classifications, metaphors, &
analogies. These tasks, on
different occasions, are
directed by both teacher and
students.
Students are occasionally
engaged in activities that
require identification of
similarities and differences
as teacher periodically
incorporates their use in
lessons.
Teacher is just beginning to
explore the use of
similarities and differences
as an instructional strategy.
Summarizing
and Note-
taking
Teacher is just learning
about summarizing
techniques, including but not
limited to, rule-based,
summary frames, &
reciprocal teaching
strategies. There is little
evidence of note-taking
strategies on the part of
students that is encouraged
by teacher.
Teacher regularly facilitates
students’ use of
summarizing techniques.
Students also engage in
note-taking strategies, i.e.,
using teacher-prepared
notes, note-taking formats,
& combination strategies.
Teacher occasionally
encourages the use of
summarizing techniques by
students. Some students
occasionally make use of
some note-taking strategies.
Reinforcing
Effort &
Providing
Recognition
Teacher occasionally
informs students about their
effort and provides some
tracking of student effort and
achievement. Teacher also
periodically recognizes and
praises students.
Teacher is just becoming
aware of how to inform
students regarding effort
and how to track their effort
and achievement. Teacher
infrequently engages in
recognition and/or praise of
students.
Teacher regularly educates
students about effort and
continuously tracks student
effort and achievement.
Teacher also provides
personalized recognition;
makes use of pause,
prompt, and praise
strategies; & uses concrete
symbols or tokens of
recognition on a recurring
basis.
Homework &
Practice
Teacher either does not
have a clearly established
homework policy or does
not communicate policy to
parents. Feedback to
students is minimal. Practice
of skills is not planned to
reinforce complex skills
and/or does not occur in
class.
Teacher has a clear
homework policy that is only
occasionally communicated
to parents. The frequency
and quality of feedback to
students is not necessarily
consistent. Practice of
complex skills occurs when
time allows.
Teacher clearly
communicates an
established homework
policy with parents,
articulating the purpose &
outcome of the
assignments. Teacher uses
a variety of approaches to
provide feedback to
students. Practice
assignments focus on
specific elements of
complex skills & students
are provided with planning
time to increase their
conceptual understanding.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
277
Nonlinguistic
Representati
on
Teacher frequently uses
graphic organizers, i.e.,
structuring common
patterns, making physical
models, generating mental
pictures, drawing pictures, &
engaging in kinesthetic
activity.
There is little evidence of
the use of graphic
organizers or non-linguistic
representations in the
classroom.
Teacher occasionally uses
of graphic organizers or
non-linguistic
representations.
Cooperative
Learning
Teacher occasionally
incorporates cooperative
learning activities into
lessons that incorporate
strategies to make the
efforts particularly
successful.
In the process of regularly
incorporating cooperative
learning strategies into their
lessons, the teacher uses a
variety of criteria for
grouping students; creating
informal, formal and base
groups; managing group
size; and/or combining
cooperative learning with
other classroom structures.
The use of cooperative
learning techniques and
strategies are at the
beginning stages or non
existent.
Setting
Objectives &
Providing
Feedback
Teacher occasionally
establishes goals or
contracts for students.
Feedback strategies are
used periodically to inform
students.
The establishment of goals
and the use of feedback
strategies are minimally
used.
Teacher has specific but
flexible goals and/or uses
contracts with students.
Teacher also uses criterion-
referenced feedback for
specific types of knowledge
and skills or facilitates
student-led feedback.
Generating &
Testing
Hypotheses
Teacher is in the beginning
stages of using strategies
that incorporate the
generation and testing of
hypotheses.
Teacher frequently uses a
variety of structured tasks to
guide students through
generating and testing
hypotheses, making sure all
students can explain their
hypotheses and
conclusions.
Teacher occasionally makes
use of activities that engage
students in generating and
testing hypotheses.
Cues,
Questions &
Advanced
Organizers
Teacher regularly uses
straightforward, explicit
cues, and uses questions
that require analysis or that
elicit inferences on a
recurring basis. Teacher
also uses advance graphic
organizers for expository
and narrative texts, as well
as for skimming.
Teacher seldom uses cues
and questions in their
lessons. There is little
evidence of the use of
advance graphic organizers.
Teacher periodically makes
use of cues and questions
during their lessons.
Teacher occasionally uses
advance graphic organizers.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
278
Appendix M
Key to Field Use of Teacher Implementation of Instructional Strategies:
Innovation Configuration
Identifying Similarities
& Differences
3 2 1
Summarizing and Note-
taking
1 3 2
Reinforcing Effort &
Providing Recognition
2 1 3
Homework & Practice
1 2 3
Nonlinguistic
Representation
3 1 2
Cooperative Learning
2 3 1
Setting Objectives &
Providing Feedback
2 1 3
Generating & Testing
Hypotheses
1 3 2
Cues, Questions &
Advanced Organizers
1 3 2
3 = Fully Implemented 2 = Partially Implemented 1 = Just Getting started
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
279
Appendix N
School Level Survey
Classroom Instruction that Works (Nine Strategies)
Thank you very much for agreeing to participate in this study. Your contribution is
valued and your answers are completely confidential. As part of this study, your
responses will provide educators and professional organizations with important
information.
Please mark the following demographic data that best applies to you for purposes
of this study only. Your anonymity will be maintained. The numbers in parenthesis
following each response are for coding purposes only.
1. Your Gender:
□ Female
(i)
6 . Years of Experience in Education:
□ Male (2)
□ 0-2 Years
(i)
□ 3-5 Years (2)
Highest Educational Degree You □ 6-10 Years (3)
Possess: □ 11-20 Years (4)
□ Bachelor’s
(1)
□ 20+ Years (5)
□ Master’s (2)
□ Doctorate (3)
7. Years in Current Position:
□ 0-2 Years
(i)
Credentials You Possess: □ 3-5 Years (2)
□ Clear
0 )
□ 6-10 Years (3)
□ Preliminary (2)
□ 11-20 Years (4)
□ Intern (3)
□ 20+ Years (5)
□ Pre-Intern (4)
□ Emergency (5)
8. Ethnicity:
□ African American
0)
Your Current Position: □ Asian (2)
□ Administrator
(1)
□ Hispanic/Latino (3)
□ Teacher Leader (Dept. Chair) (2) □ Native American (4 )
□ Teacher (3)
□ Pacific Islander (5)
□ White (6)
Grade Level Currently Teaching: a Other (7 )
□ K - 6 (1 )
£
00
1
,5
r-
□
(2)
□ 9th. 12th
(3)
□ Not teaching (4)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 8 0
9. Courses Currently Teaching:
□ Self-contained (Elementary School) (i)
□ Fine Arts (2)
□ Foreign Language (3)
□ English/Language Arts (4)
□ Mathematics (5)
□ Physical Education (6)
□ Science
(? )
□ Social Science
(8)
□ Elective (9)
□ Other (10)
□ Not Teaching
(1 1 )
10. How often do you use the nine strategies from Classroom Instruction that
Works?
□ Everyday (i)
□ Most Days (2)
□ At least twice a week (3)
□ About once per week (4)
□ Once per month (5)
□ Not at all (6)
□ Not familiar with the nine strategies (7)
11. What type of training have you had in using the nine strategies from
Classroom Instruction that Works?
□ More than five days (i)
□ Three to five days (2)
□ One to two days (3)
□ Less than one day ( 4)
□ No training (5)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 8 1
Classroom Instruction that Works (Nine strategies)
Please take time to read each item carefully and circle the number that best reflects
your response to the item.
Don’ t
Know
0
Strongly
Disagree
1
Somewhat
Disagree
2
Somewhat
Agree
3
Strongly
Agree
4
1. I am aware of the Classroom Instruction that
Works strategies for improving teaching and
learning.
0 1 2 3 4
2. The district supports standard-based
instruction.
0 1 2 3 4
3. The Classroom Instruction that Works
strategies are centered on improving student
learning.
0 1 2 3 4
4. The Classroom Instruction that Works
strategies call for the use of multiple
measures to assess student performance.
0 1 2 3 4
5. The Classroom Instruction that Works
strategies include ways to identify acceptable
levels of student performance.
0 1 2 3 4
6. The district believes that all students can
meet high standards.
0 1 2 3 4
7. The Classroom Instruction that Works
strategies foster a collaborative approach to
improving student performance.
0 1 2 3 4
8. Leaders in the district want everyone to use
the Classroom Instruction that Works
strategies.
0 1 2 3 4
9. The school is supportive in implementing the
Classroom Instruction that Works strategies.
0 1 2 3 4
10. The school offers frequent professional
development to raise knowledge of the
Classroom Instruction that Works strategies.
0 1 2 3 4
11. Systematic efforts to implement the
Classroom Instruction that Works strategies
were communicated.
0 1 2 3 4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
282
Don’ t
Know
0
Strongly
Disagree
1
Somewhat
Disagree
2
Somewhat
Agree
3
Strongly
Agree
4
12. Teacher training and assistance were
provided and conducted in a timely manner.
0 1 2 3 4
13.1 have attended professional development
training on the Classroom Instruction that
Works strategies in the past six months.
0 1 2 3 4
14. I know a great deal about the Classroom
Instruction that Works strategies.
0 1 2 3 4
15. Teachers have opportunities to provide input
on how to implement the Classroom
Instruction that Works strategies at my
school.
0 1 2 3 4
16. I would like to modify the way my school
uses the Classroom Instruction that Works
strategies based on the experiences of my
students.
0 1 2 3 4
17. Teachers are involved in the change process
and development/selection of suitable
materials to support Classroom Instruction
that Works strategies.
0 1 2 3 4
18. I am developing mastery of the Classroom
Instruction that Works strategies.
0 1 2 3 4
19. School leadership support was a key element
in assisting this site in the implementation of
the Classroom Instruction that Works
strategies.
0 1 2 3 4
20. Financial, staff, and material resources are
allocated to facilitate the implementation of
the Classroom Instruction that Works
strategies.
0 1 2 3 4
21. My school’s vision, mission, and goals are
aligned with the Classroom Instruction that
Works strategies.
0 1 2 3 4
22. All teachers are committed to the
implementation of the Classroom Instruction
that Works strategies.
0 1 2 3 4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 8 3
Don’ t
Know
0
Strongly
Disagree
1
Somewhat
Disagree
2
Somewhat
Agree
3
Strongly
Agree
4
23. The Classroom Instruction that Works
strategies are consistently used to improve
student learning.
0 1 2 3 4
24. The Classroom Instruction that Works
strategies are used to help students who
perform below grade level.
0 1 2 3 4
25. The Classroom Instruction that Works
strategies assist students to meet state
performance standards.
0 1 2 3 4
26. The Classroom Instruction that Works
strategies are used by other departments in
my school.
0 1 2 3 4
27. I connect the Classroom Instruction that
Works strategies to the curriculum.
0 1 2 3 4
28. I connect and use the Classroom Instruction
that Works strategies to teach my students
the standards.
0 1 2 3 4
29. I use the Classroom Instruction that Works
strategies to help students understand
textbook content.
0 1 2 3 4
30. Students are aware of and use the Classroom
Instruction that Works strategies to improve
their academic achievement level.
0 1 2 3 4
31.1 use the Classroom Instruction that Works
strategies to improve my instruction.
0 1 2 3 4
32. I am comfortable using the Classroom
Instruction that Works strategies in my
classroom.
0 1 2 3 4
33. Using the Classroom Instruction that Works
strategies has improved my teaching.
0 1 2 3 4
34. The Classroom Instruction that Works
strategies are effective in improving student
learning.
0 1 2 3 4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 8 4
Don't
Know
0
Strongly
Disagree
1
Somewhat
Disagree
2
Somewhat
Agree
3
Strongly
Agree
4
35. My instructional delivery has changed by
using the Classroom Instruction that Works
strategies.
0 1 2 3 4
36. The Classroom Instruction that Works
strategies have increased student motivation
toward learning.
0 1 2 3 4
37. The Classroom Instruction that Works
strategies are not expensive to implement.
0 1 2 3 4
38. The Classroom Instruction that Works
strategies were presented to teachers in a
timely manner.
0 1 2 3 4
39. Training provided to teachers on the
Classroom Instruction that Works strategies
was useful.
0 1 2 3 4
40. On-going support is provided at my school
on the Classroom Instruction that Works
strategies.
0 1 2 3 4
41. Feedback is provided to teachers on their use
of the Classroom Instruction that Works
strategies.
0 1 2 3 4
42. The Classroom Instruction that Works
strategies are worth keeping.
0 1 2 3 4
43. The Classroom Instruction that Works
strategies have helped students to be
successful in my class.
0 1 2 3 4
44. The Classroom Instruction that Works
strategies are feasible in improving teaching
and learning.
0 1 2 3 4
Thank you very much for your time. Please return this survey in the self-addressed
stamped envelope. If you have misplaced the envelope, extra self-addressed
envelopes are available in the front office.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 8 5
Appendix O
Teacher Questionnaire
Classroom Instruction that Works (Nine strategies)
Years of Teaching___________
The purpose of this questionnaire is to identify concerns regarding use of the nine
strategies from Classroom Instruction that Works. You may find some of the items
on this questionnaire irrelevant. For the completely irrelevant items, please circle
“0” on the scale. Other items will represent those concerns you do have, in varying
degrees of intensity, and should be marked higher on the scale. The results of this
questionnaire are confidential and will only be used for dissertation research in
summative form.
0 1 2 3 4 5 6 7
Irrelevant Not true of me
Somewhat true
of me
Very true of me
now
1 I am concerned about students’ attitudes towards the
use of the nine strategies from Classroom
Instruction that Works.
0 1 2 3 4 5 6 7
2 I now know of some other approaches that might
work better than the Classroom Instruction that
Works strategies.
0 1 2 3 4 5 6 7
3 I don’t know what the Classroom Instruction that
Works strategies are.
0 1 2 3 4 5 6 7
4 I am concerned about not having enough time to
organize myself each day in relation to using the
Classroom Instruction that Works strategies.
0 1 2 3 4 5 6 7
5 I would like to help other faculty in their use of the
Classroom Instruction that Works strategies.
0 1 2 3 4 5 6 7
6 I have a very limited knowledge of the Classroom
Instruction that Works strategies.
0 1 2 3 4 5 6 7
7 I would like to know how the use of the Classroom
Instruction that Works strategies affects my
classroom, my position at my school, and my
professional status.
0 1 2 3 4 5 6 7
8 I am concerned about conflict between my interests
and my responsibilities when using the Classroom
Instruction that Works strategies
0 1 2 3 4 5 6 7
9 I am concerned about revising my lesson plans to
use the Classroom Instruction that Works strategies.
0 1 2 3 4 5 6 7
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 8 6
10 I would like to develop working relationships with
both our faculty and outside faculty who use the
Classroom Instruction that Works strategies.
0 1 2 3 4 5 6 7
11 I am concerned about how the Classroom
Instruction that Works strategies affect students.
0 1 2 3 4 5 6 7
12 I am not concerned about the Classroom Instruction
that Works strategies.
0 1 2 3 4 5 6 7
13 I would like to know who will make the decisions
on how I use the Classroom Instruction that Works
strategies.
0 1 2 3 4 5 6 7
14 I would like to discuss the possibility of using the
Classroom Instruction that Works strategies.
0 1 2 3 4 5 6 7
15 I would like to know what resources are available to
support the Classroom Instruction that Works
strategies.
0 1 2 3 4 5 6 7
16 I am concerned about my inability to manage all
that the Classroom Instruction that Works strategies
require.
0 1 2 3 4 5 6 7
17 I would like to know how my teaching skills are
supposed to change with the use of the Classroom
Instruction that Works strategies.
0 1 2 3 4 5 6 7
18 I would like to familiarize other departments or
persons with the progress of using the Classroom
Instruction that Works strategies.
0 1 2 3 4 5 6 7
19 I am concerned about evaluating my impact on
students in relation to the Classroom Instruction
that Works strategies.
0 1 2 3 4 5 6 7
20 I would like to revise the Classroom Instruction that
Works strategies instructional approach.
0 1 2 3 4 5 6 7
21 I am completely occupied with other things besides
the Classroom Instruction that Works strategies.
0 1 2 3 4 5 6 7
22 I would like to modify the school’s use of the
Classroom Instruction that Works strategies based
on the experiences of my students.
0 1 2 3 4 5 6 7
23 Although I am not familiar with the Classroom
Instruction that Works strategies, I am concerned
about aspects of the Classroom Instruction that
Works strategies.
0 1 2 3 4 5 6 7
24 I would like to excite my students about their part in
the Classroom Instruction that Works strategies.
0 1 2 3 4 5 6 7
25 I am concerned about time spent working with
nonacademic problems related to the Classroom
Instruction that Works strategies.
0 1 2 3 4 5 6 7
26 I would like to know what the use of the Classroom
Instruction that Works strategies will require in the
immediate future.
0 1 2 3 4 5 6 7
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
287
27 I would like to coordinate my effort with others to
maximize the effects of the Classroom Instruction
that Works strategies.
0 1 2 3 4 5 6 7
28 I would like to have more information on time and
energy commitments required by the Classroom
Instruction that Works strategies.
0 1 2 3 4 5 6 7
29 I would like to know what other faculty are doing as
they use the Classroom Instruction that Works
strategies.
0 1 2 3 4 5 6 7
30 At this time, I am not interested in learning about
the Classroom Instruction that Works strategies.
0 1 2 3 4 5 6 7
31 I would like to determine how to supplement,
enhance, or replace the Classroom Instruction that
Works strategies.
0 1 2 3 4 5 6 7
32 I would like to use feedback from students to
change the Classroom Instruction that Works
strategies.
0 1 2 3 4 5 6 7
33 I would like to know how my role as a teacher will
change when I am using the Classroom Instruction
that Works strategies.
0 1 2 3 4 5 6 7
34 Coordination of tasks and people in relation to using
the Classroom Instruction that Works strategies is
taking too much of my time.
0 1 2 3 4 5 6 7
35 I would like to know how the Classroom Instruction
that Works strategies are better than what we have
had in the past.
0 1 2 3 4 5 6 7
36 I am concerned about how the Classroom
Instruction that Works strategies affect students.
0 1 2 3 4 5 6 7
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
288
Appendix P
Document Review Guide
The document review guide will be used to align district documents to the research
questions developed in the study; design, factors shaping design, extent of
implementation and effectiveness of design and implementation.
Document Title: Author & Date:
Descrip
R esearch
Q uestions
tion and Purpose:
S
w
o
e x
«
a
a .
o
o c
ELEMENTS OF
DISTRICT’S DESIGN
Researcher Notes
RQl: Design - RQ2: School Level Effort — RQ3: Extent of Im plementation - RQ4: Effectiveness of th e design
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Districtwide instructional improvement: A case study of Algiers Elementary School in the Orchard Unified School District

PDF

Districtwide instructional improvement: A case study of a high school in the media unified school district

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in Los Coyotes High School District

PDF

Districtwide instructional improvement: A case study of an elementary school in the Beach Promenade Unified School District

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Los Coyotes High School District

PDF

How districts and schools utilize data to improve the delivery of instruction and student performance: A case study

PDF

Data: Policies, strategies, and utilizations in public schools

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Orchard Field Unified School District

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Beach Promenade Unified School District

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Media Unified School District

PDF

Factors of a successful transition into a new superintendency

PDF

How effective schools use data to improve student achievement

PDF

An analysis of program planning in schools with emerging excellence in science instructional design

PDF

An analysis of the elementary principal's role in implementing school accountability within California's high -priority school: A case study

PDF

California teacher evaluation and improved teacher practice in a high -performing urban school

PDF

Design, implementation and adequacy of using student performance data and the design's link to state context for assessing student performance: A case study

PDF

Districtwide instructional improvement: A case study of a high school in the Los Coyotes High School District

PDF

A case study of instructional supervision, including teacher evaluation, and the impact on teacher practice

PDF

A study of the principal's role in increasing achievement in literacy in a low -performing elementary school

PDF

Implementation challenges of a school district's technology plan at the middle school level

Asset Metadata

Creator Kaufman, Sylvia Martinez (author)

Core Title Districtwide instructional improvement: A case study of a high school in the Los Coyotes High School District

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, administration,education, curriculum and instruction,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Marsh, David D. (committee chair), Bowman, Gregory (committee member), Brown, Richard (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-384919

Unique identifier UC11336729

Identifier 3180469.pdf (filename),usctheses-c16-384919 (legacy record id)

Legacy Identifier 3180469.pdf

Dmrecord 384919

Document Type Dissertation

Rights Kaufman, Sylvia Martinez

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA