University of Southern California Dissertations and Theses

A case study of teacher evaluation and supervision at a high -achieving urban elementary school

(USC Thesis Other)

A case study of teacher evaluation and supervision at a high -achieving urban elementary school

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content A CASE STUDY OF TEACHER EVALUATION AND SUPERVISION
AT A HIGH-ACHIEVING URBAN ELEMENTARY SCHOOL
by
Roberta Rosalin Lansman
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2006
Copyright 2006 Roberta Rosalin Lansman
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 3233791
Copyright 2006 by
Lansman, Roberta Rosalin
All rights reserved.
INFORMATION TO USERS
The quality of this reproduction is dependent upon the quality of the copy
submitted. Broken or indistinct print, colored or poor quality illustrations and
photographs, print bleed-through, substandard margins, and improper
alignment can adversely affect reproduction.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
®
UMI
UMI Microform 3233791
Copyright 2006 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
DEDICATION
This dissertation is dedicated with love and appreciation to my wonderful
family. My husband Robert, my loving partner, and my best friend has supported
and encouraged me as I opened new doors of educational advancement. There
were never complaints of missed meals. There was only unconditional encourage
ment. My two beautiful and brilliant daughters, Melissa, who recently received her
Master of Arts Degree, and Hillary, who just received her Bachelor of Arts degree,
have both been my biggest cheerleaders throughout the doctoral degree and disser
tation writing experience.
This dissertation is also dedicated to my mother and in memory of my
father. As a child of immigrant parents, I was taught to believe that schools were
institutions of equity and social justice. I was always the idealistic teacher, feeling
as if I could “teach the world.” I still feel, after more than 25 years, that there is no
loftier goal than to be an educator. As the years and opportunities prevailed, I
found myself led into a variety of directions, yet all in the field of education. I
thank my parents for always encouraging me to study and learn, and for always
telling me to reach for the stars because you can do everything.
So I thank my extraordinary and loving family for their support; I love them
all.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ACKNOWLEDGMENTS
It is with a great deal of gratitude and appreciation that I acknowledge those
special people who encouraged and supported me throughout my USC doctoral
program.
I would like to thank the three wonderful members of my dissertation com
mittee. First, I would like to thank my dissertation chair, Dr. Stuart Gothold, who
along with imparting his vast knowledge, was encouraging, supportive, a strong
and guiding leader, and an outstanding mentor. He also filled me with the USC
Trojan Spirit. Fight On, Dr. Gothold! Next, I would like to thank Dr. Robert
French, who has been a great leader, role model and superintendent for me, both
professionally and academically. His guidance and encouragement was greatly
appreciated in my quest to always “go for the touchdown.” I greatly appreciated
the leadership opportunities that he provided for me in Orange Unified, as well as
the annual USC football calendars. Finally, I would like to thank Dr. David Marsh,
who inductively guided me into seeing the vision of how the policy, practice,
impact, and influence of my study could be analyzed. It truly gave me my struc
ture. To these three great educators, I am eternally grateful and appreciative.
I would also like to thank my Orange County USC Cohort. These fine
people were my colleagues, study buddies, confidants, and friends. I would espe
cially like to thank Lynn Matassarin, my fellow administrator, friend, support, and
listening ear throughout this doctoral adventure. In addition, I would like to ac
knowledge the teamwork and support of my 15-member dissertation cohort. I
would especially like to thank my Team #1 subgroup, Tish Traynor-Nilsen, Sharon
iii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Nordheim, Leslie Rapkine-Miller, Cyndi Paik, and Ben Wolf, who became my
USC family.
As a working professional educator and doctoral student, I need to thank
some special people who have been encouraging and supportive, even if it meant
more work for them. I would like to thank my West Orange Team, my entire staff,
and especially my support staff. I would also like to thank my former Running
Springs staff for their continued support and friendship. A special thanks goes to
Jeanette Boyd for her flexibility, understanding, and help, and to Katherine Topor
for her excellent transcribing. The support from my Orange Unified “family” has
allowed my educational opportunities to go from good to great!
Thanks go to my study site school and district. I am grateful and apprecia
tive of all of the quality time and opportunity that they gave to me as an outsider to
become an insider and investigate first-hand what it takes to be a successful high-
achieving urban elementary school.
Finally, I would like to acknowledge my extended family and friends. To
my sisters, Sherri Chapman and Annette Caplin, and their families, thanks for the
encouragement and for taking care of the holidays. I would like to apologize to all
of them for all of the times I said, “I can’t ... because I have class.” They have all
been so supportive encouraging, and proud of me, even when I would practically
fall asleep in front of them. I feel fortunate to have them all as family and friends.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
TABLE OF CONTENTS
DEDICATION ............................................................................................................. ii
ACKNOWLEDGMENTS.......................................................................................... iii
LIST OF TABLES ....................................................................................................viii
LIST OF FIGURES.....................................................................................................xi
ABSTRACT ............................................................................................................... xii
CHAPTER
1. OVERVIEW OF THE STUDY ....................................................................1
Introduction............................................................................................. 1
Policy................................................................................................ 3
Practice.............................................................................................3
Impact...............................................................................................4
Influence...........................................................................................5
Statement of the Problem........................................................................5
Purpose of the Study............................................................................... 7
Research Questions..................................................................................8
Importance of the Study.......................................................................... 8
Assumptions........................................................................................... 10
Delimitations of the Study.................................................................... 10
Limitations of the Study....................................................................... 11
Definition of Terms...............................................................................11
Summary.................................................................................................16
2. REVIEW OF RELATED LITERATURE.................................................. 17
Introduction............................................................................................ 17
What is Teacher Evaluation?................................................................19
Formative Evaluation....................................................................24
Summative Evaluation...................................................................26
Stull Bill................................................................................................. 27
Historical Perspective........................................................................... 29
Standards-Based Reform and Accountability..................................... 33
No Child Left Behind........................................................................... 37
Teacher Quality......................................................................................39
Sample Performance-Based Standards................................................42
Improving Teacher Practice..................................................................43
Portfolio Assessment.....................................................................45
Peer Review....................................................................................46
Tennessee Value-Added Assessment System.............................47
Leadership.............................................................................................. 49
v
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Trends, Innovations and National Reforms........................................ 51
Conclusion............................................................................................ 53
3. METHODOLOGY....................................................................................... 55
Theoretical B ase................................................................................... 55
Conceptual Framework.........................................................................56
Research Questions Restated............................................................... 56
Design of the Study...............................................................................57
Sample and Population.........................................................................59
Criteria for Selection.....................................................................59
Case Study Site..............................................................................60
Instrumentation..................................................................................... 63
Development of the Survey Protocol........................................... 64
Development of the Interview Protocol....................................... 67
Development of the Observation Protocol.................................. 70
Data Collection..................................................................................... 71
Qualitative Data Analysis.....................................................................73
Survey.............................................................................................74
Interviews...................................................................................... 76
Documents..................................................................................... 77
Observation.................................................................................... 78
The Validity and Reliability of the Findings...................................... 81
Conclusion.............................................................................................81
4. THE FINDINGS...........................................................................................83
Overview of the Study Site...................................................................83
Findings by Research Question........................................................... 85
Findings for Research Question 1 ................................................85
Summary of Findings for Research Question 1...........................95
Findings for Research Question 2 ................................................97
Summary of Findings for Research Question 2.........................109
Findings for Research Question 3 ..............................................109
Summary of Findings for Research Question 3.........................124
Findings for Research Question 4 ..............................................125
Summary of Findings for Research Question 4.........................135
Discussion of the Findings..................................................................136
Policy Lens...................................................................................138
Practice Lens.................................................................................138
Impact Lens..................................................................................139
Influence Lens............................................................................. 141
Four Frames Aligned...................................................................141
Recurring Themes........................................................................ 144
5. SUMMARY, CONCLUSIONS, AND IMPLICATIONS.......................147
Summary of Background.................................................................... 147
Purpose of the Study.................................................................... 150
Methodology.................................................................................151
Data Collection and Analysis..................................................... 152
Summary of Findings.......................................................................... 154
Research Question 1 .................................................................... 154
vi
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Research Question 2 ....................................................................156
Research Question 3 ....................................................................156
Research Question 4 ....................................................................157
Conclusions..........................................................................................158
Implications and Recommendations..................................................160
REFERENCES...........................................................................................................162
APPENDIX A ............................................................................................................169
APPENDIX B.............................................................................................................170
APPENDIX C.............................................................................................................172
APPENDIX D ............................................................................................................177
APPENDIX E .............................................................................................................179
APPENDIX F .............................................................................................................180
APPENDIX G ............................................................................................................181
APPENDIX H ............................................................................................................182
APPENDIX 1..............................................................................................................186
vii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
LIST OF TABLES
1. Student Enrollment at Catson Elementary School by Ethnic
Group, CBEDS Data 2005............................................................................61
2. Catson Elementary School Academic Performance Index (API)
Growth Data (Released 2004/2005 School Year by California
Department of Education).............................................................................61
3. Certificated Staff at Catson Elementary School According
to Ethnicity..................................................................................................... 62
4. Certificated Staff Stability at Catson Elementary School............................ 62
5. Certificated Staff Status at Catson Elementary School.................................63
6. Rotated Component Matrix............................................................................ 65
7. Pearson Correlation Coefficients...................................................................66
8. Instrumentation Organization........................................................................75
9. Research Questions Triangulated With Broad Framework and
Bolman/Deal Frames.....................................................................................79
10. Best Practices of High-Performing School Systems Framework............... 80
11. Survey Findings for Research Question 1, How Is Teacher
Evaluation Practiced at the School Site? Administration’s
Involvement in the Teacher Formal Evaluation Process as
Mandated by California Education Code..................................................... 92
12. Survey Findings for Research Question 1, How Is Teacher Evaluation
Practiced at the School Site? Administration’s Involvement in the
Supervision Process Through Observation, Data Collection,
Feedback, Goal Setting, and Improvement Strategies................................ 93
13. Observation Tool Findings Related to Research Question 1, How Is
the Teacher Evaluation Process Practiced at the School Site?.................. 96
14. Survey Findings for Research Question 2, How Does the Teacher
Evaluation Process Impact Teacher Behavior? Administration’s
Involvement in the Teacher Formal Evaluation Process as
Mandated by California Education Code..................................................... 98
viii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
15. Survey Findings for Research Question 2, How Does the Teacher Evaluation
Process Impact Teacher Behavior? Administration’s Involvement in the
Supervision Process Through Observation,
Data Collection, Feedback, Goal Setting, and Improvement
Strategies................................................................................................................99
16. Findings for Open-Ended Survey Items Related to Research
Question 2, How Does the Teacher Evaluation Process Impact
Teacher Behavior?............................................................................................... 101
17. Observation Tool Findings Related to Research Question 2, How
Does the Teacher Evaluation Process Impact Teacher Behavior?................. 104
18. Survey Findings for Research Question 3, What Factors Exist
at the School That Positively or Negatively Impact the Teacher Practice?
Teachers’ Perceptions of School-Based Procedures and Activities, Not
Including Direct Supervision, That May Have
Led to School Improvement................................................................................111
19. Survey Findings for Research Question 3, What Factors Exist at
the School That Positively or Negatively Impact the Teacher
Practice? Teachers’ Perceptions of “the Way We Do Things
Around Here” .......................................................................................................112
20. Survey Findings for Research Question 3, What Factors Exist at
the School That Positively or Negatively Impact the Teacher
Practice? W4 and W 5.......................................................................................... 116
21. Observation Tool Findings for Research Question 3, What Factors
Exist at the School That Positively or Negatively Impact the
Teacher Practice?................................................................................................. 119
22. Survey Findings for Research Question 4, How Does the District/
Site Leadership Influence the Teacher Evaluation Process?
Teachers’ Knowledge and Perceptions of Official Documents
Regarding the Teacher Evaluation Process........................................................126
23. Survey Findings for Research Question 4, How Does the District/
Site Leadership Influence the Teacher Evaluation Process?
Administration’s Involvement in the Teacher Formal Evaluation
Process as Mandated by California Education Code.........................................127
24. Survey Findings for Research Question 4, How Does the District/
Site Leadership Influence the Teacher Evaluation Process? Administration’s
Involvement in the Supervision Process
Through Observation, Data Collection, Feedback, Goal Setting,
and Improvement Strategies................................................................................128
ix
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
25. Survey Findings for Research Question 4, How Does the District/
Site Leadership Influence the Teacher Evaluation Process?
Teachers’ Perceptions of Policy, Process, and Implementation.......................129
26. Survey Findings for Research Question 4: How Does the District/
Site Leadership Influence the Teacher Evaluation Process?
Responses to Open-Ended Survey Items........................................................... 130
27. Observation Tool Findings for Research Question 4, How Does
the District/Site Leadership Influence the Teacher Evaluation
Process?................................................................................................................135
28. Observation Tool Frame Analysis: Responses Tallied According
to Bolman and Deal’s Leadership Frames......................................................... 143
x
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
LIST OF FIGURES
1. Performance-based standards for Colorado teachers....................................42
2. Performance-based standards for Kentucky teachers....................................43
3. What leads to student learning........................................................................56
4. Research questions.......................................................................................... 58
5. Certificated staff interview questions.............................................................68
6. School and district administrator interview questions..................................69
7. Overview lens of the Bolman and Deal four-frame model.......................... 71
8. Survey findings for research question 3, what factors exist at the
school that positively or negatively impact the teacher practice?
W1: Describe which experiences and/or activities have the greatest
impact on your teaching practice and professional development 113
9. Survey findings for research question 3, what factors exist at the
school that positively or negatively impact the teacher practice?
W3: describe which types of interactions with administration
impact your teaching practices, either positively or negatively 114
x i
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ABSTRACT
The purpose of this study was to determine whether the teacher evaluation
process impacts teacher performance in support of student learning in an urban
school district. This qualitative case study examined a high-performing urban
elementary school in the Los Angeles county area of southern California to identify
key elements of its success by determining the effects of the evaluation process
connecting teacher evaluation, teacher effectiveness, and student achievement.
The methodology was based on four research questions developed by a
cohort of doctoral students. The four research questions drove the development of
a teacher survey, a teacher and administrator interview protocol, an observation
tool, and a document analysis tool, all of which were used as data collection in
struments. The findings were triangulated to respond to the research questions.
Participants included classroom teachers, the site principal, and a district adminis
trator. District and site policy and procedure documents were reviewed, site and
classroom observations were made, and confidential survey questionnaires were
collected from 19 classroom teachers. One-on-one interviews were conducted with
10 classroom teachers, the site principal, and the district assistant superintendent of
human resources.
Data were examined data through four distinct but related lenses: the
policy, the practice, the impact and influence of the teacher evaluation process, and
its relationship to teacher practice, and ultimately student achievement at a single
school site.
Analysis of the questionnaire results, interview responses, and observations
by the researcher indicated that, at times, there was a disconnect between the intent
xii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
and the practice with regard to the full implementation of the district’s teacher
evaluation process design. The results of this study showed that, at a successful
urban school, if there was strong leadership, collaboration, and accountability, the
teacher evaluation process was effective in improving teacher practice.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 1
OVERVIEW OF THE STUDY
Introduction
Anyone who is concerned with K-12 education in the United States wants
to know what works best to promote greater student achievement. In this current
age of quality assurance there have been increased accountability and regulation in
public schools across the nation with the recent legislation of the federal No Child
Left Behind Act of 2002 (NCLB). The premise of the NCLB accountability pro
vision is that all states must set clear timelines for improving student achievement
and ensure that all students are taught by highly qualified classroom teachers
(NCLB, 2002).
If a highly qualified teacher holds the golden key to unlock opportunities for
students to achieve, then a less-qualified teacher will lock out students so that they
may be left behind. In order to unlock achievement opportunities for all K-12
students, standards of “high expectations, organized effort, caring, commitment,
and talented teachers in every classroom” are needed (U.S. Department of Educa
tion, 1998). To achieve these standards there are numerous debates concerning
appropriate innovations and practices within the American education system such
as new curricula (including the role of computer technology), structural/organiza-
tional mandates (e.g., extended school year, grade retention, reduced class size)
nontraditional choices for schooling (e.g., charter, accelerated, alternative schools),
teaching and learning programs derived from popular learning theories and instruc
tional models (e.g., social constructivism, cooperative learning), standards-based
1
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
preparation programs for teachers and administrators, and new approaches to certi
fication, licensing and evaluation of teachers (Ellett & Teddlie, 2003).
Ultimately, it is the classroom teacher, and the evaluation of that teacher’s
daily practice, that is essential in shaping student achievement. Therefore, what
teachers know and are able to do is of critical importance to the nation, including
the task of preparing and supporting effective and continuous development of a
teacher’s knowledge and skills (Darling-Hammond & Ball, 1998). One cannot
improve student achievement without striving to improve teacher quality. In an age
of increasing accountability and quality assurance, examining teacher quality
through the lens of evaluation may be one way to measure a correlation between
improved teacher practice and improved student achievement.
During the past 3 to 4 decades education, as practiced in this country, has
been strongly concerned about producing, selecting, and assessing effective
teachers, as well as striving to understand connections among effective teaching,
teacher evaluation, and school effectiveness (Ellett & Teddlie, 2003, p. 102). In
addition, research studies have discovered again and again that teacher expertise is
one of the most important factors in determining student achievement (Darling-
Hammond & Ball, 1998). Since it is teacher expertise—what teachers know and
can do—that affects all core tasks of teaching, effective teachers must receive
feedback and support of their own practice in order that they may better teach the
children of the nation to the high standards that the challenges of the 21st century
demand.
Although research has shown a link between effective teachers and effective
schools, there appears to be very little research to connect a teacher’s mandatory
yearly evaluation with ongoing teacher improvement as it relates to increased
2
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
student achievement (Haycock, 2004). Most teacher evaluation systems seem to
concentrate on observed outcomes and do not promote ongoing teacher improve
ment. In examining a teacher evaluation system as it relates to increased student
achievement, one must address the policy, practice, impact, and influence of such a
system.
Policy
Under NCLB federal accountability provisions, states are required to define
a qualified teacher, set clear timelines for improving student achievement with
particular emphasis on closing achievement gaps between low-income and minority
students, and ensure that low-income and minority students are not taught dispro
portionately by inexperienced, unqualified, or out-of-field teachers (Education
Trust, 2003). NCLB policy provides strong leverage for guaranteeing all children
an opportunity to learn from “highly qualified” teachers. Although a highly
qualified teacher is important, most parents would want their child to have a highly
effective teacher whose instructional efforts (the act of teaching) result in high rates
of student learning.
Practice
Each state is required by NCLB policy (U.S. Department of Education,
Office of the Deputy Secretary, 2004) to define a qualified teacher, and so each
state maintains the practice of its own teacher evaluation system. An example of
one such system is California’s teacher evaluation system, regulated by the 1972
state Stull Bill (California Education Code, 2005b, Article 11, §§ 4460 et seq.).
The Stull Bill is a process-product evaluation system grounded in the notion of
teaching that prevailed in the 1970s, based on the direct instruction model of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Madeline Hunter and the development of “clinical supervision” as a way of enhanc
ing instruction (Danielson & McGreal, 2000). Paper and pencil “check-off’ forms
and rating scales became the norm. The idea of marking off what is determined
“good,” “meets standards,” or “satisfactory,” as well as the design and structure of
the form, is still the basis of the teacher evaluation method that is used in most
evaluations today. As part of the California Education Code, the impact of the
Stull evaluation is twofold. First, it is a summative tool. As long as a teacher
receives an overall satisfactory evaluation, the teacher’s job is intact and nothing
necessarily changes. Second, most evaluation systems in California schools
include as least one classroom observation. While still being able to satisfy
accountability demands, it is hoped that continual observation, which is formative
in practice, will support teacher growth and student achievement in the classroom
(California Education Code, 2005a).
Impact
With the onset of standards-based instruction, the movement to adopt
content standards for student learning must now have, as its parallel, new
approaches toward the evaluation of teaching (Danielson & McGreal, 2000).
Currently, the state of California has adopted California Standards for the Teaching
Profession (CSTP; California Commission on Teacher Credentialing, 1997). The
impact of the CSTP is that, under the legal umbrella of the California Stull Bill,
most public school districts in the state have incorporated the following six CSTP
standards (appendix A) into their evaluation process:
1. Engaging and supporting all students in learning
2. Creating and maintaining effective environments for student learning
4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3. Understanding and organizing subject matter for student learning
4. Planning instruction and designing learning experiences for all students
5. Assessing student learning
6. Developing as a professional educator
Influence
Designing or changing evaluation and professional development systems to
something new is a major event, whether it is in California or in any state in the
nation. In the current age of standards and accountability, the influence of good
teaching matters, and relevant evaluations should serve more than a methodical
purpose of compliance accountability. The teacher evaluation process should
enhance and support improvement for all teachers on the quest for greater student
achievement.
Statement o f the Problem
There are many sizes, shapes, and cultural backgrounds among teachers.
There are also differences in training and experience. Yet teaching, which is
unique among the vast array of professions, makes the exact same demand on all
practitioners.
No one would expect a prospective surgeon, straight from medical school,
to take charge of a complex operation. Nor would a new architect be asked
to design, single-handedly, a large office building. Yet, the job of teaching
for a novice is identical to that of a seasoned veteran; and the procedures
used to evaluate them are identical. (Danielson & McGreal, 2000, p. 5)
From the first moment that the new teacher enters the classroom, there is no
opportunity for a “do-over.” The beginning teacher is held to the same standards as
the experienced instructor and, as educators, both teachers are subject to the exact
same evaluation procedures on their performance in support of student learning.
5
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Thus, whatever evaluative checklist the site administrator may use for the most
experienced teacher on the staff, the first-year novice will be subject to the very
same list (Danielson & McGreal, 2000). This leads the researcher to question
whether there is are links among the teacher’s evaluation, teacher improvement,
and how well the students are achieving in the classroom.
By law or regulation, in every state across the nation, all public school
districts have a formal procedure for the evaluation of teachers (Danielson &
McGreal, 2000). It is commonly recognized that teachers are committed to
continual improvement of their professional performance and that the evaluation
process is found to be positive in nature and intent. By recognizing a teacher’s
individual strengths and providing a means for support and improvement, a teacher
evaluation is designed to be a communication tool to promote excellence in teach
ing and learning. According to Danielson and McGreal, most of the current evalua
tion systems that are used in the public school system were originated in the early
to mid-1970s and tend to rely heavily on the documentation of a small number of
“observable behaviors,” such as “writing the learning objectives on the board,”
“smiling at students as you greet them,” and similar actions (p. 3). Following this
common format, the classroom teacher does all of the perfunctory tasks that are
prescribed and the site administrator checks from the list those practices that have
occurred or those behaviors that have been observed, such as what occurs in the
summative system practiced in California (Danielson & McGreal).
It is unfortunate that there are evaluation systems that are not really helpful
for teachers who are truly looking to improve professional practice. Poor teacher
evaluation programs that do not improve teachers’ instructional skills or permit the
dismissal of incompetent teachers cheat children from potential achievement gains
6
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
that they could acquire if they were well taught (Stanley & Popham, 1988).
Despite this, the truly fundamental flaw in the evaluation system is the assumption
that the presence of good practice during one particular teacher observation would
equate to teacher growth and the entire academic achievement of the students.
Teacher effectiveness through classroom instruction and the learning gains
of students should involve evaluation of the act of teaching, as well as the results of
teaching. Unfortunately, well-intended evaluation systems that should serve the
purpose of providing both feedback and guidance toward improving a teacher’s
professional practice are often used as accountability documentation rather than as
a guide for teacher growth and improvement. Although most educators are in
agreement that they are responsible for student learning, evaluations based on
measures of student learning have been avoided. As observed by Mike Schmoker
(1999), “When we begin to more systematically close the gap between what we
know and what we do, we will be on the cusp of one of the most exciting epochs in
the history of education” (p. 70).
Purpose o f the Study
The purpose of this study was to determine whether the teacher evaluation
process impacts teacher performance in support of student learning in an urban
school district. This case study examined a high-performing urban elementary
school in California and identified key elements of its success in determining the
effects of the evaluation process connecting teacher evaluation, teacher effective
ness, and student achievement. A qualitative review of the evaluation process was
made with regard to how the evaluation system is implemented at the school
through utilization of the following: interviewing teachers and administrators,
7
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
administering a survey to staff, observing best practices, and examining evaluation
documents and district policies.
This qualitative case study was part of a study by a larger thematic disserta
tion group that examined how the teacher evaluation process impacts teacher per
formance in support of student learning at 15 high-performing urban schools
(elementary, middle, and high schools) in southern California.
Research Questions
The research questions for this qualitative case study were as follows:
1. How is the teacher evaluation process practiced at the school site?
2. How does the teacher evaluation process impact teacher behavior?
3. What factors exist at the school that positively or negatively impact
teacher practice?
4. How does the district’s/school’s leadership influence the teacher
evaluation process?
Importance o f the Study
The findings of this study are relevant to professional organizations, district
administrators, site administrators, and teachers. This study was part of a larger
study at the University of Southern California (USC) investigating evaluation
profiles of 15 high-performing urban schools in California to determine whether
teacher evaluation was a factor of success in their achievement.
Professional organizations offer support and guidance to schools and
districts by assisting them in their efforts to improve instruction. With an expanded
understanding of learning and knowledge of what constitutes good teaching, pro
fessional organizations have a great deal of influence on teacher effectiveness.
8
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Professional organizations can use this study to provide better training based on
current research-based procedures and reform measures, as well as through
examples of strategies used by a high-achieving and successful school.
District office administrators should provide guidance and support to
schools that are implementing effective instruction in the classroom. District
administrators must see that they have an obligation to support schools and ensure
that teachers are able to help students meet high standards. The evaluation process
should not be only a directive of compliance and accountability. This study
encourages district administrators to support teachers with high-quality staff
development and teacher training.
Site administrators are the true implementers of the reform effort. They are
the key evaluators of classroom teachers. A site administrator should understand
that an evaluation is a judgment, an assessment of teaching, and that it must be
made as objectively and fairly as possible, with no appearance of favoritism toward
individuals based on friendship or bias grounded in irrelevant matters. This study
can help administrators to distinguish strengths and weaknesses in both summative
and formative evaluations, as well as to understand the latest reform strategies in
teacher effectiveness and accountability.
Teachers are the critical connection to this study as the evaluatees. This
study provides teachers with the knowledge that teacher expertise is one of the
most important factors in determining student achievement. Teachers should be
able to see that those teachers who know much about teaching and learning and
who are effective in the classroom are the most critical element of successful
student learning.
9
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Assumptions
The following assumptions were made in this case study:
1. There was a link between the teacher evaluation process and teacher
performance in the classroom.
2. The site principal conducts all teacher evaluations at the school and was
viewed as an archetype instructional leader.
3. Distinct contributing factors, such as school climate/culture, professional
development, and school leadership were present at the school site, and to some
extent had a positive or negative impact on student achievement.
Delimitations o f the Study
This is a qualitative case study of one K-5 elementary school. Generaliza-
bility is not the goal of this qualitative study, and any applicability is in how the
reader interprets and applies similarities due to depth of coverage in this study. The
school selected for this study was based on the following criteria:
1. The school has schoolwide Title I status.
2. The school has at least a 40% poverty rate.
3. The school at least doubled its Academic Performance Index (API)
target of achievement for at least 2 years on the California STAR (state assessment)
test.
4. The school has a high percentage of English language learners.
5. The principal at the school site has been there at least 3 years.
4. The teachers at the school site have had at least one evaluation.
10
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Limitations o f the Study
1. Although all school districts in California are mandated to the provisions
of the California Education Code regarding the Stull Bill, this limitation restricts
making generalizations or replications of the findings for all schools and/or districts
in California. This qualitative case study was limited to interpretation and use of
the Stull Bill by this singular school and district.
2. This study was one of 15 similar studies investigating the teacher evalua
tion process at high-achieving urban schools. The researcher was confined to inter
viewing only teachers who volunteered to participate. The data analysis and find
ings may be subject to researcher bias and interpretation.
Definition o f Terms
Academic Performance Index (API): California’s numerical indicator of
student achievement, used as a basis for a comparative ranking of schools
statewide.
Accountability: The responsibility for implementing a process or pro
cedure, for justifying decisions made, and for results or outcomes produced.
Teachers are often said to be accountable for their students’ learning in the assigned
subject area, within the limits of the students’ abilities and the time and resources
available.
Assessment: The process of or instrument for measuring, quantifying,
and/or describing those aspects of teaching related to the attributes covered by the
evaluation.
Beginning Teacher Support and Assessment (BTSA): Since 1988,
California has sought to provide intensive learning experiences for beginning
teachers through the auspices of the California Commission on Teacher
11
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Credentialing and the California Department of Education (CDE). These two
agencies administer the BTSA program, supporting first- and second-year teachers
at the outset of their professional careers in partnership with local education
agencies and institutions of higher education. As a part of this work, research on
best practices in teacher induction and the development of program components
that support research-based practice have been a strong focus.
Behaviorism: The learning theory that describes learning as a response to
an environmental stimulus. Behaviorism advocates that children learn through a
change in behavior, so the use of rewards and punishment result in the establish
ment of good behaviors or the extinction of bad behaviors.
Benchmark: A referenced behavior for comparing observed performance at
a given level.
California Standards Tests (CST): These are tests administered by the state
of California Their purpose is to implement state standards and ensure that all
schools are accountable to meet their standards in each subject area in grades 2
through 12; scores are rated as Far Below Basic (FBB), Below Basic (BB), Basic
(B), Proficient (P), and Advanced (A).
Capacity: The potential for acquiring skills and competencies through such
means as self-study, on-the-job training, mentoring, coaching, and professional
development activities.
Clinical supervision: A process of collaboration between the teacher and
the supervisor or administrator designed to improve the teacher’s performance.
This process usually includes a pre-observation conference, observation and data
collection, data analysis, a post-observation conference, and a post-observation
conference evaluation report.
12
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Coaching: The assistance given to teachers in ways that will improve their
job performance, not just for the purpose of doing better on the evaluation but also
for the purpose of improved teaching and increased student learning. Coaching can
include reviewing teacher products related to the domains being assessed, tutoring
on the attributes being assessed, and offering feedback on the teachers’ strengths
and weaknesses. Coaching can also refer to teachers coaching students so that the
students will perform better on a measure used as an indicator of the teacher’s
performance.
Conceptual framework: A consistent and comprehensive integration of
research literature, theories, and other pertinent information that is the basis for the
analysis of findings within the study.
Content standards: As the foundation of a standards-based system, content
standards describe what content knowledge and skills students must master
(American Federation of Teachers, 2001).
Data: The information and evidence gathered during the assessment pro
cess for use in determining the level of teaching performance.
Data-driven decision making: The process of making decisions about
curriculum and instruction based on the analysis of classroom data and standard
ized test data in order to continuously improve student performance.
Evaluation stages: The major steps in the teacher evaluation process (e.g.,
orientation meeting; distribution of copies of job descriptions, evaluation pro
cedures, and timelines; and scheduling of the first conference and the first formal
observation).
Formative evaluation: An evaluation conducted primarily for the purpose
of improving the teacher through identifying the teacher’s strengths and
13
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
weaknesses. Formative evaluation is usually done by a supervisor or another
teacher rather than by an administrator; it is typically part of professional
development.
Induction: The initial period of exposure to a school or work setting during
which the teacher learns local policies and practices and the norms of teaching in
that setting.
Informal observation: The conducting of an assessment or evaluation
activity without a prescribed plan or structure, with little or no advance notice.
K-12: Grades kindergarten through 12.
No Child Left Behind (NCLB): This is the reauthorization of the Element
ary and Secondary Education Act of 1965. It creates mandatory testing of students
and improvement for schools. It sets a minimum level of qualifications for teachers
to meet to be “highly qualified” compliant. The deadline for the compliance is
June 2006.
Peer Assistance Review (PAR): Peer Assistance Review was introduced as
an amendment to the Stull Bill and entered into the California Education Code. It
provides referral to the PAR program for teachers who receive an unsatisfactory
performance on their formal Stull Bill evaluation. It is a nondisciplinary support
system.
Portfolio assessment: This is a method to assess performance via a
collection of a teacher’s artifacts of performance. This might include a teacher’s
lesson plans, student work, professional development participation, and other
evidence of performance.
14
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Professional development: Training and professional learning that has as
its purpose the improvement and enhancement of a professional’s performance and
knowledge.
Reform: A change effort that is undertaken to improve the educational
system.
Standard: The level of performance on the criterion being assessed that is
considered satisfactory in terms of the purpose of the evaluation.
Standards-based reform: The change to an educational system that utilizes
subject-matter benchmarks to measure student achievement, assessments aligned
with standards to measure student performance, and accountability systems that
provide rewards or sanctions to the district, school, and students based on student
performance.
Stull Bill (AB 293): A bill enacted by the California legislature in 1971 that
created the teacher and certificated evaluation process that is mandated for all
certificated staff in the state of California.
Summative evaluation: An evaluation conducted primarily for the purpose
of making personnel decisions about the teacher (e.g., merit pay, reassignment,
promotion, dismissal, tenure). Summative evaluation usually is performed by an
administrator rather than by a supervisor or another teacher.
Teacher effectiveness'. The attribute of a teacher who has the capability or
potential of having a positive impact on student learning, behavior, and attitudes.
Tenure: A common term used to identify a teacher’s permanent employ
ment status following a particular school district’s probationary status.
15
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Triangulation: The attempt to obtain more valid results by using multiple
sources of data about one aspect of performance, multiple methods of collecting
data, and/or multiple interpretations of the same data.
Summary
Since the purpose of this study was to determine whether the teacher
evaluation process impacts teacher performance in support of student learning in an
urban school district, an in-depth review of the literature presents theories and
research regarding various aspects of the teacher evaluation process. Specifically,
Chapter 2 examines a defining overview of teacher evaluation that includes both
formative and summative aspects of evaluation, a reflection of the historical-
political movement of accountability in both California and the United States, and
the operational Stull Bill and standards-based reform. Finally, a review of the
literature clarifies teacher evaluation with regard to national reform movements,
teacher quality, trends, innovations, and improving teacher practice.
16
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 2
REVIEW OF RELATED LITERATURE
Since the purpose of this study is to determine whether the teacher evalua
tion process impacts teacher performance in support of student learning in an urban
school district, an in-depth review of the literature presents theories and research
regarding various aspects of the teacher evaluation process. Specifically, chapter 2
examines a defining overview of teacher evaluation that includes both formative
and summative aspects of evaluation, a reflection of the historical-political move
ment of accountability in both California and the United States, and the operational
Stull Bill and standards-based reform. The review of the literature clarifies teacher
evaluation in regard to national reform movements, teacher quality, trends,
innovations, and improving teacher practice.
Introduction
Traditionally, teacher evaluation is used to determine a teacher’s suitability
for continued employment. Nevertheless, it is common practice throughout the
literature to find that a primary goal of such an evaluation is educational improve
ment. With both thoughts in mind, the goal of this literature review is to examine
how teacher evaluation within the past 5 years has or has not improved perform
ance of all teachers who deliver instructional programs directly to the students in
their charge.
School administrators face myriad tasks to reach their ultimate goal of
providing the best quality education for all students (Valentine, 1992). Due to
recent insights, accountability movements, and progress in educational research
that define and measure good teaching, policymakers have practiced using teacher
17
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
evaluation in a variety of new ways. Some states’ educational policies now use
evaluations to identify a teacher’s professional development needs. Others have
begun to link teacher evaluation with continuing licensure requirements in order to
raise the bar for teacher performance, providing a high-quality work force. Either
way, the educational research findings all seem to indicate that teacher expertise is
the most important factor in student achievement, and policymakers know now
more than ever before that having good teachers in the classroom is crucial
(Danielson, 2001; Darling-Hammond & Ball, 1998; National Commission on
Teaching and America’s Future [NCTAF], 1996). It is fortunate that a number of
factors have converged to create a renewed focus on teacher evaluation systems.
Current research literature has shown factors such as teacher quality, effect
ive teaching, teacher preparation, standards-based reform, the NCLB accountability
movement, state governance, and improvement of teacher evaluation models to be
relevant to improved teacher practice. Therefore, this study, which is focused on
how the teacher evaluation process impacts a teacher’s performance in support of
student learning, closely examines the aforementioned factors.
It is a common assumption that, although teachers are the single most
important determinant of educational quality, they are all evaluated in some way or
another. A number of school administrators and teachers view teacher evaluation
systems as extremely stressful, of little or no value, and a barrier to high staff
morale (Sawa, 1995). This literature study examines theories and research with
regard to the current practice of teacher evaluation in order to refute a disturbing
and inherent paradox seemingly recognized by analysts that teachers are simultane
ously taken to be the potential saviors of the American educational system while at
the same time being noted as the source of most of its problems. Initially, this
18
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
chapter defines positive and negative aspects of teacher evaluation, along with an
overview and summary of historical background. The chapter continues with a
study of governance, to include a perspective of California legislation and subse
quent law as it pertains to teacher evaluation. Trends, innovations, and national
reforms in relation to how the evaluation process impacts teacher performance in
support of student learning is reviewed, but first, it is important to define teacher
evaluation and its current practice.
What is Teacher Evaluation?
Teacher evaluation is one of the most important and potentially far-reaching
activities in school systems. If the evaluation is done well, it can advance the
quality of professionalism, enhance the quality of teaching, and improve the learn
ing environment for students. If the evaluation is done badly, it can undermine the
credibility of the profession, destroy the morale of a professional community, and
decrease opportunities for student achievement (Saskatchewan, School Board
Association, 2005). In this respect, “Teacher evaluation is very much a high-stakes
enterprise” (p. 7).
Frequently, some of the same skills used in program evaluation (e.g., data
gathering, data analysis) are involved in teacher evaluation, even though it is funda
mentally a separate process requiring different strategies and techniques (Glickman,
Gordon, & Ross-Gordon, 2005). Evaluation is the process by which teachers are
assessed professionally (Goldrick, 2002). By definition, teacher evaluation is based
on an individual system used in a particular school district, usually conducted by
principals or school administrators; it may include classroom observation as well as
verification of continuing education and professional development activities. By
19
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
current practice, evaluation is based on a principal or supervisor report, which in
turn is based on a classroom visit (Bridges, 1992; Lewis, 1982; Peterson &
Chenoweth, 1992) or a checklist of what good teaching should involve (Good &
Mulryan, 1990).
Goldrick (2002) stated that teacher evaluation, sometimes conceived
incorrectly, is distinct from teacher licensure. Teacher licensure is a means of
credentialing beginning teachers and often requires completion of a basic-skills test
and evidence of minimum competence. The re-licensure of veteran teachers is
seldom, if ever, contingent upon the data collected from their evaluations.
In most school districts, teacher evaluation is typically designed to be used
as a personnel action rather than as a tool to guide instructional improvement.
Although evaluation serves as a mechanism for assessing job performance, in
practice it is “often cursory, subjective, based upon insufficient observation, and
seldom results in the termination of truly poor teachers” (Goldrick, 2002, p. 2).
Teacher evaluation has been described as “an activity that is done to teachers”
(Danielson, 2001, p. 2). A Massachusetts educator consortium has described
teacher evaluation as “a task that teachers endure and a task conducted by already
overextended school administrators (MassPartners for Public Schools, 2002, p. 4).
Within the current context of state and district policies, school leaders have
broad latitude in designing an evaluation process. Only a few educational
researchers and developers have worked on the evaluation of the key performers of
the curriculum and the classroom: the teachers (Peterson, 2000). Unfortunately, as
studied by Peterson, poor practice in teacher evaluation is quietly accepted by
teachers, administrators, and researchers. Principals may evaluate teachers annu
ally, then only observe them every 2 to 4 years, awarding a teacher permanency
20
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
based on evaluations that may be far disconnected from student learning and that
do not seem to help improve teaching practice (Odden, 2001). With current
reforms in education, it appears that more and more policy makers are “scrapping”
a traditional approach to teacher evaluation in favor of performance-based teacher
evaluations (Goldrick, 2002). In fact, Peterson has suggested that teacher evalua
tion requires 12 new directions:
1. Emphasize that the function of teacher evaluation is to seek out,
document and acknowledge GOOD teaching.
2. Use good reasons to evaluate teachers (e.g., to discover what
training and development are required).
3. Place the teacher at the center of evaluation.
4. Use more than one person to judge teacher quality and performance.
5. Limit the judgmental role of administrators.
6. Use multiple data sources to inform judgments about teacher
quality.
7. Include student achievement data
8. Vary data sources depending on the individual teacher’s roles and
approaches.
9. Spend time and resources on recognizing good teaching.
10. Use research on teacher evaluation correctly.
11. Attend to the sociology of teacher evaluation by recognizing the
complex organizational factors at play in any educational institution.
12. Use the results of teacher evaluation to encourage professional
dossiers, publicize results and support teacher promotion, (p. ix)
Another researcher stating the purposes of teacher evaluation, Donald
Haefele (1993, as cited in Danielson & McGreal, 2000), pointed out that a clear
sense of purpose should govern the design of a teacher evaluation system. He
identified the following purposes that must be served, reasoning that a teacher
evaluation system should
21
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
screen out unqualified persons from certification and selection processes;
provide constructive feedback to individual educators;
recognize and help reinforce outstanding service;
provide direction for staff development practices;
provide evidence that will withstand professional and judicial scrutiny;
aid institutions in terminating incompetent or unproductive personnel; and
unify teachers and administrators in their collective efforts to educate
students, (p. 8)
Other researchers also believe that current teacher evaluation practices are
in grievous need of change: “Teacher evaluation is a disaster. The practices are
shoddy, and the principles are unclear” (Scriven, 1981, p. 244). “Evaluators are
mistaken if they assume they are observing the typical behavior of a teacher with
the usual evaluation procedure” (Stodolsky, 1984, p. 17). “Principals lacked
sufficient resolve and competence to evaluate accurately” (Wise, Darling-
Hammond, McLaughlin, & Berstein, 1984, p. 22). “Teachers see nothing to be
gained from evaluation” (Wolf, 1973, p. 160). “Teachers regard the practice as an
institutional obligation to be endured rather than an opportunity to be seized”
(Johnson, 1990, p. 266). “If a school can justify evaluating all teachers through
identical procedures, then the school is probably devoid of innovations” (Travers,
1981, p. 22). “An approach based on this kind of [classroom observation-based]
research cannot be a legitimate method of teacher evaluation” (Scriven, 1987, p. 9).
“In most school districts, the norms and expectations that surround teacher evalua
tion preclude a meaningful activity” (McLaughlin, 1990, p. 404).
Teacher evaluations that are well designed and integrated with curriculum
and professional teaching standards can attain more than just assuring basic
competence. Goldrick (2002) stated that a purposeful evaluation system measures
22
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
teaching outcomes, not simply teaching behavior. A solid teacher evaluation can
help states and districts to measure the effectiveness of teachers at various points in
their careers. The evaluation can identify highly skilled teachers and offer specific
recommendations to improve teaching, inform professional development, and
demonstrate accountability for student achievement. As stated by Goldrick, policy
makers should treat teacher evaluation as “an integrated component of a compre
hensive strategy to improve overall teaching quality” (p. 2).
During the past 3 to 4 decades, education as practiced in this country has
been strongly concerned with (a) producing, selecting, and assessing effective
teachers; and (b) understanding connections among effective teaching, teacher
evaluation, and school effectiveness (Ellett & Teddlie, 2003, p. 102). Toward any
improvement effort, informative and timely feedback is paramount. Essentially,
teachers need support and feedback not only on the act (the formative aspect) of
teaching but on the result (the summative aspect) of teaching. Thus, teacher
evaluation systems should be designed and are intended to serve the purpose of
providing both formative and summative feedback and guidance for improving a
teacher’s professional practice (Tucker, P., & Stronge, 2005).
A good principal or administrator should be dedicating a large amount of
time to the supervision of teachers. In reality, most administrators spend more of
their time on evaluation of teachers, even if many feel that the evaluation is lacking
in credibility (Carroll, 1997). Most of the supervision today falls into the category
of summative evaluation. Administrators often see the responsibilities of both
supervision (formative) and evaluation (summative) as one and the same when, in
fact, they are not (pp. 3-4).
23
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Formative Evaluation
A formative focus on the evaluation process promotes a climate of openness
and trust where the teacher may feel more apt to admit a lacking or inadequacy in a
particular area and request feedback that would lead to improved instructional
performance (Glickman et al., 2005). A formative evaluation is usually conducted
during the development or improvement of a program or person in order to assist
and support. With a focus on the aspects of teaching and learning, formative evalu
ation is typically conducted more than once (Glickman et al.; Scriven, 1981). A
significant finding on the study of formative evaluations concludes that “evaluation
systems designed to support teacher growth and development through an emphasis
on formative evaluation techniques produced higher levels of satisfaction and more
thoughtful and reflective practice while still being able to satisfy accountability
demands” (Danielson & McGreal, 2000, p. 15).
However, when teachers feel that the information from formative confer
ences may be used in their summative, personnel decision-making conference,
communication and mutual trust are often lost (Daresh, 1989). A principal fre
quently faces a conflict between the formative role of supervisor and the summative
role of evaluator. The teacher realizes this same conflict by choosing to rely on the
principal for “formative” assistance, or instead worry and possibly face criticism.
It is ironic that teachers put themselves into this dilemma constantly with their
students in the classroom. The teacher has the formative responsibility to help a
student learn but he or she must also evaluate that same student’s progress
(Acheson & Gall, 1980).
The formative evaluation process should have two main goals as its focus.
One goal is to improve instruction through coaching and feedback, and the other
24
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
goal is to provide professional growth for the teacher to correct any deficiencies
that may have been uncovered in the formative evaluation process (Carroll, 1997).
An example of teachers supported in this manner are first- and second-year teachers
in California who are mandated at the outset of their professional careers to partici
pate in the BTSA Program (California Commission on Teaching Credentialing and
CDE, 2001). Through the BTSA program beginning teachers are introduced to the
California Formative Assessment and Support System for Teachers (CFASST),
which consists of a systematic set of standards-based formative assessment activi
ties designed to be used by beginning teachers for the sole purpose of improving
teaching and enhancing student learning. The result is that formative evaluation
will assist teachers in doing a better job in the classroom rather than penalize them
for what they might be doing incorrectly (Blake & DeMont, 1989, as cited in
Carroll).
Finally, a key element in the formative supervision process is that it does
not necessarily mean that an administrator must carry out the supervision. Effect
ive formative supervision can include any building administrator, a peer teacher, or
another district faculty member. In fact, the key element to this process is not who
the supervisor may be but rather what level trust and collegiality exists between the
two persons involved (Carroll, 1997).
Peer coaching is an excellent example of formative supervision that has
been implemented in many schools across the country. Peer coaching provides
support, experimentation, honest and open communication, collegiality, and most
of all, utilization of the knowledge of effective teaching. Another form of forma
tive supervision is the area of self-analysis. According to Carroll (1997), teachers
should be trained and encouraged to make judgments and reflections about the
25
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
effectiveness of their own teaching. The realistic fact is that a supervisor cannot be
physically in the classroom to observe every single lesson. Therefore, the class
room teacher should be trained in using reflection in order to make appropriate
changes in his or her own instructional procedures.
Summative Evaluation
Every district should examine its supervision and evaluation process to
determine how it should impact teacher performance and accountability in support
of student learning, knowing that the process should perform two educational
functions. First, as previously mentioned, a formative evaluation with the teachers
should improve teacher performance on the district’s standards of instruction while
moving toward the ultimate goal of improving students’ performance on the
district’s curriculum standards. Concurrently, the second function is that the evalu
ation process should summatively assess teacher performance as measured against
the district’s articulated standards for curriculum and instruction (Ribas, 2000).
Evaluation is generally a summative process that results in personnel deci
sions conducted by an administrator, usually the site principal. One stated purpose
of a summative teacher evaluation is the improvement of classroom instructional
performance (Carroll, 1997). Unfortunately, many studies have shown that this is
not occurring. A key complaint on behalf of the teachers is the efficacy of the
many arbitrary evaluative criteria on the summative evaluation form used by
schools that were developed at other institutions. In other words, does the evalua
tion have the ability to do what it was designed to do?
Frequently, the summative teacher evaluation processes that govern
instructional practices have been mandated at the state level. Many states have
26
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
adopted summative teacher evaluation instruments, and there is genuine confidence
that the instrument will have the ability to promote excellence in the teaching
profession. Others feel that this mandated type of evaluation “views teachers as
infants in the instructional process and threatens any sense of professionalism”
(Carroll, 1997, p. 15). In a summative evaluation, the focus is on the end product,
rather than on any concern or problems that teachers might be experiencing.
California is one of the states that has evaluation mandated at the state level, by
means of the California Stull Bill.
Stull Bill
Most states have laws that require the evaluation of all public school
teachers. One of the most important pieces of legislation that has occurred in
California education is AB 293, commonly referred to as the Stull Bill. The Stull
Bill was introduced into the 1971 Regular Session of the California Legislature by
Assemblyman John Stull on January 28, 1971. This law affects all certificated
employees, from the classroom teacher to the superintendent. It has remained the
single evaluation guide for certificated teachers and administrators in California for
over 30 years. R. F. Hernandez (1972) noted that this legislation was a new
approach to teachers by the coupling of teacher evaluation standards in each school
district with a new set of specific and definite guidelines. Historically, the Stull
Bill has also been described as the “teacher tenure law” and the “teacher evaluation
law” (Hernandez). The Stull Bill provides a uniform system for evaluation, and
regulates both how certificated educators are evaluated and how they are dismissed.
These are typical roles of summative evaluations that are found in other states.
Known in the California Education Code as “Article 11, Evaluation and
27
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Assessment of Performance of Certificated Employees,” the following sections in
the Education Code (reprinted in appendix B) specifically deal with evaluation
based upon the passing of AB 293:
§ 44660 Legislative intent; establishment of uniform system.
§ 44661 Advice of certificated instructional personnel.
§ 44661.5 Objective evaluation and assessment guidelines; agreement
to include objective standards from National Board for Professional
Teaching Standards of California Standards for the Teaching Profession.
§ 44662 Evaluation and assessment guidelines.
§ 44663 Written evaluation and assessment of certificated employees
and certificated noninstructional employees; copy to employee; written
reaction; discussion of evaluation and assessment.
§ 44664 Frequency; areas of employment; unsatisfactory perform
ance; exclusion.
§ 44665 “Employing authority.” (California Education Code, 2005b)
The Stull Bill provides for the school boards of every district in the state of
California to adopt written and specific guidelines to include a uniform set of
objective evaluation protocols for use in evaluating the professional competency of
its certificated personnel.
The Stull Bill specifies that the written guidelines must include:
1. Establishment of standards of expected student progress in each area of
study and of techniques for assessment of that progress.
2. Assessment of certified personnel competence as it related to established
standards.
3. Assessment of other duties normally required to be performed by
certificated employees as adjunct to their regular assignments.
28
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4. Establishment of procedure and techniques for ascertaining that certifi
cated employee is maintaining proper control and is preserving suitable learning
environment. (California Education Code, 2005a; Stull Bill, AB 293).
As part of the Stull Bill evaluation process, it is the legal right of the certifi
cated employee to be given follow-up counseling for deficiency. Furthermore,
evaluations must be given in writing, and guidelines must be published so that all
certificated employees know the required expectations prior to their written evalua
tion. In the state of California there have been no significant changes in the legis
lated evaluation process for more than 30 years.
Historical Perspective
After defining the current teacher evaluation system and practice, it would
seem of interest to examine a historical perspective in order to answer the question,
how did we get to where we are now in the teacher evaluation process?
Historically, what has fundamentally shaped teacher practice occurred
during the post-World War II years. However, in many respects, the characteristics
of problems in what many consider dysfunctional school districts are part of the
design of what was developed in the first decades of the past century by a group of
educational reformers known as administrative progressives (Burney, Canada,
Corcoran, & Erskine, 2002). These progressives sought a remedy for the “patron
age and provincialism” of the school governance system of the 19th century and
mimicked the growing manufacturing economy by trying to create the “one best
system” (Tyack, 1974, as cited in Burney et al.) that would produce assimilated,
productive citizens as efficiently as Ford’s factories produced cars (p. 2). Pro
fessionals were to make and enforce policies (teacher evaluation) that would be
29
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
carried out by the “workers” (teachers) in the schools. Thus, standardization of
inputs, not outputs, was the goal.
What was learned from these administrative progressives was that, by
rewarding compliance over professional judgment and separating the school from
the community, a system was created that almost guaranteed that “innovation
would be thwarted” (Burney et al., p. 2). “Good instruction and good schools are
idiosyncratic rather than pervasive, and lessons from successful school and districts
are not widely learned or heeded” (p. 2). This connects to the current study of
evaluation in that another legacy of the administrative progressives was their failure
to free school systems from politics. Political jockeying became much more
important than addressing the needs of children. This can cause unproductive,
adversarial relationships between district and union leaders to move true educa
tional concerns to a back burner (such as the collective bargaining agreement on
evaluation practices being more of a concern than learning to read), while special
interest groups lobby for advantages at the cost of “other people’s children”
(Delpit, 1995).
The lack of adequate teacher preparation was condemned by both internal
and external critics for its “lack of intellectual rigor, selectivity standards, structural
arrangements, research base, and failure to achieve positive results in schools and
classrooms” (Cochran-Smith & Zeichner, 2005, p. 71). As stated by Cochran-
Smith and Zeichner with regard to declarations by liberal arts scholars and teacher
educators, “Extremists on one side proclaimed that teachers were being taught how
to teach but not what to teach, [while] extremists on the other side claimed that if
one really knew how to teach, one could teach anything” (p. 74).
30
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Following the 1957 launching of the Soviet satellite Sputnik, American
public education was outwardly criticized for becoming mediocre, instigating a
renewed demand for rigor and excellence in public schools. Hence, the 1960s
ushered increased efforts among educational researchers to identify effective
teaching methods, and a new era of classroom-based teaching methods research
grounded in the popular “behaviorism movement” in psychology and education
was developed (Ellett & Teddlie, 2003). With a push to enhance basic skills and
improve science and mathematics, researchers began to look at links between
teaching practices (behaviors) and student outcomes.
The 1970s brought an era of competency-based teacher education (CBTE).
The National Teachers Exam was introduced, maintaining that teachers would have
a core set of behaviors and skills deemed necessary for effective teaching in
American classrooms (Ellett & Teddlie, 2003). During this decade important
methodological studies of teacher effectiveness research (TER), including dis
cussions of classroom observations and teacher evaluations, encompassed the
educational research literature. Examining teacher evaluation literature on the use
of direct observations of teaching, as well as linking observations and evaluations
of teaching measures and achievement to on-task behaviors, was common.
Research examined what teachers did or could do to improve basic skills. The
process-product generation of teacher evaluation systems is grounded in the notion
of teaching that prevailed in the 1970s. Many systems are based on the direct
instruction model of Madeline Hunter, with her seven steps of lesson design
(anticipatory set, statement of objective, instructional input, modeling, checking for
understanding, guided practice, and independent practice). At this time researchers
were developing “clinical supervision” as a way of enhancing instruction, and
31
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
observation instruments were designed that allowed more accurate depictions of
what was occurring in classrooms (Danielson & McGreal, 2000). Paper-and-pencil
“check-off’ forms and rating scales became popular in these observations. The
idea of marking off what is determined good is the design and structure that is still
the basis of the teacher evaluation method used in many evaluations today.
The 1980s were marked by public concern about the state of American
education with the issuance of a federal report titled A Nation at Risk (National
Commission on Excellence in Education, 1983). This report criticized public
education for failing to prepare students to become productive member of the
nation’s workforce and to compete in the current global economy (Jennings, 2000).
This report stimulated a wave of educational reform initiatives, such as mandating
Local Educational Agencies (LEAs) to be accountable for the educational outcomes
of students; however, it did not address the classroom teacher. Then in the 1986
report^ Nation Prepared: Teachers for the 21st Century (Carnegie Forum, 1986)
the focus was on the teacher, and the document’s leading recommendation called
for establishing a National Board for Professional Teaching Standards (NBPTS).
Thus, in 1989, the NBPTS issued its policy statement, What Teachers Should Know
and Be Able To Do, which served as a basis for all standards development work
and guided school districts, states, colleges, and universities with a strong interest
in strengthening what teachers should know and be able to do (NBPTS, 2002).
“This last decade has heralded a significant change in conceptualizing the
teacher evaluation process” (Avery, 2001, p. 1). With the onset of standards-based
instruction, all students of the 1990s and were to be guaranteed the same rigor and
quality of instruction. Therefore, schools, the state, and the nation currently face
the challenge of developing and implementing standards for instruction, as well as
32
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
professional teaching standards, while ensuring that all teachers are able to perform
at those high levels.
Standards-Based Reform and Accountability
Almost everywhere and at every level across the country, the notion that
all students should be held to high standards and that educators should be held
accountable for getting them there has taken hold (Tucker, M. S., & Codding,
1998). The standards movement has moved from an outlier status to the very
center of education policymaking in the United States.
In the early part of the past century the administrative progressives favored
a system that supported the idea that the proper function of an educational system is
to sort out the vast majority of students who would “not be expected to achieve
more than an eighth-grade level of literacy, from a much smaller number who
should be selected to receive a curriculum that will fit them to become the
managers and professionals who will run the country” (p. vi). Standards-based
reform posited the direct opposite and very powerful understanding that it is both
possible and necessary for virtually all students to master a curriculum that is
rigorous and intellectually demanding. The rhetoric of standards and accountability
saturated the educational platform of both the Democratic and Republican candi
dates during the presidential election of 2000. With the passage of NCLB in 2002
the federal government endorsed the idea of standards-based education and made it
the law of the land (Tucker, M. S., & Codding, 1998).
If students are to achieve high standards, there can be nothing less from
educators. Thus, the movement to adopt content standards for student learning
must now have as its parallel new approaches toward the evaluation of teaching
33
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
(Danielson & McGreal, 2000). As stated by Darling-Hammond and Ball (1998), in
order to help students succeed in meeting the new standards, the first priority must
be to reach agreement on what teachers should know and be able to do. With the
increased importance of accountability and role that standards serve in education,
standards-based instruction provides the impetus for teachers to focus on the core
subject area knowledge and skills in which students are expected to become pro
ficient (National Research Council, 2003). Teaching to effective standards is
designed to promote high levels of student achievement and accountability from
one school year to the next. Thus, in a high-stakes, standards-based classroom
environment, a teacher should be provided with regular, ongoing, and consistent
feedback.
As there is continual and growing agreement that many schools are failing
children from low-income and minority backgrounds, accountability is now seen as
the vehicle for improvement using the assumption that teachers will try harder and
become more effective in meeting goals for student performance when the goals are
clear, when the information on the degree of success is available, and when there
are real incentives to meet the goals. Within the past few years, external agencies
have become much more interested in outcome data, such as test scores, than ever
before. There is a belief that using these outcome data systematically as part of an
accountability system will motivate positive change (Datnow & Schneider, 2004).
The three professional bodies that set national teaching standards linked to
student standards in order to outline a coherent, career-long continuum of teacher
development are the National Council for Accreditation of Teacher Education
(for teacher education), the Interstate New Teacher Assessment and Support
Consortium (for beginning teacher licensing), and the NBPTS (for advanced
34
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
certification of accomplished veteran teachers). To advance teacher standards, the
following accountability recommendations were made by the National Commission
on Teaching (NCT) for all states in the union:
• Establish professional standards boards.
• Insist on professional accreditation for all schools of education.
• License teachers based on demonstrated performance of their ability to
teach to the new standards, including tests of subject matter knowledge,
teaching knowledge, and teaching skill.
• Use National Board standards as the benchmark for accomplished
teaching. (Darling-Hammond & Ball, 1998, p. 14)
In addition, so that teachers would have continuous access to the most
current knowledge about teaching and learning, the Commission also recom
mended that, along with states, schools and colleges must:
• Organize teacher education and professional development around
standards for students and teachers.
• Institute extended, graduate-level teacher preparation programs that
provide year-long internships in a professional development school.
• Create and fund mentoring programs for beginning teachers that provide
support and assess teaching skills [in California it is BTSA, which is
discussed later].
• Create stable, high-quality sources of professional development, and
allocate one percent of state and local spending to support it. (Darling-
Hammond & Ball, 1998, pp. 14-15)
According to the Association of California School Administrators (ACSA)
Task Force on Student Performance and School Accountability, Californians hold
the same high expectations for all students and believe that every child can aspire to
and achieve high standards. Educators want and expect to be held accountable for
student learning. Therefore, a comprehensive and continuous improvement
accountability system in which all elements are aligned and work together for the
35
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
purpose of promoting and improving student learning is the expectation for
California’s accountability system (ACSA, 1997). Essential to an effective state
wide comprehensive standards-based accountability system that focuses on student
learning and achievement and represents the best practices of states and districts
around the country are the following five key components for California accounta
bility:
• Content standards-to define what students should know and be able to
do;
• Performance standards-to define levels of proficiency;
• Appropriate assessments-to define level of proficiency;
• Resources and support-to assure all schools can successfully implement
a standards-based accountability system; and
• Shared responsibility by all stakeholders-to ensure that students,
parents, educators and community members have clearly defined roles
and accept their responsibilities. (ACSA, 1997, p. 1)
With the implementation of an integrated, standards-based accountability
system, schools and districts must be able to devise and implement their own plans
for success. Now that states identify academic standards for what every child needs
to know, it must be clearly defined what a highly qualified teacher needs to know
and be able to do (Goldrick, 2002). Options must be provided to schools so that
barriers to improvement can be removed. Currently there is no one coherent
accountability system in place for all of California’s schools, and districts are asked
to comply with a variety of regulations for each state and federal program (ACSA,
1997). If there is a desperate need for one integrated accountability system that
coordinates all local, state, and federal requirements to focus on improved student
learning, then the current system, including the evaluation process, must be
36
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
changed to measure gains in student achievement rather than adherence to rules and
regulations.
No Child Left Behind
The NCLB Act of 2001 was signed into law on January 8, 2002. The
NCLB legislation, reauthorizing the Elementary and Secondary Education Act
(ESEA), is the federal government’s attempt to establish a national framework for
accountability. The accountability provisions of NCLB require states to set clear
timelines for improving student achievement, with an emphasis on closing achieve
ment gaps between low-income students, minority students, and their peers. In
order to accomplish this daunting expectation, every state and school district is
responsible for ensuring that students meet state standards for proficiency in read
ing and mathematics by 2014. Schools must use disaggregated data of significant
student subgroups to ensure that all groups of students are making adequate yearly
progress (AYP). In addition, all states must define a qualified teacher and ensure
that low-income and minority students are not taught disproportionately by inex
perienced, unqualified, or out-of-field teachers. The NCLB law is meant to make
meaningful school reform a reality. The premise of the NCLB law is to guarantee
all children an opportunity to learn from a high-quality curriculum, delivered by a
highly qualified teacher, with high standards as the expectation for all (NCLB,
2002).
National mandates such as NCLB’s demand for “highly qualified teachers,”
along with new state teaching standards, require that all teachers be credentialed
and have the ability to teach in an ever more complex learning environment, par
ticularly in urban schools, where it is critical that education be transformed to be
37
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
successful for all learners (U.S. Department of Education, Office of the Under
secretary, 2004).
Under NCLB, as states have identified academic standards for what every
child needs to know, it must be clearly defined what a highly qualified teacher
needs to know and be able to do (Goldrick, 2002). Previously cited recent studies
offer compelling evidence that teachers are one of the most critical factors in how
well students achieve. For instance, studies in both Tennessee and Texas found
that students who had highly effective teachers largely outperformed students who
had ineffective teachers. It was noted in the Tennessee study that students with
highly effective teachers for 3 years in a row scored 50 percentage points higher on
a test of mathematics skills than those whose teachers were ineffective (Sanders &
Rivers, 1996). Hence, NCLB includes provisions that all teachers in core academic
areas must be highly qualified in the core academic subjects that they teach by the
end of the 2005-2006 school year (NCLB, 2002). There is also a requirement that
newly hired teachers in Title I programs or Title I schools be highly qualified
immediately (U.S. Department of Education, Office of the Deputy Secretary,
2004). Under quality teaching, the term highly qualified is specifically defined by
provisions stated in NCLB. The NCLB law outlines a list of minimum require
ments that are related to content knowledge and teaching skills that a highly
qualified teacher would meet, such as the required Bachelor’s degree and full state
certification, along with demonstrated content knowledge in the subjects taught
(U.S. Department of Education, Office of the Deputy Secretary). The NCLB law
recognizes the importance of state and local control of education and, therefore, it
provides the opportunity for each state to develop its own definition of highly
qualified, decide what is necessary for certification, and then determine subject
38
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
matter competency that remains consistent with NCLB as well as with the unique
needs of the particular state. States have flexibility to design ways to allow
teachers to demonstrate competency in the subjects that they teach; this is especi
ally desirable for teachers with a number of years of experience. It may seem
obvious, but NCLB focuses on the importance of all teachers knowing the subject
matter that they teach. The intuitive claim by parents, students, and educators is
that a teacher’s knowledge of subject matter is critical if students are to achieve to
high standards (U.S. Department of Education, Office of the Deputy Secretary,
2004). If research findings show that teachers who know the subject matter are
more effective in the classroom, then having teachers who know the content well
will result in classes that lead to improved student learning (Monk, 1994). Sandra
Feldman, president of the American Federation of Teachers clearly stated, “You
can’t teach what you don’t know well” (Feldman, 2001, p.l#). A prevailing
concern is that highly qualified is not synonymous with highly effective and that a
highly qualified teacher is not necessarily a teacher o f high quality.
Teacher Quality
Besides being one of the most prominent aspects of NCLB, teacher quality
is one of the most challenging areas for states to address (Goldrick, 2002). By the
end of the 2005-2006 school year, states must ensure that all teachers of core
academic subjects are “highly qualified” (Goldrick; NCLB, 2002). This brings
back the dilemma over the meaning of teacher quality. Individual states have
defined academic standards for what every child needs to know. States must also
clearly define what a highly qualified teacher needs to know and be able to do,
before they can purposefully construct a teacher evaluation policy (Goldrick).
39
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Since NCLB uses the term highly qualified teacher and the NCT has
consistently used the term highly qualified teachers, in its reports, one is led to
believe that the research-based characteristics developed by the Commission are
important. According to NCTAF (2003), highly qualified teachers:
• possess a deep understanding of the subjects they teach;
• evidence a firm understanding of how students learn;
• demonstrate the teaching skills necessary to help all students achieve
high standards;
• create a positive learning environment;
• use a variety of assessment strategies to diagnose and respond to
individual learning needs;
• demonstrate and integrate modem technology into the school
curriculum to support student learning;
• collaborate with colleagues, parents and community members, and other
educators to improve student learning;
• reflect on their practice to improve future teaching and student
achievement;
• pursue professional growth in both content and pedagogy; and
• instill a passion for learning in their students, (p. 5)
In the past, there have been no highly qualified teacher criteria, and past
practice teacher licensure requirements have been minimal. Candidates needed to
hold a Bachelor’s degree, maintain a minimum grade point average, and pass a
criminal background check. In the past few years, states have become more
stringent by establishing tougher licensure requirements. Currently, most states
have set minimum scores for licensing exams, and many now require candidates to
have majored in an academic subject. In addition, 43 states require teaching
candidates to pass at least a basic skills test in order to receive a license (“Quality
40
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Counts,” 2002) or enter into a recognized teacher preparation program (U.S.
Department of Education, 2004).
There are currently several national efforts to define what constitutes good
teaching and to develop ways to measure it. The bottom line is, “Traditional
licensure does not guarantee teacher quality” (Goldrick, 2002, p. 3). States are now
designing performance-based licenses for teaching that require demonstration of
subject knowledge and teaching skill instead of basing licenses on course credits
and hours of professional development. More than half of the states require
teachers to take the Praxis II written tests of subject knowledge and pedagogy to
even receive a provisional teaching license. Finally, states are beginning to use
performance-based systems to create “tiered professional designations,” which may
include classifications of initial, provisional, professional, and master teacher
licenses (Hirsch, Koppich, & Knapp, 2001).
According to research data, more than 20 states have adopted universal
standards and discipline-specific standards that were developed by the Interstate
New Teacher Assessment and Support Consortium (INTASC) for beginning
teachers (Hirsch et al., 2001). As stated earlier in this literature review, the NBPTS
has also created standards and research-based assessments to identify accomplished
teachers. These outstanding and experienced teachers are encouraged to seek
NBPTS recognition for a monetary incentive and for the distinction of being among
the best teachers in the country.
Even this model is not without criticism. Finn and Wilcox (2000), of the
conservative Thomas B. Fordham Foundation, stated, “Unless the NBPTS can
guarantee that teachers who earn its credential do an outstanding job of imparting
skills and knowledge to their pupils, generous rewards to those teachers do not
41
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
make sense” (p. 1). They contended that the NBPTS ignores classroom results, and
it can prove that its credential guarantees “a boon for students,” California is better
off reserving its $30,000 bonuses for teachers who have demonstrated their capa
bility in the one area that really matters: pupil achievement. “Greatness is in the
hard evidence of how much and how well one’s pupils learn” (p. 2).
Nevertheless, the models from INTASC and NBPTS for beginning and
experienced teachers are good starting points for states that are developing per
formance-based teaching standards and models for those states that have created
their own.
Sample Performance-Based Standards
Figures 1 and 2 present samples of performance standards and criteria from
Colorado and Kentucky, respectively.
Performance-Based Standards for Colorado Teachers
Standard—The teacher shall be knowledgeable about strategies, planning processes, assessment
techniques, and appropriate accommodations to ensure student learning in a standards-based
curriculum.
Performance Criteria—The teacher has demonstrated the ability to design short- and long-range
standards-based instructional plans; develop valid and reliable assessment tools fo r the
classroom; develop and utilize a variety o f informal and formal assessments; assess, compare,
and contrast the effects o f various teaching strategies on individual student performance relative
to content standards; use assessment data as a basis fo r standards-based instruction; provide
effective verbal and written feedback that shape improvement in student performance on content
standards; prepare students for [state assessments]; ensure that instruction is consistent with
school district priorities and goals and [state content standards].
Figure 1. Performance-based standards for Colorado teachers. From Improving
Teacher Evaluation to Improve Teaching Quality: Issue Brief, Education Policy
Studies Division, by L. Goldrick, 2002, Washington, DC: National Governors
Association Center for Best Practices.
42
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Kentucky Performance Standards—Experienced Teacher Standards
Standard—The teacher designs and plans instruction and learning climates that develop student
abilities to use communication skills, apply core concepts, become self-sufficient individuals,
become responsible team members, think and solve problems, and integrate knowledge.
Performance Criteria—The extent to which the teacher’ s plan focuses instruction on one or more
o f Kentucky's student academic expectations; develops the student’ s ability to apply knowledge,
skills, and thinking processes; integrates skills, thinking processes, and content across
disciplines; proposes learning experiences that challenge, motivate, and actively involve the
learner; proposes learning experiences that are developmentally appropriate fo r learners and
describes experiences fo r multiple levels o f complexity to accommodate students at different
levels o f performance incorporates strategies that address physical, social, and cultural
diversity and show sensitivity to differences; establishes physical classroom environments to
support the type o f teaching and learning that is to occur; includes creative and appropriate
uses o f technology as a tool to enhance student learning; includes appropriate assessment
strategies and processes; includes comprehensive and appropriate school and community
resources that support learning; includes learning experiences that encourage students to be
adaptable, flexible, resourceful, and creative.
Figure 2. Performance-based standards for Kentucky teachers. From Improving
Teacher Evaluation to Improve Teaching Quality: Issue Brief, Education Policy
Studies Division, by L. Goldrick, 2002, Washington, DC: National Governors
Association Center for Best Practices.
As the attributes for a true quality teacher unfold, states are beginning to
specify the necessary knowledge and skills of a “highly qualified teacher.” If there
is no clear framework about what teachers should know and be able to do at the
different stages of their careers, evaluation itself lacks a clear purpose and will
never be able to provide consistently reliable and useful information about what is
happening in the classroom.
Improving Teacher Practice
The effects on student achievement by working for consecutive years with
highly effective or ineffective teachers are known. The school students attend, and
43
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
what their teachers know and do are more important influences on student achieve
ment than any family characteristics or ethnicity (Kaplan & Owings, 2003). In
addition, results on standardized tests with a 100-point scale show that the cumulat
ive impact over 3 years of effective elementary teachers is estimated to produce
more than a 50-point difference in mathematics scores and a 35-point difference in
reading scores (Haycock, 1998). Haycock also found that 5 consecutive years with
highly effective teachers produced dramatic achievement gains in all ability level
groups, while those with ineffective teachers showed virtually no growth. There is
agreement between politicians and educators that a positive and clear predictive
relationship exists between the basic skills of teachers and student achievement
(Darling-Hammond, 1998; Haycock, 1998). Even with all of the NCLB elements
of a highly qualified teacher, there is no evidence to suggest that possessing content
knowledge alone is sufficient to be an effective teacher (Barnett Barry, 2001, as
cited by Kaplan & Owings, 2003). What is it that improves teacher practice?
According to Kaplan and Owings, the bottom line—and a “hotbed” of disagree
ment—is that, even though teachers’ strong content knowledge and verbal skills
have demonstrated some higher student achievement, “they may be necessary but
not sufficient conditions for high-quality teaching and learning” (p. 2). Research
acknowledges that, among teacher candidates who have come from strong educa
tion schools where students have content knowledge linked to teaching practices
and many real-world opportunities to integrate and use what they learn with
students in well-supervised settings, their preparation and certification can be a
strong predictor of teacher quality (p. 4). Therefore, being a “content expert” alone
is no guarantee of potential teaching effectiveness.
44
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Although teacher evaluation typically serves as a process to measure job
performance, it can be an informational tool to help administrators to identify
teachers who need special help or support toward improving their teaching practice.
Evaluation can help a teacher by providing a venue for constructive feedback and
securing professional development for the teacher that is aligned with performance-
based teaching standards.
Very few states have developed any sort of meaningful teacher assessments
to transform the traditional evaluation and licensure processes. Three states—
Connecticut, North Carolina, and Ohio—require teachers to demonstrate competent
instructional practices prior to moving beyond the initial teacher’s license
(Goldrick, 2002). In fact, Ohio is the first state in the nation to fully implement
Praxis III, which is a performance-based teacher assessment tool to determine
whether a beginning teacher will move from a provisional to a professional
teaching license. In order to improve teacher practice, two other teacher evaluation
strategies are found to have promise in informing professional improvement:
portfolio assessment and peer review.
Portfolio Assessment
The idea of teachers assembling a professional portfolio as part of an evalu
ation system seems to have gained popularity in the past few years (Danielson &
McGreal, 2000). A portfolio can be utilized as an excellent reflective evaluation,
but there must be clear criteria for the collected evidence and teachers must know
the difference between a portfolio versus a “collection of stuff.” A portfolio used
for assessment is a collection, but it is a collection of a teacher’s work that, when
assembled, will demonstrate to the evaluator that the teacher has a true command of
45
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
subject content and instructional skill (Danielson & McGreal; Goldrick, 2002). A
portfolio can help teachers to improve their instructional practices as well as be
used for self-evaluation. Connecticut and Vermont use portfolio assessment as part
of their evaluation systems (Goldrick). In fact, in Connecticut, a second-year
teacher must prepare a portfolio as a part of the initial licensure; in Vermont, all
beginning teacher candidates must have a portfolio as part of the initial licensure
application, and experienced teachers must construct a portfolio as part of their re
licensure procedures.
Peer Review
Peer review can also be used in an overall teacher evaluation. Although
peer review developed from peer coaching and mentoring, peer review allows
consulting teachers to conduct formal evaluations of colleagues’ performance and
recommend professional assistance or even dismissal (Goldrick, 2002). Currently,
very few school districts have implemented peer review programs. Implemented as
a way to improve the quality of teaching in their most troubled urban high schools,
Chicago, as well as Columbus and Toledo, have implemented the best-known peer
review programs.
In 1999 California enacted the first state peer review law. Known as Peer
Assistance and Review (PAR), the review process is generally, but not exclusively,
used with nontenured teachers (Danielson & McGreal, 2000). In this system,
teachers conduct formal, summative evaluations of their colleagues, and their
recommendations for a teacher’s employment status are typically accepted by the
school district. In a peer assistance and review program (such as PAR in
California), peer assistance (formative assessment) is combined with peer review
46
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
(summative assessment related to continuing employment status). Schools and
districts that apply PAR usually use it for new teachers or experienced teachers
who are having difficulties in teaching. “The primary purpose of this kind of
program is to assist teachers whose employment status may be threatened, and to
increase retention” (p. 58).
More states are looking for ways to measure the effectiveness of teachers
based, in part, on student learning. Unfortunately, there is a “lack of assessments,
data systems, and evaluation processes that are capable of capturing the complex
ity of teaching skills—knowledge, pedagogy, classroom management—and their
impact on student leaming”(Goldrick, 2002, p. 4). Although testing dominates the
approaches to ensure teacher quality nationally, tests alone do not develop the
essential connection between teacher performance and student achievement
(Wasley & McDiarmid, 2004). In order to address that connection directly, value-
added assessment is being used in several states.
Tennessee Value-Added Assessment System
William Sanders and colleagues developed a method for comparing student
test scores over time, which they called the Tennessee Value-Added Assessment
System (TVAAS; Goldrick, 2002; Tucker, P., & Stronge, 2005; Wasley &
McDiarmid, 2004). Value-added assessment tracks the annual learning gains of
students. It can gauge the impact of an individual school or a teacher on student
achievement more accurately and fairly than traditional assessments that are
designed to measure cumulative achievement over time. Percentiles are tradition
ally used to rank a student against peers. In a value-added approach, a student’s
current level is ascertained in a particular subject area and the measurement is made
47
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
from that point. Aggregating scores by teacher allows this type of assessment to be
used in identifying the impact of the teacher on student growth. Researchers track
student test scores from one year to the next and analyze results using hierarchical
linear modeling that controls for socioeconomic and other factors (Wasley &
McDiarmid). Researchers contend that they can credibly identify the “value” that
an individual teacher adds to a student’s performance by measuring the gains made
from one year to the next. Rather than measure one class of students against
another, the individual student’s baseline scores become the control (p. 13).
Sanders and colleagues suggested that using value-added data provides teachers
with useful formative data that would be available nowhere else. The connection
to teacher evaluation is that, by having the baseline data for each student at the
beginning of the year, a teacher can evaluate his or her own performance based on
student growth. Currently, school districts in more than 20 states are adopting
value-added modeling (VAM) to apply in their local districts and schools (p. 14).
Value-added models have developed a wide appeal and provide important
and useful information to schools, districts, and teachers. The value-added data are
especially appealing to low-socioeconomic status (SES) schools to enable them to
measure their progress by providing year-to-year information on the value that they
are adding to student achievement. If a student is assessed at the beginning and end
of the school year, the “value” (or growth) is a more accurate way of looking at
student achievement. Unfortunately, teachers and union leaders are fearful that
data from value-added models could be used in punitive ways if individual scores
did not show growth. The concern is that teachers identified as ineffective, based
on a VAM, could be drummed out of the profession by outside sources, such as the
media (since test scores are public), rather than get the support needed to improve.
48
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Concerns about the reliability of test scores have influenced policymakers
against holding teachers accountable for student learning. This is because student
achievement reflects numerous influences that are unrelated to an individual
teacher, such as past schooling, divorce in the family, behavioral and health
problems, or other external factors (Goldrick, 2002; Tucker, P., & Stronge, 2005;
Wasley & McDiarmid, 2004). Therefore, student tests using a value-added assess
ment system that is closely aligned with the state’s academic standards should be
only one factor in the teacher evaluation system. Other factors, such as classroom
observation, student work, teacher portfolios, self-evaluation, peer review, and
verification of appropriated credentials and licensure, are also critical to the evalua
tion system. The final factor that plays an important role in evaluation is that of
leadership.
Leadership
According to research by William B. Ribas (2000), it is fortunate that, in
most of the relationships between an evaluator and a teacher, the primary focus is
on the educational and social/emotional processes. According to James Walsh,
superintendent of the Public Schools of Brookline, Massachusetts, and past deputy
superintendent of the Boston Public Schools, “Leadership is more psychology than
anything else. The process of being evaluated is a significant emotional event for
many people” (as cited in Ribas, p. 31). The way that one perceives oneself in the
work environment, meaning how one sees oneself in one’s own professional career,
is an extremely important component to one’s own self-image in the work place.
Therefore, a threat to a teacher’s career, whether it is real or perceived, often
triggers a fight-or-flight response, making evaluation an emotionally charged
49
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
activity. These emotions intensify exponentially when the message (perceived or
real) to the classroom teacher is, “Your performance is less than satisfactory”
(Ribas). Thus, the leadership of the evaluator takes on an important social/emo
tional role.
Ribas (2000) reported that it is very important for an administrator who is
evaluating teachers not only to recognize the excellent performance that the major
ity of the teachers provide each and every day but also to develop a culture of early,
honest, and sensitive notification to teachers when performance in an instructional
area is less than acceptable. This includes the important fact that the school district
should develop and maintain a high level of consistency (interrater reliability)
among its evaluators. Teacher associations often hear concerns from teachers about
the varying evaluation standards across departments or buildings. Without raising
this interrater reliability, teachers tend to feel that performance evaluations are
more subjective rather than objective assessments. Ribas found that increasing
interrater reliability improved morale in general and pushed low-performing
teachers to recognize their deficiencies, as opposed to identifying them as the
inaccurate perceptions of the evaluator. When examining the differences between a
low-performing teacher who improves performance in support of student learning
and stays in teaching and a low-performing teacher who eventually leaves the
system, it is the degree to which the teacher can own the performance-related
problems and face this difficult reality early in the process.
It is important that the school leader know the differences related to
effective, efficient, and good practice. Administration is defined by Sergiovanni
(1991) as a process of working with and through others to accomplish school goals
efficiently. “The essential elements of this definition are action, goals, limited
50
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
resources, and working with other people.. . . When a principal is successful in
matching their actions to their goals, they are considered to be effective” (p. 15). It
is typical for a principal to work in an environment that is known to be of limited
resources. These resources tend to be money, time and space. “The leader must
figure out a way to obtain maximum results for the school with minimal resources.
When a principal is able to do that, he or she is considered efficient” (p. 15).
Sergiovanni identified effectiveness and efficiency as the two major concerns of
school administration. “The leader must realize that neither effectiveness nor
efficiency is the same as good. Goodness is not something one discovers but
something one decides” (p. 16). According to Paul W. Taylor, “When it comes to
goodness we must decide what ought to be the case. We cannot discover what
ought to be the case by investigating what is the case” (1961, as cited in
Sergiovanni, 1991, p. 16).
Just as teachers need training, evaluators need preservice training oppor
tunities to conduct more accurate and effective teacher assessments. Training could
focus on a variety of skills, such as analyzing effective teaching practice, determin
ing a teacher’s impact on student learning, and providing leadership for pro
fessional development and remedial assistance (Goldrick, 2002). Developing and
preparing school leaders for the act of teacher evaluation tasks “can enhance the
quality of instruction, promote student achievement, and make evaluations more
accurate and more consistent” (p. 6).
Trends, Innovations and National Reforms
The trends, innovations, and national reforms with regard to the teacher
evaluation process and how it impacts teacher performance in support of student
51
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
learning has taken a new direction. According to Fredric Genck (1991), the
solution to urban crisis and school failure and to America’s future progress and
success is a balance of participative management with performance measures and
school board accountability. On one hand, a school must perform effectively with
demonstrated good results to justify the funding needed for good schools and fair
salaries, while on the other hand, teachers need supportive participative manage
ment with planning, information, evaluation and compensation, recognition and
reward, teamwork, trust and confidence, communication, and participation.
Through action on three levels—board, administrators, and teachers—in order to
improve learning, the concept of school governance, management and performance
can be simplified by thinking of three components of a triangle. The three com
ponents inside the triangle contain the following:
1. Accountable boards', policies and leadership for teamwork and
success, with boards evaluating performance (not just “rubber stamps”);
2. Participative management', positive support for teachers and
administrators; performance-based planning, organization information,
evaluation and compensation;
3. Performance measures', measures of learning, confidence, morale,
and cost-effectiveness; plus an outside audit for credibility and as the
catalyst for measurement and improvement, (pp. 47-48)
Implementation of the “triangle” has three specific action steps:
1. Replace traditional “rubber stamp” school boards, and the conflict of
recent years, with teamwork and accountability.
2. Build a participative and result-oriented management system, to
replace traditional authoritarian school administration.
3. Measure school results, starting at the bottom with parent/teacher
satisfaction and student learning, (p. 49)
Ever effective school should have a plan. This plan models support in
exchange for measures of results and performance-based compensation (Genck,
52
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1991). With a plan and a good evaluation system, there should be greater overall
productivity and increased job performance for staff, with an upward spiral of
teamwork, student learning, and success instead of conflict and decline.
Conclusion
Based on an in-depth review of the literature, a good deal remains to be
done to ensure a high-quality teacher for every classroom. There is a strong
relationship between student achievement and a teacher’s skills, knowledge, and
practices. Many studies have reported that what teachers know and can do are
crucial to what students learn. In fact, the research contends that teacher expertise
affects all of the core tasks of teaching.
The literature reveals that educators must be held accountable for student
learning. Comprehensive and continuous improvement using performance-based
standards for teacher assessment, formative and summative evaluation systems,
qualities of a “highly qualified” teacher, value-based assessment, standards-based
reform, and capable leadership is identified as part of an accountability system in
which all elements are aligned and work together for the purpose of promoting and
improving student learning. There are approaches and methods for the novice
teacher as well as for the veteran teacher. There must be a continued examination
of not only what the students learn in the classroom of beginning teachers but what
these beginning teachers actually do. Finally, multiple measures are needed to
assess what students know and can do in order to determine what the teachers are
teaching and whether students are actually learning. Establishing what teachers
actually do in the classroom and why they do what they do is important as new
accountability measures are introduced.
53
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Many studies have connected school-wide initiatives with teacher practice
to determine whether that is what leads to increased student learning. Studies have
identified the importance of good professional development based on the individual
needs of teachers as a result of evaluation systems. Research supports policies and
practices that build a comprehensive standards-based accountability system for
continuous improvement of multiple teacher evaluation systems and how they may
or may not effect increased student achievement. It appears that the gap in the
literature appears to be the connection of the teacher evaluation process and its
impact on teacher performance in support of student learning, especially in an
urban school district. This qualitative study strives to bridge the gap in the litera
ture by its identification of the connections among the teacher evaluation process,
its impact on teacher practice, and its result of greater student learning in a high
achieving urban elementary school as identified by the state of California.
54
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 3
METHODOLOGY
This chapter describes the research methodology used in the study. As this
study addressed the research questions through a qualitative tradition of inquiry, a
case study approach was deemed the most appropriate to use (Creswell, 1998). The
purpose of this qualitative study was to determine whether the teacher evaluation
process influences teacher performance in support of student learning. The school,
as an organization, is designed to fulfill specific mandates and thereby achieve
specific objectives. If the school is thought of as a business, and the customer is
the student, then the school is accountable to its community for fulfilling its pur
pose, which would be quality teaching in order to provide the ultimate goal to the
primary customer (the student): the opportunity to learn (Van Der Linde, 1998).
This phenomenological case study examined the teacher evaluation process in place
at a high achieving urban elementary school. Furthermore, this study closely
examined whether such an evaluation process had an impact on teacher perform
ance in support of student learning. The data collection techniques, the instru
mentation tools, and data analysis are described. Multiple methods of interviews,
observations, surveys, and document analysis were used in this study.
Theoretical Base
The theoretical base that guided and directed this proposed study main
tained that the quality of the school was determined by teacher performance in the
classroom and that evaluation played a key role in the monitoring of a teacher in
order to constantly improve the teacher’s practice.
55
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Conceptual Framework
The conceptual framework shown in Figure 3 depicts the relationship of
supervision to teacher evaluation, teacher evaluation to improved teacher practice,
and improved teacher practice to student learning. A medley of factors other than
teacher evaluation lead to improved teacher practice, such as professional staff
development, data-driven analysis, parent involvement, school culture, principal
leadership, and BTSA. In this particular relationship, teacher evaluation was
studied as the primary factor depicted in the model that leads to student learning.
Primary Factor
V
Accountability
Standards
NCLB
Stull Bill
......-HJ\ Instruction/ 1 — Teacher
U \
Improved '— ' \. Student
Context: |— Supervision j — Evaluation
vV
Practice i — > S Learning
V
Other Factors
Professional Development
Leadership
Beginning Teacher Support and Assessment (BTSA)
Data Driven
School Culture
Parent Involvement
Figure 3. What leads to student learning.
Research Questions Restated
Prior to formulating the research questions for this study, common themes
within the literature were identified and closely examined. The research questions
were as follows:
56
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1. How is the teacher evaluation process practiced at the school site?
2. How does the teacher evaluation process impact teacher behavior?
3. What factors exist at the school that positively or negatively impact
teacher practice?
4. How does the district’s/school’s leadership influence the teacher
evaluation process?
The development of the four research questions was reached as a result of
the conceptual framework (Figure 3). These formulation of these questions
followed the recommendations of Miles and Huberman (1994, as cited in Creswell,
2003) in limiting the number of questions to those topics specifically investigated
in a systematic way through interviews, observations, documents, and archival
material. The research questions (see Figure 4) made the theoretical assumptions
explicit, actually utilized the conceptual framework, and presented what the
researcher planned to investigate.
Design of the Study
A case study, in which the researcher investigates in depth a program, a
process, an event, an activity, or one or more individuals, is qualitative research in
that the case is bounded by time and activity and the researcher collects detailed
information using a variety of data collection methods over a sustained period of
time (Stake, as cited in Creswell, 2003). Recognizing that all research methods
have limitations, the researcher concurred with Creswell (1998) that the research
questions lent themselves well to a qualitative design study, the factors being
investigated warranted a detailed view, the study took place within the natural
context of the school setting, and the researcher was an active learner telling the
57
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Research Question #1:
How is the teacher
evaluation process
practiced at the school
site?
Research Question #4:
How does the
district’s/school’s
leadership influence
the teacher evaluation
process?
Research Question #2:
How does the teacher
evaluation process
impact teacher
behavior?
Research Question #3:
What factors exist at
the school that
positively or
negatively impact
teacher practice?
Figure 4. Research questions.
story from the participants’ perspective. Due to the descriptive nature of using a
qualitative design and investigating nonquantifiable relationships among existing
variables, this study focused on a mixed methodology and included a triangulation
of data approach in which the same dependent variable was investigated using
multiple procedures, including interviews, surveys, direct observation and partici
pant observation, documents, archival records, and physical artifacts.
58
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Sample and Population
Criteria fo r Selection
Purposefully selected sites are used in a qualitative study to help the
researcher to understand the problem and answer the research questions (Creswell,
2003). The urban school selected for this case study was identified by the CDE as
a high-achieving Title I school. Purposive sampling techniques were employed as
the process of selection for this study to locate an urban elementary school that met
high-achieving Title I school requirements and ensured that the participants met
specific criteria. The CDE identified the following criteria as eligibility require
ments:
1. The poverty index at the school had to equal at least 40% of all students
enrolled, not just those tested on the annual California state assessment.
2. Schools had to have 2 consecutive years of assessment data in the state
accountability system for growth results to be calculated for the API.
3. All eligible Title I schools had to have made AYP for 2 consecutive
years, as defined under NCLB and calculated by CDE in accordance with the
State’s NCLB Accountability Workbook.
4. An eligible Title I school had to demonstrate an achievement level of
twice the schoolwide API growth target and twice the API growth target for the
socioeconomically disadvantaged subgroup for 2 consecutive years.
5. An eligible Title I school had to attain or surpass the following current
API growth median score at the appropriate level: (a) elementary schools 729, (b)
middle schools 685, and high schools 668 (CDE, 2004).
59
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Case Study Site
Catson Elementary School (pseudonym) met the stated qualifications of
high-achieving Title I school status and was the focus of this single-school case
study. This mid-size urban elementary school (kindergarten through grade 5, or K-
5) in the Rio Hondo Unified School District (RHUSD; pseudonym) is located in the
southern California area, specifically in the Los Angeles County area. The school
has an average daily attendance (ADA) of 630 students and a staff of 29 certificated
teachers. The school is one of 18 elementary schools in the district.
Catson has shown sustainable growth in test scores as the school has con
tinued to meet all federal NCLB requirements in regard to making AYP. Accord
ing to AYP data, the three significant subgroup populations at Catson—English
Language Learners, Hispanic or Latino students, and socioeconomically disad
vantaged students—met the AYP target score of 24.4% scoring proficient. Table 1
describes the ethnic breakdown of Catson Elementary School for all students in
grades K-5. Since 86% of the students in the school are Hispanic, it is more than
just a subgroup of the school; it is the majority of the student population.
In addition to the federal AYP, the state of California maintains an API
proficiency goal of 800. For the past 3 years Catson has made significant growth
toward reaching the state target (see Table 2).
RHUSD has an enrollment of approximately 24,000 students within its 18
elementary schools, 7 middle schools, and 4 high schools. This urban school dis
trict has a large minority population of predominately Hispanic students. Although
it was not a criterion of site selection, of interest to the researcher was the diversity
as well as the stability of the school staff, with approximately one half of the 29
teachers acknowledging themselves as minority (information received from
60
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 1
Student Enrollment at Catson Elementary School by Ethnic Group, CBEDS Data
2005
Ethnic group %
American Indian or Alaska Native 0
Asian 1
Pacific Islander < 1
Filipino 1
Hispanic or Latino 86
African American 5
White 6
Note. CBEDS = California Basic Educational Data System.
Table 2
Catson Elementary School Academic Performance Index (API) Growth Data
(Released 2004/2005 School Year by California Department o f Education)
2001 to 2002 2002 to 2003 2003 to 2004
Percent tested 100 99 100
API growth score 689 746 754
the Catson site administrator based on the 2004 California Basic Educational Data
System [CBEDS] report). As Tables 3 and 4 show, there is a variety of ethnicities
among the Catson teaching staff, but there is stability of the staff by years of
61
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 3
Certificated Staff at Catson Elementary School According to Ethnicity
Ethnic group
/
American Indian or Alaska Native 0
Asian 2
Pacific Islander 1
Filipino 0
Hispanic or Latino 10
African American 1
White 15
Table 4
Certificated Staff Stability at Catson Elementary School
Tenure at the school
/
1 year 0
2-5 years 2
6-10 years 13
11+ years 14
teaching at the school. The researcher found it noteworthy that approximately one
half of the teachers had been teaching at Catson for 11 or more years. Table 5
shows the certificated staff status of the teachers at Catson Elementary School.
62
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 5
Certificated Staff Status at Catson Elementary School
Certificated status
/
Permanent teacher 28
Probationary teacher 1
Provisional/temporary teacher 0
University intern teacher 0
Catson Elementary School was one of 15 schools selected for part of a
larger study conducted at USC. Stratified purposeful sampling was utilized in
selecting the 15 schools that participated in the study. These schools equally
represented elementary schools, middle schools, and high schools at high-
achieving, Title I schools in the southern California region.
Instrumentation
The researcher utilized a triangulation of the data to analyzing the following
instruments: a survey, interviews, observations, and a collection of documents and
other artifacts related to the teacher evaluation process. This study was part of a
larger study conducted at the University of Southern California consisting of 15
similar studies conducted throughout the Los Angeles County and Orange County
areas of southern California at purposefully selected elementary, middle, or high
school sites with common factors of being Title I schools and high-achieving
schools. The development of the instruments was a collective effort of the
63
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
15-member cohort of doctoral students. Each subgroup of 4-6 researchers
developed one of the instruments that was used in this research.
Development o f the Survey Protocol
The survey for this study was developed by a subgroup of five researchers
in the thematic dissertation group of 15 doctoral students, based on Ebmeier’s
(2003) teacher’s survey used to illustrate a link between teacher efficacy and
commitment. The intent of the survey was to generalize from a sample of staff to
the entire school population. Both teacher and administrator versions were created
for this study to collect data on the perceptions of both. Included in the survey
were statements in the following domains: Background Information, Policy,
Teacher Evaluation, Ongoing Teacher Supervision, School Efforts, School Culture,
and Personal Efficacy/Commitment to Teaching. Respondents’ evaluations of each
statement (with the exception of statements in the Background Information section)
were based on a 4-point Likert-type scale of 0 = Strongly Disagree, 1 = Disagree,
2 = Agree, and 3 = Strongly Agree (Gall, Gall, & Borg, 2003).
A pilot study was conducted by a previous group of researchers (2005
doctoral candidate cohort) with the teacher version of the survey on 11 respondents.
Their data were tabulated in Statistical Package for the Social Sciences (SPSS®)
version 11.5. Four types of statistical tests were performed on the data to deter
mine validity and reliability. First, descriptive statistics were produced in order to
understand the frequencies and means of each item answered. Second, an explora
tory factor analysis test was conducted to identify potential factors between strong
self-efficacy (Q48 = Agree) and items regarding perceptions of evaluation and
64
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
supervision of instruction. The rotated component matrix (Table 6) indicated that
the variables loaded in two separate components.
Table 6
Rotated Component Matrix
Component
Survey item 1 2
8. I am confident in my administration’s ability to monitor
my instructional practice. .824 .303
19. I am comfortable going to my administrators for support. -.043 .979
20. I am aware of the goals and objectives of this school. .904 -.389
The thematic subgroup found that these results suggested that two factors
linked teacher self-efficacy with their perceptions of both evaluation and super
vision: (a) personal beliefs (Component 1 = Q8 and Q20), and (b) administrative
responsiveness (Component 2 = Q19). A reliability coefficient was calculated for
all items of the survey and yielded alpha = .8912 (based on eight cases in which
respondents evaluated all of the statements). Pearson correlation coefficients were
determined for all items. Although many items showed strong positive
correlations, Table 7 shows that some items were exceptionally significant.
The results of the field tests by the thematic dissertation subgroup suggested
that the survey met standards of both validity and reliability and showed signs of
generating data that demonstrated connections among self-efficacy, school culture,
65
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 7
Pearson Correlation Coefficients
Item 1 Item 2 p value Sig. level
(1) I am aware of and
understand the district’s
policy regarding teacher
evaluation.
(3) My administrator
frequently observes my
classroom for the purpose of
evaluation.
.832 .001
(2) I agree with the goals and
objectives o f my district’s
policy on teacher evaluation.
(10) The administrator
frequently observes my
classroom.
(10) The administrator
frequently observes my
classroom.
(8) I am confident in my .858 .001
administration’s ability to
monitor my instructional
practice.
(12) I see the administration’s .866 .001
supervision of instruction as
separate and non-evaluative.
(22) I am satisfied with the .873 .001
professional competency and
leadership ability of the
administration.
(12) I see the administration’s
supervision of instruction as
separate and non-evaluative.
(13) I believe that my
administration’s supervision
of instruction improves my
instructional practice.
(13) I believe that my
administration’s supervision
of instruction improves my
instructional practice.
(13) I believe that my
administration’s supervision
of instruction improves my
instructional practice.
(22) I am satisfied with the .932 .000
professional competency and
leadership ability of the
administration.
(5) I see my administrator’s .804 .003
implementation of the teacher
evaluation policies as an
integral part of my
professional growth.
(8) I am confident in my .865 .001
administrator’s ability to
evaluate my instructional
practice.
(16) I believe that my .. 804 .003
participation in the teacher
evaluation process at this
school has led to my
professional growth.
66
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 7 (continued)
Item 1 Item 2 p value Sig. level
(13) I believe that my
administration’s supervision
of instruction improves my
instructional practice.
(19) I receive meaningful
feedback following an
evaluation.
.909 .000
(13) I believe that my
administration’s supervision
of instruction improves my
instructional practice.
(23) I am aware of specific
things that administrators
look for when visiting my
classroom.
.830 .002
leadership, and the effects of teacher supervision and evaluation on teacher
practice. Appendix C contains a copy of the survey instrument.
Development o f the Interview Protocol
The interview protocol was initially created by a six member subgroup of
the thematic dissertation group on teacher evaluation, and it was subsequently
modified by the entire group of 15 researchers. The questions were designed to
elicit in-depth answers to the study’s four research questions. The interview
questions were organized by their corresponding research question to aid in the
analysis process. Figure 5 presents the certificated staff interview questions (see
also appendix D) organized by corresponding research questions. Figure 6 shows
the additional questions designed for school and district administrators (see also
appendix E).
67
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Research Question 1: Describe the evaluation process at the school site.
1.1 How involved are you in developing your evaluation for the school year?
1.2 Are you familiar with all the documents you need to complete for your evaluation?
1.3 How do you know what your principal is looking for during your classroom
observation?
1.4 How does your principal share with you what s/he observes?
1.5 How many observations do you have in your formal evaluation year?
1.6 What is the duration o f each observation?
1.7 In what ways are your contractual guidelines from the collective bargaining
agreement integrated into your evaluation process?
1.8 What modifications would you recommend fo r the teacher evaluation process that
would improve your practice?
Research Question 2: In what ways does the teacher evaluation process impact
teacher behavior?
2.1 How does the evaluation process make a difference in your teaching practices?
2.2 How does the recommendation affect your teaching practice?
2.3 What modifications would you recommend for the teacher evaluation process that
would improve your teaching practice?
Research Question 3: What factors exist at the school that positively or negatively
impact the teacher practice? Why is this school successful?
3.1 To what extent does your school culture impact your teaching practice?
3.2 To what extent does professional development impact your teaching practice ?
3.3 To what extent does collaboration and collegiality impact your teaching practice?
3.4 To what extent does leadership impact your teaching practices?
3.5 What school programs and/or strategies been implemented to improve teacher
practice?
3.6 How are these factors connected to the teacher evaluation?
Research Question 4: In what ways does the district or site leadership influence the
teacher evaluation process?
4. How does the district office leadership contribute to improve teacher practices?
4.2 In what ways does the school’ s leadership contribute to improved teacher
practices?
4.3 Who do you view as school leaders and how do they impact teacher practice?
Figure 5. Certificated staff interview questions.
68
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Principal:
How is it decided who you will evaluate each year?
Have you received any training on how to observe/evaluate teachers?
Do you feel you have the necessary knowledge to properly evaluate teachers? What would
you like more training on?
How do you follow-up to see if your suggestions to teachers are acted upon?
Are you satisfied that your school’ s evaluation system accurately identifies teachers’
abilities, from poor to excellent? What happens to teachers who are identified as weak,
average, or excellent?
Do you attribute your school’ s recent success in any way to your evaluation practices?
What changes would you make?
District Office Personnel:
What types o f training does the district provide its school administrators in the area o f
teacher evaluation?
Do you feel that the site administrators do a good jo b evaluating teachers? Why/why not?
Is the number o f teachers who have received an unsatisfactory evaluation over the last few
years an accurate reflection o f the number ofpoor teachers in the district?
Are you satisfied that the district’ s evaluation system accurately identifies teachers ’
abilities, from poor to excellent? What happens to teachers who are identified as weak,
average, or excellent?
Do you attribute this school’ s recent success in any way to your district’ s evaluation
practices?
What changes would you make?
Figure 6. School and district administrator interview questions.
69
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Development o f the Observation Protocol
The observation tool (appendix F) was used as a means to gather data for
the case study at each school site. The development of the observation tool was a
team effort by four researchers who constituted a subgroup of the thematic disserta
tion group of 15 doctoral students studying teacher evaluation. Each one of the
four researchers examined one aspect of the four frames of leadership developed by
Bolman and Deal (1997). The premise was to write questions that fit the lens of
each frame to get a better understanding of the school, its ambiance, its teacher
evaluation process, and its practices that led to student achievement. Researchers
wrote questions based on Bolman and Deal’s (2003) political, structural, symbolic,
and human resource four-frame model (Figure 7). Once questions were written, the
four-member team met to combine the questions into a one-page observation tool
consisting of 25 questions. All questions were designed to receive yes' or no
answers on the observation tool. The team created a numbering and notation
system for each question concerning its order and leadership frame reference. In
the final survey the entire cohort of 15 doctoral students modified the observation
tool so that 4 questions focused on the political frame, 12 questions focused on the
structural frame, 6 questions focused on the symbolic frame, and 3 questions
focused on the human resource frame.
The final observation survey was brief, focused and encompassed all
aspects of school culture, climate, leadership, and practices. The observation tool
enabled each case study researcher to create a body of data with a similar frame of
reference, which added to the generalizability of the 15 case studies.
70
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
SYMBOLIC FRAME
Issues o f meaning and
faith
Ritual, ceremony, story,
play, and culture as heart
o f organizational life
POLITICAL FRAME
Competitive Arenas
Competing interests
Struggles for power
and advantage
HUMAN
RESOURCE FRAME
Understanding of
people (strengths
and foibles)
Reason and emotion
Desires and fears
STRUCTURAL
FRAME
Architecture of the
organization
Design o f units
Rules and roles
Goals and policies
Figure 7. Overview lens of the Bolman and Deal four-frame model. Source:
Refraining Organizations: Artistry, Choice, and Leadership (3rd ed.), by L.
Bolman & T. Deal, 2003, San Francisco: Jossey-Bass.
Data Collection
Four methods of data collection were utilized to obtain the required
information to triangulate the data. The Institutional Review Board (IRB) per
mission to conduct the study was received prior to data collection to protect the
rights of human participants.
71
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The principal of Catson Elementary School was contacted by the researcher
in June 2005 and invited to participate in this qualitative study. The Assistant
Superintendent of Human Resources for RHUSD, in agreement with the Superin
tendent and the district’s legal department, gave written permission for the case
study to take place at the school (appendix G). The IRB approval was granted for
the case study in November 2005. The data were collected during December 2005
and January 2006 in five stages. At all times, documents and tape recordings were
kept secure, and confidentiality was maintained.
The principal, Ms. Jeanette Cueva (pseudonym), was the link between the
district and the school in implementing the design of this study. Consequently, the
principal was the first participant who provided information in the first stage of
data collection. Staff members representing both primary and upper grades were
invited to participate in the study. Although participation by staff members was
voluntary, the principal assisted in reassurance to those staff chosen to participate
in the formal interviews that this research was a confidential process. The principal
helped to solicit participation in the completion of the surveys. The participants in
the study included the principal, leadership team members, and classroom teachers.
The criteria for selection of certificated staff included permanent status and at least
3 years of teaching at the site (all certificated staff at Catson met the criteria).
Participants were informed of their rights as research subjects, reminded that their
participation was voluntary, and reassured of confidentiality and anonymity
(appendix H).
During the second stage the researcher met with the staff to answer
questions with regard to the confidentiality and the purpose of the research. The
teachers who volunteered to participate in the survey or interview placed their
72
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
responses in separate envelopes, using a prestamped enveloped addressed envelope
to the researcher or the sealed box in the staff lounge marked USC Research.
The third stage of the data collection began with observations of classrooms
and of the campus in general. The observation tool (appendix F) was used to guide
the researcher in data collection. Observations occurred over a 6-day span before
school, after school, and during the instructional school day to give the researcher
an opportunity to view the entire school community and partake in the culture and
climate of the campus.
The fourth stage consisted of the actual teacher interviews, which were held
either before school or after school over a period of 2 weeks. The researcher used
an audiotape machine to record the interviews (again, assuring confidentiality).
Each of the 10 interviews lasted approximately 40 minutes. Each interview was
completed individually with the researcher. The researcher also interviewed the
principal at Catson Elementary School using the Principal Interview Protocol
(appendix E).
The fifth stage of data collection occurred at the district office. The
Assistant Superintendent of Human Resources of RHUSD was interviewed at the
district office. The interview, approximately 50 minutes was also tape recorded.
The District Office Interview Protocol (appendix E) was also used to interview the
assistant superintendent.
Qualitative Data Analysis
The qualitative research model, which has emerged from research efforts in
the social sciences, was particularly appropriate for this study because it was
especially effective in understanding the total environment in which teacher
73
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
training and instruction takes place (Creswell, 1998). Four research questions acted
as a guide for the researcher for data collection and analysis. The researcher
collected data from multiple sources to determine how the teacher evaluation
process affected teacher performance in creating classrooms of effective student
achievement.
A protocol used by the researcher to record demographic, descriptive, and
reflective information according to the four research questions was based on
interviews at the selected school site, observations, reviews of artifacts related to
the site and the site’s process for evaluation, a survey, and an analysis of district
documents and relative data. All of these data were collected in an attempt to
triangulate and produce a strong qualitative case study (Table 8).
The instrumentation organization chart was developed by a subgroup of
researchers in the thematic dissertation group of doctoral students to guide the
researcher in the analysis of the data. The data collected by each researcher
reflected that researcher’s uniquely selected case study site.
Survey
In this mixed-method study a voluntary survey was used for data collection.
The intent was to generalize from a sample of staff to the entire school population.
In addition, the survey was used to drive additional questions for the interview,
with an opportunity for the researcher to ask follow-up questions regarding
information that was not collected by responses to the survey questions or open-
ended questions. Results of the survey were coded and the responses were grouped
according to the four research questions, with the findings presented in tables.
74
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced w ith permission o f th e copyright owner. Further reproduction prohibited without permission.
Table 8
Instrumentation Organization
Research
questions
Teacher
survey
questions
Teacher
interview
questions
Document
review
Observational
tool
questions
District
interview
questions
Administrator
interview
questions
1 How is the teacher
evaluation practiced
at the school site?
3,4, 6, 7,
10, 11
1.1, 1.2,
1.3, 1.4,
1.5, 1.6,
1.7
Stull Bill,
Contract evalua
tion forms,
District memos,
Board policy
6, 10, 17,19,
21,22, 23,
24, 25
2,6 1,3, 4, 7
2 How does the teacher
evaluation process impact
teacher behavior?
5, 13, 14,
W1,W2,
W3
2.1, 2.2,
2.3
Contract peer
assistance review
4, 6, 8, 10
11,20,21
22, 23, 24,
25
5 4
3 What factors exist at the
school that positively or
negatively impact the
teacher evaluation?
15, 16, 17,
18, 20,21,
W l, W3,
W4, W5
3.1, 3.2,
3.3, 3.4,
3.5, 3.6
School plan
Professional
development plan,
Beginning
Teacher Support
and Assessment
1,2, 3, 4, 5, 7,
9, 10, 11, 12,
13, 14, 15, 16,
School culture
and climate
4 How does the district/
site leadership influence
the teacher evaluation
process?
1,2, 5, 6,
8, 9, 12,19,
22, 24, 25,
W2, W3
4.1 8, 11, 16, 18 1 3
Note. W = written response to questions.
Interviews
Using interviews in this qualitative design is naturalistic to the extent that
the research took place in a real-world setting. The researcher conducted face-to-
face interviews that were semistructured, allowing for open-ended responses, and
were repeated multiple times with different staff members. The objective of these
interviews was that the phenomenon of interest unfolded naturally, and the teachers
were interviewed with open-ended questions in a familiar setting under conditions
that were comfortable and amicable to them. The intent was to get high-quality
data from the interview participants in a comfortable context where they could
consider their own views and opinions. It was important throughout the interview
to let the participants speak for themselves.
The responses to the interview questions were coded. They were analyzed
for patterns that would provide an overview of the phenomenon of the study and its
impacts. The coding was assisted through the use of NVivo® software for con
sistency in the data analysis. By using this software for coding, the researcher
revealed a thematic overview of the phenomenon of the study. Coded patterns
were organized in such a way that the overall themes were clear. The semistruc
tured interview of approximately 10 questions plus follow-up questions in a
standard open-ended interview format that best fit this qualitative paradigm was
conducted by the researcher with the site principal, the assistant superintendent of
human resource, and 10 teachers. The purpose for conducting this interview with
the site administrator and assistant superintendent stemmed from the researcher’s
intent to understand the perspective of the evaluator as well as the viewpoint of the
teacher who was the evaluatee.
76
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Documents
During the process of data collection the researcher collected documents
(see Appendix A). Some of these documents were public documents, such as
district written policy of the evaluation process, the collective bargaining agree
ment, and current formal evaluation and informal observation documents. In
addition, private documents were utilized, such as e-mails, letters, principal’s
“walk-through” notes, and formal observation calendar. A subgroup of the
dissertation cohort developed a protocol for document review. According to
Creswell (2003), there are distinct advantages and disadvantages in document
analysis. The advantages are that the researcher is able to obtain the language and
words of the participants. The researcher analyzed the documents at her conveni
ence. Using documents saved the time and expense of transcribing. One of the
disadvantages of this process is that some documents were protected from both
public or private access. The researcher searched for information in hard-to-find
places. Materials were incomplete, and some documents were possibly not
authentic or accurate. The researcher used the document review protocol (appendix
I) to respond to the following guiding questions:
1. Does the form meet the Stull Bill criteria?
2. Does the formal evaluation incorporate the Professional Standards for
the Teaching Profession? Does it include content standards?
3. Does the district have a “how to” manual on the formal evaluation
process? If so, does it include a rubric for teacher performance? Is it presented in
professional training meetings only by the district, or is it presented with the
association present?
77
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4. Does the certificated collective bargaining agreement provide explicit
timelines for the formal evaluation process and options for use of AB 954 (the
Goldberg Bill), and does it define unsatisfactory performance?
5. Does the formal evaluation process provide the administrator and the
teacher the option to select at least one goal of their choice?
6. Does the formal evaluation process incorporate a component to provide
for coaching, professional growth opportunities, and/or next steps?
7. Does the evaluator’s observation template include a Likert-type scale or
some form of rubric to rate the evaluatee’s performance?
8. Is there a pre and/or post process used with regard to classroom
observations as a described part of the formal evaluation process?
9. Is the formal evaluation document designed to allow the evaluator to
individualize comments and address particular needs or recommendation?
10. Is the formal evaluation process primarily summative, primarily
formative, or some combination of summative and formative?
Observation
The researcher spent several hours in selected teachers’ classrooms and the
school site as a whole for collection of observation data. This occurred with sug
gestion and guidance from the site principal, along with permission and voluntary
participation by the teachers. During observations, the researcher took field notes
on the behavior and activities of the staff member being observed. The researcher
followed a semistructured protocol (appendix F).
Through data collection both the observations and the documents were
analyzed, using two sets of tools. Table 9 presents observations that were analyzed
78
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 9
Research Questions Triangulated With Broad Framework and Bolman/Deal
Frames
Research question
Broad
framework Bolman/Deal frames
1. How is the teacher evaluation
practiced at the school site?
1,3, 4,5 Human Resource,
Structural
2. How does the teacher evaluation
process impact teacher behavior?
2,3,5 Political, Human Resource
3. What factors exist at the school
that positively or negatively impact
the teacher practice?
1,2, 3, 4,5 Political, Human Resource,
Structural, Symbolic
4. How does district/site leadership
influence the teacher evaluation
process?
2,5 Political, Human Resource,
Structural, Symbolic
using the protocol developed for the study based on Bolman and Deal’s four frame
model (see Figure 7) of structural, political, human resource, and symbolic frames
to understand the climate and culture of the school. In addition, the researcher used
the rubric created by the National Center for Educational Accountability and the
Broad Foundation known as Best Practices of Fligh-Performing School Systems
(Table 10) to analyze the collected artifacts and the observations as they may have
related to factors identified in this research study. The rubric for analysis presented
in Table 10 was used to compare specific elements related to the school site and to
the district with regard to factors identified by the Broad Foundation research group
regarding characteristics found in effective and high-achieving schools.
79
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced w ith permission o f th e copyright owner. Further reproduction prohibited without permission.
Table 10
Best Practices o f High-Performing School Systems Framework
Organizing themes District practices School practices Classroom practices
1 Curriculum and
Academic Goals
Define clean and specific
academic objectives by
grade and subject
Center school plan on
explicit improvement
of specific academic
objectives
Ensure teaching
content is based
on specified academic
objectives
2 Staff Selection,
Leadership and
Capacity Building
Provide strong instructional
leaders, highly qualified
teachers, and aligned
professional development
Select, develop, and
allocate staff based on
student learning
Collaborate in grade
or subject level teams
focused on student
work
3 Instructional
Programs and
Practices
Arrangements
Provide evidence-based
instructional programs
Ensure use of evidence-
based programs, practices,
and arrangements in
every classroom
Use evidence-based
programs, practices,
and arrangements
4 Monitoring: Com
pilation, Analysis,
and Use of Data
Develop student assessment
and data monitoring systems
to monitor school performance
Monitor teacher
performance and
student learning
Monitor student
learning
5 Recognition,
Intervention, and
Adjustments
Recognize, intervene, or
adjust based on school
performance
Recognize, intervene,
or adjust based on
teacher and student
performance
Recognize, intervene,
or adjust based on
student performance
00
o
The Validity and Reliability of the Findings
Creswell (2003) stated that, although it is limited, qualitative researchers
can use reliability to check for consistent patterns of theme development among the
investigations studied. According to Yin (1989), researchers can also generalize
particular areas of multiple case analysis to other cases. Basically, reliability and
generalizability play only a minor role in qualitative research (Creswell).
While reliability plays a minor role, validity is seen as a major strength of
qualitative research in that “it is used to suggest determining whether the findings
are accurate from the standpoint of the researcher, the participant, or the readers of
an account” (Creswell & Miller, as cited in Creswell, 2003, pp. 195-196). While
reading qualitative research studies, the reader often comes across terms such as
authentic, credibility, trustworthiness, and so forth. Although some researchers
contend that using these terms in analyzing research positions the study as valid
(Creswell & Miller, as cited in Creswell, 2003), others see it as a highly debated
topic (Lincoln & Guba, 2000).
Conclusion
This chapter provided an overview and detailed description of the methodo
logy used in this single-school case study. The chapter introduced the theoretical
base that guided the study as well as the conceptual model. Requirements of a
high-achieving Title I school were included as well as a descriptive analysis of the
case study elementary school. Included in this chapter were a discussion and
overview of the four research questions, the sample and population, data collection
instruments, and the methods of data analysis. Data collection and analysis were
based on a triangulation of survey, interview, observation, and document review
relating to the factors of policy, practice, impact, and influence of teacher
81
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
evaluation affecting teacher practice as it impacts student achievement. Chapter 4
presents the findings and an analysis of the findings in response to the research
questions.
82
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 4
THE FINDINGS
In this chapter the findings of the study are presented and discussed. The
findings are explained in terms of the four research questions that align to the
purpose of this qualitative case study, which was to determine whether the teacher
evaluation process impacted teacher performance in support of student learning in
an urban school district. The findings and procedures used at the selected element
ary school study site were examined within each of the following four research
questions:
1. How is teacher evaluation practiced at the school site?
2. How does the teacher evaluation process impact teacher behavior?
3. What factors exist at the school that positively or negatively impact the
teacher practice?
4. How does the district/site leadership influence the teacher evaluation
process?
Through focused data collection and analyses that were driven by surveys,
interviews, observation, and document review, each research question is analyzed
individually. A discussion involving an analysis and reflection of the research
findings conclude the chapter.
Overview of the Study Site
It was an early January morning at Catson Elementary School. There was a
distinct chill in the air. After a few days of warm weather, it felt unusually cold for
a southern California morning. At 6:45 a.m. all was silent on the Catson School
playground.
83
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The school awakened as the day custodian unlocked the restrooms, the
gates, and the office doors. Quietly observing the campus, the researcher noticed
that the school appeared clean and maintained. Catson Elementary School had
been undergoing modernization, and three wings of classrooms had all been
upgraded.
Soon the researcher heard the bounce of a single ball on the blacktop and
saw one lone figure wearing a gray hooded sweatshirt. In a matter of moments the
sole ball player was joined by a handful of early arrivals. The ball cart was
wheeled out to the playground by the morning playground supervisor. She
methodically hung all of the tetherballs onto the pole hooks. A nearby basketball
game began, and by the aroma in the air, it was obvious to the researcher that a
warm meal was being served to those students arriving hungry and cold for the
early morning school breakfast program. A tiny girl in a lurry pink jacket rushed to
the ball cart to triumphantly grab the last red rubber ball off the cart. Teachers
walked the hallways to class, giving friendly waves to seemingly happy children.
Laughing and shouts of “hello” and “come play” could be heard from the handball
court. A single teacher with arms filled with “stuff’ received help from a peer
carrying the load to a nearby classroom.
The early morning quiet had disappeared. The pulse of the school had
quickened, the day at Catson began long before the ringing of the first morning
bell.
Catson Elementary School was specifically selected as the research site for
this qualitative case study. A case study is one in which the researcher “explores in
depth a program, an event, an activity, a process, or one or more individuals”
(Creswell, 2003, p. 15). As stated by Creswell, the case is “bounded by time and
84
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
activity” (2003, p. 15) and the researcher collects “detailed information using a
variety of data collection procedures over a sustained period of time (Stake, 1995,
as cited by Creswell, 2003, p. 15). Catson was chosen because it was a typical
mid-size urban elementary school in the Los Angeles County area of southern
California that beat the odds of other adjacent schools in the area reaching the
status of a high academic achieving Title I elementary school. The researcher
collected data during December 2005, and January 2006. Site observations
occurred over a 2-week span within that time period. Two visits were made to the
RHUSD office building, and an additional follow-up visit was made to Catson at
the beginning of February 2006.
Findings by Research Question
In a qualitative study, research questions focus on a single phenomenon or
concept (Creswell, 2003). This is a single study of the teacher evaluation process
as it impacts teacher performance in support of student learning at an urban
elementary school. The following four research questions begin with the words
what or how to convey an open and emerging design consistent with qualitative
research (Creswell, 2003, p. 106).
Findings fo r Research Question 1
Research question 1 asked, How is teacher evaluation practiced at the
school site? According to the Catson Elementary School Accountability Report
Card (SARC) published during the 2004-2005 school year, data included are
consistent with State Board of Education guidelines, which are available at the
CDE Web site (CDE, 2004). With regard to teacher evaluation practice, the
85
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
document states that teachers are evaluated at intervals determined by the collective
bargaining contract in the following areas:
Engaging and supporting all students in learning
Creating and maintaining effective environments for student learning
Understanding and organizing subject matter for student learning
Planning instruction and designing learning experiences for all students
Assessing student learning
Developing as a professional educator
Permanent teachers are evaluated every three years, while all others once or
twice a year.
After teacher feedback, copies are provided to the teacher, the evaluator,
and for the teacher’s personnel file. (CDE, SARC, 2004, n.p.)
During individual teacher interviews the researcher asked each teacher, “In
what ways are your contractual guidelines from the collective bargaining agreement
integrated into your evaluation process?” The results revealed that not one teacher
knew the answer, yet the SARC openly stated that teachers were evaluated at inter
vals determined by the collective bargaining contract. Teacher G stated “Sounds
like Greek to me. I don’t really know how to answer that,” while Teacher H
responded, “Not at all.” A couple of teachers said that they could not respond to
that question because they just did not know the answer. Teacher B said, “I’m not
sure. You know, I’m not sure how exactly it states in the contract where it would
be with the observation. I wouldn’t know the wording of it, or anything like that to
see the connection.”
Prior to examining findings related to teacher evaluation practice, a review
of district policy in regard to certificated evaluation of teachers was deemed
necessary. Therefore, by following the researcher-designed document review
86
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
protocol (appendix I), in addition to the S ARC, the researcher examined a medley
of policy and practice documents relating to the evaluation process for certificated
teachers. According to the document review protocol’s guiding questions, Rio
Hondo has very comprehensive evaluation policies and procedures that follow the
tenets of the Stull Bill and the California Education Code.
First, the researcher carefully reviewed district policy by examining the
Handbook for the Evaluation o f Certificated, Classified and Administrators for the
RHUSD, as presented by the Division of Human Resources. Within this detailed
notebook are all of the policies and documents used in the district with regard to the
evaluation process. In the section entitled “Regulations & Policies Governing the
Evaluation of Certificated Employees” California Education Code sections are
presented and defined (see appendix B for complete definitions), including (a) Edu
cation Code 44660: Article 11. Evaluation and Assessment of Performance of
Certificated Employees, (b) Education Code 44661: Guidelines and Procedures,
and (c) Education Code 44662: California Peer Assistance and Review Program
for Teachers (California Education Code, 2005).
The Certificated Contract language of the RHUSD Article IX, “Procedures
to be Utilized in the Evaluation of Teachers,” contains the following comprehens
ive and fully explained sections per district policy and procedure:
A. Purpose of the Evaluation Process
B. Frequency of Evaluation
C. Evaluation Procedure
D. Stull Objectives
E. Development of Objectives and Standards
F. Evaluation of One Adjunct Duty Required
87
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
G. Use of Public Charges in Evaluation
H. Serious Situations Between Evaluations
I. Rebutting Derogatory Information
J. Evaluation Conference
K. Regular Evaluation Track Procedures
L. Alternative Evaluation Track Procedures
M. Use of Public Charges in Evaluation
N. Serious Situations Between Evaluations
0 . Rebutting Derogatory Information
P. Evaluation Conference
Q .
Filing of Evaluations
R. Procedural Calendar for Evaluations
S. Non-Tenured Teachers
T. Permanent Teachers
U. Adult Education Teachers and
V. Participation in Peer Assistance Review Program (RHUSD
Evaluation Handbook)
The second section of the handbook, “Certificated Observation and
Evaluation Guidelines,” contains an NCLB compliance policy as follows:
When observing elementary Self-Contained or Middle School Multiple
subject Core teachers the evaluating administrator should ensure that the
formal evaluation reflects a summary of observations that cover all of the
core subjects (Social Studies, Math, Science, Language Arts). Observations
and the resulting formal evaluation shall be within the context of the K-12
State Content Standards, the California Standards of the Teaching Pro
fession, established teaching practices, District standards and District
curriculum guidelines. (RHUSD HR Handbook)
RHUSD has included district guidelines and an evaluation timeline, which
are both included in this section as a matter of district policy. Of special interest to
the researcher were some policies and procedures found in RHUSD that were
unique to the district but still followed California’s Stull guidelines.
88
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
In Section B, “Frequency of Evaluation,” permanent teachers (and all
nonteaching certificated personnel) with satisfactory evaluations are evaluated once
every 3 years.
Permanent teachers and all other non-teaching personnel, shall be evaluated
at least once every three years, unless they are working under the provisions
of a Performance Action Plan (PAR) in which case the permanent teacher
shall be evaluated every year until the level of performance has improved to
the extent they are no longer on a Performance Action Plan. (RHUSD HR
Handbook)
These same staff members follow Section K, “Regular Evaluation Track Pro
cedures” (appendix H). If a permanent teacher (or certificated nonteaching person
nel) has completed at least two sequential evaluations by the regular procedure in
RHUSD, he or she may opt to pursue an alternative evaluation known as the Pro
fessional Options Plan, commonly called POP (located in Section K, “Alternative
Evaluation Track Procedures,” seen appendix H). RHUSD policy states that, in
order for a teacher to POP, the teacher must abide by the following:
1. After a permanent teacher or non-teaching personnel, who has completed
two or more successful, and sequential performance evaluations (where all
six categories of the evaluation instrument were rated “Meets Expectation,”)
the individual evaluatee may elect, with the approval of the site principal, to
pursue an alternative evaluation track, called the Professional Options Plan
(POP).
2. The POP is a process that is employee centered, encourages peer sharing
and support and includes voluntary self-assessment and goal setting. The
rating administrator and teacher share the joint responsibility for developing
the POP. Any projects must align with District goals, standards, and
objectives. (RHUSD HR Handbook)
Dr. HR (pseudonym), the Assistant Superintendent of Human Resources of
RHUSD, has held his position in the district since 1992. He was hired as the
Assistant Superintendent of Human Resources from outside the district, and he had
some personal visions that he has diligently strived to fulfill. In discussing the
89
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
practice of teacher evaluation in the district, he stated the following comments with
regard to POP:
It took me some time to work with the association, that is the teachers
association, to accept that. They were very traditional and they had some
thing in place and they didn’t want to muck with it. They didn’t want to
change it in any way. So it took some time. But it was important, I believe,
that we had some options and , again, people are at different levels at
different stages in their careers. They can bring to the table different kinds
of things. And if you can harness this, you know, to help the school. . .
because I remember when I was principal, I would bring ideas, and test
them with my staff. And they really enjoyed that. And so I had the flexi
bility of doing certain things. As a result of that, people were being
measured in different ways. And for me, that was very workable because,
again, people are at different stages, different places in their careers and I
found that we, as a result of that, brought some wonderful programs and
projects (what the teachers had to do in order to POP) and ideas to our
programs.
The researcher found a fine line between “intent” of the policy and
“practice” of the policy. According to Dr. HR,
There is a tremendous effort on the part of the district in expenditure to
provide the tools that people need to be more effective in the classroom.
And we’re starting with our principals, with our administrators. Principals
and Assistant Principals, teaching them the tools (in regard to evaluation)
and they then turn around and are teaching the teachers the tools. As a
result of that, we again believe that we are making an impact. But some
times we see some things that don’t connect, you know, they just don’t
really make sense. And we wonder sometimes what has happened here.
Although the researcher received only a 66% survey response (19 of 29
teachers), the respondents were fairly cohesive in their agreement to most survey
items. Since the survey was given to the teachers at the onset of the study, the
researcher hoped that the survey responses would drive additional questions to ask
during the individual interview process. The goal was that all pertinent data would
be collected. Referring to Table 4, the data reflect an experienced teaching staff at
Catson, with most teachers having taught at this particular school site for more than
7 years. Since the first research question was in regard to how the evaluation is
90
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
practiced at the school, the data responses were surprisingly mixed with regard to
the administrator’s involvement in the Teacher Formal Evaluation Process as
mandated by California Education Code.
The teacher survey contained 25 questions that required a forced response,
using a 4-point Likert-type scale. Results were expressed as the mean of teacher
responses made on the 4-point scale, where 0 = disagree strongly, 1 = disagree, 2 =
agree, and 3 = agree strongly, hi addition to the mean, the researcher was
interested in the percentages of response by teachers who agreed or agreed
strongly. Both of these figures are stated in the tables.
Survey questions 3, 4, 6, 7, 10 and 11 (Tables 11 and 12) asked teachers to
respond to the “practice” of the teacher evaluation at the school site. Table 4
illustrates that 93% of the Catson Elementary teachers had taught 7 or more years
at Catson. The principal had been at the site for 10 years. The survey results shown
Tables 11 and Table 12 suggest very mixed responses.
The results suggest that the principal does not frequently observe class
rooms, yet 95% of the respondents stated that the principal frequently observes
classrooms for the purpose of evaluation. Although 84% of the teachers agreed
that they were aware of the goals and objectives of the school, only 42% stated that
they discussed with the principal instructional strategies that they use in their
classrooms. Finally, 89% of the teachers, addressing teacher evaluation practices at
the school site, said that they receive timely and meaningful feedback regarding the
teaching observations.
During her interview, Ms. Cueva (the principal) expressed very strong
feelings with regard to the teacher evaluation practice. “I get a list of who is in
what I call the evaluation cycle, and then it is up to me to decide with permanent
91
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 11
Survey Findings fo r Research Question 1, How Is Teacher Evaluation Practiced at
the School Site? Administration’ s Involvement in the Teacher Formal Evaluation
Process as Mandated by California Education Code (N = 19)
Item Mean % Agreea
3. My administrator frequently observes my 2.3 95
classroom for the purpose of evaluation.
4. My administrator and I often discuss the 1.2 42
instructional strategies I use in my classroom.
6. When my administrator visits my classroom, he/she looks 1.6 68
for things which we agreed upon at pre-conference.
7. I receive timely and meaningful feedback 2.1 89
regarding my teaching observations.
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 = agree, 3 = agree strongly). Number of
teachers =19 out of 29 (66%) who responded.
aScored 2 or 3.
teachers if they’re going to be (traditionally) evaluated or if they’re going to follow
what’s called a Professional Options Plan.” Elaborating on the POP, Ms. Cueva
stated, “They [the teachers] have to do a self-study, and then pose a question that
they want to investigate and come up with an end product that they would share.”
In describing the evaluation process at Catson Elementary School, teachers
seemed aware of the practice. Teacher H stated,
I’m just aware that I’m on my toes to make sure that I know that I’m cover
ing everything that I know I should cover, and then after she’s come in to
make her observation, then there’s a meeting to go over the lesson I’ve pre
sented. And from there, she gives me pointers where she thinks I could
have made some improvement and she also gives her observation and
comment about things she really liked and noting different students as the
way they behaved in the classroom as far as responding to questions.
92
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 12
Survey Findings for Research Question 1, How Is Teacher Evaluation Practiced at
the School Site? Administration’ s Involvement in the Supervision Process Through
Observation, Data Collection, Feedback, Goal Setting, and Improvement Strategies
(N= 19)
Item Mean % Agree3
10. The administration frequently observes my classroom. 1.3 37
11. Other instructional support staff (e.g. coaches, counselors,
BTSA/PAR support providers, etc.) frequently observe
my classroom.
1.5 53
2 0 .1 am aware of the goals and objectives of this school. 2.3 84
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 = agree, 3 = agree strongly). Number of
teachers = 19 out of 29 (66%) who responded.
aS cored 2 or 3.
Teacher F stated that, when the principal comes into the classroom,
She will interact so she doesn’t just sit there and watch us. She’ll say,
perhaps you can do this, or she’ll add to it or she’ll say, why don’t you try
this technique. So I really like that. She doesn’t just walk in and write
notes, she actually interacts with us at the time, saying did you think about
doing it this way? Or, you know, let me try, or she’ll even take over with
the kids so she’s done that in the past. So, my first two years she was in six
times during the year, and now it’s once a year and every other. So we do
POPS, we do self-evaluation and she does .... I think next year I’m up for
an evaluation again.
The principal interacting with the staff and students was very observable,
including during evaluation. Teachers seemed to be comfortable with this practice.
Regarding her training, Ms. Cueva stated that there had been only limited training
of principals on evaluation, “but not any, what I consider quality professional
development.” On the other hand, the principal said,
93
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
I think I have a good eye for teaching. I can watch a lesson and kind of
analyze it and say, well wait, that’s not working. Why isn’t that working? I
have a good eye for if kids are picking up concepts. I think what is really
helping now, which is interesting is a piece of software. It came from
ACS A called the walk-in. It’s so wonderful because it matches the
California Standards for the Teaching Profession. So if you focus in on,
let’s say we’re going to look at creating an effective classroom environment
and that’s all I’m going to watch during the observation, then that software
really helps me because it poses questions. And I’m like, oh, I never
thought of asking that. So that’s very interesting.
When asked how involved they were in developing their evaluation for the
school year, teachers’ responses ranged from “I’m not” to “Pretty involved.”
Teacher E stated,
I would say I was pretty involved because we actually meet and talk about
the Stull Bill, and certainly I have input into the Stull Bill. Jeanette
[principal] is always open to new ideas or maybe an innovative way of
doing something. She is very open and very . . . you don’t feel intimidated,
I guess that would be the word, at all.
The interview results suggested that teachers were generally aware of
evaluation documentation and formal observation schedules. When asked how
they knew what the principal was looking for during classroom observation,
Teacher B responded, “I really don’t look at it that way. I don’t think she ever
comes to us with a hidden agenda.” She elaborated, “I don’t think that she’s going
to try to trip us up or trick us. I am going to be me, and I’m going to do what I’m
going to do, and I don’t worry about it.”
Most of the Catson teachers stated that there was usually a defined standard,
a selected curricular area, or a lesson plan form to fill out the day before the
observation. Teacher A stated, with regard to what the principal was looking for
during classroom observation, “One of the things that we do is submit a lesson plan
ahead of time so that she has an idea as to our lesson, what it consists of (then) she
has a pretty good idea as to what she is looking and hearing for when we have any
discussion.” She continued, “And we have a follow-up also to have that discussion
94
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
for feed-back.” The interviews revealed that 100% of the teachers received one-to-
one feedback conferences from the principal following a formal observation in their
evaluation practice at Catson.
The researcher used the 2 5-item Observation Tool (appendix F) as a check
off sheet (yes = observed, no = not observed) for responses that were aligned with
Bolman and Deal’s (1997) four frame model. Table 13 indicates those items that
were observed by the researcher, each significant frame relating to research
question 1, and the practice of evaluation at Catson Elementary School. The data
revealed that the structural frame is most prevalent in response to the practice of the
teacher evaluation process at Catson Elementary School.
Summary o f Findings fo r Research Question 1
Findings from district office administration contend that Catson Elementary
School tends to display a high level of agreement between the intention and the
practice of their policies. Many times, the basis for that kind of belief may simply
be a “feeling” that is held by the policymakers (the school board and/or the district
office) or the administrators that people would be doing what is expected of them
(Tunison, 1998). The intent of the district office in RHUSD is that all policies and
procedures are practiced exactly as written. The Assistant Superintendent helped to
develop the POP as an alternative to experienced teachers. This is advantageous to
teachers who have been through a satisfactory evaluation process on a number of
occasions and are now provided an opportunity for a learning challenge. As the
Assistant Superintendent, Dr. HR concurred, the POP projects are good for the
teacher, good for the school, and good for the district. Through triangulation of the
data that included survey responses, researcher observation, and staff interviews,
95
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 13
Observation Tool Findings Related to Research Question 1, How Is the Teacher
Evaluation Process Practiced at the School Site?
Frame Question Yes No
10 SY There is positive interaction between staff and
administration. V
14 H Teachers are engaged in school activities y l
17 P Students remain on task when administrators enter
the classroom
V
19 S Content standards for the lesson are visible V
20 S The agenda/objectives are visible to the students V
21 s Student work is displayed in the classroom. V
22 s Guided practice was observed during the lesson V
23 s A variety of learning activities (direct instruction,
lecture, group work, projects, etc.) are utilized. V
24 s High levels of questioning are evident in the class V
25 s Assessments demonstrate multiple measures to
evaluate student work.
V
Note. H = Human Resources, P = Political, S = Structural, SY = Symbolic.
this researcher found that the POP process was innovative and successful as an
alternative to the traditional evaluation practice for experienced teachers in
RHUSD.
According to site administrator interview results, there appears to be a
disconnect between intent and practice of the teacher evaluation policies involved
in the training provided to school administrators. Dr. HR stated that, under the
96
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
California Education Code, “I believe its 44662, we have to do annual training. We
have to actually certify that principals are capable to evaluate. So we do annual
training plus individual training for our administrators.” On the other hand, the
principal at Catson, Ms. Cueva reported, “The training we tend to have gotten has
been more on the progressive discipline FRISK kind of model, but not any, what I
consider, quality professional development on the evaluation process.” The
principal stated that more specific training on the practice of evaluation based on
the CSTP was an avenue for next steps in providing a practice of good evaluation.
I think we all should have decided how to implement the California stand
ards for the teaching profession. Because, are we going to just do an over
view of all the standards? Because there’s just no way. I mean, that in one
or two or three observations you can go in-depth. You can get a picture, a
broad picture of teaching practice in the frame of the standards for the
teaching profession. But it would have been nice to say, you know, let’s
focus this observation for everybody on Standard One. And then talk about
it. It’s just a different implementation, I guess.
Findings fo r Research Question 2
Research question 2 asked, How does the teacher evaluation process impact
teacher behavior? Based on a discussion of following teacher evaluation policies
by intent, or by practice, the researcher found aspects of teacher evaluation that
revealed that the process itself had an impact on teacher behavior. The second
research question further investigates the evaluation process by seeking a response
as to how the teacher evaluation process impacts teacher behavior.
Beginning with the survey process, when teachers responded to the state
ment, “I view my administrator’s implementation of the teacher evaluation policies
as an integral part of my professional growth,” only about half of the respondents
agreed with that statement (Table 14). On the other hand, the teachers stated that
the administrator’s supervision of instruction and additional support and feedback
97
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 14
Survey Findings fo r Research Question 2, How Does the Teacher Evaluation
Process Impact Teacher Behavior? Administration’ s Involvement in the Teacher
Formal Evaluation Process as Mandated by California Education Code (N= 19)
Item Mean % Agree3
5. I view my administrator’s implementation of the teacher 1.3 52
evaluation policies as an integral part of my professional
growth.
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 = agree, 3 = agree strongly). Number of
teachers =19 out of 29 (66%) who responded.
aS cored 2 or 3.
(survey questions 13 and 14) improved instructional practice. Both of these areas
had an agreement level above 70% (see Table 15).
After responding to the 25 forced-response questions on the survey,
participating teachers responded to five open-ended survey questions. Three of the
five questions (W l, W2, and W3, where W = open-ended written response)
revealed answers that contributed to the findings with regard to the question of how
the teacher evaluation process impacts teacher behavior.
When Catson teachers were asked to describe which experiences and/or
activities had the greatest impact on their teaching practice and professional
development, the majority of the responses could be categorized into the three
areas: staff development opportunities, collaboration, and support from colleagues
(Table 16). When teachers were asked to describe which experience and/or
activities have the greatest impact on student achievement at Catson, responses
98
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 15
Survey Findings for Research Question 2, How Does the Teacher Evaluation Pro
cess Impact Teacher Behavior? Administration’ s Involvement in the Supervision
Process Through Observation, Data Collection, Feedback, Goal Setting, and
Improvement Strategies (N= 19)
Item Mean % Agreea
13.1 believe that my administration’s supervision of
instruction improves my instructional practice.
1. 7 79
14.1 believe that other instructional support staff's
feedback regarding my instruction improves my
instructional practice.
1.7 74
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 = agree, 3 = agree strongly). Number of
teachers =19 out of 29 (66%) who responded.
aS cored 2 or 3.
varied. Of the open-ended results most often stated, themes were centered around
grade level and whole school collaboration, school-wide cohesiveness, high
expectations for students, staff development, and cohesiveness of school-wide
programs (such at Step Up to Writing, Mountain Math, and Mountain Language).
Throughout the responses to the third open-ended item (W3), interactions
with the principal was cited most often. Therefore, by examining the open-ended
survey responses to question W3, leadership was the abounding theme (with 11
specific individual comments on the leadership of the school) that positively
impacted the teaching practices at Catson Elementary School.
In order to validate these responses, the researcher spent time observing in
classrooms (where the teachers had given permission for a 15-minute observation),
as well as observing the entire school on a class-by-class walkthrough with the
99
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
principal. As reflected in Table 16, there was a positive and dynamic interaction
between the staff and the administration (principal). In fact, as observed by the
researcher, Ms. Cueva was revered by the entire school community. There was
definitely a positive relationship between the observation, the survey results, and
the interview responses with regard to the relationship and impact that Ms. Cueva
has on her teaching staff. Teachers stated that the site administrator was an
accessible leader who gave positive feedback and support. They said, “She is
extremely helpful and trustworthy.” Although Ms. Cueva was an icon of sorts,
there was distributed leadership among the teachers at Catson, with strong grade-
level leaders and an active Instructional Leadership Team (ILT). Teachers stated
that they were trusted to make well-informed decisions about teaching methods and
practices. They also worked toward consensus rather than feeling coerced into
doing something. Teachers reported that they felt empowered to try new ideas
without negative consequences. Nevertheless, the principal was the supervisory
force that influenced teacher behavior in the evaluation process at Catson
Elementary School.
As suggested by the findings in Table 17, using Bolman and Deal’s (1997)
four frame model, the majority of observation findings from the 25 items on the
Observation Tool (appendix F) that related to research question 2 could be assigned
to the structural frame. In fact, most of the items that were in the structural frame
were those things that visibly impacted teacher behavior. Some structurally labeled
findings identified on the observational tool were displayed student work in the
classroom, visible content standards for the lesson in the classroom, and assess
ments used to demonstrate multiple measures in evaluation of student work.
100
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 16
Findings for Open-Ended Survey Items Related to Research Question 2, How Does
the Teacher Evaluation Process Impact Teacher Behavior? (N= 19)
Question Responses
1. Describe which
experiences
and/or activities
have the
greatest impact
on your
teaching
practice and
professional
development.
The discussions that we have during our team meetings have
the greatest impact on my teaching.
Staff development at the school site.
Staff development directed by the administrator and staff
development where teachers share what works best in the
classroom.
An administrator who thinks like a teacher.
Having the freedom to try new things not just stick to district
pacing schedules.
Tremendous support that we receive from our site
administrator while still holding high expectations.
Having time to meet with other teachers and discussing our
practices help in my professional development as well as our
bi-weekly inservices (staff development with the principal).
Grade level team planning. (5)
Grade level collaboration for year long planning.
Attending professional development conferences.
Regular planned staff meetings used to meet school goals.(6)
Sharing with colleagues.
Time to reflect.
Freedom to experiment with different approaches.
New ideas presented, but freedom to accept, modify, or reject
them.
Collaboration with other professionals regarding strategies to
support student needs.(3)
District staff development.
101
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 16 (continued)
Question Responses
2. Describe which
experiences
achievement at
this school.
and/or activities
have the
greatest impact
on student
I believe the decisions that we make during team meetings. (4)
The on-site professional development that we receive.
Faces to Data.
Staff Development trainings. (2)
Small group differentiated instruction.(2)
Attitudes where all (students, teachers, support staff, parents) at
the school work toward a common goal.
Vocabulary development.
Integration of technology and music.
Having high expectations of students.(2 )
The manner that each grade has similar programs so that
students see consistency and repetition throughout each grade
level. (2)
Strategies for improving teaching practice as well as student
achievement.
Music instruction as a literacy tool.
Our after school tutoring has had a great impact on student
achievement.
Rotating students into other classrooms to support a specific
skill a student might be. lacking has helped students.
Whole school collaboration with grade level teams working
closely together.
Careful analysis of data which then drives instruction.
Sharing.
Staff meetings.
Instruction focused on meeting standards.
Support staff.
Success builds on success (I can do it attitude!).
Personal streamlined attention.
102
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 16 (continued)
Question Responses
3. Describe which
experiences
and/or activities
have the
greatest impact
on your
teaching
practice either
positively or
negatively (+ =
positive; - =
negative
response).
The conferences that I have with my principal after an
observation positively impact my teaching practices.+
Post observation meetings with the principal. +
When I have a problem, I go to my team and the administrator.
I am directed to a solution by the administrator. Either I will
read, or observe or I will meet with someone. +
Feedback and conversation that is non-evaluative is positive.+
Focusing on a few goals each year and developing expertise
within our school.+
Site administrator is accessible at all times, and makes frequent
classroom visits with positive feedback and support.+
Principal wants us to come to consensus instead of feeling
coerced into doing something (that is positive).+
Our biweekly staff development is positive.+
Having a great relationship with our administrator has allowed
us to have discussions without feeling uncomfortable.+
Principal is always supportive of new ideas we may present.+
Our administrator is good at working with grade level teams
and then making that work as a whole school collaboration.+
Our principal is extremely positive, helpful and trustworthy.+
She (the principal) trusts her teachers to make well-informed
decisions about teaching methods and practices.+
Sharing ideas is positive.+
Meetings: SCD, Leadership Team, Grade level team, and
Staff.+
Informal discussions and brainstorming with the principal.+
Freedom to try new ideas without negative consequences +.
Positive is collaboration, negative is not including me in the
interactions.+
Taking the time to individually discuss with a te a c h e r.+
Note. The number in parentheses follow ing a statement indicates respondents with
similar answers. Som e teachers wrote more than one response.
103
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 17
Observation Tool Findings Related to Research Question 2, How Does the Teacher
Evaluation Process Impact Teacher Behavior?
Frame Question Yes No
4 H There is positive interaction between the staff
and administration. V
6 S Staff members are familiar with the vision and can speak
to the goals of the vision. V
8 s Describe the leadership style of the administration:
Yes - Distributed, No = Top-Down. V
10 SY There is positive interaction between the staff and
administration. V
11 SY The administration is visible in all areas of the school. <
14 H Teachers are engaged in school activities. V
17 P Students remain on task when administrators enter
the classroom. V
18 S Substitutes are considered effective instructors. V
19 s Content standards for the lesson are visible. V
20 s The agenda/objectives are visible to the students. V
21 s Student work is displayed in the classroom. V
22 s Guided practice was observed during the lesson. V
23 s A variety of learning activities (direct instruction,
lecture, group work, projects, etc.) are utilized. V
24 s High levels of questioning are evident in the class. V
25 s Assessments demonstrate multiple measures to evaluate
student work. V
Note. H = Human Resources, P = Political, S = Structural, SY = Symbolic.
104
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
According to the interview protocol, responses in sections 2.1, 2.2, and 2.3
allowed teachers to communicate how they felt the teacher evaluation process
impacts teacher behavior. There was a variety of strong responses from teachers to
the question, How does the evaluation process make a difference in your teaching
practices? Teacher H revealed that he is open-minded.
I never see evaluation as a threat, and I am never on the defensive because
since I already look at it like I know what I’m going to do, and I’m not
going to pretend. I’m just going to do my job and if there’s something that
comes up, I’m going to be open to whatever it is that I didn’t know that I
was doing incorrectly to fix it.
Teacher G reflected that the evaluation process makes a difference in her teaching
practice.
It makes me a better teacher. Yeah, I think you know that I don’t always
agree with her (the principal, referring to other interview responses), you
know I may think, that might work in a perfect classroom, but I don’t have a
perfect classroom and I need to do it this way. But always, I am very open
to her suggestions and will consider them. And if I think it’s something that
will work or that I can try, I think that’s really important.
Responding to the same question, Teacher E stated,
Well, I think for one thing, it really makes me sit down and take a look at
standards, perhaps more that I would if I weren’t being evaluated and being
more aware of the standards. I think just to really try to put good teaching
practices . . . maybe that, you know, we get a little like, I don’t want to say
lazy because I don’t think we’re ever lazy, comfortable perhaps. Like just
planning this lesson that I’m planning, it just makes me think, okay, well
this is a good idea and how can I extend this and I can use this for a center
later, and so I just think that the preparation [for this formal observation] it
just makes me a better teacher.
With regard to the evaluation process making a difference in their teaching
practice, a number of teachers said that informal “evaluations” (as the teachers call
them) make more difference than the formal ones. Teacher B remarked,
The formal ones only happen every few years, but the informal ones are the
ones in which she’s [the principal] coming around.. . . We don’t know
when she’s coming, so you’re kind of like on your toes in a sense. But you
also know that you need to do a correct job, and do the right job.
105
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Teacher C affirmed that the evaluation process does make a difference in
her teaching practice.
It make you more cognizant of what you’re doing. And, you know, am I
reaching all levels for the students, all levels of language ability, all levels
of learning styles? Am I calling on an equal number of boys and girls, and
everybody in the room? And so you just become more aware in thinking of
your own teaching style and how you can improve it.
When Catson teachers were asked (interview question 2.2) how a recom
mendation from their principal affected their teaching practice, they responded
positively. Teacher A stated that she tried to implement those changes as best as
possible: “If I ever were to have a problem, I would just go to the principal and
say, how do you think it would be best for me to do it because I don’t think I’m
doing it right.” Teacher F said that she would definitely take the principal’s recom
mendation into consideration: “Usually she has really sound advice, and really
sound recommendations. I respect her as a teacher and as a principal. She’s been
in the classroom and so . . . I’m always open to better my practices with my
students.” Teacher F disclosed,
Well, I just think I take it to heart. Just like she [the principal] pointed out
last time, the lesson was too long. I knew it was, and yet I thought, well
she’s here and I better keep going.. . . So, I just think. It makes me reflect
and just try to put into practice what she [the principal] recommends.
Teacher D realized that, as she was doing the “day-in and day-out” teaching
every day, the principal’s recommendations
kind of give you someone else’s perspective as to how they think you’re
doing, and our principal is pretty good about just sticking to the facts. . . .
You don’t feel like you’re being judged. She [the principal] is really there
to help you become better, and to focus on what you need to develop.
Teacher J remarked,
Recommendations from the principal affect my teaching practice a lot. It’s
another chance for reflection and wondering, how? It makes you think
about what’s best for the children and lets you feel less self-absorbed and
106
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
wondering whether or not you’re a good teacher. You think about what’s
best for the kids.
Teacher I summed it up:
I think we all take her [the principal] words to heart. So, if she tells us that
we kind of need to work on something, we respect her decision, or respect
the way that she looks at our teaching, and then we, you know, I, try to
change the way that I’m doing things.
Teacher responses to interview question 2.3 related to the impact of teacher
behavior with regard to the evaluation practice by asking teachers to indicate what
modifications they would recommend for the teacher evaluation process that would
improve their teaching practice.
In response to interview question 2.3, several teachers were not sure of any
modifications that they could recommend. A few agreed with the following
statement by Teacher E.
You know, I hesitate to say it, nobody likes to be observed, but I suppose
perhaps more frequent observations would be a recommendation. But
again, I hesitate to say that because, of course, we all get a little on edge
when we’re being observed. So, I suppose, more frequent feed-back, if it
were possible.
Novel responses were given by Teacher A and Teacher H. Teacher A said
that, if the lesson did not go well when the principal was observing, the teacher
should have a chance for a “do-over” with the same lesson as the first, but this time
the teacher would incorporate the principal’s written feedback into the lesson.
Teacher H said that the following recommendation for the teacher evaluation
process would improve teaching practices.
I think other principals could shadow each other. Because I’ve heard from
other teachers, and I’ve been observed myself by another person from my
credential program who didn’t observe me, who would sit and write on this
form while I was teaching, and so I think it would benefit other principals,
you know, just, no matter how long they’ve been doing it, just to see
another person and go, oh yeah, hey, that’s a good idea, or I am doing well,
or no matter how long you’ve been either teaching or being a principal, you
can still learn something new.
107
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
With regard to research question 2, contemplating how the teacher evalua
tion process impacts teacher behavior, Dr. HR (Assistant Superintendent of Human
Resources of RHUSD) was asked whether he was satisfied that the district’s evalu
ation system accurately identified teachers’ abilities from poor to excellent. The
researcher also asked what happened to teachers who were identified as weak,
average, or excellent.
One of the things that we did several years ago was to align our evaluation
instrument with the standards. So, they’re now one in the same. So, we do
believe that the instrument itself at least addresses what should be happen
ing in that classroom. And if the principals are adept at finding, well I
mean, looking at what is happening, and they are identifying what is good
about it and what’s bad about it in assisting people, it is an evaluation
system used to assist people not to just talk about. We believe th at. . . we
have seen some growth.
Affirming and examining the documents identifying a policy and procedure
for identified weak teaching, the researcher heard the following from the Assistant
Superintendent.
We have a PAR program. We have a teacher support program and PAR is a
component of that. The evaluation tool itself, or instrument itself, does... if
you receive an unsatisfactory in any of the areas of the standards, you are
then, you know, you go to Par and you get up to 18 months of assistance
through the PAR program to help you improve your deficiencies. In the last
few years here, those who gave gone to PAR and still continue to be unsuc
cessful, we have placed on 90-day notices with an intent to dismiss. And
that has moved some people and has improved others.
When the researcher asked Ms. Cueva (the site principal) to reflect upon her
school’s recent success and respond to the question, “Did you attribute your
school’s success in any way to your evaluation practices?” she stated,
Absolutely not, because I don’t. Okay, along with my idea of getting rid of
tenure, and let’s put the achievement on the table, I believe that people are
doing the best they know how in any given situation. And when they know
better, they’ll do better. I don’t believe you evaluate teachers to a higher
level of performance. I mean, I can write N and put in action plans, but I
don’t think that’s the behavior of what makes teachers improve or lets
teachers improve.
108
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Summary o f Findings fo r Research Question 2
Summarizing the findings of how the teacher evaluation process impacted
teacher behavior at Catson Elementary School, the researcher concluded that the
process positively impacted teacher behavior. Findings from surveys and inter
views indicated that teachers felt comfortable with the principal’s observations and
feedback. Teachers wanted to know what they needed to work on and how they
could improve their practice. As one teacher stated, “It really is about the relation
ship with the administrator.” Although teachers at Catson were very experienced
teachers, they seemed to feel enough trust to be open to suggestions made by the
principal.
As a result of the evaluation process, teachers utilized professional develop
ment opportunities, support from colleagues, and collaboration with team members
to improve their practice in the classroom. At Catson, if a teacher was directed to a
solution from the principal, it meant that the teacher would read professional
literature, observe a colleague, or meet with someone (an expert, coach, or mentor)
to improve teaching practice. Modified Wednesday schedules allowed for
collaboration time so that grade-level team meetings, collaboration, or professional
development could occur. The mantra heard throughout the campus was that “the
answers are in the building.” The principal valued opportunities for teachers to
observe each other teaching so they could learn from and model each other in order
to improve their own practice. The results of the evaluation process seemed to
guide teacher practice.
Findings fo r Research Question 3
Research question 3 asked, What factors exist at the school that positively
or negatively impact the teacher practice? Both research question 1 (How is the
109
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
evaluation process practiced at the school site?) and research question 2 (How does
the teacher evaluation process impact teacher behavior?) were queries into the
policy, practice and impact of the teacher evaluation process at Catson. When the
researcher examined which factors existed at the school that positively or nega
tively impacted teacher practice, care had to be taken that it would not be suggested
that one particular isolated factor would impact teacher practice in and of itself,
independent of the context in which it occurs. A consideration of the roles played
by students and materials, or even the teacher’s own personal or professional
history, could not be ignored as positively or negatively impacting teacher practice.
From the data in Table 18, responses to survey items 15, 16, 17, 18, 20, and
21 reflect factors that teachers felt positively impacted teacher practice but that did
not include direct supervision. There was total agreement on item 16, acknowledg
ing that teachers are encouraged to collaborate on instructional matters on a regular
basis. Another revealing factor was the teachers’ awareness of the school’s goals
and objectives and how much the teachers considered that their involvement in
setting and developing goals and objectives mattered in improving instruction.
Table 19 identifies responses to survey items 23 and 24, which had an
almost 90% agreement by the responding teachers that Catson teachers discuss
curriculum and instruction issues and that they take an active role in initiating
efforts toward school achievement. This can be seen in the use of the school-wide
initiatives of Step Up to Writing, Mountain Language, and Mountain Math, which
were supplementary instructional programs positively endorsed by teachers.
By observation in classrooms and by examining local Catson documents,
such as the single school plan and the School Accountability Report Card (SARC),
investigation revealed a positive relationship between the forced-response items
110
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 18
Survey Findings for Research Question 3, What Factors Exist at the School That
Positively or Negatively Impact the Teacher Practice? Teachers ’ Perceptions o f
School-Based Procedures and Activities, Not Including Direct Supervision, That
May Have Led to School Improvement (N= 19)
Item Mean % Agree3
15.1 have multiple opportunities throughout the year to
participate in professional development activities. 2.0 89
16. Teachers are encouraged to collaborate on instructional
matters on a regular basis. 2.6 100
17.1 have a clear understanding of the professional
development goals for my school. 2.4 89
18. Teachers have an active role in developing professional
development goals and objectives. 2.5 95
2 0 .1 am aware of the goals and objectives of this school. 2.3 84
21. The goals and objectives of this school have contributed
to our school’s improvement 2.3 100
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 = agree, 3 = agree strongly). Number of
teachers =19 out of 29 (66%) who responded.
aS cored 2 or 3.
and the open-ended items W l, W3, W4, and W5 (see appendix C). The extent to
which positive experiences and activities impacted teacher practice are illustrated
in Figure 8, reporting responses to open-ended survey question W l.
Results indicate that 100% of the teachers agreed that collaboration with
grade-level teams in year-long planning was a positive experience. A theme that
resonated throughout the school was that “the answers are in the building.” Sixteen
of 19 teachers stated that discussions during these team meetings had the greatest
111
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 19
Survey Findings for Research Question 3, What Factors Exist at the School That
Positively or Negatively Impact the Teacher Practice? Teachers ’ Perceptions o f
“the Way We Do Things Around Here ” (N = 19)
Item Mean % Agree3
23. Faculty discussions regarding curriculum and instruction
impact our school’s ability to improve. 2.2 89
24. Teachers have an active role in initiating efforts toward
school improvement. 2.3 89
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 - agree, 3 = agree strongly). Number of
teachers =19 out of 29 (66%) who responded.
aS cored 2 or 3.
positive impact on their practice. Although the survey results specified that 17 of
19 teacher responses specifically said that sharing with colleagues was significant,
the researcher concluded that there is a distinct difference between the responses
“discussions with your team” and “sharing with your colleagues.” The grade-level
team consists of colleagues, but the term colleagues in this survey refers to all
teachers on the staff at any grade level, which broadens the response.
Twelve teachers responded that regular planned staff meetings used to meet
school goals had a great impact on teaching practice. Professional development at
the school site, at the district office, and at outside conferences was deemed a posi
tive experience and an activity that impacted teacher practice at Catson.
Continuing with the findings from the open-ended survey items, Figure 9
displays the responses to survey item W3, “Describe the types of interactions with
administration impact your teaching practices either positively or negatively.” Data
112
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The discussions that we have during our team meetings have the greatest impact
on teaching. (16)
Staff development at the school site directed by the administrator and staff
development where teachers share what works best in the classroom. (8)
Observation of other teachers. (2)
Having the freedom to try new things not just stick to district pacing schedules.
Grade level team meetings for collaboration and for year long planning. (19)
Attending professional development conferences. (7)
Regular planned staff meetings used to meet school goals. (12)
Sharing with colleagues. (17)
Time to reflect.
Freedom to experiment with different approaches. (4)
New ideas presented, but freedom to accept, modify, or reject them.
District staff development. (7)
Figure 8. Survey findings for research question 3, what factors exist at the school
that positively or negatively impact the teacher practice? W 1: Describe which
experiences and/or activities have the greatest impact on your teaching practice and
professional development (A T =19) The number in parentheses following a
statement indicates number of respondents with similar answers.
showed that 100% of the teachers had some sort of positive response to their non-
supervisory interactions with Ms. Cueva, the Catson Elementary School Principal.
Teachers responded to the principal’s attributes. As illustrated in Figure 9,
the principal was considered to be positive, helpful, and trustworthy. It was
reported that she listens, is a problem solver, provides nonevaluative feedback to
teachers, allows for freedom of ideas and experimentation, gives of her time to
113
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
When I have a problem, I go to my team and the administrator. I am directed to
a solution by the administrator. Either I will read, or observe or I will meet with
someone. (8)
Feedback and conversation that is non-evaluative is positive. (14)
Focusing on a few goals each year and developing expertise within school.
Our site administrator makes herself accessible at all times, and makes frequent
classroom visits with positive feedback and support. (10)
Principal wants us to come to consensus instead of feeling coerced into doing
something (that is positive).
Our biweekly staff development is positive.
I believe having a great relationship with our administrator has allowed us to
have discussions without feeling uncomfortable. (9)
The principal is always supportive of any new ideas we may present.
An administrator who thinks like a teacher.
Our administrator is good at working with grade level teams and then making
that work as a whole school collaboration. (7)
Our principal is extremely positive, helpful and trustworthy.
She (the principal) trusts her teachers to make well-informed decisions about
teaching methods and practices.
Sharing ideas is positive.
Informal discussions and brainstorming with the principal. (8)
Freedom to try new ideas without negative consequences.
Taking the time to individually discuss with a teacher. (6)
Figure 9. Survey findings for research question 3, what factors exist at the school
that positively or negatively impact the teacher practice? W3: Describe which
types of interactions with administration impact your teaching practices, either
positively or negatively (N = 19). The number in parentheses following a statement
indicates number of respondents with similar answers.
114
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
discuss individually with a teacher, is good at guiding people to work together, and
is always accessible.
Table 20 presents the teachers’ responses to open-ended item W4, regarding
positive or negative interactions with colleagues that impact teaching practices, and
responses to open-ended item W5, regarding the most important factors that have
contributed to student learning at Catson Elementary School.
Generally, teacher interactions at Catson were extremely positive. As
illustrated in Table 20, teachers responded to open-ended survey item W4 that
grade-level collaboration meetings, planning together, staff meetings, even lunch
room talk positively impacted Catson teacher practice. One teacher responded that
it was a positive impact on her teaching practice when staff were striving for
avenues of student success but negative when “teacher talk” was on the futility of
student achievement. Contributing to this negativity and, at times, frustration were
comments by teachers with regard to the federal guidelines of NCLB and the
seemingly impossible AYP goals that must be met by the year 2014.
Responses to open-ended survey item W5 described a plethora of activities
and experiences that teachers claimed to positively impact their teaching practices.
Key to their practice was the attitude that all students can learn at Catson and that,
as teachers, they were willing to try new things—a “whatever it takes” philoso
phy—in order to teach their students. The researcher noted that the teachers
consistently referred to students at Catson using the phrase our students. The
researcher never heard a reference to those students.
As suggested by the findings recorded on the Observation Tool and
summarized in Table 21, aligned with Bolman and Deal’s (1997) four frame model,
the majority of observation findings were in the Symbolic frame. On the 25-item
115
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 20
Survey Findings for Research Question 3, What Factors Exist at the School That
Positively or Negatively Impact the Teacher Practice? W4 and W5 (N= 19)
Question Responses
W4: Describe
which types of
interactions with
colleagues and
other staff
members impact
your teaching
practices, either
positively or
negatively.
• Grade-level team meetings. (8)
Staff meetings. (4)
Eating lunch with our team.
The team is our grade level. We do all the same lessons.
Each member has a few tasks to do for the others.
Planning time with each other.
Observation of each other.
Since we “bank time” and have early release
Wednesdays, grade level teams meet twice a month and
collaborate on planning and teaching strategies.
Our team meetings have allowed teachers to talk to each
other about our teaching concerns. This has helped each
other in our teaching practice.
Team planning and collaboration for year-long plans.
Our entire staff works well together. Our first focus is
student success.
Sharing with each other.
District Staff development..
Lunch room discussions.
Positive is when we are striving for avenues of student
success.
Negative is when we talk about the futility of student
achievement.
116
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 20 (continued)
Question Responses
W5: What do you
feel are the most
important factors
that have contri
buted to student
learning at this
school?
Our professional development is focused on one or two
elements that we get really good at.
Our principal is very supportive of grade level decisions.
Strong visionary leadership from the principal.
Focusing on the standards.
Addressing specific learning needs of students.
Everyone working together toward a common goal.
Everyone helps each other. If someone is an expert,
he/she helps the others or teaches them how.
Cohesiveness among grade levels, and adopting practices
across the school (Mountain Language, Mountain Math,
Step Up to Writing).
Kids are able to gain understanding when concepts are
repeated
Vocabulary development has been important.
Our use of technology keeps kids engaged.
The fact that teachers and staff truly care for the students
and their families.
We have a strong sense of community and we have very
high expectations for ourselves and our students.
Teachers are aware of students’ learning styles and utilize
small group instruction to target students at different
levels.
Our collaboration in teams has contributed to student
achievement. (3)
We try really hard to work together.
Uniform practice of teaching the material in the grade
levels, same homework in each grade level.
Parent involvement.
Music instruction as a literacy tool. (3)
117
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 20 {continued)
Question Responses
• After school tutoring.
• Team meetings and staff development.
• Our belief that every student can learn!
• As teachers, our willingness to try new things in order to
teach our students.
• Ongoing data collection and analysis.
• An administrator who is always looking for cutting edge
practices and is willing to investigate and implement
them.
• Students are number one at our school site.
• We believe that students are capable and challenge them
to higher levels of cognition.
• People tell us we are good, we don’t know why.
• Time for grade level teams to plan and work together.
• Cross grade level sharing and discussion.
• Entire staff works together to meet our “goal.”
Note. The number in parentheses following a statement indicates number of
respondents with similar answers.
Observation Tool (appendix F), 14 responses related to research question 3,
delineating the factors that existed at the school that positively or negatively
impacted teacher practice. Symbolically, most of these observations were
concerned with a positive feeling on the campus and among staff, and positive
interactions among staff members, students, and community. The interview
responses validate these researcher observations.
118
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 21
Observation Tool Findings for Research Question 3, What Factors Exist at the
School That Positively or Negatively Impact the Teacher Practice?
Frame Question Yes No
1 P The campus is clean and well-kept. V
2 P There are display cases throughout the school showing
awards, student work, and events. V
3 P There is a sense of safety and security on campus. V
4 H There is positive interaction between the staff and
administration. V
5 SY There is positive school spirit displayed by staff and
students alike. V
6 S Staff members are familiar with the vision and can
speak to the goals of the vision. V
7 S There is a local reward/recognition program in place
for students and staff. V
8 S Describe the leadership style of the administration:
Yes = Distributed, N = Top-Down. V
9 SY There is positive interaction between the staff, students,
parents, and community. V
10 SY There is positive interaction between staff and administration, a /
11 SY The administration is visible in all areas of the school. V
12 SY The office is warm and friendly when customers enter. V
13 SY There are rituals and events throughout the year to mark
positive learning, social, and environmental happenings. V
14 H Teachers are engaged in school activities. V
15 H The administration has positive interactions with students. V
16 H There is a friendly and positive environment in staff meetings. V
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 21 (continued)
Frame Question Yes No
18 S Substitutes are considered effective instructors. V
21 S Student work is displayed in the classroom. V
Note. H = Human Resources, P = Political, S = Structural, SY = Symbolic.
The researcher found that the school climate and culture at Catson could be
summed in three themes: attitude, relationships, and leadership. A positive attitude
permeates the Catson campus. People seem to enjoy working together. Whether
whole school or grade level, the attitude from staff is that it is all about student
achievement. In addition, every practice at Catson tends to focus on relationships.
At Catson, the term relationship is synonymous with collaboration. People work,
discuss, and collaborate. The final theme of the triad is leadership. The researcher
observed that Ms. Cueva is a strong, respected, and knowledgeable visionary
leader. She is truly at the helm of the Catson ship. She is the captain that sets the
sail for the entire crew to succeed, but she is aware that the ship will not sail with
out a strong crew working together. In the interview, the principal mentioned a
quote by Elliot Eisner that she said strongly embodies her philosophy of teacher
practice in the classroom and student learning:
“The kind of net we know how to weave determines the kind of net we cast,
which determines the kind of fish we catch.” All teachers know how to
weave a simple net, which maybe they just catch the fish that are the easiest
to catch, those are the kids that intrinsically learn things. So, “come here,
you’re an easy student, and I’m going to catch you.” But, what I’m looking
for are teachers that can be sophisticated weavers of nets and weave nets in
layers so we can scoop up all the fish, even the ones that are hard to catch.
And that only happens, I believe, when I can get the evaluator role off the
120
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
table and get into the heart-to-heart conversation about, “I’m a teacher,
you’re a teacher, why are you teaching it that way?”
During the individual teacher interviews, responses to questions 3.1, 3.2,
3.3, 3.4, 3.5, and 3.6 related to research question 3 (appendix D) regarding what
factors exist at the school that positively or negatively impact the teacher practice.
Responses to questions 3.1 and 3.2 overlapped in describing reasons that the
teachers considered Catson Elementary School to be successful. According to
Teacher H,
I think we have a good atmosphere among the staff. The teacher relations
with each other are very good.... If there is something that comes along,
we share things . . . papers, books . . . and the staff has always been very
friendly and outgoing towards one another. Never possessive of their
materials or their things, always willing to help each other. I’ve noticed that
since I started here 16 years ago.
Teacher G’s response is in agreement with Teacher H, who stated, “I don’t think
it’s just our practices, I think it’s the teachers that are on campus; they really try to
work with the kids in different ways.”
Teacher F reflected on the many reasons that Catson School is successful.
I think that we have an awesome administrator. She’s a great leader. She
allows us to be teachers and teach . . . we’re in the classroom and she values
that, she values our opinions; she values our ability to bring good practices
to our kids. And she’s very open. If we come to her and say this is not
working, we want to challenge our kids and this is not going to work. And
she was like, okay, she allowed us to go ahead. She trusts us; I guess is
what I want to say. She really trusts us as teachers to make good decisions.
Teacher C responded that teacher collaboration made Catson successful.
The researcher noted that her entire response was in the “we” form rather than the
first person form.
We work really well together. We work in teams and we plan as a team and
we compare at a team and we think that makes a difference. And within
those teams, when we get together at our staff meetings as a school group,
everybody really respects each other and is willing to hear new ideas. It can
be really easy to be set in your ways and just say I’m going to tech this, but
we’re not like that. We’re always open to looking for new strategies and
121
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
techniques to improve. We think that’s evident in our scores. We look sat
the scores and say, okay, we’re lacking in this area; we need to do some
thing to change it. And we just don’t do it as individual teachers, but as a
team.
Additional responses as to why Catson is so successful included the
response by Teacher E:
I think that we’re here to support each other, and I think that every year
there’s more continuity between the grades, and from grade to grade....
We are really interested in what children need, not just what the district says
to do or what we feel like we want to do.
Teacher D commented,
I think that people here are dedicated to what they’re doing. We work
together really well. We hold our students to high standards, and we don’t
make excuses for them because they are Title I or they speak another
language. At Catson, we all have a similar school focus.
When asked about the impact of the Catson School culture (interview
question 3.3), Teacher F stated that is was a major impact.
I think that is why we developed a school-wide vocabulary [program], and
then a school-wide writing program with Step Up to Writing. We already
had Mountain Math and Mountain Language, and we wanted to have some
things be all K to 5 because it makes such a big school impact.
Teacher B stated that the school culture impacts her teaching practice
because of the trust that she and others have for the school principal.
I know that initially when she came in she didn’t make changes right away.
She was able to honor the way people did things whether she believed them
to be right or wrong. She honored them. And then it wasn’t until a few
years later that she started implementing more of a demand of what things
should be. You know, structurally. Before it was my students, and now I
think the philosophy has gone from my students to our students. We now
know we have a community even though that child might be in someone
else’s class, they’re still part of my responsibility to work together, which
goes on to our rotations that we have from time to time and year to year.
That is the culture of our school.
Teachers expressed very strong support for professional development at
Catson and its impact on teacher practice (interview question 3.4). Teacher C
responded (with regard to professional development), “That’s very powerful. I
122
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
think our staff meetings, the majority of time is spent on staff development. And
that has a definite influence on all of our teaching styles and improving our
program.”
On the other hand, Teacher H did not agree that professional development
impacted his practice. When asked to what extent professional development
impacted teaching practice, Teacher H responded,
I would have to say very little to be honest. Not that I’m negative with
professional development. I just think that the professional development
that has been presented reassures me that I’m doing something right
already. I don’t ever think that it’s given me some new revelation. All that I
feel it’s doing is reassuring me about what I’m doing. Yes, I’m still on the
right path and it’s more like a reassurance staff development.
Teacher A stated that staff development
definitely improves my teaching practice. There are most teachers who will
take the information, and adjust it the way they think might be best in their
classroom but, at this school they’re willing to try it and definitely try new
methods and ideas. Absolutely.
Continuing to examine what factors at the school positively or negatively
impact the teacher practice, interview question 3.5 asked teachers to what extent
collaboration and collegiality impacted their teaching practice. Teacher H was
emphatic:
I would say it does, for sure. Because, we work together as a team. We
give the same homework, we talk about what we’re doing in the classroom
with each other, and we try to stay close to the same page in our language
and math so that all of our homework is the same.
Teacher F remarked that teacher collaboration and collegiality are significant at
Catson.
I think that it’s a big reason why we have done as well as we have because
we do collaborate and we do work together. And like I said, not only as
individual teams, but within. Ms. Cueva is really good about during staff
meetings she’ll have us work, and there are times where we will work as
teams and there are times where she’ll cross grade. She’ll have us in cross
grade teams, so it’ll be a teacher from K, 1,2, 3, 4, and 5 that works
together on an idea. She wants the fifth grade teachers to see what the
123
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
straggles are for the third grade teachers, and so on. She wants us to see
what others are doing at the different grade levels so that we know their
challenges. She makes an effort to do that. I think that’s really important.
Interview question 3.6 reflected on factors at the school that positively or
negatively impact teacher practice by asking teachers to express the extent to which
leadership impacted their teaching practices. Teacher D stated, “Leadership helps
to give us focus on what we’re doing instead of walking blindly along. It just helps
us constantly refresh our memory in keeping us focused.” Teacher A responded in
a thoughtful manner:
You know, we are doing well at our school. Again, I think when you have
somebody who is in that leadership role and who has a direction and knows
what they want to do, and what they want to accomplish, and what really
values the other teacher opinions and ideas and strategies, I think . . . we’re
more willing to give to her. I think it a reciprocal relationship. She gives to
us, and we give to her, and I think that when it is shown that she is a very
capable leader, then we’re willing to work and we do.
Teacher C also stated that leadership had an important impact on her
teaching practice.
It’s, you know, you feel like you can’t try new things, or if you’re not
getting supported, then it would be very discouraging in your job.... We
are fortunate. Because the principal is such a positive person, I think
because she is so positive and because she has such a big vision, she kind of
makes us stretch. It’s not always completely comfortable, but I think she
certainly makes us stretch and try to always do the very best we can do.
Summary o f Findings fo r Research Question 3
To summarize the findings for the third research question (What factors
exist at the school that positively or negatively impact the teacher practice?), the
triad of the themes of attitude, relationships and leadership prevailed. Once the
door is closed in the classroom, it is the attitude and the belief system of the
teachers at Catson that their students are capable and that they can be challenged to
higher levels of achievement. Relationships are vital at the school. They include
124
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
collaboration in planning, collaboration in instruction and teams, and respecting
and listening to one another as colleagues and friends that make them always want
to push a little harder.
Leadership plays an important role in positively impacting teacher practice
at Catson. The principal is willing to investigate new practices and implement
them. As one staff member explained, “We have a leader who is the principal and
who is a teacher. She is very involved with us, helping us to develop as educators
and as professionals. She is very supportive and hands-on.” The findings show
that the principal, as the leader of Catson, works to improve teacher practice by
weaving a thickly layered net in order to scoop up all of the fish, especially the
ones that are so difficult to catch.
Findings fo r Research Question 4
Research question 4 asked, How does the district/site leadership influence
the teacher evaluation process? Research questions 1, 2, and 3 dealt primarily with
the policy, practice, and impact of the teacher evaluation process in relation to
student achievement. The fourth research question investigated the evaluation
process by examining the influence of the evaluation process by the district office
and by the site leadership.
In triangulating the data the researcher first investigated the teacher survey
responses. Forced-response survey items 1, 2, 5, 6, 7, 8, 9, 12, 19, 22,24, and 25,
and open-ended survey questions W2 and W3 (appendix C) pertained to the influ
ence of the district and/or site leadership on the teacher evaluation process, if any.
Table 22 shows that 100% of the teachers who responded to the survey
understood and were aware of the intent and policies of the evaluation process.
125
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 22
Survey Findings for Research Question 4, How Does the District/Site Leadership
Influence the Teacher Evaluation Process? Teachers ’ Knowledge and Perceptions
o f Official Documents Regarding the Teacher Evaluation Process (N= 19)
Item Mean % Agree3
1. I am aware of and understand the district’s policy
regarding teacher evaluation. 2.6 100
2. I agree with the goals and objectives of my district’s
policy on teacher evaluation. 2.7 100
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 = agree, 3 = agree strongly). Number of
teachers =19 out of 29 (66%) who responded.
aScored 2 or 3.
Illustrated in Table 23, the staff was not in complete agreement as to the site
administrators’ involvement in the formal teacher evaluation process. In response
to survey item 5, only approximately 50% of the teachers considered the principal’s
implementation of the evaluation policies as an integral part of their professional
growth. This is a good example of the disconnect between policy intent, policy
practice, and policy implementation of the teacher evaluation process. According
to the survey results, only 68% of the teachers agreed that, when the principal
formally observed a lesson, she looked for pre-agreed evidence. Also, only 68% of
the teachers had confidence in the principal’s ability to formally evaluate their
instructional practice. This response did not seem to match the open-ended and
individual interview responses concerning the principal’s abilities in evaluating
teacher practice. In the individual interview and open-ended survey responses,
teachers stated strongly that the principal had the ability to evaluate lessons. In
126
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 23
Survey Findings for Research Question 4, How Does the District/Site Leadership
Influence the Teacher Evaluation Process? Administration’ s Involvement in the
Teacher Formal Evaluation Process as Mandated by California Education Code
(N — 19)
Item Mean % Agree3
5. I view my administrator’s implementation of the teacher
evaluation policies as an integral part of my professional
growth. 1.3 52
6. When my administrator visits my classroom, he/she looks
for things, which we agreed upon at pre-conference. 1.6 68
7. I receive timely and meaningful feedback regarding my
teaching observations. 2.1 89
8. I am confident in my administrator’s ability to evaluate
my instructional practice. 1.7 68
9. There are alternative evaluation opportunities (e.g., PAR,
portfolio, etc.) available at my school site. 2.1 79
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 = agree, 3 = agree strongly). Number of
teachers =19 out of 29 (66%) who responded.
aScored 2 or 3.
fact, as shown in response to survey item 7, almost 90% of the teachers responded
that they received timely and meaningful feedback regarding their teaching
observations. During the individual interviews, most teachers knew about the
alternative evaluation opportunities. Permanent teachers could opt for a POP; if
needed, PAR would be an alternative for remediation. Only 79% responded that
they were aware of these alternatives.
127
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
As shown in Table 24, all Catson teachers did not particularly see the
supervision of instruction as separate from the formal evaluation process. At
Catson Elementary, 42% of the teachers who responded to the survey were aware
that the supervision process and formal evaluation process are separate.
Table 24
Survey Findings for Research Question 4, How Does the District/Site Leadership
Influence the Teacher Evaluation Process? Administration’ s Involvement in the
Supervision Process Through Observation, Data Collection, Feedback, Goal
Setting, and Improvement Strategies (N=\9)
Item Mean % Agreea
12.1 see the administration’s supervision of instruction as
nonevaluative and separate from the formal evaluation
process. 1.4 42
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 = agree, 3 = agree strongly). Number of
teachers =19 out of 29 (66%) who responded.
aScored 2 or 3.
On the other hand, responses to survey items 19, 22, 24, and 25, shown in
Table 25, suggest that teachers at Catson had a high agreement in their comfort
level interacting with the principal for needed support, and all of the teachers
expressed satisfaction with the professional competence and leadership ability of
their principal.
The teachers reported that they have an active role in initiating school
improvement efforts, and they responded overwhelmingly (95%) that most of the
128
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 25
Survey Findings for Research Question 4, How Does the District/Site Leadership
Influence the Teacher Evaluation Process? Teachers ’ Perceptions o f Policy,
Process, and Implementation (N= 19)
Item Mean % Agree3
19.1 am comfortable going to my school administrators
for support. 2.3 84
22.1 am satisfied with the professional competence and
leadership ability of the administration. 2.5 100
24. Teachers have an active role in initiating efforts
towards school improvement. 2.3 89
25. The majority of school improvement efforts at this
school have been initiated by the district and/or
site administration. 2.3 95
Note. Results expressed as the mean of teacher responses made on a 4-point scale
(0 = disagree strongly, 1 = disagree, 2 = agree, 3 = agree strongly). Number of
teachers =19 out of 29 (66%) who responded.
aScored 2 or 3.
improvement efforts at Catson were initiated by site administration. This matches
previous data collected with regard to Catson site improvement efforts.
Table 26 reiterates teachers’ responses to the open-ended survey questions
W2 and W3. These written responses to both survey questions were also used as
findings with regard to research questions 2 and 3. Teacher survey item W2, which
asked teachers to describe which experiences and/or activities had the greatest
impact on student achievement at Catson School, resulted in multiple responses as
to what the teachers perceived impacted student achievement at Catson Elementary
School. Teacher collaboration, grade-level meetings, staff meetings, and
129
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 26
Survey Findings for Research Question 4: How Does the District/Site Leadership
Influence the Teacher Evaluation Process? Responses to Open-Ended Survey
Items (N = 19)
Item Responses
W2: Describe
which experiences
and/or activities
have the greatest
impact on student
achievement at this
school.
I believe the decisions that we make during team meetings. (4)
The on-site professional development that we receive.
Faces to Data.
Staff Development trainings. (2)
Small group differentiated instruction. (2)
Attitudes where all (students, teachers, support staff, parents) at
the school work toward a common goal.
Vocabulary development.
Integration of technology and music.
Having high expectations of students.(2 )
The manner that each grade has similar programs so that
students see consistency and repetition throughout each grade
level. (2)
Strategies for improving teaching practice as well as student
achievement.
Music instruction as a literacy tool.
Our after school tutoring has had a great impact on student
achievement.
Rotating students into other classrooms to support a specific
skill a student might be. lacking has helped students.
Whole school collaboration with grade level teams working
closely together.
Careful analysis of data which then drives instruction.
Sharing.
Staff meetings.
Instruction focused on meeting standards.
Support staff.
Success builds on success (I can do it attitude!).
Personal streamlined attention.
130
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 26 (continued)
Question Responses
When I have a problem, I go to my team and the administrator.
I am directed to a solution by the administrator. Either I will
read, or observe or I will meet with someone. (8)
W3: Describe
which types of
interactions with
administration
impact your
teaching practices,
either positively or
negatively.
Feedback and conversation that is non-evaluative is positive.
(14)
Focusing on a few goals each year and developing expertise
within our school.
Our site administrator makes herself accessible at all times, and
makes frequent classroom visits with positive feedback and
support. (10)
Principal wants us to come to consensus instead of feeling
coerced into doing something (that is positive).
Our biweekly staff development is positive.
I believe having a great relationship with our administrator has
allowed us to have discussions without feeling uncomfortable.
(9)
The principal is always supportive of any new ideas we may
present.
An administrator who thinks like a teacher.
Our administrator is good at working with grade level teams
and then making that work as a whole school collaboration. (7)
Our principal is extremely positive, helpful and trustworthy.
She (the principal) trusts her teachers to make well-informed
decisions about teaching methods and practices.
Sharing ideas is positive.
Informal discussions and brainstorming with the principal. (8)
Freedom to try new ideas without negative consequences.
Taking the time to individually discuss with a teacher. (6)
Note. The number in parentheses following a statement indicates number of
respondents with similar answers.
131
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
professional development were stated as teacher support activities that positively
impacted student achievement. In addition, careful analysis of data, sharing,
putting faces to data, and focusing instruction on meeting standards were practices
reflected in the teachers’ responses to item W2 as having a major impact on student
achievement at Catson.
In response to open-ended survey item W3, teachers described which types
of interactions with administration impacted their teaching practices, either
positively or negatively. That survey question provided teachers the opportunity to
ponder the more comprehensive research question 4 (How does the district/site
leadership influence the teacher evaluation process?). Teachers at Catson agreed
strongly that they could go to their administrator for resolutions to problems and
support. Data illustrated that feedback and conversations with the principal were
extremely positive. A number of teachers responded how positive it was to have a
great relationship with their principal that allowed them to have open discussions
without feeling uncomfortable. Teachers identified that an important and positive
impact to their teaching practice was to have the principal accessible and visible in
their classrooms.
In addition to the open-ended survey responses, responses to the individual
teacher interview questions 4.1 and 4.2 gave the researcher insight into how the
district and the site leadership influence the teacher evaluation process.
In response to question 4.1 (How does the district office leadership contri
bute to improved teacher practice?), most of the responses were negative. Those
teachers did not agree that the district contributed to teacher practice at all. On the
other hand, Teacher F stated that the district office was supportive. “I know that
the district has a lot of respect for our principal and that’s because she’s getting
132
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
results. So, I think that they are more willing to support what we’re interested in
doing.” Teacher E concurred: “I could say that perhaps they (the district office)
allow our principal to fly like a bird, and do what she sees fit here at this school.
I guess you could say that was contributing.” With regard to district office leader
ship contributing to improved teacher practice, Teacher B commented,
I think it is a struggle because, for example right now they’re [the district
office] implementing a shared reading program to have 90 percent of
students reading at grade level by 2007. I think they’re helping in the sense
that they’re giving us coaches and we have one of our teachers as a lead
teacher getting training so he can also teach the other teachers. Also they’re
here one Wednesday a month at our staff development to look at new
reading strategies. So, in that way it is improving our strategy.
From the district perspective on how district leadership influences the
teacher evaluation process and impacts teacher practice, according to Dr. HR,
Assistant Superintendent of Human Resources,
We [the district] have an initiative, a literacy initiative to have 9 out of 10
students proficient by 2 0 0 7 .... There is a tremendous effort on the part of
the district in expenditure to provide the tools that people need to be more
effective in the classroom. And we’re starting with our principals, with our
administrators. Principals and Assistant Principals, teaching them the tools
and they then turn around and are teaching the teachers the tools. As a
result of that, we again believe that we are making an impact. But some
times we see some things that don’t connect, you know, they just don’t
really make sense. And we wonder sometimes what has happened here.
Research question 4, asked how the district/site leadership influenced the
teacher evaluation process. There were multiple responses to teacher interview
question 4.2 (Who do you view at school leaders and how do they impact teacher
practices?). Teacher B commented,
Whatever the district mandates for us to do, the principal is our leader, and
tries to give us what she knows we can do. Like she doesn’t give us all that
we need to do. So she kind of acts like the filter in that sense.
Teacher E added,
We have one grade level representative from each grade level on an Instruc
tional Leadership Team. So I would say those people kind of lead the
133
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
school. They give direction. As team leaders, they keep the team focused,
and sometimes act as the go-between, between the team teachers and the
principal.
Teacher A responded,
You know, it’s interesting because I honestly think all of our teachers here
are leaders. In this school we all have a leadership role to play. And per
haps there is a teacher who is strong in a certain area and will step up to the
plate. Meanwhile, there is another teacher strong in another area, and
he/she will pickup the other part of the road to keep going.
When the principal was interviewed in regard to the influence of site
leadership and the teacher evaluation process, she responded,
Okay, talking straight truth. I think the whole system of evaluation for
teachers is broken. Because if there’s going to be accountability, then there
should be accountability built into the teacher’s observation and evaluation.
If we’re going to say, you will raise scores by this much, you should be part
of it. It’s like, we’re trying to run the school on one system and then just
hoping and praying that student achievement is going to rise on the other
system with no connection between the two. I mean, seriously.
So, you know what you’re going to be held accountable to for student
achievement, then let’s write it into the evaluation form. And let’s talk
about it. You know, it’s just plain wrong that. . . . But you know, I am an
iconoclast on this. It’s plain wrong that teachers get tenure. Why?
Why should they [teachers] have tenure? To me, I’m sorry, because I really
am a fan of teachers, to me, tenure protects incompetent workers. That’s all
it does. It doesn’t help at all to raise student achievement. And I don’t
believe that you have teachers that are so intellectually free-thinking that
tenure is going to protect them because they’re going to say something.
This whole tenure thing, in my opinion, is out. And everybody needs to be
like everybody else in the school system. You know what? You get a
March 15th letter. Classified people get a two-week notice. Everybody
needs to be on . . . let’s level the playing field. So, I completely disagree
with the whole teacher evaluation process [as is].
Using the Observation Tool (appendix F), aligned with Bolman and Deal’s
(1997) four frame model, the researcher examined those items that were reflected
in research question 4 (How does the district/site leadership influence the teacher
evaluation process?).
As illustrated in Table 27, the findings showed that the site leadership
appeared to be distributed. Although there was no doubt that, as principal,
134
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 27
Observation Tool Findings for Research Question 4, How Does the District/Site
Leadership Influence the Teacher Evaluation Process?
Frame Question Yes No
8 S Describe the leadership style of the administration:
Yes = Distributed, N = Top-Down. V
10 SY There is positive interaction between the staff and
administration. V
11 SY The administration is visible in all areas of the school. V
Note. H = Human Resources, P = Political, S = Structural, SY = Symbolic.
Ms. Cueva was the site leader, there was also a strong ILT and active grade-level
team leaders known as lead teachers. The principal seemed very open, and all
interactions with staff appeared positive and supportive. As stated by Teacher C,
I think that teacher evaluation will have an impact on teacher practice when
it’s done in a certain way. And the best way, is when it’s done out of how
to help the teacher, as opposed to the administrator being a watch dog
looking for something that’s wrong.
Teachers wanted the principal on campus. There were some grumblings that the
principal was too often off at a meeting somewhere. Sometimes, as Teacher C
noted, “The principal is spread too thin.” From the observation data collected by
the researcher, the principal was very visible and highly revered by students, staff,
and parents.
Summary ofFindings fo r Research Question 4
When summarizing the findings of the fourth research question, the
researcher again examined the findings regarding the influence of district and site
135
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
leadership on the teacher evaluation process. Observing the teacher evaluation
process in policy, intent, practice, and impact, the researcher identified that the final
descriptor, in response to research question 4, was influence.
Dr. HR, Assistant Superintendent of Human Resources at the district office,
stated that he wished the district office could take credit for the success at Catson.
He stated, “What is happening at Catson Elementary had more to do with a lot of
factors, and a lot of variables that are difficult to pinpoint exactly.” Dr. HR stated
that the group at Catson is, of itself, a special group. He noted the strong energy in
the staff at Catson and that they all seemed to build on one another.
Having a very compatible group of people who like working with each
other, and who are very creative, and they have a dynamite leader, I mean,
with all these things, it’s just like aligning all the stars. I think that with
Catson, the stars have aligned. As a result of that, having an instrument that
helps them along with everything else that they’re doing, w hy.. . that’s
why they are doing so well. It’s a very special school.
The scope of the principal’s influence at Catson Elementary School is
enormous. She is all things to all people. She sets the sail and steers the rudder.
She is truly captain of the Catson Elementary “ship.” Her influence goes beyond
the teacher evaluation process. She is all about teacher practice. At the formal
evaluation conference with the teachers, she added one part, which she called
“Reflections to Make the Lesson Better.” She said that she does with the teacher,
and that this is her influence for making teachers better.
Discussion o f the Findings
The purpose of this qualitative case study was to determine whether the
teacher evaluation process impacted teacher performance in support of student
learning in a successful urban school. The researcher examined Catson Elementary
136
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
School, a high-achieving Title I school in the RHUSD located in the Los Angeles
County region of southern California.
The methodology for this study was based on four research questions
developed by a 2006 USC doctoral student cohort. The four research questions
drove the development of the survey, the teacher and administrator interview proto
col, and the observation tool, all of which were used as data collection instruments.
The findings were triangulated to respond to these research questions. The
researcher focused on examining data concerning the teacher evaluation process in
three ways. First, the researcher examined the teacher evaluation process through
four lenses: policy, practice, impact, and influence of the teacher evaluation pro
cess. Second, the researcher aligned all four research questions on the Observa
tional Tool to Bolman and Deal’s (1997) four frame leadership model. Third, the
researcher identified recurring themes: academic goals, leadership, teacher prac
tices, accountability, and intervention. These themes were prevalent throughout the
analysis and could be aligned easily to the Broad Best Practice Framework.
The four research questions for this study were:
1. How is the teacher evaluation practiced at the school site?
2. How does the teacher evaluation process impact teacher behavior?
3. What factors exist at the school that positively or negatively impact the
teacher practice?
4. How does the district/site leadership influence the teacher evaluation
process?
Based on these four research questions, the researcher focused findings on
examining data concerning the teacher evaluation process through four lenses of
137
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
the teacher evaluation process: the policy lens, the practice lens, the impact lens,
and the influence lens.
Policy Lens
The need to examine policy was implied prior to responding to research
question 1. It was implied because, without a teacher evaluation policy, there
would be no practice, impact, or influence of the teacher evaluation process.
The policy lens was examined by reviewing a medley of policy documents
related to the evaluation process for certificated teachers. The district has very
comprehensive evaluation policies and procedures which follow the tenets of the
Stull Bill and the California Education Code. There was an evaluation notebook at
every site. In the section entitled “Regulations and Policies Governing the Evalua
tion of Certificated Employees” were the policy and procedure guidelines for
teachers.
Practice Lens
The practice lens was examined through data collected and analyzed to
address research question 1. There were definite district policies and procedures
with regard to the teacher evaluation process but there was a disconnect between
intent and practice. Since all of the teachers who participated in the survey and
interviews were permanent employees, the researcher closely examined their
evaluation requirements. There was only one Probationary II teacher, who had
come from another district and was an experienced teacher; the other 28 teachers
were permanent. The practice of teacher evaluation in the district was that a
permanent teacher was evaluated every 3 years. After two successful sequential
evaluations, a permanent teacher could opt to choose the POP as an alternative to
138
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
the traditional evaluation. The district had a PAR program in place for remediation
due to an unsatisfactory evaluation.
Impact Lens
Examining the impact lens of the teacher evaluation was viewed in two
ways. The first impact lens was with regard to research question 2, examining how
the teacher evaluation process impacted teacher behavior. The second impact lens
was with regard to research question 3, examining what factors existed at the
school that positively or negatively impacted the teacher practice.
Most teachers felt that the evaluation kept them “on their toes.” The
teachers stated that the post-observation meeting was beneficial. As the data
showed, the principal had a bit of her own way of handling the evaluation. She
was in “borderline” compliance with the district policies and procedures, mostly
because she did not agree with them. Here was the difference between intent and
actual practice. The data suggested that the respondents to the survey and inter
views felt very comfortable with the evaluation process at Catson.
Factors at Catson Elementary School that positively impacted teacher
practice but did not include teacher supervision suggested three themes: attitude,
collaboration, and accountability.
The data revealed that the teachers maintained a positive and collegial
attitude. In fact, they embraced a “whatever it takes” attitude in working together
toward a common goal. They had a strong sense of community and very high
expectations for themselves and for their students. They had the attitude that their
students were capable, and they constantly challenged the students to higher levels
139
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
of cognition. A positive attitude was present in the strong belief at Catson that
every student can learn.
The data suggested that collaboration was a key factor that positively
impacted teacher practice and contributed to student achievement at Catson.
Through collaboration, teachers helped each other. As one teacher stated, “If
someone is an expert, he/she helps the other or teaches them how.” Teachers stated
that their collaboration in grade-level teams had positively contributed to their
practice. The principal was very supportive of grade-level decision making.
Scheduled time for grade-level teams to plan and work together and cross-grade-
level sharing and discussion were also factors that positively impacted teacher
practice at Catson Elementary School.
The manner of accountability at Catson positively impacted teacher practice
and strongly contributed to the school’s continued success in student achievement.
Ongoing data collection and analysis had become part of the Catson culture and the
“way we do business around here.” Teachers had positively responded to monthly
and quarterly “faces to data” reports that had carefully tracked the progress of each
student at Catson. Additional factors that positively impacted teacher practice were
school-wide vocabulary, writing, mathematics, and language programs. Accounta
bility was in follow-through, standards-based instruction, and implementation of
professional development based on best practices. Professional development had
positively impacted teacher practice at Catson because the professional develop
ment was focused on only one or two elements until the staff “gets good at it.”
Accountability to the parents and parent involvement was a factor that positively
impacted teacher practice and contributed to greater student achievement.
140
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Influence Lens
The influence lens related to how the district and site leadership affected the
teacher evaluation process. Most teachers did not agree that district leadership had
any influence on the evaluation practice; all of the teachers agreed that the site
leadership was extremely influential in regard to the entire evaluation process.
Teachers took to heart the feedback that they received from the principal. They
regarded her as the expert. They stated that formative ongoing evaluation would be
more beneficial than their current practice of a formal evaluation once every 3
years. Site leadership was influential in the evaluation process with regard to how
teachers stated the trust, respect, and confidence that they felt for their visionary
principal.
Four Frames Aligned
The instrumentation chart that was used to analyze observational data was
developed by a subgroup of a 2006 doctoral cohort at USC. The chart was
developed to guide the researcher in the analysis of the data in an organized manner
in order to triangulate it with the survey and interview protocols. It was aligned
according to Bolman and Deal’s (1997) four frame leadership model. Each
research question was individually analyzed, and the data were presented on a table
within each research question analysis. Table 13 analyzed observations of how the
teacher evaluation was practiced, Table 17 analyzed observations of how the
teacher evaluation impacted teacher behavior, Table 21 analyzed observations of
factors that positively or negatively impacted teacher practice, and Table 27
analyzed observations of how the district and site leadership influenced the teacher
evaluation process.
141
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 28 is a summary of the data collected using the Observational Tool.
With regard to research question 1 (how teacher evaluation is practiced), 7 (70%)
of the observations were in the Structural frame. With regard to research question
2 (how teacher evaluation impacted teacher behavior), 10 (67%) of the observations
were in the Structural frame. With regard to research question 3 (factors that
positively or negatively impacted teacher practice), 6 (33%) of the observations
were in the Symbolic frame and 5 (28%) were in the Structural frame. With regard
to research question 4 (how district and site leadership influenced the teacher
evaluation process), 2 ( 67%) of the observations were in the Symbolic frame.
Examining the results vertically, the data revealed that 11% of the responses
were in the Political frame, 50% of the responses were in the Structural frame, 24%
of the responses were in the Symbolic frame, and 15% of the responses were in the
Human Resource frame.
Examining the 50% response in the Structural frame as an organizational
process, Bolman and Deal (2003) viewed evaluating in a structural frame as a way
to distribute rewards or penalties and control performance. Under the process of
goal setting in the Structural frame, Bolman and Deal stated that it keeps the
organization headed in the right direction. The researcher expected to see a greater
response in the Human Resource frame, where the evaluating process was inter
preted as a process for helping individuals to grow and improve. Under goal set
ting in the Human Resource frame, the process was to keep people involved and
communication open. The researcher had expected the greater percentage of
responses to be in the Human Resource frame, but the results were not as expected.
The results shown in Table 28 indicated that the teacher evaluation process
at Catson followed the structural frame assumption that “the right formal
142
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 28
Observation Tool Frame Analysis: Responses Tallied According to Bolman and
Deal’ s Leadership Frames
Research
question
Political
frame
Structural Symbolic
frame frame
Human
resource
frame
Question 1: How is the teacher evalua
tion practiced at the school site?
1 7 1 1
Question 2: How does the teacher
evaluation process impact teacher
behavior?
1 10 2 2
Question 3: What factors exist at the
school that positively or negatively
impact the teacher practice?
3 5 6 4
Question 4: How does the district/site
leadership influence the teacher
evaluation process?
0 1 2 0
Percentage by frame summed vertically. 11% 50% 24% 15%
Note. Number denotes number of responses based on data from Tables 13,17, 23,
and 27.
arrangements of social architecture and organizational design minimize problems
and maximize performance” (Bolman & Deal, 2003, p. 44). In other words, the
evaluation process in the structural frame existed to achieve established goals and
objectives, increase efficiency, and enhance performance at Catson School (in the
RHUSD) and was designed to fit the organization’s goals, workforce, and
environment.
143
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Recurring Themes
Along with examining the data through the four lenses of policy, practice,
impact and influence, the researcher found that recurring themes of academic goals,
leadership, teacher practices, accountability, and intervention were prevalent
throughout the analysis and could be easily aligned to the Broad best practice
framework introduced earlier (see Table 10). The best practice framework
“provides an organizational schema to examine the practices of consistently high-
performing school systems in a variety of settings (National Center for Educational
Accountability, 2005, p. 1). Through the framework, the National Center for
Educational Accountability has created a method by which to compare practices at
high performing schools. Rigorous school selection and research criteria has led to
the identification and study of high-performing schools within and across states
throughout the country.
Five themes were prevalent at Catson Elementary School, aligned with the
Broad Best Practices of High-Performing School Systems Framework (2005). For
this study, the themes were aligned according to school and classroom practices.
1. Curriculum and Academic Goals—’’ What is Taught and Learned.” This
theme focused on the learning target. At Catson Elementary School a strong theme
seen throughout the data findings was academic goals. At Catson, the school
marquee states the school-wide and district-wide academic goal that 9 of 10
students will score proficient by 2007. The school also uses an ongoing “faces to
data” approach of analysis to continually monitor agreed academic goals. The
learning goals are for all K-5 students at Catson, and they are not negotiable.
2. Staff Selection, Leadership, and Capacity Building— “Selecting and
Developing Leaders and Teachers.” This theme focused on the selection and
144
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
development of the people (teachers and administration). The Catson theme that
was consistent throughout the findings was leadership. The site leader was con
sidered capable, trustworthy, knowledgeable, and a visionary leader. The leader
ship of the teachers focused on lead teachers and the ILT. The principal was the
driving force for the school.
3. Instructional Programs, Practices, and Arrangements—’’ The Right Stuff,
Time and Tools.” This theme focused on the “things” that successful schools use:
the arrangement of time, instructional resources and materials, technology, and
so forth. At Catson, teacher practices was the theme that aligned in this area.
Wednesdays were on a modified day schedule so that, by banking minutes on the
other days of the week, teachers were able to collaborate, receive professional
development, look at student work, and have grade-level planning meetings.
4. Monitoring-Compilation, Analysis, and Use of Data— “Knowing the
Learners and the Numbers.” In this theme, clear identification of what should be
taught and learned by grade and subject ensured that the school was equipped with
the staff and tools to deliver the curriculum successfully. At Catson, accountability
was the theme that aligned in this area. Teachers turned in “faces to data” reports
on monthly and quarterly bases to the principal. Measurable goals were posted by
grade level throughout the school. All instruction was standards based at Catson,
with teachers using the California state standards for content and the CSTP for
instruction. Grade-level collaborative data analysis was a part of the school culture
at Catson.
5. Recognition, Intervention, and Adjustment—’’Ensuring All Children
Learn.” The most important teacher practice was to teach, but the teacher had to
follow and monitor student performance. The prevalent theme at Catson was
145
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
intervention. From the study findings it was clear that teachers at Catson provided
repetition of similar strategies across grade levels, they used specific strategies for
improving teaching practice as well as student achievement, there was an after
school tutoring program, students were rotated into other classrooms to support
a specific needed skill, and there was small-group instruction, differentiated
instruction, and multiple parent-teacher-student conferences.
By collecting and examining data, the findings of this qualitative case study
were triangulated to determine how the teacher evaluation process impacted teacher
performance in support of student learning in a successful urban elementary school.
146
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 5
SUMMARY, CONCLUSIONS, AND IMPLICATIONS
Summary o f Background
Traditionally, teacher evaluation has been used to determine a teacher’s
acceptability for continued employment in a school district. A majority of teacher
evaluation systems are seen by many school administrators and teachers to be
extremely stressful, of little or no value, and a barrier to high staff morale
(Saskatchewan School Trustees Association, 1995). Therefore, administrators may
wonder whether the evaluation process contributes to growth in teacher practice
and student achievement.
Fundamentally, most parents would wish to have a guarantee that their child
would have an effective teacher. If having an effective teacher means that the
teaching quality makes a significant difference in the student’s learning, then
NCLB (2002) has solidified this into law. NCLB requires a “highly qualified”
teacher in every classroom by the 2005-2006 school year and achievement gains by
all students over time (Goldrick, 2002). As the public appears to strongly favor
educational reform tied to accountability, they also equate educational improve
ment with quality teaching. There appears to be a consensus that teaching quality
is a critical influence on how and what students learn (Cochran-Smith, 2003).
In light of current research findings that indicate that teacher expertise is
one of the most important factors in student achievement, policy makers are aware
now, more than ever before, that having good teachers in the classroom is not only
necessary but critical (NCTAF, 1996). Several factors have converged to create a
renewed focus on the teacher evaluation process.
147
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
For decades traditional forms of teacher evaluation, including examining
credentials, supervisor (mainly the site principal) formative and summative
observations, and summative check-off sheet ratings have served the K-12 schools
and teaching profession for many years. Within this renewed focus on the teacher
evaluation process are two methods of evaluation that seem to be at the center of
contemporary policy debates: testing of teachers and using students’ test scores to
evaluate teachers (Glass, 2004).
Requiring teacher candidates to take a paper-and-pencil test in the subject
that they teach, or in general teaching methods, has apparently become more
prevalent. In fact, passing an examination is one of the requirements under NCLB
(U.S. Department of Education, 2004). Performance tests of teaching ability are an
alternative to paper-and-pencil tests, but the cost is simply too prohibitive.
Although performance tests are a part of the National Board Certification procedure
for teachers, the approach is so time consuming and expensive that only a few
teachers can afford to take the test. A key question with regard to this testing is
whether the paper-and-pencil test score correlates with or predicts teaching per
formance. Glass (2004) found that validity investigations of teachers’ performance
on the subject matter tests of the National Teacher Examinations (NTE) have failed
to reveal consistent relationship between these tests of subject matter knowledge
and teacher performance in terms of students achievements and supervisors’
ratings. Another significant problem, or rather drawback, with the “paper-and-
pencil-testing,” is that “any such selection test must have what is called a cut score”
(p. 7.7), or the score on the test that separates those who are selected from those
who are rejected. There is no set agreement on the arbitrarily set cut point (Jaeger,
1990, as cited in Glass). The researcher shares in raising a thought-provoking
148
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
question: How can passing a multiple-choice test ensure that one will be a good, or
even minimally competent, teacher?
With an agreement that teaching quality has a significant influence on how
and what students learn, researchers such as William Sanders have concluded that
individual teachers are the single largest factor that adds value to student learning
(Sanders & Horn, 1998). With that in mind, using students’ scores on standardized
achievement tests to evaluate teachers is another innovation in the renewed interest
in accountability and evaluation. In this method of evaluation, the beginning-of-
year to end-of-year gain for students on a standardized achievement test appears to
be attributed only to the efforts and ability of the teacher. In some areas, teachers
are rewarded with merit pay increases for meeting or exceeding a specific targeted
gain or “punished” in various ways for failing to meet the target (Glass, 2004).
The notion that a test score gain can be attributed to a particular teacher’s
efforts and abilities is often referred to as the “value-added” approach to teacher
evaluation. In other words, what value does this particular teacher add to the
learning of the students in his or her class (Glass, 2004)?
The Tennessee Value Added Assessment System (TVAAS) Center at the
University of Tennessee, under the direction of Professor William L. Sanders, is the
originator of a measurement and statistical analysis system “that promises to
measure validly and reliably the value that teachers add to the performance of the
students in their charge” (Glass, 2004, p. 7.3). This approach has been criticized by
some educators. Some feel that attributing achievement gains made by a group of
students solely to the efforts and skill of a single teacher ignores the reality of
schools and classrooms. In an elementary classroom, the controversy is that the
progress that a child makes in the third grade may well have much to do with the
149
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
teaching by that child’s second-grade teacher. Others argue that teacher evaluation
approaches that focus so heavily on standardized testing will make the test and the
entire curriculum synonymous (Glass).
Whatever research has shown using value-added methods for evaluation,
this method has received an enthusiastic reception by politicians and policy makers.
On the other hand, value-added teacher evaluation methods attempting to evaluate
teachers in terms of the standardized achievement test score gains of their students
has drawn heavy criticism from measurement experts and raises serious concerns
about fairness (Glass, 2004).
Purpose o f the Study
The purpose of this study was to determine whether the teacher evaluation
process impacted teacher performance in support of student learning in an urban
school district. This case study examined a high-performing urban elementary
school in the Los Angeles County area of southern California to identify key ele
ments of its success by determining the effects of the evaluation process connecting
teacher evaluation, teacher effectiveness, and student achievement. A qualitative
review of the evaluation process was made with regard to how the evaluation sys
tem was implemented at the school through utilization of the following: interview
ing teachers and administrators, administering a survey to staff, observing best
practices, and examining evaluation documents and district policies. The research
study questions for this qualitative case study were:
1. How is the teacher evaluation process practiced at the school site?
2. How does the teacher evaluation process impact teacher behavior?
150
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3. What factors exist at the school that positively or negatively impact
teacher practice?
4. How does the district’s/school’s leadership influence the teacher
evaluation process?
Methodology
A case study approach was used as the most appropriate way to address the
four research questions through a qualitative tradition of inquiry. Recognizing that
all research methods have limitations, the researcher concurred with Creswell
(1998) that the research questions lent themselves well to a qualitative design
study. The factors being investigated warranted a detailed view and the study took
place within the natural context of the school setting. The researcher was an active
learner telling the story from the participants’ perspective. Due to the descriptive
nature of using a qualitative design and investigating nonquantifiable relationships
between existing variables, this study focused on a mixed methodology and
included a triangulation of data approach in which the same dependent variable was
investigated using multiple procedures gathered from interviews, surveys, direct
observation, and a review of pertinent documents.
Through the use of purposeful sampling, an urban high-achieving Title I
elementary school was selected for this case study. The selection criteria were
identified by the CDE as to those factors that qualified the school for the state Title
I Academic Achievement Award. The CDE (2004) identified the following
criteria as Title I school eligibility requirements:
1. The poverty index equaled at least 40% of the entire school population.
2. The school had 2 consecutive years of assessment data.
151
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3. The school made its AYP in all areas for 2 consecutive years.
4. The school made twice its API target for 2 consecutive years.
5. The school attained or surpassed a current API median score of 729.
Data Collection and Analysis
The researcher utilized a triangulation of the data approach to analyzing the
following instruments: surveys, interviews, observations and examination of a
collection of documents. The data for this study were collected in December 2005
and January 2006, with one additional visitation in February 2006. This study was
part of a larger study conducted at USC. Fifteen similar studies by the USC 2006
doctoral student cohort took place at various high-achieving Title I elementary
school, middle school, and high school sites in the Los Angeles and Orange County
regions of southern California. The development of the instruments was a collect
ive group effort by the 15-member USC doctoral cohort. Each of three subgroups
of 4-6 researchers developed one of the following three instruments that was used
in this research.
The survey was developed by a subgroup of USC researchers in the 2006
doctoral cohort, based on a previous survey developed by a USC 2005 doctoral
cohort. This survey was adapted from Ebmeier’s (2003) teacher survey used to
illustrate a link between teacher efficacy and commitment. The intent of the survey
was to generalize from a sample of staff to the entire school population. There
were 25 forced-response items scored on a Likert-type scale and five open-ended
written response items. At the case study school, 19 of 29 (66%) of the teachers
completed the voluntary survey (appendix C).
152
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The interview protocol was initially created by a six-member subgroup of
the larger 15-member 2006 USC doctoral cohort thematic dissertation group on
teacher evaluation and was subsequently modified by the entire group. The ques
tions were designed to elicit the most in-depth answers to the study’s four research
questions. The interview questions were organized by their corresponding research
question in order to aid in the analysis process. Interview questions were designed
for one-one-one interviews with teachers (appendix D). Ten individual teacher
interviews were conducted. Additional interview questions were created for site
and district administrators (appendix E). One principal interview and one district
office (assistant superintendent of human resources) administrator interview were
conducted.
The observation tool (appendix F) was used to gather data for the case study
at the school site. The observation tool was initially created by a four-member
subgroup of the larger 15 member 2006 USC doctoral cohort thematic dissertation
group on teacher evaluation and was subsequently modified by the entire group.
The observation tool was aligned to Bolman and Deal’s (1997) four frame model of
leadership: Questions were written for the observation tool that fit into the struc
tural, political, symbolic, and human resource frames. Observations were subse
quently analyzed by specific frame.
Data were collected over five stages in December 2005 and January 2006.
All participation was voluntary and confidential. Individual interviews were tape
recorded and lasted approximately 40 minutes. Teachers volunteered for individual
15-minute classroom observations. Administrator interviews lasted approximately
50 minutes. The principal was interviewed at the case study school site and the
assistant superintendent interview was held at the district office.
153
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Results from the survey were coded and the responses were grouped
according to the four research questions, with the findings presented in tables. The
responses to the interview questions were also coded and analyzed to look for
patterns and themes to provide an overview of the phenomenon of the study and its
impacts. Documents were examined and analyzed using the cohort-created docu
ment review protocol (appendix I). Observations were made at the school site over
a 2-week period, both in and out of classrooms. Data were collected on the cohort-
created semistructured observation tool (appendix F). An instrument organization
chart was developed by a subgroup of researchers in the 2006 USC thematic
dissertation cohort of doctoral students to guide the researcher in the analysis of the
data (Table 8).
Summary o f Findings
Through analysis of the data collected, the findings and conclusions for
each of the four research questions were presented. A summary of selected find
ings by research question was reviewed to determine whether the teacher evalua
tion process impacted teacher performance in support of student learning in an
urban elementary school.
Research Question 1
Research question 1 asked, How is teacher evaluation practiced at the
school site? Prior to examining the practice of teacher evaluation at the study site,
the researcher examined the policy and procedures of the district. With regard to
how the evaluation is practiced at the site, the findings revealed practice intent and
practice implementation. The district has a very comprehensive set of evaluation
policies and procedures that follow the tenets of the Stull Bill and the California
154
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Education Code. Teachers with permanent status are formally evaluated every 3
years. There is an alternative for those permanent teachers who have had two
consecutive successful evaluations: a project-based POP. With a seasoned staff,
most teachers opt for the POP and tend to be vague on the practice and procedures
of the traditional evaluation. Teachers strongly stated that the principal observed
classrooms for the purpose of evaluation, but less than half agreed that they dis
cussed instructional strategies used in the classroom with the principal. This did
not seem to connect with the almost 90% response that the principal gives timely
and meaningful feedback regarding teacher observations. Teachers stated that they
would like to raise the frequency of principal visitations. There was only a 37%
agreement that “the administration frequently observes my classroom.” The
teachers expressed positive attitudes toward the teacher evaluation practice at the
site, saying it “keeps me on my toes” and “the principal is not trying to trick us, or
catch us doing something wrong.” Interview responses revealed that teachers
wanted the supervision and appreciated the input and feedback following the
observations. The principal said that, as an administrator, she had not received
quality training as an evaluator from the district. She stated that there should be
more specific training for evaluation based on the CSTP. Findings showed an
strongly positive interaction between staff and administration. Teachers were on-
task and engaged in teaching. Students remained on task when the administrator
entered the classroom. There appeared to be no “down time” and a real sense of
school-wide cohesiveness.
155
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Research Question 2
Research question 2 asked, How does the teacher evaluation process impact
teacher behavior? Teachers at the study site stated that the teacher evaluation
process was an integral part of their professional growth. More than 70% of the
responses by teachers agreed that supervision of instruction and additional support
and feedback improves their instructional practice. Teachers expressed that it was
not the evaluation policy per se that was an integral part of their professional
growth; rather, it was the supervision of instruction by their principal that impacted
their behavior. One teacher commented that being evaluated positively impacted
her behavior and reported that she must look at content standards carefully and that
she might not do it as much if she knew that she was not being evaluated. A num
ber of teachers stated that having teacher evaluations impacted teacher behavior
because it kept them from becoming too comfortable in their practice. The teachers
said that they respected the principal, could comfortably ask the principal for help,
and most of the time she would provide sound advice and sound recommendations.
The principal’s leadership was revealed to be the major factor impacting the teacher
evaluation process. There was clearly a positive relationship between the observa
tion, the survey results, and the interview responses with regard to the relationship
and impact that the principal had with her teaching staff.
Research Question 3
Research question 3 asked, What factors exist at the school that positively
or negatively impact the teacher practice? This research question examined those
factors that impacted teacher practice that were not equated with supervision.
Findings indicated that teachers were well aware of the goals and objectives of the
school. There was 100% agreement that collaboration was a key factor that
156
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
positively impacted teacher practice. The teachers stated that collegial discussion
and team planning were vital to their successful practice. There was a positive
response to professional development. The teachers appreciated that they were not
bombarded with a multitude of new things but that there was an opportunity to try
something new and gain expertise. The teachers highly respected and responded
positively to the principal’s attributes, considering her to be positive, helpful, and
trustworthy. They stated that she listens and is a problem solver. A key positive
factor in impacting teacher practice was that the principal consistently provides
nonevaluative feedback to the teachers. She allows for freedom of ideas and
experimentation. She gives of her time to discuss matters individually with a
teacher, and she is good at guiding people to work together. She is accessible to
the staff and is willing to investigate new practices and implement them. As one
staff member explained, “We have a leader who is the principal, and who is a
teacher.” There is a strong belief at the school that all students can (and will) learn,
and a “whatever it takes” attitude. The researcher noted that teachers consistently
referred to the students at the school as “our” students; there was never a reference
to “those” students. The school climate and culture can be summed in three posi
tive themes that positively impact teacher practice: attitude, relationships, and
leadership.
Research Question 4
Research question 4 asked, How does the district/site leadership influence
the teacher evaluation process? The teachers did not report any difference or
separation between supervision of instruction and the formal evaluation process.
Less than half of the teachers responded that there was a difference. There was
157
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
100% agreement that the teachers were satisfied with the professional competence
and leadership ability of their principal. The teachers reported that they had an
active role in initiating school improvement efforts and that most of the positive
improvement efforts that were implemented were initiated by the influence of the
site administrator. With a strong ILT and lead teachers at every grade level, there
is strong distributed leadership among the staff. Within that influential role,
teacher collaboration, grade-level meetings, staff meetings, and professional
development were reported as teacher support activities that positively impacted
teacher practice and the evaluation process.
Most teachers reported that the district office was not influential. Most of
the teachers stated that the district did not contribute to teacher practice; others
realized that the district leadership contributed to improved teacher practice by
developing goals that the staff was striving to reach. One such goal was the school-
wide and district-wide goal of 90% of the students reading at grade level by 2007.
The site principal was very influential in setting high expectations for everyone and
demanding strong accountability. The school was exemplary at using data and data
analysis to examine what they know, what they do not know, and to drive new
instruction.
Conclusions
Three themes emerged from the analysis and triangulation of the data:
leadership, collaboration, and accountability. The relationship of the teacher and
the principal and the relationship of the teachers to each other seemed to be a key
factor in the school’s success. With a strong and experienced staff, there was not
an attitude by the principal of looking at the evaluation process as a task to see
158
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
which teachers should be cultivated and which teachers should be weeded out;
everyone was there to grow. Each teacher was aware of how his or her practice
impacted student achievement. Data analysis was an embedded practice at the
school site. They used the “name to faces” model so that every child was known
by his or her name and ability level.
A strong lead teacher was examining her class test scores. She had always
been a teacher that had moved her class to the highest levels of student achieve
ment. In the past year her language arts scores were maintained in the proficient
and advanced performance bands but her math test scores were the lowest of the
grade level. Perplexed, she tried to analyze what went wrong. What had she done
differently in the past year from all other years? Among other things, she had
had a baby. Still, her language arts scores were good; it was just the math that
plummeted. In her careful analysis she realized that, during her pregnancy, she did
not feel well after morning recess—when she taught math. The impact of how she
felt related to how the students achieved was a valuable discovery. More than any
professional development training, this teacher learned that quality teacher practice,
what the teacher knows, and the quality of what the teacher does in her teaching
makes the greatest impact on student achievement.
If the elementary school teacher’s classroom were a car and the students
were the passengers, the teacher would be the driver. The students will get only
where the driver takes them. This is a powerful analogy to teacher practice and
student achievement.
Teacher evaluation practice at the study site was valued by the staff as an
improvement guide, whereas the principal regarded it as a compliance report.
159
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Both the teachers and the principal valued fair, formative, ongoing supervision as
a process to impact teacher practice and improve student achievement.
Implications and Recommendations
If there is a disconnect between the evaluation intent and the evaluation
practice, and the principal has commented that the training provided by the district
is not good, then revising the professional development for administrators is
recommended. The implication is that, if the principal as the evaluator were better
trained, the evaluation process would be more meaningful. It would also be
important for the classroom teacher to understand the evaluation process for per
sonal and professional growth. Professional development for the administrator and
for the teacher should focus on the CSTP, since the final evaluation document is
based on those six standards.
The researcher recommends two mandatory training programs (based on
recommendations by Tunison, 1998), one for supervisors and one for teachers.
1. A mandatory training program for supervisors (site administrators)
would reflect the practices, competencies, and attitudes described by the super
vision and evaluation literature as necessary for effective supervision. The program
would address (a) the purpose of supervision and evaluation, (b) specific super
visory techniques (data collection and data analysis), (c) appropriate links between
supervision and staff development (and methods develop those links), (d) practice
with nondirective language, (e) training related to the CSTP, and (t) a professional
reading program to keep principals current on new and emerging practices.
2. A mandatory training program for teachers would include practices,
competencies, and attitudes described by supervision literature as necessary for
160
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
effective supervisory practice. The program would address (a) the purpose of
instructional supervision and evaluation, (b) methods for peer coaching, (c) analy
sis of teaching practice to identify areas of weakness in practices and assist peers to
improve, (d) training related to the CSTP, and (e) a professional reading program to
keep teachers current on new and emerging practices.
161
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
REFERENCES
Acheson, K. A., & Gall, M. D. (1980). Techniques in the clinical supervision o f
teachers: Preservice and inservice applications. New York: Longman.
American Federation of Teachers. (2001). Making standards matter 2001. Re
trieved July 5, 2005, from http://65.110.81.56/pubs-
reports/downloads/teachers/msm2001 .pdf
Association of California School Administrators, Task Force on Student Per
formance and School Accountability. (1997). Standards-based account
ability: A design for student success (special report). Retrieved March 1,
2006, from http://www.acsa.org/publications/accountability.cfin
Avery, L. (2001). Teacher evaluation: To enhance professional practice. Roeper
Review, 23(4), 240.
Bridges, E. M. (1992). The incompetent teacher. Philadelphia: Falmer.
Bolman, L., & Deal, T. (1997). Reframing organizations: Artistic choices and
leadership (2nd ed.). San Francisco: Jossey-Bass.
Bolman, L., & Deal, T. (2003). Reframing organizations: Artistry, choice, and
leadership (3rd ed.). San Francisco: Jossey-Bass.
Burney, D., Canada, G. Corcoran, T. & Erskine, R. (2002). School communities
that work. Providence, RI: Annenberg Institute for School Reform at
Brown University.
California Department of Education. (2004). School adequate yearly progress
report. Retrieved July 5, 2005, from http://www.ayp.cde.ca.gov/reports
California Commission on Teacher Credentialing. (1997). California standards
for the teaching profession: A description ofprofessional practice for
California teachers. Sacramento: Author.
California Commission on Teaching Credentialing and California Department of
Education. (2001). California formative assessment and support system for
teachers: Resources for professional practice. Sacramento: Author.
California Education Code. (2005a). Article 4. High-achieving/improving schools
program. Sections 52056-52057. Sacramento: Office of the Secretary of
State.
California Education Code. (2005b). Article 11. Evaluation and assessment o f
performance o f certificated employees. Sections 44660-44665. Sacramento:
Office of the Secretary of State.
162
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Carnegie Forum on Education and the Economy. (1986). A nation prepared:
Teachers for the 21st century. New York: Carnegie Corporation of New
York.
Carroll, D. H. (1997, January). A comparison o f clinical supervision and evalua
tion. Paper presented at the Annual Meeting of the Southwest Educational
Research Association, Austin, TX.
Cochran-Smith, M. (2003). Teaching quality matters. Journal o f Teacher Educa
tion, 54(2), 21-32.
Cochran-Smith, M., & Zeichner, K. M. (Eds.). (2005). Studying teacher educa
tion: The report o f the AERA panel on research and teacher education.
Mahwah, NJ: Erlbaum.
Creswell, J. (1998). Qualitative inquiry and research design: Choosing among
five traditions. Thousand Oaks, CA: Sage.
Creswell, J. (2003). Research design: Qualitative, quantitative, and mixed meth
ods approaches (2nd ed.). Thousand Oaks, CA: Sage.
Danielson, C. (2001). New trends in teacher evaluation. Educational Leadership,
2, 14.
Danielson, C., & McGreal, T. L. (2000). Teacher evaluation: To enhance profes
sional practice. Alexandria, VA: Association for Supervision and Cur
riculum Development.
Daresh, J. C. (1989). Supervision as a proactive process. New York: Longman.
Darling-Hammond, L. (1998). Teachers and teaching: Testing policy hypotheses
from a national commission report. Educational Researcher, 27(1), 5-15.
Darling-Hammond, L., & Ball, D. L. (1998). Teaching for high standards: What
policymakers need to know and be able to do. Washington, DC: National
Commission on Teaching and America’s Future and Consortium for Policy
Research in Education.
Datnow, A., & Schneider, N. (Eds.). (2004). The age o f accountability and the
need for data-driven decision making. Los Angeles: University of South
ern California, Rossier School of Education.
Delpit, L. (1995). Other people’ s children: Cultural conflict in the classroom.
New York: Free Press.
Ebmeier, H. (2003). How supervision influences teacher efficacy and commit
ment: An investigation of a path model. Journal o f Curriculum and Super
vision, 18(2), 110-141.
163
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Education Trust. (2003). Telling the whole truth (or not) about highly qualified
teachers. Retrieved July 22, 2005, from http://www.edtrust.org
Ellett, C., & Teddlie, C. (2003). Teacher evaluation, teacher effectiveness, and
school effectiveness: Perspectives from the USA. Journal o f Personnel
Evaluation in Education, 27(1), 101-128.
Feldman, S. (2001). White House Conference on Preparing Tomorrow’ s Teach
ers. Retrieved July 22, 2005, from http://www.ed.gov/admin/tchrqual/
leam/preparingteacheresconference/feldman
Finn, C. E., Jr., & Wilcox, D. D. (2000, January 13). Teachers should be graded
on how well their students are learning. Los Angeles Times, p. 9.
Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An intro
duction (7th ed.). San Francisco: Allyn and Bacon.
Genck, F. H. (1991). Renewing America’ s progress: A positive solution to school
reform. New York: Praeger.
Glass, G. V. (2004). Teacher evaluation policy brief. Tempe: Arizona State
University, Education Policy Research Unit.
Glickman, C., Gordon, S., & Ross-Gordon, J. (2005). The basic guide to supervi
sion and instructional leadership. Boston: Pearson.
Goldrick, L. (2002). Improving teacher evaluation to improve teaching quality:
Issue brief Education Policy Studies Division. Washington, DC: National
Governors Association Center for Best Practices.
Good, T. L., & Mulryan, C. (1990). Teacher ratings: A call for teacher control a
and self-evaluation. In J. Millman & L. Darling-Hammond (Eds.), The new
handbook o f teacher evaluation: Assessing elementary and secondary
school teachers (pp. 191-215). Newbury Park, CA: Sage.
Haefele, D. L. (1993). Evaluating teachers: A call for change. Journal o f Per
sonnel Evaluation in Education, 7(1), 21-31.
Haycock, K. (1998). Good teaching matters: How well-qualified teachers can
close the gap. Thinking K-16, 3(2), 1-2. Retrieved July 2, 2005, from
http ://www2. edtrust.org/NR
Haycock, K. (2004). The real value of teachers: If good teachers matter, why
don’t we act like it? Thinking K-16, <§(1), 1-2. Retrieved July 2, 2005, from
http://www2.edtrust.org/NR
Hernandez, R. F. (1972). A recommended system o f evaluation for use in Califor
nia secondary school bands with reference to the Stull Bill (AB 293). Un
published master’s thesis, University of Southern California.
164
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Hirsch, E., Koppich, J., & Knapp, M. (2001). Revisiting what states are doing to
improve the quality o f teaching: An update on patterns and trends. Seattle:
University of Washington Center for the Study of Teaching and Policy.
Jennings, J. (2000). Title I: Its legislative history and its promise. Phi Delta Kap-
pan, 81, 516-522.
Johnson, S. M. (1990). Teachers at work: Achieving success in our schools. New
York: Basic Books.
Kaplan, L. S., & Owings, W. A. (2003). No child left behind: The politics of
teacher quality. Phi Delta Kappan, 84(9), 687.
Lewis, A. C. (1982). Evaluating educational personnel. Arlington, VA: Ameri
can Association of School Administrators.
Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions,
and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Hand
book o f qualitative research (2nd ed., pp. 163-168). Thousand Oaks, CA:
Sage.
MassPartners for Public Schools. (2002). Teaching matters: Strengthening
teacher evaluation in Massachusetts. Monson, MA: Author.
McLaughlin, M. W. (1990). Embracing contraries: Implementing and sustaining
teacher evaluation. In J. Millman & L. Darling-Hammond (Eds.), The new
handbook o f teacher evaluation (pp. 403-415). Newbury Park, CA: Sage.
Monk, D. H. (1994). Subject area preparation of secondary mathematics and sci
ence teachers and student achievement. Economics o f Education Review,
13, 125-145.
National Board for Professional Teaching Standards. (2002). What teachers
should know and be able to do. Arlington, VA. www.nbpts.org
National Commission on Excellence in Education. (1983). A nation at risk.
Washington, DC: U.S. Government Printing Office.
National Commission on Teaching and America’s Future. (1996). What matters
most: Teaching for America’ s future. New York: Author.
National Commission on Teaching and America’s Future. (2003). “No dream
denied”: A pledge to America’ s children. Washington, DC: Author.
National Research Council. (2003). Assessment in support o f instruction and
learning: Bridging the gap between large-scale and classroom assessment.
Washington, DC: National Academy Press.
No Child Left Behind Act, 20 U.S.C. Sections 6301 et seq. U.S.C. (2002).
165
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Odden, A. (2001). Rewarding expertise. Retrieved March 1, 2006, from
http://www.educationnext.org/2001 sp/16odden.html
Peterson, K. D. (2000). Teacher evaluation: A comprehensive guide to new di
rections and practices. Thousand Oaks, CA: Corwin Press.
Peterson, K. D., & Chenoweth, T. (1992). School teachers’ control and involve
ment in their own evaluation. Journal o f Personnel Evaluation in Educa
tion, 6, 177-189.
Quality counts. (2002, January 10). Education Week, pp. 78-79.
Ribas, W. B. (2000). Ascending the ELPS to excellence in your district’s teacher
evaluation. Phi Delta Kappan, 81, 585.
Sanders, W., & Horn, S. (1998). Research findings from the Tennessee Value-
Added Assessment System (TVAAS) database: Implications for educa
tional evaluation and research. Journal o f Personnel Evaluation in Educa
tion, 12, 247-256.
Sanders, W., & Rivers, J. (1996). Cumulative and residual effects o f teachers on
future student academic achievement. Knoxville: University of Tennessee
Value-Added Research and Assessment Center.
Saskatchewan School Boards Association. (2005). Profession-based instructional
supervision: A toolkit o f resources. Saskatoon, Saskatchewan, Canada:
Author.
Saskatchewan School Trustees Association. (1995). To cultivate or to weed: An
assessment o f teacher evaluation policies and practices in rural Saskatche
wan school divisions. Saskatchewan, Canada: Author.
Sawa, R. (1995). Teacher evaluation policies and practices: A summary o f a
thesis by Rick Sawa. Retrieved March 1, 2006, from http://www.ssta.sk.ca/
research/instruction/95-04.htm
Schmoker, M. (1999). Results: The key to continuous school improvement (2nd
ed.). Alexandria, VA: ASCD.
Schmoker, M. (2003). First things first: Demystifying data analysis. Educational
Leadership, 60(5), 22-44.
Scriven, M. (1981). Summative teacher evaluation. In J. Millman & L. Darling-
Hammond (Eds.), The new handbook o f teacher evaluation: Assessing ele
mentary and secondary school teachers (pp. 244-271). Newbury Park, CA:
Sage.
Scriven, M. (1987). Validity in personnel evaluation. Journal o f Personnel
Evaluation in Education, 1, 9-24.
166
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Sergiovanni, T. J. (1991). The principals hip: A reflective practice perspective
(2nd ed.). Boston: Allyn & Bacon.
Stanley, S., & Popham, W. (Eds.). (1988). Teacher evaluation: Six prescriptions
for success Los Angeles. Arlington, VA: Association for Supervision and
Curriculum Development.
Stodolsky, S. S. (1984). Teacher evaluation: The limits of looking. Educational
Researcher, 13(9), 11-18.
Travers, R. M. W. (1981). Criteria of good teaching. In J. Millman (Ed.), Hand
book o f teacher evaluation (pp. 14-22). Beverly Hills, CA: Sage.
Tucker, M. S., & Codding, J. B. (1998). Standards for our schools: How to set
them, measure them, and reach them. San Francisco: Jossey-Bass.
Tucker, P., & Stronge, J. (2005). Linking teacher evaluation and student learning.
Alexandria, VA: Association for Supervision and Curriculum Develop
ment.
Tunison, S. (1998). Instructional supervisory policies and practices. Saskatche
wan, Canada: SSTA Research Centre.
U.S. Congress. (2001). Public Law 107-110: The No Child Left Behind Act.
Washington, DC: U.S. Government Printing Office.
U.S. Department of Education. (1998). Promising practices: New ways to im
prove teacher quality. Retrieved March 1, 2006, from
http://www.ed.gov/pubs/PromPractice/index.html
U.S. Department of Education, Office of the Undersecretary. (2004). No Child
Left Behind: A toolkit for teachers. Washington, DC: Author.
Valentine, J. W. (1992). Principles and practices for effective teacher evaluation.
Needham Heights, MA: Allyn and Bacon.
Van Der Linde, C. (1998). Clinical supervision in teacher evaluation: A pivotal
factor in the quality management of education. Education, 119, 328-334.
Wasley, P. A., & McDiarmid, G.W. (2004). Connecting the assessment o f new
teachers to student learning and to teacher preparation. Seattle: Univer
sity of Washington College of Education and the National Commission on
Teaching and America’s Future.
Wise, A. G., Darling-Hammond, L., McLaughlin, M. W., & Berstein, H. T.
(1984). Teacher evaluation: A study o f effective practices. Santa Monica,
CA: RAND.
167
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Wolf, R. (1973). How teachers feel toward evaluation. In E. House (Ed.), School
evaluation: The politics and process (pp. 156-168). Berkeley, C A:
McCutchan.
Yin, R. K. (1989). Case study research: Design and methods. Newbury Park,
CA: Sage.
168
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX A
California Standards for the Teaching Profession
STANDARD ONE: ENGAGING & SUPPORTING ALL STUDENTS IN LEARNING
1.1 Connecting students’ prior knowledge, life experience, and interests with learning goals
1.2 Using a variety o f instructional strategies and resources to respond to students’ diverse needs
1.3 Facilitating learning experiences that promote autonomy, interaction, and choice
1.4 Engaging students in problem solving, critical thinking, and other activities that make subject
matter meaningful
1.5 Promoting self-directed, reflective learning for all students
STANDARD TWO: CREATING & MAINTAINING EFFECTIVE ENVIRONMENTS FOR
STUDENT LEARNING
2.1 Creating a physical environment that engages all students
2.2 Establishing a climate that promotes fairness and respect
2.3 Promoting social development and group responsibility
2.4 Establishing and maintaining standards for student behavior
2.5 Planning and implementing classroom procedures and routines that support student learning
2.6 Using instructional time effectively
STANDARD THREE: UNDERSTANDING & ORGANIZING SUBJECT MATTER FOR
STUDENT LEARNING
3.1 Demonstrating knowledge of subject matter content and student development
3.2 Organizing curriculum to support student understanding of subject matter
3.3 Interrelating ideas and information within and across subject matter areas
3.4 Developing student understanding through instructional strategies that are appropriate to the
subject matter
3.5 Using materials, resources, and technologies to make subject matter accessible to students
STANDARD FOUR: PLANNING INSTRUCTION & DESIGNING LEARNING EXPERIENCES
FOR ALL STUDENTS
4.1 Drawing on and valuing students’ backgrounds, interests, and developmental learning needs
4.2 Establishing and articulating goals for student learning
4.3 Developing and sequencing instructional activities and materials for student learning
4.4 Designing short-term and long-term plans to foster student learning
4.5 Modifying instructional plans to adjust for student needs
STANDARD FIVE: ASSESSING STUDENT T F.ARNTNG
5.1 Establishing and communicating learning goals for all students
5.2 Collecting and using multiple sources of information to assess student learning
5.3 Involving and guiding all students in assessing their own learning
5.4 Using the results o f assessments to guide instruction
5.5 Communicating with students, families, and other audiences about student progress
STANDARD SIX: DEVELOPING AS A PROFESSIONAL EDUCATOR
6.1 Reflecting on teaching practice and planning professional development
6.2 Establishing professional goals and pursuing opportunities to grow professionally
6.3 Working with communities to improve professional practice
6.4 Working with families to improve professional practice
6.5 Working with colleagues to improve professional practice
Source: California Standards for the Teaching Profession: A Description o f Pro
fessional Practice for California Teachers, by California Commission on Teacher
Credentialing, 1997, Sacramento: Author.
169
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX B
California Education Code Sections 44660-44665:
44660. It is the intent o f the Legislature that governing boards establish a uniform system of
evaluation and assessment o f the performance of all certificated personnel within each school dis
trict of the state, including schools conducted or maintained by county superintendents o f education.
The system shall involve the development and adoption by each school district o f objective evalua
tion and assessment guidelines which may, at the discretion of the governing board, be uniform
throughout the district or, for compelling reasons, be individually developed for territories or
schools within the district, provided that all certificated personnel o f the district shall be subject to a
system o f evaluation and assessment adopted pursuant to this article. This article does not apply to
certificated personnel who are employed on an hourly basis in adult education classes.
44661. In the development and adoption of guidelines and procedures pursuant to this article, the
governing board shall avail itself of the advice of the certificated instructional personnel in the dis
trict’s organization of certificated personnel; provided, however, that the development and adoption
o f guidelines pursuant to this article shall also be subject to the provisions of Article 1 (commencing
with Section 7100) of Chapter 2 o f Part 5 of Division 1 of Title 1.
44661.5. When developing and adopting objective evaluation and assessment guidelines pursuant
to Section 44660, a school district may, by mutual agreement between the exclusive representative
o f the certificated employees of the school district and the governing board of the school district,
include any objective standards from the National Board for Professional Teaching Standards or any
objective standards from the California Standards for the Teaching Profession if the standards to be
included are consistent with this article. If the certificated employees o f the school district do not
have an exclusive representative, the school district may adopt objective evaluation and assessment
guidelines consistent with this section.
44662. (a) The governing board of each school district shall establish standards of expected pupil
achievement at each grade level in each area of study.
(b) The governing board of each school district shall evaluate and assess certificated employee per
formance as it reasonably relates to:
(1) The progress of pupils toward the standards established pursuant to subdivision (a)
and, if applicable, the state adopted academic content standards as measured by state
adopted criterion referenced assessments.
(2) The instructional techniques and strategies used by the employee.
(3) The employee’s adherence to curricular objectives.
(4) The establishment and maintenance of a suitable learning environment, within the
scope of the employee’s responsibilities.
(c) The governing board of each school district shall establish and define job responsibilities for
certificated noninstructional personnel, including, but not limited to, supervisory and administrative
personnel, whose responsibilities cannot be evaluated appropriately under the provisions of subdivi
sion (b) and shall evaluate and assess the performance o f those noninstructional certificated employ
ees as it reasonably relates to the fulfillment o f those responsibilities.
(d) Results o f an employee’s participation in the Peer Assistance and Review Program for Teachers
established by Article 4.5 (commencing with Section 44500) shall be made available as part o f the
evaluation conducted pursuant to this section.
(e) The evaluation and assessment of certificated employee performance pursuant to this section
shall not include the use o f publishers’ norms established by standardized tests.
(f) Nothing in this section shall be construed as in any way limiting the authority o f school district
governing boards to develop and adopt additional evaluation and assessment guidelines or criteria.
44663. (a) Evaluation and assessment made pursuant to this article shall be reduced to writing and a
copy thereof shall be transmitted to the certificated employee not later than 30 days before the last
170
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
school day scheduled on the school calendar adopted by the governing board for the school year in
which the evaluation takes place. The certificated employee shall have the right to initiate a written
reaction or response to the evaluation. This response shall become a permanent attachment to the
employee’s personnel file. Before the last school day scheduled on the school calendar adopted by
the governing board for the school year, a meeting shall be held between the certificated employee
and the evaluator to discuss the evaluation.
(b) In the case of a certificated noninstructional employee, who is employed on a 12-month basis,
the evaluation and assessment made pursuant to this article shall be reduced to writing and a copy
thereof shall be transmitted to the certificated employee no later than June 30 of the year in which
the evaluation and assessment is made. A certificated noninstructional employee, who is employed
on a 12-month basis shall have the right to initiate a written reaction or response to the evaluation.
This response shall become a permanent attachment to the employee’s personnel file. Before July
30 of the year in which the evaluation and assessment takes place, a meeting shall be held between
the certificated employee and the evaluator to discuss the evaluation and assessment.
44664. (a) Evaluation and assessment o f the performance o f each certificated employee shall be
made on a continuing basis as follows:
(1) At least once each school year for probationary personnel.
(2) At least every other year for personnel with permanent status.
(3) At least every five years for personnel with permanent status who have been employed at least
10 years with the school district, are highly qualified, as defined in 20 U.S.C. Sec. 7801, and whose
previous evaluation rated the employee as meeting or exceeding standards, if the evaluator and
certificated employee being evaluated agree. The certificated employee or the evaluator may with
draw consent at any time.
(b) The evaluation shall include recommendations, if necessary, as to areas of improvement in the
performance of the employee. If an employee is not performing his or her duties in a satisfactory
manner according to the standards prescribed by the governing board, the employing authority shall
notify the employee in writing o f that fact and describe the unsatisfactory performance. The em
ploying authority shall thereafter confer with the employee making specific recommendations as to
areas o f improvement in the employee’s performance and endeavor to assist the employee in his or
her performance. If any permanent certificated employee has received an unsatisfactory evaluation,
the employing authority shall annually evaluate the employee until the employee achieves a positive
evaluation or is separated from the district.
(c) Any evaluation performed pursuant to this article which contains an unsatisfactory rating o f an
employee’s performance in the area of teaching methods or instruction may include the requirement
that the certificated employee shall, as determined necessary by the employing authority, participate
in a program designed to improve appropriate areas o f the employee’s performance and to further
pupil achievement and the instructional objectives of the employing authority. If a district partici
pates in the Peer Assistance and Review Program for Teachers established pursuant to Article 4.5
(commencing with Section 44500), any certificated employee who receives an unsatisfactory rating
on an evaluation performed pursuant to this section shall participate in the Peer Assistance and
Review Program for Teachers.
(d) Hourly and temporary hourly certificated employees, other than those employed in adult educa
tion classes who are excluded by the provisions o f Section 44660, and substitute teachers may be
excluded from the provisions of this section at the discretion o f the governing board.
44665. For purposes of this article, “employing authority” means the superintendent o f the school
district in which the employee is employed, or his designee, or in the case of a district which has no
superintendent, a school principal or other person designated by the governing board.
171
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX C
Staff Survey
A research group from the University of Southern California is studying highly successful,
urban schools. Your school was selected because of the high CST growth you have
achieved over the last two years with a sizeable at-risk population. We are very interested
in gathering your perceptions as a member of the staff with regard to how you have
achieved such remarkable student success. We are also interested in the role that the
teacher evaluation and supervision process may have played in that success. Attached is a
short survey containing a series of statements and open-ended questions for you to answer
based on your knowledge/experience at your school. Your assistance is crucial to the
project; we thank you for your voluntary participation in this important research. (All
aspects of this study are completely confidential!
DIRECTIONS
Please answer all questions as completely and accurately as possible.
Circle the most appropriate response.
Background Information:
1) Number o f years as a teacher: (circle one) 0-1 yrs. 2-5 yrs. 6-10 yrs. 11-19 yrs.
20 yrs.+
2) Number of years at this location: (circle one) 0-1 yrs. 2-5 yrs. 6-10 yrs. 11-19 yrs.
20 yrs.+
3) Participant in the Beginning Teacher Support and Assessment (BTSA)? Yes No
4) If yes, what was your role? Support Provider Beginning Teacher Both
5) What is your highest academic degree? (check one)
fk Bachelor’s degree (B.A., B.S., etc.)
A Bachelors degree plus additional units
Master’s degree (M.A., M.S, etc.)
Doctoral degree (Ph.D., or Ed.D., J.D. etc)
172
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Survey: Teacher Evaluation and Supervision (Teacher Version)
Please be as honest as possible. Your responses are completely confidential
and will only be used to produce findings on issues related to this study on
teacher evaluation and supervision. Please rate each of the statements below on
a four-point Likert Scale as follows:
0 = Strongly Disagree 1 = Disagree 2 = Agree 3 =
Strongly Agree
Policy: Teachers ’ knowledge and perceptions of official documents regarding the teacher
evaluation process.
SD D A SA
1) I am aware of and understand the district’s policy regarding teacher evaluation.
0 1 2 3
2) I agree with the goals and objectives of my district’s policy on teacher evaluation.
0 1 2 3
Teacher Evaluation: The extent to which the administration actively participated in the
formal evaluation process, as mandated by the California Education Code, which may
have included observations, data collection, feedback, goal setting, and improvement
strategies.
SD I) A SA
3) My administrator frequently observes my classroom for the purpose of evaluation.
0 1 2 3
4) My administrator and I often discuss the instructional strategies I use in my classroom.
0 1 2 3
5) I view my administrator’s implementation of the teacher evaluation policies as an integral
part of my professional growth.
0 1 2 3
6) When my administrator visits my classroom, he/she looks for things which we agreed upon
at a pre-conference.
0 1 2 3
7) I receive timely and meaningful feedback regarding my teaching observations.
0 1 2 3
8) I am confident in my administrator’s ability to evaluate my instructional practice.
0 1 2 3
173
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9) There are alternative evaluation opportunities (e.g. PAR, portfolio, etc.) available at my
site.
0 1 2 3
Ongoing Teacher Supervision: The extent to which the administration actively partici
pated in a supervision process through observations, data collection, feedback, goal
setting, and improvement strategies.
SD D A SA
10) The administration frequently observes my classroom.
0 1 2 3
11) Other instructional support staff (e.g. coaches, counselors, BTSA/PAR support providers,
etc.) frequently observe my classroom.
0 1 2 3
12) I see the administration’s supervision o f instruction as non-evaluative and separate from the
formal evaluation process.
0 1 2 3
13) I believe that my administration’s supervision o f instruction improves my instructional
practice.
0 1 2 3
14) I believe that other instructional support staffs feedback regarding my instruction im
proves my instructional practice.
0 1 2 3
School Efforts: Teacher’ s perception of school-based procedures and activities, not in
cluding direct supervision, which may have led to school improvement.
SD D A SA
15) I have multiple opportunities throughout the year to participate in professional develop
ment activities.
0 1 2 3
16) Teachers are encouraged to collaborate on instructional matters on a regular basis.
0 1 2 3
174
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
17) I have a clear understanding of the professional development goals for my school.
0 1 2 3
18) Teachers have an active role in developing professional development goals and objectives.
0 1 2 3
School Culture: Teacher’ s perception o f “the way we do things around here. ”
SD D A SA
19) I am comfortable going to my school administrators for support.
0 1 2 3
20) I am aware o f the goals and objectives of this school.
0 1 2 3
SD D A SA
21) The goals and objectives of this school have contributed to our school’s improvement.
0 1 2 3
22) I am satisfied with the professional competence and leadership ability o f the administration
0 1 2 3.
23) Faculty discussions regarding curriculum and instruction impact our school’s ability to
improve.
0 1 2 3
24) Teachers have an active role in initiating efforts towards school improvement.
0 1 2 3
25) The majority of school improvement efforts at this school have been initiated by the district
and/or site administration.
0 1 2 3
Please provide feedback to each question below.
1) Describe which experiences and/or activities have the greatest impact on
your teaching practice and professional development.
175
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2) Describe which experiences and/or activities have the greatest impact on
student achievement at this school.
3) Describe which types o f interactions with administration impact your teach
ing practices, either positively or negatively?
4) Describe which types o f interactions with colleagues and other staff
members impact your teaching practices, either positively or negatively?
5) What do you feel are the most important factors that have contributed to
student learning at this school?
By a 2006 doctoral student cohort, Rossier School of Education, University of
Southern California, based on a document developed by a 2005 doctoral student
cohort.
176
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX D
Interview Questions for Certificated Staff
Research Questions Interview Questions
Research Ouestion One:
How is the teacher evalua
tion process practiced at
the school site?
Describe the evaluation process at the school
site?
• Suggested Follow Up: H o w involved
are you in developing your evaluation
for the school year?
• Suggested Follow Up: Are vou fa
miliar with all the documents you need
to complete for your evaluation?
• Suggested Follow Up: How do vou
know what your principal is looking
for during your classroom observa
tion?
• Suggested Follow Up: How does vour
principal share with you what s/he ob
served during the observation?
• Suggested Follow Up: How many ob
servations do you have in your formal
evaluation year? What is the duration
o f each observation?
• Suggested Follow Up: In what wavs
are your contractual guidelines from
the collective bargaining agreement
integrated into your evaluation
process?
What modifications would you recommend for the
teacher evaluation process that would improve
your practice?
177
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Research Ouestion Two:
How does the teacher
evaluation process impact
teacher behavior?
How does the evaluation process make a differ
ence in your teaching practices?
How does the recommendation affect your
teaching practice?
What modifications would you recommend for the
teacher evaluation process that would improve
your teaching practices?
Research Ouestion Three: Why is this school successful?
• Sussested Follow U d: To what extent
does your school culture impact your
teaching practice?
• Sussested Follow U d : To what extent
does professional development impact
your teaching practice?
• Sussested Follow U d : To what extent
does collaboration and collegiality im
pact your teaching practice?
• Sussested Follow U d : To what extent
does leadership impact your teaching
practices?
What school program and/or strategies have been
implemented to improve teacher practice?
How are these factors connected to the teacher
evaluation?
What factors exist at the
school that positively or
negatively impact the
teacher practice? (How, if
at all, is teacher evaluation
connected to teacher prac
tice?)
Research Ouestion Four: How does the district office leadership contribute
to improved teacher practices?
How does the school’ s leadership contribute to
improved teacher practices?
Who do you view as school leaders and how do
they impact teacher practices?
How does the district/site
leadership influence the
teacher evaluation process?
By a 2006 doctoral student cohort, Rossier School of Education, University of
Southern California.
178
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX E
Interview Questions for Site and District Administrators
Principals and Assistant Principals:
1. How is it decided who you will evaluate each year?
2. Have you received any training on how to observe/evaluate teachers?
3. Do you feel you have the necessary knowledge to properly evaluate teachers?
What would you like more training on?
4. How do you follow-up to see if your suggestions to teachers are acted upon?
5. Are you satisfied that your school’s evaluation system accurately identifies teach
ers’ abilities, from poor to excellent? What happens to teachers who are identified
as weak, average or excellent?
6. Do you attribute your school’s recent success in any way to your evaluation prac
tices?
7. What changes would you make?
District Office Administrators:
1. What types of training does the district provide its school administrators in the
area of teacher evaluation?
2. Do you feel that the site administrators do a good job evaluating teachers?
Why/why not?
3. Is the number of teachers who have received an unsatisfactory evaluation over
the last few years an accurate reflection of the number of poor teachers in the
district?
4. Are you satisfied that the district’s evaluation system accurately identifies
teachers’ abilities, from poor to excellent? What happens to teachers who are
identified as weak, average or excellent?
5. Do you attribute this school’s recent success in any way to your district’s
evaluation practices?
6. What changes would you make?
By a 2006 doctoral student cohort, Rossier School of Education, University of
Southern California.
179
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX F
Campus Observation Tool
Date:___________ Tim e:_____________ Place:
# frame Yes No
1 P The campus is clean and well-kept.
2 P There are display cases throughout the school showing awards,
student work, and important events.
3 P There is a sense of safety and security on campus.
4 S There is positive interaction between the staff and administration.
5 SY There is positive school spirit displayed by staff and students
alike.
6 S Staff members are familiar with the vision and can speak to the
goals of the vision.
7 s There is a local reward/recognition program in place for students
and staff.
8 s Describe the leadership style o f the administration:
Y = Distributed, N = Top-Down.
9 SY There is positive interaction between the staff, students, parents,
and community.
10 SY There is positive interaction between the staff and administration.
11 SY The administration is visible in all areas of the school.
12 SY The office is warm and friendly when customers enter.
13 SY There are rituals and events throughout the year to mark positive
learning, social, and environmental happenings.
14 H Teachers are engaged in school activities
15 H The administration has positive interactions with students.
16 H There is a friendly and positive environment in staff meetings.
17 P Students remain on task when administrators enter the classroom.
18 S Substitutes are considered effective instructors.
19 S Content standards for the lesson are visible.
20 s The agenda/ objectives are visible to the students.
21 s Student work is displayed in the classroom.
22 s Guided practice was observed during the lesson.
23 s A variety of learning activities (direct instruction, lecture, group
work, projects, etc...) are utilized in the classroom.
24 s High levels of questioning are evident in observed lessons.
25 s Assessments demonstrate multiple measures to evaluate student
work.
By a 2006 doctoral student cohort, Rossier School of Education, University of
Southern California.
180
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX G
(This is a copy of the original letter which is on file with the IRB. This copy contains the
pseudonym school and district, Catson Elementary School in the Rio Hondo Unified
School District).
October 29, 2005
Dr. Wynne R. Waugaman
Chair
University Park Institutional Review Board
3601 Watt W ay-G FS 306
Los Angeles CA 90089-1695
RE: Dr. Stuart Gothold, Dissertation Chairman,
University of Southern California, Rossier School of Education
Roberta (Bobbie) Lansman (doctoral candidate, investigator)
“A Case Study of Teacher Evaluation and Supervision at a High Achieving
Urban Elementary School”
Dear Dr. Waugaman:
This letter is to convey that Ewe have reviewed the proposed research study being
conducted by Roberta (Bobbie) Lansman intended to conduct a voluntary and con
fidential certificated staff survey, conduct approximately ten certificated staff vol
untary and confidential one hour after-school interviews, and conduct voluntary
classroom observations over a period of two weeks (at the school’s convenience) at
Catson Elementary School (Jeanette Cueva, Principal), and find “A Case Study of
Teacher Evaluation and Supervision at a High Achieving Urban elementary
School” acceptable. Ewe give permission for the above investigator to conduct
research at this site. Ewe will review, abide by, and comply with the procedures
approved by the University of Southern California’s University Park IRB. If you
have any questions regarding this permission letter, please contact the undersigned
at [INSERT TELEPHONE NUMBER OR CONTACT INFORMATION],
Sincerely,
Assistant Superintendent of Human Resources
Rio Hondo Unified School District
181
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX H
( ; Sample o f the original approved IRB application on file)
University of Southern California
Rossier School of Education
Waite Phillips Hall
University of Southern California
Los Angeles, CA 90089-4038
INFORMED CONSENT FOR NON-MEDICAL RESEARCH
CONSENT TO PARTICIPA TE IN RESEARCH
A Closer Look At T eacher Evaluation
You are asked to participate in a research study conducted by Roberta (Bobbie)
Lansman and Stuart Gothold, Ed.D. from the Rossier School of Education at the
University of Southern California. You were selected as a possible participant since
you are a “highly qualified teacher” as defined by No Child Left Behind and
employed at a high performing school that serves needy students.
The teaching staff from Catson Elementary School was selected to participate in
survey, interviews and observations. The expectation is to conduct a total of ten
teacher interviews and one administrator at the school site and one administrator at
the district office. The subjects will be randomly selected from those who indicate
their interest. Your participation is voluntary. You may read the information below,
and ask questions about anything you do not understand, before deciding whether
or not to participate
PURPOSE OF THE STUDY
To determine how the teacher evaluation process, and other practices, impacts
teacher performance in support of student learning in urban schools which are im
proving in student achievement. The focus is to identify effective evaluation prac
tices that have an impact on student learning.
182
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
PROCEDURES
If you volunteer to participate in this study, I would ask you to do the following
things:
Interview: One-time, 45-minutes, 20 questions about the teacher evaluation
process, preferably audio-taped. The interviews will be conducted during the site
visit at a time and place convenient to the subject. Two sample questions are: How
involved are you in developing your evaluation for the school year? and Does the
evaluation process make a difference in your teaching practices?
Classroom Observation: One-time, 15 minutes during instructional time. No inter
actions with teacher or students. I will be observing classroom culture.
POTENTIAL RISKS AND DISCOMFORTS
There are no anticipated risks associated with this research study. During the inter
view, you may feel uncomfortable answering some questions. Any discomforts
you may experience with questions may be managed by not answering these ques
tions. Other discomforts in this process may be time away from work and the dis
comfort of being audio-taped and observed while teaching.
POTENTIAL BENEFITS TO SUBJECTS AND/OR TO SOCIETY
You will not benefit from your participation in this research study.
The potential benefits to society may lead to better practice of the teacher evalua
tion process in schools, which in turn may benefit students at those schools.
PAYMENT/COMPENSATION FOR THE PARTICIPATION
There will be no payment for participating in this study.
CONFIDENTIALITY
Any information that is obtained in connection with this study and that can be
identified with you will remain confidential.
183
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
When the results of the research are published or discussed in conferences, no in
formation will be included that reveals your identity. Pseudonyms will be used for
each interviewed participant. If audio-tape recordings of you will be used for edu
cational purposes, your identity will be protected or disguised by the researcher.
Notes and audio-tapes will be kept separately to avoid positive identification by
unauthorized personnel. Audio-tapes will be secured and stored for three years in
the home of the investigator. As a participant, you have the right to review and/or
edit the tapes in the presence of the researcher.
Classroom observations will also be conducted, but they will be coded with num
bers to maintain anonymity. Numbers 1-30, for example, will be assigned to each
participant and the notes identifying each classroom will be kept separately in order
to maintain anonymity. Notes will be kept in the researcher’s home for three years
before disposing of all relevant information. Again, when the results of the re
search are published, no information will be included that reveals your identity.
Personal information, research data, and related records will be coded, stored for
three years, and secured by the researcher to prevent access by unauthorized per
sonnel. All audio-tapes used in interviews during this study will be stored in the
researcher’s home and destroyed three years after the completion of the study.
PARTICIPATION AND WITHDRAWAL
You can choose whether to be in this study or not. If you volunteer to be in this
study, you may withdraw at any time without consequences of any kind. You may
also refuse to answer any question and still remain in the study. Furthermore, if you
wish to participate and not be audio-taped, observed or have your students’ data
analyzed, you may do so. However, the investigator may withdraw you from this
research if circumstances arise which warrant doing so.
IDENTIFICATION OF INVESTIGATORS
If you have any questions or concerns about the research, please feel free to contact
the investigator, Roberta (Bobbie) Lansman (XXX) XXX-XXXX, email at
lansman@usc.edu or at bobbiel@orangeusd.kl2.ca.us
RIGHTS OF RESEARCH SUBJECTS
You may withdraw your consent at any time and discontinue participation without
penalty. You are not waiving any legal claims, rights or remedies because of your
participation in this research study. If you have questions regarding your rights as a
research subject, contact the University Park IRB, Office of the Vice Provost for
Research, Grace Ford Salvatori Building, Room 306, Los Angeles, CA 90089-
1695, (213) 821-5272 or unirb@usc.edu.
184
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
SIGNATURE OF RESEARCH SUBJECT
I understand the procedures described above, and I understand fully the rights of a
potential subject in a research study involving people as subjects. My questions
have been answered to my satisfaction, and I agree to participate in this study. I
have been given a copy of this form.
□ I agree to be audio-taped during a 45-minute interview.
□ I do not agree to be audio-taped during the interview
□ I agree to be observed once for 15 minutes during class time.
□ I do not agree to be observed during class time.
Name of Subject (Print)
Signature of Subject Date
SIGNATURE OF INVESTIGATOR
I have explained the research to the subject and answered all of his/her questions. I
believe that he/she understands the information described in this document and
freely consents to participate.
Name of Investigator
Signature of Investigator Date (must be the same as
subject’s)
185
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX I
Document Review (Researcher Records and Reflections)
Criteria for Review of documents including:
• Certificated Formal Evaluation Documents/Process/Manuals
• Collective Bargaining Agreement (Certificated) Relevant Article/s
• Board Policy/Administrative Regulations
• Site Plan
• Professional development information regarding the Certificated Formal
Evaluation Process
Guiding Questions
Question Yes No Not
Applicable
Comments Document
Reviewed
1. Does the form meet the Stull
Bill criteria?
2. Does the formal evaluation
• Incorporate the Pro
fessional Standards
for the Teaching Pro
fession?
• Include Content Stan
dards?
3. Does the district have a “how
to” manual on the formal
evaluation process?
• If so, does it include a
rubric for teacher per
formance, i.e. satisfac
tory, unsatisfactory,
needs improvement?
• Is it presented in profes
sional trainings in
meetings only by the
district or is it presented
with the association
present?
186
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4. Does the certificated collec
tive bargaining agreement pro
vide:
• Explicit timelines for
the formal evaluation
process?
• Options for use of AB
954, the Goldberg Bill?
• Does it define not satis
factory performance?
5. Does the formal evaluation
process provide the adminis
trator and the teacher the op
tion to select at least one goal
of their choice?
6. Does the formal evaluation
process incorporate a com
ponent to provide for coach-
professional growth oppor
tunities, and/or next steps?
7. Does the evaluators observa
tion template include a Likert
scale or some form of a rubric
to rate the evaluatee’s
performance?
8. Is there a pre and/or post pro
cess used with regard to
classroom observations as a
described part of the formal
evaluation process?
9. Is the formal evaluation
document designed to allow
the evaluator to individualize
comments, address particular
needs or recommendations?
10. Is the formal evaluation
process:
• Primarily summative?
• Primarily formative?
• Some combination of
summative and formative?
By a 2006 doctoral student cohort, School of Education, University of Southern
California.
187
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

A closer look at the impact of teacher evaluation: A case study in a high performing California elementary school

PDF

California teacher evaluation and improved teacher practice in a high -performing urban school

PDF

Beating the odds: Does the formal teacher evaluation process make the difference in teacher performance?

PDF

A case study of teacher evaluation and supervision at a high performing urban elementary school

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Los Coyotes High School District

PDF

A case study: An analysis of the adequacy of one school district's model of data use to raise student achievement

PDF

A case study of instructional supervision, including teacher evaluation, and the impact on teacher practice

PDF

A substitute teacher preservice staff development program: A case study of the Los Angeles County Office of Education

PDF

An analysis of student -level resources at a California comprehensive high school

PDF

Differential characteristics of beginning teachers

PDF

A case study of the California teacher evaluation system and its impact upon teacher practice in an alternative education high -performing urban high school

PDF

An evaluation of the current level of Korean parent involvement at Third Street Elementary School

PDF

A comparative study of the role of the principal in conventional public schools and in charter schools

PDF

Districtwide instructional improvement: A case study of an elementary school in the Beach Promenade Unified School District

PDF

Factors of a successful transition into a new superintendency

PDF

Curriculum policy and educational practices: A study of primary classroom music education in Kern County, California

PDF

Elementary teacher perceptions of school violence: Safe school elements and responsibility.

PDF

A case study of social promotion and retention policies, strategies, and programs

PDF

An investigation of early literacy achievement with an emphasis on urban public schools

PDF

Administrative evaluations and their impact on school leadership in public and private secondary schools: A comparison study of governance structure

Asset Metadata

Creator Lansman, Roberta Rosalin (author)

Core Title A case study of teacher evaluation and supervision at a high -achieving urban elementary school

Contributor Digitized by ProQuest (provenance)

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, administration,education, elementary,Education, Teacher Training,OAI-PMH Harvest

Language English

Advisor Gothold, Stuart (committee chair), French, Robert (committee member), Marsh, David D. (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-622439

Unique identifier UC11336430

Identifier 3233791.pdf (filename),usctheses-c16-622439 (legacy record id)

Legacy Identifier 3233791.pdf

Dmrecord 622439

Document Type Dissertation

Rights Lansman, Roberta Rosalin

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA