University of Southern California Dissertations and Theses

Teacher perception of the implementation of the educator effectiveness system

(USC Thesis Other)

Teacher perception of the implementation of the educator effectiveness system

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

1

TEACHER PERCEPTION OF THE IMPLEMENTATION OF THE EDUCATOR
EFFECTIVENESS SYSTEM

by

Jana Une Hai Fukada

_____________________________________________________________________

A Dissertation Presented to the
FACTULY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

December 2015

Copyright 2015 Jana Une Hai Fukada

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

2
ACKNOWLEDGEMENTS
I would like to acknowledge my parents, Ed and Patti Fukada, for supporting me through
this journey. Thank you for sacrificing so much for me over the years so that I could receive the
best education possible and always encouraging me to pursue any and all aspirations that I have.
Thank you also to my big brother, Gavin, for setting the pace and being someone I could always
aspire to be like. I would also like to thank Brad and Gizmo for believing in me and giving me
the time and space to get my studying done. This dissertation would not have been possible
without the cooperation of the Complex Area Superintendents, Principals, Educational Officers,
and teachers, so to them, I am truly grateful. A huge thank you must be given to my USC cohort.
The hours of studying and writing have paid off and I will always hold the time we spent
together near to my heart. Finally, I would like to thank my dissertation committee, Dom, Larry,
and Monique, for your guidance, patience, and encouragement.

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

3
TABLE OF CONTENTS
Acknowledgements 2
List of Tables 5
List of Figures 6
Abstract 7
Chapter 1: Overview of the Study 8
Background of the Problem 9
Statement of the Problem 15
Purpose of the Study 16
Importance of the Study 17
Limitations and Delimitations 17
Definition of Terms 18
Organization of the Dissertation 21
Chapter 2: Literature Review 22
Human Capital 22
Teacher Evaluation 33
Teachers’ Unions 42
Teacher Evaluation in Hawaii 44
Conceptual Framework 56
Summary 58
Chapter 3: Methodology 59
Research Questions 59
Research Design 59
Sample and Population 60
Instrumentation 61
Data Collection 64
Data Analysis 64
Summary 65
Chapter 4: Results 66
Research Question 1 Findings 68
Research Question 2 Findings 83
Summary 88

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

4
Chapter 5: Discussion 90
Summary of Findings 91
Implications for Practice 91
Implications for Research 93
Limitations 94
Conclusion 95
References 97
Appendices 112
Appendix A: Teacher Focus Group Interview Protocol 112
Appendix B: EES EO Interview Protocol 113
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

5
LIST OF TABLES
Table 1. Five Duties of Teachers as Determined by PEP-T with Accompanying 45
Description
Table 2. Three Point Rating Scale Used to Rate Teachers on the Five Duties as 45
Determined by PEP-T
Table 3. Expectations and Responsibilities of Classroom Teachers and Administrators 50
as Defined by the EES
Table 4. Four Levels of Evaluation 57
Table 5. Focus Group Questions Aligned to Research Questions 62
Table 6. Interview Questions Aligned to Research Questions 63
Table 7. Summary of Initial Educator Effectiveness System Training by Complex 70
Area and School
Table 8. Summary of Ongoing Support Received by Teachers by Complex Area 72
and School

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

6
LIST OF FIGURES
Figure 1. Human Capital Management framework 23
Figure 2. An overview of Hawaii’s Educator Effectiveness System framework 48

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

7
ABSTRACT
This study applies the Four Levels of Evaluation from the performance evaluation literature to
understand teacher’s perceptions of the implementation of an evaluation system. The purpose of
this study was to investigate, analyze, and describe elementary school teacher’s perceptions of
the implementation of the Educator Effectiveness System within the Hawaii Department of
Education during the 2013-2014 school year. Using focus groups and interviews, the experiences
and perceptions of nine teachers and three Educational Officers were explored. The experiences
of the teachers and Educational Officers were analyzed to identify emergent themes. Findings
from this study indicate that teachers did not feel adequately prepared to implement the Educator
Effectiveness System during the 2013-2014 school year and received various types of
implementation trainings and ongoing support during the implementation year. This study
provides insight for further consideration as the State of Hawaii continues to improve the
Educator Effectiveness System.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

8
CHAPTER 1
OVERVIEW OF THE STUDY
In a global economy where the most valuable skill you can sell is your knowledge, a
good education is no longer just a pathway to opportunity – it is a pre-requisite. Right
now, three-quarters of the fastest-growing occupations require more than a high school
diploma. And yet, just over half of our citizens have that level of education. We have one
of the highest high school dropout rates of any industrialized nation. And half of the
students who begin college never finish. This is a prescription for economic decline,
because we know the countries that out-teach us today will out-compete us tomorrow.
That is why it will be the goal of this administration to ensure that every child has access
to a complete and competitive education – from the day they are born to the day they
begin a career. (Obama, 2009)
President Obama’s words speak to the urgency of improving America’s education system
in response to the challenges that globalization presents. Research (Hickman & Olney, 2011;
Cheng, 2004; Spring, 2008; Mittelstrass, 2001) defines globalization as a long-term
multidimensional process that involves the free transfer of raw materials, knowledge,
technology, social values, labor, and promoting the development of organizations and societies
across different countries and communities. Mittelstrass (2001) asserts, “In a global economy it
is not only economic structures that change but also social structure. Institutional isolation
dissolves. This development affects both education and research” (p. 394). As the economic and
social structures change, the type of education children across the nation receive must also
change.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

9
Improving educational achievement is a national priority that starts with obtaining,
supporting, and retaining effective teachers. As the most important school based factor affecting
student achievement (Darling-Hammond, 1999) it is essential that schools be filled with high-
quality educators. Obtaining effective teachers is the result of purposeful hiring and training that
enables teachers to prepare their students to succeed in a global economy. Research finds that a
vital tool for improving teacher effectiveness is the teacher evaluation (Danielson & McGreal,
2001; Howard & McColskey, 2001). Consequently, it is important that teacher evaluation
systems are implemented effectively as a means of ensuring an accurate and fair measure of
teacher performance and effectiveness.
This study will investigate, analyze, and describe the teachers’ perspective of the
implementation of the Educator Effectiveness System (EES) in Hawaii. The findings of this
study could provide information and insight on how teacher evaluation systems can be most
effectively and efficiently implemented to ensure student success and improve teacher practice.
Background of the Problem
Teacher quality is the single most important school based influence of student
achievement (Darling-Hammond, 1999; Pretorius, 2012). However, U.S. students are falling
behind their peers in countries around the world highlighting the urgency of improving teacher
quality across the nation. Students in the United States made little progress on recent global
achievement exams and slipped deeper in the international rankings as the competition abroad
continues to progress.
In 2012 the Program for International Student Assessment (PISA) revealed that U.S
students continue to perform poorly on international tests. According to the National Center for
Education Statistics (NCES, n.d.-a), PISA is an international assessment that measures the
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

10
reading, mathematics, and scientific literacy of 15-year-old students. Unlike other international
assessments such as the Trends in International Mathematics and Science Study (TIMSS) and the
National Assessment of Educational Progress (NAEP) which measure mastery of students’
specific knowledge, skills, and concepts, the content of the PISA focuses on assessing the
application of skills and knowledge to everyday situations. Compared to the 2000 PISA results,
15-year-old students in the U.S. have struggled to show significant growth despite reform efforts
such as more frequent testing, publicizing student proficiency rates, and opening public schools
to competition from charter and public schools. In math, the U.S. scored below the PISA mean
and ranks 26
th
out of the 34 Organisation for Economic Co-operation and Development (OECD)
countries on par with countries such as Hungary, Russia, and the Slovak Republic. In science,
the U.S. came in two points above the PISA mean and ranked 21
st
which was the same position
as Italy, Latvia, and Portugal. The U.S. posted its best score in reading literacy with a rank of
17
th
in the world on equal footing with the United Kingdom, France, and Austria (NCES, 2013a).
According to the PISA report (NCES, 2013a), one in four U.S. students did not reach the PISA
baseline level two of mathematics proficiency. At level two students begin to demonstrate the
skills necessary to participate effectively and productively in life. Additionally, the report also
indicated that even the top students in the United States are two years of formal schooling behind
students in Shanghai-China, the number one country in reading, mathematics, and scientific
literacy.
Confirming the modest performance of U.S. students are the scores reported by the 2013
Nation’s Report Card which shows National Assessment of Educational Progress (NAEP) math
and reading assessment for 376,000 fourth-graders and 341,000 eight graders in public and
private schools in all 50 states including the District of Columbia and Department of Defense
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

11
schools (Nation’s Report Card, 2013). According to the report, the average mathematics scores
for fourth- and eighth-graders in 2013 were one point higher than in 2011, and 28 and 22 points
higher respectively in comparison to the first assessment year in 1990. Since 2007, the average
fourth-grade score improved two points while the average eight-grade score improved 12 points.
The average reading scores for eighth-graders were two points higher in 2013 than in 2011, but
there was no significant change in the average fourth-grade score since 2011 on average across
the nation. Since 2007, the average fourth-grade reading score improved one point while the
average eight-grade reading score improved five points. Furthermore, the Trends in Academic
Progress study published by the U.S. Department of Education’s National Center for Education
Statistics in 2013 (NCES, 2013b) reported that in math and science today’s high school seniors
have made no significant gains since the early 1970s. The aforementioned results presented by
The Nation’s Report Card and the Trends in Academic Progress study highlight the sluggish
progress students across the nation are making in spite of education reforms highlighting the
importance of improving teacher instruction.
Despite slow national progress, Hawaii’s fourth- and eighth-graders continue to progress
in mathematics and reading as measured by the NAEP. In math, Hawaii’s fourth-graders showed
a four point improvement scoring about the national average for the first time. Hawaii’s eight-
graders improved three points but fell short of the national average by four points. In reading, the
average Hawaii fourth-grade score improved one point while the average eighth-grade score
improved three points. Although gains were made by the fourth- and eighth-graders both scores
did not meet the national average. The progress that Hawaii’s students have made is
encouraging, but the progress that it must continue to make to keep up with students across the
nation call for improved instruction by all teachers (Nation’s Report Card, 2013).
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

12
Strong growth and performance on national and international assessments are only part of
the puzzle. Today’s children will soon enter a global economy in which they will be required to
transform their knowledge into ability to compete and succeed in a rather uncertain global
workforce (Mittelstrass, 2001). Advancements in technology have contributed to rapid changes
in the global workforce. However, today slightly more than three-quarters of students graduate
high school (NCES, 2012a) and in some of America’s largest cities fewer than half of all
students graduate (Dillon, 2009). Moreover, for students who do graduate many leave high
school without adequate numeracy and literacy skills (NCES, 2012a). Jerald (2009) argues that
although it is unknown what skills students will need to compete in the job market of the 21
st

century, strong math and reading skills are essential since they form the foundation for complex
communication and expert thinking. Unfortunately, students across the nation and Hawaii
struggle to demonstrate strong aptitude in either of these areas.
Although a longstanding mission for America’s public schools has been to provide a high
quality free education to America’s youth, it is laden with inequalities and inadequacies. The
1983 publication of A Nation at Risk (National Commission on Excellence in Education, 1983)
highlighted and spearheaded an ongoing investigation of America’s education system and the
need for higher quality teaching and learning experiences in lieu of technological advancements.
Compared to twenty-five years ago when 95% of jobs required low skills such as following basic
procedures, today low-skills jobs constitute only 10% of the entire economy (Darling-Hammond
& Barron, 2008). Technological advances have contributed to a rise in demand for skilled
workers to operate and create innovative products to keep up with changes. Darling-Hammond
and Barron (2008) argue, “Education today must focus on helping students learn how to learn, so
they can manage the demands of changing information, technologies, jobs, and social
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

13
conditions” (p. 3). Autor, Levy, and Murnane (2003) assert that although computers reduce the
demand for workers to complete routine tasks, there is an increase in demand for those whom are
able to complete analytic or nonroutine tasks. This highlights the importance of a quality
education and its ability to prepare students with higher-level thinking and problem solving
skills.
In response to rapid globalization, increased competition from students around the world,
growing enrollment, and diversity in the public school system the climate of public education
across the nation has changed. According to the National Center for Education Statistics (NCES,
2013c), from school years 2010-2011 through 2021-2022, public school enrollment is projected
to increase by 7% from 49.5 to 53.1 million students. Additionally, the racial/ethnic enrollment
in public schools has and will continue to change. From fall 2000 through fall 2010 the number
of White students enrolled in public schools decreased from 28.9 million to 25.9 million, which
is in contrast to growing Hispanic enrollment of 7.7 million to 11.4 million. Enrollment changes
are accompanied by an increase in the percentage of school-aged children living in poverty.
According to the National Center for Educational Statistics, in 2011 approximately 21% of
school-aged children in the United States were in families living in poverty (NCES, 2015a).
Darling-Hammond and Post (2000) assert that “Few Americans realize that the U.S educational
system is one of the most unequal in the industrialized world, and students routinely received
dramatically different learning opportunities based on their social status” (p. 127). The
challenges the public education system faces are deep and complicated and require that
significant changes be made to what students learn and how teachers teach to prepare all students
to succeed in this rapidly changing world.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

14
In response to inconsistent academic standards across the nation the state school chiefs
and governors that comprised the Council of Chief State School Officers (CCSSO) and the
National Governors Association (NGA) Center organized a movement to develop the Common
Core State Standards (CCSS). The CCSS were launched in 2009 as a set of high-quality
academic standards in mathematics and English language arts/literacy (ELA). The CCSS provide
schools across the country with clearly defined markers of what students should be able to do at
each grade level within their K-12 education to adequately prepare them to be college and career
ready (Common Core State Standards Initiative [CCSSI], 2014). According to the Common Core
State Standards Initiative (2014) the standards are internationally benchmarked and backed by
evidence showing that students’ mastery of them leads to preparedness for higher education and
the work force. U.S. Secretary of Education Arne Duncan (2010) says:
The [common core standards] will help teachers, students and parents know what is
needed for students to succeed in college and careers, and will enable states, school
districts and teachers to more effectively collaborate to accelerate learning and close
achievement gaps nationwide. (n.p.)
Although the CCSS clearly outlines what needs to be taught, it does not provide educators with
guidelines on how the standards should be taught. The National Governors Association (NGO)
and Council of Chief State School Officers (CSSO) assert that the standards are not a curriculum,
rather a set of shared expectations to help students succeed. They go on to say that it is up to
teachers, principals, and superintendents to decide how the standards are to be met (CCSSI,
2014). It is for this reason that classrooms be filled with high quality effective teachers to ensure
that the standards are met. Moreover, as the number one contributor to student academic
achievement in school (Darling-Hammond, 1999; Pretorius, 2012) it is vital that teachers provide
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

15
high quality instruction that engage, challenge, and inspire innovation and problem solving
through the implementation of the CCSS. To ensure that students are receiving intentional high
quality instruction, an effective teacher evaluation program must be in place that will objectively
assess teachers while providing them with adequate feedback to improve their practice.
The evidence is clear; teacher instruction must improve across the nation to prepare
America’s youth to be successful contributors to tomorrow’s workforce (Mittelstrass, 2001;
Dillon, 2009; Jerald, 2009). In addition to new standards (CCSSI, 2014) and a changing student
population, today’s teachers are faced with the task of preparing students for jobs that have not
yet been created. Teachers are no longer preparing students to perform specific tasks but rather
teaching them how to think critically, be responsive, and problem solve. As the role of teachers
transforms, instructional practices will need to change and is essential that teachers receive
adequate support and feedback on their instruction to ensure effective instruction is provided.
Statement of the Problem
According to the PISA (NCES, 2013a), students in the U.S. lag behind their peers in
acquiring key knowledge and skills that are essential for full participation in modern societies.
The 2012 Program for International Student Assessment (PISA) revealed that that U.S. students
rank 36
th
overall of 66 countries in spite of having the highest per student spending (NCES,
2013a). According to the PISA students across America are not performing as well as their peers
throughout the world. As the number one school expense and influence of student outcomes,
ensuring teacher quality is extremely important. To ensure teacher quality, teacher evaluations
can be used to identify effective teachers and those that require additional support to ensure all
students receive a quality education. As states across the nation move towards adopting
comprehensive teacher evaluation systems in response to increased accountability and urgency to
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

16
improve student achievement, effective implementation will influence the overarching goal of
improving student achievement.
Purpose of the Study
The purpose of this study was to investigate, analyze, and describe the evaluation of
human capital in the Hawaii Department of Education. This study focused on the perceptions and
experiences of eight elementary school teachers from seven schools and seven district personnel
from seven complex areas throughout Oahu. Through semi-structured interviews and focus
groups, classroom teachers’ and district personnel’s perceptions regarding, training, support, and
implementation of the EES was explored. An evaluation of the implementation of the Educator
Effectiveness Evaluation System in Hawaii was completed using the framework of Clark and
Estes (2008), an adaptation of Kirkpatrick’s (1998) basic four-level model.
The research questions for this study are as follows:
1. What are the perceptions of teachers regarding the Educational Effectiveness System
as it was implemented?
a. How were teachers prepared for the implementation of the Educator
Effectiveness System?
b. How were teachers supported through the implementation of the Educator
Effectiveness System?
c. How do teachers perceive the EES has influenced their instruction?
2. Which factors contributed to the perceptions of teachers regarding the Educator
Effectiveness System as it was implemented?
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

17
Importance of the Study
It is widely understood that higher quality teacher instruction will improve student
achievement but performance management is a relatively new concept in public education
systems (Odden, 2011a). Sanders and Rivers (1996) assert that as teacher effectiveness increases
lower achieving students are the first to benefit. Furthermore, identifying ineffective teachers is
critical, as it has been found that students who are assigned to several ineffective teachers in a
row have significantly lower achievement gains in comparison to student who are assigned to
highly effective teachers (Sanders & Rivers, 1996). As performance gaps widen and the skills
students need to be successful intensify, it is crucial that teachers receive a fair and applicable
evaluation to ensure all students receive a high quality education from an effective teacher. The
findings of this study of the implementation of the EES in Hawaii public elementary schools can
be used to inform future initial and follow-up EES trainings and support sessions for Hawaii
Department of Education teachers. Additionally, the findings can be used by other states to
ensure successful implementation of a teacher evaluation system.
Limitations and Delimitations
A limitation and threat of external validity of this study was that it only investigated the
experiences of classroom teachers in elementary school teachers on the island of Oahu. Although
the components of the Educator Effectiveness System are the same for all classroom teachers,
the experiences of middle and high school teachers should also be investigated. Moreover, the
experience of teachers on other islands and non-classroom teachers would also provide
meaningful data that should be considered. An additional limitation is that this study did not take
into account the experiences of school administrators such as principals or vice principals. A
fourth limitation is that this study only focused on the experiences of teachers. It does not look at
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

18
the impact that the EES has made on teacher instruction and student achievement. A
delimitation of this study is that it was designed to gather data from a small sampling of Hawaii
Department of Education teachers.
Definition of Terms
To clarify key terms that appear throughout this study, the following definitions are
provided:
A Blueprint for Reform: The Reauthorization of the Elementary and Secondary Act is a
plan the government created for the Department of Education that builds upon reforms made in
response to the American Recovery and Reinvestment Act of 2009 (U.S. Department of
Education [USDOE], 2010a).
American Recovery and Reinvestment Act of 2009 was signed into law by President
Barack Obama in an effort to jumpstart the economy, save and create millions of jobs, and start
addressing long-standing challenges such as education, healthcare, and clean energy technologies
(The White House, n.d.-a).
Common Core State Standards (CCSS) refers to the set of academic standards in
mathematics and English language arts/literacy (ELA) that define what students (K-12) should
be able to do and know at the end of each grade (CCSSI, 2014).
Council of Chief State School Officers (CCSSO) refers to a nonpartisan, nationwide,
nonprofit organization of public officials who lead departments of elementary and secondary
education in the state, the District of Columbia, the Department of Defense Education Activity,
and the five U.S. extra-state jurisdictions (Council of Chief State School Officers, 2014).
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

19
Educator Effectiveness System (EES) refers to the teacher performance evaluation system
within the Hawaii Department of Education (State of Hawaii Department of Education [HIDOE],
n.d.-a).
Hawaii State Department of Education is Hawaii’s single school district. It is comprised
of 256 public schools, which are distributed into 42 complexes. Complexes are geographically
grouped into 15 complex areas, which are distributed into seven geographic school districts.
National Assessment of Educational Progress (NAEP) refers to the largest national
computer based assessment of what American students know and can do in mathematics,
reading, science, writing, the arts, civics, economics, geography, U.S. history (NCES, 2015b).
National Center for Education Statistics (NCES) refers to the principal federal group for
collecting and analyzing data related to education (NCES, n.d.-b).
National Governors Association (NGA) refers to the bipartisan organization of the
nation’s governors (National Governors Association, 2011).
Nation’s Report Card is a report that informs the public about the academic achievement
of elementary and secondary students in the United States on the National Assessment of
Educational Progress (NCES, 2012b).
No Child Left Behind Act of 2001 refers to the reauthorization of the Elementary and
Secondary Education Act signed into law by President Bush in 2002 (NCLB, 2002).
Organisation for Economic Co-operation and Development (OECD) is group comprised
of representatives from 34 countries that work with governments to foster prosperity and fight
poverty through economic growth and financial stability (Organisation for Economic Co-
operation [OECD], n.d.).
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

20
Professional Evaluation Program for Teachers (PEP-T) was the Hawaii State
Department of Educations teacher evaluation. The Department in collaboration with the Hawaii
State Teachers Association created the PEP-T (HIDOE, 2011).
Program for Student Assessment (PISA) refers to the international assessment that
measures 15-year-old students’ reading, mathematics, and science literacy. PISA has been
administered every three years since 2000.
Race to the Top (RttT) refers to the competitive grant program funded by the American
Recovery and Reinvestment Act of 2009. The program was designed to encourage and rewards
States for creating conditions for education reform, increasing student achievement, closing
achievement gaps, improving high school graduation rates, ensuring students are prepared to
succeed in college and careers, and implementing plans for core education reform (USDOE,
2009).
Smarter Balanced Assessment Consortium (SBAC) is a state-led consortium that was
awarded funding from the U.S. Department of Education in 2010 to develop an assessment
aligned to the Common Core State Standards to be implemented in school year 2014-2015
(Smarter Balanced Assessment Consortium, 2012).
Trends in International Mathematics and Science Study (TIMSS) collects data on the
mathematical and science achievement of U.S. students. TIMSS data is collected every four
years and is used to compare U.S. student performance to that of students in other countries
(NCES, n.d.-c).
Classroom Assessment Scoring System (CLASS) is an observational instrument developed
at the University of Virginia to assess teacher quality in PK-12 classrooms (Center for Advanced
Study of Teaching and Learning, 2014).
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

21
Educational Officers (EO) exist within the HIDOE at the school, district, and state levels.
Principals and vice principals are known as school-level educational officers. At the district and
state level, educational officers include program specialists, office and branch directors, and
superintendents.
Organization of the Dissertation
This dissertation is organized into five chapters. Chapter 1 provides an overview of the
study. This chapter provides background information on student achievement data in the United
States and its relevance to teacher evaluations. Within this chapter the purpose of the study is
presented as well as the importance of the study along with limitations and delimitations. The
end of Chapter 1 contains a list of terms and their definitions. Chapter 2 includes a synthesis of
the literature on Human Capital Management and each component within it. This chapter also
includes a synthesis of the literatures on the history, the components, the role of teacher unions,
and the implementation of teacher evaluation systems. This chapter closes with a summary of
teacher evaluation practices in Hawaii with regards to the Educator Effectiveness system.
Chapter 3 outlines the methodology of the study and includes information on the sample and
population, instrumentation, and data collection procedures. This chapter concludes with an
analysis of the data organized by research question. The fourth chapter contains the results of the
study and insights about what the results mean. The final chapter includes a discussion of the
findings and implications for practice and future research.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

22
CHAPTER 2
LITERATURE REVIEW
The purpose of this literature review is to provide background on Human Capital
Management and more specifically the evaluation component as it pertains to Hawaii
Department of Education teachers. The progression of information will be reviews of the
research on Human Capital Management and its individual components, national trends and
reforms of teacher evaluation, the history and present situation of the evaluation system of
Hawaii Department of Education, the history of Teacher Unions, and the role that Hawaii’s
Teacher Union played the adoption of the current evaluation system.
Human Capital
Human capital refers to the knowledge, ideas, skills, and health of individuals (Becker,
2002; Eide & Showalter, 2010; Coff, 2002; Flamholtz & Lacey, 1981) and is, according to
Becker (2002), the most important form of capital in modern economies. Constituting over 70%
of the total capital invested in the United States (Becker, 2002) and one of the largest
investments organizations make (Schiemann, 2006), human capital management is vital to the
economic success of individuals and organizations.
1

Human Capital and Organizational Performance
Maximizing the impact and efficiency of human capital in organizations is one of the
cornerstones of industrial and organizational psychology inquiry (Crook, Todd, Combs, Woehr,
& Ketchen, 2011). Research shows that investing in human capital can yield positive individual
as well as organizational benefits. Compensation for employees and managers is strongly related

1
The description of Human Capital Management was developed collaboratively by, Julie Elting, Jana
Fukada, Richard Gonsalves, Maureen Ikeda, Roger Kiyomura, Keoni Kuoha and Melia Tauvela.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

23
to the education and experience they possess. Superior human capital leads to sustainable
performance advantages because talent tends to expand over time.
Human Capital Management (HCM) is a complex but critical system that can enhance an
organization's success (Joshi, 2012; Becker & Gerhart, 1996; Huselid, 1995). The literature
defines HCM as a systematic approach to attracting, selecting, engaging, developing, and
retaining the highest ability individuals (Joshi, 2012; Becker & Gerhart, 1996; Huselid, 1995;
Phillips & Roper, 2009; McMahan & Wright, 1992). Phillips and Roper (2009) created a
framework upon which this study of HCM will be based (Figure 1).

Figure 1. Human Capital Management framework. Adapted from Phillips and Roper (2009)

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

24
The HCM framework in Figure 1 consists of five key elements; attracting, selecting,
engaging, developing, and retaining employees. At the center of the framework are the
organizations values and competencies, which are aligned to the five key elements and the
continuous process of strategizing, executing, and evaluating. Research emphasizes that all
decisions made by the organization must be aligned to one another and linked to the strategic
needs of the organization (McMahan & Wright, 1992).
Human Capital Theory
Formal economic theories of developing human capital as a common good date back
approximately 50 years although awareness of positive outcomes from investing in people is far
from new (Becker, 1962; Eide & Showalter, 2010; Schultz, 1961). The modern usage of human
capital took root in the economic growth and societal changes that followed World War II.
Contemporary economists such as Schultz (1961) and Becker (1962) observed that increased
levels of education and training were creating a knowledge-based economy in which individuals
could better control their destinies. When individuals acquired knowledge, skills, and attitudes as
a form of human capital, it could not be separated from them as was possible with other forms of
capital. Ultimately, the success of these individuals would impact not only their own lives, but
have far-reaching effects on society as a whole (Becker, 1962; Eide & Showalter, 2010; Schultz,
1961).
The benefit to an organization from investment in their workers is also a focus of human
capital theory. Ample research provides evidence that the knowledge, skills, and abilities (KSAs)
possessed by employees at a variety of levels contribute to organizational success (Blair, 2011;
Byerly, 2012; Crook et al., 2011; Guest, 2011; Lepak, Takeuchi, & Swart, 2011; Zimmerman,
Gavrilova-Aquilar, & Cullum, 2013). Boxall and Purcell (2011) adds that the configuration of
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

25
human capital, how managers allocate resources and recognize potential talent worth investing
in, is critical. Additionally, ongoing evaluation of human capital is a critical component to the
success of an organization. Clark and Estes (2008) argue that it essential when seeking to
improve or close performance disparities.
Human Capital Management Model
Phillips and Roper (2009) proposed a comprehensive model that depicts the
interrelationship of five stages of human capital management (HCM): attracting, selecting,
engaging, developing, and retaining employees (Figure 1). Central to the model is the alignment
of employee and organizational values supported by adequate KSAs and ongoing evaluation and
feedback (Bauer & Erdogan, 2012; Phillips & Roper, 2009). Linking the five stages to
organizational values and competencies are employee opportunities for performance evaluations
and learning. Life-long learning, a catch-phrase of 21
st
century educational jargon, is integral to
HCM. Each HCM stage is connected to the next one by organizational strategic planning and the
actual implementation of those plans. Evaluating the effectiveness of the plan can help determine
if goals of the stage were met (Phillips & Roper, 2009).
Five Key Elements of the Human Capital Management Model
Attracting and recruiting. In effort to streamline the human capital development cycle,
recruitment of employees has emerged in recent years as its own area of study. As a primary
means of acquiring human resources, recruitment strategies, policies, and practices serve an
important role in the selective value of recruitment (Ployhart, 2012). Recruitment strategies are a
critical component of every organization’s overall framework for selecting and hiring qualified
individuals (Ployhart & Kim, 2014). KSAs are often used to define human capital resources and
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

26
allow organizations to identify the personnel that will allow them to achieve their goals (Ployhart
& Moliterno, 2011).
Selecting and hiring. Labor, like all organizational resources, is highly dependent on
supply and demand of human talent (Mueller & Baum, 2011). Huang and Cappelli (2010) affirm
that a central research question in economics is how to ensure that employees do not avoid their
workplace responsibilities. Research pertaining to this issue focuses on initiatives that provide
incentives to employees to follow through with responsibilities and to act in the best interest of
the organization such as performance evaluations and promotions (Huang & Cappelli, 2010). To
that end, selection of potential employees within organization is paramount in the success and
productivity of any organization.
Engagement and development of employees. The third and fourth stages, engagement
and development, can be viewed on a continuum. The employer and employee share the
responsibilities of these stages. Engagement is early organizational socialization, the process in
which new employees are acclimated to the institutional structure and job role (Bauer &
Erdogan, 2012). Engagement should naturally flow into the development stage. Developing
human capital, the continuing investment in employees, has been shown to increase retention of
effective employees (Bauer & Erdogan, 2012; Becker, 1962; Blair, 2011; Byerly, 2012; Crook et
al., 2011). Offering opportunities for professional development or ongoing training of employees
produces increased general and firm specific knowledge and skills (Bauer & Erdogan, 2012;
Crook et al., 2011). Committed employees will use this knowledge to advance the organizational
mission.
In a nationwide survey of workers and their preferences conducted by Dychtwald,
Erickson, and Morison (2006), “the opportunity to learn and grow and try new things” ranked
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

27
third most important of basic elements. Personal and professional development are valued by
employees at all job levels (Phillips & Roper, 2009). Research indicates that it is beneficial to the
organization to train employees and have supervisors and mentors dedicated to discussing and
evaluating their performance (Wagner & Harter, 2006).
Retaining employees. Retaining a skilled workforce and decreasing unwanted employee
turnover are necessities for companies and organizations world-wide (Belbin, Erwee, & Wiesner,
2012). Preserving the most skilled workers can be critical in determining whether a company can
maintain a competitive advantage while ensuring that operations are preserved (Cardy &
Lengnick-Hall, 2011, Frank, Finnegan, & Taylor, 2004). Losing talented employees translates to
losses in human capital, as the time and financial resources spent on the departed employee are
now lost (Cardy & Lengnick-Hall, 2011). If the best employees are not retained, an organization
can be negatively affected on both the operational and strategic levels (Cardy & Lengnick-Hall,
2011). As organizations compete to hire the most talented employees, a strong focus is made to
retain gifted employees (Govaerts & Kyndt, 2010) to foster the operational success of an
organization (Belbin et al., 2012).
According to Hytter (2007), workplace factors such as rewards, leadership style, career
advancement opportunities, training and skills development, physical working conditions, and
work-life balance have an impact on employee retention. Echols (2007) states that on the job
learning and development processes, coupled with promotions and salary increases, contribute to
increased retention. Rodrigez (2008) found that retention is positively influenced when
employees feel that they are learning and growing on the job, while limited growth and scarce
opportunities lead to decreased retention. Development and learning are critical to attracting and
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

28
maintaining high-quality employees (Govaerts & Kyndt, 2010). Work experience and tenure
were also found to have positive influences on retention (Gunz & Gunz, 2007).
Evaluation
Evaluation of human capital occurs throughout the HCM process and is essential when
attempting to improve or close performance gaps (Clark & Estes, 2008). It is the process of
providing employees with performance feedback and assessing employees’ value to the
organization (DeNisi & Pritchard, 2006; Flamholz & Lacey, 1981; Roberts, 1994) while
determining the connections between performance gaps, improvement programs, and cost-
effectiveness (Clark & Estes, 2008). Effective performance evaluation systems affect the
selection, development, and retention of employees in any organization. For example, as
employees seek to develop themselves (Dychtwald et al., 2006), evaluation processes can be
used to determine areas of need. Moreover, as a process valued by employees (Phillips & Roper,
2009) it can improve the performance and retention of valuable employees (Govaerts & Kyndt,
2010).
The goal of a human capital evaluation system is to provide information to the employee
and organization that will enhance employee effectiveness and performance. Clark and Estes
(2008) assert that evaluation is one of the most cost-effective ways to improve performance.
According to Kirkpatrick (1984) human capital evaluation systems are most effective when the
purpose and design is aligned to organizational goals and areas of need. Effective performance
appraisal systems ensure that evaluations and outcomes are structured so that the employee will
focus their actions in the ways desired by the organization, and will result in the kind of
performance that is needed (DeNisi & Peters, 1996).
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

29
Human capital evaluations, also known as performance appraisal systems have evolved
over the last 65 years. It has shifted from a person-based measurement system to a job-related
performance based system. Early forms of performance appraisal methods involved ranking and
comparing individuals to one another and were laden with problems. As a result, organizations
transitioned to job-related performance measurements, which require that job expectations and
the performance appraisal process are precise and clearly understood (Kirkpatrick, 1984) by all
parties involved.
Performance appraisals, also called employee evaluations, are formal evaluations that are
typically comprised of several components with clearly stated performance dimensions and/or
criteria that are used in the evaluation process (DeNisi & Pritchard, 2006; Roberts, 1994, 2002).
Research (Roberts, 1994, 2002; DeNisi & Pritchard, 2006; Levy & Williams, 2004) has found
that employee participation, collaboration, acceptance, the rater, and feedback are critical
components of an effective performance appraisal system.
Employee participation. When developing a performance appraisal system, Roberts
(2002) asserts that employee participation is an important intrinsic motivational strategy that
facilitates worker growth and development. Employee participation has been shown to improve
employee acceptance of the evaluation system and outcomes (Longenecker, Scazzero, &
Stansfield, 1994; Roberts, 1994, 2002). Opportunities for employee participation can occur in the
development of the performance appraisal system and in its operation (Roberts, 1994).
Participation empowers individuals to provide feedback on ratings and documentation and it has
been linked to increased confidence of employees in the fairness of the appraisal process.
Participation can occur through self-appraisal, interviews, and goal-setting (Longenecker et al.,
1994; Roberts, 1994, 2002). Self-appraisal or self-evaluation gives the employee the opportunity
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

30
to assess their own performance which increases overall preparation and readiness, enhancing
satisfaction and perceived fairness, and reducing self-defensive behavior (Roberts, 1994, 2002).
Employee collaboration. Collaboration also increases the accuracy and validity of the
appraisal process by allowing the employee to provide valid, unique, and relevant performance
information and insight that is unavailable or unobservable by the rater. Additionally, within a
participatory appraisal system, the employee attains “ownership” over the process enhancing
acceptance and trust in the success of the evaluation system. Finally, a participatory appraisal
system generates an atmosphere of cooperation and employee support, which encourages the
development of productive coaching or counseling relationships, thereby reducing appraisal-
rating tension, defensive behavior, and rater-ratee conflict.
Employee acceptance. Employee acceptance of a performance appraisal system is
essential for optimal effectiveness (Levy & Williams, 2004; Kirkpatrick, 1984; Roberts, 1994).
Acceptance of the evaluation tool and outcomes increases the likelihood that the employee will
be motivated to use the feedback they receive to change their behavior (DeNisi & Pritchard,
2006). Employees must understand the measurement process, agree on the value orientation of
the system, share consensus on the standards being used, be confident in the measurement
system, and perceive an absence of rater bias (Longenecker et al., 1994).
Employee acceptance of the performance appraisal system is also improved when
multiple sources of data are used (Gosselin, Werner, & Halle, 1997). DeNisi and Pritchard
(2006) identify several indicators such as a standards based measurement, goal setting, self-
evaluation, and interviews. As mentioned, Roberts (2002) asserts that employees must be
involved in developing and implementing any and all appraisal measurements to enhance
employee acceptance. A standards based measurement allows appraisers to measure employees
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

31
against a mutually understood set of performance standards. The standards clearly articulate and
measure factors such as essential job duties and responsibilities. Effective goal setting allows the
employee to set specific and moderately difficult goals to work towards rather than focusing on
past behavior which cannot be changed. The self-evaluation portion of a performance appraisal
system provides the employees with an opportunity to systematically assess their own
performance. Although managerial and employee ratings frequently disagree, the process
contributes towards achieving consensus and acceptance over time. In addition to a standards
based measurement tool, goal setting, and self-evaluation, performance appraisal interviews are
an additional source of data to measure employee performance. Roberts (2002) asserts that an
effective performance appraisal interview reinforces the open communication established and
analyzes employee strengths, needs, and future opportunities. Additionally, during the interview
the ratee is given the opportunity to challenge perceived discrepancies to avoid future
difficulties. Although there are tremendous benefits to a performance appraisal interview, they
can be lost and employee motivation reduced if it is handled poorly.
Rater. In addition to developing an accepted and accurate measurement tool it has been
found that the rater also influences the success of a performance appraisal systems. Roberts
(1992) asserts that the task of the rater in the appraisal process is to fairly and accurately observe
and produce a comprehensive rating of the ratee. Bernardin (1979) and Gosselin et al. (1997)
found that ratees identified their immediate supervisors as the most trusted and preferred
individuals to be primarily responsible for their performance appraisal. Although appraisals
conducted by an immediate supervisor can be daunting, they are often perceived as having the
most knowledge and influence on rewards and promotions.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

32
Although a great deal of research has been conducted on different types of rater training
concerned with eliminating rating errors such as leniency, halo, and range restriction, DeNisi
(1996) concluded that rater error training does not improve rater accuracy. It has been found that
a more effective approach is to study performance expectations (DeNisi, 1996; London, Mone, &
Scott, 2004). This type of training allows appraisers to develop a deeper understanding of the
performance standards to enable them to provide more constructive and descriptive feedback.
Feedback. Appraisal feedback has also been identified as a factor that influences
appraisal acceptance. In addition to facilitating two-way communication, feedback also allows
the employee to clarify feedback and identify ways to improve their performance (Roberts,
1994). Landy, Barnes, and Murphy (1978) found that periodic feedback reduces the likelihood of
discrepancies of performance standards subsequently increasing perceptions of rating fairness.
Although most organizations use annual appraisals (Locher & Teel, 1998), it has been found that
employees prefer to be rated more than once a year (Bernardin, 1979).
Summary of Human Capital Management and Evaluation
Human Capital Management is a complex system comprised of attracting, selecting,
engaging, developing, and retaining employees that posses the knowledge, skills, and abilities
(Joshi, 2012; Becker & Gerhart, 1996; Huselid, 1995; Phillips & Roper, 2009; McMahan &
Wright, 1992). When coupled with effective performance evaluation systems and aligned to the
strategic needs of the organization, performance goals and values can be realized (McMahan &
Wright, 1992).
The process of developing and implementing a performance appraisal system is complex
and laden with challenges for all parties involved. Although the aforementioned research
indicates that numerous factors should be taken into consideration when designing and
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

33
implementing an appraisal system, employee participation, collaboration, acceptance, the rater,
and feedback are vital components for optimal effectiveness. An effective performance appraisal
system can close achievement gaps and improve the overall performance of the organization
(Clark & Estes, 2008).
Teacher Evaluation
It is evident that education systems must improve to ensure that the next generation of
learners are prepared to compete in a global workforce. To close the growing achievement gap
and improve the overall performance of education systems we must look at teachers, as they are
the most influential school-based factor of student achievement (Darling-Hammond, 1999;
Pretorius, 2012). Clark and Estes (2008) highlight the importance of employing effective
performance appraisal systems to aid in this outcome, which is supported by Danielson and
McGreal (2001) and Howard and McColskey (2001) who assert that teacher evaluations area
vital tool for improving teacher effectiveness. Additionally, as the largest component of
educational expenditure (Dolton, 2010; Santibanez, 2010) decisions regarding teachers must be
made cautiously to optimal student and organizational outcomes supporting the need for
effective performance evaluation systems.
History of Teacher Evaluation
Teacher evaluations have existed since the 1700s when clergy were responsible for hiring
teachers and making judgments about their teaching from a moralistic and ethical perspective
(Marzano, Frontier, & Livingston, 2011). These early teacher evaluations measured teacher
quality based on qualities such as personality traits. Subsequent teacher performance measures
dating back to the 1950s were based on observable behaviors (Ellett & Teddlie, 2003), and were
proven to be an ineffective indicator of quality instruction and student learning.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

34
Although teacher evaluations have undergone many transformations since the 1950s,
Danielson and McGreal (2001) assert that many of the teacher evaluation systems in use today
were developed in the early to mid-1970s and reflect what educators believed about teaching at
that time. Based on work done by Madeline Hunter, many of the current systems use the
documentation of small observable behaviors to identify quality teachers. These observable
behaviors include writing the learning target on the board and completing appropriate
paperwork. These evaluation systems were supported by student achievement measures such as
norm-referenced, machine-scoreable, multiple-choice tests that measured fairly low-level
knowledge. Although these measures were well suited for the 1970s, they are not reflective of
recent educational research on factors such as the nature of the brain and how it learns. Over the
past 25 years educational research has provided the a wealth of knowledge to inform what
students should be learning and how teachers should be teaching thus teacher evaluation must
follow suit. According to a study completed by The New Teacher Project less than 1% of
teachers are rated unsatisfactory despite high percentages of students who fail to meet basic
academic standards (Weisberg, Sexton, Mulhern, & Keeling, 2009). According to Donaldson
(2009), teacher evaluations often suffer from the Lake Wobegon effect: Most if not all teachers
receive satisfactory evaluation ratings. Although it is possible that all teachers are above average
in some schools, there is usually more variation in teacher effectiveness.
There are multiple factors that influence the over identification of satisfactory teachers:
vague district standards, poor evaluation instruments, and overly restrictive collective bargaining
agreements. Within schools, the absence of high-quality professional development for
evaluators, a school culture that discourages critical feedback and negative evaluation ratings,
and a district culture that offers little oversight and few incentives for administrators to evaluate
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

35
accurately, also contribute to exaggerated teacher ratings. Additionally, traditional evaluation
methods have few negative or positive consequences, which reduce evaluators’ will to evaluate
accurately and thoroughly, and teachers’ incentives to take evaluations seriously (Donaldson,
2009).
Major Practices in Teacher Evaluation
Current teacher evaluation systems serve to devalue teacher effectiveness by producing
outcomes that reflect virtually no variation among teachers. Excellent teachers go unrecognized,
support and development is neglected, and poor performance goes unaddressed (Odden, 2011b).
A study conducted by Brandt, Mathers, Oliva, Brown-Sims, and Hess (2007), found that teacher
evaluations are primarily used to decide whether to retain or release new teachers. Moreover, in
response to the findings presented by The Widget Effect (Weisberg et al., 2009) a study
conducted by The New Teacher Project, Secretary of Education Arne Duncan (2009) said,
“Schools need to have evaluation systems that fairly and accurately identify teachers” (n.p.). The
study asserts that our school systems treat teachers as interchangeable parts rather than
professionals. In addition to failing to recognize quality teaching, a study completed by Toch and
Rothman (2008) found that most evaluation practices were “superficial, capricious, and often
didn’t directly address the quality of instruction, much less students learning” (p. 1). Poor
performance goes unaddressed and excellence unrecognized.
In addition to failing to recognize differences between teacher instruction, traditional
evaluation systems also neglect to provide the professional development function (Danielson &
McGreal, 2001). This finding was also supported by The Widget Effect (Weisberg et al., 2009)
study, which also found that teacher evaluations fail to identify developmental areas upon which
teachers could use to improve. More specifically, “73 percent of teachers surveyed said their
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

36
most recent evaluation did not identify and developmental areas, and only 45 percent of teachers
who did have developmental areas identified said they received useful support to improve” (p.
6). Effective teacher evaluation systems must identify and measure individual teachers’
strengths and weaknesses accurately and consistently, and provide feedback to teachers that can
be used to improve their practice.
Reforms to Teacher Evaluation
Over the last 30 years three major teacher quality reforms have developed from the
modern school reform. In 1983 in the wake of A Nation at Risk (National Commission on
Excellence in Education, 1983) which highlighted “the rising tide of mediocrity” in the nation’s
schools and teacher workforce, interest in reforming teacher evaluation grew. During this time
teacher evaluation and merit pay drew support to improve instruction and student learning but it
quickly lost steam when the economy weakened. The second phase of reform began in the 1990s
and focused on providing students with challenging academic standards and the use of high-
stakes assessment of those standards. The third phase dates from the 1996 publication of What
Matters Most: Teaching for America’s Future (National Commission on Teaching & America’s
Future, 1996) and drove the concept of teacher quality to the forefront of the policy agenda. The
report proposed a blueprint for recruiting, preparing, and supporting excellent teachers to reach
the goal of providing every student in America with access to competent, caring, qualified
teaching in schools organized to succeed. To attain this goal the report offered five major
recommendations: get serious about standards for teachers and students, reinvent teacher
preparation and professional development, fix teacher recruitment, encourage and reward teacher
skill, and create schools that are organized for student and teacher success.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

37
More recently, teacher evaluation gained increased attention with the adoption of the No
Child Left Behind Act of 2001 (NCLB, 2002). The NCLB Act increased school and teacher
accountability forcing schools to focus on teacher practice as a means of improving overall
student achievement. Although intended to increase accountability and ultimately student
achievement, NCLB faced a great deal of criticism. In March 2010, the U.S. Department of
Education released A Blueprint for Reform: The Reauthorization of the Elementary and
Secondary Education Act (the Blueprint) (USDOE, 2010a) which retains the NCLB’s focus on
test-based accountability while focusing on five priority areas: college and career ready students,
great teachers and leaders in every school, equity and opportunity for all students, raising the bar
and rewarding excellence, and promoting innovation and continuous improvement.
Accountability remains at the forefront of the Blueprint and will be measured by students’
academic growth during the school year. Similar to NCLB, the Blueprint also speaks to the
importance of teacher quality. Using the term, highly effective, teacher effectiveness will be
determined using factors such as student growth and value added measurements. Moreover, the
Blueprint offered NCLB waivers in which states could receive flexibility from NCLB by
adopting a plan to implement college and career ready standards, creating a comprehensive
system of teacher and principal development, developing an evaluation and support system, and
setting performance targets to improve student achievement and close achievement gaps (The
White House, n.d.-b). States across the nation applied for the NCLB waiver and adopted teacher
evaluation measures intended to improve teacher practice and student learning. In addition to
creating the Blueprint, the Obama Administration also implemented the Race to the Top (RttT)
(USDOE, 2010b) competitive grant program, which offered significant incentives to states
willing to make systemic reforms to improve teaching and learning.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

38
The RttT competition was created as a part of the American Recovery and Reinvestment
act of 2009, and created political cover for state education reformers to develop an innovation
plan and to help states construct the administrative capacity to implement these innovations
effectively (McGuinn, 2012). Joanne Weiss, the RttT program’s initial director highlighted three
predominant features to the philosophy behind RttT: shifting from a focus on means to a focus
on ends, shifting from sanctions to incentives as a way of motivating state reform, and shifting
the Department of Education away from compliance-monitoring to capacity building and
innovative (Nee, 2010). To qualify for up to $700 million, states were required to change their
education laws and policies to meet five conditions: join the Common Core Standards Initiative,
upgrade data systems, create a teacher evaluation system that was in part based on student test
scores, implement a turnaround model proposed by the Administration, and lift legal limits on
the number of charter schools allowed. Prior to the implementation of RttT most federal
education funds were distributed through categorical grant programs that allocated money to
districts based on need-based formulas (Beam & Conlan, 2002). In contrast, the RttT grant
allowed states to allocate resources to four administrative priorities: developing common
standards and assessments, improving teacher training, evaluation, and retention policies,
creating better data systems, and adopting preferred school-turn around strategies (USDOE,
2010b). The aforementioned changes to national education policy and federal funding have once
again brought teacher quality to the forefront of teachers, teacher unions, and school districts
across the nation. In 2010, Hawaii applied for and was awarded a four year, $75 million federal
RttT grant to improve student achievement throughout the state.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

39
New Teacher Evaluation Practices
As with any performance evaluation, teacher evaluations serve to connect instructional
practice with student achievement (Odden, 2011b; Milanowski & Heneman, 2001; Goe, Bell, &
Little, 2008). The literature (Glazerman et al., 2011; Goe et al., 2008; Papay, 2012; Hill &
Grossman, 2013) asserts that this can be achieved by incorporating multiple sources of data
within a teacher evaluation system. Several components are mentioned throughout the literature:
value-added models and classroom observations.
Goe et al. (2008) defines value-added models as a summary score of the contribution a
teacher makes towards growth in student achievement. According to Milanowski and Heneman
(2001) the contribution of a teacher to student learning can be determined by finding the
difference between the expected and actual level of achievement. Although value-added models
are relatively new measures of teacher effectiveness, supporters argue that they provide an
objective measure of determining which teachers are successful in improving student learning.
Critics of value-added models suggest that scores be interpreted with caution due to huge
variation in value-added scores within schools and its weak correlation to what teachers do in
their classrooms (Goe et al., 2008; Darling-Hammond & Haertel, 2012; Darling-Hammond,
Amrein-Beardsley, Haertel, & Rothstein, 2011). They argue that the value-added model focuses
on data from standardized test, which are not necessarily valid or reliable indicators of student
learning (Milanowski & Heneman, 2001) and that teacher ratings are significantly affected by
differences in the students who are assigned to them. Additionally, although value-added models
are intended to identify the contribution a teacher made to student achievement, they do not
provide feedback to teachers that indicate how they can improve their instruction. Darling-
Hammond, Amrein-Beardsley, Haertel, and Rothstein (2012) also found that teachers’ value-
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

40
added ratings are significantly affected by differences in the students who are assigned to them.
This could create disincentives for teachers working with high-needs students and could
reinforce inequalities as teachers avoid such student populations.
Unlike value-added models, which are based on standardized test scores, classroom
observations are typically conducted by a school administrator or an outside evaluator and are
intended to measure general teaching practices or subject-specific techniques. A critical
component of classroom observations are the use of appropriate and valid instruments and well-
trained and calibrated observers (Goe et al., 2008). Milanowski and Heneman (2001) recommend
that classroom observations include multiple competency levels through the use of rating scales
or rubrics that provide examples of each performance levels. Two common observation protocols
being used throughout the nation are Charlotte Danielson’s Enhancing Professional Practice: A
Framework for Teaching (2007) and the Universities Classroom Assessment Scoring System
(CLASS). Milanowski and Heneman (2001) assert that observations should be focused on key
aspects of teacher instruction that are key drivers of student learning and should be conducted on
multiple occasions to ensure consistency overtime. He goes on to argue that observations should
be conducted by multiple observers to account for principal bias and/or lack of content
knowledge and to provide more objective perspectives.
Teacher Evaluation Implementation
Similar to performance appraisal systems, it has been found that the successful
implementation of a teacher evaluation system depends on clear communication between
administrators and teachers (Darling-Hammond, Wise, & Pease, 1983). Involving teachers in the
design and implementation of an evaluation system increases the likelihood that they will see the
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

41
evaluation as an opportunity to gauge and improve their teaching capacity rather than a
summative report required to meet district or state requirements (Zepeda, 2002).
Danielson (2001) argues that to ensure a credible evaluation system, evaluators need to
be able to assess accurately, provide meaningful feedback, and engage teachers in productive
conversations about their practice. Evaluation systems are found to be more effective when they
ensure evaluators are well-trained, evaluation and feedback are frequent, mentoring and coaching
are available, and process such as Peer Assistance and Review systems are in place to support
due process and timely decision making by an appropriate body (Wise, Darling-Hammond,
McLaughlin, & Bernstein, 1984; Darling-Hammond et al., 2011).
Summary of Teacher Evaluation
Teacher evaluations have existed since the 1700s and have experienced tremendous
change due to educational and political changes. Most recently with the RttT competitive grant,
it has taken precedence within educational systems throughout the nation. Research shows that
effective teacher evaluation systems use multiple sources of data such as value-added models
and classroom observations to produce a comprehensive evaluation of a teacher’s effectiveness.
Two classroom observation protocols that are currently being used throughout the nation are
Charlotte Danielson’s Enhancing Professional Practice: A Framework for Teaching (2007) and
the Universities Classroom Assessment Scoring System (CLASS). In addition to essential
components, teacher evaluations must be implemented intentionally to optimize its effectiveness.
Two vital components of implementation are clear communication between administrators and
teachers and well-trained evaluators that are able to provide frequent and meaningful feedback.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

42
Teachers’ Unions
For more than 160 years, teachers’ unions have unified teachers across the nation to
improve public education. The two largest national teachers unions in the United States, the
National Education Association (NEA) and the American Federation of Teachers (AFT), support
more than 4.6 million members working at every level of education from pre-school to university
graduate programs throughout more than 14,300 communities. Upon its conception in 1857, the
National Education Association (NEA) has played a vital role in improving working conditions
for teachers and learning conditions for students. Prior to its formation, state education
associations existed in 15 of the 31 states in the Union but there was no national organization to
serve as single voice for America’s educators (National Education Association, n.d.). The AFT
was founded in 1916 and worked alongside the NEA to improve issues such as work conditions,
compensation, tenure, and equality for teachers and students (American Federation of Teachers,
n.d.).
History of Teachers’ Unions
Following the civil war, the NEA, then known as the National Teachers Association
(NTA), sought out and attained federal aid to help support the education of newly freed slaves of
all ages. In 1867 the NTA successfully lobbied congress to establish a Federal Department of
Education to provide and standardize education. Although the NTA had been open to minority
educators it wasn’t until 1866 that membership became accessible “persons” rather than just
“gentlemen.” Throughout the early 1900s the NEA tackled issues such as a child labor laws,
teacher salaries, tenure, and pensions. In 1966, after years being on the same course for justice
and equality for the nation’s children, the NEA and the American Teachers Association (ATA)
agreed to merge. The merger was met with some resistance from affiliates around the nation but
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

43
over time the NEA and ATA enacted state and federal laws. Today the NEA has affiliate
organizations in every state, which includes the Hawaii State Teachers Association in the State
of Hawaii (National Education Association, n.d.).
History of Teachers’ Unions in Hawaii
The Hawaii State Teachers Association (HSTA) was formally incorporated on January 1,
1971, but the transition from the existing system, the Hawaii Federation of Teachers (HFT) was
not easy. The HSTA was created from a new generation of public school teachers who, in the
mid-60s “had no qualms bout asserting their rights and standing up for what they believed in”
(Hawaii State Teachers Association [HSTA], n.d.). Alongside the civil rights movement, the
“young turks” as they were known, worked to unify the teachers and in 1968 the Hawaii State
Constitutional Convention made collective bargaining a constitutional right. This opened the
door for teachers to let their voices be heard and take action for themselves. In 1971 the public
school teachers of Hawaii selected the HSTA over the existing HFT to represent them. Although
the change was met with a great deal of resistance from those who were satisfied with the ways
things had always been most were ready for a change and to take a more active role in deciding
how to improve Hawaii’s education system for the teachers and students.
Upon its acceptance as the representative of Hawaii’s teachers, the HSTA tackled issues
such as teacher compensation, teacher development, leadership development, job sharing, and
grievances. After years of progress, in 2001 after two years of bargaining a new contract, 99% of
teachers went on a 21-day strike. Having gone through a similar bargaining just four years
before, the teachers were unwilling to agree to a package that included regressive measure and
virtually no pay increase. In the end, the teachers received more than 16% in pay raises, and
improvements in working conditions. More recently, on April 17, 2013 the members of the
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

44
HSTA voted to ratify a new four-year contract after more than two years of working without a
contract. The contract, which took effect on July 1, 2013, included a 3% pay increases above the
restored 5% cut in July 2012, 3 and 3.2% increases over subsequent years, and restoration of
health insurance premium percentages all of which was taken away as a result of a statewide
budget cut. Under the 2013 contract, a new evaluation system was established to improve
teaching practices and learning experiences for students. Also included in the new contract were
raises for existing teachers that will be tied to performance evaluations beginning July 1, 2015
(Hawaii State Teachers Association, n.d.).
Teacher Evaluation in Hawaii
Teachers throughout the state of Hawaii have participated in teacher evaluations for many
years. Prior to the current system, the Educator Effectiveness System, probationary, temporary,
and tenured teachers were rated on five duties on differentiated schedules. The current system,
the EES, was rolled out statewide in lieu of the 2013 teacher contract.
Program for Teachers (PEP-T)
Prior to the teacher evaluation system being investigated in this study, teachers in the
Hawaii Department of Education (HIDOE) were evaluated against the Professional Evaluation
Program for Teachers (PEP-T). Similar to the new teacher evaluation system, HIDOE in
collaboration with the Hawaii State Teachers Association (HSTA) developed the PEP-T.
According to the PEP-T manual (HIDOE, 2011), this evaluation served two purposes: (1) to
evaluate teacher effectiveness on a continuous basis and (2) to provide ratings for temporary,
probationary and tenured teachers in the public schools on a regular, specific schedule. The
outcomes of PEP-T ratings resulted in employment actions such as: continuation of contract,
granting tenure, extension of probation, non-renewal of probationary contract, or termination.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

45
Teachers were evaluated against Five Duties and given a rating of satisfactory, marginal, or
unsatisfactory. The Five Duties and a description of each are displayed in Table 1. The three
rating scales are displayed in Table 2.

Table 1
Five Duties of Teachers as Determined by PEP-T with Accompanying Description
Duty Description
Duty 1 Designs and Implements Effective Strategies to Develop Self-
Responsible/Independent Learners
Duty 2 Creates and Maintains a Positive and Safe Learning Environment
Duty 3 Uses Assessment Data
Duty 4 Demonstrates Professionalism
Duty 5 Reflects on Practice
Adapted from the PEP-T Manual (HIDOE, 2011)

Table 2
Three Point Rating Scale Used to Rate Teachers on the Five Duties as Determined by PEP-T
Rating Description
Satisfactory Teacher meets the Duty(ies) as stated in the Professional Program for
Teachers
Marginal Teacher needs improvement in order to meet the Duty(ies) as stated in
the Professional Program for Teachers
Unsatisfactory Teacher does not meet the Duty(ies) as stated in the Professional
Evaluation Program for Teachers
Adapted from the PEP-T Manual (HIDOE, 2011)

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

46
Teachers demonstrated their understanding of and ability to execute the Five Duties
through student records, documents, interviews, judgment, and scheduled observations. Upon
completing the PEP-T process, the evaluator met with the teacher to inform them of their ratings.
At this point the teacher and evaluator signed and dated the Rating form.
Similar to the findings of Brandt et al. (2007), which found that many teacher evaluation
systems are focused on new teachers, PEP-T was conducted with new teachers until they
received tenure while tenured teachers were evaluated once in a five-year cycle based on the last
digit of their social security number. The exceptions to this evaluation cycle were tenured
teachers who received marginal or unsatisfactory ratings during a previous rating. Teachers who
fell into this category were evaluated yearly until they could demonstrate satisfactory ratings for
the Five Duties or terminated. Probationary teachers who were unable to earn tenure after four
years were terminated. According to the PEP-T manual (HIDOE, 2011), probationary and
tenured teachers who received a rating of marginal or unsatisfactory were notified no later than
the third Friday in May with the rating form and all relevant Summary of Conference(s)
submitted to the Personnel Regional Office.
Educator Effectiveness System (EES)
Teacher evaluation has gained significant attention in lieu of No Child Left Behind
(NCLB) and the Race to the Top (RttT) competition. In 2010, 46 states applied for grants to
improve their education system. State applications were graded on a 500 point scale according to
the rigor of the reforms proposed and its compatibility with four administrative priorities:
developing common standards and assessments, improving teacher training, evaluation and
retention policies, creating better data systems, and adopting preferred school-turn around
strategies (USDOE, 2010b). In August 2010, Hawaii’s educational plan was awarded a four-
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

47
year, $75 million federal RttT grant. Within Hawaii’s RttT educational plan are five areas of
focus that are aligned to the Department of Education and Hawaii State Board of Education’s
Strategic Plan and national priorities.
The first of five focus areas is tying high-quality college- and career-ready standards and
assessment to a statewide curriculum. Throughout the state the Hawaii Common Core Standards
(CCSS) were fully implemented in school year 2013-2014 and in the school year 2014-2015 the
Smarter Balanced Assessment (SBAC) will replace the Hawaii Standards Assessment. The
second of five focus areas is improving longitudinal data collection and use. Hawaii created a
longitudinal database that teachers and administrators can access to monitor student progress and
growth using more than 100 different metrics and reports. The third focus area addresses
cultivating, rewarding, and leveraging effective teaching and leading. Under this focus area two
evaluation systems were created to support and evaluate educators throughout the state. The
Educator Effectiveness System (EES) was created for teachers and the Comprehensive
Evaluation System for School Administrators for principals. Providing targeted support to
struggling schools and students is the fourth element of the RttT educational plan. This
component is intended to close achievement gaps, turn around persistently low-performing
schools, and ensure all student are prepared for college, career and citizenship. The final focus
area is aligning organizational functions to support reform outcomes by creating a performance-
focused organizational structure (HIDOE, n.d.-b).
Similar to teacher evaluation systems that have been implemented throughout the nation,
Hawaii’s EES was designed to measure teachers’ professional practice, their impact, and provide
feedback and support to teachers to improve their effectiveness with students. The evaluation
focuses on teacher practice and student achievement. Under these two broad categories,
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

48
classroom teachers received feedback, support, and evaluation on five components: classroom
observations, core professionalism, student surveys, student growth, and student learning
objectives. Figure 2 illustrates Hawaii’s EES Framework design.

Figure 2. An overview of Hawaii’s Educator Effectiveness System framework. Reprinted from
The Educator Effectiveness System Manual (HIDOE, 2013).

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

49
As with any organization, schools are filled with various types of teachers. The EES
applies to all Bargaining Unit 5 employees with in the HIDOE, which includes classroom
teachers (PK-12), non-classroom teachers at the school level and non-school level, and teachers
with multiple roles. The various teacher classifications complicate EES and it is for this reason
that this study focused on elementary school classroom teachers of tested (grades 4-6) and non-
tested grades (K-3). As illustrated in Figure 2, the EES comprises several measures under two
categories, teacher practice, and student growth and learning which each account for 50% of a
teacher’s annual effectiveness rating. Table 3 outlines the expectations and responsibilities of
teachers and administrators for each of the EES components. Working portfolios have been
omitted from the following table, as they are not used as a data source for classroom teachers.

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

50
Table 3
Expectations and Responsibilities of Classroom Teachers and Administrators as Defined by the
EES
EES Measure Teacher Responsibilities Administrator Responsibilities
Core Professionalism
Acknowledges the value
of teacher dedication and
the numerous
contributions teachers
make to their profession
and their schools.
Document evidence aligned
with the Framework for
Teaching rubric for 4f.
Document evidence aligned with
the Framework for Teaching
rubric for 4f.
Communicate a deadline for
final submission of core
professionalism evidence.
Review evidence and assign a
rating for each teacher.
Classroom
Observations
Acknowledges the value
of targeted feedback
based on clear
performance expectations
and evidence
Facilitate two classroom
observations from a school
administrator (one per
semester)
Participate in pre-observation
conference with an
administrator before each
observation
Review observation in PDE3
Participate in post-observation
conference with an
administrator within two
weeks of observation
Reflect on the cycle using the
post conference form
Conduct two full-cycle
observations for every classroom
teacher
Provide teachers with a
minimum of 25 hours’ notice
before each pre-observation
conference
Hold pre-observation conference
with each teacher before each
classroom observation
Hold post-observation
conference with each teacher
within two weeks of classroom
observation
Enter data into PDE3
Tripod Student Survey
Acknowledges the value
of student perceptions as
an essential gauge of
school and classroom
quality.
Complete the Tripod roster
verification process in the fall
and spring using online tool
Administer the Tripod student
survey in the fall and spring
Give final approval of submitted
Tripod rosters in the fall and
spring using the online tool
Oversee the administration of the
Tripod student survey in the fall
and spring
Prepare staff to received and
analyze Tripod student survey
results
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

51

Table 3, continued
EES Measure Teacher Responsibilities Administrator Responsibilities
Hawaii Growth Model
Acknowledges the value
of student growth and the
belief that teachers
deserve to be recognized
for the gains students
make
Complete roster verification
for the Hawaii Growth Model
during the spring, tracking the
enrollment of students in
classes of tested grades and
subjects using an online tool
Give final approval of submitted
rosters for the Hawaii Growth
Model in late spring using the
online tool
Receive growth model data for
the school
Lead efforts to analyze growth
model data
Student Learning
Objectives
Acknowledges the value
of high expectations for
student achievement and
the process of linking
planning, classroom
instruction, and
assessment with student
outcome goals.
Complete two SLOs
Participate in beginning-of-tem
conference
Schedule and participate in a
midterm check-in as necessary
Schedule and participate in an
end-of-tem conference
Enter SLO data into PDE3
Oversee SLO development
Schedule and conduct beginning-
of-term conference with each
teacher
Conduct midterm check-ins as
necessary
Hold end-of-term conference and
assign final SLO ratings for each
teacher
Enter SLO data into PDE3
Final Effectiveness
Rating
Receive final rating in May Input all necessary evaluation
data into PDE3
Assign teachers a final
effectiveness rating no later than
May 16, 2014 or a mutually
agreed to alternate date
Source: Educator Effectiveness System Manual (HIDOE, 2013)

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

52
The Framework for Teaching (Danielson, 2007), previously mentioned, is a research-
based tool developed by Charlotte Danielson that establishes criteria and expectations for
evaluating teacher practice. Hawaii Department of Education classroom teachers selected this
evaluation tool after exploring multiple options during the first year of the EES pilot (SY 2011-
2012). The Framework is organized into four domains: planning and preparation, the classroom
environment, professional responsibilities and instruction. Within each of the aforementioned
domains are between five to six components and 15 to 23 elements. Using the Framework for
Teaching rubrics, teachers were rated using four performance levels: unsatisfactory, basic,
proficient, and distinguished. The Framework for Teaching rubrics were used to evaluate
teachers Core Professionalism and Classroom Observations. Once individual performance
ratings were determined using the appropriate rubrics, the results for each teacher were
quantified using the following point values:
• Distinguished: 4 Points
• Proficient: 3 Points
• Basic: 2 Points
• Unsatisfactory: 0 Points
Teacher Practice Measures
Core professionalism. According to the EES Manual (HIDOE, 2013) core
professionalism provides the teachers with the opportunity to demonstrate how they serve and
lead others in a manner is critical to their students and their school. The criteria and expectations
for core professionalism are articulated in The Framework for Teaching within Domain 4
(Professional Responsibilities) Component 4f: Showing Professionalism. Component 4f was
selected by the HIDOE and various stakeholders for the 2013-2014 school year because it most
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

53
closely captured several of the PEP-T Duty 4 values. Throughout the year the EES Manual
suggests that teacher and administrators collect evidence aligned to the elements of 4f.
Classroom observations. Classroom observations and accompanying conferences serve
as a collaborative process between teachers and administrators to understand and develop teacher
practice. Within The Framework, Domains 2 (classroom environment) and 3 (instruction) are
aligned to classroom observations. Although there are ten observable components between the
two domains, HIDOE chose to focus on five, which are aligned with statewide priorities. The
following five components were used to evaluate teacher performance during classroom
observations:
• 2d: Establishing a Culture for Learning
• 2d: Managing Student Behavior
• 3b: Using Questioning and Discussion Techniques
• 3c: Engaging Students in Learning
• 3d: Using Assessment in Learning
The classroom observation component of the EES required that teachers be visited once
per semester by an observer certified by the HIDOE. Each observation cycle was comprised of
pre-observation conference, classroom observation, post-observation conference, and
observation cycle conclusion during which time the teacher completes a Post-Observation
Summary Form and the observer reviewed it. Performance levels were collaboratively decided
upon during the post-observation conference through the analysis of observations and artifacts.
Tripod student surveys. The tripod student survey was developed over ten years ago
through a partnership between Cambridge Education and Dr. Ron Ferguson of Harvard
University. It is a classroom-level analysis and reporting system that asks students to provide
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

54
feedback on their classroom experience in relation to seven indicators of teaching practice
known as the 7Cs (caring, control, challenge, clarify, captivate, confer, and consolidate).
Adapted to the needs of Hawaii, specific surveys have been developed for early elementary (K-
2), upper elementary (3-5), and secondary students (6-12). Tripod surveys were administered
twice a year, at the end of the first and third quarters and individual teacher scores were ranked
to calculate a normal curve equivalent (NCE), which was used to determine individual EES
rating.
Student Growth and Learning Measures
In addition to evaluating teacher practice, the EES also used student growth and learning
outcomes to assign teacher effectiveness performance ratings. The two components of this
branch of the EES were the Hawaii Growth Model and Student Learning Objectives.
Hawaii Growth Model. The Hawaii Growth Model uses student growth percentile
(SGP) scores as one of two measures to capture student growth and learning for classroom
teachers. The Hawaii Growth Model utilized a longitudinal data system to group students who
scored similarly on the Hawaii State Assessment (HSA) and the SGP to understand the progress
the student is made. For example if a student attained an SGP of 60 for math, he/she would have
scored higher than 60% of similar students take the same test. Similarly, median student growth
percentiles (MSGPs) were used to capture the growth of groups of students. The MSGP for a
student group is the SGP value where half of the group did better and half did worse. Given that
HSA scores are validated and finalized in the summer, MSGPs used for evaluation lagged by at
least one school year. For example, the HSA scores from SY 2012-2013 will determine the
MSGPs used for SY 2013-2014. Classroom teachers of tested grades were evaluated using
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

55
individual MSGPs and classroom teachers of non-tested grades were evaluated using school
English Language Arts (ELA) MSGPs.
Student Learning Objectives. The final component of the EES system, Student
Learning Objectives (SLOs) provided teachers with an opportunity to document and highlight
the impact they made on student learning. SLOs allow teachers to set specific, measurable, data
driven goals for their students that were aligned to state or national standards and create a long-
term plan to achieve these goals. Although teachers must submit and follow through with two
SLOs, only one was used to determine the final SLO rating. SLO ratings were determined at the
end-of-term conference and were based on individual student growth in relation to expected
targets during the SLO process.
Implementation of the Educator Effectiveness System
The EES Manual (HIDOE, 2013), which provides a timetable, and implementation
strategies that should work for most classroom teachers, asserts that all teachers attend an
orientation describing EES protocols and available resources and an EES overview training by
the end of July 2013. The manual also indicated that by August 31, 2013 teachers should attend
the Charlotte Danielson’s Introduction to the Framework for Teaching training. Although the
manual does not specify who should conduct the orientation and training sessions a survey
conducted by the Hawaii State Teacher Association (HSTA) in September 2013 revealed 58.6%
of respondents were trained by Administrators or Educational Officers and 16% of respondents
were trained by Bargaining Unit 5 members (teachers). The survey also revealed that 6% of
respondents attended none of the trainings, 35.9% attended the EES Overview, 79.4% attended
the Charlotte Danielson training, 20.5% attended the EES Orientation with the EES manual, and
62.3% attended the EES orientation with the EES manual.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

56
Summary of Teacher Evaluation in Hawaii
The teacher evaluation system in Hawaii has recently undergone a tremendous amount
change in response to the federal funds received from the RttT competition grant. In school year
2013-2014 the Department implemented the Educator Effectiveness System to evaluate teacher
effectiveness. Under the new evaluation system teachers are evaluated against their practice and
student growth and learning. The teacher practice rating is comprised of data from core
professionalism and classroom observations. The student growth and learning rating is
determined using data collected from Student Learning Objectives, Tripod Surveys, and the
Hawaii Growth Model. Although the EES Manual provides a suggested timeline for
implementation of the new evaluation system it does not articulated who will conduct the
orientation and training sessions.
Conceptual Framework
Clark and Estes (2008) assert that evaluation is a critical element when attempting to
close performance gaps and improve performance. This study utilized the framework outlined by
Clark and Estes (2008) in Table 4 to guide the evaluation of the Educator Effectiveness System
in the State of Hawaii Department of Education.

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

57
Table 4
Four Levels of Evaluation
Level Description
Level 1: Reactions Are the participants motivated by the program? Do they value it?
Level 2: Impact During
the Program
Is the system effective while it is being implemented?
Level 3: Transfer Does the program continue to be effective after it is implemented?
Level 4: Bottom Line Has the transfer contributed to the achievement of the
organizational goals?
Adapted from Clark and Estes (2008)

Clark and Estes (2008) identify level 1 of performance evaluation as reactions. Reaction
evaluation is conducted with participants and is used to determine whether they like and value
the program. Although reaction information can be useful in determining if people are motivated
to persist and invest additional effort into the performance program it cannot be used to
determine if participants gained anything useful or if the program supports organizational goals.
The second level of evaluation, impact during the program, examines changes in knowledge
gaps and learning that might be taking place. Evaluations conducted at this level must focus on
the element that is central to the performance improvement program. Outcomes can be used to
identify, diagnose, and correct learning problems or training presentation glitches. Level 3,
transfer, is used to identify if the gains made during levels 1 and 2 persisted after the program’s
completion. Clark and Estes (2008) assert that ongoing monitoring of the transfer and persistence
of change program is essential in light of the research that shows that nearly all training
programs suffer from lack of transfer (Ford & Weissbein, 1997). The final level of evaluation,
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

58
bottom line, refers to the outcomes and answers the question of whether the program made any
difference to the organizational goal achievement. Given the nature of this study and the context
in which it is being conducted, only level 1 of the Four Levels of Evaluation will be utilized to
frame this study.
Summary
Teacher evaluation continues to remain at the forefront of education reform as efforts are
made to close achievement gaps and increase student learning and performance across the nation.
As a recipient of the RttT grant, the Hawaii Department of Education designed and implemented
a comprehensive teacher evaluation system in SY 2013-2014. Based on preliminary reactions
from teachers as indicated by the HSTA EES Survey (HSTA, 2014) the EES is not completely
understood by all stakeholders. This is concerning as research (Longenecker et al., 1994; Levy &
Williams, 2004; Kirkpatrick, 1984; Roberts, 1994) shows that clear expectation help to ensure an
evaluation system is valued by employees and seen as a means of improving their practice.
This study utilized focus groups and interviews to investigate the experiences and
reactions of elementary school teachers in the Hawaii Department of Education with the Hawaii
Educator Effectiveness System. The research methodology is explained in detail in the next
chapter.

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

59
CHAPTER 3
METHODOLOGY
The purpose of this study was to describe the experiences of elementary school teachers
regarding the Hawaii Department of Education Educator Effectiveness System. This chapter
describes sampling procedure, population, instrumentation, and procedures for data collection
and analysis.
Research Questions
1. What are the perceptions of teachers regarding the Educational Effectiveness System
as it was implemented?
a. How were teachers prepared for the implementation of the Educator
Effectiveness System?
b. How were teachers supported through the implementation of the Educator
Effectiveness System?
c. How do teachers perceive the EES has influenced their instruction?
2. Which factors contributed to the perceptions of teachers regarding the Educator
Effectiveness System as it was implemented?
Research Design
Investigating the experiences of elementary school teachers in the Hawaii Department of
Education in regards to the implementation of the EES was intended to reveal the perspectives of
teachers to help improve future implementation and training. This study was designed to be a
descriptive case study of the perceptions of nine elementary school teachers through a qualitative
design method. Maxwell (2013) asserts that one of the goals of qualitative research is
“understanding the meaning, of participants in the study, of the events, situations, experiences,
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

60
and actions they are involved with or engaged in” (p. 30). The purpose of this case study was
also consistent with the four characteristics of a qualitative study: the focus is on meaning and
understanding, the researcher is the primary instrument, the process is inductive, and the product
is richly descriptive (Merriam, 1998). Although the Hawaii State Teachers Association and the
Hawaii Department of Education have sought teacher feedback on the EES, the purpose of this
study was to dig deeper into understanding the experiences of teachers and shed light upon ways
to improve implementation processes in the future.
Sample and Population
This qualitative study utilized purposeful sampling or criterion sampling, which is
defined by Maxwell (2013) as means of deliberately selecting people to provide information that
is particularly relevant to ones goals and can not be gotten as from other choices. Due to sheer
number of teachers in the Department of Education, this study focused on the experiences of
elementary school classroom teachers from five schools in four school complexes on the island
of Oahu. The four participating complexes were Honolulu, Leeward, Central, and Windward.
One to two elementary schools were randomly selected from each complex and all the teachers
at the selected schools were invited to participate in the focus group. In addition to focus groups
and interviews with classroom teachers, semi-structured interviews were conducted with the
Complex Area Support Team (CAST) member responsible for the Educator Effectiveness
System from four complex areas to gain their implementation experiences. The selection of the
EES participants also used criterion sampling as they were selected based upon their association
with the CAST.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

61
Instrumentation
This study utilized two research-designed instruments: focus groups and interviews.
Focus Groups
Focus groups were chosen as the primary means of data collection for the classroom
teachers as the topic was not highly personal (Merriam, 1998) as scores and individual ratings
were not discussed. Being that the EES was something all teachers were experiencing allowing
the teachers to hear each others responses allowed the teachers to build off of each other’s
experiences while maintaining the integrity of their own experience. The focus group questions
were designed to answer the research questions identified at the beginning of this study. The
research questions and corresponding focus group questions can be found in Table 5. In addition
to the questions mentioned in Table 5, participants were also be asked to reveal demographic
data such as: grade level, years of teaching experience within the DOE, years of teaching
experience outside the DOE, and roles within the school.

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

62

Table 5
Focus Group Questions Aligned to Research Questions
Research question Focus Group Questions
What are the perceptions of
teachers regarding the
Educational Effectiveness
System as it was implemented?
What can you tell me about the training you received with
regards to the implementation of the Educator Effectiveness
System?
How were teachers prepared
for the implementation of the
Educator Effectiveness
System?
Did you feel prepared to pilot the EES system this during
School Year 2013-2014? Why or why not?
What component did you feel the most comfortable with
implementing? Why or why not?
What component did you feel the most comfortable with
implementing? Why or why not?
How were teachers supported
through the implementation of
the Educator Effectiveness
System?
Were you supported during SY 2013-14 with the
implementation of EES? (CD, SLO, Tripod, Professionalism)
Give examples (did you get a stipend, time off… )
How do teachers perceive the
EES has influenced their
instruction?
What was your CD observation experience like? What type
of feedback did you receive? Did you understand it?
What was your second observation experience like? Was it
any different from the first?
What was your experience with the first tripod survey
results?
What was your experience with the second tripod survey
results?
What was your experience with the SLO component like?

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

63
Interview
A semi-structured interview approach was selected as a means of collecting data from
CAST members. Merriam (1998) defines this type of interview as one that is guided by questions
or issues to be explored while allowing the researcher to respond to the thoughts and ideas
presented by the respondent. Similar to the focus group questions, the interview questions were
also designed to investigate the experiences of those involved with the EES. The research
questions and corresponding interview questions can be found in Table 6. In addition to the
interview questions, demographic data was collected in relation to the respondent’s job title and
description, years of service with the DOE, and prior classroom experience.

Table 6
Interview Questions Aligned to Research Questions
Research question Interview Questions
What are the perceptions of teachers
regarding the Educational Effectiveness
System as it was implemented?
What can you tell me about the training you received
in regards to the implementation of the Educator
Effectiveness System?
How were teachers prepared for the
implementation of the Educator
Effectiveness System?
What was your role in preparing teachers for the
implementation of the EES?
How were you prepared to do this?
Were you comfortable with this role? Why or why
not?
How were teachers supported through
the implementation of the Educator
Effectiveness System?
What is your understanding of the support teachers
received during SY 2013-2014 with the
implementation of the EES?
Were you involved with this? How?
How do teachers perceive the EES has
influenced their instruction?
Do you think the EES has influence teacher
instruction? How? Please give some examples.

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

64
Data Collection
Prior to data collection the University of Southern California and the Hawaii Department
of Education Internal Review board were consulted for approval of the methodology. Given that
the HSTA and the Department of Education have conducted similar studies, data from these
reports were collected for analysis and comparison. Additionally, complex and individual school
data published by the Accountability Resource Center Hawaii (ARCH) and the School Status
Improvement Report (SSIR) was gathered for comparative analysis. These reports were used to
identify potential bias in teacher or CAST member response.
Prior to conducting the interviews and focus group sessions, primary consent was
obtained in person from all participants. Additionally, conformation of date, time, and location of
the sessions was arranged through email or phone correspondence. Upon obtaining consent, each
participant was asked if they are comfortable with recording the conversation.
Data Analysis
The data collected from the interviews and focus groups was used to analyze to answer
the research questions of this study. Immediately following each data collection session, the
interview data was reviewed and transcribed. Following the data analysis process outlined in
Merriam (1998) each transcript was read and coded. Coding is identifying data that appears to be
interesting, important, or potentially relevant to the study by writing codes or notes in the
margins (Merriam, 1998). Unlike data analysis, this process was used to identify valuable
information that will be analyzed in subsequent sessions. Once the data was coded it was
organized into categories to answer each of the research questions.
In addition to transcripts of the conversations, reflective field notes were kept to provide
context to the transcripts during the data analysis process. Bogdan and Biklen (2007) define
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

65
reflective field notes as “sentences and paragraphs that reflect a more personal account of the
course of inquiry” (p. 122). The reflective field notes contained observations such as the
arrangement of the room, placement and engagement of the participants and overall impression
of the conversation.
Summary
This chapter provides an overview of the research methodology of this study including a
description of the data collection instruments, processes, and analysis strategies utilized. It also
outlines the purpose of the study and the research questions this study seeks to answer. Although
this study was designed to obtain a representative sample of teacher perceptions throughout the
state of Hawaii, it is possible that the data presented in the following chapter will not accomplish
this. Moreover, an additional limitation may have occurred in the data collection process, as the
focus group facilitator was unable to engage all participates in such away to foster conversations
that allowed for honest and open discussions.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

66
CHAPTER 4
RESULTS
The purpose of this study was to investigate, analyze, and describe the teachers’
perceptions of the implementation of the Educator Effectiveness System within the Hawaii
Department of Education during the 2013-2014 school year. The experiences of individual
elementary school teachers, groups of elementary school teachers, and Educational Officers (EO)
were described and analyzed to determine what agreements and discrepancies in perception
existed. Chapter 4 presents the findings for the following research questions:
1. What are the perceptions of teachers regarding the Educational Effectiveness System
as it was implemented?
a. How were teachers prepared for the implementation of the Educator
Effectiveness System?
b. How were teachers supported through the implementation of the Educator
Effectiveness System?
c. How do teachers perceive the Educator Effectiveness System has influenced
their instruction?
2. Which factors contributed to the perceptions of teachers regarding the Educator
Effectiveness System as it was implemented?
This chapter will present findings significant to answering the research questions.
Findings will be organized by research question, and emergent themes will be identified and
elaborated upon. In response to Question 1, the interviews revealed that initial training practices
varied throughout the state. The second finding was that amount support teachers received during
the first year of EES implantation also varied depending in which complex the teacher worked.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

67
These first two findings suggest that although teachers received initial implementation training
and some level of support throughout the first year, the teachers did not feel adequately prepared
to implement the Hawaii Department of Education Educator Evaluation System (EES) during the
2014-2015 school year.
In response to Question 2, the interviews disclosed that teachers’ perceptions of the EES
were influenced by unclear expectations with regards to the Student Learning Objectives (SLO)
and Core Professionalism, the type of initial training received, and the type of ongoing support
they received throughout the school year. As discussed in the findings related to Question 1,
feeling inadequately prepared to implement the EES affected teachers’ perceptions of the EES
system as it was implemented. Although teachers shared that all components of the EES
presented challenges, they were most comfortable with the Charlotte Danielson observation and
least comfortable with the Student Learning Objective (SLO) component.
The findings related to Question 2 also showed dimensions of the EES implementation
that were perceived as being effective and challenging in supporting teachers. The first theme to
emerge was that initial training programs were perceived as more effective when they were
provided by a credible source and over a period of time. The second theme to emerge was that
ongoing support by school level personal was perceived as ideal but it was sometimes difficult to
achieve due to factors such as school level personal limitations.
The insights of the participants were based on their experiences during the 2013-2014
school year. The findings will be presented in conjunction with parts of the literature review to
ground teachers experiences.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

68
Research Question 1 Findings
Research Question 1 asked: What are the perceptions of teachers regarding the
Educational Effectiveness System as it was implemented? All nine respondents shared that they
did not feel adequately prepared to utilize the EES as implemented in the 2013-2014 school year.
The research (Roberts, 1994, 2002; DeNisi & Pritchard, 2006; Levy & Williams, 2004) in
Chapter 2 identified five critical components to an effective performance appraisal system:
employee participation, collaboration, acceptance, the rater, and feedback. The three sub-
questions were aimed to identify which of the critical components of an effective performance
appraisal system contributed to the teachers overall perception of the EES.
EES Initial Training
To understand the background of each teacher’s EES experience, all nine teachers were
asked to share how they were prepared for the implementation of the EES. Of the five schools
participating in the study, all five received a different type of implementation training during the
first semester of the school year. The interviews and focus groups revealed that teachers
throughout all Oahu complexes received at least one initial training during the summer of 2013
or first semester of school from an administrator or Educator Effectiveness System Educational
Officer (EES EO). More specifically, four of the five schools received their training from the
EES EO while the remaining school received their training from their principal. A summary of
the type of training each school received is in Table 7. The teachers in the Mahalo Complex
were given the opportunity to attend two paid initial EES training sessions during the summer
prior to the start of the school year. Teachers that were unable to attend the summer trainings
were provided with substitutes to attend trainings within the first quarter. The teachers in the
Kokua Complex were provided with substitutes and received initial trainings during the first
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

69
semester. Teachers in the Mahina Complex received their initial training during four two-and-a-
half-hour Wednesday meetings during the first quarter of school from their EES Educational
Officer and their Principal. The teachers in the Wikiwiki complex received their training in
pieces throughout the first semester by various parties, which included the EES EO and a
Charlotte Danielson trainer from a private institution. With regard to when the training was
provided, one school attending training during the summer prior to school starting while the
other four received their training during the school year during after school meetings or during
work hours with a substitute provided to cover their class. Although the method of presentation
varied between complexes, all respondents shared the initial trainings included an overview of
EES and an introduction to each component with a focus on the Charlotte Danielson Observation
and SLO components.
The data gathered cannot be used to determine if one method of initial implementation
training was more effective than another, but the feedback from the teachers revealed that
receiving training in smaller parts was more manageable than the two full day trainings. The
teacher from the Mahalo complex who attend two full day implementation training sessions
shared:
It was hard to make sense of it all. On the first day they went over everything. We learned
about all of the components but it was so general. On the second day they talked about
the Charlotte Danielson part. They gave us the book and we watched the video but I
didn’t understand what it all meant. I remember leaving there so with many questions.

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

70
Table 7
Summary of Initial Educator Effectiveness System Training by Complex Area and School
Complex
Area
Elementary
School
When was the initial
training conducted?
When was the initial
training presented?
Initial Training
Trainer Content
Kokua
Complex
Aloha
Elementary
School
Between August and
December of the 2013-
2014 school year
Teachers were
provided with half day
subs to attend complex
area training
Wednesday meetings
Complex Area
Educator
Effectiveness
System Educational
Officer (EES EO)
• EES Overview
• Charlotte
Danielson
• SLO
• Tripod Survey
• Core
Professionalism
Pua
Elementary
School
Between August and
December of the 2013-
2014 school year
Teachers were
provided with half day
subs to attend complex
area training
Wednesday meetings
Complex Area
Educator
Effectiveness
System Educational
Officer (EES EO)
• EES Overview
• Charlotte
Danielson
• SLO
• Tripod Survey
• Core
Professionalism
Mahina
Complex
Nani
Elementary
School
Between August and
December of the 2013-
2014 school year
Wednesday meetings • Principal
• Video
• EES Overview
• Charlotte
Danielson
• SLO
• Tripod Survey
• Core
Professionalism
Wikiwiki
Complex
Mele
Elementary
School
Between August and
December of the 2013-
2014 school year
Wednesday meetings
Teachers were
provided with full day
subs to attend complex
trainings
• EES EO
• School
Administration
• EES Overview
• Charlotte
Danielson
• SLO (very little)
• Tripod Survey
Mahalo
Complex
Kuleana
Elementary
School
Summer 2014 2 days of paid complex
area training
• EES EO
• CAST Members
• EES Overview
• Charlotte
Danielson
• SLO
• Tripod Survey
• Core
Professionalism

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

71
Although the EES EO an the CAST provided training during the summer in preparation
for the school year, the teacher felt that the information was “a lot to digest” and “it wasn’t
something that I could think about. I was thinking about the beginning of the school year and
getting ready for that.” This was in contrast to a teacher from the Aloha complex who shared that
“receiving the training in smaller chunks made it easier to understand. We knew that we were
going to see our EES EO again so we could bring our questions to the next meeting.”
Furthermore, when asked if she minded being pulled out for training the teacher said:
It is not ideal to leave your classroom, especially during the first quarter, but because they
were only half day trainings it wasn’t too bad. Learning about EES during the school
year also helped me make connections to what I was already doing. I could see where I
could apply the EES parts. Like the SLO. When I learned about it I knew that I wanted to
do writing because that was our schools focus.
The experiences that the teachers shared indicate that although it was difficult to leave the
classroom during the first quarter, receiving the training during the school year supported
teachers in understanding how the EES is connected to what they are already doing in their
classrooms.

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

72
EES Support
Similar to the initial training, teachers throughout Oahu also received varied levels of
support throughout the first year of the EES implementation. A summary of the support teachers
received during the first year of statewide EES implementation can be found in Table 8.

Table 8
Summary of Ongoing Support Received by Teachers by Complex Area and School
Complex
Area Elementary School
Did you
receive
ongoing
support?
Who provided
ongoing support
during the 13-15
school year?
What did you receive
support with?
Aloha
Complex
Kokua Elementary
School
Yes • EES EO
• Grade Level
• SLO
Pua Elementary
School
Yes • EES EO
• Grade Level
• SLO
Mahina
Complex
Nani Elementary
School
Yes • School level –
grade level and
other grade
level teachers
• SLO
Wikiwiki
Complex
Mele Elementary
School
Yes • School level –
administration
• SLO
• Core
Professionalism
Mahalo
Complex
Kuleana Elementary
School
Yes • School level –
Vice principal
• CAST
members
• SLO
• Charlotte Danielson
Observation
• Tripod Survey

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

73
During the first year of implementation teachers from Kuelana School, which is a part of
the Mahalo complex, shared that their EES EO and Vice Principal provided and offered support
throughout the first year of implementation. Members of the Complex Area Support Team
(CAST) came to Professional Development days to support the teachers while they worked on
their SLOs and to answer questions the teachers had regarding the EES. The CAST also made
themselves available to the teachers via email throughout the school year. Additionally, the Vice
Principal provided the teachers with three afternoon trainings to share EES updates, changes, and
additional information that the State was providing. The teacher shared that “it was helpful to
hear about the EES changes from our VP rather than in an email. Hearing about the changes face
to face gave us the opportunity to ask questions.” The teachers from Aloha School in the Kokua
complex also received support during the year from two members of their CAST during
Wednesday meetings: “It was helpful when they would come to our meetings. Sometimes they
would have things to share and other times they just walked around and listened to what we were
working on and give feedback.” The teachers from Aloha school were also provided with time
during weekly grade level articulation to work on their SLOs but they did not receive any school
level or complex level support during these meetings. One respondent shared, “it was nice to
have the time to work on our SLOs but in the beginning we didn’t know what we were doing.
We spent a lot of time understand the SLO and understanding what was expected of us.” When
asked if their administrator provided additional support, the teachers from Aloha school
responded that:
Although we have a strong relationship with our principal we didn’t want to bother her.
We knew that she had a lot of things to do and we weren’t sure if she would be able to
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

74
answer our question without having to ask someone else. We didn’t want to give her any
more work.
The teachers from Nani Schools in the Mahina complex were also provided with time during
Wednesday afternoon meetings to work on their SLOs. Although they were able to contact their
EES EO via email they received most of their support from school level administrators and one
another: “We just worked on it together. It wasn’t easy but we knew it had to get done so we did
it.” Similar to the Mahina complex the teachers from Mele Elementary School in the Wikiwiki
complex received no district support once the initial trainings were complete. They were able to
ask their Vice Principals who did not always have the answers but were willing to find them.
Beyond the support that the teachers received from their CAST and the school level
administration, all of the respondents shared during the interviews that they spent a great deal of
time with the teachers in their grade levels and or their school to make sense of the expectations
set forth by the various components of EES.
To corroborate the implementation training and ongoing support data provided by the
teachers, EES EOs from three of the four complex areas were conducted. The interviews with the
EES EOs revealed that although all EOs were provided with the same initial trainings, State
directives were not provided to support the EOs with designing and implementing complex level
initial trainings and support. All three EOs received intensive training for each component of the
EES. They received training from the Danielson Group for Danielson Certification; the Center
for Assessments provided preparation for the SLO component, and State EES leads conducted
the tripod and core professionalism trainings. Of the three EES EOs interviewed, two felt
confident in being able to share information to the teachers based on what they had learned.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

75
More specifically, the EES EO from the Mahina complex who previously supported teachers in
implementing an evaluation system said:
I felt comfortable supporting the teachers because of my past experience but it’s still new
for everyone. I think because it is tied to an evaluation people are more sensitive about it
but my message to people is that it is going to get better every year.
The Mahalo Complex EES EO stated:
Implementation of a new system always comes with challenges and so did the EES
rollout. I was confident in relaying the message to the teachers based on what I knew at
the time. It was a challenge however that the state was making changes as we were
conducting the trainings and as the school year started. That made it difficult for use to
prepare the teachers.
The third EES EO did not feel prepared initially but spent a great deal of time with another
CAST member to make sense of the information to make it their own. He compared the
implementation process to building a plane while it was taking off:
Initially I didn’t feel like I was really prepared. I didn’t really understanding everything
myself. Especially the SLO process when it first came out but through sitting down and
working with my CAST we were able to process through the information and make it our
own. Only after making it our own and delivering trainings to administrators and being
prepared for that did I begin to feel more comfortable with my role as an EES EO.
Although the EES EOs confidence in implementing the initial trainings cannot be directly
correlated to the type of training the teachers received, it does shed light upon how the EOs
experiences might have influenced the training they provided.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

76
Role Perceptions
The experiences of the eight teachers interviewed was further affirmed when each of the
EES EOs were asked to share their role in the EES implementation process. This question
revealed that each EES EO defined their role differently. The EES EO from the Mahalo complex
shared that:
As a member of the CAST I was responsible for conducting the trainings that the teachers
received in summer of 2013. I also found resources for the schools to use and built
capacity within the CAST by training them in all components of EES.
The EES EOs from the Mahina and Kokua complexes shared that they worked more closely with
school principals and used a “train the trainers” model. The Mahina EES EO stated:
I worked with the principals to develop their capacity as leaders so they could be the first
point of contact for the teachers at their schools. If there were teacher concerns that the
principals could not address I got involved but the teachers’ primary point of contact was
the principals.
The EES EO from the Kokua complex took a similar approach but also included a support plan:
My goal was to team up with the administrators and the academic coaches to present the
trainings. I felt like if I could build ownership within the schools and work along side
them we could build capacity in them as well as build capacity within their teachers. I
also worked with principals and academic coaches on developing a plan to train and
support their teachers throughout the process. I knew that having that one shot training
would not work.
The varied responses from the EES EOs indicate that although the EOs received the same EES
trainings, the role they assumed as the EES EO varied which may have influenced the type of
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

77
initial training and level of support that the teachers received. The aforementioned EOs responses
corroborate the varied implementation trainings and levels of support that the teachers received
throughout the state.
Opinions of EES Components
In addition to the types of training and support received, the teachers were also asked to
share which components of EES they felt the most and least comfortable implementing to
understand how the initial trainings may have contributed to the teachers’ perceptions.
Respondents from all five school sites shared that the Charlotte Danielson observation was the
least challenging to implement because it was similar to the PEP-T rating system. A teacher from
Melemele School shared:
Danielson was the easiest component. Being observed is nothing new for us and our
administration is by the book so we knew what to expect in terms of what we would be
evaluated against. They also made it easy for use to conduct our pre and post
conferences. We use Google docs so we just signed up on Google docs for a conference
time before or after school. For people like me who like to know right away how we did
we could sign up for a lunch appointment right after the observation.
Similarly the teacher from Aloha School shared that they were most comfortable with the
Charlotte Danielson observation because “it was the least different from what we do. We have a
lot of classroom visits so we are used to people coming in and watching us teach.” However,
although the teachers were comfortable with being observed they were not comfortable with the
Charlotte Danielson rubric. One teacher shared:
We were told that no teachers lives in the distinguished part of the rubric and that the
rubric was intended to be used purely as an observation in order to start a conversation
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

78
between teachers and their principals in order for growth to constantly occur … not as an
evaluation. However, this is not what happened. It is an evaluation and we were expected
to live in the distinguished portion of the rubric. I was not concerned about being
observed or receiving feedback, but because the rubric was so specific it was very
stressful. During my conference my principal said she has seen student initiated
conversations in my classroom but because she didn’t see any in the lesson she observed
I couldn’t receive a distinguished rating.
Additionally, when asked to share about their experience two of the seven teachers found the
process and feedback useful while the remaining five had less positive experiences. The teachers
who did not find the observation to be useful spoke of two issues: inconsistent ratings within the
school and lack of follow up to support teachers improve their instruction. With regard to
inconsistent ratings the teachers expressed concern regarding subjectivity between evaluators. A
teacher from Aloha School shared:
It depended on who you had. One administrator was more by the book while the other
was not as objective so the ratings were higher or lower. I also heard that some of the
administrators believe that no teacher deserves distinguished while others do.
Similarly, another teacher from Aloha school shared that “some administrators evaluated
teachers strictly by the Charlotte Danielson rubric while others gave teachers the benefit of the
doubt if the teacher could justify a higher rating.” The teachers also shared frustration regarding
a lack of resources to support them in improving the areas in which they did not receive a
distinguished rating. One teacher said:
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

79
All she did was tell me what I needed to work on. I already knew that I needed to work
on questioning. How do I do that? She didn’t help me develop a plan or offer any
resources or suggestions to help me improve upon my instructional practices.
Although the teachers found the observation to be the easiest to implement, questions about its
validity and usefulness have contributed to some frustration regarding this component of EES.
The teachers’ comfort with the Charlotte Danielson component of the EES might be attributed to
the time that was provided for training. Of the five components of the EES, the Charlotte
Danielson component received the most amount of training during the initial implementation
training. It is not clear if this added to the comfort teachers experienced but it can be assumed
that it contributed to it.
In addition to the component that they felt the most confident to implement, the teachers
were also asked to share which component they felt the least comfortable implementing. All
eight teachers involved in this study identified the SLO as the most challenging component to
implement for the following three reasons: no exemplars, unclear data collection procedures, due
dates. The lack of SLO exemplars was the most pressing issue for the teachers due to the
specificity they were asked to use when writing their SLO. One teacher shared,
When we started to write the SLOs we all had a different interpretation of what the terms
meant. For example, learning goal. Some of us thought that it was the learning target
others thought it was the standard. There was also some confusion about the big idea,
instructional strategies, and expected targets. Each grade level was defining it differently
and when we talked about it as a staff we some of us had to rewrite our SLOs based on
the school wide definition. It would have been nice to have exemplars for us to use from
the start. Eventually we got some samples but we were told that they were not to be used
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

80
as exemplars but rather guides. That was frustrating because we didn’t know what part of
the sample SLO was considered to be good and which part was not good.
In addition to the exemplars the teachers were also frustrated with the data collection
expectations with regard to the pre-assessments, formative assessments, and summative
assessments. A teacher from Melemele school mentioned:
We knew that we needed to collect data which wasn’t a problem since we collect data
any way but we didn’t know how many data points would be enough, how or where to
record the data, and if the data points that we collected would be considered good enough
or valid.
The final concern for some teachers were the due dates, more specifically the end of final SLO
due date and the mid-year review. The teachers knew that there was an opportunity to revise
their expected targets but many missed it due to confusion regarding the mid-year review date or
poor communication. One of the teachers from Kuleana School said,
There was just so much going on all the time I couldn’t remember when the SLO mid
review was and before I knew it the date had passed. I know that I am an adult and should
be able to remember due dates but an email from administration or the CAST reminding
us to setup SLO mid year review would have been helpful. We were asked to set our
targets so early in the year. I barely knew my students.
The end of the year due date also concerned some teachers as the SLO was due long before the
end of the school year. A teacher from Melemele School shared,
We wrote our SLOs on standards we knew we could scaffold and build upon throughout
the school year but the due date was so early. We didn’t have enough time to teach so a
lot of my students weren’t able to meet their expected targets so I was penalized. I didn’t
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

81
think that was fair. I don’t know how to fix that. I guess next year I just need to pick a
less rigorous standard.
Although the SLO proved to be the most challenging component, teachers in two of the five
schools likened the process to Data Teams and the Instructional Leadership Team process
already in place within their schools. The teachers from Aloha Schools shared,
The SLO was frustrating because we didn’t know how to write it and if we were doing it
right but the EES EO encouraged us to write the SLO on our Data Team and Instructional
Leadership Team (ILT) focus. That made it seem less overwhelming.
Similarly the teacher at Kuleana School said, “Our EES EO showed our ILT how the SLO is on
going Data Team cycles which we already do so that helped us to connect the SLO with
something that we were already doing at our school.” Unclear expectations was contributing
factor to teachers frustrations, which is a critical factor of success according to Kirkpatrick
(1984). However, the additional support that the EES EOs and school level administrators were
able to provide to the teachers helped the teachers make sense of the process and connect it to
processes already being used within the school.
The successes and challenges that the Charlotte Danielson observation and SLO
presented for the elementary teachers were also evident in the EES EOs experiences. The
Mahalo complex EES EO stated that he “spent a great deal of his time working with the teachers
and administrators to understand that Charlotte Danielson process as well as writing and
rewriting their SLOs.” He also shared that in response to unclear SLO data collection procedures
he created a data collection sheet for the schools in his complex to use. The Kokua EES EO
shared that throughout the first semester the principals in his complex had a lot of questions
regarding SLOs that he was unable to answer. He denoted that he received some support from
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

82
those at the state level but because things were still being defined and solidified it was difficult
for anyone to provide him clear and consistent answers. The Mahina EES EO shared that what
he saw at the schools he works with was compliance:
The teachers were so overwhelmed with the various components and that they didn’t see
how the EES could be used to improve their instructional practice, especially the SLO.
The SLO is the best part of EES because it gives all the control to the teachers.
In spite of the additional support provided to the teachers and EES EOs regarding the Charlotte
Danielson and SLO components, the SLO component proved to be the most challenging due to
unclear expectations and changes made once the initial implementation was provided.
The findings related to Research Question 1 reveal that although the teachers felt
unprepared to implement the EES during the 2013-2014 school year, they were more
comfortable with the Charlotte Danielson component. Although no direct correlations can be
made due to a specificity of this study, their level of comfort might be attributed to the additional
training. Additionally, their level of comfort might also be attributed to the similarity between
the Charlotte Danielson component and the PEP-T and the specificity that the rubrics and manual
provided. The interviews also revealed that the teachers were the least comfortable with the SLO
process. Although no direct correlation can be made it can be assumed that their discomfort can
be attributed to unclear expectations and procedures and an absence of exemplars. The
discomfort that the teachers experienced was consistent with the EES EOs struggles and the
literature (Kirkpatrick, 1984). Moreover, the interviews also revealed that teachers were provided
with varying levels of training and support further contributing to the various perceptions and
experiences shared by the teachers.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

83
Research Question 2 Findings
Research Question 2 asked: Which factors contributed to the perceptions of teachers
regarding the Educator Effectiveness System as it was implemented? Upon reviewing the
interview data from the teachers and EES EOs three factors emerged; lack of clarity with regard
to expectations and procedures, the type of initial training the teacher received, and the type of
ongoing support the teachers received.
Lack of Clarity
A consistent finding from the teacher interviews, focus groups, and the three EES EO
interviews was that the lack of clarity with regard to expectations, procedures, and consistent
responses was an area of frustration. The EES EOs shared that the teachers and administrators
turned to them for answers but they did not always know what the answers were. One of the EES
EOs stated:
I gave the teachers as much information as I could but some things were just not ready to
be shared. In the beginning there were a lot of things that were not worked out. As we
rolled out the EES we were able to work out the kinks and learn from each other. I know
the teachers were frustrated with the changes and unanswered questions but we did the
best we could with what we had.
Similarly a teacher from Pua School shared,
It just felt like everything was changing while we were trying to make sense of it. We
thought we understood what we were supposed to do or what was expected of us then we
would get an email or be told in a meeting something different. I know it wasn’t the EES
EOs fault. It was just frustrating.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

84
The interviews revealed that the frustrations that the teachers experienced arose from receiving
inconsistent and unclear answers to their questions. One teacher shared that her grade level asked
the EES EO and the principal a question about their SLO and got two different answers. She
revealed:
It was really frustrating to not be able to get answers to something so important. I asked
the EES EO and the principal a question about my SLO and got two different answers. I
mean my EES rating might be used to determine how much money I make. I want to
understand how I will be evaluated so I can know how I can improve my teaching and get
a good rating.
The lack of clarity and ongoing changes to the EES affected the teachers and EES EO
experiences with the initial implementation of the EES throughout the state.
Initial Training Dates
In addition to unclear expectations, the interview data also revealed that teachers received
their initial trainings at different times during the first year of implementation. Some teachers
received their initial EES training prior to the start of school during two full day trainings while
others received it during the first semester during half-day professional development days or
during afterschool meetings. Educators in the Mahalo complex received their two-day initial
training during the summer prior to starting the school year. The respondent from the Mahalo
complex stated that she appreciated receiving the information prior to the start of the school year.
The teacher also mentioned that:
It was hard to make sense of it all. On the first day they went over everything. We learned
about all of the components but it was so general. On the second day they talked about
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

85
the Charlotte Danielson part. They gave us the book and we watched the video but I
didn’t understand what it all meant. I remember leaving there so with many questions.
Teachers in the Kokua and Wikiwiki complexes received their initial trainings during the school
year during afterschool Wednesday meetings or at full or half-day complex trainings. Teachers in
the Kokua complex received all of their initial trainings by the EES EO. They were provided
with substitutes to attend two half-day trainings on the Charlotte Danielson observation during
the first quarter while their EES overview training was spread across a few Wednesday meetings
afterschool. Although the teacher still had questions regarding the various components of the
EES she mentioned that:
Receiving the training in smaller chunks made it easier to understand. We had time to
digest what we learned and try to make sense of it. It was comforting to know that we
were going to see our EES EO again so we could bring our questions to the next meeting.
The teachers from the Wikiwiki complex also received their training during the school year but
they were provided with full day substitutes and received their EES overview on day one and a
private trainer provided the Charlotte Danielson training on day two. The teachers from the
Wikiwiki complex shared that there was some confusion regarding their initial training as the
EES EO thought she was coming to answer questions while the teachers were expecting an EES
overview training. The teachers from the Wikiwiki complex shared that the initial
implementation training consisted of the EES EO sharing the EES website and providing
handouts to the teachers. Unlike the aforementioned complexes, the teacher from the Mahina
complex received her initial training from her principal during Wednesday meetings. She shared
that:
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

86
Things were confusing at first but we figured it out. I felt a little better hearing the
information from my principal. It made me feel like if I had a question I could ask her
and she would know how to help me. She seemed to know what we were expected to do
and how we were going to be evaluated.
The initial training experiences of the teachers revealed that the teachers’ perceptions of the EES
were affected by the timing of the training and who delivered the training. Although teachers
appreciated receiving their training prior to the beginning of the school year, they also valued
receiving their initial trainings during the first semester in shorter training sessions.
Support
In addition to various types of initial training, the teachers also received various levels of
support throughout the first year. Teachers in the Mahalo complex received majority of their
support from members of the CAST. The CAST supported the teachers in writing and revising
their SLOs. Teachers in the Mahina complex received their support during afterschool
Wednesday staff meetings conducted by the school’s administration. During the meetings the
teachers shared that they “worked together to understand EES and support each other.” Teachers
in the Wikiwiki complex shared that the only type of support that they received was from their
administration, the curriculum coaches, and one another. One teacher said, “It was like the blind
leading the blind. We just asked each other what they did and copied that.” The teachers from the
Kokua complex shared that once they received their initial training they were left to work on the
components on their own. One teacher shared that:
Although we have a strong relationship with our principal we didn’t want to bother her.
We knew that she had a lot of things to do and we weren’t sure if she would be able to
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

87
answer our question without having to ask someone else. We didn’t want to give her any
more work.
The interviews and focus groups revealed similar findings regarding the initial trainings, teachers
from the five data collection sites received various types and levels of support from EES EOs,
administrators, and colleagues.
The teachers experiences were corroborated with the EES EOs perceived role in
implementation of the EES and consistencies and inconsistencies emerged. The initial trainings
developed and spoken of by the EES EOs were received as intended. As presented in the
findings of Research Question 1, the Mahina complex EES EO used the train the trainers model
and the teacher in the Mahina complex received almost all or her training and support from her
principal. Similarly, the EES EO from the Kokua complex also worked on building capacity at
the school level to provide on going support for teachers. Unfortunately, the teachers did not feel
as supported by their administration throughout the first year as the EES EO had intended which
was revealed in their discomfort with asking their principal for help. Further discussion with the
Kokua EES EO revealed that some principals in his complex decided to hire a Curriculum Coach
for the 2014-2015 school year to provide additional instructional support for teachers and to
alleviate some of the EES responsibilities. The EES EO from the Mahalo complex built capacity
at the complex level within the CAST team and teachers within the complex felt supported by
the CAST team throughout the school year. It is unclear if the type of training the teachers in the
Wikiwiki complex received was what was intended because at the time of data collection the
Wikiwiki complex EES EO was unable to be reached.
It cannot be determined at this time, which type of training or support was found to be
most effective. However, the data indicates that teachers who received support throughout the
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

88
school year from their EES EO or their school administration were able to help make
connections between the EES process and what was already being done at the school. For
example, the teachers in the Mahalo complex spoke of using the Data Team cycle as data
collection points for her SLO. Additionally, the teachers in the Kokua complex used the SLO to
support the work they were doing with their Instructional Leadership Team. When asked how
they decided to connect the SLO with the existing initiative they shared that their EES EO
suggested it at the SLO training.
Summary
Chapter 4 presented emergent themes from data collection. Although the teachers did not
feel adequately prepared to implement the EES during the 2013-2014 school year there were
components that they felt more comfortable with.
The first research question sought to explore teachers’ perceptions of the EES system as
it was implemented in the 2013-2014 school year. The data indicates that teachers did not feel
adequately prepared to implement the EES due to inadequate knowledge regarding expectations
and procedures as well a lack of support throughout the school year. The teachers’ experiences
with the EES were further influenced by feelings of frustration regarding inconsistencies within
the Charlotte Danielson observations, unclear expectations, and short timelines for the SLO.
Although the teachers shared frustrations with regards to the other components of the EES such
as the Tripod survey, Core Professionalism, and Hawaii Growth Model the implementation
training they received for the SLO component proved to be the most inadequate. The teachers’
experiences were corroborated with those of the EES EO to identify inconsistencies. The
findings revealed that both parties recognized that the initial trainings were not sufficient for full
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

89
implementation for reasons including ongoing changes to expectations and procedures and lack
of support throughout the first semester.
The second research question explored which factors contributed to the teachers’
perceptions of the implementation of the EES. The interviews and focus groups revealed that the
lack of clarity, initial trainings, and ongoing support were factors that influenced teachers’
perceptions. More specifically, as the teachers received their initial implementation trainings
expectations were being clarified and changes were being made. In addition to changes, the
teachers also engaged in various types of implementation trainings and support plans. Some of
the EOs approached the implementation training and support as train the trainers model while
others worked directly with teachers at the school level. One possible inference is that the types
of training approach taken by the EES EOs play a more significant role in implementation than
previously thought.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

90
CHAPTER 5
DISCUSSION
The purpose of this study was to investigate, analyze, and describe the teachers’
perceptions of the Educator Effectiveness System within the Hawaii Department of Education.
The experiences of the individual elementary school teachers, groups of elementary school
teachers, and Educational Officers were described and analyzed to determine what agreements
and discrepancies in perception existed. Chapter 4 presented the findings for the following
research questions:
1. What are the perceptions of teachers regarding the Educational Effectiveness System
as it was implemented?
a. How were teachers prepared for the implementation of the Educator
Effectiveness System?
b. How were teachers supported through the implementation of the Educator
Effectiveness System?
c. How do teachers perceive the Educator Effectiveness System has influenced
their instruction?
2. Which factors contributed to the perceptions of teachers regarding the Educator
Effectiveness System as it was implemented?
Chapter 5 will briefly summarize the findings, suggest implications for practice and
research, and discuss limitations of the study, and provide the conclusion of the dissertation.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

91
Summary of Findings
In response to Research Question 1 the findings were as follows:
• Teachers did not feel adequately prepared to implement the Educator Effectiveness
System during the 2013-2014 school year
• Teachers received similar trainings but delivery methods and timelines varied
between complexes
• Teachers felt the least prepared to implement the SLO component and the most
prepared to implement the Charlotte Danielson component
In response to Research Question 2, the findings were as follows:
• Unclear expectations, type of initial training, and access to ongoing support
influenced teachers perceptions
Implications for Practice
The findings revealed several implications for practice, because the perceptions of the
teachers and EES EOs revealed commonalities that provided insight in way of improving the
effectiveness of implementing the EES. This study revealed three implications for further
consideration as the State of Hawaii continues to improve the EES. The first implication involves
clarifying expectations and procedures at the State level prior to training implementation and the
second involves building capacity at the school level to better support the classroom teachers.
The first implication was that decisions regarding expectations must be clarified prior to
implementation training. Although the EES EOs provided administrators and teachers with new
and revised information as it was provided to them, the changes, modifications, and additions
proved to be a challenge. As teachers learned about the evaluation system and worked to make
sense of it, changes were being made furthering confusion and frustration with the high stakes
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

92
evaluation. This implication is supported by the research of Longenecker et al. (1994), which
states that the employees understanding of the measurement process is a critical component of
accepting any evaluation-tool and its outcomes. Moreover, the acceptance of an evaluation tool
increases the likelihood that the employee will be motivated to use the feedback they receive to
change their behavior (DeNisi & Pritchard, 2006).
A second implication is that that the State considers building capacity at the school level
to offer more support for teachers with regards to EES protocols, SLO development, and
instructional practices. The literature review in Chapter 2 illustrated that the rater also influences
the success of a performance appraisal systems. Roberts (1994) asserted that the task of the rater
in the appraisal process is to fairly and accurately observe and produce a comprehensive rating of
the ratee. Bernardin (1979) and Gosselin et al. (1997) found that ratees identified their immediate
supervisors as the most trusted and preferred individuals to be primarily responsible for their
performance appraisal. Building the capacity of school level administrators will increase their
level of understanding and enhance their ability to support teachers, which will allow teachers to
see their administrators as more knowledgeable evaluators and thus more credible ratees.
Moreover, building capacity at the school level would provide additional means of calibrating
evaluation tools to ensure that teachers are receiving objective evaluations.
In addition to building evaluator capacity, a third implication from this study is that
teachers would benefit from working with their administration and curriculum coaches to
develop a plan to improve their instruction based on the feedback they received from their
evaluations. More specifically, Danielson (2011) states that credible evaluations allow teachers
to engage in productive conversations about their practice. Involving curriculum coaches in these
conversations would give teachers the opportunity to develop a plan to improve their practice
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

93
and subsequent student learning. Wise et al. (1984) and Darling-Hammond et al. (2011) also
support engaging school level personnel in providing teachers with mentoring and coaching
opportunities as their feedback and support has been shown to motivate teachers to improve their
practice based on their evaluation outcomes.
Implications for Research
The findings of this study also exposed possible research topics, and those are highlighted
in this section. The first area of potential future research is the method of implementation
training provided. Across the state teachers received their initial trainings differently. Some
receive it during two full day trainings while others received it in smaller parts spread out over a
few months during full workdays and afterschool meetings. It is unclear at this time if either type
of training had an impact on teachers’ perceptions or understanding of the EES but additional
research in this area could be used to identify any correlations.
The findings also suggest that additional research regarding the trainer would help to
improve future implementation trainings. The research presented in Chapter 2 highlights the
importance of including employees in developing and operating the appraisal system (Roberts,
1994). However, little research has been done to understand what role the appraisal system
trainer plays employee perception of the evaluation system. Exploring how the trainer affects the
employees’ perception and understanding of the evaluation system would provide useful
information for organizations when developing implementation plans.
Within the Hawaii Department of Education additional research regarding
implementation training and EES document quality would also shed light upon the effectiveness
of the training and support each complex area provided. More specifically, since the SLO proved
to be the most challenging component of the EES for teachers, a closer look at this element
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

94
would provide the state and teachers with valuable information for future development. Samples
of SLOs can be collected throughout the state, evaluated against a set criteria to determine its
quality and compared to the type of training they received to identify trends and patterns. A
closer investigation into the outcomes of complex trainings would also facilitate further learning
and collaboration between complexes.
Limitations
There are several limitations that were not anticipated in Chapter 1 that were revealed
throughout this study. The most significant limitation that arose was the small sample size.
Initially data was to be collected from groups of five teachers from eight schools across the state
but the participation rates were low for classroom teachers. Of the nine schools contacted three
schools responded with a single participant, two responded with two and five teachers
respectively, and one school did not respond. Additional emails and letters were sent to elicit
additional participants with no success. Participation from additional teachers would have
provided additional perspectives which would have added to the overall extensiveness of the
data.
Prior to approaching schools each Complex Area Superintendent (CAS) was approached
to approve the study and give their consent to collect data within their complex. All but one CAS
granted permission to conduct research within their complex. This limited the amount of data
that could be collected from the teachers as well as the EES EO.
The final unexpected limitation of this study was the limited number of remaining EES
EOs responsible for initial implementation during the 2013-2014 school year. When the EES
implementation training was rolled out during the summer of 2013 there were eight EES EOs
(one from each complex area) responsible for implementation training. At the start of the 2014-
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

95
2015 school year four remained, one of which was in the complex that access was denied. The
number of available EES EO limited the data related to how the trainings and support systems
were designed for the teachers across the state.
Conclusion
Teachers are the most significant school based factor to improve student achievement and
learning (Darling-Hammond, 1999). To improve student achievement and in response to the
Race to the Top Competition, the Hawaii Department of Education implemented the Educator
Effectiveness System during the 2013-2014 school year. Research has found that a critical
instrument for improving teacher effectiveness is teacher evaluation (Danielson & McGreal,
2001; Howard & McColskey, 2001). Successful teacher evaluation systems are designed and
implemented to provide teachers with meaningful feedback that allow teachers to engage in
productive conversations about their practice (Danielson, 2011). Additionally, research (Roberts,
1994, 2002; DeNisi & Pritchard, 2006; Levy & Williams, 2004) has found that critical
components of an effective appraisal system are employee participation, acceptance and
collaboration.
This study was designed to investigate elementary school teachers’ perceptions of the
implementation of the EES during the 2013-2014 school year. The results indicate that teachers
did not feel adequately prepared to implement the EES. Although all teachers participated in
initial EES training with an emphasis on the Charlotte Danielson component and the SLO, the
findings revealed that teachers struggled with making sense of the SLO process and ongoing
changes to procedures and receiving a limited amount of support throughout the year. Interviews
with complex area EES Educational Officers revealed that similar to the teachers’ experience,
they were also plagued with ongoing changes and expectations. In addition to the
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

96
aforementioned challenges, the EES EOs were also tasked with designing and implementing
their own trainings for the teachers within their complex, which resulted in a variety of
approaches being used throughout the state. It is not clear which or if any of the implementation
training approaches was more effective than the other, but teachers that received ongoing support
from their administration or a CAST member found were able identify benefits to the Educator
Evaluation System.
The extensive amount of research pertaining to designing and using employee appraisal
systems highlights and confirms the importance, value, and challenges evaluation systems
present. The implementation of the Hawaii Department of Education EES was riddled with
challenges, but the results of this study indicate that there were practices that teachers found to
be useful, such as ongoing support and the ability to connect current practices with new
evaluation measures. As the Hawaii Department of Education moves forward with the EES, the
results of this study could be used improve to improve teachers’ perceptions of the EES by
improving implementation training and increasing ongoing support.

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

97
REFERENCES
American Federation of Teachers. (n.d.). History. Washington, D.C.: Author. Retrieved from
http://www.aft.org/about/history
Autor, D., Levy, F., & Murnane, R. (2003). The skill content of recent technological change: An
empirical investigation. Quarterly Journal of Economics, 118, 1279-1333.
Bauer, T., & Erdogan, B. (2012). Organizational socialization outcomes: Now and into the
future. In C. Wanberg (Ed.), The Oxford handbook of organizational socialization (pp.
97-112). New York, NY: Oxford University Press.
Beam, D. R., & Conlan, T. J. (2002). Grants. In L. M. Salamon (Ed.), The tools of government: A
guide to the new governance (pp. 340-380). New York, NY: Oxford University Press.
Becker, B., & Gerhart, B. (1996). The impact of human resource management on organizational
performance: Progress and prospects. Academy of Management Journal, 39(4), 779-801.
Becker, G. (1962). Investment in human capital: A theoretical analysis. The Journal of Political
Economy, 70(5), 9-49.
Becker, G. S. (2002). The age of human capital. In E. P. Lazear (Ed.), Education in the twenty-
first century (pp. 3-8). Stanford, CA: Hoover Institution Press.
Belbin, C., Erwee, R., & Wiesner, R. (2012). Employee perceptions of workforce retention
strategies in a health system. Journal of Management and Organization, 18(5), 742-760.
Bernardin, H. J. (1979, November, 15). A study to identify feasible appraisal systems for
employees of the U.S. Geological Survey (Final technical report to the U.S. Geological
Survey). Washington, D.C.: Government Printing Office.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

98
Blair, M. (2011). An economic perspective on the notion of human capital. In A. Burton-Jones &
J.-C. Spender (Eds.), The Oxford handbook of human capital (pp. 334-357). New York,
NY: Oxford University Press.
Bogdan, R. C., & Biklen, S. K. (2007). Qualitative research for education: An introduction to
theories and methods (5th ed.). Boston, MA: Allyn and Bacon.
Boxall, P., & Purcell, J. (2011). Strategy and human resource management (3rd ed.).
Basingstoke, UK: Palgrave Macmillan.
Brandt, C., Mathers, C., Oliva, M., Brown-Sims, M., & Hess, J. (2007, November). Examining
district guidance to schools on teacher evaluation policies in the Midwest Region (Issues
& Answers Report, REL 2007-No. 030). Washington, D.C.: U.S. Department of
Education, Institute of Education Sciences, National Center for Education Evaluation and
Regional Assistance, Regional Educational Laboratory Midwest. Retrieved from
http://ies.ed.gov/ncee/edlabs/regions/midwest/pdf/REL_2007030_sum.pdf
Byerly, B. (2012). Measuring the impact of employee loss. Performance Improvement, 51(5),
40-47. doi:10.1002/pfi.21268
Cardy, R. L., & Lengnick-Hall, M. L. (2011). Will they stay or will they go? Exploring a
customer-oriented approach to employee retention. Journal of Business and Psychology,
26(2), 213-217.
Center for Advanced Study of Teaching and Learning. (2014). Classroom Assessment Scoring
System
TM
. Charlottesville: University of Virginia, Curry School of Education. Retrieved
from http://curry.virginia.edu/research/centers/castl/class
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

99
Cheng, Y. C. (2004). Fostering local knowledge and human development in globalization of
education. The International Journal of Educational Management, 18(1), 7-24.
doi:10.1108/09513540410512109
Clark, R. E., & Estes, F. (2008). Turning research into results. Charlotte, NC: Information Age.
Coff, R. W. (2002). Human capital, shared expertise, and the likelihood of impasse in corporate
acquisitions. Journal of Management, 28(1), 107-128. doi:10.1177/014920630202800107
Common Core State Standards Initiative. (2014). About the Standards. Retrieved from
http://www.corestandards.org/about-the-standards/
Council of Chief State School Officers. (2014). Who we are. Washington, D.C.: Author.
Retrieved from http://www.ccsso.org/Who_We_Are.html
Crook, T. R., Todd, S. Y., Combs, J. G., Woehr, D. J., & Ketchen Jr, D. J. (2011). Does human
capital matter? A meta-analysis of the relationship between human capital and firm
performance. Journal of Applied Psychology, 96(3), 443.
Danielson, C. (2001). New trends in teacher evaluation. Educational Leadership, 59(5), 12-15.
Danielson, C. (2007). Enhancing professional practice: A framework for teaching (2nd ed.).
Alexandria, VA: Association for Supervision and Curriculum Development.
Danielson, C. (2011). Evaluations that help teachers learn. Educational Leadership, 68(4), 35-39.
Danielson, C., & McGreal, T. L. (2001). Teacher evaluation to enhance professional practice.
Alexandria, VA: Association for Supervision and Curriculum Development.
Darling-Hammond, L. (1999). Educating teachers: The academy’s greatest failure or its most
important future? Academe, 85(1), 26-33.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

100
Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E. H., & Rothstein, J. (2011). Getting
teacher evaluation right: A background paper for policy makers. Washington, D.C.:
National Academy of Education.
Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E. H., & Rothstein, J. (2012). Evaluating
teacher evaluation. Phi Delta Kappan, 93(6), 8-15. doi:10.1177/003172171209300603
Darling-Hammond, L., & Barron, B. (2008). Powerful learning: What we know about teaching
for understanding. San Francisco, CA: Jossey-Bass.
Darling-Hammond, L., & Haertel, E. (2012, November 5). A better way to grade teachers. Los
Angeles Times. Retrieved from http://articles.latimes.com/2012/nov/05/opinion/la-oe-
darling-teacher-evaluations-20121105
Darling-Hammond, L., & Post, L. (2000). Inequality in teaching and schooling: Supporting high-
quality teaching and leadership in low-income schools. In R. D. Kahlenberg (Ed.), A
notion at risk: Preserving public education as an engine for social mobility (pp. 127-
167). New York, NY: Century Foundation Press.
Darling-Hammond, L., Wise, A. E., & Pease, S. R. (1983). Teacher evaluation in the
organizational context: A review of the literature. Review of Educational Research, 53(3),
285-328.
DeNisi, A. S. (1996). Cognitive approach to performance appraisal: A program of research.
New York, NY: Routledge.
DeNisi, A. S., & Peters, L. H. (1996). Organization of information in memory and the
performance appraisal process: Evidence from the field. Journal of Applied Psychology,
81(6), 717-737. doi:10.1037/0021-9010.81.6.717
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

101
DeNisi, A. S., & Pritchard, R. D. (2006). Performance appraisal, performance management and
improving individual performance: A motivational framework. Management and
Organizational Review, 2(2), 253-277. doi:10.1111/j.1740-8784.2006.00042.x
Dillon, S. (2009, April 22). Large urban-suburban gap seen in graduation rates. New York Times.
Retrieved from http://www.nytimes.com/2009/04/22/education /22dropout.html
Dolton, P. (2010). Teacher supply. In D. Brewer & P. McEwan (Eds.), Economics of education
(pp. 251-259). San Diego, CA: Academic Press
Donaldson, M. L. (2009, June 25). So long, Lake Wobegon? Using teacher evaluation to raise
teacher quality. Washington, D.C.: Center for American Progress. Retrieved from
https://www.americanprogress.org/issues/education/report/2009/06/25/6243/so-long-
lake-wobegon/
Duncan, A. (2010, June 2). Statement on National Governors Association and State Education
Chiefs Common Core Standards. Washington, D.C.: U.S. Department of Education.
Retrieved from http://www.ed.gov/news/press-releases/statement-national-governors-
association-and-state-education-chiefs-common-core-standards
Dychtwald, K., Erickson, T. J., & Morison, R. (2006). Workforce crisis: How to beat the coming
shortage of skills and talent. Boston, MA: Harvard Business School Press.
Echols, M. E. (2007). Learning’s role in talent management. Chief Learning Officer, 6(10), 36-
40.
Eide, E. R., & Showalter, M. H. (2010). Human capital. In D. Brewer & P. McEwan (Eds.),
Economics of education (pp. 27-31). San Diego, CA: Academic Press.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

102
Ellett, C. D., & Teddlie, C. (2003). Teacher evaluation, teacher effectiveness and school
effectiveness: Perspectives from the USA. Journal of Personnel Evaluation in Education,
17(1), 101-129.
Flamholtz, E. G., & Lacey, J. (1981). The implications of the economic theory of human capital
for personnel management. Personnel Review, 10(1), 30-40.
Ford, J., & Weissbein, D. (1997). Transfer of training: An updated review and analysis.
Performance Improvement Quarterly, 10(2), 22-41.
Frank, F. D., Finnegan, R. P., & Taylor, C. R. (2004). The race for talent: Retaining and
engaging workers in the 21st century. Human Resource Planning, 27(3), 12-25.
Glazerman, S., Goldhaber, D., Loeb, S., Raudenbush, S., Staiger, D. O., & Whitehurst, G. J.
(2011, April 26). Passing muster: Evaluating teacher evaluation systems. Washington,
D.C.: The Brookings Institution, Brown Center on Education Policy. Retrieved from
http://www.brookings.edu/research/reports/2011/04/26-evaluating-teachers
Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating teacher effectiveness: A research
synthesis. Washington, D.C.: National Comprehensive Center for Teacher Quality.
Gosselin, A., Werner, J. M., & Halle, N. (1997). Ratee preferences concerning performance
management and appraisal. Human Resource Development Quarterly, 8(4), 315-333.
Govaerts, N., & Kyndt, E. (2010). Influence of learning and working climate on the retention of
talented employees. Journal of Workplace Learning, 23(1), 35-55.
Guest, D. (2011). Human resource management and performance: Still searching for answers.
Human Resource Management Journal, 21(1), 3-13.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

103
Gunz, H., & Gunz, S. (2007). Hired professional to hired gun: An identity theory approach to
understanding the ethical behaviour of professionals in non-professional organizations.
Human Relations, 60(6), 851-87. doi:10.1177/0018726707080079
Hawaii State Teachers Association. (2014, April 7). EES joint survey results. Honolulu, HI:
Author. Retrieved from http://www.hsta.org/index.php/news/ees-joint-survey-results
Hawaii State Teachers Association. (n.d.). About us. Honolulu, HI: Author. Retrieved from
http://www.hsta.org/who-we-are/about-us/
Hickman, D. C., & Olney, W. W. (2011). Globalization and investment in human capital.
Industrial and Labor Relations Review, 64(4), 652-670.
Hill, H. C., & Grossman, P. (2013). Learning from teacher observation challenges and
opportunities posed by new teacher evaluation systems. Harvard Educational Review,
83(2), 371-401.
Howard, B. B., & McColskey, W. H. (2001). Evaluating experienced teachers. Educational
Leadership, 58(5), 48-51.
Huang, F., & Cappelli, P. (2010). Applicant screening and performance-related outcomes. The
American Economic Review, 100(2), 214-218.
Huselid, M. A. (1995). The impact of human resource management practices on turnover,
productivity, and corporate financial performance. Academy of Management Journal,
38(3), 635-672. doi:10.2307/256741
Hytter, A. (2007). Retention strategies in France and Sweden. The Irish Journal of Management,
28(1), 59-79.
Jerald, C. D. (2009). Defining a 21st century education. Alexandria, VA: The Center for Public
Education.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

104
Joshi, A. (2012). Human capital management: Strategies for retention of productive work force.
International Journal of Management Research and Review, 2(11), 1943-1951.
Kirkpatrick, D. (1984). Two ways to evaluate your performance appraisal system. Training and
Development Journal, 38(8), 38-40.
Kirkpatrick, D. (1998). Evaluating training programs: The four levels. San Francisco, CA:
Berrett-Koehler.
Landy, F. J., Barnes, J., & Murphy, K. (1978). Correlates of perceived fairness and accuracy of
performance appraisals. Journal of Applied Psychology, 3(6), 751-754.
Lepak, D., Takeuchi, R., & Swart, J. (2011). Aligning human capital with organizational needs.
In A. Burton-Jones & J.-C. Spender (Eds.), The Oxford handbook of human capital (pp.
334-357). New York, NY: Oxford University Press.
Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and
framework for the future. Journal of Management, 30(6), 881-905.
Locher, A. H., & Teel, K. S. (1998). Assessment: Appraisal trends. Personnel Journal, 67, 139-
145.
London, M., Mone, E. M., & Scott, J. C. (2004). Performance management and assessment:
Methods for improved rater accuracy and employee goal setting. Human Resource
Management, 43(4), 319-336. doi:10.1002/hrm.20027
Longenecker, C. O., Scazzero, J. A., & Stansfield, T. T. (1994). Quality improvement through
team goal setting, feedback, and problem solving: A field experiment. International
Journal of Quality & Reliability Management, 11(4), 45-52.
Marzano, R. J., Frontier, T., & Livingston, D. (2011). Effective supervision. Alexandria, VA:
Association for Supervision and Curriculum Development.
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

105
Maxwell, J. A. (2013). Qualitative research design. Thousand Oaks, CA: Sage.
McGuinn, P. (2012). Stimulating reform: Race to the top, competitive grants and the Obama
education agenda. Educational Policy, 26(1), 136-159. doi:10.11 77/0895904811425911
McMahan, G. C., & Wright, P. M. (1992). Theoretical perspective for strategic human resource
management. Journal of Management, 18(2), 295-320.
doi:10.1177/014920639201800205
Merriam, S. B. (1998). Qualitative research and case study applications in education. San
Francisco, CA: Jossey-Bass.
Milanowski, A. T., & Heneman, H. G. (2001). Assessment of teacher reactions to a standards
based teacher evaluation system: A pilot study. Journal of Personnel Evaluation in
Education, 15(3), 193-212. doi:10.1023/A:1012752725765
Mittelstrass, J. (2001). New challenges to education and research in a global economy.
Prospects, 31(3), 393-398. doi:10.1007/BF03220079
Mueller, J. R., & Baum, B. (2011). The definitive guide to hiring right. Journal of Applied
Business and Economics, 12(3), 140-153.
The Nation’s Report Card. (2013). 2013 mathematics and reading. Washington, D.C.: U.S.
Department of Education, Institute of Education Sciences, National Center for Education
Statistics. Retrieved from http://nationsreportcard.gov/reading_math_2013/#/executive-
summary
National Center for Education Statistics. (2012a). Public high school graduation rates.
Washington, D.C.: U.S. Department of Education, Institute of Education Sciences.
Retrieved from http://nces.ed.gov/programs/coe/indicator_coi.asp
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

106
National Center for Education Statistics. (2012b). Mathematics 2011: National assessment of
educational progress at grades 4 and 8 (NCES 2012-458). Washington, D.C.: U.S.
Department of Education, Institute of Education Sciences. Retrieved from
http://nces.ed.gov/nationsreportcard/pdf/main2011/2012458.pdf
National Center for Education Statistics. (2013a). Selected findings from PISA 2012.
Washington, D.C.: U.S. Department of Education, Institute of Education Sciences.
Retrieved from http://nces.ed.gov/surveys/pisa/pisa2012/pisa2012highlights_1.asp
National Center for Education Statistics. (2013b). Trends in academic progress: Reading 1971-
2012; Mathematics 1973-2012 (NCES 2013-456). Washington, D.C.: U.S. Department of
Education, Institute of Education Sciences. Retrieved from
https://nces.ed.gov/nationsreportcard/subject/publications/main2012/pdf/2013456.pdf
National Center for Education Statistics. (2013c). Fast facts: Enrollment trends. Washington,
D.C.: U.S. Department of Education, Institute of Education Sciences. Retrieved from
https://nces.ed.gov/fastfacts/display.asp?id=65
National Center for Education Statistics. (2015a). Children living in poverty. Washington, D.C.:
U.S. Department of Education, Institute of Education Sciences. Retrieved from
http://nces.ed.gov/programs/coe/indicator_cce.asp
National Center for Education Statistics. (2015b). National Assessment of Educational Progress
(NAEP): Overview. Washington, D.C.: U.S. Department of Education, Institute of
Education Sciences. Retrieved from http://nces.ed.gov/nationsreportcard/about/
National Center for Education Statistics. (n.d.-a). Program for International Student Assessment
(PISA): Overview. Washington, D.C.: U.S. Department of Education, Institute of
Education Sciences. Retrieved from http://nces.ed.gov/surveys/pisa/
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

107
National Center for Education Statistics. (n.d.-b). Welcome to NCES. Washington, D.C.: U.S.
Department of Education, Institute of Education Sciences. Retrieved from
https://nces.ed.gov/
National Center for Education Statistics. (n.d.-c). Trends in International Mathematics and
Science Study (TIMSS): Overview. Washington, D.C.: U.S. Department of Education,
Institute of Education Sciences. Retrieved from https://nces.ed.gov/TIMSS/
National Commission on Excellence in Education. (1983). A nation at risk: The imperative for
educational reform. Washington, D.C.: Author. Retrieved from
http://www2.ed.gov/pubs/NatAtRisk/index.html
National Commission on Teaching & America’s Future. (1996). What matters most: Teaching
for America’s future. New York, NY: Author.
National Education Association. (n.d.). Our history. Washington, D.C.: Author. Retrieved from
http://www.nea.org/home/1704.htm
National Governors Association. (2011). About. Washington, D.C.: Author. Retrieved from
http://www.nga.org/cms/about
Nee, E. (2010, Spring). Joanne Weiss. Stanford Social Innovation Review. Retrieved from
http://www.ssireview.org/articles/entry/qa_joanne_weiss
No Child Left Behind (NCLB) Act of 2001, Pub. L. No. 107-110, § 115, Stat. 1425 (2002).
Obama, B. (2009, February 24). Remarks of President Barack Obama — Address to Joint
Session of Congress. Washington, D.C.: The White House. Retrieved from
http://www.whitehouse.gov/the_press_office/Remarks-of-President-Barack-Obama-
Address-to-Joint-Session-of-Congress
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

108
Odden, A. R. (2011a, April 1). Manage ‘human capital’ strategically: Managing people wisely
should be at the core of all district improvement work. Education Week. Retrieved from
http://www.edweek.org/ew/articles/2011/04/01/kappan_odden.html
Odden, A. R. (2011b). Strategic management of human capital in education: Improving
instructional practice and student learning in schools. New York, NY: Routledge.
Organisation for Economic Co-operation and Development. (n.d.). About the OECD. Paris,
France: Author. Retrieved from http://www.oecd.org/about/
Papay, J. P. (2012). Refocusing the debate: Assessing the purposes and tools of teacher
evaluation. Harvard Educational Review, 82(1), 123-167.
Phillips, D. R., & Roper, K. O. (2009). A framework for talent management in real estate.
Journal of Corporate Real Estate, 11(1), 7-16.
Ployhart, R. E. (2012). Personnel selection and the competitive advantage of firms. In G. P.
Hodgkinson & J. K. Ford (Eds.), International Review of Industrial and Organizational
Psychology, 27, 153-196. Indianapolis, IN: Wiley.
Ployhart, R. E., & Kim, Y. (2014). Strategic recruitment. In D. Cable & K. Y. T. Yu (Eds.), The
Oxford handbook of recruitment (pp. 5-20). New York, NY: Oxford University Press.
Ployhart, R. E., & Moliterno, T. P. (2011). Emergence of the human capital resource: A
multilevel model. Academy of Management Review, 36, 127-150.
Pretorius, S. G. (2012). The implications of teacher effectiveness requirements for initial teacher
education reform. Journal of Social Sciences, 8(3), 310-317.
Roberts, G. E. (1992). Linkages between performance appraisal system effectiveness and rater
and rate acceptance. Review of Public Personnel Administration, 12(3), 19-41.
doi:10.1177/0734371X9201200303
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

109
Roberts, G. E. (1994). Maximizing performance appraisal system acceptance: Perspectives from
municipal government personnel administrators. Public Personnel Management, 23(4),
525-549.
Roberts, G. E. (2002). Employee performance appraisal system participation: A technique that
works. Public Personnel Management, 31(3), 333-341.
Rodriguez, R. (2008). Learning’s impact on talent flow. Chief Learning Officer, 7(4), 50-64.
Sanders, W. L., & Rivers, J. C. (1996). Cumulative and residual effects of teachers on future
student academic achievement. Knoxville: University of Tennessee, Value-Added
Research and Assessment Center.
Santibanez, L. (2010). Teacher incentives. In D. Brewer & P. McEwan (Eds.), Economics of
education (pp. 251-259). San Diego, CA: Academic Press.
Schiemann, W. A. (2006). People equity: A new paradigm for measuring and managing human
capital. Human Resource Planning, 29(1), 34-44.
Schultz, T. (1961). Investment in human capital. The American Economic Review, 51(1), 1-17.
Smarter Balanced Assessment Consortium. (2012). Smarter Balanced Assessment Consortium.
Retrieved from http://www.smarterbalanced.org/about/
Spring, J. (2008). Research on globalization and education. Review of Educational Research,
78(2), 330-363. doi:10.3102/0034654308317846
State of Hawaii Department of Education. (2011, April). Professional Evaluation Program for
Teachers (PEP-T): Manual for evaluators and participants (RS 11-1144). Honolulu, HI:
State of Hawaii Department of Education, Office of Human Resources, Performance
Management Section. Retrieved from
http://www.nctq.org/docs/Hawaii_Eval_Manual.pdf
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

110
State of Hawaii Department of Education. (2013, July). Hawaii Educator Effectiveness System:
Manual for evaluators and participants (RS 14-0017). Honolulu, HI: State of Hawaii
Department of Education, Office of Human Resources, Performance Management
Section. Retrieved from
http://www.hawaiipublicschools.org/DOE%20Forms/Educator%20Effectivness/Educator
%20Effectiveness%20System%20Manual.pdf
State of Hawaii Department of Education. (n.d.-a). Educator Effectiveness System. Honolulu, HI:
Author. Retrieved from
http://www.hawaiipublicschools.org/TeachingAndLearning/EducatorEffectiveness/Educa
torEffectivenessSystem/Pages/home.aspx
State of Hawaii Department of Education. (n.d.-b). Race to the Top. Honolulu, HI: Author.
Retrieved from
http://www.hawaiipublicschools.org/VisionForSuccess/AdvancingEducation/RaceToThe
Top/Pages/home.aspx
Toch, T., & Rothman, R. (2008). Rush to judgment: Teacher evaluation in public education.
Washington, D.C.: Education Sector.
U.S. Department of Education. (2009, November). Race to the Top program: Executive
summary. Washington, D.C.: Author. Retrieved from
http://www2.ed.gov/programs/racetothetop/executive-summary.pdf
U.S. Department of Education. (2010a, March). A blueprint for reform: The reauthorization of
the Elementary and Secondary Education Act. Washington, D.C.: Author. Retrieved from
http://www2.ed.gov/policy/elsec/leg/blueprint/blueprint.pdf
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

111
U.S. Department of Education. (2010b). Race to the Top program: Guidance and frequently
asked questions. Washington, D.C.: Author. Retrieved from
http://www2.ed.gov/programs/racetothetop/faq.pdf
Wagner, R., & Harter, J. K. (2006). Twelve: The elements of great managing. Washington, D.C.:
The Gallup Organization.
Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: Our national
failure to acknowledge and act on differences in teacher effectiveness (2nd ed.).
Brooklyn, NY: The New Teacher Project. Retrieved from
http://widgeteffect.org/downloads/TheWidgetEffect.pdf
The White House. (n.d.-a). About the Recovery Act. Washington, D.C.: Author. Retrieved from
http://www.whitehouse.gov/recovery/about
The White House. (n.d.-b). Reforming No Child Left Behind. Washington, D.C.: Author.
Retrieved from http://www.whitehouse.gov/issues/education/k-12/reforming-no-child-
left-behind
Wise, A. E., Darling-Hammond, L., McLaughlin, M. W., & Bernstein, H. T. (1984). Case
studies for teacher evaluation: A study of effective practices. Santa Monica, CA: Rand
Corporation.
Zepeda, S. (2002). Linking portfolio development to clinical supervision: A case study. Journal
of Curriculum and Supervision, 18(5), 83-102.
Zimmerman, T., Gavrilova-Aguilar, M., & Cullum, P. (2013). Rethinking human resource
strategies: A shift in the treatment of contingent workers. International Journal of
Business and Management, 8(7), 28-24. doi:10.5539/ijbm.v8n7p28

TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

112
APPENDIX A
TEACHER FOCUS GROUP INTERVIEW PROTOCOL

Interviewee: Date:
Time: Location:

Years with the DOE: Position: Years in position:

• What can you tell me about the training you received with regards to the implementation
of the EES?

• Did you feel prepared to pilot the EES system during SY 2013-2014? Why or why not?
• What component did you feel the most comfortable implementing? Why?
• What component did you feel the least comfortable implementing? Why?
• Were you supported throughout SY 2013-2014 with the implementation of EES? Please
provide examples

• What was your CD observation experience like? What type of feedback did you receive?
Did you understand the process and feedback you received?

• What was your second CD observation experience like? Was it any different from the
first?

• What was your experience with the first tripod survey results?
• What was your experience with the second tripod survey results?
• What was your experience with the SLO component?
TEACHER PERCEPTION OF EDUCATOR EFFECTIVENESS SYSTEM

113
APPENDIX B
EES EO INTERVIEW PROTOCOL

Interviewee: Date:
Time: Location:

Years with the DOE: Position: EES EO Years in position:

• What can you tell me about the training you received in regards to the implementation of
the Educator Effectiveness System?

• What was your role in preparing teachers for the implementation of the EES?
• How were you prepared to do this?
• Were you comfortable with this role?
• Why or why not?
• What is your understanding of the support teachers received during SY 2013-2014 with
the implementation of the EES?

• Were you involved with this? How?
• Do you think the EES has influence teacher instruction? How? Please give some
examples.

Abstract (if available)

Abstract This study applies the Four Levels of Evaluation from the performance evaluation literature to understand teacher’s perceptions of the implementation of an evaluation system. The purpose of this study was to investigate, analyze, and describe elementary school teacher’s perceptions of the implementation of the Educator Effectiveness System within the Hawaii Department of Education during the 2013-2014 school year. Using focus groups and interviews, the experiences and perceptions of nine teachers and three Educational Officers were explored. The experiences of the teachers and Educational Officers were analyzed to identify emergent themes. Findings from this study indicate that teachers did not feel adequately prepared to implement the Educator Effectiveness System during the 2013-2014 school year and received various types of implementation trainings and ongoing support during the implementation year. This study provides insight for further consideration as the State of Hawaii continues to improve the Educator Effectiveness System.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Teacher retention and classroom instructors who coach interscholastic athletics or advise extracurricular activities

PDF

The Hawai'i State Teachers’ Association and Race to the Top: HSTA’S role in the RTTT grant

PDF

Teacher perceptions of evaluation policy in Hawaii

PDF

An investigation of hiring English language learner paraprofessionals in Hawaii

PDF

Enhancing professional development aimed at changing teachers' perceptions of Micronesian students

PDF

Overcoming the cultural teaching gap: an evaluative study of urban teachers’ implementation of culturally relevant instruction

PDF

The role of international school teacher leaders in building leadership capacity within their teams

PDF

The vortex of homophobic bullying: the reporting behavior of teachers

PDF

Exploratory study on Race to the Top schools and the impact a school principal has on a school’s academic performance

PDF

Assessing the meaning and value of traditional grading systems: teacher practices and perspectives

PDF

The challenges teachers face effectively implementing science, technology, engineering, and mathematics (STEM) curricula: an evaluation study

PDF

A qualitative study on Hawaii's use of Race to the Top funding on extended learning time in a Zone of School Innovation

PDF

A case study analysis of high school principals' criteria in the teacher selection process

PDF

Understanding the teachers' perception of impeding bullying in the middle school classroom

PDF

Teachers' perceptions of an effective teacher evaluation system and its key components in China

PDF

Addressing the principal shortage: women teachers

PDF

The teacher evaluation: key components that positively impact teacher effectiveness and student achievement

PDF

Elementary STEM policies, practices and implementation in California

PDF

Teachers as bystanders: the effect of teachers’ perceptions on reporting bullying behavior

PDF

An exploratory study of professional development to improve student reading: a case study in Oahu Hawaii

Asset Metadata

Creator Fukada, Jana Une Hai (author)

Core Title Teacher perception of the implementation of the educator effectiveness system

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 09/15/2015

Defense Date 09/01/2015

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag OAI-PMH Harvest,teacher evaluation implementation

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Brewer, Dominic (committee chair), Datta, Monique (committee member), Picus, Lawrence (committee member)

Creator Email jfukada@usc.edu,juhfukada@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-180966

Unique identifier UC11274554

Identifier etd-FukadaJana-3907.pdf (filename),usctheses-c40-180966 (legacy record id)

Legacy Identifier etd-FukadaJana-3907.pdf

Dmrecord 180966

Document Type Dissertation

Format application/pdf (imt)

Rights Fukada, Jana Une Hai

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA