University of Southern California Dissertations and Theses

A case study of instructional supervision, including teacher evaluation, and the impact on teacher practice

(USC Thesis Other)

A case study of instructional supervision, including teacher evaluation, and the impact on teacher practice

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content A CASE STUDY OF INSTRUCTIONAL SUPERVISION, INCLUDING TEACHER EVALUATION, AND THE IMPACT ON TEACHER PRACTICE by Kevin Edward Astor A Dissertation Presented to the FACULTY OF THE ROSSIER SCHOOL OF EDUCATION UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF EDUCATION May 2005 Copyright 2005 Kevin Edward Astor Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UM I Number: 3180459 Copyright 2005 by Astor, Kevin Edward All rights reserved. INFORMATION TO USERS The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleed-through, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. ® UMI UMI Microform 3180459 Copyright 2005 by ProQuest Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P.O. Box 1346 Ann Arbor, Ml 48106-1346 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. DEDICATION This dissertation is dedicated to the Astor family - my grandfather, A.M. A., for starting a long history of education at the University of Southern California almost 90 years ago, my Uncle Harry and Aunt Margie for always providing encouragement and guidance, my big sisters, Laura and Susie, for always letting me know they were proud of me, and my parents Arthur and Antonia for providing everything I needed and more than I ever wanted. Mom and dad, all I have accomplished comes out of what you have given me. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ACKNOWLEDGEMENTS iii I acknowledge that God has blessed me to know every person listed on this page. Without His love the road would have been immensely more challenging. I owe all glory and honor to Him. I offer my most sincere expression of gratitude to my committee members: Chairperson, Dr. Stuart Gothold; Dr. Dennis Hocevar; and Dr. Greg Bowman, for the efforts they have put forth to support me in completing this endeavor at the highest level possible. Dr. Gothold the time and energy you have invested in me this past year is something for which I could never repay you. I want to extend my deepest admiration and appreciation to Dale Hillyer, without whom I cannot imagine having finished this three year journey. We have spent so much time together during this adventure that I am amazed we found the chance to start families. I cherish the memories we made and the friendship that continues to grow. To my colleagues at Cypress High School I am indebted. You have done a great deal to help me in this process. I thank you for giving me encouragement, time, and the reassurance that you were always ready and willing to provide support when I was feeling overwhelmed. Most importantly, I want to express my gratefulness for having taken this voyage with my wife, Diane, at my side. Diane, by the grace o f God you have been my comfort and sanctuary as I have battled long nights, short deadlines, and Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. seemingly insurmountable challenges. You have always been there to take the brunt of my frustration. I celebrate with you the completion of this degree, the arrival of our first child, and all the blessings that God has designed for us. You my love and my best friend. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE OF CONTENTS v DEDICATION ii ACKNOWLEDGEMENTS iii LIST OF TABLES viii LIST OF FIGURES ix ABSTRACT x CHAPTER 1 THE PROBLEM 1 Introduction 1 Statement of the Problem 6 Purpose of the Study 8 Research Questions 9 Importance of the Study 9 Limitations, Delimitations and Assumptions 10 Definitions 11 2 REVIEW OF THE LITERATURE 14 A Historical Perspective 14 The Nature of Supervision and Evaluation 21 Beginning Teacher Support and Assessment 27 California Standards for the Teaching Profession 30 California Formative Assessment and Support System for Teachers 32 Processes to Improve Teacher Practice Involved in Successful Supervision 33 Issues of Public Opinion and Governance 36 General Practices 38 Effective Program Models 41 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. vi TABLE OF CONTENTS (continued). CHAPTER METHODOLOGY 48 Purpose of the Study 48 Research Questions 49 Sample and Population 53 Instrumentation 58 Data Collection 60 Data Analysis 65 THE FINDINGS 68 Introduction 68 Data Collection Tools 68 Data As Viewed by the Four Frames 74 The Structural Frame 74 The Human Resource Frame 79 The Political Frame 88 The Symbolic Frame 93 Emergent Themes 95 Common Focus 96 School Wide Instructional Strategies 97 Collaboration 98 Instructional Feedback 98 Response to the Research Questions 100 Research Questions 100 Summary 104 5 CONCLUSIONS, RECOMMENDATIONS AND IMPLICATIONS Introduction Purpose of the Study Summary of the Findings Recommendations for Practice Implications for Further Research BIBLIOGRAPHY 105 105 107 108 112 115 117 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. vii TABLE OF CONTENTS (continued). APPENDIX A TEACHER SURVEY 122 B DOCUMENT REVIEW FORM 128 C OBSERVATION TEMPLATE 130 D THE INTERVIEW 132 E FREQUENCY REPORT FROM SURVEY 137 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. viii LIST OF TABLES TABLE 1. School Enrollment by Grade Level. 54 2. School Enrollment by Racial/Ethnic Category. 54 3. Student Performance from 2003-2004 STAR API Report. 56 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. LIST OF FIGURES FIGURE 1 Conceptual Framework 2 Six Steps of Data Analysis Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ABSTRACT x The purpose of this study was to investigate the impact of various elements of instructional supervision on teacher classroom practice with emphasis on the role played by the formal teacher evaluation process. For the purposes of this study, instructional supervision was defined to include any programs, policies, or practices, formal and informal, which support instruction within the classes. Grants providing support in academic planning and execution, the role of program support personnel, strategies implemented school wide, and the perception among the teaching staff o f program and policy implementation are examples of what was investigated at the school site. This study is part of thematic group including 13 individual studies. One public high school, grades 9 through 12, was chosen as the subject of this case study. The school was chosen for possessing two major characteristics: high performing (as defined by having met or exceeded California Academic Performance Index targets on state testing for the past three years or more) and urban (defined as having a majority of non-white, socioeconomically disadvantaged students). These parameters ensured an examination of a successful program whose success can be more heavily attributed to instructional practice as opposed to a wealthy community that provides academic support. Four data collection tools were used to collect information: observation, document review, a survey, and interviews. The school site was visited seven times and various Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. xi campus activities were observed. Documents such as the teachers’ bargaining agreement, School Accountability Report Card, and forms used in the evaluation process were reviewed. A survey was administered to the teaching staff of 88 individuals with a return rate of 33%. Six interviews were conducted. The participating staff consisted of the principal, the Categorical Programs Coordinator, and four teachers from three different departments with three, seven, eight, and nine years o f experience at the school site. Analysis of the data generated by all four tools indicates a strong, common focus of student achievement among the staff, school wide instructional strategies, high levels of staff collaboration, and a desire for critical, instructional feedback. The study concludes that the formal evaluation is not potent in terms of improving instructional practice, but what is cited to be effective could be incorporated into the process. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 1 1 THE PROBLEM Introduction There are almost as many school improvement initiatives or models for improvement in the nation as there are school districts. A library holdings search at one university for the key words “school improvement” generated 1,024 listings. The No Child Left Behind initiative and similar policies have every school official seeking the most direct path leading to higher student achievement. The common factor contributing to student success continues to be the teacher. Multiple studies have shown that more skilled teachers lead to higher student achievement (Brophy, 1986; Ebmeier, 2003; Goddard, Hoy & Hoy, 2000; Stronge & Tucker, 2000). The California academic environment is no exception. Student achievement can be approached in two different ways with regards to teaching practices. Schools can either search for new programs to implement, offering different strategies that may be more effective than current practices, or schools can focus on modifying what is already being done and policies already in place to generate improved teacher, or instructional, practices. Supervision of instruction includes components such as professional development, instructional approaches, learning communities, and the evaluation of teachers. This last aspect of instructional supervision is reviewed extensively in the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 2 literature. California State Educational Code 44660 through 44665 mandates that every school district defines a set of standards by which all certificated teachers will be evaluated - every year for those not permanent and at least every other year for those who are tenured. The section of the educational code, 44662(b), generally outlines the content of the evaluation. It is to include sections regarding student progress towards established standards, instructional techniques and strategies, adherence to curricular objectives, and the establishment and maintenance of a suitable learning environment. The educational code goes on to address, briefly, the timeline in implementing the evaluation. Beyond this, the districts, and to a certain degree, the individual school sites have the opportunity to construct the evaluation process in such a way that best serves their goals and objectives. To properly comprehend the dynamics of instructional supervision today, it is necessary to understand how it has evolved. John Smyth (as cited in Reitzug, 1997) explains that the very word “supervision” has Latin roots to the term that meant to scan or review a text for errors and ways that it differed from the original. The literature clearly tells a story of how supervision began as an inspection by lay people, transitioned into an inspection by officially appointed bureaucrats, and is now most commonly seen more as collaboration with a strong element of inspection still remaining (Ebmeier & Nicklaus, 1999; Ellett & Teddlie, 2003; Siens & Ebmeier, 1996; Sullivan & Glanz, 2000). In a study included in their book, Glickman, Gordon and Ross-Gordon (2004) generated a list of words that teachers associate with “teacher supervision.” This, in turn, generated some insight Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 3 in how strong the inspection element has endured with words like control, directive, big brother, dog and pony show, paperwork, and authority, among others. Cogan and Goldhammer led others in the development of clinical supervision. At its essence, clinical supervision took a collaborative approach between teacher and supervisor in which there was a pre-conference discussion regarding goals, an in-class observation, and a post-conference to discuss what was observed citing areas of mastery and areas of growth with plans to attain that growth (Acheson & Gall, 2003; Ebmeier & Nicklaus, 1999; Sullivan & Glanz, 2000; Van der Linde, 1998). As Sullivan and Glanz (2000) pointed out in their book Supervision That Improves Teaching, there are two forces at play. There is traditional evaluation that dates back to the Industrial Revolution and the beginnings of bureaucratic inspection pitted against performance improvement of clinical supervision, based on democratic practices. Many other sources note this tension between what is termed formative and summative evaluation (Chow, Wong, Yeung & Mo, 2002; Gullatt & Ballard, 1998; Van der Linde, 1998; Zepeda, 2002). Formative evaluation is best characterized by the intent of the clinical supervision model. A staff works together with the over-arching goal of improving instructional practice - an ongoing dialogue that exists to examine what is currently happening and work toward an understanding of how it could be better. Summative evaluation, on the other hand, is more directly tied to the inspection roots. “The function of this type of evaluation is to determine whether to dismiss a Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 4 teacher, grant tenure, place on probation, or grant merit pay,” for example (Gullatt & Ballard, 1998,1). The literature speaks both indirectly and directly to a “crisis” in terms of the effectiveness o f the supervision and evaluation processes today (Sullivan & Glanz, 2000; Sullivan & Glanz, 2000) and address alternative approaches to traditional evaluation systems (Phillips, 2003; Zepeda, 2002). At the same time, there are no definitive studies identifying what works well and what can be built upon with regards to the supervision and evaluation process. Multiple textbooks talk about effective implementation of clinical supervision, but there are no inside views on how the supervision and evaluation processes are working to improve teacher practices, and, eventually, student achievement (Glickman, Gordon & Ross- Gordon, 2004; Sullivan & Glanz 2000). Even with that, Reitzug (1997) criticized a great number of textbooks of supervision for presenting a theory of shared inquiry and collaboration embedded in a reality of hierarchical inspection. Reitzug (1997) went on to cite Smyth who points out in his 1991 article that an apparent contradiction exists when clinical supervision is imposed upon a teacher to diagnose and give direction on how to fix the deficiencies. This conflict has led many districts to adopt new supervision programs structured to facilitate collegial participation, shared vision, self-reflection, and in many cases, an increased focus on student achievement. Heneman and Milanowski (2003) described a program in Cincinnati, Ohio that consists of various domains with multiple standards that are measured by rubrics. There is a portfolio Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 5 component along with multiple classroom observations. There is a differentiation for beginning and tenured teachers, with tenured teachers having the comprehensive assessment with a less intensive version for the “o ff’ years. Interestingly enough, the process still leads to a summative decision that carries punitive consequences such as non-renewal, placement on remediation, etc. Another program mentioned extensively in the literature is the Professional Assessment and Comprehensive Evaluation System (PACES) from Miami-Dade County Public Schools (Davis, Ellett & Annunziata, 2002; Ellett & Teddlie, 2003; Peterson, Kelly & Caskey, 2002). According to the Miami-Dade County Public School’s website, “The primary focus of PACES is to promote collaboration among teachers to advance teaching and learning through continuous professional growth.” Donaldson and Stobbe’s (2000) article in Thrust fo r Educational Leadership seems to be the only documentation of a modification of pre-existing program to meet the challenges of what the last decade or so has produced to define effective supervision. The article focused on how the Pajaro Unified School District is directly addressing California’s state mandated evaluation system and augmenting it to create a vehicle that will carry the district to meet its goals of dramatic impact on student achievement. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Statement of the Problem 6 It is not clearly known how the application of a teacher evaluation process impacts teacher practice; nor is it known what factors within instructional supervision other than the teacher evaluation process are affecting teacher practice. The literature is replete with examples of teacher evaluation systems (Glickman, Gordon & Ross-Gordon, 2004; Strong & Tucker, 2000; Sullivan & Glanz, 2000; Zepeda, 2003) offering model after model but falls short of a complete contextual investigation describing what factors of evaluation models impact teacher practice within a school setting; including the other components of instructional supervision and organizational culture. Schools are complex organizations that, like other complex structures, have countless elements that don’t exist in isolation, but rather influence each other in a continual interplay. The literature lacks detailed, qualitative research providing a rich and thick description of instructional supervision with an emphasis on the teacher evaluation process. There have been some strong investigative articles regarding specific supervision programs adopted by districts to meet current challenges and deficiencies (Davis, Ellett & Annunziata, 2002; Donaldson & Stobbe, 2000; Ellett & Teddlie, 2003; Peterson, Kelly & Caskey, 2002). Still, with individual state control over evaluation requirements, multiple, state-specific studies, if available, would be invaluable for districts to move forward. There also appears to be a shortage of documented examples showing how to modify existing policy, as Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 7 opposed to exchanging it for an entirely new program. Policy change in California would be more powerful if there was a clear understanding of how California schools are improving teacher practice through instructional supervision. It is trite to say that looking at a problem from multiple angles is quite beneficial in understanding the core issue, or issues, as well as developing a solution. With this in mind, the educational community is remiss in not seeking out, in greater detail, the perspective of teachers with regards to teacher supervision. Ebmeier (2003) was very clear in his study of teacher efficacy and commitment that the “Teacher’s belief in the importance principals attached to the teachers’ instructional activities seemed to be of great value in predicting teacher efficacy and, indirectly, teacher commitment” (110). Although studies have begun to consider, and even focus on measuring teacher perspectives (Chow, Wong, Yeung & Mo, 2002; Ovando, 2001), there is still a great need to document how teachers perceive all aspects of supervision and measure that against intentions as clarified in policy. Lastly, there are elements of a school’s culture that indirectly and directly influence policies and programs that impact teacher learning and improved practice within a school site. Schools are complex organizations with diverse cultures that influence what form teacher supervision takes on and its level of success (Davis, Ellett & Annunziata, 2002; Ebmeier, 2003). The culture needs to be examined through a specific lens, such as the Four Frames of Bolman and Deal (2003), to distill and analyze specific elements and their effect on the supervision process. In Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 8 terms of effective opportunities for teacher growth, which have grown informally out o f the culture, there is a deficiency in understanding how informal practices coexist with the formal policy and procedure and could potentially be replicated and continue on a formal level. Purpose of the Study The purpose of this case study is to understand and describe the policies and procedures governing the evaluation and supervision of teachers in one high- performing, urban high school, and the perceptions of those policies and procedures held by the teaching and administrative staff with regards to their impact on teacher practices. The policies and procedures will be those emanating from the district, as well as the site, and affecting the evaluation process of certificated employees and other support elements o f instructional supervision, such as professional development opportunities. This study will also examine any informal structures that afford teachers an opportunity to grow and improve in their classroom practices. Finally, this study will seek to describe the school’s organizational culture through the lens o f Bolman and Deal’s (2003) Four Frames to understand what elements o f the culture impact the overall teacher supervision process in both negative and positive ways. At this stage in the research, the evaluation and supervision o f teachers will be generally defined as instructional supervision. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Research Questions 9 1. What is the district’s policy and strategy for carrying out teacher evaluation? 2. How does the school carry out teacher evaluation? 3. What factors have shaped this school level effort? 4. How effective is the teacher evaluation process at the school level? Importance of the Study This study could potentially have far reaching effects. With improved instructional supervision generating increased growth in teacher practices, and stronger teacher practices producing higher levels of student achievement, all school stakeholders benefit. Specifically, this study would provide deeper insight for teachers and administrators into what works. This can allow teachers to approach the process more intentionally - knowing what elements will generate the most impact. If teachers have a clear understanding of what brings true growth and improvement in the instructional supervision process, they will perceive the activities in which they are directed to engage to be more valuable - a major ingredient in efficacy. Administrators are not only able to engage in the process with more confidence and more purposefully make choices that bring on the most growth, they can influence district policy to formalize what is working on an informal, maybe inconsistent, basis. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 10 Lastly, the policy makers can get a window into what effectively produces improvement in teacher practices. State legislature can consider programs that go well beyond the mere inspection of whether or not teachers are meeting a minimum level of operating criteria. Regardless of the schools that strive beyond the level to which they are required, and the negative image of state mandated standards and testing requirements, the truth is that state mandated expectations lead to effort and much achievement at schools statewide that might not occur otherwise. This study has the potential to discover and articulate what works and could work well with the teacher supervision and evaluation process to make the process more meaningful and powerful. Limitations, Delimitations and Assumptions The phenomenon at the core of this study is the high level of success achieved by some urban schools as compared to their similar schools as defined by the California Testing and Accountability program. This success is determined by high student achievement. O f all the factors contributing to increased student achievement, the study is delimited to a description of the impact o f the teacher evaluation process and other elements of instructional supervision on improved teacher practice within one grade 9 to 12 public high school that can be characterized as high performing and urban, a sample of the population of K-12, high-performing, urban, public schools. The correlation of improved teacher Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 11 practice to student achievement is assumed by evidence presented in the literature. This study is limited by the characteristics of the population. The purposive sampling may decrease the extent to which the findings can be generalized to K-12, public schools. In addition, the study is also limited in that the data could give rise to other interpretations beyond what is included. Definitions Academic Performance Index (A.P.I.) - The numerical representation of a school’s “academic progress and growth” (http://www.cde.ca.gov, retrieved 8/10/04). “It is a numeric index (or scale) that ranges from a low of 200 to a high of 1000” (http://www.cde.ca.gov/ta/ac/ap/ apidescription.asp, retrieved 8/13/04). The score is made up of performance indicators including the STAR program (the CSTs and CAT/6 Survey) the California High School Exit Exam. Cornell Notes - A note-taking system in which notebook paper is divided into two columns: one for documenting information and the other to write questions or comments. The final component is a summary completed at the end of a note-taking session. Four Frames - Intellectual framework developed by Bolman and Deal (2003) to sort the various elements of an organization into the categories of Structural, Human Relation, Political, and Symbolic. These categories or “frames” Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 12 can be thought of as . . windows, maps, tools, lenses, orientations, and perspectives . . (Bolman & Deal, 2003, 12). High-Performing - Schools that have met or exceeded their Academic Performance Index score for the last 3 years and have scored a seven or higher on their similar schools ranking. Instructional Supervision (also listed Supervision of Instruction) - includes all activities and policies that impact both content curriculum and methods of instruction. Examples are, but not limited to, professional development plans, curriculum committees, learning communities, etc. STAR Program - The Standardized Testing and Reporting (STAR) “... program is to help measure how well students are learning basic academic skills” (http://www.cde.ca.gov. retrieved 8/12/04). The last test offering in the spring of 2003 included the California Standards Tests (CSTs) and the California Achievement Test, Sixth Edition Survey (CAT/6 Survey). Similar Schools - Schools that are grouped together for the basis of comparison as required by the Public Schools Accountability Act. Schools are determined to be “similar” by specific demographic characteristics including: pupil mobility, pupil ethnicity, pupil socioeconomic status, percent of teachers who are fully credentialed, percent of teachers who hold emergency credentials, percent of pupils who are English learners, average class size per grade level, and whether the schools operate multi-track year-round educational programs (http://www.cde. ca.gov/ta/ac/ap/documents/simschl03b.pdf. retrieved 3/1/05). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 13 Similar Schools Rank - “This information shows where a school ranks academically on a scale of 1-10, compared with 100 other schools with similar demographic characteristics” ten being the best (http://www.cde.ca.gov/ta/ac/ap/ documents/simschl03b.pdf, retrieved 8/10/04). Teacher Evaluation - refers to the district/school’s implementation of state mandated review of teachers’ performance as outlined in state educational code 44660 through 44665. Urban - The population of schools determined to be “urban” are those with large populations o f students coming from low socioeconomic households and families of ethnic minority. The non-white population of the school could be a diverse collection or predominantly one ethnicity. Teachers Assisting Students to Excel in Learning Mathematics (TASEL-M) Project - refers to a project grant that partners Cal State Fullerton and multiple school sites, including Grove High School and its feeder school sites. The grant monies pay for a full-time support educator, release time for teachers, professional development, and supplies used in classroom instruction. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 2 REVIEW OF THE LITERATURE 14 A Historical Perspective The history o f instructional supervision, the monitoring of classroom practices, and the evaluation of teachers can be characterized as a long road of growth from a humble by-product of formalized schooling to a focus o f research, debate, policy, and contract negotiation. Understanding where supervision has been helps to comprehend the current policy and practices in California. Ellett and Teddlie (2003) summarized the evolution of the evaluatory component to supervision quite well: [Evaluation] is as old as the education system in the USA and it has been through many trends and cycles as roles of teachers have changed, as values and beliefs about effective teaching and teacher responsibilities have changed, as perceptions of how students best learn have changed, and as societal demographics and teaching contexts have changed (103). Supervision of our public schools, in an official sense, emerged around the turn of the century coming out of a desire for organization and structure that was heavily influenced by the industrial revolution (Ebmeier & Nicklaus, 1999; Sullivan & Glanz, 2000). The acceleration and praise for the industrial age and factory model of organization inspired the hierarchical structure with regards to school supervision (Ebmeier & Nicklaus, 1999). While the early forms of supervision were carried out by lay committees, this gave way to control by officially appointed Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 15 bureaucrats (Glickman, Gordan & Ross-Gordan, 2004). It is important to understand that early supervisors were better classified as inspectors (Siens & Ebmeier, 1996) and, up through to the 21st century, the practice of supervision has not yet been completely divorced from it. Smyth traces the term “supervision” to a medieval definition of, “a process of perusing or scanning a text for errors or deviations from the original text” (1991, 121). Many early superintendents used their supervisory tools to perpetuate this search for “deviation” or deficiency among teachers. “Nineteenth century supervisors, for the most part, saw teachers as inept” (Sullivan & Glanz, 2000, 3). Sullivan and Glanz cited an 1894 document written by T.M. Balliet, a Massachusetts superintendent, who clearly communicated that educational reform was easily done by hiring an expert superintendent who would identify the inadequate teachers and lead them to improvement and cast away those for whom there was no hope. For decades the prevailing method of improving instruction was to find the deficiencies among teachers (Reitzug, 1997) and eliminate them by changing or removing the teacher. As with many processes, standardization is the first goal. This was achieved, to a degree, early on as the “inspector,” or supervisor, looked for basic implementation of the curriculum (Ebmeier & Nicklaus, 1999). The supervisor was the expert and the teacher was not involved in developing goals or focusing on a purpose (Gullett & Ballard, 1998). Ebmeier and Nicklaus discussed how the gain in standardization was a loss in collaboration and Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. formative inquiry - teachers fearing negative impacts of poor evaluations avoided seeking help from supervisors or colleagues. The structure on which supervision was being built relied heavily on what was to become a behavioralist perspective. Sullivan and Glanz (2000) pointed to Franklin Bobbitt as the initiator of “scientific management” which he identified as the key to controlling what teachers did in schools to produce the best outcomes. Supervision was still seen as the best method of control (Siens & Ebmeier, 1996). Siens and Ebmeier’s article provided an expansion of this thought to included the perspective that saw, “... increased control over the curriculum as [a way] to deal with these teacher deficiencies” (1996, 300). Throughout this era, the supervisor remained the expert prescribing the most effective “behaviors” and approaches to the teachers under their responsibility. It can be likened to a trainer who looks to elicit certain actions and behaviors on command and in certain contexts. Supervision, and the teacher evaluation process specifically, were limited in their effectiveness as the focus became that of teacher behavior and teacher performance, (Ellett & Teddlie, 2003) measured against what the bureaucratic leadership proclaimed to be appropriate. The prevailing, prescriptive approach of control found opposition early in James Hosic’s battling against the image of supervisor as leader of a work crew when he put forth his model of the consultant who conferences (as cited in Sullivan & Glanz, 2000). Concepts and perspectives, such as these, lead to a transformation into what Sullivan and Glanz term “democratic supervision” (2000, 14). “Democratic Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 17 supervision, in particular, implied that educators, including teachers, curriculum specialists, and supervisors, would cooperate to improve instruction” (15). The next great leap towards modem supervision was in the application of scientific thought in gathering data. “... [T]he idea that supervision involved improving instruction based on classroom observation gained momentum” (17). Ebmeier and Nicklaus documented how these two perspectives began to grow together and in the mid 20th century a number of educators began to view, “supervision as a vehicle for the professional development of the teacher through observation, conferencing, and feedback cycles” (1999,123). This allowed, “teachers to take active roles in determining the focus of their own growth and shifted the function of supervision from inspection to a vehicle for active experimentation and individual development” (124). The introduction of clinical supervision by Cogan, Goldhammer and others, in many ways, marks the birth of modem supervision and evaluation, as it is practiced in many school districts today. Clinical supervision came out of dissatisfaction with educational practice (Sullivan & Glanz, 2000) in the 1950s and 1960s. “The premise o f clinical supervision was that teaching could be improved by a prescribed, formal process of collaboration between teacher and supervisor” (19). Cogan (1973) defined clinical supervision as: . . . the rationale and practice designed to improve the teachers’ classroom performance. It takes its principal data from the events of the classroom. The analysis of these data and the relationship between teacher and supervisor form the basis of the program, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 18 procedures and strategies designed to improve the students’ learning by improving the teacher’s classroom behavior (9). Clinical supervision continued to be refined and modified within educational settings (Zepeda, 2003). The influence of the clinical model slowly changed the thrust of supervision. The goal became to engage in reflective practice as opposed to adhering to a prescribed list of behaviors (Siens & Ebmeier, 1996) and supervisors began to acknowledge the contextual element to instruction (Ebmeier & Nicklaus, 1999). A major shift in thought regarding clinical supervision is seen in how it, “... was originally designed to continue in cycles, with each cycle (pre observation, observation, and post-observation) informing future cycles and identifying the activities needed to help teachers meet their learning objectives” (Zepeda, 2003, 58). Sergiovanni and Starrat (1983) defined clinical supervision as the constant revision or refining of instructional practices directed by the overarching goal of professional improvement. This reference to the “refining” process speaks to the “cycles” much like the silversmith who must engage in repeated cycles of purification to produce an ever improving metal. Carroll (1997) referred to Goldhammer’s definition where, “the word clinical is used to indicate face to face interaction between teacher and supervisor” (6). Carroll further explained that, “clinical supervision is a structured system of observing teachers and conferencing with teachers in order to improve teaching and school effectiveness” (6). The structural conception of clinical supervision still remains Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 19 the pre-conference to discuss goals and objectives, the observation of teaching in context, and a post-conference for dialogue and reflection upon the findings and their implications (Carroll, 1997; Glickman, et al., 2004; Sullivan & Glanz, 2000; Zepeda, 2003). Pajak’s (2002) four families of clinical supervision, including characteristics such as humanistic, technical, didactic, and reflective show the extent to which this concept has been embraced, permeated both educational research and practice and is reflective o f so many elements of humanity. The clinical supervision model was as influential in California as it was in many other parts of the nation. The foundation of contemporary, state-mandated evaluation and site- executed formal supervision is the Stull Act of 1971 which formed the educational code that has evolved and still governs the teacher evaluation process in California’s public K-12 schools. Enacted on July 20, 1971 as Chapter 361 of the Educational Code of the State of California, the act was a bold step into accountability for California educators (Flanigan, 1975). The Stull Act dealt with tenure, but Article 5.5 had for its central theme, evaluation and assessment. Article 5.5 mandates the development by local school districts of a plan for the evaluation and assessment of performance of certificated employees; the plan must include the following elements: (1) The establishment of standards of expected student progress in each area of study and of techniques for the assessment of that progress; (2) Assessment of certificated personnel competence as it relates to the established standards; Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 20 (3) Assessment of other duties normally required to be performed by certificated employees as an adjunct to their regular assignments; (4) The establishment of procedures and techniques for ascertaining that the certificated employee is maintaining proper control and is preserving a suitable learning environment” (Flanigan, 1975, 1). This has been modified, slightly, to form today’s mandated teacher evaluation procedure found in California Educational Code Sections 4460-44665. The educational code states, in part, the following: It is the intent of the Legislature that governing boards establish a uniform system of evaluation and assessment of the performance of all certificated personnel within each school district of the state .... The governing board of each school district shall establish standards o f expected pupil achievement at each grade level in each area of study .... The governing board of each school district shall evaluate and assess certificated employee performance as it reasonably relates to: (1) The progress of pupils toward the standards established . (2) The instructional techniques and strategies used by the employee. (3) The employee’s adherence to curricular objectives. (4) The establishment and maintenance of a suitable learning environment, within the scope of the employee’s responsibilities” (retrieved from www.leginfo.ca.gov on 6/18/04). This portion of the educational code served and continues to serve to officially articulate the basic elements of the evaluation procedures for the majority of school districts. It was very common to meld this content with a delivery structure built on the concepts of the clinical supervision model discussed earlier. Supervision and evaluation are terms that, although they carry significant distinctions, will continually be tied together in the minds of many educators. The Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 21 following sections attempts to shed light on the characteristics that define both, which will, in turn, help to clarify two significant aspects to supervision and evaluation in California, the Beginning Teacher Support and Assessment and the California Standards for the Teaching Profession, that have developed in recent years. The Nature of Supervision and Evaluation Harold Spears (1953) stated very clearly, “Supervision is and always will be the key to the high instructional standards of America’s public schools” (462). McQuarrie and Wood (1991) defined supervision as the method by which instruction is monitored with a focus to encourage teachers to grow and cultivate their instructional practices while supporting them to this end. According to Zepeda (2003), “Instructional supervision aims to promote growth, development, interaction, fault-free problem solving, and a commitment to build capacity in teachers” (89). Although it can be argued that instructional supervision creates the space for teacher evaluation to take place (Gullett & Ballard, 1998), it should by no means be considered the same in theory. Tunison (2001) appropriately quoted Ellis from a 1986 article in which he explains that a well structured supervision system, that is supported by all participating staff members, and focused on improving teacher practice, “. . . can be an effective and dynamic agent for educational renewal” (86). Supervision is Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 22 looking at the big picture, not just documenting teaching performance against a minimum expectation. Van der Linde (1998) shared that, “continuous improvement, or ‘kaizen’, as the Japanese call it, is a necessity in order to raise the quality of the teacher’s classroom practice” (331). Zepeda (2003) cited numerous articles to come up with a list describing the intentions of instructional supervision that includes ongoing learning, building relationships, improved classroom practice that generates higher levels of student learning, etc. Supervision can also be a vehicle to foster reflective practice that can support teachers in professional inquiry, increase their comprehension of appropriate teacher practices, and provide a deeper teaching repertoire (Siens & Ebmeier, 1996, 302). To investigate instructional supervision and not discuss teacher evaluation is like studying the game of football while ignoring the forward pass - they’re not synonymous, but the latter is integral in having a complete understanding of the larger picture. According to Zepeda (2003), “The intents of evaluation are to meet state statutes and district policies, assign teachers a rating at the end of the year, and determine whether a teacher will return to work” (92). Stronge (1997) suggested that, “the two most frequently cited purposes of personnel evaluation are accountability and performance improvement” (3). Districts are constantly striving to develop the most effective teacher evaluation program and the greatest motivation behind this is seeking the highest student success possible (Ovando, 2001). Davis, Ellett and Annunziata (2002) took the time to point out the very simple concept that teacher evaluation holds enormous value in that its primary Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 23 focus on ensuring proficiency in teacher practice represents a large potential to cultivate teachers’ skills. Teacher evaluation is the mandated arm of supervision. As Danielson and McGreal (2000) stated, “virtually every public school district, by order of state law or regulation, has a formal procedure for the evaluation of teachers” (7). Throughout the literature, it becomes clear that often there is a disconnection between theory, policy, and practice. Evaluation of teachers is easily supported by rational argument because it proposes to ensure the best possible practitioners for our kids, just as we want the best doctors, for example. Despite this, negative perceptions of teacher evaluation abound. Some teachers see observation visits as meaningless interruptions of their teaching and have no real impact on their instructional practices (Tunison, 2001). Davis, et al. (2002) presented a very pessimistic view on the process as it is implemented in most schools. According to them it is clear that “... opportunities are being missed for assessment of the teaching and learning processes to support collegiality and collaboration, to identify professional growth needs, and to acquire the contextually specific data necessary to improve student learning” (287). Gullett and Ballard (1998) borrow from a Gitlin and Price article from 1992 by framing the problem of evaluation structurally with several deficits as a result. The hierarchal structure facilitates one-way communication where teachers are compared to a one-size-fits- all set of standards, and not much of the classroom context, outside of the observation(s) is considered. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 24 Although, when dealing with supervision and evaluation, it is hard to completely extricate one from the other, a fatal flaw is to forget their distinctions. The reality of practice is that many teachers consider supervision and evaluation to be the same (Holand & Garman, 2001; Reitzug, 1997; Zepeda & Ponticell 1998). Reitzug (1997) attributed the lack of distinction to the fact that the principal facilitates both processes. A major theoretical difference between the more general process of supervision and the more specific element of evaluation is the summative and formative nature. Danielson and McGreal (2000) simply defined summative evaluation as being, “for the purpose of making consequential decisions,” and formative evaluation, “for the purpose of enhancing the professional skills of teachers” (8). For the purpose of this investigation into the literature, any summative elements, whether termed supervision or evaluation are to be considered as evaluation. Conversely, those processes labeled formative are to be thought of as supervision, even if labeled evaluation. Egelson (1994) pointed out formative qualities such as ongoing, promoting professional growth, and improving practice. These include all the elements that cultivate improvement without leading to a final judgment. “Summative evaluation is a judgmental decision of the quality and worth of an individual teacher over a specified time frame” (Gullatt & Ballard, 1998, 1). Dagley and Orso (1991) mentioned accountability and minimum standards and Gullatt and Ballard stated, “The function of this type of evaluation is to determine whether to dismiss a teacher, grant tenure, place on probation, or grant merit pay” Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 25 (1). Educational systems are left with supervision, the big picture, which is meant to support growth in classroom practices without final judgment, and, the more focused evaluation, that labels effective or not, meets standards, or not and leads to determinations of employment status, position, pay, etc. Although the theory easily makes a distinction, in practice there is conflict in successfully executing both processes. Rational thought can lead to the assumption that to grow and increase an individual’s level of skill, whatever the context, it is necessary to take a risk. Athletes, students, professionals, anyone attempting to move closer to their potential needs a safe place to fail. Trapeze artists practice with a net to diminish the penalty for a failed maneuver. Systems that include an evaluation process that leaves no room for a safe, supervisory environment become an obstacle to the same goals for which they were designed. According to Evans (1992), “over the years, well-intentioned efforts to combine formative and summative teacher evaluation have created dysfunctional, counterproductive, personnel appraisal schemes” (61). In some of the worst-case scenarios, “supervision and evaluation are often practiced as the same; a single classroom observation towards the end of the year yields an immediate rating” (Zepeda, 2003, 92). In their study, Zepeda and Ponticell (1998) noted that, “if teachers perceived that the ultimate purpose of supervision was the completion of the required evaluation form, then it is not surprising that for about half the teachers in both states [referring to the subjects of the study] the result of supervision was Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 26 perceived to be” (140) a summative checklist without any of the key formative support that makes supervision successful. Tunison (2001) correctly identifies that the core of the tension arising when a supervisor attempts to facilitate a context of formative supervision when the teacher sees that supervisor as the ultimate word in all summative decisions made regarding that teacher. In light of the significant contrast between summative and formative supervision, educators such as Evans (1992) advocated a literal separation with the duties distributed to different supervisors. Many times one system takes over the other. Sullivan and Glanz (2000) discussed the concept of bureaucratic supervision (summative) and democratic supervision (formative) and how the bureaucratic mandates, rooted in inspection, supersede any formative efforts. Evaluation done to the exclusion of teachers has tainted the concept of supervision to such an extent that the Association for Supervision and Curriculum (emphasis added) considered a name change because many teachers felt alienated (Holland & Garman, 2001). Holland and Garman (2001) sited a Texas supervision process that uses a clinical supervision approach, but because of a mandate to check against prescribed “ideal” teacher behaviors, the system quickly morphs into a summative evaluation activity. The literature does support, though, strategies that allow supervision and evaluation to coexist is such a way that the summative and formative natures are reflected. Fenwick (2001) explained that defined programs can be implemented that require, “schools and staffs to invest time and resources in teachers’ learning, and not evaluation” (406) thereby creating space dedicated to formative activities. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 27 Evans (as cited in Gullatt and Ballard, 1998) advocates separating the two functions of summative and formative evaluation and assigning them to different individuals, stating that the qualities of formative evaluation are not compatible with summative evaluation’s concerns related to job security (3). Peterson (2000) listed 12 new focuses for the teacher evaluation process that can open up space for true formative supervision. Gullatt and Ballard (1998) proposed with confidence that: . . . since formative evaluation is an on-going process designed to improve the teacher’s performance, it is possible to combine summative and formative procedures into an integrated model. Each procedure informs and contributes to the overall goals of professional improvement and can direct the sequence of activities required in evaluation (5). There is no definitive structure to facilitate both formative and summative supervision in a school organization, but the literature seems to suggest that a clear understanding of the characteristics of each can allow for the process to separate. Beginning Teacher Support and Assessment In the mid and late 1980s, there was attention directed towards the numbers of new educators entering the ranks of teaching and the levels of success they encountered. A significant number of teachers were leaving the profession and, as Young put it, “the high attrition rate of beginning teachers nationwide has been attributed to various conditions” (1999, 1). Young continued by referring to Natale’s (1993) argument that decisions to leave education by new teachers are heavily impacted by feelings of isolation and the lack of perceived support from Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 28 colleagues filling a “mentoring” type of role. Olebe (2001) referred to the attention of policymakers drawn to the condition of new teachers in 1988. “Their interest was spurred by concerns about the lack of retention of new teachers in urban and rural environments, including the especially high turnover of new minority teachers” (72). California’s first formal step in addressing the issue was SB 148, the Bergeson Act, enacted to, “examine alternative molds for supporting and assisting the professional induction of first- and second-year teachers, and assessing their competence and performance in the classroom” (72). The California New Teacher Project (from 1988 to 1992) served as a pilot program funding 37 local programs to provide support and assessment for teachers in their first and second years and the result of this work led to the enactment of SB 1422, establishing the Beginning Teacher Support and Assessment (BTSA) program in 1992 (Olebe, 2001; Young, 1999). The BTSA web site enumerates the program goals: (1) Provide an effective transition into the teaching career for first- and second- year teachers in California. (2) Improve the educational performance of students through improved training, information, and assistance for new teachers. (3) Enable beginning teachers to be effective in teaching students who are culturally, linguistically, and academically diverse. (4) Ensure the professional success and retention of new teachers. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 29 (5) Ensure that a support provider provides intensive individualized support and assistance to each participating beginning teacher. (6) Improve the rigor and consistency of individual teacher performance assessments and the usefulness of assessment results to teachers and decision makers. (7) Establish an effective, coherent system of performance assessments that are based on the California Standards for the Teaching Profession. (8) Examine alternative ways in which the general public and the education profession may be assured that new teachers who remain in teaching have attained acceptable levels of professional competence. (9) Ensure that an individual induction plan is in place for each participating beginning teacher and is based on an ongoing assessment of the development of the beginning teacher. (10) Ensure continuous program improvement through ongoing research, development, and evaluation (retrieved from http://www.btsa.ca.gov/ BTSA basics.htm on 9/18/04). Olebe (2001) reported findings that showed new teachers participating in support programs were more likely to use powerful instructional strategies leading to higher student achievement, found more success in motivating diverse student population, and retention rates in the professional were higher as well. All support and direction established through BTSA are guided by data collected through multiple methods (Young, 1999) to achieve the clearest picture possible. The Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 30 element of assessment is achieved by way of a self-assessment survey, a portfolio, direct observation o f classroom practice, and conversations with the support providers. This data is then used in a number of components to provide support for the BTSA participant. These components include, but are not limited to, an assigned support provider (from the same school site, same academic discipline, or both), beginning teacher orientations, release time, workshops, etc. (Young, 1999). “Retention rates in the profession for first- and second-year BTSA teachers were approximately 93% across all programs in the 1999-2000 school year” (retrieved fr°m http://www.btsa.ca.gov/BTSA basics.htm on 9/18/04). It was also significantly noted that findings from an independent evaluation that year indicated that, “retention rates did not vary significantly among programs serving schools with different degrees o f urbanity, programs at different levels of maturity, or programs of different sizes” (retrieved from http://www.btsa.ca.gov/BTSA basics.htm on 9/18/04). California Standards for the Teaching Profession “From 1988 to 1992, the work of the California New Teacher Project (CNTP) revealed the need for a commonly understood set of expectations about the knowledge, skills and abilities required by beginning teachers” (Whittaker, Snyder & Freeman, 2001, 87). With continued state efforts, this evolved and they, “. . . revealed a need for common language and a new vision of the scope and Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 31 complexity of teaching that would enable teachers to define and develop their practice” (CCTC/CDE, 1997, 3). This was the genesis leading to the adoption of the California Standards for the Teaching Profession (CSTP). The CSTP serve and have served multiple purposes from supporting new teachers, guiding pre-service teacher education programs, influencing the state licensing of teachers, and providing the base structure on which to support the state mandated certificated evaluation process in many California districts. The following is a brief timeline of development: 1991 - 1992 The “six domains” are developed by the Far West Lab (now WestEd) 1993 - 1994 West Lab revises domains and adds sub-domains and reflective Questions 1994 - 1995 Statewide taskforce revises domains as “Draft Framework” 1995 - 1996 Draft Framework revised as the “California Standards for the Teaching Profession” (CSTP) 1997 The state officially adopts the CSTP Source: (Whittaker, et al., 2001, p. 88) Many key philosophical and pedagogical decisions went into the development of the CSTP. It was decided that all educators needed guiding standards, not just those beginning, and, recognizing the many contexts of teaching, the standards had to be applicable to all environments (Whittaker, et al., 2001), this had to be about educating teachers. “The group [State appointed task force reviewing and revising original domains that Far West Lab had developed] had no disagreement.. . that Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 32 the fundamental purpose of these standards should be to guide and support practice” (92). The California Standards for the Teaching Profession were officially endorsed by the State Board of Education in the summer of 1997, marking the official adoption (Whittaker, et al., 2001). The CSTP have come to be a foundational element of the supervision of teacher practice in California since. California Formative Assessment and Support System for Teachers To implement the goals of BTSA, all districts must have a system in place and for those without, the California Formative Assessment and Support System for Teachers (CFASST) was instituted (Olebe, 2001). CFASST represents the embodiment of current research supporting the inclusion of formative assessments in the supervision of instruction (Olebe, 2001), as is discussed in the next section. “... [T]his formative assessment system offers multiple opportunities for participating teachers to learn and demonstrate knowledge, understanding, and applications of the California Standards for the Teaching Profession (CSTP) through a structured series of twelve critical thinking activities designed to be used by participating teachers and their support providers” (retrieved from www.omsd.kl2.ca.us/teacher/cfasst.pdf on 9/18/041. As with the CSTP, a strength of the CFASST program is the flexibility in that it can serve the interests and needs of all teachers, regardless of the context in which they find themselves. The California Formative Assessment and Support System for Teachers answers the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 33 call found at the end of Whittaker, et al.’s article on the development of California’s teaching standards: “the tendency to impose mechanistic models on the complex and holistic process of teaching and learning must cease in the interest of children” (106). Processes to Improve Teacher Practice Involved in Successful Supervision The literature has identified multiple elements that are a part of successful supervision processes. Reflection, described as a key component to effective supervision of instruction and teacher growth in classroom practice, is pervasive (Costa & Kallick, 2000; Davis, et al., 2003; Ebmeier & Nicklaus, 1999; Sullivan & Glanz, 2000; Weasmer & Woods, 2003). According to Costa and Kallick, “To be reflective means to mentally wander through where you have been and to try to make some sense o f it” (2). Weasmer and Woods stated, “Through reflection, a teacher examines her or his pedagogy and the motive driving her or his planning, activities, and assessment” (65). Weasmer and Woods cited Schon’s theory on reflection in which he distinguishes between reflection-in-action, which focuses on experiences as they occur, and reflection-on-action, which allows teachers to analyze a past event to better understand or modify future strategies. Although it can happen naturally without clear, defined intention, there must be a structure for reflective practice to occur at the optimum level. Ebmeier and Nicklaus (1999) pointed to the process of supervision by indicating that, “... Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 34 the critical task o f the supervisor is to help teachers more successfully engage in reflective behavior, which is thought to be a necessary element of professionalism” (2). Costa and Kallick (2000) are very clear in their proposal that, “in teaching, as in life, maximizing meaning from experiences requires reflection” (1). They go on to list four main opportunities that structured reflection provides: the perspectives of others helps augment the meaning of a teacher’s work; meaning can be extrapolated beyond the context in which it occurred; it forces the participant to more strongly commit to change and modification of future implementation; and it provides a forum to document what is learned in order to share with others. Even though reflection can occur meaningfully in isolation, reflection in a culture of collaboration and collegiality has more potential. Collaboration is another powerful element to successful supervision. “Collaboration among and between teachers is often cited as a desirable process that serves to decrease teacher isolation, increase the exchange of pedagogical information, increase coordination, within departments and buildings, and promote job satisfaction in general” (Ebmeier & Nicklaus, 1999, 11). They go on to cite three powerful ways in which collaboration dramatically strengthens the supervision process. First, the exchange of thoughts focused on the teacher’s practice increases the feeling of relevancy. Second, the feeling of ownership increases because the teacher is part of the process from the beginning, as opposed to being a passive recipient. Third, the teacher participated in developing the goals and objectives instead of having them imposed by a supervisor and this increased the perception of the long-term benefit. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 35 There has always been an adversarial tension in the background of education between supervisors (or administration) and the teachers. This can be traced back to the elements of inspection that have already been discussed in this review of the literature. Collaboration is what can make the shift to a healthier climate. Zepeda (2003) pointed out that collaboration is built upon and perpetuates the feeling of unity towards making organization goals and meeting organizational challenges. “Healthy school cultures thrive in environments built through collaboration, trust, and care for the members of the school” (47). Collaboration is also the balancing point on which rests the necessity of bureaucracy and standardization versus the teachers’ desire to lead their own professional growth (Davis, et al., 2002). Simply put, the collaborative spirit must allow teacher and supervisor to work together, formulating the best path to meet goals and objectives within the established bureaucracy (Davis, et al., 2002). Education constantly operates under a spotlight of public scrutiny - from the local to the national level. Public educational agencies do not operate in isolation and accountability does not end with the board o f education. This next section speaks briefly to the impact of government policy and perspectives held by the American public. Although they are not the only driving forces in education reform, both must be considered, alongside other factors, to truly understand the larger context of education, regardless of the particular aspect of education on which the focus is being directed. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Issues of Public Opinion and Governance 36 Public schools exist within an atmosphere of accountability. There is a structure of reporting: teachers to administrators, administrators to school board members to the community, etc. Education in the United States exists with a constant background o f tension between the schools serving the public’s perception of what is needed and appropriate and the teachers having the space for control over their own professional practice (Davis, et al., 2002). The duality of summative and formative supervision is reflected in conflict with public opinion. “On one hand, teacher supervision is influenced by public pressures for greater accountability ...” (Fenwick, 2001, 402) while, “on the other hand, discourses of reflection and continuous growth have become ubiquitous . . . ” (402). Rose and Gallup (2003) presented some clear findings in the 35th Annual Phi Delta Kappa/Gallup Poll o f the Public’ s Attitudes Toward the Public Schools. “The data show what they have shown every year . . . [t]he public gives the school high marks, the closer people are to the schools” (44) - meaning, the stronger the connection, and, presumably, the better the link to the true nature of the school, the more positive the opinion. A majority of the public also sees that getting and keeping good teachers as problems for the schools (Rose & Gallup, 2003). Supported by this evidence, Wenglinsky (2000) was right in saying, “... policy makers are correct in emphasizing the importance of improving teacher quality as a mechanism for Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 37 improving student academic achievement” (7). It is difficult to ascertain with certainty when commission reports, government policy, and conference summaries are influencing public opinion or are the product of preexisting perceptions by the public. Darling-Hammond (1996) reviewed the National Commission on Teaching and America’s Future’s report entitled, “What Matters Most: A competent teacher for every child.” The title alone clearly indicates the thrust of the report is drawing attention to the impact that teachers make on student learning. “We propose an audacious goal... by the year 2006, America will provide all students with what should be their educational birthright: access to competent, caring, and qualified teachers” (as cited in Darling-Hammond, 1996,1). The reauthorization of the Elementary and Secondary Education Act of 1965 by President Bush’s signing into law the No Child Left Behind Act o f 2002, propelled every state of the union into a new era of accountability that put at least some of the spotlight on teachers. Increased student mastery of standards is the product of the law’s focus on “highly qualified teachers.” The policy is very clear on what is expected. As it is found in The Secretary’s Third Annual Report on Teacher Quality (retrieved from http://www.ed.gov/about/reports/annual/teachprep/ 2004/teacherquality pg5.html#chapter-1 on 9/18/04): The No Child Left Behind Act (NCLB) requires that all public school teachers of core academic subjects meet the highly qualified requirements of their state by the end of the 2005-2006 school year, and that new teachers in school programs serving high-need student populations (i.e., Title I-targeted assistance programs or school-wide program schools) meet the highly qualified requirements immediately. To be highly qualified, a teacher must Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 38 possess at minimum a bachelor's degree, have full state certification and demonstrate subject matter mastery in each subject taught. The directives of this policy fortunately fell in line with what the state of California was working on through initiatives such as BTSA and the CSTP, but the alignment process still had to take place. At this moment the additional governance factor of contract negotiations and unions comes into play. “The impact of collective bargaining since the authorizing legislation in 1975 has been pervasive: it plays a central role in each district’s decision making process” (EdSource, 1999, 1). District policy is not the sole decision of the elected trustees. “School districts can implement most new laws - from class size reduction to teacher evaluation - only after they have ‘bargained’ the impact” (EdSource, 1999,1). Federal mandates and policy effect state educational code. California Ed. Code falls under what is termed “prohibited” or “non-negotiable” items and cannot be brought to the bargaining table. Where collective bargaining comes in is when Educational Code leaves room for districts to develop a process that meets the outlined mandates. General Practices This paper has discussed how the literature presents both reflection and collaboration as potent ingredients in a successful supervision program. There are processes built upon these elements that serve as effective practices with regards to Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 39 the supervision o f instruction and specifically to the evaluation of teachers. The portfolio, for example, is emerging as a powerful tool (Zepeda, 2002). “A portfolio is an individualized, ongoing record of growth that provides the opportunity for teachers to collect artifacts over an extended period of time - an entire school year, even from year to year” (85). According to Sullivan and Glanz (2000), “[the] portfolio not only documents the development of innovative and effective practices, it is a central vehicle for the growth of the teacher through self-reflection, analysis, and sharing with colleagues through discussion and writing” (7-8). Tucker, Stronge, Gareis, and Beers (2003) mentioned numerous authors in their description of portfolios. Some of the key words are collection, purposeful, evidence, demonstration, documentation, and analysis. Tucker, et al. (2003) continued by acknowledging that the diversity in portfolio nuances allows for a flexibility in which portfolios can play a part in formative, on-going support, allowing for continued reflection and refining, as well as, more structured, summative evaluation that leads to decisions of promotion, retention, etc. “Teacher portfolios are appealing for many reasons including their authentic nature, recognition of task complexity, active involvement of participants, encouragement of reflection and self-assessment, and facilitation of collaborative interaction” (575). Zepeda (2002) presented a model for the use of portfolios that is integrated with the clinical supervision model to extend its effectiveness. At the same time, the portfolio supports differentiated learning, in that individual teacher needs would Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 40 help determine what was put into the portfolio, as well as, the fact that the portfolio adds clarity and purpose to the element of in-class observation (Zepeda, 2002). “Although it does not replace classroom observation, the portfolio extends and enhances the professional discussion by going beyond what is observed in the classroom on a given day” (Sullivan & Glanz, 2000, 8). Zepeda (2002) shifted the focus back to the value of reflection with a quote by John P. Murphy from a paper presented in 1994, “[the] power of a portfolio is found in its ability to become a tool for an individual to reflect on the real task of education - teaching. The most important outcome from a teaching portfolio is self-reflection” (92). The task of reflection can happen individually, but another door opens when it can be done in relationship with others. Collaboration has a powerful effect on improving teacher practice. One road to formalize this process is that of learning communities. Learning communities allow for a structure in which professional development can be an ongoing process. In the context of learning communities, “professional development also creates forums for teachers to have collegial conversations about curriculum programs as well as instructional problems and solutions” (Phillips, 2003, 242). Phillips pointed out a number of different activities in which teachers are able to collaborate from observing each other teach, to solving obstacles in teaching, to engaging in action research projects. The ongoing nature of learning communities fosters a type of rhythm among the staff members who can habitually engage in developing their classroom practices. “Learning communities create ‘spaces’ for teachers to form professional Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 41 relationships, to share information, and to provide collegial support. The creation of small, collegial communities of practice contrasts sharply with the traditional approach to staff development” (Phillips, 244). Study of the available literature also provides a window through which to view fully implemented programs that include various components previously mentioned. State regulations, culture, and history can influence what would and would not work with regards to the supervision of instruction. It is effective to dissect programs operating within the context of real teachers and students, as opposed to those floating in theory. Effective Program Models Tucker, et al. (2003) posited that, “... improving practice is the ultimate goal of teacher evaluation ...” (594). Ovando (2001) added, “. .. school districts across the nation are developing teacher evaluation initiatives that promise to foster teacher professional growth through comprehensive appraisal systems .. .” (218). In the Miami-Dade County Public Schools (M-DCPS), Davis, et al. (2002) reported on, “... a state of the art teacher evaluation system” (288) - the Professional Assessment and Comprehensive Evaluation System (PACES). The PACES program replaces the former, teacher-centered system that M-DCPS had been using for about fifteen years (Ellett & Teddlie, 2003). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 42 “The PACES was designed to promote the development of increasingly competent professional teachers with the ability to participate in a collegial, professional community of educators committed to continuous improvement of teaching and learning” (Davis, et al., 2002, 290). Beyond the collaborative element that is becoming so crucial, the Miami-Dade system shifts in its focus of what to measure. “The PACES is a learner-centered, classroom-based assessment and professional development system intended to improved the cultural context of teaching and learning in schools through new role taking on part o f school administrators and teachers” (290). The PACES program doesn’t tell the teacher how to teach but allows them to explore their context and be creative - the focus is on evidence o f student engagement and learning (Ellett & Teddlie, 2003). Ellett and Teddlie continued to describe the PACES program as being a deeper assessment than other programs - looking beyond just teaching. The PACES facilitates reflection and a transformation in school culture that “supports a community of leaders” (109). There is a shift from identifying prescribed teacher behaviors to assessing student outcomes (Ellett & Teddlie, 2003). For example, instead of documenting teacher did “x,” the phrase might read that students were engaged in “y.” “Thus, the language of the PACES requires a shift in focus for the evaluator from teacher performance to the active involvement of learners in the development of concepts” (109). The PACES program is designed in line with previously mentioned perspectives that summative and formative supervision can coexist. Key to its success is the emphasis on both Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 43 colleague collaboration and the facilitation of reflective conversations (Davis, et al., 2002). Policy 2.1.5 entitled “Teacher Growth, Supervision and Evaluation” from the Alberta, Canada website, reads as follows: A teacher employed by a school authority or ECS operator . . . is responsible for completing during each school year an annual teacher professional growth plan that: (I) reflects goals and objectives based on an assessment of learning needs by the individual teacher; (II) shows a demonstrable relationship to the teaching quality standard; and (III) takes into consideration the educational plans of the school, the school authority and the Government or the program statement of an ECS operator (retrieved from http://www.learning.gov.ab.ca/educationguide/pol- plan/polregs/215.asp on 9/19/04). Fenwick (2001) put it in simpler terms. She explained that the Alberta supervision policy, “... requires all teachers to maintain a teacher professional growth plan (TPGP), which is reviewed annually by a supervisor, and is to conform to government-specified teaching quality standards” (401). Fenwick goes on to explain that this program came out of a duality of goals - a clear set o f government standards guiding teacher growth and, at the same time, opportunities for reflection and continual, life-long learning. Although this is a mandated activity, the TPGP, as a whole, or in part, is not to be used for the purposes of evaluation (Fenwick, 2001). The union, after a rough start, ended up in favor of the policy because it created, “.. . a positive model of teachers as responsible professionals and self directed continuous learners, rather than a negative deficit model of teachers’ requiring regular evaluation to ensure satisfactory performance” (406). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 44 The TPGP contributed extensively to the push for formative supervision that is distinguishable from summative evaluation decisions. “In requiring schools and staffs to invest time and resources in teachers’ learning, not teacher evaluation, while upholding individual teachers’ rights to direct this learning, TPGPs grant a space of possibility or teacher supervision that is generative [and] liberating .. (Fenwick, 2001, 406). Despite this, there was apprehension as the process began. Many teachers admitted a strong level of anxiety when they realized a likelihood of not completing a goal they had set for themselves (Fenwick, 2001). The administration sensed the same and found that partnering with the teachers, focusing on praising, trying to communicate care, and being as flexible as possible, had a tremendous impact on the teachers’ perception and brought down their level of stress (Fenwick, 2001). “Thus, although the mandated regulation of TPGPs embeds certain potential tensions and contradictions, practicing administrators and teacher managed to work through these in ways that opened useful possibilities for school improvement” (418). A last example is another incident of educators realizing how essential collaboration and reflection are to creating the best environment possible for teacher growth. Pajaro Valley Unified School District has implemented a new approach to teacher evaluation, an approach based on continuous professional growth and assessment. The annual ritual of evaluation that used to take the time of administrators and teachers - with questionable effect - has changed. It is now a collaborative process, focused on teacher selection of a course of professional Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 45 growth designed to increase student success (Donaldson & Stobbe, 2000, 1). The Pajaro system will now be defined in terms of what the students are doing, how they are engaged, and to what levels they are achieving (Donaldson & Stobbe, 2000). The Pajaro system is built on the California Standards for the Teaching Profession, which was discussed earlier in this paper. The intent was to build on this and expand the scope of the system. One step was implementing a self- assessment component, utilizing the Continuum of Teacher Abilities, developed by UC Santa Cruz and the Santa Cruz New Teacher Project (Donaldson & Stobbe, 2000). Another component of the Pajaro system is the use of differentiated supervision. The district is aware that as teachers become tenured and grow in their profession, they need different support systems to continue improving their practice. There are three options for a tenured teacher. They can: “(1) Develop a coaching relationship with another teacher; (2) Develop a coaching relationship with an administrator; or (3) Develop a portfolio to share with other teachers and an administrator” (2). The last major element to the Pajaro system is that of inquiry. Both probationary and tenured teachers are to use the California Standards for the Teaching Profession to focus on an area o f growth - not to make up for a deficit, but rather to channel their development (Donaldson & Stobbe, 2000). The Pajaro district is realistic about the challenges that face education in terms of limited Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 46 resources of both time and money, and the constant barrage of external pressures from the Federal, State, and local levels. With all that in mind, Pajaro believes that the effort required to empower teachers to direct their own professional development, to investigate how particular teaching skills impact student learning, and to monitor and assess their own professional growth in collaboration with their colleagues and administrators is both valuable and necessary, and will be reflected in increased student achievement (3). The status of instructional supervision and teacher evaluation is a complex organism that constantly changes appearance depending where it is and continues to change as educational research deepens. In addition, every person shades the concepts differently depending on personal experience. The preceding pages offer some clarification to the how supervision and evaluation have developed, some characteristic differences, external pressures that affect both, and examples of each in theory and context. To a certain degree, the picture of supervision and evaluation will continue to change as additional individuals offer their own perspective. Continued research will hopefully serve to identify both positive and negative trends that prove true in various contexts allowing for a stronger and more definitive common language for continued discussion and exploration. A missing piece o f the puzzle is an in-depth examination of what is happening at a high-achieving public school in terms of instructional supervision - formally within the evaluation process and informally outside o f it. The intent of this study is to spend time at a school site that, despite facing social and economic challenges among its clientele, is successful as measured by California state testing Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Al data. Through observation and document review, the researcher will gain an understanding o f what classroom practices exist to improve instruction, and, by way of a survey and interviews, gather teacher perspective on what role, if any, general instructional supervision and the teacher evaluation process play in the equation. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 3 48 METHODOLOGY Purpose of the Study The purpose of this case study is to understand and describe the policies and procedures governing the evaluation and supervision of teachers in one urban high school, identified by its high rating when compared to its similar schools listing, and the perceptions of those policies and procedures by the teaching and administrative staff with regards to their impact on teacher practices. The policies and procedures are those emanating from the district, as well as the site, and affecting the evaluation process of certificated employees and other support elements of instructional supervision, such as professional development opportunities. This study examines the informal structures that afford teachers an opportunity to grow and improve in their classroom practices. Finally, this study describes the school’s organizational culture through the lens of Bolman and Deal’s (2003) Four Frames to understand what elements of the culture impact the overall teacher supervision process in both negative and positive ways. At this stage in the research, the evaluation and supervision of teachers is generally defined as instructional supervision. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 49 Research Questions 1. What is the district’s policy and strategy for carrying out teacher evaluations? 2. How does the school carry out teacher evaluation? 3. What factors have shaped the school level effort? 4. How effective is the teacher evaluation process at the school level? This study is one of fourteen case studies investigating the same phenomenon with the same research questions and the same data collecting tools. All the studies are part of the same Thematic Dissertation group. The hope is that any emergent themes common over multiple case studies will have increased generalizability and importance. Figure 1 illustrates the overall structure of what the study explored and within what context this took place. High performance is determined by the level of achievement o f the students. This success if most importantly measured by performance on the California State Testing and Reporting (STAR) system. Multiple factors can have an effect on how the students perform (some of them are listed in gray to the left). Working backwards from that point, this study focuses on the factor of effective teacher practice and assumes its correlation (supported by the literature) to student achievement. The study focuses on what impact the supervision of instruction, with emphasis on the teacher evaluation process, may or may not have on improving teacher practices. Figure 1 also illustrates the awareness that the supervision of instruction is influenced by both various level policies and the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. culture of the organization - in terms of the school and the larger district organization. 50 State & District Policy Organizational Culture Adm inistrative Supervision of Instruction at Site Level Teacher Evaluation Process Professional Development Opportunities A Learning communities 7 IMPROVED TEACHER PRACTICES Classroom Instructional Strategies Lesson / Curriculum Development Teacher Evaluation Process Assumed Correlation O th er fa c to rs: \ HIGH STUDENT ACHIEVEMENT •e/e. } Course grades Performance on High Stakes Testing ---- — - ------------ - High Performing Urban School FIGURE 1. Conceptual Framework Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 51 The driving force behind this study - to fully describe, within its natural context, the phenomenon of how components of instructional supervision impact teacher practice in a high performing urban school - is most strongly channeled through a multi-method case study. The researcher sought the richest description possible while generating data with the highest attainable levels of validity, reliability, and generalizability. Qualitative research is most effective when trying to understand a concept or phenomenon that is not heavily supported by research. “Qualitative research is exploratory and is useful when the researcher does not know the important variables to examine” (Creswell, 2003, 22). While quantitative researcher can effectively search for the correlation and causation between distinct elements, qualitative helps to sort through the context to identify the elements that are at play (Creswell, 2003). The researcher sought emergent themes that arose to the surface throughout the investigation and following analysis. Qualitative studies are also effective when existing perspectives and theories do not apply to specific samples or populations, such as California, urban schools, in this case. Among the different qualitative approaches, the case study was the most obvious vehicle to propel this study forward in the most effective fashion. According to Gall, Gall and Borg (2003) in the book Educational Research: An Introduction, the case study “. . . it is an in-depth study of instances of a phenomenon in its natural context and from the perspective of the participants involved in the phenomenon” (436). This allows for the potential application of the study’s research findings to other schools with similar characteristics. This study is Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 52 very interested in the teachers’ and administrators’ perceptions towards instructional supervision. Gall, Gall and Borg (2003) stated, “the study of individual’s interpretations of social reality must occur at the local, immediate level” (17). A case study allows for a deeper understanding o f a complex phenomenon or concept “as experienced by its participants” (Gall, Gall & Borg, 2003,438). This empowers the researcher to deliver the findings in such a way as to transport the audience, to some degree, into the school context that has produced the themes, enabling them to see the data in a more real and true light. Just as with a master craftsman, the right tools can forge truth and insight from blocks of stone or cuts of lumber. The researcher collected data through document review, surveys, interviews, and observations. Each method stands alone, serving its own purpose in the overall project, but together, all four approaches serve to support each other, shining a brighter light on what is found. The multi-method approach came out of the realization that all methods have both their limitations and biases. Combining methods could potentially minimize or cancel out completely the negative effects of any particular single method. This gave rise to the formal practice of triangulation. Triangulation increases the validity of the data and it allows one method to inform the other. For example, the survey administered in this case study can potentially identify surface trends that can be investigated in deeper detail through interviews and observations (Creswell, 2003). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 53 Sample and Population This study collects data from a public, K-12, high performing, urban school within the state of California. Time and resources limited the study to one school site. High-performing is defined as having met or exceeded the API growth targets for the past three years and ranking a seven or higher in Similar Schools. Urban is defined as schools with significant percentages o f the student body being qualified as socioeconomically disadvantaged and as being of minority, non-white ethnicities. Grove High School, in Grove, California, having met the outlined requirements, was chosen. Grove High School’s website holds an informative timeline. The school was voted into existence in 1903 serving four towns that have become, in essence, different areas of Grove, California. The first school building was constructed in 1905 and its present campus was opened in 1953. Grove High is one of four comprehensive high schools within the Grove Unified School District. Like many other schools within the county, it has seen dramatic change is pupil demographics in the last thirty years - a reflection of the county. The current demographic information is found in Tables 1 and 2. According to state data from the 2004 STAR school report, of the 1,593 students tested, 1,012, or 64%, were categorized as socioeconomically disadvantaged having qualified for the Federal Free or Reduced Lunch program. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE 1. School Enrollment by Grade Level. 54 Grade Level Enrollment Grade 9 695 Grade 10 562 Grade 11 461 Grade 12 448 Total Enrollment 2166 Retrieved from Grove High School’s website on 8/7/04 TABLE 2. School Enrollment by Racial/Ethnic Category. Racial/Ethnic Catesorv Number of Students Percentage of Students African-American 36 1.70% American Indian or Alaska Native 24 1.10% Asian 198 9.10% Filipino 36 1.75 Hispanic or Latino 1,367 63.10% Pacific Islander 18 0.80% White (Not Hispanic) 487 22.50% Multiple or No Response 0 0.00% Retrieved from Grove High School’s website on 8/7/04 Both finding the right site and “working with the ‘gatekeepers’ to obtain necessary permission, are critical first steps in a case study” (Gall, Gall & Borg, 2003, 445). It is simply human nature to be more open when someone in trying to understand your success, as opposed to your failure. The measurement o f “high performing” was also quite intentional. Utilizing the Academic Performance Index (A.P.I.) rating system provided control for certain factors. It enabled the researcher to identify a school that is doing significantly better when compared to 100 schools Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 55 of similar demographic make-up. This allowed pursuit of the question, “How can they be doing so much better than most of the schools just like them?” Seeking a school with a three-year trend of success helps to control for random surges in performance. As with all case studies, a purposive approach was used in selecting the sample from which to collect the data. This was done to ensure the most appropriate context within which to study and each characteristic was determined to provide the most fruitful data. A high-performing school was selected because the researcher could describe the factors in a school that is finding success and this, in and of itself, is potent. To illustrate the achievement, Table 3 shows the most recent Grove High School student performance as reported on the 2003-04 STAR report that continues a four year trend in meeting the School wide API growth target. The right hand column of Table 3 clearly illustrates that Grove High School met the individual target for each of its four significant subgroups: Asian students, Hispanic or Latino students, White students, and Socioeconomically Disadvantaged students. Although Grove High School had made is school wide growth target for a total of three years, this was the first year the school met each o f the subgroups. The most powerful aspect to the school’s performance is that the state testing scores from four and five years ago were poor enough to identify the school as Program Improvement. This meant that a lack of growth would result in various state sanctions including reconstitution o f the school where all staff are essentially released and must reapply to continue working at the site. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced w ith permission o f th e copyright owner. Further reproduction prohibited without permission. TABLE 3. Student Performance from 2003-2004 STAR API Report. Number Of Pupils Included in 2004 API Numerically Significant in both Years 2004 Sub group API Growth 2003 Sub group API Base 2003-04 Sub-group Growth Target 2003-04 Sub-Group Growth Met Sub-Group Growth Target Ethnic/Racial African American (not of Hispanic origin) 32 No American Indian or Alaska Native 15 No Asian 136 Yes 759 722 7 37 Yes Filipino 32 No Hispanic or Latino 1028 Yes 606 568 7 38 Yes Pacific Islander 17 No White (not of Hispanic origin) 333 Yes 734 721 7 13 Yes Socioeconomically Disadvantaged 1012 Yes 616 578 7 38 Yes Retrieved from the California Department of Education website on 3/1/05 O n 57 Finally, an urban school implies inherent challenges associated with high percentages of ethnic minority and low socioeconomic students. Student success under these circumstances is more likely the result of what is done at the school site, as opposed to wealthy; mostly Anglo schools where the families provide services and experiences outside of school that dramatically support their children’s learning. Finally, there are more students at schools facing various levels of challenges regarding significant populations of ethnic minorities and low socioeconomic status. This boosts the generalizability of the findings that emerge from the study. Beyond meeting the predetermined characteristics for the case study, Grove High School has the potential to make a large contribution to the understanding of this phenomenon. First, Grove is a perfect representation of the ever-growing “urban-suburban” schools located throughout California. Urban is no longer confined to the downtowns of Los Angeles, San Francisco and Oakland. Suburban neighborhoods throughout the state have experienced many changes through which the schools have taken on urban qualities of ethnic and economic diversity. Many of these communities continue to lament the change instead of building a strong future like Grove. Many of the staff members currently working remember “how it used to be” and their insight into how Grove High School has overcome the transformation could be quite informative. Also, studying a senior high school provides an examination of the most complicated organization within the K-12 public school system. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Instrumentation 58 The instruments for data collection - surveys, document review template, interview questions, and the observation template - were chosen to gather the most possible information answering the four research questions (see appendices A, B, C, and D for full copies of the instruments mentioned). The tools were developed collaboratively as the thematic dissertation group met throughout the spring, summer and early fall of 2004. Each instrument supports each of the questions in different ways. Question one seeks understanding of the district’s policy and strategy for carrying out teacher evaluation. Document review of board policy and the teacher contract shows official scope and intent with regards to the evaluation process and potentially other aspects of instructional supervision. The forms used for documenting the evaluations also provide a rich description, again, of the district’s official intent in carrying out the process. The survey of teachers generates a cursory review of the perceptions held by both groups. Interviews allow for open-ended questions to investigate both teachers’ and administrators’ understanding of district policy and strategy in this area. Direct observation provides the least amount o f insight to this question. Observation deals with watching and listening to the reality - policies in action, or inaction. Observations of faculty meetings and various stages of the teacher evaluation process directly gather data concerning the second question of how the school carries out teacher evaluation. Surveys again focus on perceptions Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 59 of how the process occurs and the interviews will provide members of the sample with an opportunity to give more detailed information with regards to how they see they are evaluated and, on the part of the administrators, how they intend to evaluate. This is bounced off the “reality” as it is visible through direct observation by the third party researcher. All four instruments support the last two questions - what factors have shaped the school level effort and how effective in the teacher evaluation process at the school - in much the same ways as the previous two. Document review also provides insight into the “official story.” Everything that the organization is willing to put into print for internal and public distribution has no guarantee that it happens that way or is perceived to be happening that way by those involved. Surveys will provide a certain depth of insight regarding teacher and administrator perceptions of items contained within the survey. Interviews will provide the chance to probe deeper with open-ended questions and the ability to read body language and other non-verbal cues. The observation template will grant the opportunity to collect data regarding what is happening - messages, symbols, and interactions occurring throughout the school in various formal and informal settings. These instruments were chosen through large group discussion including all parallel dissertation members after it was decided to pursue the topic through a case study. General discussion initiated thoughts on how each tool was to be fleshed out. Next, three groups were assigned the task to develop prototypes of each tool Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 60 for review by the entire group. The initial drafts of each were brought to the designated meeting where each was reviewed with suggestions for deletions and modifications. The entire group worked on the protocols for implementation of both the surveys and interview questions at this same meeting. At the end of the meeting, the group had agreed upon the content and format for each tool and individuals volunteered to edit the changes and electronically mail the final versions to the rest o f the group. Data Collection Collecting information to shed light on a phenomenon can be quite inefficient without a minimal amount o f structure to build upon. The other extreme of narrowly bounding what is to be collected can limit the insight that is generated. To help walk this line, the researcher employed the conceptual framework of Bolman and Deal’s (2003) Four Frames. Their four organizational categories of Structural, Human Resource, Political, and Symbolic did not serve as limits to what data was gathered but rather lenses to enhance what was seen, as well as, what was read and heard. “... the structural frame emphasizes goals, specialized roles, and formal relationships .... The human resource frame, based particularly on ideas from psychology, sees an organization as much like an extended family, made up of individuals with needs, feelings, prejudices , skills, and limitations .. . [The political fram e] . .. sees organizations as arenas, contests, or jungles [with concepts Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 61 such as] .. . bargaining, negotiation, coercion .. . coalitions . .. [and] power .... The symbolic fram e, drawing on social and cultural anthropology, treats organizations as tribes, theaters, or carnivals [seeing] . . . organizations as cultures, propelled more by rituals, ceremonies, stories, heroes, and myths than by rules, policies, and managerial authority” (Bolman and Deal, 2003, 14-15). The researcher went into the school site looking for elements within each of the categories. For example, within structural, what are the official roles of instructional supervision assigned to administrators; in political, who has the power to influence decisions on teacher practice; in symbolic, what is celebrated within the school; and in the human resource, how are perceived teacher instructional needs being addressed? The researcher will not code data, forcing every bit of information into one of the categories as it is gathered. Again, the Four Frames will serve to enhance, not limit the information that is collected. The researcher engaged in various activities and approaches to facilitate the greatest participation among the individuals in the sample. At the earliest point possible, the researcher attended a faculty meeting to clarify the study placing strong emphasis on the success of the school that qualified it as part of the population. Questions were fielded to rectify any potential misconceptions or assuage any apprehension at how the data will be used. On the occasion of the formal introduction, the researcher dressed professionally with coat and tie. In subsequent days on site while conducting interviews, observations and other data collection, the researcher dressed professionally, but less formally with the intent of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 62 creating a more comfortable environment, especially with the teacher, reminding them more of an educational colleague and less of a school administrator. Reliability in data collection is always important. In the context of this study it becomes absolutely crucial if commonalities in the findings among the various case studies are to be considered significant. The protocols for both the surveys and interviews are of paramount importance in this process. Much work was invested to ensure that both tools were implemented uniformly, not only within the same school site, but at the other school sites as well. Both of the templates for the document review and the observation were constructed to create applicability for all situations and to generate consistency in how the data is collected. The surveys, document review, and observations began as soon as the researcher gained entry into the school. The survey served as both an introduction of the areas of investigation for the sample participants, as well as a “scratch the surface” measurement regarding how teachers and administrators perceive the various factors which the study seeks to describe. Document review gets directly at the intentions of the organization and the “official” story of what is happening. This can then be checked against the observation data throughout the process. The observations, although beginning early, continued throughout most of the study in order that the researcher can observe all of the various formal and informal meetings and activities that are considered germane to the study and were able to be attended within the time parameters. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 63 One survey was distributed to all members of the teaching faculty. The researcher offered small incentives for completion to increase the return rate. A lottery ticket was given in exchange for each completed survey. Participants were encouraged not to consult with others on responding to the statements, but the participants were not monitored during the process, allowing them to finish with some flexibility of time and location. The templates for document review and observations were used in electronic form within a database allowing for retrieval by key word or phrase. Observations were obviously done on site of the activity while document review was done both on and off site, depending on issues of time and availability o f the documents. The interviews began after the overall process had started. Interviews serve the study in multiple ways. Researchers have used interviews as long as they have been investigating phenomena involving people to collect their perspective. Beyond this, the interviews in this study could increase the validity of findings if similar trends are identified through the various tools of collection and they also served as a follow-up instrument. Potential themes seemingly emerged quite quickly after beginning the research. The interview process granted the researcher the opportunity to direct open-ended questions to the members of the sample that are most connected to the patterns or those who, through the surveys and observations, seemed to indicate a certain perspective. One important aspect to the interview process is that of field notes - how the information will be recorded and what, if any, preparations need to occur for Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 64 analysis. During the interview process, the researcher used an audio recording device to collect all that is said. He simultaneously took written notes on the general themes and, seemingly, the most significant comments related to the study. The recordings were transcribed and entered into a database program so the information could be recalled by question - looking at all of the responses to Question 3, for example. The information could also be sorted by interviewee - looking at all of one individual’s responses. By the end o f the data collection, the researcher had visited the Grove High School campus on seven different occasions. These visits were spent introducing the study, distributing the surveys, collecting documents, observing classrooms, attending a faculty meeting, and conducting interviews. Twenty-nine surveys were returned from a teaching staff of 88 teachers for a return percentage of 33%. Six interviews were conducted with both administrative and teaching staff members. The principal was interviewed first. The Categorical Programs Coordinator, who oversees primarily the Title I and English Language coordinators, was also interviewed. Four teachers participated in the interviews. There was an attempt to interview at least one teacher within their first two to three years, a core curriculum department chair, and a math teacher participating in the TASEL-M grant project because of its elements of curricular support and supervision. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Data Analysis 65 According to Creswell, data analysis," ... involves preparing the data for analysis, conducting different analyses, moving deeper and deeper into understanding the data, representing the data, and making an interpretation of the larger meaning of the data" (2003, 190). To engage in this process, the researcher employed a six-step process presented by Creswell (2003). Organization Preparation Read for a General Sense “Chunk” the Information Description Themes Create Narration Interpret Data Adopted from Creswell 2003 FIGURE 2. Six Steps of Data Analysis Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 66 Each step refines the information moving closer to the study's findings. Figure 2 illustrates each step in order. Step one is the organization and preparation of the data. This entails many of the aforementioned activities of processing the raw data into clean and logical ink and paper representations, including transcribing, sorting, etc. The second step is to read through all of the data to gain a holistic, over-all view, making any appropriate initial generalizations. Step three begins the focusing process. The data was placed into broad chunks that were given titles that Creswell refers to as "en vivo" terms. Bolman and Deal's (2003) Four Frames were gain relied upon help create the crude categories that will begin the classification of the data. The data documents were then coded with these various terms within a database system that, again, will facilitate quick and easy retrieval. The coding process enabled step four, describing the people, events, and physical location of the school site. Along with rich description, themes were developed about the phenomenon as experienced at the school. Each of the themes was supported by contextual information lifted directly from the raw data, as well as, a corroboration of themes as seen through multiple data collection methods. Step five is a synthesis and presentation of the identified themes in the form of a narrative. This allowed the researcher to focus on the internal connectedness of the emergent themes. Someone else reading this would take a brief walk down the road of discovery traveled by the researcher. The sixth and final step in the process consists of interpreting, or making meaning of, the data. Lincoln and Gobi Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 67 (as cited in Creswell, 2003) focus on the questions, “What were the lessons learned?” This empowered the researcher progress towards a comparison with existing studies citing differences and similarities or bring the researcher to new questions that need to be answered or new phenomenon to describe. As with any study, potential problems exist and the most glaring is that of time. Quite often, questions beget questions, especially in research. After the initial survey of the data, the researcher wanted to conduct follow-up interviews, put together a new focus group, and perform additional observations. This was quite limited, simply by the boundary of time. Another potential problem is truly gaining access to how staff really perceives what is happening. Issues of trust and the fear of negative consequences, as a result of sharing concerns about what is not working well, can stand as obstacles to truth. All feasible approaches were taken to mitigate or eliminate these issues. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 4 68 THE FINDINGS Introduction Grove High School represents a complex and rich organization with a great story to be told about the formal and informal policies and practices that work together to make a successful school. A number of data collection tools were used to generate a great deal of information about what works, what doesn’t, and why. This chapter includes observations, reviews of documentation, and perceptions of the school by staff members generated through a survey and interviews. The chapter is divided into sections with the following information: data collection tools and procedures, the data as organized by the Four Frames, the emergent themes, and a response to the research questions. Data Collection Tools The researcher generated data by four different methods. Various district and site documents were reviewed for direct and indirect implications on various aspects of instructional supervision, and more specifically, the teacher evaluation process. The researcher had the opportunity to observe a faculty meeting, as well as, the general day-to-day operations of the school, both in and out of the classroom. A survey was administered to the teaching faculty which generated an Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 69 initial picture of Grove High School. Finally, six staff members engaged in an interview responding to an identical group of questions. Written publications were reviewed to begin collecting information on the official “story” of the school site. Various documents emanating from both the district and the school site which address aspects of the study were examined. To gain understanding of the formal teacher evaluation process, the researcher first reviewed the portion of the district’s bargaining agreement with the teachers’ union, in which the parameters and expectations of the evaluation process are set forth. In addition, various school site forms that are used by the administration to execute the evaluation were examined. To gain general understanding of instructional programs at Grove High School, both their School Accountability Report Card and the Single Plan for Student Achievement were assessed. During the observations a sense of the school began to develop. Although there are some differences that set the Grove staff aside from other schools, there were clear indications that Grove is host to dynamics that are found on many campuses which are achieving various levels of success. The faculty meeting held on January 21 was attended by pods of teachers as opposed to one staff. In other words, different groups o f teachers sat in various spots of the school theatre leaving many empty seats in both the front and the middle sections. The crowd needed a few reminders to initially focus on the meeting, but was attentive to the various presenters. Mixed in with a review of the new tardy policy and a presentation on a pilot on-line parent interface system, were the usually litany of announcements of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 70 up-coming events, volunteer opportunities, and other general reminders. There was no evidence of an extraordinary interest among the staff to meet and work through information towards the goal of improved student achievement. Despite some questions and comments from the staff throughout various parts of the meeting, the level of engagement was not, in any way, overwhelming. A lot o f information was disseminated to those in attendance while they passively participated in the process. The most significant portion of the meeting seemed to be the PowerPoint presentation run by the principal, which served as a “State of the School” address, as it was entitled. The leading slide proclaimed the school to be, “OUT OF PROGRAM IMPROVEMENT!” Although the style of presentation seemed to communicate this information being new, the reaction of the staff seemed to indicate that they already knew this. There wasn’t an evident exultation by the teachers present. The ensuing slides provided information regarding performance on the 2004 STAR exams, success of the AVID (Advancement Via Individual Determination) program, school wide instructional strategy goals, update on the 5-Year Plan, continued planning and implementation, and future goals. Nothing demonstrated a visible opposition to the items shared in the presentation, but there were no other visible reactions either. The information, which appeared to be both positive and well organized, didn’t incite any significant reaction among the staff. Campus “walk arounds” - opportunities to go in and out of classes and pass through different parts of the campus - also began to tell a story about the Grove High School organization. The physical trappings of High School, USA were Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 71 clearly evident - everything from posters advertising the next school dance to plaques acknowledging the generosity of past graduating classes. Any visitor could also see the great diversity in educational programs. Course offerings were plentiful for the English Learners, as well as those students desiring classes reflective of the Advanced Placement and honors curriculum. In addition, the visual and performing arts are strongly represented. Grove High School can also boast a well established agriculture program with onsite facilities. A cursory visual inspection identifies continuity in presentation with school wide posters displayed in offices and classrooms that address instructional strategies and local philosophy. There are posters that speak to the process of and idea behind using both Cornell Notes and graphic organizers. Classrooms not displaying these visual reminders are in the minority. Flowing from the school wide focus on writing is a poster found throughout the school that says, “Math gets you into college, but writing keeps you there.” The largest poster displayed on school walls is the Peacock Pride poster. It queries the onlooker, “Did you know ...,” and then proceeds to list a number of accomplishments achieved by the students and staff over the last few years. It makes references to API scores, sports championships, AVID program performance, and Junior Reserved Officer Training Corps (J-ROTC), among other accomplishments. These themes are also reflected in the school site’s web page. Sprinkled throughout the pages are lists of accomplishments and various thoughts regarding writing, the writing process, and the advantages of having strong writing skills. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 72 The survey (see Appendix A) was distributed to all teachers on campus by way of their mailbox. This was done following a brief opportunity to meet with the teaching staff. They were meeting in various rooms throughout the campus preparing for the upcoming accreditation visit. The principal briefly made introductions. The general goals of the study and an explanation of what piece the survey played in the process were described by the researcher with an opportunity for a few quick questions by the teachers. Multiple reminders were sent to staff by way of the staff bulletin and the principal sent an e-mail memo to all the teachers asking that they take the time to fill out the survey. Twenty-nine surveys were returned from a teaching staff of 88 teachers for a return percentage of 33%. This provided a cursory indication o f trends present at Grove regarding teacher perceptions and provided a comparison to the data retrieved from the individual interviews. Appendix E is the Frequency Report for the survey as generated by SPSS program software. This first section contains the items of the survey along with the amount of valid responses, missing responses, and a mean rating correlating to the responses from 0 for Strongly Disagree to 3 for Strongly Agree. A mean of 3 would indicate that every respondent marked Strongly Agree and 2 would indicate every respondent marked “Agree,” etc. The remaining section of the report generated the same information in terms of percentages of respondents who agree or disagree. These percentages appear throughout this and the next chapter. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 73 The final piece of data collection was the collection o f individual interviews. Six interviews were conducted with Grove High School staff members. The principal and the Categorical Programs Coordinator represented the administrative/curriculum support staff. Representing the teaching staff were four members of the faculty: Teacher A - a male science teacher with nine years of experience; Teacher B - a male English teacher in his third year; Teacher C - a male math teacher with seven years of experience; and Teacher D - a female English teacher in her eighth year at Grove High School and who serves as chair of her department. Teachers C and D had experience teaching at other schools, while teachers A and B have spent all of their contracted teaching years at Grove High School. The researcher sought teachers with specifically different experiences to participate in the interviews. By design, there was to be at least one teacher who was new - with three years or less of experience and for comparison one or more teachers would have over five years of experience. One teacher would be a department chairperson and have participated in site leadership. Another teacher would be from the math department because of their participation in the TASEL-M program that generates a significant amount of instructional support. The questions from each of the six categories (Policy, Teacher Evaluation, Teacher Supervision, School Efforts, School Culture, and Teacher Beliefs) were posed to each of the staff members interviewed with the exception of the coordinator of Categorical Programs. The Teacher Evaluation section was omitted Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 74 in her interview, because she had no direct contact with either side of the formal evaluation process. The interviews were conducted at school except for Teacher A whose interview was done off-site. The interviews lasted from 35 to 56 minutes and were transcribed into a database for easier access and analysis. Each response was coded by the interviewee and the question and all data could be sorted by both codings. Although all four methods generated data that added to the understanding of Grove High School, the interviews were the most powerful. The open-ended quality of the questions, and the direct connection to rich, vivid, teacher perceptions combined to provide a very detailed picture. This data was categorized by Bolman and Deal’s Four Frames referenced in chapter three of this document. Data As Viewed by the Four Frames In order to organize the raw data generated from the four collection tools, but not limit the analysis, the researcher employed the conceptual framework of Bolman and Deal’s (2003) Four Frames. Their four organizational categories of Structural, Human Resource, Political, and Symbolic did not serve as limits to what data was gathered but rather lenses to enhance what was seen, as well as, what was read and heard. The Structural Frame According to Bolman and Deal, the definition is: .. the structural frame emphasizes goals, specialized roles, and formal relationships ...” (Bolman and Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 75 Deal, 2003, 14). This first section lists the data collected that falls in the Structural Frame of the Grove High School organization. There is information that pertains mostly to the policies governing action, systems in place through which to execute the policy, and the different roles played by the members of the organization that participate in various activities supporting instructional supervision and teacher evaluation. Structures are in place which makes evident that Grove High School operates in traditional ways found at many schools across the country. The faculty meeting had elements that have become iconic in school folklore. The seating and presentation arrangement were typical in and of themselves. The theatre served as the locale with the presenters disseminating information to the audience of teachers. The whole event appeared to be very traditional with a long list of information and the teachers playing a passive role, with the exception of commentary, usually meant for the immediate group of teachers. There was no evidence of innovative decision making or information processing having occurred, nor was there any indication that this meeting was not representative of the organized faculty meetings that occur throughout the year. Some school wide strategies and objectives were evident by simply observing posters displayed in the classrooms. School wide objectives quickly became clear after posters illustrating writing, Cornell Notes, and graphic organizers were visible in numerous classrooms, as well as, administrative offices. The survey corroborated with 96% of the respondents indicating that they were Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 76 aware of the school’s goals and objectives. Eighty-two percent were aware of the professional development plan which has reflected these same elements of the past two or three years. In addition, the strategies and objectives have most likely been communicated clearly as evidenced by responses in the interviews. Although there was a variance in term of execution of school wide strategies, there was a fairly consistent acknowledgement and awareness of those same elements. In other words, there was a clear understanding that all teachers should be directing students to use Cornell Notes, while, at the same time, teachers admitted to not requiring the notes to the degree they were directed. This carried onto the school’s website in which thoughts on the importance of writing appear on most every page. The website, general survey, visual evidence, administrative and teacher interviews all have a strong, consistent factor in the area o f school wide strategies and objectives. The formal teacher evaluation process, as a focus of this investigation, was directly investigated. There is a clearly defined structure as presented in the collective bargaining agreement established between the school district and the teachers’ union. By a mark of 96%, the teachers surveyed both are aware of and understand the district’s policy regarding the evaluation process and 82% agree with the goals and objectives as they understand them. The bargaining agreement contains much information, such as, who is to be evaluated each year, what are the general parameters o f the class visitations, and the timeline by which dates each portion of the process must be completed. A review of the notebook produced by the principal and handed out to the staff members he will be evaluating continues to Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 77 inform on the process. The folder includes various forms which guide the teacher in gathering and producing information that will be discussed prior to the observation and reviewed after the observation. The process seems to be very structured and purposefully designed from an examination of the documents teachers receive. There was little criticism regarding the evaluation. The concerns arise with the execution o f the policy. Even so, 82% of the teachers surveyed are satisfied with the administration carrying out the evaluation process. The next item of the survey indicated that 69% of the teachers don’t think the administrators frequently visit classrooms for evaluation. This may not be viewed as a negative by all teachers. Teacher A illustrates that, for some, more visits could be considered a burden. Teacher A says, “... [it has] only taken up 30 minutes each year: 15 minutes with them in the classroom and 15 minutes in their office saying, ‘hey, you're doing a great job.’ That's the only strength for me.” He continued to explain that if the experience has no value, less time spent on it is more positive. Teacher D, on the contrary, spoke about the value she placed on the interaction which can come out of the evaluation process and would like more “pop-in” visits from the administrator evaluating her. “I like ‘pop-in’ evaluations because I believe they are more honest. You don't have the dog and pony show.” The district and site implement an evaluation model that includes, what is now, a traditional clinical model of pre-observation, observation and post-observation. No modifications to the structure were mentioned by staff members, but their needs expressed Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 78 implications for the execution segment of evaluations. These are brought forth in the Human Resource frame. The staff also revealed perceptions regarding aspects of instructional supervision outside of the formal evaluation process. According to the principal, the school district started to mandate this year that administrators spend 50% of their time in the classrooms, informally observing. The principal shared that this mandate inspired an idea to place a sign-in sheet on the wall of every classroom at the opposite end from the door, on which the visiting administrator could sign their name. The principal has also set aside two specific “coaching days” per week in which he is out of the office and in the classes. In the face of these goals and efforts, 70% of the teachers surveyed reported that administrators do not frequently observe classes outside the evaluation process and 30% say they do. With regards to support staff outside of administration, 37% of the teachers agree that they observe classes frequently. This refers primarily to support staff members coming out of the categorical programs office. Working out of this office are the Categorical Programs Coordinator, the Title I Coordinator, and the English Language Coordinator. Seventy-four percent of teachers surveyed report discussing teaching with non-administrative support staff while only 55% say the same with regards to administrative personnel. An important distinction that came of the survey was that 60% of respondents saw supervision of instruction as separate from the evaluation process, leaving 40% seeing both as the same. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 79 The Human Resource Frame “The human resource frame, based particularly on ideas from psychology, sees an organization as much like an extended family, made up of individuals with needs, feelings, prejudices, skills, and limitations . . (Bolman and Deal, 2003, 15). The Human Resource frame is rich with teachers’ perceptions of what is effective for them and their impressions of what they need and how processes are carried out on campus. The survey generated data that shed an extremely positive light over the evaluation process. O f those responding, 65% see the evaluation process as an integral part o f their professional growth. Sixty-seven percent agree that feedback is timely from the evaluation and 63% believe that feedback assists them in improving instruction. Seventy-seven percent express confidence that the administrator is able to evaluate instructional practice, but 74% are not confident that administration can monitor their instructional practice. This could be influenced by the response in which some teachers do differentiate between formal evaluation and instructional supervision. The same survey has 84% responding that they believe administrative supervision of instruction improves their instructional practice. In addition, 81% agree that instructional support staffs supervision improves their instructional practice. In terms of the effectiveness of the evaluation process, the interviews yielded a less positive perception. It is important to keep in mind that the evaluation process was modified for this year. With tenured teaching staff being Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 80 evaluated every other year, not all the staff members had direct contact with the new process built on the California Standards for the Teaching Profession. Having said this, not one teacher made reference to the process being significantly different - even with three out o f the four having been evaluated by means of both systems. Teacher A was emphatic about his perception of the process as fairly meaningless. “For me it's been pretty rote - they just come in. They might come in once a year and watch me for 10 minutes and then tell me I'm doing great.” This same teacher pointed out a strength to be the fact that the process, although fairly meaningless, doesn’t take up a lot of his time. Teacher C did not have such a strong, negative reaction but still expressed a perspective that it didn’t have a lot of impact on his teaching. Teacher B, the interviewee with the least amount of years teaching, stated that he enjoyed the process, highlighted positive aspects such as the pre conferencing, but did not give a strong impression of how the process powerfully impacted his teaching practice. Teacher D was the most positive in saying that, “it matters a lot to me.” Her perspective was that the process, in theory, was powerful - having someone observe and critique her teaching offering a place to focus for improvement and growth. She did not provide clear examples of how the executed program at Grove impacted her. In fact, she stated that, “. .. there have been several times I have felt the critique is too positive; you’re giving me nothing to work on - I need something to work on.” Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 81 The teachers were very clear and consistent in the interviews with respect to their need and desire for constructive feedback on their teaching and how that feedback could be tainted or accentuated. Teacher A is open about his desire for informative feedback. He wants, “somebody who knows my teaching and tells me, ‘this is good, this is not good.’” The frustration lies in what he gets instead. I feel like, most of the feedback I get - 1 feel like it's worthless, because it's coming from people who haven't seen me teach. People will pop in for a second and say, “It was great to see this.” That's good, they were taking Cornell notes. You're happy about that because you're trying to get everyone to use Cornell notes - that doesn't necessary mean I'm a good teacher. Teacher B, having a more positive experience with the evaluation process, still emphasized the feedback he receives as being very crucial. That feedback is so very important. I'm not afraid of what's to be said. I'm especially looking forward to somebody saying this was good, but maybe you could do better here; maybe you could not do this and do this instead. Out of a negative experience, Teacher C is able to clarify better the feedback he would like. Since he has not received any information from the formal observation he had, he would like the feedback to be timelier. In addition, he seeks a specific type of feedback that he doesn’t believe administrators, for the most part, are prepared to deliver through formal or informal observation. An administrator coming in is not going to make a comment, “In that lesson, you should have done this,” or something like that... [another, more knowledgeable math person] says, “Maybe you could explain the Pythagorean Theorem this way instead of that.” An administrator is not going to bring that up. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 82 Teacher D also seeks real feedback - observations of what she is doing well and on what she can improve. She more greatly fears not being sharpened by critical analysis of her teaching. I will tell you sometimes the way that I feel is that I'm not being told enough things that I could improve on. I don't know if I should be sitting back going, “well, maybe I don't have anything to improve on.” I sometimes feel that surely, there's gotta be something that somebody could offer or something because I don't always get that. Usually it's way too positive. I don't want to be that person who has a false sense of security. The interviews also shed light on obstacles to teachers receiving the feedback in such a way that allows reflection and growth. The principal, Teacher B, and Teacher D were very clear that issues of trust and anxiety can tremendously affect how the feedback given is viewed. The principal acknowledges the dangers of mistrust immediately. Now I look at evaluation - it is my responsibility to improve teacher instruction and performance to maintain teachers here at this school - not to get rid of them, and that's a different mind set than some of the teachers are coming from. So I'm always looking for those positives, and reinforce the good things and at the same time I'm trying to get the teachers to have enough trust in the administration that when we are critical, they don't take it personally. The perspective shared by Teacher B seems to corroborate the same thought process. I try not to be intimidated or afraid of these types of things. With all due respect, some people, for legitimate reasons, are afraid of these kinds of things. I go into it trying to have an open mind and not think of it as, “Oh my gosh I'm being evaluated and if I don't do good I'll be fired.” I look at it as an opportunity for me to grow, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 83 to accept my faults, shortcomings, or whatever it is so that I can improve. I want to constantly improve. Teacher D shares the same attitude. She approaches the process confident in her abilities and hopeful for the opportunity to hear detailed feedback on how she could improve her teaching. As a department chair, she very closely works with many other teachers - especially those new to her department, many of whom are new to teaching. When asked if newer teachers experience a greater degree of anxiety that is counterproductive to benefiting from feedback in the context of formal evaluations, she answered without hesitation: Oh, yes. Someone that I recruited is here this year and so it's their evaluation year as well. They were very stressed - very, very stressed about it. There was nothing I could say - some people I'll say, “it's not that bad, you can ask questions, look at it...”. I try to put it in a positive light. Some people can't look at it that way. They look at it, “what if I mess up.” Almost, “what if I don't - if we take a test and I don't have spare pencils?” They start, almost honing in on the slightest little thing and that's how stressed I've seen people get. At the same time, 77% o f respondents to the survey feel comfortable going to the administration for support. This question appears in the section of the survey dealing with the school’s culture - how things are done around here - and does not specify for purposes of instructional dialogue. Teachers may feel comfortable going to administrators for support, but not instructional support. The feedback, o f which the teachers are so strongly in favor, does not just come from the administration during formal observations. Collaboration seems to be a much less intimidating word and a process in which the teachers at Grove Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 84 readily engage. The survey indicated that 89% feel the staff is encouraged to collaborate regularly on instructional matters and 85% of those surveyed reported that they do collaborate on a regular basis. Teacher A very easily could describe his collaborative interactions with other staff members: I definitely have conversations with other science teachers - especially with the new science teachers. I try to get a chance to talk with them, sometimes observe their classrooms, sometimes talk about where we're going, and how I do it, how I cover a subject - things like that. Responding to the same question about instructional interaction by stating, “Primarily, it’s with my colleagues - especially those in the English department.” He went on to give examples. . . . when we studied Henry David Thoreau, I had a revelation, like, “Oh my Gosh.” So I typed up this quick little lesson plan and shared it with my colleagues and they went and did it in their class and they came back and said, “This is genius, this is great.” That kind of feedback makes me go, “okay, I'm heading in the right direction.” I'm regularly getting feedback, formally and informally from other staff. They either hear about what I'm doing, or come in and actually see what I'm doing. He goes on to talk about the “ladies in the resource office.” The coordinators of Categorical Programs, Title I, and English Language all work out of the same office and they work very hard to provide direct instructional support to teachers, beyond managing the resources of their respective programs. The coordinator of categorical programs expands on her and the others’ role in support: . . . as a resource person, I interact on an individual basis with teachers as part of giving support services and instructional strategies - working to improve whatever. An example, I had a Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 85 math teacher in the other day who wants to do student led conferences, so I worked with him in how to set that all up. She went on to explain how the whole office directly supports teachers in the classroom, “In fact, our EIA LEP Coordinator does that a lot, because she goes in with her EL teachers and gives immediate feedback.” The interviewee with the most years experience at Grove High School, Teacher D, had the most enthusiastic response in terms of collaboration both on a group and individual level. For example, she says, “... we are always talking about instructional practice within our department. I think that’s what made it stronger.” She continues to describe very powerful illustrations with her on both sides of significant instructional collaborations. Formally she was part of the Instructional Leadership Team and was given extra time during the school day for providing others’ curriculum support. She recounted opportunities allowing her to teach lessons in colleagues’ classrooms to help them with a particular challenge they were facing. According to her own admission, this teacher was equally on the receiving end o f support. I have had one of our teachers come in and . . . I said, “Look, I'm having a hard time getting this concept across to this class of mine. Could you help me?” They're like, “Hey, I'll come in and teach the lesson.” That's what they did. They came in and taught the lesson on this particular thing. I was in the back taking notes. Then I taught the lesson the next time that class period came up. Those are just the kinds of things we do. Another example conveyed how staff collaboration truly helped her past a very challenging moment in her teaching. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 86 I was struggling with one class and I had just gotten to the point that I didn't like the class anymore. I actually looked on my schedule for people who had that particular period - they had conference. I was seeking out people to come in and watch. There's something going on in this class - there's a dynamic and it's the first time I've taught this particular curriculum. It's a brand new class for me and so I know that's going to be an issue, but I also know that something's going on, there's a dynamic that's going on and I'm not picking up on what exactly it is. I got some really good information. Probably not information I was expecting. This same teacher expressed a longing for past opportunities in which she could provide more instructional support to her department. Using Program Improvement monies, Grove High School was able to establish an Instructional Leadership Team with team members that had common planning periods to meeting and make instructional decisions and additional time within the school day to go out to classes and observe, do demo lessons, or simply discuss classroom concerns with other teachers. Teacher B also made comments in which he identified time as a major obstacle to the type of collaboration and reflection in which teachers should really engage themselves. Teacher C, as part of the math department, was able to report on a very unique experience in terms of feedback and support. Grove High School’s math teachers participate in a grant through a local university that provides monies and personnel for curriculum support in terms of supplies, formal professional development, and informal support and dialogue. The grant, called TASEL-M (Teachers Assisting Students to Excel in Mathematics), provides money to pay for an on-site support person working directly with math teachers (at Grove and its two Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 87 feeders), a budget for various instructional supplies, a university liaison who also provides professional support, and funds to pay for teachers attending professional development opportunities throughout the school year and summer. Teacher C was very clear on the impact that this program had on instruction in terms of collaboration and opportunities for feedback. When asked how the TASEL-M Project improved his classroom practice he said, “. .. [it's] mostly due to us working together. We had more communication through TASEL-M. I don’t it forced us, but it gave us time to get together and gave us, “here’s what works, here’s what you should do.” The staff development aspect to the program was rated as more effective by him than other opportunities. He said: Well, maybe because TASEL-M is math specific. All of the other stuff we had is generic to everybody. This is math specific, number one. Number two, is we’re kind of running it partially ourselves. We have a coach, but it’s input ourselves. A large part of the Human Resource Frame is value - the value that individuals perceive they hold within an organization. All six individuals interviewed responded with no hesitation that they felt valued as a member of the Grove High School organization. This sense of value came from students as well as other staff members. Teacher B shared the experience of receiving Teacher of the Month. Teacher C described the principal coming into one of his classes and praising him to the students by acknowledging his receipt of many votes for teacher of the year, despite his not having won. Teacher D feels administration’s efforts to keep her as department chair to be a clear validation of her efforts in the post. The Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 88 survey results support this in that the four questions dealing with teachers’ role in impacting students, 89% or more of the respondents acknowledged their influence in student achievement. Despite this, those interviewed very much downplayed their own part in Grove’s high levels of student achievement. Very little light was shone on the teachers’ efforts in the class when asked, “What are the most significant ingredients in student success?” The Political Frame [The political fram e] . .. sees organizations as arenas, contests, or jungles [with concepts such as] . . . bargaining, negotiation, coercion .. . coalitions . . . [and] power .. . (Bolman and Deal, 2003, 15).” The Political Frame sees the organization as an arena where human interaction centers on negotiation, the distribution of power and the roles played by members of the organization within that context. The faculty meeting displayed very traditional power roles. The principal organized the event moderating the flow of presenters while the audience of teachers wielded what power they had by taking their time to settle in and focus and sitting in social groupings, rather than in the closest proximity to the dissemination of information. The principal saved the last position on the meeting’s agenda for him to share a PowerPoint slide show regarding the current status of academic achievement, progress towards present goals, and the road to formulate new objectives. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 89 The survey indicated a strongly positive perception of the ability of staff members, teachers and non-teachers to serve in leadership roles at the school. In spite of this, very few individuals were identified as being strong leaders in the school. The principal was mentioned numerous times and some identified the program coordinators from the resource office, but, among the teachers, no one was mentioned as having any true influence over the decisions made affecting Grove High School. There could be a connection to the exodus of teachers over the past eight years or so. Many veteran teachers that were outspoken and respected are no longer working at the school. There are younger, very talented teachers, who don’t have the experience and “time on the job” to be significantly influential. Both perceptions and realities of decision-making can dramatically affect the success of an organization in terms of how a staff views the leaders. Schools are no different. The data collection generated no evidence o f any underlying dissatisfaction with the leaders and decision makers of Grove High School that can plague many school organizations. In fact, there is a great satisfaction among colleagues as they survey their coworkers. Seventy-five percent of the respondents to the survey agree that they are satisfied with the teaching ability o f other staff members. In this same vein, 85% are satisfied with the competence and leadership ability of teachers in leadership roles and 75% are satisfied with the ability and leadership of other instructional support staff. Eighty-two percent are satisfied with the ability and leadership of the administrators. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 90 Information collected also suggests that staff members, overall, are happy with who makes decisions and how they’re done. The principal expressed a strong, clear philosophy regarding decision making. As the principal, I have the pencil last. I don't believe that I have the wisdom, that I have the complete global picture that I can make every decision. Most decisions - some decisions, I need input from as many people as I can get. And there are certainly spur of the moment decisions that need to be made. There are decisions that I reserve to make because I am principal. It's my responsibility and that's why I became a principal. Major decisions that affect what goes on at this school are made by the management team or the instructional leadership team. Shared leadership is so critical. There is some discrepancy among staff members with regards to who makes decisions. The survey indicates that 57% of the teachers think they have an active role in making decision for the school with a significant 43% disagreeing. At the same time, 82% reported that teachers have initiated school improvement efforts with 67% agreeing that the majority come from the site and/or the district. Teachers have a sense of their ability to impact decision making, but to varying degrees and in different ways. There is not a common understanding of what staff are part of what committees that share in decision making. The principal and the Categorical Programs Coordinator quickly identified the Management Team (consisting of administration and non-academic department heads) and the Instructional Leadership Team (consisting of academic department chair persons) as main decision-making bodies. Teacher D, having participated in the Instructional Leadership Team, also identified their roles in decision-making. In Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 91 contrast, the other three teachers did strongly identify the part these two teams play in making decisions. Teacher A shared his perspective: I was actually talking to some other teachers about this and a lot of them feel, “Oh well, you know the principal talks to a few people and then makes the decision.” I know there is the school site council that apparently has parents and teachers and I have no idea what goes on in those meetings. As the conversation continued he was asked whether there was an Instructional Leadership Team. He responded, “There could be. If there is, I think it's just the department chairs at the moment.” In considering the Instructional Leadership Team of past years which had structured time to make more decisions, Teacher A was not convinced it made a significant impact. “As far as school wide decisions, I don't think it was any better because we had more people in each department taking a period to do that.” Teacher B talks about the School Site Council and expressing great pride in his participation and feels good about the opportunity for this group of different staff and community members to work together in analyzing different school issues. Although, the other members interviewed don’t put it at the forefront of decision making for the school. Teacher B also clearly sees the administration making decisions on their own and as an extension of the district. The administration certainly makes a lot of decisions for us. It could be that they are directed by the district and I assume that what it is most of the time - that the district - they say you've got to Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 92 implement this. The site administrators here then do what their told. Sometimes it's to our chagrin and some times it's like we understand. As he continues, Teacher B also highlights his impression that the administration gives over authority for decision making to the departments and individual teachers in many areas o f classroom instruction. This is concurred in by Teacher C in his response. Stuff within the department or our common agreement - more classroom related stuff, as long as we've got the school wide writing . . . Cornell Notes and graphic organizers, we're - like in math, we geometry teachers will decide, or whatever course you're teaching. As far as day to day operation, we are kind of left on our own. He also shares the perception that certain decisions are handed down by the district with no option but to implement on the site level. This teacher also mentioned that there are many occasions where e-mails or memos will be distributed soliciting input from the staff on modifying a policy, for example. He intimated that on many occasions staff would not offer suggestions but complain later at the resulting policy decision. Teacher D spoke about the different responsibilities staff members have the ability to make decisions regarding their department or sphere of responsibility. She also made it clear that with, “anything that goes on regarding school practice usually - 1 rarely ever hear the principal just make a decision by himself. Maybe that's something I see as a good thing.” As the only teacher having been involved Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 93 in the Instructional Leadership Team of previous years, Teacher D spoke to her perspective on that group’s worth and purpose. I actually miss that - we were able to do a lot of work with regards to, “okay, what do we need to do to help our kids?” We could have that kind of discussion because it was built into the school day. Now we don't have that, so we're looking at from a department chair angle. If we were to have more meetings in that regard, I think it would help us discuss more instructional practice. Again, it is interesting that Teacher D, the principal, and the Categorical Programs Coordinator all heavily praised the Instructional Leadership Team as it operated with the common planning periods, while the other three didn’t even mention its existence without a prompt. The Symbolic Frame “The symbolic frame, drawing on social and cultural anthropology, treats organizations as tribes, theaters, or carnivals [seeing] . . . organizations as cultures, propelled more by rituals, ceremonies, stories, heroes, and myths than by rules, policies, and managerial authority” (Bolman and Deal, 2003, 15). Although it may not appear in manuals, regulation books or accountability reports, what is held sacred, and celebrated within an organization is as vital to its function as any flow chart or policy. Grove High School is replete with celebration. As a school that has seen its centennial graduating class already walk along the stage with their diplomas, there is a strong sense of history among the staff. This history seems to be the launching pad from which to rejoice in what is good about Grove. The Peacock Pride Poster Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 94 that adorns walls all over campus, in and out of the classrooms, highlights a multitude of accomplishments. The poster asks onlookers, “Did you know . . and goes to list accolades and accomplishments within academics, athletics, special programs, and community service events. The principal misses no opportunity to highlight achievements of staff and students. In his opinion the yield to investment ratio in highlighting teacher achievement is immense. If there is a teacher of the year award, of any kind, whether its California Bank or something else, I'm gonna find a teacher to nominate. My philosophy is that piece of paper costs nothing. I will spend 20 minutes making up a certificate on my own computer - it doesn't really have to mean anything. You give a teacher a certificate that says, ‘You're an exemplary teacher,’ or ‘you've done a great job,’ it goes such a long way. Teacher B relates a story demonstrating the power in extending celebration beyond traditional awards and achievements. One of the things I like about [Grove] is that everything is celebrated, everything. This is a strange example, but last month there was a group of students that performed a civil disobedience - they did a sit-in protest and that was actually celebrated. [The principal] came up and said, “I want to applaud these students for protesting peacefully.” They had a complaint and rather than going and vandalizing or being violent, they did a respectful, peaceful protest where they did this big sit-in. So, he even celebrated that. Teacher A feels strongly about celebration as well. I think there's a lot of celebration. As far as teachers go, we have a Golden Apple award. One teacher wins and gives it to another teacher the next time. We always celebrate the penny drive because it's number one in the nation for the last 5 years. We celebrated the class that did the best on the [STAR] exams in April the year before and then give them a day off... Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 95 Teacher D concurs in the celebratory nature of Grove High School and is happy that it extends beyond academics to the nature and character o f the student body. What we have celebrated, number one is which I really appreciate - it's not all about scores. The scores are celebrated. We talk a lot and we tell the students - it's done in assemblies, it's said in faculty meetings, we often get - anything that's written about us in the register is put in our boxes .... All of those kinds of things are celebrated school wide with academics. We often celebrate that our school is really generous. Our students are very, very generous - more generous than any school in our district. So, we're celebrating, oh, gosh I'm tearing up, [long pause] I would say that our kids are good - they're just good kids and we try to tell them that. Emergent Themes Examination of the raw data distilled multiple themes that are strongly represented and begin to tell the story of who Grove High School is and how it has achieved the success of student achievement that it enjoys. The different collection methods drew information from different sources and each complimented much of what became evident from the other tools. There are four major themes identified from the data. First, there is a common focus among the staff members on improving academic performance of students. Second, school wide instructional strategies have been implemented. Third, collaboration among staff members, especially within in departments is valued and sought after. Fourth, teachers desire instructional feedback and they can articulate what type is powerful for them in terms of improving their instruction. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 96 Common Focus Grove High School enjoys an organized push in the same direction; a common focus with eyes set on improving student achievement as measured by state testing. Since the school was identified as “under performing” five years ago a tremendous amount of effort and financial resources have been spent to increase test scores and exit out of program improvement status. The faculty meeting PowerPoint presentation by the principal was anchored in the achievement of officially climbing out of program improvement and then focusing on the next set of goals. Walking past the Peacock Pride poster, one cannot help but have their eyes fall upon the first two bullet points, “increased its state API score by 66 points in 2 years,” and, “exited Program Improvement (PI) and Under-Performing Status (II/USP) in a record 2 years.” Survey questions 34 through 37 directed support the unified progress towards school goals. Ninety-six percent o f those surveyed are aware of the goals and objectives of the school, more than 85% agree with the same goals and objectives, 77% believe the goals and objectives to be consistent with their own, and 96% agree that the goals and objectives have contributed to the schools improvement. Teacher C in response to the question of why is this school successful said: I think because as an administration, group, everybody says, “we want you people to do better and we're not gonna give you grades. The tests are important. Do good, etc.”. ... If the students don't care, maybe 10% are done in 10 minutes - what have they done? They've guessed. They've figured out, the older the kids, “this test doesn't mean a thing to me. Why should I put in the effort?” So Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 97 you have to change the mindset of most kids and say, “do the best you can.” Teacher A had a similar response to the same question. . . . we went from 2/7 to 4/10 in the last 3 years - that means 3 years ago, we weren't where we are now. Over that time, I think it might have to do with the motivation of the principal - he really wants to this to happen. You can tell that and I think the students can tell that because he pushes it every chance he gets. They make posters to go around the school and I think that has some effect. I don't know if it's just the effect of "hey, these tests matter," so the kids are going to try on them, which, of course, always has a big .... I think the motivation to do well on it pushes through the whole school year, not just to try on the test, but I think it pushes a lot of the teachers to do more .... School Wide Instructional Strategies Grove High School makes the claim to have implemented school wide strategies and the evidence backs them up. The campus is littered with posters that acknowledge writing, Cornell Notes, and graphic organizers as being a part of every aspect to the school’s curriculum. School documents indicate these strategies as being integral in a program trying to continually raise the achievement level of its students. The faculty meeting review of achievements, current efforts and future goals and objective clearly lists these strategies as essential parts of classroom practice. The interviews made two things very clear. First, the level of implementation of the strategies was not occurring at consistent levels, which is expected with almost 90 instructors operating with a great deal of autonomy. Second, and more important, is that each teacher, despite their particular degree of implementation, was well aware of the strategies and the school’s desired Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 98 application. An outcropping of this was the Common Agreement mentioned by two of the teachers. Common Agreements are the intention of departments to come together and identify commonalities in policy and procedure that will further unite the school and help create a single, connected education experience for the students. Collaboration Collaboration is the third emergent theme evident in the study of Grove High School. There are opportunities for teachers to get together and they value the experience as a manner through which to improve their instruction. The survey showed that 89% of the respondents agree with being encouraged to regularly collaborate on instructional matters 85% say they regularly engage in that activity. All four of the teachers interviewed acknowledge collaboration with their department colleagues as being one of, if not the, most powerful activity for improving their instruction. The TASEL-M project is very tangible evidence of how collaboration can be made more effective by providing resources and structured time for that purpose. Instructional Feedback The final theme is that of instructional feedback. Teachers want specific, critical feedback on what they are doing in the classroom, but without having to worry about critiques leading to the loss of employment. The survey clearly indicates the teacher perception of instructional feedback. Almost 63% agree that Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 99 formal teacher evaluation feedback assists them in improving their teacher practice. When they get feedback after an informal observation by administration or other support staff, 62% feel the feedback is meaningful. Seventy-seven percent o f the survey respondents feel comfortable going to their administration for support which can open the door for instructional feedback opportunities. All of the teachers interviewed recounted in varying levels of detail the kind of feedback they receive that truly has a positive impact on their classroom practice. The common thread in the stories is that these occurrences of feedback, although critical at times, were free from any summative judgments that could affect their employment status. As a department chair, Teacher D works closely with many teachers new to Grove High School and new to teaching. She related high levels of stress among new teachers worried about their position which precluded them from accepting critical feedback as a tool for self-improvement. The survey supports this in that 40% respondents don’t see supervision of instruction as non-evaluative and separate from the formal evaluation process. The evidence indicates that teachers who are more secure in their employment - feeling like they are meeting all minimum levels of performance to continue working - can move past fear and anxiety to truly desire critiques that indicate areas to focus growth efforts. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 100 Response to the Research Questions The driving force behind this study emanates from the four research questions first introduced in Chapter 1 and each is directly addressed in this section. Research Questions 1. What is the district’s policy and strategy for carrying out teacher evaluation? 2. How does the school carry out teacher evaluation? 3. What factors have shaped this school level effort? 4. How effective is the teacher evaluation process at the school level? The cornerstone of this study is the teacher evaluation process. The first level of discovery is how the policy is presented on paper. What is the district’s policy and strategy for carrying out teacher evaluation? The Grove Unified School District complies with all state guidelines regarding the teacher evaluation process. The teachers’ bargaining agreement states, “The evaluation and assessment of the performance o f unit members shall be made on a continuing basis.” Teachers will be evaluated annually until they achieve tenured status, at which point they will be evaluated every other year, unless they receive, “... any ‘unsatisfactory’ or ‘needs improvement’ subcategory rating ...” and will then be formally evaluated each year. The bargaining agreement continues on to describe a clear timeline by which Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 101 elements of the process need to be executed and a description of the parameters of the formal observation. The policy was modified for the 2004-05 school year to incorporate the California Standards for the Teaching Profession into the existing evaluation structure. As explained in a district memo for the Human Resource Office: The new certificated Final Evaluation and Rating Form reflects all six of the standards along with 31 of the 32 key elements contained in the California Standards for the Teaching Profession (CSTP). The six CSTP standards have replaced the categories of ‘Instructional Skills,’ ‘Management Skills,’ and ‘Personal and Professional Characteristics’ on the evaluation form. The category of ‘Supports District and School Goals, Plans, Policies and Procedures’ is carried over to the new evaluation. The second question focuses on execution - how the policy is brought to life on the school campus - how the school carries out teacher evaluation. There is evidence that the change in district policy may have had an effect on school site implementation. Teacher A, the only teacher who participated in the interviews and was not being evaluated this year valued the process much less then the other participants. Even so, no other interviewee clearly mentioned the new system’s impact as compared to the previous model. The principal presented a very comprehensive evaluation notebook distributed to staff members being evaluated that guided them through process from pre-observation planning to the observed lesson to the reflection upon evidence gathered from that observation. All of this material was generated because of the shift in focus of the evaluation to the California Standards for the Teaching profession. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 102 With multiple administrators carrying out the evaluations, their own characteristics influence, then, what the teachers experience. Feedback from Teachers B and D expressed timely adherence to the progression of evaluation events while Teacher C acknowledged delays up to months between observation and the return of written reports on what was seen. Teacher B shares, “This year, it was scheduled, it happened, it was a very thorough process. For the most part I think it was worthwhile - 1 definitely enjoyed it.” Teacher D, on the other hand, addressed the delays in getting feedback. “In fact, this time since I've been evaluated I haven't seen it. I know eventually, I'm gonna get it back, I think, but I haven't seen it.... It was first semester - [two months ago] I think in December.” The third research question seeks to understand what else is going on, formally or informally, that has had an impact on the school in such a way as to contribute to it success. What factors have shaped this school level effort? With any organization, the more coordinated the effort, no matter how diverse the participants, the more effective the system is for achieving the desired goals and objectives. Grove High School has a strong sense of itself in terms of what it wants to accomplish. Having been labeled Program Improvement in 2001 resulted in a united push - a common focus - towards the objectives necessary to move ahead and out of the Program Improvement status. School wide strategies are understood by all teachers and, to varying degrees, being implemented across the campus, across the curriculum. All teachers interviewed clearly articulated the school’s emphasis on writing, incorporating graphic organizers and using Cornell Notes. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 103 Collaboration among department colleagues is maximizing the skill base at Grove. TASEL-M, as an example, provides teachers ownership of their own growth and the resources of time and money to implement the plans they develop. All of these efforts are made possible due to, in large part, the principal. Yet, he has not been the mastermind behind every idea. There is strong evidence of shared leadership and participation, but his passion has been a large factor in the momentum of, not choosing the path, but continuing down it. The final question addresses how valuable the formal teacher evaluation system is in terms of the schools’ success. How effective is the teacher evaluation process at the school level? The results are mixed. The survey strongly indicates that teachers perceive the process to be effective. Over 70% of the respondents indicated that they view the administrator’s implementation of the teacher evaluation policies as an integral part of their professional growth. However, the interview provided a wide range of perspectives from the process being meaningless to neutral to effective. Aside from one teacher, the element that was cited as having the most potential to aide in growth was feedback. Other comments indicated that instructional feedback was more sought after and effective when it came from colleagues who understood the curriculum and had no connection to any summative employment decisions. There is evidence that the evaluation process is moving in that direction. The principal is very happy that the evaluation system now incorporates agreed upon teaching standards and he wants to build trust Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 104 through the process that allows for critical analysis and dialogue of teacher practice that will lead to growth and success. Summary The data collection generated information regarding the leadership and decision-making practice of the school, the formal evaluation process, staff perceptions of what impacts classroom practice, and school wide practices regarding both a common goal and instructional strategies. Although the understanding among staff differs, there is a belief that teachers are a part of the decision making process. There is confidence in the leadership abilities of other staff members. A common goal among the staff to improve student achievement as measured by state testing clearly emerged. This was cited as having a positive impact on classroom instructional practice, along with school wide instructional strategies and critical, instructional feedback. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 5 105 CONCLUSIONS, RECOMMENDATIONS AND IMPLICATIONS Introduction The No Child Left Behind initiative and similar policies have every school official seeking the most direct path leading to higher student achievement. The common factor contributing to student success continues to be the teacher. Multiple studies have shown that better skilled teachers lead to higher student achievement (Brophy, 1986; Ebmeier, 2003; Goddard, Hoy & Hoy, 2000; Stronge & Tucker, 2000). The California academic environment is no exception. Student achievement can be approached in two different ways with regards to teaching practices. Schools can either search for new programs to implement, offering different strategies that may be more effective than current practices, or schools can focus on modifying what is already being done and policies already in place to generate improved teacher, or instructional, practices. To properly comprehend the dynamics of instructional supervision today, it is necessary to understand from where it has evolved. John Smyth (as cited in Reitzug, 1997) explained that the very word “supervision” has Latin roots to the term that meant to scan or review a text for errors and ways that it differed from the original. The literature clearly tells a story of how supervision began as an inspection by lay people, transitioned into an inspection by officially appointed Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 106 bureaucrats, and is now most commonly seen more as collaboration with a strong element of inspection still remaining (Ebmeier & Nicklaus, 1999; Ellett & Teddlie, 2003; Siens & Ebmeier, 1996; Sullivan & Glanz, 2000). In a study included in their book, Glickman, Gordon and Ross-Gordon (2003) generated a list of words that teachers associate with “teacher supervision.” This, in turn, generated some insight in how strong the inspection element has endured with words like control, directive, big brother, dog and pony show, paperwork, and authority, among others. The literature speaks both indirectly and directly to a “crisis” in terms of the effectiveness of the supervision and evaluation processes today (Sullivan, 2000; Sullivan & Glanz, 2000) and address alternative approaches to traditional evaluation systems (Phillips, 2003; Zepeda, 2002). At the same time, there are no definitive studies identifying what works well and what can be built upon with regards to the supervision and evaluation process. Multiple textbooks talk about effective implementation of clinical supervision, but there are no inside views on how the supervision and evaluation processes are working to improve teacher practices, and, eventually, student achievement (Glickman, Gordon & Ross- Gordon, 2004; Sullivan & Glanz, 2000). Even with that, Reitzug (1997) criticizes a great number o f textbooks of supervision for presenting a theory of shared inquiry and collaboration embedded in a reality of hierarchical inspection. Reitzug (1997) went on to cite Smyth who points out in his 1991 article that an apparent contradiction exists when clinical supervision is imposed upon a teacher to diagnose and give direction on how to fix the deficiencies. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 107 This chapter serves to summarize the findings of this study in a way that begins to chip away at questions brought up by the vast amounts written regarding effective instructional supervision and teacher evaluation. The purpose of the study is reviewed and the findings are summarized. The paper draws conclusions and ends with implications for further practice and study. Purpose of the Study The purpose of this case study is to understand and describe the policies and procedures governing the evaluation and supervision o f teachers in one urban high school, identified by its high rating when compared to its similar schools listing, and the perceptions of those policies and procedures by the teaching and administrative staff with regards to their impact on teacher practices. The policies and procedures will be those emanating from the district, as well as the site, and affecting the evaluation process of certificated employees and other support elements of instructional supervision, such as professional development opportunities. This study will also examine any informal structures that afford teachers an opportunity to grow and improve in their classroom practices. Finally, this study will seek to describe the school’s organizational culture through the lens of Bolman and Deal’s (2003) Four Frames to understand what elements of the culture impact the overall teacher supervision process in both negative and positive Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 108 ways. At this stage in the research, the evaluation and supervision of teachers will be generally defined as instructional supervision. Summary of the Findings The data collected from the four different tools worked together to present a picture of Grove High School that illustrates four themes contributing to its success: a common focus on student achievement through state exams, school wide instructional strategies, collaboration within departments, and a desire for critical feedback on classroom instruction. These themes inter-relate to one another, have significance to the school’s success as individual factors, and influence other structures such as teacher evaluation. The teacher evaluation process at Grove High School, as established by district and site policy, does not provide any innovation to the greater body of knowledge. It follows the same state guidelines as all other public schools in California and bases its structure on the clinical model of supervision established by Cogan, Goldhammer and others. In some ways, having only switched to a focus on the California Standards for the Teaching Profession recently, the system is catching up to many other districts. There was little to no evidence of open hostility towards the policy or how evaluations are implemented, but some teachers indicated that it was a hoop - a bureaucratic process that is hoped to be painless. Even the teachers that indicated they enjoyed the process and were better teachers Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 109 for it did not praise the entire evaluation process as much as they did the feedback component, which they equally enjoyed from other sources. The true excitement came through the description of colleague interaction which generated both discussion of instructional issues and feedback regarding classroom practice. In the interviews Teacher D relayed a very passionate story about struggling with a class period of a course she had never taught. She sought out other teachers with conference periods that would allow them to observe her. The insight and critical analysis she received as a result were not only meaningful, but it remedied the problem she was experiencing. The other interviewees recounted similar experiences where they received meaningful input from collaboration with, not only, other teachers, but support staff as well. The survey clearly indicated that teachers value the contribution in terms of instructional supervision and feedback that comes from the site’s support staff. This seems to exclusively come from the “ladies in 301.” The Resource Office is the location out of which the coordinators of Categorical Programs, Title I, and English Language work. The accounts provided by the Categorical Programs Coordinator and the Teacher B, who had the most contact with the office, were very consistent. The coordinators would formally and informally observe classroom practice and provide critical insight. They would help plan or brainstorm curricular activities and also help the teachers execute major and minor aspects to those programs. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 110 When working together, the teachers reported that they encountered significant success with various instructional situations. All the teachers expressed through the interviews how they valued curricular dialogue with departmental colleagues. Teachers B and D, who were both part of the English/Language Arts Department, expressed an exceptional amount of gratitude for all opportunities to brainstorm, exchange ideas, and critique classroom practice with their fellow English teachers. Teacher C’s description and review o f the TASEL-M project was quite convicting regarding its effectiveness. The program embodies both critical feedback and opportunities for collaboration. The power is not only in working together in the context of professional development, but also having the opportunity to work together to design and choose professional development activities. There is high level of ownership. The feedback is powerful in its application. The comments and suggestions provided by the knowledgeable project coach specifically target the elements of math instruction present in a lesson, as well as general classroom strategies. The common factor in all descriptions of powerful instructional feedback is the disconnectedness from any summative judgment. This supports the literature’s interpretation of how formative and summative evaluation impact teachers in different ways. Whether the summative element is extracted by the teacher’s attitude towards the process or by virtue of the observer having no hire/fire powers, this appears to be the key in a meaningful exchange of information. The principal of Grove clearly recognizes this by his own admission of trying to create a level of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. I l l trust with teachers that would produce an environment of two colleagues simply talking and sharing about how a good lesson could be better. Another powerful theme clearly evident from the study of Grove High School is the power of having a common understanding among the staff members of the organizational focus. There is room for disagreement and dialogue, but true synergy can only be achieved by the momentum created with multiple energy sources moving in the same direction. The survey, observation, and interviews, leave no doubt as to what the common, over-arching goal is for Grove High School - continue to build more effective instructional practice with the objective of increasing student performance as measured by the Standardized Testing and Reporting system. This common focus both supports and flows out of the implementation of school wide instructional strategies. It always seems that whether one is considering overcoming grief, a drug problem, or other situation, acknowledgment is the first step. The same holds true with uniform implementation of policy and program. The nature of teaching has created highly autonomous creatures and implementing any policies across classrooms is challenging. Although Grove teachers differ in their level of implementation, there is continuity in the awareness of expectation. School wide writing samples, Cornell Notes, and graphic organizers are recognized by all as being part of the plan to achieve the greatest levels of student success. This concept is then extended to the idea of the Common Agreements coming out of the TASEL-M project. Other departments are working Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 112 on Common Agreements in which some standardization is achieved in policy and procedures regarding grading, homework, tests, etc. The students will benefit from an unified, integrated educational experience as opposed to a disjointed set of six or more academic events that have little commonality. Recommendations for Practice Insight drawn from the data supplied by the study o f Grove High School may not be universal, but provides appropriate, and potentially powerful, considerations in dealing with elements in instructional supervision, including formal teacher evaluation. This study adds credence to the already established argument that formative evaluation has more potential for affecting growth in teacher practice when compared to the inspection-inspired elements of summative evaluation where the goal, on the part of the teacher, ceases to be professional growth and becomes presenting the right image to maintain or secure a job. Anxiety must be eliminated to reach the full potential in providing critical feedback. In many instances, instructional feedback is softened to be “friendly” enough to assuage the overly worried teacher. The result, as clearly explained by teachers through the interviews, is meaningless information that doesn’t provide any direction in making their teaching better. On the other hand, critical feedback that highlights elements upon which the teacher can improve won’t be accepted for worry of the effect on their employment status. With the reality of determining job Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 113 status of temporary and probationary teachers, this is a tenuous task when new staff members are involved. Formally separating out the processes and building capacity for observations and dialogues done by staff other than the direct supervisor, is one possible remedy. Grove’s former Instmctional Leadership Team with built-in opportunities for instructional leaders to provide this support was seemingly a step in this direction. The label of Program Improvement brought a significant amount of money that the school now no longer receives. Those funds enabled the school to create an Instructional Leadership Team of which its members had addition non-teaching periods that were used to meet collaboratively as a team and go into classrooms to provide support directly to teachers. Also, new state policy passed during the writing of this study now allows tenured teachers meeting certain minimum requirements of proficiency to be eligible for a five-year reprieve from formal evaluation. By eliminating the evaluation, the state and districts are missing an opportunity to modify the system. Evidence from this study, which puts staff collaboration with members of their department and other support staff in such a positive light, could serve as a guidepost. Districts could offer tenured teachers meeting certain proficiency requirements an alternative instead of a reprieve. An alternative form of evaluation in which teachers team up with a colleague to mentor, be mentored with respect to a specific aspect o f their teaching, or collaborate on an element of curriculum development could afford teachers formal, structured time to engage in activities proven to be of great worth in their development as an educator. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 114 Finally, the power of coordinated efforts and an understanding of common goals have a tremendous potential to positively affect students. A student’s sense of worth with regards to a test, a class or a project will most likely determine the level of effort expended on the task and the resulting product. Multiple interviews highlighted the power of what a student believes regarding a task set in front of them - especially such a task as the STAR exams. The principal of Grove High School led his staff in continual proclamations of the importance of state tests and the significance o f each and every victory made by student progress in terms of gaining testing goals set before them. The Grove High School community makes it difficult for a student to pass through its halls without hearing, at least, and agreeing, at most, that these tests are important and each person’s efforts are needed. School wide instructional strategies and programs, like writing days, Cornell Notes and graphic organizers, work in two different ways. Simply, powerful instructional strategies serve to improve the performance of both teachers and students with regards to teaching and learning the curriculum, respectively. Strongly implemented writing strategies, for example, will help students organize and express their thoughts which, in turn, make them a better learner. Additionally, each school wide element where the student sees the teachers working collectively reinforces the idea that all efforts are working together for increased student success. Students may begin to think, “We do these things because they are the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 115 best for our ability to achieve - not because teacher x or teacher y simply wants to do them.” Implications for Further Research The body of literature concerned with instructional supervision’s impact on classroom practice and student achievement is ever growing. The need to understand what elements of supervision, within, and separate from, the formal evaluation process, will continue until all schools perform at a level where all students are mastering the essential information needed to be successful in post graduate activities. This study opens a window onto one organization’s approach to supporting effective classroom practice within their specific cultural context and set of social norms. This study, and the others done in conjunction with the thematic group, should be the first in another thrust of studies that continue to examine the elements of instructional supervision that lead to the highest quality of classroom practice. The analysis of the data from this study suggests four additional areas of research to be carried on: 1. A vertical analysis of high achieving, middle achieving, and low achieving schools within the same similar schools grouping to compare and contrast elements of instructional supervision, including teacher evaluation systems, that may encourage or inhibit strong classroom practice. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 116 2. Alternative forms of evaluation for tenured teachers who meet proficiency requirements that enable them to enter into a productive, formative supervision process focused on what they see as their greatest instructional challenge. 3. A comparison of school structures which provide space and resources to facilitate defined instructional support by non-administrative personnel: lead teachers, program coordinators, coaches, department chairpersons, etc. 4. The impact of school wide strategies and how the formal evaluation system or elements of instructional supervision can support the most effective implementation. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. BIBLIOGRAPHY 117 Alberta Teachers Association Policy. (2004). Retrieved August 15, 2004, from http://www.1eaming.gov.ab.ca/educationguide/pol-plan/polregs/215.asp. Acheson, K.A. and Gall, M.D. (2003). Clinical Supervision and Teacher Development: Preservice and Inservice Applications (5th ed.). New York, NY: John Wiley & Sons. Bolman, L.G. and Deal, T.E. (1997). Reframing Organizations: Artistry. Choice, and Leadership (2n d ed.). San Francisco: Jossey-Bass. Brophy, J.E. (1986). Teacher influences on student achievement. American Psychologist 41. 1069-1077. BTSA Overview. (2001). Retrieved on September 16, 2004 from http://www.btsa.ca.gov/BTSA basics.htm. California Education Codes 44660-44665. Retrieved on July 18, 2004, from http://www.leginfo.ca.gov/calaw.html. California Formative Assessment and Support System for Teachers - What is it? (2001). Retrieved from www.omsd.kl2.ca.us/teacher/cfasst.pdf. California Standards for Teaching Profession. (2000). Retrieved on August 15, 2004, from http ://www.etc.ca. gov/reports/cstpreport.pdf. California Commission on Teacher Credentialing (CCTC)/Califomia Department of Education (CDE). (1997). California Standards for the Teaching Profession. Sacramento: California Commission on Teacher Credentialing. Carroll, D.H. (1997, January). A Comparison of Clinical Supervision and Evaluation. Paper presented at the annual meeting of the Southwest Educational Research Association, Austin, TX. Chow, A.P.Y., Wong E.K.P., Yeung, A.S., and Mo, K.W. (2002). Teachers’ perceptions of appraiser-appraisee relationships. Journal of Personnel Evaluation in Education 16(2), 85-101. Cogan, M.L. (1973). Clinical Supervision. Boston: Houghton Mifflin. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 118 Costa, A.L., and Kallick, B. (2000). Getting into the habit of reflection. Educational Leadership 57(7), 60-62. Creswell, J.W. (2003). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (2n d ed.). Thousand Oaks: Sage Publications. Dagley, D.L. and Orso, J.K. (1991). Integrating formative, summative, modes of evaluation. National Association of Secondary School Principals 53(6), 72- 82. Danielson, C. and McGreal, T. (2003). Teacher Evaluation: To Enhance Professional Practice. Alexandria, VA: Association for Supervision and Curriculum Development (ASCD). Darling-Hammond, L. (1996). What matters most: A competent teacher for every child. Phi Delta Kappan 78(31.193-200. Davis, D.R., Ellett, D.D., and Annunziata, J. (2002). Teacher evaluation, leadership and learning organizations. Journal of Personnel Evaluation in Education 16(4), 287-301. Department of Education. (2003). The secretary’s third annual report on teacher quality. Retrieved on September 18, 2004 from http://www.ed.gov/about/ reports/annual/ teachnrep/2004/teacherqualitv pg5.html#chapter-l. Donaldson, C. and Stobbe, C. (2000). Teacher evaluation: A self-directed, inquiry based approach. Thrust for Educational Leadership 29(3), 30-32. Ebmeir, H. (2003). How supervision influences teacher efficacy and commitment: An investigation of a path model. Journal of Curriculum and Supervision 18(2), 110-141. Ebmeier, H. and Nicklaus, J. (1999). The impact of peer and principal collaborative supervision on teachers’ trust, commitment, desire for collaboration, and efficacy. Journal of Curriculum and Supervision 14(4), 351-378. EdSource. (1999). Collective Bargaining: Explaining California’s System. Retrieved on August 15,2004, from http://www.edsource.org/ pub abs collective.cfm. Egleson, P. (1994) Teacher evaluation plans that support professional growth. ERIC Document Reproduction Service, No ED 376 83. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 119 Ellett, C.D. and Teddlie, C. (2003). Teacher evaluation, teacher effectiveness and school effectiveness: perspectives from the USA. Journal of Personnel Evaluation in Education 17, 1 (March), 101-128. Evans, D. (1992). The instructional leader must go. ERIC Document Reproduction Service. No. ED 350 649. Flanigan, G.E. (1975). Implementation of the Stull bill within the San Diego unified school district. Fenwick, T.J. (2001). Teacher supervision through professional growth plans: balancing contradictions and opening possibilities. Educational Administration Quarterly 37, 3 (August), 401-424. Gall, M.D., Gall, J.P., and Borg, W.R. (2003). Educational Research: An Introduction (7th ed.). San Francisco: Allyn and Bacon. Glanz, J. (2000). Supervision for the millennium: A retrospective and prospective. Focus on Education. (Fall). Glickman, C.D., Gordon, S.P., and Ross-Gordon, J.M. (2004). Supervision and Instructional Leadership: A Developmental Approach 16th ed.T San Francisco: Allyn and Bacon. Goddard, R.D., Hoy, W.K., and Woolfolk Hoy, A. (2000). Collective teacher efficacy: Its meaning, measure, and impact on student achievement. American Educational Research Journal 37(2), 479-507. Gullatt, D.E. and Ballard, L.M. (1998). Choosing the right process for teacher evaluation. American Secondary Education 26(3). 13-17. Heneman, H.G. and Milanowski, A.T. (2003). Continuing assessment of teacher reactions to a standards-based teacher evaluation system. Journal of Personnel Evaluation in Education 17(2), 173-195. Holland, P.E. and Garman, N. (2001). Toward a resolution of the crisis of legitimacy in the field of supervision. Journal of Curriculum and Supervision 16(2), 95-111. McQuarrie, F.O. and Wood, F.H. (1991). Supervision, staff development, and evaluation connections. Theory Into Practice 30, 91-96. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 120 Olebe, M. (2001). A decade of policy support for California’s new teachers: The beginning teacher support and assessment program. Teacher Education Quarterly 28(1), 71-84. Ovando, M.N. (2001). Teachers’ perceptions of a learner-centered teacher evaluation system. Journal of Personnel Evaluation in Education 15(3). 213-231. Pajak, E. (2002). Clinical supervision and psychological functions: A new direction for theory and practice. Journal of Curriculum and Supervision 17(3), 189-205. Peterson, K.D. (2000). Teacher Evaluation: A Comprehensive Guide to New Directions and Practices (2n d ed.). Thousand Oaks: Corwin Press. Peterson, K.D., Kelly, P., and Caskey, M. (2002). Ethical considerations for teacher in the evaluation of other teachers. Journal of Personnel Evaluation in Education 1614). 317-324. Phillips, J. (2003). Powerful learning: Creating learning communities in urban school reform. Journal of Curriculum and Supervision 18(3), 240-258. Reitzug, U.C. (1997). Images of principal instructional leadership: From super vision to collaborative inquiry. Journal of Curriculum and Supervision 12, 324-343. Rose, L.C. and Gallup, A.M. (2002). The 34th annual phi delta kappa/gallup poll of the public’s attitudes toward the public schools. Phi Delta Kappan (September). Sergiovanni, T.J. and Starrat, R.J. (1983). Supervision: Human Perspectives (3rd ed.). New York: McGraw-Hill. Siens, C.M. and Ebmeier, H. (1996). Developmental supervision and the reflective thinking of teachers. Journal of Curriculum and Supervision 11(4), 299- 319. Spears, H. (1953). Improving the Supervision of Instruction. Englewood Cliffs, NJ: Prentice Hall. Stronge, J.H. (1997). Evaluating Teaching: A Guide to Current Thinking and Best Practice. Thousand Oaks: Corwin Press. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 121 Stronge, J.H. and Tucker, P.D. (2000). Teacher Evaluation and Student Achievement. New York: National Education Association. Sullivan, S. and Glanz, J. (2000). Alternative approaches to supervision: Cases from the field. The Journal of Curriculum and Supervision 15(3), 212-235. Sullivan, S. and Glanz, J. (2000). Supervision That Improves Teaching: Strategies and Techniques. Thousand Oaks: Corwin Press. Tucker, P.D., Stronge, J.H., Gareis, C.R., and Beers, C.S. (2003). The efficacy of portfolios for teacher evaluation and professional development: Do they make a difference? Educational Administration Quarterly 39(3), 572-602. Tunison, S. (2001). Instructional supervision: The policy-practice rift. Journal of Educational Thought 35(1), 83-108. Van der Linde, C.H. (1998). Clinical supervision in teacher evaluation: A pivotal factor in the quality management of education. Education 119(2), 328-334. Weasmer, J. and Woods, A.M. (2003). Mentoring: Professional development through reflection. The Teacher Educator 39(1), 64-77. Wenglinksy, H. (2000). How teaching matters: Bringing the classroom back into discussion of teacher quality. Educational Testing Service. Whittaker, A., Synder, J., and Freeman, S. (2001). Restoring balance: A chronology o f the development and uses of the California standards for the teaching profession. Teacher Education Quarterly (winter), 85-107. Young, B.L. (1999). A model for beginning teacher support and assessment. Action in Teacher Education 21(1), 24-36, Zepeda, S.J. (2002). Linking portfolio development to clinical supervision: A case study. Journal of Curriculum and Supervision 18(1), 83-102. Zepeda, S.J. (2003). The Principal As Instructional Leader. Larchmonte, NY: Eye on Education. Zepeda, S.J. and Ponticell, J.A. (1998). At cross-purposes: What do teacher need, want, and get from supervision? Journal of Curriculum and Supervision 14(1), 68-87. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. APPENDIX A TEACHER SURVEY Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. SURVEY: TEACHER EVALUATION AND SUPERVISION (Teacher Version) 123 Instructions: The Dissertation Committee is very interested in gathering your perceptions as a member of the staff with regards the direct and indirect factors affecting the teacher evaluation and supervision processes currently taking place at this school. Below is a short survey containing a series of statements for you to evaluate based on your knowledge of the school. Please be as honest as possible, as your responses are completely anonymous and will only be used to produce findings on issues related teacher evaluation and supervision. You may rate each of the statements below on a four-point Likert Scale as follows: 0 = Strongly Disagree [SD] 1 = Disagree [D] 2 = Agree [A] 3 = Strongly Agree [SA] Background Information: 1) Number of years as a teacher: (Please circle one) 0-1 yrs. 2-5 yrs. 6-10 yrs. lly rs. + 2) Number of years at this location: (Please circle one) 0-1 yrs. 2-5 yrs. 6-10 yrs. lly rs. + Policy: Teachers ’ knowledge and perceptions o f official documents regarding the teacher evaluation process. _________________________________________________________SD D A SA 3) I am aware of the district’s policy regarding teacher evaluation. 0 1 2 3 4) I understand the district’s policy regarding teacher evaluation. 0 1 2 3 5) I agree with the goals and objectives of my district’s policy on teacher evaluation. 0 1 2 3 6) I am satisfied that my site administrators are carrying out the district’s policy on teacher evaluation with integrity. 0 1 2 3 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 124 Teacher Evaluation: The extent to which the administration actively participated in the formal evaluation process, as mandated by the California Education Code, which may have included observations, data collection, feedback, goal setting, and improvement strategies. SD D A SA 7) My administrator frequently observes my classroom for the purpose of evaluation. 0 1 2 3 8) My administrator and I often discuss the instructional strategies I use in my classroom. 0 1 2 3 9) I view my administrator’s implementation of the teacher evaluation policies as an integral part of the integral part of my professional growth. 0 1 2 3 10) When my administrator visits my classroom, he/she looks for things which we agree upon at a pre-conference. 0 1 2 3 11) I receive timely feedback following my observations. 0 1 2 3 12) The teacher evaluation feedback assists me improving my teacher practice. 0 1 2 3 13) 1 am confident in my administrator’s ability to evaluate my instructional practice. 0 1 2 3 14) There are alternative evaluation opportunities (e.g. PAR, portfolio, etc.) available at my school. 0 1 2 3 15) I have participated in an alternative evaluation process before. 0 1 2 3 16) I believe that my participation in the teacher evaluation process at this school has led to my professional growth. 0 1 2 3 Ongoing Teacher Supervision: The extent to which the administration actively participated in a supervision process through observations, data collections, feedback, goal setting, and improvement strategies. _________________________________________________________SD D A SA 17) The administration frequently observes my classroom. 0 1 2 3 18) Other instructional support staff (e.g. coaches, counselors, etc.) frequently observe my classroom. 0 1 2 3 19) I receive meaningful feedback regarding my teaching following an observation. 0 1 2 3 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 125 Ongoing Teacher Supervision (continued). SD D A SA 20) I have discussions with administrators regarding my teaching. 0 1 2 3 21) I have discussions with instructional support staff (not including site administrators) regarding my teaching. 0 1 2 3 22) I see the administration’s supervision of instruction as non-evaluative and separate from formal evaluation processes. 0 1 2 3 23) I am aware of the specific things that administrators look for when visiting my classroom. 0 1 2 3 24) I am confident in my administration’s ability to monitor my instructional practice. 0 1 2 3 25) I believe that my administration’s supervision of instruction improves my instructional practice. 0 1 2 3 26) I believe that other instructional support staffs supervision of instruction improves my instructional practice. 0 1 2 3 School Efforts: Teacher’ s perception o f school-based procedures, and activities, not including direct supervision, which may have led to school improvement. SD D A SA 27) I have multiple opportunities throughout the year to participate in professional development activities. 0 1 2 3 28) Teachers are encouraged to collaborate on instructional matters on a regular basis. 0 1 2 3 29) I collaborate with others on instructional matters on a regular basis. 0 1 2 3 30) I am aware of an official professional development plan that has been prepared to share with the district and school community at large. 0 1 2 3 31) I have a clear understanding of the professional development goals for my school. 0 1 2 3 32) Teachers have an active role in developing professional development goals and objectives. 0 1 2 3 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 126 School Culture: Teacher’ s perception o f " the way we do things around here. ” _________________________________________________________SD D A SA 33) I am comfortable going to my school administrators for support. 0 2 34) I am aware of the goals and objectives of this school. 0 2 35) I believe in the goals and objectives of this school. 0 2 36) The values of this school are consistent with my own values. 0 2 37) The goals and objectives of this school have contributed to our school’s improvement. 0 2 38) I am satisfied with the professional competence and teaching ability of my teaching colleagues. 0 2 39) I am satisfied with the professional competence and leadership ability of the administration. 0 2 40) I am satisfied with the professional competence and leadership ability of the instructional support staff (e.g., coaches, counselors, etc.) 0 2 41) I am satisfied with the professional competence and leadership ability of teachers in leadership roles at this school. 0 2 42) Faculty discussions regarding curriculum and instruction impact our school’s ability to improve. 0 2 43) Teachers have an active role in making decisions for the school. 0 2 44) Teachers have initiated efforts towards school improvement. 0 2 45) The majority of school improvement efforts at this school have been initiated by the district and/or site administration. 0 2 3 3 3 3 3 3 3 3 3 3 3 3 3 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Ill Teacher Beliefs: The teacher’ s belief that he/she can make a difference in student learning. _________________________________________________________SD D A SA 46) I believe that all students can learn. 0 1 2 3 47) When I really try, I can get through to most students. 0 1 2 3 48) If a student masters a new concept quickly, this might be because I knew the necessary steps in teaching that concept. 0 1 2 3 49) When a student is having difficulty with an assignment, I am usually able to adjust it to his/her level. 0 1 2 3 50) When the grades/proficiency levels of my students improve, it is usually because I used my effective teaching approaches. 0 1 2 3 51) I believe that all students can achieve at high levels. 0 1 2 3 52) The influence of a student’s home environment can be overcome by good teaching. 0 1 2 3 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 128 APPENDIX B DOCUMENT REVIEW FORM Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 129 D O C U M E N T R E V IE W F O R M Title of Document Date of Review Type of Document Author/Dccision- Makcr Location of Source How closely is this document related to school policy on teacher evaluation? How closely is this document related to other school level efforts? Linkages stated within document to improve practice Reflections Further questions for consideration . Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. APPENDIX C OBSERVATION TEMPLATE Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. O B S E R V A T IO N T E M P L A T E 131 ! i N otes Purpose o f A ctivity Dale/1 .ocation Participants General Sense Is this activity reflective o f policy/m andate im plem entation? Is this activity reflective o f the teacher evaluation process? D ocs this activity reflect teacher im provem ent or student achievem ent strategies? 1 D iscuss the instructional i leadership present. ....I. What are the perceptions o f the school site and its functions? D escribe the relationships'interactions taking place. ........... j . r ; - M X m Dom inant Frame and rationale ! ! ......................................... .............................................................. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. APPENDIX D THE INTERVIEW Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Teacher ID: Interview Questions Date Interviewed: 133 Policy: Teachers’ knowledge and perceptions of official documents regarding the teacher evaluation process.____________________________________________________ Question As E\ idcnccd Dominant Frame Rationale 1, How would you describe your district’s policy on teacher evaluation? 2. How does the teacher evaluation process affect your practice in the classroom? 3. What modifications would you recommend for the teacher evaluation policy that would improve your practice? Teacher Evaluation: The extent to which the administration actively participated in the formal evaluation process, as mandated by the California Education Code, which may have included observations, data collection, feedback, goal setting, and improvement strategies. Question As Evidenced Dominant Frame Rationale 4. Describe your experiences with the evaluation process. 5. What are the strengths and weaknesses of the evaluation process at this school? 6. What elements of the evaluation process are effective in improving teacher practice? Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Interview Questions 134 Teacher ID :____________ Date Interviewed:___________ Teacher Supervision: The extent to which the administration actively participate in the supervision process through observations, data collections, feedback, goal setting, and improvement strategies. Question As Evidenced Dominant Frame ; Rationale 7. Outside of the evaluation process, what types of interactions do/have you engaged in with administration, colleagues and other educators regarding instruction? 8. Outside the evaluation process, what types of instructional feedback are you given? School Efforts: Teacher’s perception of school-based procedures, and activities, not including direct supervision, which may lead to school improvement. Question As Evidenced Dominant Frame Rationale 9. Why is this school successful? 10. What school wide programs and/or strategies have been implemented to improve student achievement? 11. What efforts impact instructional practice? 12. How are these school efforts reflected in the teacher evaluation process? Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Interview Questions 135 Teacher ID: Date Interviewed: School Efforts (continued). Question As Evidenced Dominant Frame Rationale 13. Explain your school’s plan for professional development and its link to the school’s success. 14. How are individual teacher professional development goals and objectives linked to the evaluation process? 15. Who decides what and how professional development is offered? School Culture: Teacher’s perceptions of “the way we do things around here.” Question As Evidenced Dominant Frame Rationale 16. How are school site decisions made? 17. Where does the leadership come from at the school site? 18. How is conflict resolved at the school? 19. How is information communicated at the school site? 20. Whom do you view as school leaders? 21. What is celebrated here? Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 136 Interview Questions Teacher ID: ___________________ Date Interviewed:_ Teacher Beliefs: The teacher’s belief that he/she can make a difference in student learning. Question As Evidenced Dominant Frame Rationale 22. Do you believe that you are a valued member in this school? Why/Why not? 23. What factors do you believe contribute to student learning? Notes Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. APPENDIX E FREQUENCY REPORT FROM SURVEY Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. NOTES 138 Output Created 22-FEB-2005 19:01:26 Comments Input Data F :\USCYDissertation\kevinastorsurve yanalysis.sav Filter <none> Weight <none> Split File <none> N of Rows in Working Data File 29 Missing Value Handling Definition of Missing User-defined missing values are treated as missing. Cases Used Statistics are based on all cases with valid data. Syntax FREQUENCIES VARIABLES=ql q2 q3 q4 q5 q6 q7 q8 q9 qlO ql 1 ql2 q l3 ql4 q l5 ql6 ql7 ql8 ql9q20 q21 q22 q23 q24 q25 q26 q27 q28 q29 q30 q31 q32 q33 q34 q35 q36 q37 q38 q39 q40 q41 q42 q43 q44 q45 q46 q47 q48 q49 q50 q51 q52 /STATISTICS=MEAN /ORDER= ANALYSIS . Resources Elapsed Time 0:00:00.05 Total Values Allowed 149796 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. STATISTICS 139 N Mean Valid Missing Number of years as a teacher. 29 0 2.34 Number of years at this location. 29 0 1.72 I am aware of the district's policy regarding teacher evaluation. 29 0 2.66 I understand the district's policy regarding teacher evaluation. 29 0 2.62 I agree with the goals and objectives of my district's policy regarding teacher evaluation. 29 0 2.10 I am satisfied that my site administrators are carrying out the district's policy on teacher evaluation with integrity. 29 0 2.21 My administrator frequently observes my classroom for the purpose of evaluation. 29 0 1.21 My administrator and I often discuss the instructional strategies I use in my classroom. 28 1 1.14 I view my administrator's implementation of the teacher evaluation policies as an integral part of my professional growth. 27 2 1.59 When my administrator visits my classroom, he/she looks for things which we agreed upon at a pre conference. 27 2 1.81 I receive timely feedback following my observations. 28 1 1.75 The teacher evaluation feedback assists me in improving my teacher practice. 27 2 1.67 I am confident in my administrator's ability to evaluate my instructional practice. 27 2 2.07 There are alternative evaluation opportunities (e.g. PAR, portfolio, etc.) available at my school. 26 3 .92 I have participated in an alternative evaluation process before. 29 0 1.17 I believe that my participation in the teacher evaluation process at this school has led to my professional growth. 27 2 1.70 The administration frequently observes my classroom. 27 2 1.19 Other instructional support staff (e.g., coaches, counselors, etc.) frequently observe my classroom. 27 2 1.07 I receive meaningful feedback regarding my teaching following an observation. 27 2 1.59 I have discussions with administrators regarding my teaching. 27 2 1.56 I have discussions with instructional support staff (not including site administrators) regarding my teaching. 27 2 1.85 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 140 Statistics (continued). I see the administration's supervision of instruction as non-evaluative and separate from the formal evaluation process. 25 4 1.80 I am aware of the specific things that administrators look for when visiting my classroom. 27 2 2.26 I am confident in my administration's ability to monitor my instructional practice. 27 2 2.07 I believe that my administration's supervision of instruction improves my instructional practice. 27 2 1.78 I believe that other instructional support staffs supervision of instruction improves my instructional practice. 27 2 1.93 I have multiple opportunities throughout the year to participate in professional development activities. 28 1 2.43 Teachers are encouraged to collaborate on instructional matters on a regular basis. 28 1 2.36 I collaborate with others on instructional matters on a regular basis. 28 1 2.39 I am aware of an official professional development plan that has been prepared to share with the district and school community at large. 28 1 2.25 I have a clear understanding of the professional development goals for my school. 28 1 1.86 Teachers have an active role in developing professional development goals and objectives. 28 1 1.68 I am comfortable going to my school administrators for support. 27 2.15 I am aware of the goals and objectives of this school. 28 1 2.50 I believe in the goals and objectives of this school. 28 1 2.18 The values of this school are consistent with own values. 27 2.11 The goals and objectives of this school have contributed to our school's improvement. 28 1 2.36 I am satisfied with the professional competence and teaching ability of my teaching colleagues. 28 1 2.00 I am satisfied with the professional competence and leadership ability of the administration. 28 1 2.07 I am satisfied with the professional competence and leadership ability of the instructional support staff (e.g. coaches, coordinators, etc.) 28 1 1.86 I am satisfied with the professional competence and leadership ability of the teachers in leadership roles at this school. 28 1 2.04 Faculty discussions regarding curriculum and instruction impact our school's ability to improve. 27 2 2.07 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 141 Statistics (continued). Teachers have an active role in making decisions for the school. 28 1 1.64 Teachers have initiated efforts towards school improvement. 28 1 2.14 The majority of school improvement efforts at this school have been initiated by the district and/or site administration. 28 1 1.93 I believe that all students can learn. 28 1 2.89 When I really try, I can get through to most students. 28 1 2.43 If a student masters a new concept quickly, this might be because I knew the necessary steps in teaching that concept. 28 1 2.29 When a student is having difficulty with an assignment, I am usually able to adjust it to his/her level. 28 1 2.29 When the grades/proficiency levels of my students improve, it is usually because I used my effective teaching approaches. 28 1 2.18 I believe that all students can achieve at high levels. 28 1 2.21 The influence of a student's home environment can be overcome by good teaching. 28 1 1.86 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 142 FREQUENCY TABLES Number of Years as a Teacher Frequency Percent Valid Percent Cumulative Percent Valid 2-5 yrs. 5 17.2 17.2 17.2 6-10 yrs. 9 31.0 31.0 48.3 11 yrs. and over 15 51.7 51.7 100.0 Total 29 100.0 100.0 Number of Years at This Location Frequency Percent Valid Percent Cumulative Percent Valid 0-1 yrs. 2 6.9 6.9 6.9 2-5 yrs. 11 37.9 37.9 44.8 6-10 yrs. 9 31.0 31.0 75.9 11 yrs. and over 7 24.1 24.1 100.0 Total 29 100.0 100.0 I am aware of the district's policy regarding teacher evaluation. Frequency Percent Valid Percent Cumulative Percent Valid Disagree 1 3.4 3.4 3.4 Agree 8 27.6 27.6 31.0 Strongly Agree 20 69.0 69.0 100.0 Total 29 100.0 100.0 I understand the district's policy regarding teacher evaluation. Frequency Percent Valid Percent Cumulative Percent Valid Disagree 1 3.4 3.4 3.4 Agree 9 31.0 31.0 34.5 Strongly Agree 19 65.5 65.5 100.0 Total 29 100.0 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 143 I agree with the goals and objectives of my district's policy regarding teacher evaluation. Frequency Percent Valid Percent Cumulative Percent Valid Strongly Disagree 1 3.4 3.4 3.4 Disagree 4 13.8 13.8 17.2 Agree 15 51.7 51.7 69.0 Strongly Agree 9 31.0 31.0 100.0 Total 29 100.0 100.0 I am satisfied that my site administrators are carrying out the district's policy on teacher evaluation with integrity. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 2 6.9 6.9 6.9 Disagree 3 10.3 10.3 17.2 Agree 11 37.9 37.9 55.2 Strongly Agree 13 44.8 44.8 100.0 Total 29 100.0 100.0 My administrator frequently observes my classroom for the purpose of evaluation. Frequency Percent Valid Percent Cumulative Percent Valid Strongly Disagree 6 20.7 20.7 20.7 Disagree 14 48.3 48.3 69.0 Agree 6 20.7 20.7 89.7 Strongly Agree 3 10.3 10.3 100.0 Total 29 100.0 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 144 My administrator and I often discuss the instructional strategies I use in my classroom. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 8 27.6 28.6 28.6 Disagree 10 34.5 35.7 64.3 Agree 8 27.6 28.6 92.9 Strongly Agree 2 6.9 7.1 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 I view my administrator's implementation of the teacher evaluation policies as an integral part of my professional growth. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 6 20.7 22.2 22.2 Disagree 2 6.9 7.4 29.6 Agree 16 55.2 59.3 88.9 Strongly Agree 3 10.3 11.1 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 When my administrator visits my classroom, he/she looks for things which we agreed upon at a pre-conference. Frequency Percent Valid Percent Cumulative Percent Valid Strongly 4 13.8 14.8 14.8 Disagree Disagree 3 10.3 11.1 25.9 Agree 14 48.3 51.9 77.8 Strongly 6 20.7 22.2 100.0 Agree Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. I receive timely feedback following my observations. 145 Frequency Percent Valid Percent Cumulative Percent Valid Strongly Disagree Disagree 3 6 10.3 20.7 10.7 21.4 10.7 32.1 Agree 14 48.3 50.0 82.1 Strongly Agree 5 17.2 17.9 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 The teacher evaluation feedback assists me in improving my teacher practice. Frequency Percent Valid Percent Cumulative Percent Valid Strongly 4 13.8 14.8 14.8 Disagree Disagree 6 20.7 22.2 37.0 Agree 12 41.4 44.4 81.5 Strongly 5 17.2 18.5 100.0 Agree Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 I am confident in my administrator's ability to evaluate my instructional practice. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.7 3.7 Disagree 5 17.2 18.5 22.2 Agree 12 41.4 44.4 66.7 Strongly Agree 9 31.0 33.3 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 146 The are alternative evaluation opportunities (e.g. PAR, portfolio, etc.) available at my school. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 9 31.0 34.6 34.6 Disagree 12 41.4 46.2 80.8 Agree 3 10.3 11.5 92.3 Strongly Agree 2 6.9 7.7 100.0 Total 26 89.7 100.0 Missing System 3 10.3 Total 29 100.0 I have participated in an alternative evaluation process before. Frequency Percent Valid Percent Cumulative Percent Valid Strongly Disagree 11 37.9 37.9 37.9 Disagree 8 27.6 27.6 65.5 Agree 4 13.8 13.8 79.3 Strongly Agree 6 20.7 20.7 100.0 Total 29 100.0 100.0 I believe that my participation in the teacher evaluation process at this school has led to my professional growth. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 4 13.8 14.8 14.8 Disagree 4 13.8 14.8 29.6 Agree 15 51.7 55.6 85.2 Strongly Agree 4 13.8 14.8 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 147 The administration frequently observes my classroom. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 6 20.7 22.2 22.2 Disagree 13 44.8 48.1 70.4 Agree 5 17.2 18.5 88.9 Strongly Agree 3 10.3 11.1 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 Other instructional support staff (e.g. coaches, counselors, etc.) frequently observe my classroom. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 9 31.0 33.3 33.3 Disagree 8 27.6 29.6 63.0 Agree 9 31.0 33.3 96.3 Strongly Agree 1 3.4 3.7 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 I receive meaningful feedback regarding my teaching following an observation. Frequency Percent Valid Percent Cumulative Percent Valid Strongly 5 17.2 18.5 18.5 Disagree Disagree 5 17.2 18.5 37.0 Agree 13 44.8 48.1 85.2 Strongly 4 13.8 14.8 100.0 Agree Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 148 I have discussions with administrators regarding my teaching. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 4 13.8 14.8 14.8 Disagree 8 27.6 29.6 44.4 Agree 11 37.9 40.7 85.2 Strongly Agree 4 13.8 14.8 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 I have discussions with instructional support staff (not including site administrators) regarding my teaching. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 4 13.8 14.8 14.8 Disagree 3 10.3 11.1 25.9 Agree 13 44.8 48.1 74.1 Strongly Agree 7 24.1 25.9 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 I see the administration's supervision of instruction as non-evaluative and separate from the formal evaluation process. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 2 6.9 8.0 8.0 Disagree 8 27.6 32.0 40.0 Agree 8 27.6 32.0 72.0 Strongly Agree 7 24.1 28.0 100.0 Total 25 86.2 100.0 Missing System 4 13.8 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 149 I am aware of the specific things that administrators look for when visiting my classroom. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.7 3.7 Disagree 2 6.9 7.4 11.1 Agree 13 44.8 48.1 59.3 Strongly Agree 11 37.9 40.7 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 I am confident in my administration's ability to monitor my instructional practice. Cumulative Frequency Percent Valid Percent Percent Valid Disagree 5 17.2 18.5 18.5 Agree 15 51.7 55.6 74.1 Strongly Agree 7 24.1 25.9 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 I believe that my administration's supervision of instruction improves my instructional practice. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 3 10.3 11.1 11.1 Disagree 4 13.8 14.8 25.9 Agree 16 55.2 59.3 85.2 Strongly Agree 4 13.8 14.8 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 150 I believe that other instructional support staffs supervision of instruction improves my instructional practice. Frequency Percent Valid Percent Cumulative Percent Valid Strongly 4 13.8 14.8 14.8 Disagree Disagree 1 3.4 3.7 18.5 Agree 15 51.7 55.6 74.1 Strongly 7 24.1 25.9 100.0 Agree Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 I have multiple opportunities throughout the year to participate in professional development activities. Cumulative Frequency Percent Valid Percent Percent Valid Disagree 2 6.9 7.1 7.1 Agree 12 41.4 42.9 50.0 Strongly Agree 14 48.3 50.0 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Teachers are encouraged to collaborate on instructional matters on a regular basis. Cumulative Frequency Percent Valid Percent Percent Valid Disagree 3 10.3 10.7 10.7 Agree 12 41.4 42.9 53.6 Strongly Agree 13 44.8 46.4 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 151 I collaborate with others on instructional matters on a regular basis. Cumulative Frequency Percent Valid Percent Percent Valid Disagree 4 13.8 14.3 14.3 Agree 9 31.0 32.1 46.4 Strongly Agree 15 51.7 53.6 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 I am aware of an official professional development plan that has been prepared to share with the district and school community at large. Cumulative Frequency Percent Valid Percent Percent Valid Disagree 5 17.2 17.9 17.9 Agree 11 37.9 39.3 57.1 Strongly Agree 12 41.4 42.9 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 I have a clear understanding of the professional development goals for my school. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.6 3.6 Disagree 9 31.0 32.1 35.7 Agree 11 37.9 39.3 75.0 Strongly Agree 7 24.1 25.0 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 152 Teachers have an active role in developing professional development goals and objectives. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 4 13.8 14.3 14.3 Disagree 8 27.6 28.6 42.9 Agree 9 31.0 32.1 75.0 Strongly Agree 7 24.1 25.0 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 I am comfortable going to my school administrators for support. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 2 6.9 7.4 7.4 Disagree 4 13.8 14.8 22.2 Agree 9 31.0 33.3 55.6 Strongly Agree 12 41.4 44.4 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 I am aware of the goals and objectives of this school. Cumulative Frequency Percent Valid Percent Percent Valid Disagree 1 3.4 3.6 3.6 Agree 12 41.4 42.9 46.4 Strongly Agree 15 51.7 53.6 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 153 I believe in the goals and objectives of this school. Frequency Percent Valid Percent Cumulative Percent Valid Strongly 1 3.4 3.6 3.6 Disagree Disagree 3 10.3 10.7 14.3 Agree 14 48.3 50.0 64.3 Strongly 10 34.5 35.7 100.0 Agree Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 The values of this school are consistent with own values. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.7 3.7 Disagree 5 17.2 18.5 22.2 Agree 11 37.9 40.7 63.0 Strongly Agree 10 34.5 37.0 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 The goals and objectives of this school have contributed to our school's improvement. Cumulative Frequency Percent Valid Percent Percent Valid Disagree 1 3.4 3.6 3.6 Agree 16 55.2 57.1 60.7 Strongly Agree 11 37.9 39.3 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 154 I am satisfied with the porofessional competence and teaching ability of of my teaching colleagues. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.6 3.6 Disagree 6 20.7 21.4 25.0 Agree 13 44.8 46.4 71.4 Strongly Agree 8 27.6 28.6 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 I am satisfied with the professional competence and leadership ability of the administration. Cumulative Frequency Percent Valid Percent Percent Valid Disagree 5 17.2 17.9 17.9 Agree 16 55.2 57.1 75.0 Strongly Agree 7 24.1 25.0 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 I am satisfied with the professional competence and leadership ability of the instructional support staff (e.g. coaches, coordinators, etc.) Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.6 3.6 Disagree 6 20.7 21.4 25.0 Agree 17 58.6 60.7 85.7 Strongly Agree 4 13.8 14.3 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 155 I am satisfied with the professional competence and leadership ability of the teachers in leadership roles at this school. Frequency Percent Valid Percent Cumulative Percent Valid Strongly Disagree Disagree 1 3 3.4 10.3 3.6 10.7 3.6 14.3 Agree 18 62.1 64.3 78.6 Strongly Agree 6 20.7 21.4 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Faculty discussions regarding curriculum and instruction impact our school's ability to improve. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.7 3.7 Disagree 4 13.8 14.8 18.5 Agree 14 48.3 51.9 70.4 Strongly Agree 8 27.6 29.6 100.0 Total 27 93.1 100.0 Missing System 2 6.9 Total 29 100.0 Teachers have an active role in making decisions for the school. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 2 6.9 7.1 7.1 Disagree 10 34.5 35.7 42.9 Agree 12 41.4 42.9 85.7 Strongly Agree 4 13.8 14.3 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 156 Teachers have initiated efforts towards school improvement. Frequency Percent Valid Percent Cumulative Percent Valid Strongly Disagree Disagree 1 4 3.4 13.8 3.6 14.3 3.6 17.9 Agree 13 44.8 46.4 64.3 Strongly Agree 10 34.5 35.7 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 The majority of school improvement efforts at this school have been initiated by the district and/or site administration. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 2 6.9 7.1 7.1 Disagree 7 24.1 25.0 32.1 Agree 10 34.5 35.7 67.9 Strongly Agree 9 31.0 32.1 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 I believe that all students can learn. Cumulative Frequency Percent Valid Percent Percent Valid Agree 3 10.3 10.7 10.7 Strongly Agree 25 86.2 89.3 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 157 When I really try, I can get through to most students. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.6 3.6 Disagree 1 3.4 3.6 7.1 Agree 11 37.9 39.3 46.4 Strongly Agree 15 51.7 53.6 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 If a student masters a new concept quickly, this might be because I knew the necessary steps in teaching that concept. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.6 3.6 Disagree 1 3.4 3.6 7.1 Agree 15 51.7 53.6 60.7 Strongly Agree 11 37.9 39.3 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 When a student is having difficulty with an assignment, I am usually able to adjust it to his/her level. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.6 3.6 Disagree 2 6.9 7.1 10.7 Agree 13 44.8 46.4 57.1 Strongly Agree 12 41.4 42.9 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 158 When the grades/proficiency levels of my students improve, it is usually because I used my effective teaching approaches. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.6 3.6 Disagree 2 6.9 7.1 10.7 Agree 16 55.2 57.1 67.9 Strongly Agree 9 31.0 32.1 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 I believe that all students can achieve at high levels. Cumulative Frequency Percent Valid Percent Percent Valid Disagree 6 20.7 21.4 21.4 Agree 10 34.5 35.7 57.1 Strongly Agree 12 41.4 42.9 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 The influence of a student's home environment can be overcome by good teaching. Cumulative Frequency Percent Valid Percent Percent Valid Strongly Disagree 1 3.4 3.6 3.6 Disagree 7 24.1 25.0 28.6 Agree 15 51.7 53.6 82.1 Strongly Agree 5 17.2 17.9 100.0 Total 28 96.6 100.0 Missing System 1 3.4 Total 29 100.0 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

A case study of the California teacher evaluation system and its impact upon teacher practice in an alternative education high -performing urban high school

PDF

A case study of teacher evaluation and supervision at a high performing urban elementary school

PDF

Arts in education and organizational culture: The impact of the arts on the culture of a school

PDF

California teacher evaluation and improved teacher practice

PDF

An analysis of perceptions and implementation of California's teacher evaluation process in a K--5 public school and its impact on teacher practice

PDF

An elementary school's perceptions of the effectiveness of teacher evaluation to enhance teacher practice

PDF

An analysis of the implementation of content standards in selected unified school districts of California

PDF

A comparative analysis of academic achievement for CalWORKs students in a K--12 public school system

PDF

Elementary teacher perceptions of school violence: Safe school elements and responsibility.

PDF

A closer look at the impact of teacher evaluation: A case study in a high performing California elementary school

PDF

A causal, dispositional model of common personal characteristics associated with teacher job satisfaction

PDF

A case study of teacher evaluation and supervision at a high -achieving urban elementary school

PDF

21st century skills practices and programs: a case study at an elementary school

PDF

Current and future managerial competency requirements for manufacturing, assembly, and /or material processing functions

PDF

Class size reduction: A case study

PDF

A comparison of public and private governance in new teacher induction practices

PDF

A study of doctoral student -advisor satisfaction: Considering gender and ethnic grouping at a private research university

PDF

A comparison of the perceptions of senior and junior enlisted Air Force personnel concerning deterrents to enrollment in and completion of adult education courses by junior personnel

PDF

Beating the odds: Does the formal teacher evaluation process make the difference in teacher performance?

PDF

A case study of social promotion and retention policies, strategies, and programs

Asset Metadata

Creator Astor, Kevin Edward (author)

Core Title A case study of instructional supervision, including teacher evaluation, and the impact on teacher practice

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, administration,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Gothold, Stuart (committee chair), Bowman, Gregory (committee member), Hocevar, Dennis (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-380918

Unique identifier UC11340381

Identifier 3180459.pdf (filename),usctheses-c16-380918 (legacy record id)

Legacy Identifier 3180459.pdf

Dmrecord 380918

Document Type Dissertation

Rights Astor, Kevin Edward

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA