University of Southern California Dissertations and Theses

The sustainability of superintendent-led reforms to improve student achievement

(USC Thesis Other)

The sustainability of superintendent-led reforms to improve student achievement

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content THE SUSTAINABILITY OF SUPERINTENDENT-LED REFORMS TO IMPROVE STUDENT ACHIEVEMENT by Rick Edward Bagley ____________________________________________________________________ A Dissertation Presented to the FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF EDUCATION December 2012 Copyright 2012 Rick Edward Bagley ii DEDICATION I dedicate this dissertation to my wife Karla; to my parents, Beverly and James Bagley; and to the memory of my grandmother, Ruth Kolman. They taught me I could do anything and stood by me while I tried. iii ACKNOWLEDGMENTS In 1676 letter to fellow scientist Robert Hooke, Sir Isaac Newton wrote, “If I have seen further, it is by standing on ye shoulders of giants.” Newton’s acknowledgement of others’ contributions toward his own accomplishments, discoveries, and successes was profound, and it applies to each of us in all we do. Having spent the past 52 years as a student, educator, and now scholar, Newton’s words have particular significance and meaning to me. My personal list of giants is long and bears the names of those whose contribu- tions enabled me to reach for my potential, compensate for the many challenges that I encountered throughout my journey, and ultimately helped to define the context of my life. They include all of my students, teachers, mentors, professors, colleagues, friends, and family. There is not sufficient space to list them all here, but to each I owe a debt of gratitude for helping me to see further than my eyes will ever allow. First, I acknowledge and thank my dissertation committee: Dr. Pedro Garcia (Chair); Dr. Rudy Castruita; and my colleague and friend of 20 years, Dr. Michael Matthews. I am grateful to the faculty and staff of the USC Rossier School of Education; my classmates in the EdD cohort of 2010; and my editor, Mrs. Phyllis Parmet, whose support, guidance, camaraderie, and encouragement made all the difference and inspired me to press on when I wanted to stop. Special thanks also go to the 51 superintendents who responded to my survey and the five who willingly and generously shared their time and wisdom during my interviews with them. iv My appreciation also extends to the Board of Trustees and Senior Cabinet in both the Manhattan Beach Unified School District and the Lawndale Elementary School District. Not only did these leaders encourage me to embark on this journey in the first place, but they also supported it in their words and by their deeds. Finally, my deepest appreciation goes to my family: daughters Jessica and Elyse, son-in-law Scott, and baby grandson Henry. Most of all, my wife and life partner of 34 years, Karla, sacrificed as much as I did and probably more to see me reach this long- awaited goal. v TABLE OF CONTENTS Dedication ii Acknowledgments iii List of Tables vii List of Figures viii Abstract ix Chapter 1: Overview of the Study 1 Statement of the Problem 3 Purpose of the Study 4 Research Questions 4 Significance of the Study 5 Assumptions 6 Limitations of the Study 7 Delimitations of the Study 7 Definitions of Terms 8 Organization of the Dissertation 9 Chapter 2: Review of the Literature 10 Educational Reform 11 Hoyle’s Three Periods of School Reform 12 First period: 1982 to 1986 12 Second period: 1986 to 1989 13 Third period: 1989 to 2003 13 Comprehensive School Reform (CSR) 14 Types of School Reform 15 Section Summary 16 Sustainability of School Reform Efforts 16 Challenges to Sustainability 18 Section Summary 22 Leadership 22 Superintendents as Leaders 23 Superintendent-Led Reforms 24 Superintendent Tenure 26 The Relationship Between Superintendent Tenure and Student Achievement 29 Section Summary 30 Chapter Summary 31 Chapter 3: Methodology 33 Research Design Methodology 33 Sample and Population 35 Instrumentation 37 vi Quantitative Phase 37 Qualitative Phase 38 Data Collection 39 Quantitative Phase 39 Qualitative Phase 40 Data Analysis 40 Chapter Summary 42 Chapter 4: The Findings 43 Quantitative Phase: Demographics of Study Respondents 44 District Type and Size 44 Superintendent Gender 46 Superintendent Age 47 Tenure in Current Position 48 Tenure in Prior Position(s) 49 Summary of Quantitative Demographic Data 49 Qualitative Phase 50 Research Question 1 51 Research Question 2 56 Reform strengths 57 Reform weaknesses 59 Summary 60 Research Question 3 60 Research-based elements of sustainability 61 Alignment based on perceived reform strengths 61 Alignment based on perceived reform weaknesses 64 Research-based elements of successful whole-system reform 65 Summary 67 Research Question 4 68 Likelihood of sustainability beyond the superintendent’s tenure 70 Summary 70 Discussion and Chapter Summary 71 Chapter 5: Conclusions 76 Background and Purpose of This Study 76 Summary of Findings 78 Reflections on the Findings 79 Mixed-Methods Research Design and Research Question 1 79 Elements of Sustainability and Research Questions 2 and 3 80 Optimism Regarding Sustainability and Research Question 4 80 Limitations of the Study 81 Implications for Practice 84 Future Research 87 Chapter Summary 89 References 91 Appendices Appendix A: Emailed Invitation to Participate 97 Appendix B: Online Survey 98 Appendix C: Qualitative Interview Protocol 105 vii LIST OF TABLES Table 1: Demographic Information: Superintendents by District Type and Size 45 Table 2: Comparison of Distribution of Superintendents by School District Type: Current Study Survey and California 45 Table 3: Demographic Information: Superintendent by District Type and Gender 46 Table 4: Demographic Information: Superintendent Age 47 Table 5: Demographic Information: Superintendent Tenure in Current Position 48 Table 6: Demographic Information: Superintendent Tenure in Prior District(s) 50 Table 7: Demographic Information: Qualitative Interviews 51 Table 8: Correlation Between Superintendent Tenure and Sustainability of Initiatives to Improve Student Achievement 52 Table 9: Analysis of Variance: Superintendent Age and Sustainability Rating 53 Table 10: Analysis of Variance: District Type and Sustainability Rating 54 Table 11: Independent t Test: Gender and Sustainability Rating 54 Table 12: Category of Superintendent-Led Reform to Improve Student Achievement 56 Table 13: Alignment of Sustainability Strengths and Weaknesses to Elements of Sustainability 62 viii LIST OF FIGURES Figure 1: Organization of the literature review 10 Figure 2: Reform strengths as perceived by superintendents 57 Figure 3: Reform weaknesses as perceived by superintendents 59 Figure 4: Mean ratings for Fullan’s nine essential elements of whole-system reform 67 Figure 5: Reform sustainability beyond superintendent tenure 71 ix ABSTRACT The purpose of this research was threefold. First, the study explored the possible relationship between the tenure of public school district superintendents and the sustain- ability of their reform efforts to improve student achievement. Second, the study com- pared superintendents’ perceptions of factors supporting or impeding sustainability of their reforms to elements identified in research. Third, the study sought to determine the extent, if any, to which superintendents considered sustainability when initiating reforms. A mixed-methods design was employed to combine theoretical and technical components of qualitative and quantitative analysis. Quantitative data from 51 southern California superintendents were collected via an online survey. Four southern California superin- tendents and 1 northern California superintendent were interviewed to obtain qualitative data. Analysis of the data indicated that, while the superintendents perceived a strong correlation between their tenure and reform sustainability, they noted that this correlation diminishes over time as reforms gain traction through broader support and acceptance. With regard to current reforms within their school districts, the superintendents recog- nized some but not all of the eight research-based elements deemed necessary for long- term sustainability. Finally, the data indicated that the superintendents considered sustainability when initiating reforms and were highly optimistic about their reforms lasting the test of time despite research and experience that might indicate otherwise. Implications for practice included efforts to minimize position turnover through differen- tiated strategies for recruitment and selection of superintendents, targeted professional x development around the nature of reform sustainability, and possible linkages among lifespan development, reform initiation and durability over time. 1 CHAPTER 1 OVERVIEW OF THE STUDY While people choose careers for a variety of reasons, it should come as no great surprise that many become educators to make a difference—a profound and sustainable difference—in the lives of students. In fact, educators often view their chosen profession as a “vocation,” saying that they sought it as a means of providing value to society, moral purpose, helping children, and watching them grow (Best, 1948; Fullan, 2010; Tamir, 2009). Other data suggest that educators overwhelmingly rate “reaching students” as their greatest source of professional satisfaction (Cohn & Kottkamp, 1993). For example, more than 90% of the students entering a California teacher training program said that they chose teaching because it “helps change the world and further social justice” (Tamir, 2009, p. 531). As educators with the potential for system-wide impact, school district superin- tendents cite similar reasons for selecting their career paths. In fact, a comprehensive study of the superintendency found that 34.6% of America’s superintendents had sought their current role based on a “desire to have a greater impact on student achievement” (Glass & Franceschini, 2007, p. 45). Another 9.5% reported becoming superintendents because of a “personal commitment to American public education” (p. 46), while 23.7% expressed a general desire to be in leadership. Overall, 90% of the nation’s 13,500 superintendents cited altruistic reasons for entering the profession (Glass & Franceschini, 2007). Expressing a desire to make a difference in the lives of students is not the same as doing it. As Fullan (2005) pointed out, making changes happen is a far cry from 2 sustaining them over time. “The problem of sustainability is the ultimate adaptive chal- lenge” (p. 14). Sustainability of innovation and school improvement remain the most pressing issues in educational reform (Coburn, 2003; Datnow, 2005). Some have even argued that sustainability is not possible, given the powerful forces that naturally push against it (Fink, 2000). Successful school reform is a slow process that can take from 5 to 10 years, often longer (Carter & Cunningham, 1997; Desimone, 2002). Meanwhile, research on the longevity of school superintendents throughout the United States indicates that the average district leader remains in the for 5 to 6 years (Byrd, Drews, & Johnson, 2006; Glass & Franceschini, 2007). For the nation’s 65 largest school districts, which serve more than 14% of America’s children, the average superintendent tenure in 2010 was 3.64 years (Casserly, Lewis, Uzzell, Horwitz, & Simon, 2010; Pascopella, 2011). A range of complex factors are believed to contribute to the rate of superintendent turnover, including what Fuller et al. (2003) referred to as “the iron triangle: a coalition of boards, unions and the central office” (p. 24). Irrespective of the causes behind leader- ship turnover in public school districts, the district itself, as an organization, lives on. “Some leaders may be quite gifted in solving problems personally, but if they fail to institutionalize the process, their departure leaves the system crippled. They must create or strengthen systems that will survive them” (Gardner, J. W., 1990, p. 11). Therein lies the essential question behind this study. Given that school leaders come and go, often before their ideas, initiatives, and reforms have become incorporated into the culture of the school district, to what extent are these leaders’ initiatives to improve student achievement sustainable over time? In other words, what long-term 3 impact do relatively short-term leaders have on an organization and what factors or conditions contribute to whether or not a given leader’s legacy is sustained? While ele- ments of sustainability have been conceptualized in the literature (Fullan, 2005; Gladwell, 2000), this study seeks practical evidence and definition of how sustainability is achieved, or not, in the real-world context of superintendent-led reforms to improve student achievement. Statement of the Problem Research on systems change in education, comprehensive school reform (CSR), and sustainability is abundant in the literature. So too is research on leadership, leader- ship styles, and their application to the public education milieu. Less abundant in the lit- erature, but not uncommon, is research on superintendents: their demographics and challenges, strategies for success in the position, political pressures, and turnover (Björk & Kowalski, 2005; Carter & Cunningham, 1997; Spillane & Regnier, 1997). Very little is documented relative to superintendent-led reforms to improve student achievement and the degree to which, if at all, their efforts are sustainable over time. The ability to produce change is described as a distinguishing characteristic between leaders and managers (Bolman & Deal, 1994; Northouse, 2010). Superin- tendents as leaders are expected to make change happen, particularly with regard to improving student achievement. According to Peterson (2010), superintendents stand at “ground zero” or “the epicenter of intense educational change” (p. 139) and must respond accordingly. As a result, superintendents construct entry plans, write annual goals, engage in strategic planning, and work closely with a range of constituent groups to generate 4 support for their school improvement initiatives (Björk & Kowalski, 2005). While some initiatives may be designed to have a limited shelf life with regard to sustainability, other reforms, such as those aimed at improving student achievement, are designed to be long term. Unfortunately, little is known about what happens to these initiatives once the superintendent leaves and is no longer leading the charge. Purpose of the Study This study involved three major foci. The primary purpose of the study was to identify the relationship between superintendent tenure and sustainability of the superin- tendent’s reform efforts to improve student achievement. Second, the study compared superintendents’ perceptions about factors supporting or impeding sustainability relative to sustainability characteristics identified in the literature. Third, the study was designed to determine the extent to which superintendents consider sustainability when they initi- ate reforms to improve student achievement. Research Questions This study addressed four research questions: 1. What is the relationship between superintendent tenure and sustainability of their reform efforts to improve student achievement? 2. What factors do superintendents believe will impede or promote sustainability of their reforms beyond their tenure? 3. How do the factors identified in Question 2 align with the characteristics of sustainability, as identified in research? 4. To what extent do superintendents consider sustainability when they initiate a particular reform to improve student achievement? 5 Significance of the Study The findings of this study may be helpful to current and aspiring superintendents with respect to setting goals for what they want to achieve on behalf of the students and families whom they serve. Knowing about the sustainability of various reforms within the context of superintendent tenure could help superintendents to prioritize entry plans and annual goals, providing clarity and bringing focus to their work. Also, because superin- tendents are standing at center stage with regard to justifying allocation of resources toward their initiatives and away from other potential uses, sustainability may provide strong rationale for stakeholder groups to support one particular direction over another. Finally, superintendents as leaders must deal with a constant and sometimes conflicting stream of mandates, multistructural government regulations, and bureaucratic policies (Carter & Cunningham, 1997; Spillane & Regnier, 1997). These outside forces, many of which come in the form of initiatives, may or may not be sustainable over the long term (Björk & Kowalski, 2005). An understanding of the conditions that promote or under- mine sustainability can potentially assist superintendents in choosing the best course of action for addressing these external initiatives. Stewardship of a school district’s budget is ultimately the responsibility of the board of trustees. Now more than ever, it is critical that expenditures be targeted toward initiatives having the greatest potential for lasting success, while simultaneously avoiding those that will become expensive failures. As a result, boards of education may benefit from acquiring knowledge about the characteristics of sustainable reforms. Such knowledge could give board members an additional lens for critically evaluating any ini- tiative that is presented to them for their consideration or approval. The obvious advantage to having a sense of how to judge sustainability is that boards may more 6 effectively prioritize the initiatives presented to them, resulting in greater efficiency and prudence in the utilization of limited and valuable resources. Senior district administrators, principals, and department heads may also benefit from understanding the qualities and characteristics of sustainable reforms. In addition to working with the superintendent to support and promote his or her reforms, these educa- tional leaders often promote initiatives of their own (Conley & Cooper, 2011). Teachers and other staff, meanwhile, frequently complain about the constantly revolving door with regard to reforms, characterizing them as “fads” that will eventually go away as quickly as they came (DuFour, Eaker, & DuFour, 2005). By understanding the factors that con- tribute to sustainability and avoiding those that weaken it, educational leaders can poten- tially minimize the roller coaster of change and the resulting perception on the part of staff that “this too shall pass” (Carpenter, 2000; Gallagher & Bailey, 2000; McNeil, 2000). This study may inspire educational researchers to explore further the issue of sustainability in an attempt to discover and develop tools to predict and measure it. Such tools could be invaluable on several policy and practice levels. Assumptions Ideally, a study about sustainability of superintendent-led initiatives would span many years so initiatives could be followed and observed in real time. Instead, assump- tions must be made about the reliability and validity of sustainability predictors culled from the literature. Also, a significant portion of the data from this study is derived from superintendents themselves, whose responses were assumed to be truthful and accurate. 7 Assumptions were made about the degree to which initiatives identified by superinten- dents actually improve student success. Limitations of the Study The major limitation of this study is that it is not longitudinal in design. If it were, the study would rely less on assumed predictors of sustainability and more on observation and measurement of those predictors, and perhaps others, in a real-time setting. Another limitation of the study is the sampling of superintendents whose initiatives have been analyzed. Due to the fact that this study was confined to urban superintendents in south- ern California who agreed to participate in the research, caution must be exercised in extrapolating the findings to superintendent-led initiatives beyond the scope of the sample group. Another potential limitation of this study is the lack of research-based metrics to measure or predict sustainability. As no such tool currently exists, the predic- tors of sustainability identified in literature are untested. Delimitations of the Study Much of the data from this study came from urban superintendents in southern California who willingly participated in this research. Although superintendents through- out the United States face similar challenges in their positions (Glass & Franceschini, 2007), there was no attempt in this study to substantiate external validity with regard to the generalization of responses. A broader spectrum of superintendent-led initiatives could be analyzed using the same methodology. In turn, such future analysis could expand the research by examining, comparing, and contrasting the sustainability of initi- atives within multiple subgroups of superintendents. 8 Definitions of Terms For the purpose of this study, the following terms are defined as stated below. Community: A group of people or businesses sharing the same locality. Initiative: Plan, process, procedure, or program intended to change the status quo and bring about improvement. For the purpose of this study, initiative is used inter- changeably with reform. An initiative may or may not be an original concept or idea rel- ative to the person or persons who propose it. Innovation: Similar in meaning to initiative and reform but an original concept or idea relative to the person or persons who propose it. Reform: Plan, process, procedure, or program intended to change the status quo and bring about improvement. For the purpose of this study, reform is used interchange- ably with initiative. A reform may or may not be an original concept or idea relative to the person or persons who propose it. Reform activities: Activities that alter existing procedures, rules, and requirements to enable the organization to adapt the manner in which it functions to new circum- stances. School district: Local public educational agency that operates schools and serves students in any variation along the K–12 spectrum. An elementary school district gener- ally serves students in kindergarten through Grade 5 or 6. A high school district generally serves students in Grades 6 or 9 through 12. Unified school districts serve students in kindergarten through Grade 12. Student achievement: Measurable increase in student academic performance as documented by a standardized assessment tool. 9 Student success: An improvement or enhancement in any condition involving students. This term is intentionally broad and open ended, allowing some latitude for superintendents to select a particular initiative or reform that they feel will positively benefit students. Sustainability: A reform or initiative that remains in effect and continues to provide value at least 5 years beyond the tenure of the superintendent who initiated it. Superintendent: A K–12 educational administrator hired by a board of trustees to provide overall leadership to a public school district. Urban school district: An elementary, high school, or unified district serving students in a city or suburban setting. Organization of the Dissertation This dissertation is organized in five chapters. Chapter 1 presents an overview of the role of the urban superintendent as an agent of change and the importance of sustain- ability of superintendent-led reforms over time. Chapter 1 also presents the statement of the problem, the purpose of the study, research questions, significance of the study, limi- tations and delimitations of the study, and definitions of terms. Chapter 2 is a literature review of educational change, sustainability, the superintendency, and the role of the superintendent in facilitating reforms that are sustainable beyond his or her tenure. Chapter 3 describes the methodology, population, and data collection and analysis pro- cedures for the study. Chapter 4 presents the findings of the study, including collected data and descriptive and statistical analysis of the data. Chapter 5 summarizes the find- ings and their implications for current and aspiring superintendents, school boards, and others, and presents recommendations for further research. 10 CHAPTER 2 REVIEW OF THE LITERATURE The purpose of this study was threefold. First, the study was designed to deter- mine whether a relationship exists between superintendent tenure and sustainability of superintendent-led reform efforts to improve student achievement. The second purpose of the study was to identify key factors that superintendents believed could impede or promote sustainability of their reforms and to determine how these factors aligned with the characteristics of sustainability identified in the research. Third, the study was designed to determine whether superintendents considered sustainability when they initi- ated a particular reform to improve student achievement. The path to accomplishing these outcomes begins with an in-depth overview of research in two broad areas: educational reform and leadership. Figure 1 provides an overall perspective of the literature review. Figure 1. Organization of the literature review. 11 The literature on educational reform was examined in general, with particular attention to CSR and, most important, sustainability. With regard to leadership, this liter- ature review followed a narrow focus to explore research on the role of the superintendent as an agent of change. The body of research relating to superintendent longevity was also examined. The chapter concludes with a summary that identifies spe- cific controversies, questions, or problems in the research literature that validated the need for this study. Educational Reform In order to gain perspective relative to the sustainability of superintendent-led reforms, this review examines the nature of educational reform. Over the decades, much has been written about these reform efforts and many studies have been conducted, with their findings frequently compiled into numerous meta-analyses. For example, Hattie (2012) compiled approximately 800 meta-analyses incorporating 52,637 studies, to determine the impact or effect size of various programs, policies, and innovations on the academic achievement of students. One of Hattie’s conclusions was that “almost every- thing works” (p. 15), with about 95% of all things done in education having some posi- tive influence on achievement. While this section of the study was not intended to explore the effect of educational reforms on student achievement per se, the literature is reviewed with a lens focused on the element of sustainability. School reform has likely been extant since the first American public school opened its doors in 1635 (Boston Latin School Association, 2010; Coughlin, 2009; Cubberley, 1919). In the nearly four centuries since, the U.S. system of public education has grown from a relatively minor institution to one that dominates the social, political, 12 and economic landscape (Carnoy & Levin, 1985). In Tinkering Toward Utopia: A Cen- tury of Public School Reform Tyack and Cuban (1995) noted that reform of the public schools “has long been a favorite way of improving not just education, but society” (p. 1). School reform is often viewed as the means by which conditions are created for people to believe in and pursue the ideology of the American dream (Hochschild & Scovronick, 2003). The modern-day era of educational reform began in the late 1950s as a result of Russia’s successful launch of the world’s first satellite, Sputnik (Hochschild & Scovronick, 2003). Several distinct stages of reform or “waves” (Desimone, 2002, p. 433) have occurred since then, in which “one way of thinking has more or less domi- nated reform strategies” (Hochschild & Scovronick, 2003, p. 87). Hoyle’s Three Periods of School Reform Hoyle (2005) described three distinct periods of educational reform between 1982 and 2003, as documented by a plethora of research and reports examining the purpose, condition, and performance of public schools. First period: 1982 to 1986. Beginning with the release of A Nation at Risk by D. P. Gardner (1983) and members of the National Commission on Excellence in Education, this period of educational reform began primarily with a focus on increased accountabil- ity and the need to stay competitive in an increasingly global economy (Tyack & Cuban, 1995). During this time, state legislatures expanded regulatory controls over school dis- tricts, including within the classroom, calling for improved student performance on standardized tests, assessment of school-level performance and progress, increased graduation requirements, a longer school day and year, and tightening teacher licensing 13 requirements (Hoyle, 2005). Hochschild and Scovronick (2003) referred to this period as a time when the focus was “fixing the people” (p. 88). Reform efforts from this period were later criticized for not adding capacity to the system and for relying too heavily on top-down approaches (Desimone, 2002). Second period: 1986 to 1989. During this period the importance of improving student performance and accountability continued, along with a new emphasis on meeting the needs of all children, including those from diverse backgrounds relative to race and ethnicity, socioeconomics, culture, ability level, and learning style. School bureaucracies were identified as major contributors to low academic performance and high failure rates, resulting in a call for decentralization and stakeholder participation in school governance and decision making. The notion of “fixing the schools” (Hochschild & Scovronick, 2003, p. 88) dominated in an attempt to improve the workplace by attracting, keeping, and motivating teachers to create a school culture conducive to learning and a coherent approach within each school to provide effective instruction (Hochschild & Scovronick, 2003). According to Desimone (2002), reforms from this period did not have much impact on changing the way in which schools are organized or the way in which teachers teach. Third period: 1989 to 2003. Drawing on findings from research on the needs of children, optimum learning conditions, and other pedagogical studies, researchers and policymakers during this period criticized previous reform efforts for being too narrow, focusing on structural and professional issues rather than the well-being and learning of all children. As a result, reforms during the third period centered on ensuring children’s well-being and leaving no child academically behind. Reforms were aimed at defining 14 the way in which effective schools should be organized, governed, and led, how they should interact with community-based resources, and how the roles of school leaders should be reconfigured (Björk, 1996; Hoyle, 2005). Attention turned to systems as a whole, with a call for state- and district-level policymakers to set individual and col- lective goals, develop standards consistent with the goals, and develop extensive assess- ment systems to monitor mastery and measure accountability (Hochschild & Scovronick, 2003). In response to the limited success of prior change efforts, this third period of reform brought a renewed focus on the importance of restructuring schools to “foster changes in teaching and learning” (Desimone, 2002, p. 434). Broadly, the major reform efforts of this period became known as “comprehensive school reforms.” Comprehensive School Reform (CSR) An outgrowth of the effective schools movement, CSRs were designed to broadly address a school’s shared goals, climate, school-level management, strong district and principal leadership, articulated curriculum, maximized learning time, staff development, and parent involvement. The central focus of CSR models was to influence classroom instruction by placing greater emphasis on teaching and learning (Desimone, 2002). According to Hattie, “Such comprehensive reform appeals to many superintendents and school officials as a systematic answer to the issue of improving teaching” (Hattie, 2012, p. 214). The U.S. Department of Education (USDE; 2002) defined CSR on the basis of 11 elements, centered on measurable goals, utilization of research-based methods, integra- tion of and support for instruction, assessment, classroom management, professional 15 development, parent involvement, and school leadership. The USDE stressed the importance of identifying available federal, state, local, and private financial resources to coordinate services to support and sustain the school reform effort. The USDE indicated that CSR models should meet at least one of the following requirements: Either the program has been found, through scientifically based research, to sig- nificantly improve the academic achievement of participating students; or strong evidence has shown that the program will significantly improve the academic achievement of participating children. (p. 3, item 11) Borman, Hewes, Overman, and Brown (2003) conducted a meta-analysis of all known research on the most widely implemented externally developed CSR programs, synthesizing research on the specific effects of the 29 most widely implemented models. While their study showed the effects of CSRs on academic achievement to be highly variable, the overall effects were found to be statistically significant and “greater than the effects of other instruments that have been designed to serve similar purposes and student and school populations” (Borman et al., 2003, p. 164). According to Borman et al. (2003), the effect of CSRs on student achievement fluctuated in the early years but increased significantly after 5 years of implementation (effect size d = .25). At 7 years of implementation, the effect size increased to d = .39, while implementations lasting 8 to 14 showed an effect size of d = .50). Types of School Reform Sims (2010) identified two major types or categories of school reform. The first type of school reform involves school subject matter and teaching methods. Under this category of reform, what and how students are taught in the classroom are affected. Examples of these types of reforms include the use of computers in the classroom, concept-based mathematics programs, direct instruction, and back-to-basics 16 methodology. Many professional organizations, such as the National Council of Teachers of Mathematics, are involved in this type of reform, which can influence progress at the local and state levels. The second type or category of reform involves school administration and gov- ernance procedures, affecting how schools and districts are managed and the roles played by teachers and administrators (Sims, 2010). Examples of this type of reform include site-based management and community involvement programs. Section Summary The literature on educational reform is abundant, showing an evolution of theory and practice over time. While the debate over reform effectiveness continues, with at least one researcher claiming that all of them work to some degree (Hattie, 2012), there seems to be consensus within the research that reform takes time. To that end, sustain- ability of reform within the context of what teachers often view as “this too shall pass” (McNeil, 2000) is particularly relevant to the purpose of this study. Sustainability of School Reform Efforts “As the struggle to achieve large-scale reform evolves, sustainability is becoming a rallying concept, one that contains the elaboration of strategies essential for whole- system capacity building on an ongoing basis” (Fullan, 2005, p. 13). In his analysis Fullan identified eight essential components of sustainability, emphasizing elements such as leadership, commitment, perseverance, capacity building, accountability, learning, and moral purpose. Overall, Fullan concluded that “sustainability is very much linked to con- tinuity of deepening direction over time” (p. 31). Proposing a trilevel solution involving the school/community, district and state, Fullan claimed that sustainability of reforms is 17 possible if all levels commit to a continuous, systems thinking approach focused on “a small number of interconnected priorities” (p. 87). From 1973 through 1978, the RAND Corporation conducted a national study of four federally funded programs intended to introduce and support innovative practices in the public schools (McLaughlin, 1990). The study involved a sample of 293 local projects in 18 states and concluded that the “net return to the general investment was the adoption of many innovations, the successful implementation of a few, and the long-term continuation of still fewer” (McLaughlin, 1990, p. 12). Factors contributing to successful efforts included active commitment of district leadership, timely support for teachers as they implemented changes, and locally selected implementation strategies. The study concluded that “policy cannot dictate what matters” (McLaughlin, 1990, p. 12). Snipes, Doolittle, and Herlihy (2002) examined three large urban school districts that had raised academic performance while reducing the achievement gap for minority students. The research involved comparisons of the subject districts with others that had not yet seen similar improvements. Results indicated that political and organizational stability over a prolonged period and consensus on educational reform strategies were key ingredients for meaningful change. The research also indicated that district leaders invested substantial time, effort, and resources in changing district culture and creating systemwide consensus for reform. Using the policy attribute theory developed by Porter (1994), Desimone (2002) conducted a meta-analysis of studies investigating implementation of CSRs. Desimone’s analysis concluded that each of the five policy attributes described by Porter—specificity, consistency, authority, power, and stability—influenced CSRs in meaningful ways. 18 Specificity was found to influence implementation fidelity, while power was related to obtaining immediate effects from reform efforts. Authority, consistency, and stability were found to be the driving forces behind long-lasting change, which Desimone (2002) noted can take 5 to 10 years to institutionalize. This analysis also concluded that site- based management is an essential component of successful school reform, a finding not supported by Marzano and Waters (2009) or the meta meta-analysis conducted by Hattie (2012), which concluded that there was almost no evidence to support that such “organic design features” (p. 85) had a positive effect on student achievement. Challenges to Sustainability Between 1987 and 1997, approximately 361 school reform efforts were chroni- cled in the pages of Phi Delta Kappan (Carpenter, 2000). Carpenter concluded that the results of these reforms were “less than impressive” (p. 384), citing Gallop Poll statistics showing that the public’s attitude toward the public schools declined from 28% rating public education as “A” or “B” in 1987 to 22% in 1997. Approximately 13% of the public graded public schools with a “D” or “F” in 1987, with the percentage increasing to 22% in 1997. Carpenter proposed rationale to support his conclusion that the research underlying most reforms is flawed and that reformers are often motivated more by the need to publish than creating meaningful change. In his longitudinal study of Lord Byron High School in Ontario Canada, Fink (2000) found evidence to support the notion that reform has its own “life cycle” and that innovations eventually reach a point of attrition. Fink examined the school through six lenses: context, meaning, leadership, structure, culture, and teachers’ lives and work. Fink identified factors within each lens that he claimed contributes to the demise of 19 school reforms and suggested that “the hopes and dreams of the initiators of lighthouse schools will prove disappointing in the long run” (Fink, 2000, p. 153). Forty-four publications on efforts to scale up external reforms were analyzed by Coburn (2003), who found that most studies focused primarily on schools in their first few years of implementing a new external reform, thus failing to capture sustainability. Coburn also found that schools that were successful in implementing reforms had diffi- culty in sustaining them in the face of “competing priorities, changing demands and teacher and administrator turnover” (p. 6). Additional factors that Coburn found to undermine sustainability included a short-term influx of resources, professional develop- ment, and other forms of support that tended to diminish over time. Shift in ownership is critical to schools’ and districts’ abilities to sustain and spread reform over time. Depth, sustainability, spread, and ownership broadened the definition of scale as a reform strategy. Hargreaves and Fink (2000) discussed the challenges of sustainability in the context of “scaling up”: transplanting a successful reform from one location, such as a school, district or state, to others. They cited three essential ingredients to successful edu- cational reform: depth, length or sustainability, and breadth. Citing their work with two school systems over a period of time, Hargreaves and Fink concluded that sustainability is not simply an issue of how long something lasts; more important, sustainability “addresses how particular initiatives can be developed without compromising the devel- opment of others in the surrounding environment” (p. 5). In the authors’ view, sustainability extends far beyond individual schools to include districts, states, and nations. 20 Giles and Hargreaves (2006) studied three innovative schools over a 4-year period and identified three factors that contributed significantly to a weak record of sustainable school reforms. First, innovative reforms tend to be perceived by outsiders as peculiar to the organization in which they were implemented. In other words, innovative schools and districts must be unique in some way, have additional resources, and so forth. Second, innovations appear to have a clear life span that begins with creativity and experimenta- tion and ultimately ends with stagnation as changes eventually occur in the professional staff, students, district priorities, and policies. Third, sustainability is weakened by changes in the external context, such as reductions in resources, changing power relations between states and local school districts, and so forth. Giles and Hargreaves (2006) pos- ited that schools, as learning organizations and professional learning communities, have the capacity to offset the first two forces that threaten to undermine sustainability but concluded that the third force, in the form of a standardized reform agenda, is “actively undermining the efforts and successes of those few, truly creative knowledge society schools” (p. 152). A longitudinal qualitative study of six CSR models implemented within a single district revealed that 5 of 13 schools were still implementing reforms at moderate to high levels after 3 years. Changing district and state contexts vis-à-vis policies, leadership, and political agendas were cited as having the greatest impact on institutionalization, or not, of these reforms (Datnow, 2005). Implications from the Datnow study were that reform sustainability may be related to the degree to which these reforms help educators to meet new local district and state demands, or at least not to come into conflict with them. The second implication was that reforms demanding the most ongoing resources tended to be 21 the least sustainable over time. The third implication was that high-stakes accountability systems may inhibit sustainability of reforms because staff tends to place higher priority on meeting the demands of the accountability system than on utilizing the time and effort necessary to sustain a new reform. As Datnow (2005) noted, “Educators need to work hard toward the institutionalization of a reform . . . sustainability does not come easily; it takes extensive time and effort” (p. 148). According to Stone and Clements (1997), the lack of sustainability or success of many educational innovations is a “common impediment” (p. 90) due to over-reliance on a learner-centered vision of schooling. While the authors cited extensive research in support of the notion that learning requires time, attention, and action on the part of the student, they noted that many educational reforms have focused on the extrinsic aspects of shaping the instructional environment in an effort to energize and excite students about learning. In so doing, these reforms create a condition in which student effort in school is “optional” and “expected only if the individual feels so inclined” (Stone & Clements, 1997, p. 91). The authors labeled such reforms as fads and claimed that, as long as student learning is seen as the sole responsibility of teachers, such reforms cannot be sustainable over time. DuFour and Eaker (1998) identified five reasons for the lack of sustainability of many school reform efforts: complexity of the task, misplaced focus, lack of clarity on intended results, failure to appreciate and attend to the change process, and lack of per- severance. 22 Section Summary The literature indicates that, while certain common characteristics of sustainable reform efforts can be identified, sustainability itself is fragile, somewhat elusive, and per- haps subject to a “life cycle” of its own. Research has identified challenges to sustain- ability, not the least of which is educators’ seemingly insatiable quest for “silver bullet” solutions (Carpenter, 2000; Fink, 2000; Fullan, 2005). Yet one essential factor con- sistently surfaces relative to the degree to which any reform is or is not sustainable over time: leadership. With regard to superintendents, Spillane and Regnier (1997) noted that leadership involves “getting things done right and the right things done” (p. 13). Leadership As with the topic of educational reform, leadership is a broad and complex theme. For example, entering the word leadership in a February 26, 2012, search of Google Scholar yielded more than 2,260,000 entries. A similar search at Amazon.com yielded 72,851 books with the word leadership somewhere in the title. Leadership habits, person- ality attributes, styles, and case studies of inspirational leaders of lighthouse schools are common throughout the literature (Hattie, 2012), as are attempts to define and describe leadership (Bennis & Nanus, 2003; Bolman & Deal, 2008; Northouse, 2010). For the purposes of this review, leadership as it pertains to school superintendents is the main focus. In particular, the literature is reviewed to identify superintendent leadership quali- ties, characteristics, and attributes related to facilitating change. In his book Focus, Schmoker (2011) claimed that change could be driven using the tools of simplicity, clarity, and priority. Educators at all levels, especially leaders, must anchor their work in what is taught, how it is taught, and authentic literacy (Schmoker, 2011). Instead, leaders often avoid what is simple and direct. According to 23 Pfeffer and Sutton (2000), leaders are “irrationally enamored with novelty and complex- ity, which prevents them from focusing on and implementing their core priorities” (p. 33). As Schmoker (2011) stated, “We would rather innovate than follow up to ensure our priorities are implemented” (p. 17). Leading change also requires strength of purpose, the ability to grow, persevere and abandon the unattainable (Carter & Cunningham, 1997; Carver & Scheier, 2003; DuFour et al., 2005; Fullan, 2005; Marzano & Waters, 2009). Superintendents as Leaders The role of the public school superintendent has evolved over the decades (Brewer, 2011; Carter & Cunningham, 1997; Glass & Franceschini, 2007; Kowalski, McCord, Peterson, Young, & Ellerson, 2011; Spillane & Regnier, 1997) to a point where “America’s future is . . . linked to the quality of . . . the leadership of its superintendents” (Carter & Cunningham, 1997, p. 236). Reviewing the many contributions by William Boyd related to the issue of educational governance, Brewer (2011) claimed that school leaders must be political strategists whose leadership success “is dependent on the politi- cal context of their time and place” (p. 461). In examining the instructionally oriented skills, professional and personal behav- iors, and organizational relationships established by superintendents, Peterson and Barnett (2005) reviewed six studies conducted between 1990 and 2002 and identified several common themes. The findings emphasized that successful superintendents have instructional vision, clear communication skills, and the ability to work effectively with the individuals and groups who are responsible for teaching and learning. Peterson and Barnett also found that successful superintendents maintained high levels of visibility and 24 monitored and evaluated instructional and curricular program implementation. These findings are contrary to some conventional wisdom suggesting that superintendents are too consumed with administrative and management issues to focus on the fundamental importance of curriculum and instruction. In their comprehensive analysis of research on educational leadership, Leithwood and Riehl (2003) found that successful educational leaders set a clear direction, devel- oped the capacity of their staffs as well as their organizations, responded productively to challenges, and recognized opportunities so they could take full advantage of them. The American Association of School Administrators (AASA) commissioned comprehensive studies of the American school superintendent, including research by Glass and Franceschini (2007), who surveyed 1,338 superintendents throughout the United States between 1980 and 2006. Among their many findings, Glass and Franceschini analyzed the rationale used by school boards to hire their superintendents. In 1980, 11.6% of superintendents were hired for their ability to perform as instructional leaders; by 2006, the percentage had grown to 49.2%, underscoring the significance of superintendent leadership. In another AASA-sponsored study, Kowalski et al. (2011) found the emphasis on “effective communication” (p. 75), an essential component of leadership (Bolman & Deal, 2008; Northouse, 2010), to be the most significant role of the superintendent. Superintendent-Led Reforms Educational reform is complex, often messy, and slow to inculcate within a school district. Systemwide reform, according to Carter and Cunningham (1997), involves changing the “mental models that individuals in schools use to interact with one another” 25 (p. 178). Understanding how to change peoples’ mental models within the context of the human systems within which they work requires superintendents to “elicit sustained focus and effort in an enterprise notorious for climbing on every passing bandwagon” (Carter & Cunningham, 1997, p. 179). Carter and Cunningham noted that change may take as long as 8 years of doing the “right” things, meaning that sustainability requires superintendents to provide consistent long-term support and encouragement. Cuban (2001) asserted that typical “all-purpose” reform solutions were inap- plicable to urban schools, suggesting that tasks facing urban school leaders differ in mag- nitude and kind from those found in other school districts. According to Cuban, typical reforms fail to include the critical linkages between cities, their schools, and the nation’s economic and social well-being. Proposed solutions include learning from the experi- ences of successful businesses and making clear distinctions between setting standards and standardization in educational reform. In his meta-analysis of research on successful school leadership, Leithwood, Harris, and Hopkins (2008) proposed the following “claims”: 1. School leadership is second only to classroom teaching with regard to influ- encing student learning. 2. Leaders tend to rely on a similar repertoire of basic leadership practices. 3. Application of these basic leadership practices, not the practices themselves, respond to rather than drive the contexts in which they work. 4. School leaders improve teaching and learning indirectly through their influence on staff motivation, commitment, and working conditions. 26 5. School leadership has greater influence on schools and students when it is widely distributed. 6. Some patterns of leadership distribution are more effective than others. 7. Relatively few personal traits explain a high proportion of the variation in lead- ership effectiveness. Superintendent Tenure Although the terms tenure and turnover may seem interchangeable, an important conceptual difference exists. For example, according to the AASA-sponsored research by Glass and Franceschini, (2007), average superintendent tenure, or time in the position, was approximately six years. By contrast, the national turnover rate for superintendents in 2006 was 16.9%. Another example is the 2010 decennial study of 1,867 superinten- dents by Kowalski et al. (2011), who noted that 49% of the superintendents studied did not plan to remain in the role beyond 2015. This finding suggests the likelihood of sig- nificant turnover in the position during the next few years. The focus of this study is primarily on the relationship between reform sustain- ability and tenure, or the length of time on the job in a particular school district, not the frequency with which the incumbents change. In other words, while turnover may be a symptom driving tenure and an important consideration for other research, it should not be assigned the same meaning or significance within the context of this review. While possible causes of superintendent turnover are important and research has analyzed these issues in education and many other fields, turnover causes are outside the scope of this study. 27 When school superintendents or a top leader in any organization leaves, evidence suggests that the entire organization is affected (Brady & Helmich, 1984; Comte & Mihal, 1990; Drucker, 1990; Wernet & Austin, 1991). Citing research by Borman et al. (2003), Marzano and Waters (2009) noted that the average five- to seven-year tenure of superintendents may not be enough for their reforms to become institutionalized. “With- out consistent leadership from the superintendent, non-negotiable goals for achievement and instruction have little chance of success” (p. 114). With regard to superintendents leaving their positions before change has had time to institutionalize, Natkin et al. (2002) stated, “Superintendents, believing they will likely be in a position for a short time, may be reluctant to undertake major reforms, and instead focus their efforts on objectives that can be accomplished within two or three years” (p. 4). The authors reviewed the literature on superintendent turnover and found no support for prior claims that the superintendency had become a “revolving door.” In addition, Alborano et al. conducted their own retrospective study using data received from 892 superintendents throughout the United States, representing school districts ranging in size from 1,000 to 50,000 students. Of the superintendents who responded, 274 were still in office at the time of the survey. Utilizing a Cox proportional hazards regression model, the authors found that the median tenure of superintendents between 1990 and 1994 was 6.5 years, reflecting no significant increase since 1975. The authors also found no significant difference in turnover based on district size. In 2003 the Broad Foundation and Thomas B. Fordham Institute released their “manifesto” on improving leadership at the principal and superintendent levels (Broad Foundation & Thomas B. Fordham Institute, 2003). The purpose of this report was to 28 propose a different way to recruit, select, hire, and train school and district leaders, with an emphasis on qualifications over certification, expanded authority, and board support for holding subordinates accountable for results. The paper included a summary of state practices relative to certification of public school administrators and individual profiles of several education leaders. The report indicated that the medium term of service for superintendents nationally was about six years (Snipes et al., 2002), while urban super- intendents stayed in their positions for an average of 2.5 to 4.0 years (Broad Foundation & Thomas B. Fordham Institute, 2003). Casserly et al. (2010) and Pascopella (2011) found the average superintendent tenure in the nation’s 65 largest school districts to be 3.64 years, a figure substantially lower than the averages reported by Glass and Franceschini (2007) and others. In examining the reform agenda, Cuban (2001) noted a higher degree of expecta- tions facing district superintendents and dispelled common myths that large urban school districts are ungovernable, the superintendency is a revolving door, and schools alone can improve the chances of poor children. Cuban concluded with an agenda for action, high- lighting the importance of urban leaders in school reform and discussing the challenges ahead for those who believe in the civic and moral obligations that accompany the improvement of both cities and schools. According to Cuban, “The often-repeated claim that most urban superintendents serve a couple of years and depart, is, in a word, false” (p. 8). Natkin, Cooper, Alborano, et al. (2002) examined superintendent survival data in 292 districts from 1975 to 1999, concluding that superintendent tenure averaged 6 to 7 years and had not changed significantly since 1975 to 1979. This study refuted earlier 29 claims about superintendent longevity averaging 2 to 3 years (Johnston, 2000; McKay & Grady, 1994). Findings similar to those reported by Natkin, Cooper, Fusarelli, et al. (2002) regarding superintendent longevity were also reported by Byrd et al. (2006). Yee and Cuban (1996) analyzed superintendent tenure, finding the average term of school district leaders to be 5 to 6 years. Their research data indicated variability in superintendent tenure over the past 100 years and predicted that, although turnover rates might increase in the future, the impact of superintendent turnover on student achieve- ment was not significant. Another finding was that periods of increasing superintendent tenure were not “directly associated with successful reform; reform movements have occurred during periods of declining as well as increasing tenure” (p. 636). The Relationship Between Superintendent Tenure and Student Achievement It is a commonly held belief that superintendents influence educational outcomes, including student achievement, through their actions and behaviors as leaders (Yee & Cuban, 1996). A logical assumption is the longer a superintendent stays in his or her current position, the more he or she is able to “consolidate power, put into place their own programs and people, convince staff and the public of the worthiness of their vision and reorganize structure to support that vision” (Yee & Cuban, 1996, p. 633). Yet linking superintendents’ tenure with impact on student achievement through standard measures of effectiveness is problematic and, according to Yee and Cuban (1996), studies attempting to do so have been inconclusive. In fact, the authors claimed that research had yet to document a “predictive relationship specific administrator actions and organizational outcome” (p. 634). Their conclusion was supported in a doctoral study by Myers (2010), who concluded that the impact of superintendent tenure on student 30 achievement had the least relative impact of all of the variables that he studied. Accord- ing to Yee and Cuban (1996), no clear evidence supported the hypothesis that superinten- dents serving 5 or more years were any more successful than shorter-term superintendents with regarding introducing and sustaining reform-oriented programs. The conclusion by Yee and Cuban (1996) about a lack of evidence establishing a relationship between superintendent tenure and student achievement was not supported by Marzano and Waters (2009), who reviewed two studies documenting such a relation- ship. The weighted average correlation was .19, significant at the .05 level, indicating that superintendent longevity influenced student achievement. Marzano and Waters suggested that the effect may manifest itself in as early as 2 years. In discussing the notion of human strength, Carver and Scheier (2003) reflected on the “ability to transform the world, to turn visions arising from one’s imagination into reality” (p. 87). They noted that, as people age, the will to transform may diminish natu- rally as part of lifespan development—what they referred to as “giving up” (p. 89). According to Kowalski et al. (2011), the model American superintendent in 2010 was between the ages of 56 and 60 years. While younger leaders are more likely to take on challenges that enhance their growth and maximize the use of personal resources, older leaders may view goal selection as more loss based, substituting goals that are no longer attainable with those that are viewed as realistically within reach (Baltes, 1997; Marsiske, Lang, Baltes, & Baltes, 1995). Section Summary Like school reform, the essence of district leadership has transformed and advanced over the years. Superintendents, as central figures of district leadership, are 31 responsible for balancing and leveraging the complexities of their leadership roles on behalf of many stakeholders. While research on the superintendency has been conclusive regarding the importance of district leadership per se, evidence supporting a predictive link between successful superintendent leadership and student achievement is mixed. While the research on superintendent tenure seems conclusive that 5 to 6 is the national average tenure, relatively little is known about the relationship, if any, between superin- tendent tenure and the sustainability of the reforms that they lead in support of improving student achievement. Chapter Summary Several main points may be drawn from this review of the literature. First, school reforms take time to implement—at least 8 years to reach their greatest statistical effect (Borman et al., 2003). Second, superintendent tenure currently averages 5 to 7 years in the United States (Cuban, 2001; Glass & Franceschini, 2007; Kowalski, et al., 2011; Marzano & Waters, 2009; Natkin, Cooper, Alborano, et al., 2002; Natkin, Cooper, Fusarelli, et al., 2002; Snipes et al., 2002; Yee & Cuban, 1996). Third, leadership in gen- eral plays a key role in promoting student success (Carter & Cunningham, 1997; Leithwood et al., 2008). Fourth, the data with regard to the establishment of a predictive positive link between superintendent tenure and the sustainability of reform efforts to improve student achievement are mixed (Marzano & Waters, 2009; Myers, 2010; Yee & Cuban, 1996). Using the essential information from this review, the current study was designed to determine whether a connection existed between superintendent tenure and reform sustainability, identify factors that most influenced sustainability, and measure the degree 32 to which superintendents considered it a part of their work to improve student achieve- ment. 33 CHAPTER 3 METHODOLOGY This chapter describes the research design, sampling method, and data collection and analysis methods used to address the research questions: 1. What is the relationship between superintendent tenure and sustainability of their reform efforts to improve student achievement? 2. What factors do superintendents believe will impede or promote sustainability of their reforms beyond their tenure? 3. How do the factors identified in Question 2 align with the characteristics of sustainability, as identified in research? 4. To what extent do superintendents consider sustainability when they initiate a particular reform to improve student achievement? Research Design Methodology According to Teddlie and Tashakkori (2009), research questions are methodology dependent, meaning that they inspire study design. In social sciences, the three most commonly used research designs are quantitative, qualitative, and mixed methods (Bergman, 2008). Quantitative research is statistical in nature, theoretically driven and deductive, studying from the general to the specific. Statistical data analysis is typically utilized in quantitative studies, which can often be generalizable due to larger sampling than found in most qualitative research (Teddlie & Tashakkori, 2009). Quantitative research is descriptive and frequently based on a hypothesis (Bergman, 2008). 34 By contrast, qualitative studies are more narrative, inductive, and thematic in nature, using coding to interpret data and fostering the use of inquiry to develop a deeper understanding of an event or a phenomenon (Patton, 2002). Unlike quantitative research designs, qualitative studies start with specifics and progress outward toward the general. Sampling is typically small, resulting in limited generalizability (Teddlie & Tashakkori, 2009), and challenges defining cause and effect (Bergman, 2008). Mixed-methods designs combine theoretical and technical elements of quantita- tive and qualitative approaches into “one of the fastest-growing areas in research meth- odology” (Bergman, 2008, p. 1). Mixed-methods designs can be both inductive and deductive, employing one or more of the following five models: triangulation, concurrent embedded, explanatory, exploratory, and sequential embedded (Bergman 2008). The appropriate design depends on the research questions, the data types, and the implemen- tation timeline (Bergman, 2008; Teddlie & Tashakkori, 2009). According to Bergman (2008), mixed-methods research can provide richer, more in-depth analysis and is accepted by the broader research community. In the case of the current study, a mixed-methods approach was utilized, employ- ing triangulation (Jick, 1979) through sequential data collection and analysis (Creswell, Plano Clark, Gutmann, & Hanson, 2003). The rationale in support of this approach is based on the nature of the research questions. The initial approach involved collection of data through the use of a survey instrument developed specifically for this study. Data were analyzed from descriptive, relational, and predictive perspectives, relative to a spe- cific hypothesis. Qualitative data were then gathered from a subgroup of respondents, through a face-to-face, standardized, open-ended interview process (Patton, 2002) 35 designed to provide in-depth analysis. In short, by collecting and analyzing quantitative data first and qualitative data second, this study followed an implicit design path, testing variables using a large sample before drilling down for in-depth exploration (Creswell et al., 2003). As a result of this sequential or two-phased approach to the study design, quantitative and qualitative analyses were completed separately and then integrated into a final discussion. Because quantitative findings from this research directed the subsequent qualitative analysis, what Creswell et al. (2003) referred to as a “dominant-less dominant model” (p. 219), the initial assumption was that quantitative aspects would take priority over qualitative aspects. Sample and Population The purpose of this study was to examine the sustainability of superintendent-led reforms to improve student achievement. Research on the topic of school reform indi- cates that, although superintendents can have an impact on student achievement in as little as 2 years (Marzano & Waters, 2009), reform acculturation into a school or district takes much longer; at least 8 years (Borman et al., 2003). Meanwhile, research on super- intendent longevity within a school district indicates that the average superintendent tenure is 5 to 7 years (Glass & Franceschini, 2007; Kowalski et al., 2011; Yee & Cuban, 1996). Ideally, then, a study of the relationship between reform sustainability and super- intendent tenure would involve a longitudinal analysis with data collected from a variety of cohorts, including working and retired superintendents, their school boards, senior cabinet, management team, teachers, students, parents, and perhaps others. However, as mentioned in Chapter 1, limitations of this study included the time and resources necessary to conduct such a practical comprehensive analysis over time. As 36 a result, the scope of this research, both quantitative and qualitative, was confined to a population consisting of current California public school superintendents, or what Bracht and Glass (1968) referred to as the “experimentally accessible vs. target population” (p. 440). The sample population targeted for initial receipt of the survey instrument included all working superintendents throughout the state. It was not expected that all recipients of the survey would complete and return it, so the population of superinten- dents used with regard to data analysis was a subset of the original target population. Upon collection and analysis of the data from this phase of the study, a much smaller sample of 6 to 10 superintendents was selected for the second (qualitative) phase of the research. One sampling issue identified above—the expected limited return of initial sur- veys—could impact overall findings if the participating superintendents’ responses were not reflective of the overall population. Also, while it was assumed that all respondents answered survey questions truthfully, there was no system of checks and balances incor- porated within the study to verify this assumption. Perhaps equally important, there was no system incorporated within the study to test the validity of responses outside the opinions of the superintendents. In other words, even if superintendents truthfully answered survey and interview questions about their reform efforts, sustainability, and any relationship to longevity, the validity of their opinions within the context of the real world remained untested. Another potential sampling issue was the degree to which superintendent respondents uniformly understood and interpreted the survey and interview questions. 37 Absent uniform understanding, responses could vary, and such discrepancies could be incorrectly interpreted during data analysis. The sample population selected for the qualitative phase of this study may or may not have provided responses that were reflective of the population of superintendents who responded to the initial survey, much less all superintendents throughout California. While the results of the qualitative phase were expected to help in triangulating results from the quantitative phase, no provisions were taken within the study to substantiate triangulation or any lack thereof. Instrumentation This study was conducted in two phases. Thus, two instruments were used: one for the quantitative phase (Phase 1) and one for the qualitative phase (Phase 2). Quantitative Phase According to Bracht and Glass (1968), the intent of all experiments is to “gener- alize their findings to some group of subjects and set of conditions that are not included in the experiment” (pp. 437-438). To that end, the generalizability or external validity of research is a significant and worthy goal. Although a standardized measurement instru- ment with confirmed, research-based reliability and validity would have been ideal for this phase of the study, a review of the Mental Measurement Yearbook and other resources did not yield such an instrument. As a result, a survey was developed to collect data relative to the research questions. Tuckman (1999) noted that, in the development of questionnaires, researchers must exercise caution and consider the extent to which the questions might influence respondents to portray themselves in a positive manner and anticipate what the researcher 38 wants to hear or learn, and the extent to which questions ask for information that the respondent does not know. “The validity of questionnaire and interview items is limited by all three of these considerations” (p. 237). In view of these considerations, an online survey was constructed using a Web- based development tool available through www.surveymonkey.com. The survey began with a set of questions designed to capture descriptive information such as district type, student enrollment, superintendent age and gender, length of time in the current position, and total length of time serving as a superintendent. The instrument then asked a series of questions about reforms and sustainability, based on a 5-point Likert-type scale. The survey concluded with a series of questions about whether the respondent would be available for more in-depth personal interview as a follow-up to the first phase. Qualitative Phase The intent in this phase of the study was to use quantitative data from Phase 1 to select what Patton (2002) described as “information-rich cases for study in depth” (p. 230). Such purposeful sampling “yields insights and in-depth understanding rather than empirical generalizations” (p. 230). Of the 15 strategies that Patton (2002) outlined for purposeful sampling in a qual- itative study, “snowball or chain sampling” (Patton, 2002, p. 237) fit appropriately with the research design of this study. Using data from the quantitative phase in consultation with retired superintendents who are “well-situated people” (p. 237) in the field, five superintendents were selected for in-depth, face-to-face interviews. The selected super- intendents were participants in Phase 1 of the study and gave permission, through the qualitative survey instrument, to be interviewed in Phase 2. The goal in selecting specific 39 superintendents for interview was to gain insight into superintendents’ perceptions about sustainability of their reforms to improve student achievement beyond their tenure in their current districts. By using chain sampling to identify the superintendents to be inter- viewed, it was possible to gain access to district leaders who provided deeper under- standing of the data obtained in Phase 1. Whether the results of Phase 2 supported or contradicted those obtained in Phase 1, the summative findings of the quantitative and qualitative components of this study produced the most comprehensive addresses to the research questions possible, given the previously identified constraints and limitations of this research. Data Collection As described in the previous section, data collection was conducted in two distinct phases. Quantitative Phase In the quantitative phase of the study (Phase 1), data were collected via an online survey. Using a statewide database of email addresses, an introductory email was sent to 246 southern California public school superintendents in Ventura, Los Angeles, River- side, San Bernardino, Orange, and San Diego counties. The email explained the purpose of the study and solicited participation utilizing a link to the online survey. The survey instrument was designed to be completed in less than 10 minutes, using a variety of closed-end response options such as drop-down menus, radio buttons, and 5-point Likert-type scales. Other than the optional section at the end of the survey that invited respondents to enter contact information to indicate their willingness to 40 participate in Phase 2, no specifically identifiable information about respondents and their school districts was collected. Qualitative Phase In Phase 2 of the study, data were collected using the standardized, open-ended interview method (Patton, 2002). The purpose of using this method of data collection was to assemble responses that were as reliable as possible, utilizing a set of questions “care- fully worded and arranged with the intention of taking each respondent through the same sequence and asking each respondent the same questions with essentially the same words” (Patton, 2002, p. 342). This method intentionally limited flexibility with regard to follow-up or probing questions, ensuring as much consistency across interviews as possi- ble. This method of data gathering also allowed for maximum time efficiency with regard to each interview—an important element, given the busy schedules and limited time availability of school district superintendents. Data were gathered in Phase 2 via a tape recorder, hand-written notes, and coding on a process-outcomes matrix (Patton, 2002). Data Analysis In Phase 1 of the study, all data from the on-line instrument were electronically downloaded into SPSS version 20 (formerly known as Statistical Package for the Social Sciences; SPSS Inc.). Data were analyzed descriptively in an effort to organize and define the characteristics of the responses (Salkind, 2008). Superintendents’ ratings of the relationship between longevity and reform sustainability were examined relative to age, gender, district type and enrollment using several one-way analyses of variance (ANOVA) and an independent-samples t test. 41 Data in Phase 2 of the study were examined using an inductive, content-analysis approach to identify patterns, themes, and categories (Patton, 2002). Capturing superin- tendents’ ideas, thoughts, and experiences relative to reform sustainability incorporated what Patton (2002) referred to as a “phenomenological analysis” that seeks to “grasp and elucidate the meaning, structure and essence of the lived experience of a phenomenon” (p. 482). Specifically, using the process outlined by Tuckman (1999, p. 417), qualitative data were organized as follows: 1. Category labels were developed based on response patterns. 2. Sufficient data were assigned to each category to uniquely identify the cate- gory. 3. Abstract definitions were assigned to the identified category. 4. Definitions were applied to guide data collection and theoretical reflection. 5. Additional categories, as necessary, were identified as data were assigned. Such additional categories included opposites, categories with greater or lesser specificity, and so on. 6. Categories were examined to identify relationships and build hypotheses. 7. The conditions under which the categorical relationships occurred were deter- mined. 8. Connections were developed between categorized data and findings from Phase 1 of the study. Patton (2002) asserted that researchers using a mixed-methods analysis to exam- ine the same phenomenon should “not expect the findings generated by these different methods will automatically come together to produce some nicely integrated whole” (p. 42 557). Referring to the integration of quantitative and qualitative data as a form of “com- parative analysis” (p. 557), Patton advised identifying areas of convergence and diver- gence between the two methodologies in an effort to determine how each contributes to the overall understanding of the research questions. By reconciling quantitative and qual- itative data in this manner, a balanced result was obtained. Chapter Summary The sustainability of school reform efforts is dependent on a range of variables, including leadership on the part of the district superintendent. Yet when superintendents leave their positions prior to the time required for their reforms to become sustainable (a minimum of 8 years), the full-scale impact of their reforms may be jeopardized. This study examined the sustainability of superintendent-led reforms to improve student achievement in an effort to discover whether a relationship exists between superintendent tenure and the long-term viability of their change efforts. 43 CHAPTER 4 THE FINDINGS The purpose of this study was threefold. First, the study was designed to ascertain whether a relationship existed between superintendent tenure and the sustainability of their reform efforts to improve student achievement. Second, the study was an attempt to identify key factors that superintendents believed could impede or promote sustainability and how these factors aligned with the characteristics of sustainability as identified in research literature. Third, the study was designed to determine whether superintendents considered sustainability when they initiated a particular reform to improve student achievement. Four research questions guided the study: 1. What is the relationship between superintendent tenure and sustainability of their reform efforts to improve student achievement? 2. What factors do superintendents believe will impede or promote sustainability of their reforms beyond their tenure? 3. How do the factors identified in Question 2 align with the characteristics of sustainability, as identified in research? 4. To what extent do superintendents consider sustainability when they initiate a particular reform to improve student achievement? In Chapter 2, an in-depth overview of research literature was presented in two broad areas: educational reform and leadership. The literature on educational reform was examined with particular emphasis on CSR and sustainability. With regard to leadership, the literature review focused on the role of the superintendent as an agent of change. The body of research relating to superintendent longevity was examined, along with a 44 summary of specific controversies, questions, or problems in the research literature that validated the need for this study. Chapter 3 described the research methodology used in the study, with particular emphasis on the two-phase, mixed-methods approach incorporating both quantitative and qualitative analysis. This chapter presents the findings of the study in relation to each of the four research questions, beginning with an overview of the demographic information obtained in the quantitative and qualitative phases. Quantitative Phase: Demographics of Study Respondents On April 13, 2012, a personalized email was sent to the current superintendents of 246 southern California public school districts in Los Angeles, Orange, Riverside, San Bernardino, San Diego, Santa Barbara, and Ventura counties. The email message invited each superintendent to participate in a brief online survey via www.surveymonkey.com and provided a direct link to the survey instrument. Appendix A contains a copy of the email message and Appendix B contains a copy of the survey instrument. Of the 246 email invitations sent, 51 surveys were completed, representing a response rate of 20.73%. District Type and Size Table 1 summarizes demographic information about superintendent respondents to the quantitative survey relative to district type and size. The data indicated that 94.12% of the respondents were either from unified school districts (52.94%) or elementary dis- tricts (41.18%). Statewide, 91.67% of all school districts are either elementary (56.35%) or unified (35.31%). See Table 2. 45 Table 1 Demographic Information: Superintendents by District Type and Size Enrollment Elementary High school Unified Total % < 2,500 4 0 0 4 7.84 2,501-5,000 5 1 4 10 19.61 5,001-7,500 4 0 4 8 15.69 7,501-10,000 2 1 7 10 19.61 10,001-15,000 2 1 1 4 7.84 15,001-20,000 3 0 2 5 9.80 20,001-25,000 1 0 5 6 11.76 < 25,000 0 0 4 4 7.84 Table 2 Comparison of Distribution of Superintendents by School District Type: Current Study Survey and California Elementary High school Unified Source n % n % n % Total Survey 21 41.18 3 5.88 27 52.94 51 California 541 56.35 80 8.33 339 35.31 960 46 With regard to district size, 54.90% of all respondents listed their district enroll- ment between 2,501 and 10,000 students. Superintendents serving in districts with over 25,000 students represented 7.84% of the sample, with all of those districts serving a K– 12 population. Superintendents in districts with 2,500 or fewer students represented 7.84% of the sample, with all of those districts serving an elementary population. In the national study by Kowalski et al. (2011), nearly 70% of superintendents worked in dis- tricts with 3,000 or fewer students and 3.2% served in districts with 25,000 or more students. Superintendent Gender Table 3 summarizes survey response data with regard to district type and gender. Table 3 Demographic Information: Superintendent by District Type and Gender Elementary High school Unified Gender n % n % n % Total % Male 17 80.95 3 100.00 19 70.37 39 76.47 Female 4 19.05 0 0.00 8 29.63 12 23.53 In their 2010 decennial study of the American school superintendent, Kowalski et al. (2011) noted that the percentage of female superintendents had increased over the past decade from 13.2% in 2000 to 24.1% in 2010. Although the total number of respondents in this survey (51) was significantly smaller than the 1,838 responses obtained in the national survey conducted by Kowalski et al. (2011), the percentage of female 47 respondents was only slightly smaller (23.53%). When gender was disaggregated by district type, 48.72% of the male superintendents and two thirds of the female superin- tendents reported working in unified districts. Superintendent Age Table 4 summarizes demographic information about superintendent respondents relative to their age at the time the survey was taken. Table 4 Demographic Information: Superintendent Age Age (years) Elementary High school Unified Total % < 40 0 0 0 0 0.00 41–45 0 0 1 1 1.96 46–50 5 0 3 8 15.69 51–55 3 0 7 10 19.61 56–60 10 3 4 17 33.33 61–65 3 0 8 11 21.57 66–70 0 0 4 4 7.84 > 70 0 0 0 0 0.00 Total 21 3 27 51 100.00 Percentage 41.18 5.88 52.94 48 Similar to findings reported by Kowalski et al. (2011), the highest percentage of superintendent respondents in this study were in the age group 56–60. While 14.6% of respondents in the Kowalski et al. study were younger than 46 years old, only 1.96% were in that age bracket in the current study. Kowalski et al. reported 18.1% of respond- ents being over 60 years old; the current study had 29.41% in that age group. Overall, 54.90% of the respondents in this study were 56 to 65 years old and 90.2% were 46 to 65 years old; none in this study was under 40 years old or over 70 years old. Tenure in Current Position Table 5 reports the tenure of all superintendent respondents in their current school district assignments, as well as whether they were new to the superintendency. Table 5 Demographic Information: Superintendent Tenure in Current Position Measure Elementary High school Unified Total Mean 7.18 4.44 3.73 5.19 Minimum 0.67 2.83 0.33 0.33 Maximum 27.83 6.83 11.83 27.83 Median 5.83 3.67 2.83 3.83 SD 6.05 2.11 2.94 4.71 First superintendency? Yes 17 2 13 32 No 4 1 14 19 49 A majority of respondents (62.75%) indicated that this was their first assignment as a superintendent. Mean tenure in the current position was 5.19 years for all respond- ents, with a mean tenure of 3.73 years for unified districts and 7.18 years for elementary districts. Maximum tenure for this sample was one elementary district superintendent with 27.83 years, with the next longest tenure being 14.83 years. Minimum tenure in the current position was a unified district superintendent with 0.33 years, with the next short- est tenure being .67 years. Overall, seven superintendents (13.73%) had served in their current position for less than 1 year. Tenure in Prior Position(s) Of those who reported serving as superintendent before their current assignment, 10.53% had four prior superintendencies, 10.53% had three prior superintendencies, and 26.32% had two prior superintendencies. Mean tenure of superintendents serving in other districts was 5.31 years for the immediate prior assignment, 4.78 years in the second prior assignment, 4.17 years in the third prior assignment, and 10.0 years in the fourth prior assignment. Data regarding the respondents’ tenure in prior superintendent assignments are summarized in Table 6. Among all respondents, the mean number of years serving as superintendent was 8.63, with 21.56% having served as superintendent for 13 or more years. Kowalski et al. (2011) reported that 24.8% of superintendents surveyed in their study had served in the position for 13 or more years. Summary of Quantitative Demographic Data This section reviewed the demographic data obtained from a quantitative survey of 246 public school superintendents in southern California. In total, 51 superintendents 50 Table 6 Demographic Information: Superintendent Tenure in Prior District(s) Measure District 1 District 2 District 3 District 4 Count 19.00 9.00 4.00 2.00 Mean 5.31 4.78 4.17 10.00 Minimum 1.50 1.50 1.50 5.00 Maximum 11.67 10.00 6.00 15.00 Median 4.38 5.00 4.58 10.00 SD 2.65 3.06 1.96 7.07 completed the survey. The data from the survey indicated that the typical respondent was male (76.47%), between 56 and 65 years old (54.90%, currently working in a unified school district (52.94%) with current district enrollment of 2,500 to 10,000 students (54.90%), currently serving as superintendent for the first time (62.75%), and in the current assignment for 3.73 years. Overall, the surveyed superintendents had been in the top district leadership position for an average of 8.63 years. Among the 19 respondents who stated that they had previously served as superintendent in other districts, the mean tenure in the immediate past district was 5.31 years. Qualitative Phase Five currently employed California superintendents (herein designated Superin- tendent 1 through Superintendent 5) were selected for face-to-face interviews in the qual- itative phase of the study. Each superintendent was interviewed using a set of prepared 51 questions employing the standardized open-ended interview method (Patton, 2002). Appendix C contains a copy of the qualitative interview protocol. Superintendents’ responses were captured in written notes, digitally recorded on an Apple IPhone IV using the internal recording application, or both. Digital recordings and interview notes were transcribed to text for the purpose of in-depth analysis. Demographic information about the five interviewed superintendents is summa- rized in Table 7. Table 7 Demographic Information: Qualitative Interviews Superintendent, District District Years in Years as interview date Gender Age type enrollment current job superintendent 1 6/21/2012 M 59 Unified 55,571 5 12 2 7/2/2012 F 62 Unified 14,665 1 4 3 7/12/2012 M 48 Unified 8,437 6 6 4 7/31/2012 F 48 Unified 5,970 4 10 5 8/9/2012 M 56 Unified 9,870 6 17 Research Question 1 Research Question 1 asked, What is the relationship between superintendent tenure and sustainability of their reform efforts to improve student achievement? This research question was addressed by Question 15 of the survey instrument. Using a 52 5-point Likert-type rating scale (1 = no correlation to 5 = high correlation), the data shown in Table 8 were obtained. Table 8 Correlation Between Superintendent Tenure and Sustainability of Initiatives to Improve Student Achievement Rating Elementary High school Unified Total % 1 (no correlation) 0 0 0 0 0.00 2 1 0 1 2 4.00 3 2 0 5 7 14.00 4 5 1 10 16 32.00 5 (high correlation) 12 2 11 25 50.00 Total 20 3 27 50 100.00 Although one elementary superintendent did not respond to this question and thus lowered the overall sample size to 50, the data indicated that 82.0% of the respondents rated the relationship between superintendent tenure and sustainability of initiatives to improve student achievement at either 4 or 5. Half of the respondents rated the correla- tion as high. To determine whether a rating variance existed between groupings of superin- tendents, several one-way ANOVAs were run using the raw data summarized in Table 8 53 as the dependent variable and school enrollment, age, and district type as the independent variables. In the case of school district enrollment, the assumption of homogeneity of vari- ance was violated, as assessed by Levine’s Test of Homogeneity of Variance (p = .001). There was no statistically significant difference in sustainability ratings among superin- tendents according to size of district, Welch’s F(7, 14.763) = .846, p = .568. With regard to superintendent age and district type, there was homogeneity of variance, as assessed by Levine’s Test of Homogeneity of Variance (p = .553 and .653, respectively). There was no statistically significant difference in sustainability ratings among superintendents according to age or district type, F(5, 44) = .202, p = .960 and F(2, 47) = .813, p = .450, respectively. Tables 9 and 10 summarize these data. Table 9 Analysis of Variance: Superintendent Age and Sustainability Rating Sum of squares df Mean square F p Between groups 0.810 5 0.162 .202 .960 Within groups 35.270 44 0.801 Total 36.080 49 An independent-samples t test was run to measure differences in sustainability ratings between male and female superintendents. There was one outlier in the data, as assessed by inspection of a boxplot, but the single outlier was not believed to affect the 54 Table 10 Analysis of Variance: District Type and Sustainability Rating Sum of squares df Mean square F p Between groups 1.206 2 0.603 .813 .450 Within groups 34.874 47 0.742 Total 36.080 49 t test result materially. Sustainability ratings for each level of gender were not normally distributed, as assessed by Shapiro-Wilk’s test (p < .05) and there was homogeneity of variance, as assessed by Levene’s Test for Equality of Variance (p = .646). Although sustainability ratings were higher for female superintendents (M = 4.33, SD = 0.78) than for male superintendents (M = 4.26, SD = 0.89), there was no statistically significant dif- ference in sustainability ratings by gender, t(48) = -.245, p = .808 (Table 11). Table 11 Independent t Test: Gender and Sustainability Rating Levene’s Test t test for equality of means for equality of variances Equal variances F p t df p Difference Difference Lower Upper Assumed .214 .646 -.245 48 .808 -.070 .287 -.647 .507 Not assumed -.263 20.9 .795 -.070 .267 -.626 .486 55 These analyses led to the conclusion that superintendents’ ratings of the correla- tion between tenure and sustainability were consistent, irrespective of district size, district type, superintendent age, or superintendent gender. Qualitative data from superintendent interviewees were generally consistent with survey findings. However, the respondents indicated a belief that the strength of the cor- relation should diminish over time; in other words, the longer a superintendent is on the job, the less important his or her presence should be with regard to sustainability of a particular reform. For example, Superintendent 1 stated, “The longer I’m in the position and the greater the buy-in for a particular reform, the less important I become in terms of keeping that reform alive.” Superintendent 1 continued, “Changing culture needs to not be dependent on a single individual.” Superintendent 3 stated, “Having been here longer, I think that helps, I really do. . . . I think if I was here 3 or 4 years, it would be a little dif- ferent . . . but I think we’re already on that path, whether I’m here or not.” In response to a question about her thoughts regarding a superintendent’s role in sustainability of change, Superintendent 4 stated, “I think the superintendent cannot be the end-all-be-all change agent. . . . I think I can see my job primarily as mentoring other people to create change that is lasting.” Survey and interview results indicated that respondents highly correlated super- intendent tenure and the sustainability of their reforms to improve student achievement. On a Likert-type scale (1 = no correlation to 5 = high correlation), 82% rated the rela- tionship as either 4 or 5. This finding was consistent, irrespective of district type, district size, superintendent age, or superintendent gender. However, data from face-to-face 56 interviews indicated that, while the superintendents reported a strong correlation between tenure and sustainability, they indicated that the strength of the correlation should dimin- ish over time as reforms take hold. Research Question 2 Research Question 2 asked, What factors do superintendents believe will impede or promote sustainability of their reforms beyond their tenure? Based on the two major categories of school reform defined by Sims (2010), the survey asked superintendents to reflect on one reform that they had initiated in their current position. Table 12 illustrates how survey respondents categorized their chosen reform. The data indicate that 74.5% of the respondents were thinking of reforms directly related to classrooms and instruction. Table 12 Category of Superintendent-Led Reform to Improve Student Achievement (N = 51) Category n % Classroom/instructional 38 74.5 Administrative/governance 9 17.6 Other 4 7.8 Each survey respondent was then given the opportunity to share his or her per- spectives regarding the strengths and weaknesses of the chosen reform relative to sustain- ability over time. 57 Reform strengths. In thinking about perceived strengths of their reforms, 86.27% of the interviewed superintendents provided an opinion about what factors could posi- tively influence sustainability. Although responses varied, seven common themes sur- faced, as indicated in Figure 2. Approximately 59% of the respondents gave answers reflecting the opinion that a culture of success exists to some degree within their districts. Specifically, respondents reported having successful professional learning communities, collaboration and buy-in from all stakeholders, ongoing support from their boards, and shared leadership at multiple levels throughout their organizations. Approximately 12% of the respondents indicated that strong and ongoing professional development was the greatest strength of their reform effort, and 12% indicated that changes in personnel, such as adding instructional coaches or hiring the right people, were the greatest assets to reform sustainability. Figure 2. Reform strengths as perceived by superintendents. 58 The qualitative interviews yielded evidence in support of at least some of the key strengths identified from the quantitative data. For example, in developing a culture of success, Superintendent 4 spoke of the importance of reforms that “are less conceptual and more institutional,” stating that “the idea is to change the practice at the grass roots level and to show people that this [change] is effective for them.” Superintendent 1 spoke more broadly, referring to modern educators as “freedom fighters” who must “re-ignite why they got into education in the first place.” He said, “We are not running a school district, we are running a movement.” The interviewees also spoke of the importance of involving people in making changes that last. For example, Superintendent 3 stated, “Having people engaged in the process” is a major strength. Superintendent 2 agreed, emphasizing that her work on a strategic plan would be successful over time only if “everyone has been involved in the process.” Collins (2001) stressed the importance of recruitment and selection of the right people in building successful organizations. The interviewed superintendents also identi- fied this element as a key strength of sustainable reform. Superintendent 2 succinctly stated that sustainability “is about the people,” while Superintendent 3 noted, “Ninety percent of what we’ve done here is a ‘we’ thing, not an ‘I’ thing.” In terms of recruiting the best for his team and the future of the district, Superintendent 1 spoke about sending his executive cabinet to conferences, “not so much for the content as the opportunity to recruit and find talented people.” This superintendent continued, “It is our job to bring in people who are younger and smarter than us, to take our places when we are gone.” 59 Reform weaknesses. The superintendents were asked to reflect on issues that could challenge reform sustainability in some way. As with the survey question regarding strengths, responses to this question were open ended and varied. However, patterns were evident in the responses, displaying common characteristics around one of eight general themes, as identified in Figure 3. In reflecting on a single weakness that could limit sustainability of their reform effort, 32% of the respondents identified budgetary factors as a primary issue. Approxi- mately 30% of the respondents indicated that staff, board, or superintendent turnover would pose the greatest threat to their reform. Lack of focus, fear of change, and finding the time necessary to make reforms last were identified as potential weaknesses in 27% of the responses. Figure 3. Reform weaknesses as perceived by superintendents. 60 Responses in qualitative interviews echoed the notion that turnover among key staff and the board could have a negative impact on sustainability. Superintendent 1 noted, “The biggest problem regarding turnover is that school boards tend to always want the latest and greatest flavor of the month instead of looking within their own districts to find, nurture and promote talent.” Superintendent 4 noted that sustainability of her current reform efforts were dependent to a large extent on specific individuals. She feared that, if one or more of those people were to leave, her initiatives would not be in place long enough to become part of the district’s culture. While the quantitative data suggested that limited fiscal resources were viewed as a significant barrier to sustainability, the interviewed superintendents did not totally agree. For example, Superintendent 4 noted that, while teacher layoffs due to budgetary problems have an impact on human capital, reform sustainability is rooted in factors that run much deeper than the budget. Superintendent 3 agreed, emphasizing that, if people buy into a reform, they will go to extreme lengths to sustain it, regardless of what happens with the district budget. Summary. Overall, quantitative and qualitative data indicated that the participat- ing superintendents viewed the existing culture of success in their districts to be their greatest ally in sustaining their reforms to improve student achievement. On the other hand, data from the survey indicated that lack of budgetary resources and turnover were the greatest potential barriers to reform sustainability. Research Question 3 Research Question 3 asked, How do the factors identified in Question 2 align with the characteristics of sustainability, as identified in research? The essential purpose of 61 this question was to determine the extent to which superintendents utilized research- based strategies to enhance sustainability of their reform efforts to improve student achievement. The process for addressing this research question involved analysis of data from the quantitative survey and the qualitative interviews relative to research-based characteristics of sustainability and successful whole system reform. Research-based elements of sustainability. Fullan (2005, p. 14) identified eight elements of sustainability: (a) public service with a moral purpose, (b) commitment to changing context at all levels, (c) lateral capacity building through networks, (d) intelli- gent accountability and vertical relationships, (e) deep learning, (f) cyclical energizing, (g) dual commitment to short-term and long-term goals, and (h) the long lever of leader- ship. The quantitative survey contained two open-ended questions in which superinten- dents were asked to identify strengths and weaknesses of the reform of their choice. Responses to the questions were categorized in an effort to find alignment with any or all of the sustainability elements identified by Fullan (2005). Overall, superintendents’ open-ended responses regarding perceived sustainability strengths and weaknesses of their reforms aligned with some of the research-based char- acteristics of sustainability noted by Fullan (2005). These alignments are summarized in Table 13. Alignment based on perceived reform strengths. With regard to reform strengths, 21 of 43 responses (48.84%) aligned to some extent with Fullan’s (2005) notion of intelligent accountability and vertical relationships. These responses included statements such as, “Distributed leadership and a strong focus on collaboration and inter- est based problem solving” and “PLC focus at grade level in schools, across district 62 Table 13 Alignment of Sustainability Strengths and Weaknesses to Elements of Sustainability Strengths and weaknesses n % Strengths (n = 43) Intelligent accountability /vertical relationships 21 41.84 Deep learning 8 18.60 The long lever of leadership 2 4.65 Weaknesses (n = 41) Cyclical energizing 14 34.15 The long lever of leadership 10 24.39 Commitment to changing context at all levels 18 19.51 by grade, and principal PLCs all with focus on differentiated instruction, ELD and ALD.” In other words, these comments tended to focus on building capacity and accountability throughout the system. Five responses (18.60%) aligned to some degree with the fifth element of sustain- ability: deep learning (Fullan, 2005). As Fullan noted, “Sustainability by our definition requires continuous improvement, adaptation and collective problem solving in the face of complex challenges that keep arising” (p. 22). Responses aligning with this element tended to focus on professional development and included statements such as, “Extensive staff development. We have gained the support of parents and teachers in the process,” and “Provided district wide staff development for all site administrators and teachers.” 63 While staff development per se is one piece of Fullan’s notion of deep learning, it never- theless reflected some degree of alignment with this research-based element. Two responses (4.65%) aligned with what Fullan (2005) referred to as “the long lever of leadership” (p. 27). Responses aligning with this element included, “Hiring people from outside to break the closed culture we have experienced should increase the likelihood of sustainability,” and “Yes, we are building the capacity of our principals to be instructional leaders. Changing our evaluation tools for both certificated and admin- istration to reflect the changes we would like to see, and are developing coaches in the teaching ranks.” The interviews with the superintendent also revealed alignment to several of the characteristics of sustainability. Superintendent 1 spoke in depth about public service with a moral purpose and a commitment to changing context at all levels. For example, in reflecting on how he had confronted achievement gaps between high- and low- performing student groups, Superintendent 1 said, “I framed it as a Civil Rights issue and most people here never thought of it that way.” He said, “I asked people flat out if they were OK with the fact that our special needs students were outperforming our African American students on every single measure on the state’s tests.” Superintendent 1 reported spending a great deal of time in meeting with people throughout his district, engaging in difficult conversations on topics such as equity. “I asked people, ‘What does equity look like to you and how do we know when we’ve achieved it?’” Superintendent 5 also spoke about several of the sustainability characteristics in describing his approach to reform and sustainability. Number one is that we have to really do a clear job with our teachers and our administrators as to the why of things. Why are we moving with this new 64 program? Why are we moving with a different strategies and instruction? Why does brain compatible instruction make sense? Regardless of what it is, whether it’s in the instruction or the curriculum, we need to do a far better job of helping folks understand why as opposed to just that we’re going to do it and then do the training. By doing that, I think we build sustainability away from just the superin- tendent. That second piece that is so important is the fact that everyone else within the organization takes that as part of their own. When that superintendent does walk away for whatever reason, whether that is being asked to walk or they walk on their own and retire, that is part of the fabric, part of the culture of the organization. Alignment based on perceived reform weaknesses. In terms of reform weak- nesses, three patterns emerged from the survey data relative to alignment to Fullan’s (2005) eight elements of sustainability. First, 14 of 41 responses (34.14%) aligned to Fullan’s (2005) seventh sustainabil- ity element, “cyclical energizing” (p. 25). A primary focus for a number of these responses was the negative impact of inadequate school funding on sustainability, for example, “It will take significant supervision to build critical mass and current fiscal environments is making that difficult” and “The inability to pay for extended reflection time or training days.” The lack of fiscal resources and the resulting need to do more with less can wear people down and cause them to burn out (Fullan, 2005). Thus, budgetary problems may adversely affect the cyclical energizing of a district and ultimately neu- tralize this key sustainability element. Ten responses (24.39%) aligned with Fullan’s (2005) eighth element of sustain- ability, “the long lever of leadership” (p. 27). Responses tended to center on leadership turnover and included statements such as, “I believe that new superintendents bring in new initiatives; the focus is gone,” and “Elected board members require constant effort to follow the governance protocols and trust the systems. Elected officials change and require team building and education on effective systems.” Instability in key leadership 65 positions can hamper sustainability because “addressing an adaptive challenge requires complex learning in politically contentious situations where there are many inertial factors pulling us back to the status quo” (Fullan, 2005, p. 46). Frequent turnover results in continuous restarting of the learning process as each new person comes on board, thus negatively affecting this element of sustainability. Eight responses (19.51%) aligned with Fullan’s (2005) second element of sustain- ability, “commitment to changing context at all levels” (p. 15). Responses included statements such as, “People wanting to go back to what is easy for them” and “It is harder to teach this way than using other previously used strategies, so people may choose ease over quality.” These statements reflect a reluctance to commit to a changing context and therefore undermine this element of sustainability. The superintendents’ interview responses relative to sustainability barriers also revealed alignment to the characteristics of sustainability found in the literature. For example, Superintendent 5 spoke of barriers to deep learning, commitments to changing context, cyclical energizing, and capacity building when he reflected on the complacency that sometimes occurs after a degree of success has been realized, You may think about getting complacent, without actually using that word, while we’re at 79 to 80 percent, that’s incredible. That’s 20, 30 percent higher than a national average. Flip it by saying, “Okay, that 20 percent that we’re still not reaching, what about them?” If your child is one of that 20 percent, it’d be highly important. Research-based elements of successful whole-system reform. In addition to identifying the characteristics of sustainability, Fullan (2010) proposed nine elements for successful, whole system reform: A small number of ambitious goals, 66 A guiding coalition at the top, High standards and expectations, Collective capacity building with a focus on instruction, Individual capacity building linked to instruction, Mobilizing the data as a strategy for improvement, Intervention in a non-punitive manner, Being vigilant about “distractors,” and Being transparent, relentless, and increasingly challenging (Fullan, 2010, p. 21). According to Fullan, the presence of these elements within a school system indicates that the system has developed a capacity to “mobilize and engage large numbers of people who are individually and collectively committed and effective at getting results relative to core outcomes that society values” (p. 21). Question 11 of the quantitative survey asked respondents to agree or disagree with respect to each element being present in their district. A 5-point Likert-type scale was employed (1 = disagreement about the element being present to 5 = agreement that the element was present). One superintendent did respond to this question, reducing the over- all response size to 51. Figure 4 shows the mean superintendent rating for each of Fullan’s (2010) nine elements of successful whole-system reform. All elements received mean ratings between 3.84 and 4.65, indicating that respondents reported that the nine elements of successful reform were present in their districts to some degree. Being vigilant about eliminating or minimizing distractors and building individual capacity aligned to instruction received the lowest mean ratings (3.84 and 4.12, respectively). Promoting effective reform throughout the district in a 67 Figure 4. Mean ratings for Fullan’s nine essential elements of whole-system reform (N = 51). nonpunitive manner and building collective capacity with a focus on instruction received the highest mean ratings (4.65 and 4.37, respectively). The weighted average of total responses for all elements was 4.25. Summary. Multiple distinct alignments were found relative to comparison between sustainability strengths and weaknesses identified in Research Question 2 and Fullan’s (2005) eight elements of sustainability. Survey data indicated that respondents self-assessed their districts as possessing all nine elements of successful whole-system reform, at a level of 4.25 out of 5. These findings were supported by qualitative data from 68 the superintendent interviews. Collectively, the data indicated that, whether the superin- tendents instinctively or deliberately employed a research-based approach in building sustainable reforms, their actions aligned with many of the characteristics of sustainabil- ity and successful whole-system reform. What was not revealed by the data was the degree to which each superintendent individually incorporated these elements into his or her work. Further analysis on a district-by-district basis would be required to document the depth of alignment between actual practice and what is known from the research on sustainability and whole-system reform. Research Question 4 Research Question 4 asked, To what extent do superintendents consider sustain- ability when they initiate a particular reform to improve student achievement? In discussing one of the six key characteristics of effective leaders, Gardner (1990) noted, “They think longer-term, beyond the day’s crisis, beyond the quarterly report, beyond the horizon” (p. 4). To determine whether superintendents think beyond the horizon to consider sustainability when initiating a particular reform, Research Ques- tion 4 was posed directly in qualitative interviews with each of the five interviewees. Based on the interview data, these superintendents considered sustainability with regard to their reform efforts. In fact, Superintendent 2 questioned, “Why would you do anything if you didn’t think it was good practice and you want to move an organization closer to that practice?” When asked whether she considered sustainability when initiat- ing a reform, Superintendent 4 said, “absolutely.” She continued, I always think about [sustainability]. . . . I’m not sure if I think about it as a sepa- rate process, but I know if there isn’t an anchor or a structure, it probably won’t last, so how are we going to create that? So a lot of the work that I do is structure. 69 Superintendent 1 spoke of his structural work with his Board of Trustees to create a succession plan. He noted that the idea of a succession plan was at the core of his thinking in terms of promoting sustainability of the reforms already in place throughout the district. “I knew we were working with the Board on a succession plan. That’s the only way we can sustain our efforts.” Superintendent 1 also reiterated the importance of his conscious effort to move people to a point where “it would be harder to move backward than to go forward.” In describing his strategy for helping people to reach that tipping point, Superintendent 1 reflected on a scene from a classic book (Twain, 1978), saying that a superintendent must be “like Tom Sawyer. When you are painting the fence, you have to make it fun. Other people will start begging you to be allowed to paint the fence.” Superintendent 3 agreed, adding that working hard to make the “message matter” is a key to sustainability. He elaborated regarding one particular reform effort to increase the number of students applying for acceptance to 4-year universities: “Whether I’m here or not, I don’t think people can argue against that message.” In an effort to illustrate his beliefs about the importance of considering sustain- ability when initiating reform, Superintendent 5 drew on his 11-year tenure as superin- tendent in his previous district. He noted that much had been accomplished and that many reforms had been moving ahead when he left his post to join his current district. Now, 6 years later, a significant number of those reforms are no longer in place. Superintendent 5 spoke of feeling saddened that the reforms that he and others had worked hard to achieve were no longer in place. He noted that it was “hurtful to see that some of it is lost and some of the heart of it is lost. Then they’re probably doing far better in other areas but the 70 essence of what made it a magical place to be is not there.” Superintendent 5 said that he thinks frequently about what may have caused this situation to occur and reflects openly on how to prevent it from repeating in his current district. He posed a question to himself: “Okay, some things do stick, some things don’t stick and why do some things and others not?” In reflecting on his question, he stated, “I don’t know . . . one of my goals for myself is to better understand why.” Likelihood of sustainability beyond the superintendent’s tenure. On the quantitative survey participants were asked to imagine that they were to leave their current position on or about January 1, 2013, and to rate the probability of their reform sustaining beyond 8 years from the time they had initiated it. Participants were asked to rate this probability of sustainability on a Likert-type scale of 1 = highly unlikely to 5 = highly likely. One superintendent did not respond to this question, resulting in a sample size of 50. The results are shown in Figure 5. According to these data, 68% of the respondents rated the sustainability of their reform beyond their tenure at either 4 or 5, while 8% rated their reform sustainability at 1 or 2. Whether or not these superintendents had considered sustainability before initiating their reform is not known from the data, but a majority of the respondents reflected an optimistic view about sustainability of their reforms, given their hypothetically imminent departure. Summary. Quantitative survey data suggest that the superintendents believed that, if they were to leave their current posts on or about January 1, 2013, their reforms would still be in place more than 8 years after they had been initiated. Based on the face- to-face interviews with five superintendents, it was also determined that sustainability 71 Figure 5. Reform sustainability beyond superintendent tenure (N = 50). had been considered when the reforms were being contemplated. The manner in which sustainability is considered varied from superintendent to superintendent, but they were unanimous in stating that no reform would be worth undertaking without considering its long-term viability. Discussion and Chapter Summary This chapter began with a presentation of demographic data relative to 51 super- intendents who responded to the quantitative survey and 5 superintendents who partici- pated in face-to-face qualitative interviews with the researcher. Demographic data 72 included district type and student enrollment, in addition to superintendent age, gender, and current and past tenure as superintendent. The survey response sample contained greater representation from unified school districts (52.94%) than found statewide (35.31%) but the cumulative percentage of ele- mentary and unified districts in the survey was similar to the state’s cumulative percent- age (94.12% and 91.67%, respectively). With regard to gender, the percentage of female superintendents responding to the survey was 23.53%, slightly lower than the 24.1% found in a national study by Kowalski et al. (2011). Also similar to the national study by Kowalski et al. (2011), the largest per- centage of superintendents in the current study were in the 56–60 age group, with survey responses at 33.33% and the national study at 29.6%. However, survey response data reflected a larger proportion of superintendents over age 60 than in the national study (29.41% and 18.1%, respectively). A majority of survey respondents (62.75%) were in their first superintendency, as was one of the superintendents who were interviewed. Mean tenure for survey respond- ents in their current position was 5.19 years, and mean tenure in the most recent two districts was 5.31 and 4.78 years, respectively. Overall, the profile of the typical survey respondent was male (76.47%), between 56 and 65 years old (54.90%, currently working in a unified school district (52.94%) with current district enrollment of 2,500 to 10,000 students (54.90%), currently serving as superintendent for the first time (62.75%), and in the current assignment for 3.73 years Demographics of the interviewed superintendents indicated that all were from unified districts with enrollments of 5,970 to 55,571 students and ranged in age from 48 73 to 62; three were males and two were females. Four had served as superintendent in at least one prior position, and the average tenure in the current position was 4.4 years. The chapter addressed each of the four research questions in sequence, presenting relevant data from both the survey instrument and the superintendent interviews. Research Question 1 asked, What is the relationship between superintendent tenure and sustainability of their reform efforts to improve student achievement? Find- ings indicated that the superintendents perceived a high correlation between their tenure and the sustainability of their reforms, with a mean score of 4.28 out of 5, with 49.02% of the respondents rating the correlation as very high. Statistical tests were applied to the data to measure differences in correlation ratings based on district size, district type, superintendent age, or superintendent gender. No statistically significant difference was found with regard to any of these demographic factors. While interview data unani- mously supported a correlation between superintendent tenure and the scores, the respondents indicated a belief that the strength of the correlation should diminish over time, with the superintendent’s role in sustaining the reform eventually taking a back seat to the will of others who want to sustain it. Research Question 2 asked, What factors do superintendents believe will impede or promote sustainability of their reforms beyond their tenure? Based on two open-ended survey questions in which superintendents were asked to identify sustainability strengths and weaknesses of their reforms, response patterns indicated that the superintendents viewed the existing culture of success in their districts to be their greatest ally in sustain- ing their reforms to improve student achievement. In terms of barriers, survey responses indicated a lack of budgetary resources and personnel turnover as the greatest potential 74 threats to reform sustainability. Interview responses regarding sustainability strengths echoed the survey data, but four of the five interviewees were more circumspect about budget problems being a weakness. In fact, these four superintendents agreed that suc- cessful reforms should be sustainable in virtually any budget climate. Research Question 3 asked, How do the factors identified in question two align with the characteristics of sustainability, as identified in research? Utilizing the charac- teristics of sustainability and successful whole-system reform (Fullan, 2005, 2010), quantitative and qualitative data were analyzed to determine whether the superintendents had engaged in research-based practices relative to their reforms and sustainability. Findings indicated that multiple alignments existed relative to Fullan’s (2005) elements of sustainability and that the respondents self-assessed their districts as possessing all nine elements of successful whole-system reform (Fullan, 2010). These findings were supported by qualitative data from the interviews. Collectively, the data indicated that, whether superintendents instinctively or deliberately employed a research-based approach in building sustainable reforms, their actions aligned with many of the characteristics of sustainability and successful whole-system reform. However, the degree to which each superintendent individually incorporated these elements into his or her work could not be determined from the data in this study. Research Question 4 asked, To what extent do superintendents consider sustain- ability when they initiate a particular reform to improve student achievement? Survey and interview results both led to the conclusion that the superintendents considered sus- tainability when they initiated a reform to improve student achievement. In fact, survey data suggested that the superintendents believed that, if they were to leave their current 75 posts on or about January 1, 2013, their reforms would still be in place more than 8 years after the reforms had been initiated. While interview data were consistent, the manner in which sustainability was considered varied from superintendent to superintendent. Ulti- mately, all interviewees agreed that no reform would be worth undertaking without con- sidering its long-term viability. 76 CHAPTER 5 CONCLUSIONS Background and Purpose of This Study The position of school district superintendent has evolved over decades into an increasingly demanding and complex role that some have described as impossible (Edu- cation Writers Association, 2003; Fuller et al., 2003; Houston, 2001), while others have labeled it a leadership paradox: Schools are complex places with some of the most inherently difficult challenges in the world. Teaching and learning occur in a varied community of children and adults surrounded by concerned parents, graduates and neighbors. The challenges include problems, puzzles, and seeming contradictions that need to be deciphered, resolved or accepted. It takes both technical competence and symbolic sensitivity to get the job done with dignity and grace. (Deal & Peterson, 1994, p. 10) No matter how or by whom the role of superintendent is defined, within its current context the inescapable reality is that, among their many duties, responsibilities, and priorities, superintendents stand at the fore of educational innovation and are largely responsible for wielding what Fullan (2005) called “the long lever of leadership” (p. 27). Superintendents’ ability to lead reform aimed at improving student achievement requires a focus on simplicity, clarity, and priority (Schmoker, 2011). What is required of superintendents to make their reforms sustainable over time, so the positive effects of their reforms become embedded in practice for the benefit of current and future students? The central hypothesis of this study was that one of the keys to reform sustainability may not only be district leaders themselves or the reforms that they have chosen to implement in their districts, but the length of time they are able to devote to nurturing their reforms to maturity and self-sufficiency. 77 Despite the abundant research on educational change, leadership in general, and school district superintendents in particular, little is known or documented about whether superintendent tenure is connected in some way to the long-term sustainability of their reform efforts to improve student achievement. In fact, the research literature identifies a gap of 2 to 5 years between the minimum time needed for reforms to inculcate within an organization’s culture and the average tenure of today’s superintendents. The inquiry that ultimately led to this study centered on one essential question: If the average superintendent leaves the position years before his or her reforms are opera- tionalized within the district, how does successful educational innovation occur in the long term? Unfortunately for many school districts whose leaders have built elaborate strategic plans around noble and lofty goals and vision and mission statements, the simple answer to this essential question is that, all too often, innovations are not sustained, despite the fact that successful innovation in education is possible and can happen. According to Hattie (2012), who calculated the effect sizes of hundreds of meta-analyses on studies of school reform, “Almost everything works. Ninety percent of all effect sizes in education are positive (p. 15). Yet, Peter Senge wrote, “America has been trying to turn around its schools for a quarter century, with tragic results. One simplistic quick-fix nostrum after another has seized the political limelight and been ‘driven’ through the system as if it was all that was needed” (as cited in Fullan, 2010, p. vii). Since virtually every reform works but most do not last, as evidenced by the “this too shall pass” attitude of many professional educators, this study addressed the merits of shifting the focus from the reforms themselves, to tenure, or lack of it, on the part of the key leaders who initiated the reforms in the first place: school district superintendents. 78 The result was that the primary purpose of this study was to spotlight the relation- ship between superintendent tenure and sustainability of reform efforts to improve student achievement. This study also compared superintendents’ perceptions about factors supporting or impeding sustainability to factors identified in the literature. The study was also designed to determine the extent to which superintendents consider sus- tainability before initiating any reform effort. Four research questions were developed around the study’s purpose and an exten- sive review of pertinent research literature was conducted. A plan or methodology was developed to address the research questions, and data (both quantitative and qualitative) were collected. The data were analyzed to address the research questions. Summary of Findings As presented in Chapter 4, all four research questions were successfully addressed within the context of this study. The superintendents who participated in an online survey and in face-to-face interviews reported that they not only considered sustainability in the reforms that they undertook but also viewed sustainability as dependent on them contin- uing in the position long enough for their reforms to engender buy-in from a multitude of constituent groups. The findings of this study were consistent across district type and size, as well as superintendent gender, age, and level of experience in the superin- tendency. Data showed that the responding superintendents, whether through their back- ground knowledge, intuition, or a combination of the two, were aware of some research- based elements of reform sustainability. Overall, the participating superintendents were optimistic about sustainability of their reforms beyond their tenure. 79 Reflections on the Findings Mixed-Methods Research Design and Research Question 1 Perhaps the most important reflection to emerge from the study findings was that the quantitative survey data alone, while important and informative, told only part of the story. The qualitative data from face-to-face interviews with working superintendents helped to clarify the numerical findings, give them meaning, and deepen their impact. For example, one important finding was that the superintendents reported a strong correlation between their tenure as district leaders and the sustainability of their reform efforts to improve student achievement. From a pragmatic standpoint, it makes sense that a leader who devoted time, effort, and resources to bringing an innovation to life in the district would find value to staying in the position long enough to see the reform take hold. This survey finding was validating in terms of addressing Research Question 1 but it was insufficient to gain understanding of the correlation itself. Superintendent interviews solved that problem by giving respondents the latitude to expand on their survey answers regarding the correlation between tenure and sustainability. In their view, superintendent tenure matters only to the point at which others begin to take responsibil- ity for a reform’s success. The time frame for that process to occur, according to the interviewees, is situation specific and dependent on a host of factors. While it was important to learn that superintendents acknowledged a correlation between their tenure and reform sustainability, it was equally important to learn that the correlation existed within a context. Absent the qualitative data, this context would have been unknown. In short, the decision to use a mixed-methods approach to this study was helpful in gaining a complete understanding of the numerical data. 80 Elements of Sustainability and Research Questions 2 and 3 Another significant reflection involved Research Questions 2 and 3, which sought to capture the strengths and weaknesses of sustainability as perceived by superintendents and align them with sustainability characteristics identified in research. The survey and interview data revealed that, while the superintendents appeared to recognize some of the key characteristics of reform sustainability as defined in research, their knowledge of these factors was neither complete nor consistent. For example, all five interviewees shared information indicating that they understood the importance of two elements necessary for sustainability: lateral capacity building through networks and commitment to changing context at all levels (Fullan, 2005). However, only one of the superintendents spoke definitively about public service with a moral purpose (Fullan, 2005). Survey results were even more fragmented than interview data relative to the elements of sustainability. While it is possible that the superintendents may have been fully knowledgeable about all of the research-based elements of reform sustainability but chose to share only one or two of them in responding to this study, it is also possible that they were not aware of all factors needed to make their reforms last. Further study may be warranted in this area. Optimism Regarding Sustainability and Research Question 4 The third reflection centered on a particular finding relative to Research Question 4, which asked the extent to which superintendents considered sustainability when initi- ating a particular reform. Specifically, the survey data presented in Figure 5 were unexpected. These data reflect the superintendents’ responses to a hypothetical question in which they were asked 81 to imagine that they left their current position on or about January 1, 2013, and to rate the likelihood of their reform’s sustainability beyond 8 years from the time it had been implemented. Of all responses collected, 68% were highly optimistic about the sustain- ability of their reforms, even in their absence. At first this finding seemed to be in conflict with the survey responses, in which 80.39% rated the correlation between their tenure and reform sustainability as high. Upon further reflection and in consideration of the interview responses discussed initially in this section, a different conclusion could be drawn. While the superintendents rated the correlation between reform sustainability and their tenure as high, they could also have believed that they would have been in their position long enough by January 1, 2013, to be highly optimistic about the lasting impact of their chosen reform. Again, further study may be warranted to make a more definitive determination. Limitations of the Study Limitations of the study, either individually or in combination, may affect this study’s validity and reliability, as well as any generalization of findings. The first three limitations were identified in Chapter 1. 1. The study was not longitudinal in design and therefore relied on respondents’ predictors and perceptions about sustainability rather than in-the-field observation and measurement. In other words, the fact that superintendents reported a correlation between their own longevity and that of their reforms does not necessarily mean that a correlation actually exists. Validating the perceptions of superintendents would require a more thorough and structured study, over time, perhaps using control groups and random sampling. 82 2. With the exception of one northern California superintendent interviewee, all interviewees and online survey respondents were from southern California. Thus, the sample may or may not be representative of all superintendents. Care should be exercised in extrapolating the findings to all superintendents. 3. The study did not employ research-based metrics to measure or predict sustain- ability. Instead, superintendents were asked to gauge the potential sustainability of their reforms using a Likert-type rating scale. While this metric is believed to have measured superintendents’ hopes or expectations regarding sustainability, it could not measure sustainability per se. This limitation goes beyond the scope of the present study because, according to the Mental Measurement Yearbook, no metric currently exists to measure the sustainability of an educational reform. 4. In conjunction with the lack of a standardized metric to measure sustainability, another possible limitation is the reliance of several study findings on Michael Fullan’s eight elements of sustainability (Fullan, 2005). Although some or all of these elements are consistently found throughout the research literature on sustainability, it is unknown whether they actually and completely define sustainability. To the researcher’s knowledge, no conclusive studies have been conducted to validate formally these specific elements, which alone are responsible for determining sustainability of educational reforms. 5. Survey response rate was 20.73%, or 51 of the 246 surveys distributed. While the demographics of the response population were similar to those in national surveys, the extent to which respondents may have been alike relative to their thinking around reform sustainability and superintendent tenure is unknown. As with the geographic 83 limitation regarding the distribution of surveys, care should be exercised in generalizing the findings of this study to the population of superintendents. 6. Interview response size consisted of five superintendents, all from unified school districts. Unlike survey participants who were sent links to an online survey via mass email to all superintendents in southern California, all five interviewees were selected by the researcher, which could present a limitation due to selection-assignment bias. 7. Time was a limitation in that all study data had to be collected within a finite period in which superintendents would be most likely available to participate in the study. The limitation of time may have influenced responses to open-ended questions, since the respondents could have truncated their responses in order to make their most salient point or points in the shortest time possible. 8. Minimal human and other resources were available to conduct this study, lim- iting its scope and size accordingly. This limitation, arguably applicable to any research, was especially relevant in this instance. Expanding this study beyond its current scope would have required additional funding, personnel, and other resources not available within this context. 9. The researcher may have presented a limitation with regard to critical reflexiv- ity. Reflexivity calls for self-reflection, indeed, critical self-reflection and self- knowledge and a willingness to consider how who one is affects what one is able to observe, hear, and understand in the field as an observer and analyst. The observer, therefore, during fieldwork, must observe self as well as others, and interactions of self with others. (Patton, 2002, p. 299) 84 In other words, through his interactions with interviewees, the researcher himself could have had some influence on responses to interview questions. This, in turn, could have influenced qualitative data. It should be noted that the qualitative interviews conducted in this study were the first ever performed by the researcher. This lack of experience, par- ticularly with regard to reflexivity, could have influenced the findings of this study. Implications for Practice One major finding of this study is that superintendent tenure matters in terms of creating and sustaining successful reforms to improve student achievement. An implica- tion, therefore, is that school districts in general and school boards in particular should hire the leader that their district needs and keep him or her in the position until key reform efforts have taken hold and will move forward under the leadership of a broad and supportive base. In instances in which superintendent turnover has become an impedi- ment to reform sustainability, school boards may want to rethink the manner in which they select their superintendents. For example, according to University of Southern California professor Morgan W. McCall, Jr. (1998), most organizations tend to hire “right stuff” (p. 6) leaders who have risen to the top through a standardized career path and come equipped with a previ- ously identified set of skills. Meanwhile, an extensive longitudinal study of management conducted by Howard and Bray (1988) found that “the men who advanced furthest tended not to be promoted in a straight line through the same type of function” (p. 174). McCall (1998) suggested that leaders be selected based on evidence of their experience in dealing with the specific types of challenges that they will encounter in their new role. In other words, instead of school boards recruiting superintendent candidates whom they 85 assume have “the right stuff” based on past success in a structured path, McCall’s theory would suggest gaining a deeper understanding of the specific experiences of the candi- dates relative to areas critically needed in the position to be filled. According to McCall (1998), “Although it is revealing to know that a person has survived challenging situa- tions, it is more revealing to know what skills the challenges actually required the person to demonstrate. Passing irrelevant tests, however spectacular, is not helpful” (p. 11). Adopting such an approach to recruitment and selection of a new superintendent may require a paradigm shift on the part of school boards as well as the recruiters whom the board hires. However, the result could be a superintendent who is a better match for what the district needs, is successful as a result, and lasts long enough in the position to enable the reforms to take hold. Another finding of this study is that superintendents engaged in reform efforts have an awareness of some, but not necessarily all, of the research-based elements needed to maximize sustainability. An implication of this finding would be to explore creating professional development opportunities for superintendents and their boards to learn about the essential elements of successful reform and sustainability. Training could be provided through professional organizations such as the California Association of School Administrators and California School Boards Association , as well as incorpo- rated within the curriculum of graduate-level credential and degree programs for educa- tional administrators. Helping superintendents and their boards to become fully knowledgeable about all factors that promote sustainability could improve their capacity to achieve sustainability. Conversely, without this knowledge, reform sustainability may not be possible. As Fullan 86 (2005) noted, “There is no chance that large-scale reform will happen, let alone stick, unless capacity building is a central component of the strategy for improvement” (p. 10). A seemingly minor or inconsequential dataset obtained in this study was the age distribution of survey respondents. In the current study, 54.9% of the respondents reported their age at 56 to 65 years, with 29.41% of all respondents being 60 or older. While not specifically targeted in this study as an area of inquiry, the superintendent age demographics found in this study and substantiated by a national survey (Kowalski et al., 2011) indicate that a substantial number of superintendent positions will soon turn over. In fact, Kowalski et al. (2011) found that 49% of the superintendents responding to their national survey indicated that they would not be in their current position beyond 2015. Relative to the implications for practice, the turnover of significant numbers of superin- tendent positions in the next several years has significant implications. Assuming validity of the correlation found in this study relative to superintendent tenure and reform sustain- ability, significant turnover in the next several years could compromise reform efforts now in the implementation pipeline. In addition, significant superintendent turnover has implications for ongoing recruitment and selection of the leaders who will take their places. An implication for practice related to the superintendent age demographics noted above and Carver and Scheier’s (2003) discussion of giving up as a natural part of lifespan development could have consequences for sustainability of reform. According to Carver and Scheier (2003), “Humans have a vast potential for what they can become and accomplish in a lifetime. However, because time and resources are limited, people must make choices about which goals to pursue and which to give up” (p. 93). In other words, 87 as superintendents approach retirement and the end of their careers, are they as willing to confront the challenges and stresses of sustainable reform as they would have been at a different time in their careers? An answer to this question may be approached through future research and will be discussed in the next section. Nevertheless, the implications on reform sustainability of an aging superintendent population are probably inescapable in view of their diminishing biological resources and reserves (Carver & Scheier, 2003). Future Research This study identifies possibilities for future research, particularly given that superintendents’ longevity “has a positive effect on the average academic achievement of students” (Marzano & Waters, 2009, p. 9) and virtually no research has been done rela- tive to the long-term sustainability of this impact. First, as previously stated, this study relied heavily on superintendents’ personal opinions to establish a correlation between tenure and sustainability of reform efforts to improve student achievement. Further research is needed to determine whether an actual link exists between superintendent tenure and reform sustainability. For example, a lon- gitudinal study tracking similar reform efforts implemented across several school districts could provide valid and reliable evidence to substantiate or contradict the findings of this study. Further research would be helpful with regard to the elements that Fullan (2005) and others identified as essential for sustainability of any systemwide change. Because each element is complex and multifaceted in and of itself, many research opportunities exist relative to studying them individually as well as their interactions with each other. The implications for such research could be significant in real-world practice, as 88 understanding the true elements of sustainability would provide a metric against which contemplated reform efforts could be measured. This, in turn, could help district leaders to determine which reform pathways stand the best chances of succeeding for them. The ability to assess which reforms have the greatest likelihood of sustainability would ulti- mately enable superintendents and their boards to align limited fiscal and human resources for their greatest effect. Another possible area for research is quantification of the time needed for reforms to become sustainable and therefore part of a school district’s normal operating practice. Although existing research indicates that sustainability effect sizes become significant after 8 years of implementation (Borman et al., 2003), the findings of this study indicate that sustainability markers may vary depending on the reform and the variant conditions under which it is implemented. It is granted that such research would be challenging, given the number of possible variables and conditions, both internal and external, impacting school districts. However, having even an approximate idea of the time needed for a reform to become sustainable could have significant impact on real-world situations and the strategic allocation of scarce and valuable resources. The superintendency itself provides fertile ground for future research. Although numerous aspects of the superintendency have been addressed in the research literature and continue to be investigated, relatively little is yet known about Marzano and Waters’s so-called “bonus finding” (2009, p. 113) that superintendent tenure impacts student achievement. Further research is needed not only to validate this relationship but also to elaborate on it. The implications could be significant on a multitude of levels, including how districts recruit, select, evaluate, compensate, and retain their superintendents. 89 In conjunction with Carver and Scheier’s (2003) discussions about the ways in which peoples’ goals and choices evolve as they age, another potential area of research is the extent to which these natural developmental changes may impact superintendents’ commitment to leading reform, vis-à-vis their age. From the research as well as real- world practice, it is well established that leading change and transforming it into an accepted and valued part of a school districts culture takes extensive energy, focus, con- sistency, and commitment over time. Does the age of a leader have an impact on these factors; if so, how? Is it realistic to expect reform sustainability from a superintendent who is perhaps a few years away from retirement or should different strategies be employed to assure sustainability of the superintendent’s reforms? These and many other queries could be addressed with further research in the area of reform sustainability, age, and lifespan development (Carver & Scheier, 2003). Chapter Summary This chapter began with a review of the background and purpose of the study, a summary of the study’s key findings, and reflections regarding connecting the findings to the four research questions. Nine limitations of the study were discussed, as well as four implications for practice and five possibilities for future research. In bringing this study full circle and ultimately back to its roots, the often-heard refrain of “this too shall pass” must be recalled. With this simple phrase, whether spoken aloud or quietly contemplated, educators are taking a stand in favor of the status quo. While it is possible that they may be planting their feet in opposition or resistance to any change, it is also possible that they have been “down the road” of reform so many times that the call for yet another innovation by a new district leader has begun to sound like a 90 chapter from the children’s story about the boy who cried wolf. Given the relatively short tenure of school district superintendents in comparison to parents, teachers, staff, and students, skepticism about jumping aboard his or her latest reform is unfortunate but understandable. Despite its limitations, this study affirmed the importance of the relationship between superintendent longevity and reform sustainability. Yet this study also found that buy-in from all levels within the organization consistently reduces the need for persistent superintendent involvement in sustaining a reform. The ongoing challenge, then, may not be for superintendents and their boards to focus only on reform sustainability per se but also on building commitment, continuity, capacity, and shared leadership within their districts and communities. With everyone, including the board and superintendent, con- tributing to the forward movement of the district, reforms to improve student achieve- ment stand a good chance for sustained success. Perhaps most important, through their present collective efforts to make a lasting difference for generations to come, students who are not yet born may someday enter into a system that could not conceive of going back to what it once was. 91 REFERENCES Baltes, P. B. (1997). On the incomplete architecture of human ontogeny: Selection, opti- mization, and compensation as foundation of developmental theory. American Psychologist, 52, 366-380. doi:10.1037/0003-066X.52.4.366 Bennis, W., & Nanus, B. (2003). Leaders: Strategies for taking charge. New York, NY: Harper Paperbacks. Bergman, M. M. (2008). Advances in mixed-methods research: Rheories and applications. Los Angeles, CA: Sage. Best, J. W. (1948). A study of certain selected factors underlying the choice of teaching as a profession. Journal of Experimental Education, 17, 201-259. Björk, L. G. (1996). The revisionists’ critique of the education reform reports. Journal of School Leadership, 6, 290-315. Björk, L. G., & Kowalski, T. J. (2005). The contemporary superintendent: Preparation, practice, and development. Thousand Oaks, CA: Corwin. Bolman, L. G., & Deal, T. E. (1994). Looking for leadership: Another search party’s report. Educational Administration Quarterly, 30(1), 77. Bolman, L. G., & Deal, T. E. (2008). Reframing organizations: Artistry, choice, and leadership (4th ed.). San Francisco, CA: Jossey-Bass. Borman, G. D., Hewes, G. M., Overman, L. T., & Brown, S. (2003). Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research, 73(2), 125-230. Boston Latin School Association. (2010). Boston Latin School. Retrieved from https:// www.bls.org Bracht, G. H., & Glass, G. V. (1968). The external validity of experiments. American Educational Research Journal, 5, 437-474. Brady, G. F., & Helmich, D. L. (1984). Executive succession: Toward excellence in corporate leadership. Englewood Cliffs, NJ: Prentice-Hall. Brewer, C. (2011). School leaders as political strategists: William Boyd’s contributions to our understanding of the politics of leadership. Peabody Journal of Education, 86, 450-463. doi:10.1080/0161956x.2011.597276 Broad Foundation & Thomas B. Fordham Institute. (2003). Better leaders for America’s schools: A manifesto. Los Angeles, CA: Author. Byrd, J. K., Drews, C., & Johnson, J. (2006). Factors impacting superintendent turnover: Lessons from the field. Retrieved from http://eric.ed.gov/PDFS/ED493287.pdf 92 Carnoy, M., & Levin, H. (1985). Schooling and work in the democratic state. Stanford, CA: Stanford University Press. Carpenter, W. A. (2000). Ten years of silver bullets: Dissenting thoughts on education reform. Phi Delta Kappan, 81, 383-389. Carter, G. R., & Cunningham, W. G. (1997). The American school superintendent: Leading in an age of pressure (1st ed.). San Francisco, CA: Jossey-Bass. Carver, C. S., & Scheier, M. F. (2003). Three human strengths. In L. G. Aspinwall & U. M. Staudinger (Eds.), A psychology of human strengths: Fundamental questions and future directions for a positive psychology (pp. 87-102). Washington DC: American Psychological Association. Casserly, M., Lewis, S., Uzzell, R., Horwitz, A., & Simon, C. (2010). Urban school superintendents: Characteristics, tenure and salary. Washington DC: Council of the Great City Schools. Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational researcher, 32(6), 3-12. Cohn, M. M., & Kottkamp, R. B. (1993). Teachers: The missing voice in education. Albany, NY: State University of New York Press. Collins, J. C. (2001). Good to great: Why some companies make the leap—and others don't (1st ed.). New York, NY: HarperBusiness. Comte, T. E., & Mihal, W. L. (1990). CEO turnover: Causes and interpretations. Business Horizons, 33(4), 47-51. doi:10.1016/0007-6813(90)90057-i Conley, S. C., & Cooper, B. S. (2011). Finding, preparing, and supporting school leaders: Critical issues, useful solutions. Lanham, MD: Rowman & Littlefield Education. Coughlin, B. (2009). Public Latin School, 1635-1935. Retrieved from http://www.hmdb .org/Marker.asp?Marker=18511 Creswell, J. W., Plano Clark, V. L., Gutmann, M. L., & Hanson, W. E. (2003). Advanced mixed-methods research designs. In A. Tashakkori & C. Teddie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-240). Thousand Oaks, CA: Sage. Cuban, L. (2001). Leadership for student learning: urban school leadership: Different in kind and degree. Washington, DC: Institute for Educational Leadership. Cubberley, E. (1919). Public education in the United States: A study and interpretation of American educational history. An introductory textbook dealing with the larger problems of present-day education in the light of their historical development. New York, NY: Houghton-Mifflin. 93 Datnow, A. (2005). The sustainability of comprehensive school reform models in changing district and state contexts. Educational Administration Quarterly, 41(1), 121-153. Deal, T. E., & Peterson, K. D. (1994). The leadership paradox: Balancing logic and artistry in schools (1st ed.). San Francisco, CA: Jossey-Bass. Desimone, L. (2002). How can comprehensive school reform models be successfully implemented? Review of Educational Research, 72(3), 433. Drucker, P. F. (1990). Managing the non-profit organization: Practices and principles (1st ed.). New York, NY: HarperCollins. DuFour, R., & Eaker, R. (1998). Professional learning communities at work: Best practices for enhancing student achievement. Bloomington, IN: National Education Service. DuFour, R., Eaker, R., & DuFour, R. (2005). On common ground: The power of pro- fessional learning communities. Bloomington, IN: National Educational Service. Education Writers Association. (2003). Effective superintendents, effective boards: Finding the right fit. Washington, DC: Author. Fink, D. (2000). Good schools/real schools: Why school reform doesn’t last. New York, NY: Teachers College Press. Fullan, M. (2005). Leadership and sustainability: System thinkers in action. Thousand Oaks, CA: Corwin. Fullan, M. (2010). All systems go: The change imperative for whole system reform. Thousand Oaks, CA: Corwin. Fuller, H. L., Campbell, C., Celio, M. B., Harvey, J., Immerwahr, J., & Winger, A. (2003). An impossible job: The view from the urban superintendent’s chair. Seat- tle, WA: University of Washington, Center on Reinventing Public Education. Gallagher, K. S., & Bailey, J. D. (2000). The politics of teacher education reform: Strategic philanthropy and public policy making. Educational Policy, 14(1), 11- 24. doi:10.1177/0895904800014001003 Gardner, D. P. (1983). A nation at risk. Washington, DC: U.S. Government Printing Office. Gardner, J. W. (1990). On leadership. New York, NY: The Free Press. Giles, C., & Hargreaves, A. (2006). The sustainability of innovative schools as learning organizations and professional learning communities during standardized reform. Educational Administration Quarterly, 42(1), 124-156. Gladwell, M. (2000). The tipping point: How little things can make a big difference (1st ed.). Boston, MA: Little, Brown. 94 Glass, T. E., & Franceschini, L. A. (2007). The state of the American school superintend- ency: A mid-decade study. Lanham, MD: American Association of School Administrators and Rowman & Littlefield Education. Hargreaves, A., & Fink, D. (2000). The three dimensions of reform. Educational Leadership, 57(7), 30-33. Hattie, J. (2012). Visible learning for teachers: Maximizing impact on learning. New York, NY: Routledge. Hochschild, J. L., & Scovronick, N. B. (2003). The American dream and the public schools. New York, NY: Oxford University Press. Hoyle, J. R. (2005). The superintendent as CEO: Standards-based performance. Thousand Oaks, CA: Corwin. Houston, P. (2001). Superintendents for the 21st century. Phi Delta Kappan, 82, 428-433. Howard, A., & Bray, D. W. (1988). Managerial lives in transition: Advancing age and changing times. New York, NY: Guilford. Jick, T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24, 602-611. Johnston, R. C. (2000). Vacuum at the top takes heavy toll on Dallas schools. Education Week on the Web, 20(13). Retrieved from http://www.edweek.org/ew/articles/ 2000/12/13/15dallas.h20.html?qs=heavy+toll+on+dallas+schools Kowalski, T. J., McCord, R. S., Peterson, G. J., Young, P., & Ellerson, N. M. (2011). The American school superintendent: 2010 decennial study. Lanham, MD: American Association of School Administrators and Rowman & Littlefield Education. Leithwood, K., Harris, A., & Hopkins, D. (2008). Seven strong claims about successful school leadership. School Leadership and Management, 28(1), 27-42. doi:10.1080/13632430701800060 Leithwood, K., & Riehl, C. (2003). What we know about successful school leadership. Philadelphia, PA: Laboratory for Student Success. Marzano, R. J., & Waters, T. (2009). District leadership that works: striking the right balance. Bloomington, IN: Solution Tree Press. Marsiske, M., Lang, F. R., Baltes, P. B., & Baltes, M. M. (1995). Selective optimization with compensation: Life-span perspectives on successful human development. In R. A. Dixon & L. Backman (Eds.), Compensating for psychological deficits and declines: Managing losses and promoting gains (pp. 35-79). Mahwah, NJ: Erl- baum. McCall, M. W. (1998). High flyers: Developing the next generation of leaders. Boston, MA: Harvard Business School Press. 95 McKay, J., & Grady, M. (1994). Turnover at the top. Executive Educator, 16(8), 37-38. McLaughlin, M. W. (1990). The Rand Change Agent Study revisited: Macro perspectives and micro realities. Educational Researcher, 19(9), 11-16. McNeil, L. M. (2000). Contradictions of school reform: Educational costs of standardized testing. New York, NY: Routledge. Myers, S. P. (2010). A multiple regression analysis of six factors concerning school district demographics and superintendent tenure and experience in 2007-08 schools relative to student achievement on the third grade Kansas reading assessments (Doctoral dissertation). Kansas State University, Manhattan, KS. Natkin, G. L., Cooper, B. S., Alborano, J. A., Padilla, A., & Ghosh, S. (2002, April). Predicting and modeling superintendent turnover. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA. Natkin, G. L., Cooper, B. S., Fusarelli, L., Alborano, J., Padilla, A., & Ghosh, S. (2002). Myth of the revolving-door superintendency. School Administrator, 59(5), 28-31. Northouse, P. G. (2010). Leadership: Theory and practice (5th ed.). Thousand Oaks, CA: Sage. Pascopella, A. (2011). Superintendent staying power: DA editors look back over 15 years at the tenure of the men and women who have led the 10 largest districts in the nation. District Administration, 47, 31. Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage. Peterson, G. J. (2010). Superintendent leadership. In S. Conley & B. S. Cooper (Eds.), Finding, preparing, and supporting school leaders: Critical issues, useful solutions (pp. 137-154). Lanham, MD: Rowman & Littlefield Education. Peterson, G. J., & Barnett, B. G. (2005). The superintendent as instructional leader: Current practice, future conceptualizations and implications for preparation. In L. G. Bjork & T. J. Kowalski (Eds.), The contemporary superintendent: Prepara- tion, practice and development (pp. 107-136). Thousand Oaks, CA: Corwin. Pfeffer, J., & Sutton, R. I. (2000). The knowing-doing gap: How smart companies turn knowledge into action. Boston, MA: Harvard Business School Press. Porter, A. C. (1994). National standards and school improvement in the 1990s: Issues and promise. American Journal of Education, 102, 421-449. Salkind, N. J. (2008). Statistics for people who (think they) hate statistics (3rd ed.). Thousand Oaks, CA: Sage. Schmoker, M. (2011). Focus: Elevating the essentials to radically improve student learning. Alexandria, VA: ASCD. 96 Sims, R. R. (2010). Change (transformation) in government organizations. Charlotte, NC: Information Age. Snipes, J., Doolittle, F., & Herlihy, C. (2002). Foundations for success: Case studies of how urban school systems improve student achievement. Washington, DC: Manpower Demonstration Research Corporation (MDRC) for the Council of Great City Schools. Spillane, R., & Regnier, P. (1997). The superintendent of the future: Strategy and action for achieving academic excellence: Baltimore, MD: Aspen. Stone, J. E., & Clements, A. (1997). Research and innovation: Let the buyer beware. In R. Spillane & P. Regnier (Eds.), The superintendent of the future: Strategy and action for achieving academic excellence (pp. 59-97). Baltimore, MD: Aspen. Tamir, E. (2009). Choosing to teach in urban schools among graduates of elite colleges. Urban Education, 44, 522-544. doi:10.1177/0042085909339373 Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrat- ing quantitative and qualitative approaches in the social and behavioral sciences. Los Angeles, CA: Sage. Tuckman, B. W. (1999). Conducting educational research (5th ed.). Fort Worth, TX: Harcourt Brace College. Twain, M. (1978). The complete adventures of Tom Sawyer and Huckleberry Finn (Centennial ed.). New York, NY: Harper & Row. Tyack, D. B., & Cuban, L. (1995). Tinkering toward utopia: A century of public school reform. Cambridge, MA: Harvard University Press. U.S. Department of Education. (2002). Comprehensive School Reform (CSR) program guidance. Retrieved from http://www2.ed.gov/programs/compreform/ legislation.html Wernet, S. P., & Austin, D. M. (1991). Decision-making style and leadership patterns in nonprofit human service organizations. Administration in Social Work, 15(3), 1. Yee, G., & Cuban, L. (1996). When is tenure long enough? A historical analysis of superintendent turnover and tenure in urban school districts. Educational Administration Quarterly, 32(Suppl. 1), 615. 97 APPENDIX A EMAILED INVITATION TO PARTICIPATE April 9, 2012 Dear <SUPERINTENDENT NAME>, My name is Rick Bagley and I am a doctoral candidate in the Rossier School of Education at Uni‐ versity of Southern California. I am conducting a research study as part of my dissertation, focusing on the sustainability of superintendent‐led reforms to improve student achievement. You are invited to participate in my study. If you agree, you are invited to participate by com‐ pleting a short survey at <INSERT LINK TO SURVEY>. The survey is anticipated to take no more than 10 minutes to complete. At the end of the survey, you will be asked if you are willing to be considered for participation in a face‐to‐face, follow‐up interview. If selected, follow‐up interviews will last approximately 45 minutes and may be audio‐taped. Participation in this study is voluntary. Your identity as a participant will remain confidential at all times during and after the study. If you have questions, please contact me via email at rbagley@usc.edu or by phone at 818‐999‐ 9643. Thank you in advance for your consideration and participation, Sincerely, Rick E. Bagley Ed.D. Student University of Southern California 98 APPENDIX B ONLINE SURVEY 99 100 101 102 103 104 105 APPENDIX C QUALITATIVE INTERVIEW PROTOCOL UNIVERSITY OF SOUTHERN CALIFORNIA Rossier School of Education Doctor of Education METHODS AND PROCEDURES: Interview Questions Rick E. Bagley, Ed.D. Student rbagley@usc.edu Dissertation Topic: Sustainability of Superintendent‐led Initiatives to Improve Student Achievement Research Design: A mixed methods approach will be utilized employing triangulation (Jick, 1979) through sequential data collection and analysis (Creswell, Plano Clark, Gutmann, & Hanson, 2003). The rationale in support of this approach is based on the nature of the research questions themselves. The initial approach will involve collection of data through the use of a survey instrument developed specifically for this study. Data will be analyzed from descriptive, relational and predictive perspectives rela‐ tive to a specific hypothesis. Qualitative data was then gathered from a subgroup of respondents, through a face‐to‐face, standardized open‐ ended interview process (Patton, 2002) designed to provide more in‐ depth analysis. In short, by collecting and analyzing quantitative data first and qualitative data second, this study follows an implicit design path, testing variables using a larger sample before drilling‐down for more in‐depth exploration (Creswell, et al., 2003). As a result of this sequential or two‐phased approach to the study design, quantitative and qualitative analyses will be completed separately and then inte‐ grated into a final discussion. Because quantitative findings from this research are expected to direct the subsequent qualitative analysis, what Creswell, et al., (2003) refer to as a “dominant‐less dominant model” (p. 219), the initial assumption is that quantitative aspects will take priority over those which are qualitative. 106 Phase Two Interview Questions In phase two of the research, five superintendents from throughout Los Angeles County will be selected for interview. Each of the superintendents interviewed will be asked the following ques‐ tions during a face‐to‐face meeting, which will be recorded for later analysis. Q1: In regard to education reform, a common statement heard throughout the profession and well documented in the literature is, “this too shall pass.” What are your thoughts about that statement? Q2: Research indicates American superintendents, on average, serve approximately 6 years in their positions. Assuming the research is accurate, what are your thoughts about this statistic? Q3: How long have you been superintendent of this district? Q4: Prior to coming to this district, were you superintendent in any other school district(s)? If so, how long did you serve as superintendent in each of your prior school districts? Q5: In the context of your current district, please take a few moments to tell me about the most significant reform or change you initiated, to improve student achievement? Q6: What were your three most important reasons for initiating this particular reform or change? Q7: Was long‐term sustainability of your particular reform, a factor contributing to your decision to implement it? If so, why? If not, why not? Q8: Imagine for a moment that you leave this position on or around January 1, 2013. In your opinion what factors or conditions exist in your district that will contribute to sus‐ tainability of your reform for the next eight or more years? What factors or conditions in your district might jeopardize your reform’s sustainability? Q9: <REPEAT Q5 through Q7 FOR EACH PRIOR DISTRICT IN WHICH INTERVIEWEE WAS THE SUPERINTENDENT. FOLLOW‐UP QUESTIONS ABOUT EACH DISTRICT WITH …> Please tell me about the extent to which your reform currently exists in your former school district. Q10: Thinking of your current position and district, are you aware of any reforms or initiatives your predecessors led, which are still in effect today? If so, please tell me about them. Q11: <REPEAT Q10 FOR EACH PRIOR DISTRICT IN WHICH INTERVIEWEE WAS THE SUPERIN‐ TENDENT.> 107 Q12: In your opinion, does reform sustainability matter? Please explain your answer. Q13: Do you believe a correlation exists between superintendent tenure and the statement, “this too shall pass?” Please explain your answer. Q14: Is there anything I didn’t ask about that you would like to share and/or do you have any questions for me? End of interview Questions

Abstract (if available)

Abstract The purpose of this research was threefold. First, the study explored the possible relationship between the tenure of public school district superintendents and the sustain-ability of their reform efforts to improve student achievement. Second, the study com-pared superintendents’ perceptions of factors supporting or impeding sustainability of their reforms to elements identified in research. Third, the study sought to determine the extent, if any, to which superintendents considered sustainability when initiating reforms. A mixed-methods design was employed to combine theoretical and technical components of qualitative and quantitative analysis. Quantitative data from 51 southern California superintendents were collected via an online survey. Four southern California superin-tendents and 1 northern California superintendent were interviewed to obtain qualitative data. Analysis of the data indicated that, while the superintendents perceived a strong correlation between their tenure and reform sustainability, they noted that this correlation diminishes over time as reforms gain traction through broader support and acceptance. With regard to current reforms within their school districts, the superintendents recog-nized some but not all of the eight research-based elements deemed necessary for long-term sustainability. Finally, the data indicated that the superintendents considered sustainability when initiating reforms and were highly optimistic about their reforms lasting the test of time despite research and experience that might indicate otherwise. Implica¬tions for practice included efforts to minimize position turnover through differentiated strategies for recruitment and selection of superintendents, targeted

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Sustainable reform: a follow-up case study on one urban superintendent’s efforts to improve student achievement

PDF

Superintendents' viewpoint of the role stakeholders can play in improving student achievement

PDF

Effective strategies superintendents utilize in building political coalitions to increase student achievement

PDF

Leadership strategies employed by K-12 urban superintendents to improve the academic achievement of English language learners

PDF

Strategies employed by successful urban superintendents responding to demands for student achievement reform

PDF

Effective strategies that urban superintendents use that improve the academic achievement for African-American males

PDF

Leadership strategies employed by K-12 urban superintendents to improve the academic achievement of English language learners

PDF

Strategies employed by successful superintendents and boards of education resulting in increased student achievement

PDF

The role of the superintendent in ensuring school board focus on student achievement

PDF

A study of California public school district superintendents and their implementation of 21st century skills

PDF

Resource allocation to improve student achievement

PDF

The role of superintendents as instructional leaders: facilitating student achievement among ESL/EL learners through school-site professional development

PDF

An examination of traditional versus non-traditional superintendents and the strategies they employ to improve student achievement

PDF

Female superintendents in California and the role that mentoring and networking have played in their sucess

PDF

A case study of promising leadership practices employed by principals of Knowledge Is Power Program Los Angeles (KIPP LA) charter school to improve student achievement

PDF

A study of California public school district superintendents and their implementation of 21st century skills

PDF

The superintendent and reform: a case study of action by the system leader to improve student achievement in a large urban school district

PDF

Leadership traits and practices supporting position longevity for urban school superintendents: a case study

PDF

Successful communication strategies used by urban school district superintendents to build consensus in raising student achievement

PDF

Effective strategies urban superintendents utilize that improve the academic achievement for African American males

Asset Metadata

Creator Bagley, Rick Edward (author)

Core Title The sustainability of superintendent-led reforms to improve student achievement

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 10/04/2012

Defense Date 09/18/2012

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag improving student achievement,K-12,OAI-PMH Harvest,reform,superintendent,superintendent longevity,superintendent tenure,sustainability

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor García, Pedro Enrique (committee chair), Castruita, Rudy Max (committee member), Matthews, Michael (committee member)

Creator Email bagwiz@gmail.com,rbagley@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-101430

Unique identifier UC11288190

Identifier usctheses-c3-101430 (legacy record id)

Legacy Identifier etd-BagleyRick-1228.pdf

Dmrecord 101430

Document Type Dissertation

Rights Bagley, Rick Edward

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA