University of Southern California Dissertations and Theses

Charter schools, data use, and the 21st century: how charter schools use data to inform instruction that prepares students for the 21st century

(USC Thesis Other)

Charter schools, data use, and the 21st century: how charter schools use data to inform instruction that prepares students for the 21st century

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

CHARTER SCHOOLS, DATA USE, AND THE 21ST CENTURY:
HOW CHARTER SCHOOLS USE DATA
TO INFORM INSTRUCTION THAT PREPARES STUDENTS
FOR THE 21ST CENTURY

by

Lindsay Faye Butler

__________________________________________________________________

A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

May 2009

Copyright 2009 Lindsay Faye Butler
ii
Dedication
I dedicate this dissertation to my parents, Randy and Nancy Butler, and to my
grandmother, Ida Mae Monroe. Their encouragement and unwavering belief in me has
inspired my achievement and guided my success. I am lucky to have such wonderful
people who believe in me. Thank you.
iii
Acknowledgements
This dissertation would not have been possible without the guidance and
inspiration of Dr. Amanda Datnow. I was blessed to have such an amazing and patient
resource to guide me through this process. I also owe much gratitude to committee
members Dr. Pricilla Wohlstetter and Dr. Guilbert Hentschke, whose critical evaluation
of my research helped bring this topic to life for me.
I would also like to thank the schools that participated in this study for allowing
me to visit their schools and conduct my research. The enthusiasm and knowledge of
staff at each site was inspiring, and has already affected my own work.
iv
Table of Contents
Dedication ii
Acknowledgements iii
Table of Contents iv
Abstract vi
Chapter 1: Overview of Study 1
Standards, Accountability, and the 21st Century 3
Charter Schools 5
Research Questions 7
Significance of the study 8
Chapter 2: Literature Review 9
21st Century Education 10
Table 1: Summary of Definitions for 21st Century Skills 12
Charter Schools 12
Data Use in Innovation 16
Rationale for Data Use in Program Development 17
Organizational Strategies for Building Capacity 18
Figure 1. Cultures and Cycles of Inquiry 21
Supporting Effective Data Use at the School Level 25
Promising Practices and Programs 27
Programme for International Student Assessment (PISA) 28
Central Educational Center (CEC) 29
Center for Advanced Research and Technology (CART) 31
Achieve, Inc. and The American Diploma Project 32
Summary of Literature Review 33
Chapter 3: Methodology 36
Sample and Population 37
Sunny Bay Charter High School 38
City View Charter High School 39
Data Collection Procedures 40
Data Analysis Procedures 41
Ethical Considerations 42
Limitations of the Study 43
Researcher’s Subjectivity 43
Chapter 4: Data Analysis and Interpretation of Findings 45
Introduction 45
Participant Overview 46
21st Century Preparedness 47
Curricular Planning and Instruction 50
v
Sunny Bay Charter High School 51
SBCHS Design Principles 52
City View Charter High School 55
CVCHS Common Strategies 56
Comparisons between the Schools 59
Teacher Capacity for using Data and Research 59
Administrative Use of Data and Research 60
Data and Research in Professional Development 66
Student Outcomes 69
Sunny Bay Charter High School 70
City View Charter High School 71
Conclusion 74
Chapter 5: Summary of Implications and Findings 77
Introduction 77
Connections to Prior Research 79
21st Century Education 80
Charter Schools 81
Data Use in Innovation 83
Promising Practices and Programs 85
Implications for Future Research 87
Implications for Policy and Practice 88
Conclusion 91
References 92
Appendices 100
Appendix A: Interview Protocol – Teachers 100
Appendix B: Interview Protocol – Site Administration 102
Appendix C: Interview Protocol – CMO Staff 104
Appendix D: List of Codes 106

vi
Abstract
The significant advancement in technology, along with the continuing
globalization of business and industry, has raised calls to change the way American
students are being educated in order to better prepare them to excel in this constantly
evolving 21st century environment. One method of creating change and informing
direction is the practice of using data to inform decisions. Using this approach to
innovate instruction for the 21st century requires thoughtful and conscientious
leadership. In recent years, charter schools have been credited with the potential to
innovate practices across schools more successfully that traditional school districts. This
study investigated the ways in which data use is driving 21st century instructional
innovation in charter schools. To explore these issues, qualitative case studies were
conducted in two charter schools. Interviews with administrators as well as teachers and
department chairs were conducted, and complementary document analysis was
completed.
Findings show that 21st century skills are part of the instructional program at
each school, although to differing degrees. At each site, the use of data plays an
important role in the planning of instruction and curricular development. Administrators
at both schools use data as the basis for making decisions and provide many
opportunities for teachers to use (and learn to use) data during professional development
time. In regard to student preparation for the 21st century, strategies to cultivate the
appropriate skills in students are a part of the curriculum at each campus, although

vii
neither school specifically includes those strategies for the purpose of 21st century
education. Implications for policy and practice are discussed.
1

Chapter 1: Overview of Study
In the past decade, the United States has undergone a well-documented shift
toward Standards-Based Education. Federal accountability measures, namely the No
Child Left Behind (NCLB) Act of 2001, have effectively ensured that test-based
accountability systems for schools be put in place nationwide. Since this shift began,
efforts to raise student achievement and meet these standards have become a
widespread phenomenon. National, state, and local agencies were attempting to
implement programs to affect student achievement from preschool to graduation.
Educational research institutions have generated countless studies surrounding the
connection between standards and improvement in education to both facilitate and
support this process (Chatterji, 2002; McGuinn, 2006).
While raising student achievement was the explicit focus of this movement,
postsecondary institutions have found that students remain largely unprepared for the
rigors of higher education (Sanoff, 2006; Hayasaki, 2005; Levinson, 2001). Students
who meet achievement guidelines for their state in terms of content standards are often
failing placement exams, and the need for remediation at the college level is significant.
According to statistics released on their website (California State University, in press)
43 percent of freshmen entering the California State University system in 2006 were in
need of remediation in English, while 37 percent required remediation in Math. These
numbers indicate that, despite efforts to raise standards and expectations, continued
work at the secondary level to adequately prepare students for college is necessary.
2
Concern for the preparation of students by our public education system extends
beyond simply preparing them to enter college. Research confirms the long held belief
that extended academic preparation helps ensure personal success (Day & Newburger,
2002). The Commission on the Future of Higher Education further supports the need for
continued development of nationwide educational improvement, “We acknowledge that
not everyone needs to go to college. But everyone needs a postsecondary education.
Indeed we have seen ample evidence that access to postsecondary education and
training is increasingly vital to an individual's economic security” (United States
Department of Education, 2006, p. X). As global competition continues to increase in
many facets of our modern lives, so does the need to better prepare our students to
compete in this environment, despite their destination after high school.
This evolution toward a technology-driven, global work environment over the
past two decades has influenced business and school leaders to push for the
development of a target skill set in students; a new component of education deemed
necessary for success in the 21st century. At every level of the educational arena, this
paradigm shift has created a need to reassess and realign expectations and instructional
strategies to facilitate change in this direction (Achieve Inc., 2008). A call for
educational standards and practices that push schools to better prepare students for life
beyond high school in the 21st century has come from postsecondary institutions, as
well as employers.
We need to be fully prepared for a world of ever-increasing competition. In such
a rapidly-changing environment, the development of literacy, numeracy and
information-technology skills is not enough. We need to train people to do more
3
than that – to solve problems, work with others and take charge of their own
learning and development. That is why all the key skills are essential
(Roberts, 1998, p. 1)

Promising practices of all varieties and across content areas are being explored and
examined widely, and defense for existing methods seems to be taking place at every
level (e.g., Datnow, Borman, & Stringfield, 2000; Strickland & Alvermann, 2004;
Evans, 2005). However, critiques for many of these programs, and lack of decisive
results, create confusion for system leaders attempting to develop the appropriate
programs for students. The push for programs that are data-driven and research-based
has begun, in part because of NCLB, but the body of knowledge is rife with untested
hypotheses and opinionated solicitations (Suarez-Orozco, 2005; Stewart & Kagan,
2005).
As the push for innovation has converged with standards-based accountability
systems, it has created a challenge that reaches to the very root of educational purpose
(Schoen & Fusarelli, 2008; Achieve, Inc., 2008; Diamond, 2007; Suarez-Orozco, 2005).
On one hand, educators understand the need to develop a program that prepares students
for real-world success. On the other hand, meeting state and federal expectations for
student achievement weighs heavily on instructional decision-making.
Standards, Accountability, and the 21st Century
The importance of a quality instructional approach has grown significantly with
the advent of standardized test accountability. As the use of standards and more
stringent graduation requirements have become a national focus, instructional programs
that help school systems support these goals have also become available in increasing
4
quantities (Achieve, Inc., 2008). Results from these exams are scrutinized at many
levels, and schools are required to continue demonstrating growth in student
achievement across years and grade levels. Data generated from these exams has
encouraged the growth of “data driven decision making” as an instructional planning
practice at a higher rate than ever before (Supovitz & Klein, 2003).
A paradox exists between developing innovative instructional practices and the
framework of standardized accountability systems. The balance between preparing
students to perform well on standardized tests, as well as for the rigors of college and
the 21st century, has become an innovator’s conundrum. “The ever-present threat of
failing to make AYP, with its public embarrassment, stigma, and outcomes-or-else
philosophy, produces fear and conformity among educators—both of which stand in
stark contrast to the objectives of 21st-century schools” (Schoen & Fusarelli, 2008, p.
182). While the current landscape in education is focused on measures of student
achievement and school improvement, little has been done to acknowledge the learning
needs of students.
NCLB focuses on teacher quality and testing the traditional curriculum of the
20th century, without regard for new competencies required for 21st- century
schools. In contrast, the 21st-century skills movement seeks to fundamentally
restructure classroom learning experiences through student exposure to more
authentic activities such as collaborative interdisciplinary problem solving and
places less emphasis on teaching and testing subject area skills in isolation
(Schoen & Fusarelli, 2008, p. 183).

Despite the gap between the values of standards-based programs and concept of
innovation, progress toward a marriage between the two is requisite for schools and
students. Schoen and Fusarelli (2008) believe “It’s patently absurd to think that 21st-
5
century skills cannot be accurately measured and tested. Doing so, however, requires
some fundamental changes in our approach to accomplish this; more of the same will
not be sufficient (2008, p. 196). Synthesizing the elements described here to create a
whole new version of schooling is both a risky and important challenge. It is necessary
to look closely at how schools, and other organizations, are working to support this
progress.
Charter Schools
The charter school movement began in the early 1990s as an effort to depart
from traditional models of schooling, with a majority of charter schools opening with
the intention of developers to reform education and increase autonomy (Smith,
Wohlstetter & Brewer, 2007; Fusarelli, 1999). Considered to be a type of privatization
in education, charter school educators argue that they are publicly funded, and thus,
public schools (Hentschke & Wohlstetter, 2007). While a growing body of research
points toward charter schools as a successful undertaking, this finding is not universal
(Viadero, 2008, Garrison & Holifield, 2005). Among the challenges revealed in
research is the question of operational format. Smith et al. (2007) explain that
operational issues often lead to charter failure, and emphasize the importance of an
effective organizational structure. Research on effective practices for charter school
management is only just beginning (Cech, 2008).
Charter Management Organizations (CMOs) began to appear shortly after the
charter movement itself began. One of the first CMOs, Aspire Public Schools, was
formed in 1997, and received a $2.3 million dollar grant from the Bill and Melinda
6
Gates Foundation in 2000 to open five new schools (Aspire, in press; NCSRP, 2007).
The Gates grant was part $36 million to five organizations. “Over the last several years,
the number and reach of CMOs has increased dramatically, leading to a diverse array of
organizational models and school designs. In the 2007-2008 school year, there were at
least 33 CMOs operating more than 189 schools that enrolled over 57,000 students”
(Mathematica, in press). The Gates Foundation has continued their support of charter
schools, as have other philanthropic organizations. Between 2000 and 2007, the Broad
Foundation alone contributed $90 million toward charter school development
nationwide (Broad, 2008).
Traditional school districts and schools have been found to struggle with change
simply because of the context in which they exist (Schoen & Fusarelli, 2008; Arnold,
2007; Diamond, 2007). Charter schools, specifically those run by Charter Management
Organizations (CMOs), have emerged as a hopeful new format for school systems. The
New School Venture Fund (2007) defines CMOs as:
a fully integrated regional network of charter schools under a single non-profit
governance structure that typically shares a common school design and a
coherent set of scalable management systems. At their core, CMOs are designed
to enable and accelerate charter growth with consistent high quality. By
centralizing and sharing certain functions and resources across schools, CMOs
aim for greater efficiency and long-term sustainability (p. 2).

As an instrument of school reform, CMOs have been recognized for their potential to
support innovation across schools (New School Venture Fund, 2006). CMOs are
thought to possess the capability to build programs that carry broader impact than
solitary charter schools. “Because of its unique management structure – which itself
builds upon the unique structural elements of charter status – we believe the CMO
7
represents a powerful opportunity to implement an aligned instructional approach”
(NSVF, 2007, p.3). While the charter movement is nearing the end of its second decade
in existence, little information about promising practices in governance have been
revealed, and a sharing or resources is just now beginning to be facilitated by outside
organizations (Smith et al., 2007).
Research Questions
This qualitative study explored how charter schools organize high schools to
prepare students with those skills considered essential for college and the 21st century,
while continuing the work of meeting standardized accountability measures.
Specifically,
this study examined how curriculum and instruction are informed, and how the use of
data drives this development. The study focused on two charter schools that have
experienced some success in developing a program that includes “21st century” skills,
which are defined here as: initiative, creativity, flexibility, critical thinking, problem
solving, communication and collaboration in a constantly changing environment.
The overarching question this study explored is: How do charter schools use research
and data to inform curriculum and instruction in high schools to prepare students for
success in the 21st century?
Sub-questions to further address this issue were:
1. To what extent are instructional strategies and programs being used to prepare
students for the 21st century?
2. What types of evidence are being used to inform curriculum and instruction?
8
3. How are charter schools building the capacity of teachers to use research-based
instructional strategies and to be engaged in data-informed instruction?
4. According to teachers, what impact have these practices had on student preparedness
for the 21st century thus far?
Significance of the study
While the need for innovation and improvement in education is well known,
processes for going about reforming a program while continuing to meet state and
federal accountability measures has not been explored with any depth. This study
sought to add to the existing body of knowledge regarding instructional strategies and
programs that both improve student achievement on standardized measures and support
21st century preparation. By focusing on developing programs, as well as the
monitoring elements being used by the school system, this study illustrated ways in
which data driven decision-making can support innovation and improve the methods
being used to prepare students for the 21st century.
9
Chapter 2: Literature Review
Education is under pressure from many angles. Standardized testing and
accountability measures under NCLB blend with calls from industry and higher
education to better prepare students for the demands of the 21st century, creating an
environment where experimentation is both risky and required (Schoen and Fusarelli,
2008; Diamond, 2007; Conley, 2007; Wallis & Steptoe, 2006; Suarez-Orozco, 2005).
While charter schools themselves continue to develop, confidence in their capacity to
lead innovation in this area grows as well (Lake, 2007; Renzulli & Roscigno, 2007;
Raymond, 2003). How charter schools manage these demands, what tools they use to
develop and improve their programs, and how these efforts have begun to inform the
evolution into 21st century schooling carries deep significance for the educational
community and the future of teaching and learning. To what degree these forces
converge in the school environment, and actions school leaders take to manipulate
curriculum and instruction in order to best support student-learning needs was the
specific focus of this study.
The review that follows explores the existing literature within these contexts,
beginning the emerging principles of 21st Century education. The second section
provides an overview of charter schools, followed by a survey of research on
implementing data use for curricular development. Finally, this review will conclude
with a look at promising 21st Century educational programs.
The objective of this chapter is to provide clear perspective on the current
knowledge about 21st century education, charter schools, and data use in curriculum
10
development. This literature review informed the direction of the study by exposing
existing insight as well as illuminating areas for further exploration.
21st Century Education
Business and technology sectors have begun to demand that education prepare
students to meet the needs of the 21st century world (Gates, 2005). To develop an
understanding of what it means to be prepared for the 21st century requires education to
take a look at characteristics of the ideal candidate. In “Rethinking Education in the
Global Era, Suarez-Orozco (2005) explains what it means to be prepared for the 21st
century, focusing of the qualities and aptitudes required in this emerging environment.
The skills and competencies needed for identifying, analyzing, and mobilizing to
solve problems from multiple perspectives will require individuals who are:
intellectually curious and cognitively flexible, tolerant of ambiguity, able to
synthesize knowledge within and across disciplines, culturally sophisticated, and
able to work collaboratively in groups made up of individuals from diverse
backgrounds (Engaging The World section, ¶ 1).

While this description serves as a general definition of 21st century skills, critics of our
current schooling format contend that we are not currently set up to provide this type of
education, claiming that we have not designed schools to meet the needs of 21st century
students (Johnson, 2006; Gates, 2005).
Call for innovation to bring our schools, and students, into the 21st-century, “is
rooted in constructivist approaches to educational reform that increasingly have gained
acceptance internationally” (Schoen & Fusarelli, 2008, p. 183). This approach is
nothing new, but the principles remain emergent in the field of education (Gredler,
11
1997; Vygotsky, 1978). Internationally, these concepts have been infused into
instructional programs to yield positive results.
To achieve the right balance between such core knowledge and what educators
call ‘portable skills’--critical thinking, making connections between ideas and
knowing how to keep on learning--the U.S. curriculum needs to become more
like that of Singapore, Belgium and Sweden, whose students outperform
American students on math and science tests. Classes in these countries dwell on
key concepts that are taught in depth and in careful sequence, as opposed to a
succession of forgettable details so often served in U.S. classrooms (Wallis &
Steptoe, 2006, Real Knowledge in the Google Era section, ¶ 2).

These so-called “portable skills” are in high demand at the university level as well as in
the workplace. The National Research Council wrote, in 2002, that “College instructors
expect students to draw inferences, interpret results, analyze conflicting source
documents, support arguments with evidence, solve complex problems that have no
obvious answer, draw conclusions, offer explanations, conduct research, and generally
think deeply about what they are being taught” (Conley, 2007, p. 24). These qualities
and capabilities have been found to be almost fundamentally different from what is
expected of students in traditional high schools (Conley, Aspengren, Stout, & Veach,
2006). Furthermore, these skills are in high demand across the board. They are part of
the way our modern world works. Old approaches to instruction stand in contrast to the
way people are living today. In many ways, this innovation is about “closing that
yawning gap between how kids learn at school and how they do everything else”
(Wallis & Steptoe, 2006, Learning 2.0 section, ¶ 3).
There remains much work to do in order to bring American education to this
elevated level. While programs purporting to focus on preparing students for the 21st
12
century, they exist mostly in isolation (Allen, 2008). CMOs and charter schools, on the
other hand, have generated significant attention regarding the potential for broader
innovation, creating an interesting opportunity to build a 21st century skills curriculum
outside of the traditional public school setting, as explained below.
Table 1: Summary of Definitions for 21st Century Skills
Characteristics of 21st Century Skills Source
Intellectual curiosity, cognitive flexibility, tolerance of
ambiguity, ability to synthesize knowledge within and
across disciplines, cultural sophistication, and ability to
work collaboratively in groups made up of individuals
from diverse backgrounds
Suarez-Orozco (2005)
Critical thinking, making connections between ideas
and knowing how to keep on learning
Wallis & Steptoe (2006)
Draw inferences, interpreting results, ability to analyze
conflicting source documents, support arguments with
evidence, solve complex problems that have no obvious
answer, draw conclusions, offer explanations, conduct
research, and generally think deeply about what they
are being taught
Conley (2007)
Solve problems, work with others and take charge of
their own learning and development.
Roberts (1998)
Exposure to more authentic activities such as
collaborative interdisciplinary problem solving.
Schoen & Fusarelli (2008)

Charter Schools
The National Alliance for Public Charter Schools defines charters as “tuition-
free public schools created on the basis of an agreement or ‘charter’ between the school
and the community, which gives the charter school a measure of expanded freedom
relative to traditional public schools in return for a commitment to meet higher
standards of accountability” (NAPCS, in press). Charters emerged early in the 1990s as
an invigorating new possibility for schools (Lake, 2007; Renzulli & Roscigno, 2007;
13
Raymond, 2003; Kolderie, 2003). The basic philosophy behind charter schools itself is
rooted in the realm of innovation. The primary focus of the movement is “Creating
schools that are smaller, less bureaucratized, and more attuned to the needs of local
communities, parents and students,” says Dr. Lance Fusarelli, “From the theoretical
perspective of the new institutionalism, charter schools reflect an attempt to change the
fundamental roles and responsibilities of key actors within the larger framework of the
school organization, thus breaking the mold of the large urban school” (1999, p. 216).
While operational independence of charter schools is theoretically feasible, findings
suggest that the reality of charter school autonomy is much more intricate, and the
existence of constraints on a variety of levels often prevents theoretical models from
being realized in practice (Finnigan, 2007).
Even for charter schools that do not struggle with issues of local law, studies
show that management functions often consumed much of the time and energy of
operators (Griffin & Wohlstetter, 2001). CMOs emerged in part because of the
incredible effort required to open and run individual charter schools (Deal & Hentschke,
2004; Griffin & Wohlstetter, 2001; NSVF, 2006). As with Aspire Public Schools,
singular effort is not the point of the charter movement, and CMOs offer an opportunity
to create more widespread change (NCSRP, 2007).
The creation of such systems has ramped up over the last few years in response
to the need for increased scale, sustainability (financial as well as
organizational), and consistent quality among charter schools. In addition to
providing much-needed educational opportunities for low-income and other
traditionally underserved students, charter school systems may also demonstrate
effective alternative approaches to running a system of public schools (NSVF,
2006, p. 2).

14
The potential of the charter movement to influence the ways in which students are
taught, and the format of schooling itself, has been widely recognized as a driving force
in the exponential growth of charters since their inception (Lake, 2008; Lake, 2007;
Renzulli & Roscigno, 2007; Raymond, 2003). Actual research to back these claims,
however, is scarce. While studies exist that show some academic advantage for charter
school students over their peers (e.g. Lake, 2008; Lake, 2007; Greene, Forster &
Winters, 2003; Smith, 2003; RAND, 2003; Raymond, 2003), little information about
the differences in curriculum are revealed.
Similar to the schools themselves, the capacity of charter school management to
support 21st century instructional innovation has yet to be examined in great detail.
Early findings in a study by the National Resource Center on Charter School Finance
and Governance indicate that, for the most part, charter school management
organizations remain fairly traditional in the way they run schools, while only pockets
of innovation exist (Smith et al., 2007, p.18). While these findings may not directly
speak to instructional innovation, it can be inferred that traditional policies and
procedures were found in this arena as well. Research in this area does not yet exist.
Beginning in the fall of 2008, a comprehensive, longitudinal study regarding the
efficacy of CMOs will get underway, sponsored by the Bill and Melinda Gates
Foundation and New School Venture Fund. Mathematica Policy Research, Inc., and the
Center on Reinventing Public Education at the University of Washington will work as
lead research partners, examining the impact that CMOs have on student achievement,
and the structures, policies, and practices that influence outcomes (Bill and Melinda
15
Gates Foundation, 2008). Criticism for the study’s design, as well as its delayed timing,
has already begun. Specifically, critics wonder why CMOs have not been studies in
depth before (Cech, 2008). New School Venture Fund leaders, who claim that they have
been collecting data since they became involved, site the need for a large enough
sample size as a reason to begin now (Cech, 2008). Thirty-three CMOs will take part in
the study.
This current lack of evidence to suggest the existence of significant or wide-spread
innovation in regard to curriculum and structure may be a result of accountability
measures they are expected to meet. While charter schools and CMOs continue to evoke
hopefulness in regard to the possibility for instructional innovation, pressure to succeed
by way of traditional means is high. “Not only are charters pressured by their statewide
accountability system, they are also pressured to produce measurable results by
proponents of the movement itself” (Crawford & Fusarelli, 2001, p. 5). As discussed in
chapter one, there is significant disconnect between what current instructional standards
expect from students and those skills deemed necessary for success in the 21st century.
While California charter law includes some direction on this through language
in the education code, there is much room for interpretation. The law states that charter
petitions must include a description of “what it means to be an "educated person" in the
21st century, and how learning best occurs. The goals identified in that program shall
include the objective of enabling pupils to become self-motivated, competent, and
lifelong learners” (California Education Code § 47605).
16
Because of the potential for charter organizations to break the mold of
traditional schools and schooling, it is a greater possibility that they could effectively
adopt
…guidelines for assessments that measure student thinking skills, especially in a
manner that integrates core subjects, could transform the curriculum and
promote more effective educational and classroom processes. Just because
current forms of testing and assessment are of poor quality does not mean that
high-quality assessments cannot be developed and implemented (Schoen &
Fusarelli, 2008, p. 199).

In order to effectively build a program that builds students’ skills and measures program
effectiveness for continued improvement over time, constant monitoring and analysis
must take place. Data are powerful tools that can be used to this process.
Data Use in Innovation
The function of data use is to assist in the analysis process, guiding stakeholders
toward the proper answers to the questions they ask (Heritage & Yeagley, 2005). In the
case of program development, for CMOs as well as other organizations, the process
begins by asking questions about what skills students need and what can be done to
address these needs (Heritage & Yeagley, 2005). This section will explore data use in
this capacity by focusing on the following areas of research:
1. Rationale for data use in program development
2. Organizational strategies for building capacity
3. Supporting effective data use at the school level
Throughout the above-mentioned sections, the focus were on elements of data driven
decision-making in the context of organizational support for schools. While school level
17
processes were discussed, the purpose of this section is to expose the framework for
building capacity and supporting the use of data at the organizational level.
Rationale for Data Use in Program Development
Program development that is effective must pull from a variety of sources to
assess its progress and inform direction. In an era when data can make or break school
perception and, in turn, school success, it is imperative that progress takes place
(Popham, 2004).
Research-based and data-based decision-making allows schools to use research
and student performance data to improve student achievement by identifying
what works. Instead of constantly reinventing the wheel, or making decisions
through a trial and error method (or worse still, making decisions in the dark – a
not uncommon education practice), school leaders who use research and engage
in data-based decision-making are able to promote more coherent and effective
system-wide reform (Fusarelli, 2008, p. 196).

Trial and error experimentation is no longer an acceptable format for adaptation;
effective and timely change is requisite in the current educational landscape (Earl &
Katz, 2002). The ability to monitor progress, adapt programs adequately, and rapidly
produce positive academic outcomes is a new expectation that has emerged as a result
of the accountability movement (Mandinach, Honey & Light, 2006). Schools and
organizations doing the work of developing new programs, with internal assessments
that measure appropriate skills and student progress, should be interested what the data
reveal and how this can inform their next step (Black & Wiliam, 1998). Furthermore,
research indicates that educators who are knowledgeable about data use are able to
identify and address needs, emphasize strengths, and create more effective improvement
plans (Earl & Katz, 2002). In order to effectively incorporate the use of data into
18
decision-making structures on a systems level, capacity building to facilitate this effort
is necessary.
Organizational Strategies for Building Capacity
In preparation for the process of using data to drive decision-making,
organizations must first focus on setting the stage to effectively use data; this enables
the leaders themselves, educators, and school staff to make informed decisions about
the needs of the school and students. Significant credit has been given to the use of data
in instructional improvement (e.g. Heritage & Yeagley, 2005; Stiggins, 2006; Earl &
Katz, 2002; Black & Wiliam, 1998), however, this sort of sizable undertaking requires
investment on behalf of the organization to develop practices that enable the effective
and sustained use of data.
Establishing a culture of data use that was relevant to instructional decision
making and continuous school improvement required leadership at all levels to
help teachers make sense of data by defining the purpose of data use, stressing
the importance of using evidence, and emphasizing improvement efforts. All the
system leaders directly tied the use of data to improving teaching and learning
(Datnow, Park & Kennedy, 2008, p. 19).

There exist a variety of barriers to effectively using data, including a lack of training,
paucity of time, perceived risk of sharing data, and resistance to change (Datnow, Park
& Wohlstetter, 2007; Mandinach et al., 2006; Knapp, Swinnerton, Copland, & Huber,
2006). Given the variety of difficulties that exist in developing a culture of data use, it is
important for organizational leadership to carefully facilitate a transition toward regular
and effective data practice (Earl & Katz, 2002). Datnow et al. (2007) and Datnow et al.
(2008) identify key strategies for developing and sustaining data driven practices at the
elementary and secondary levels. These studies determined that, at the organizational
19
level, building a foundation through setting meaningful goals, constructing a
curriculum, establishing a culture of data use, growing capacity at the school level, and
providing a user-friendly data management system are common components of
successful systems. Use of these strategies were explored here. Additional strategies
more applicable to the school level were also included in these studies; these elements
were discussed in the next section.
Critical for beginning the process of any improvement effort, the first step
organizations must take is to set appropriate and rigorous goals; this is a “precondition
for effective data-driven decision making” (Datnow et al., 2007, p. 20). This study
further explains that “for most school systems, taking the time and space to develop
specific goals geared toward their needs ended up being a pivotal aspect of using data
purposefully. “Setting up system goals enabled school leaders to grapple with and
reflect on their history, their current progress, and future plans” (Datnow et al., 2007, p.
21). The remaining strategies, then, are driven by these goals.
While goal setting must continue at the school level to be comprehensive
(Datnow et al., 2007), organizational goals inform the beginning of a curricular
development process that continues with school level goals. While the idea of mandated
curriculum can inspire significant pushback (Sharkey & Murnane, 2006; Black &
Wiliam, 1998), it is possible to empower teachers in the development of new programs,
as “the benefits appear to dramatically outweigh the disadvantages, and more and more
of these performance driven school systems are convinced that system-wide curricula
are essential to being performance-driven” (Datnow et al., 2007, p. 24). To accomplish
20
this while acknowledging the skills of teachers, one district used curricula developed by
teachers to create curriculum guides and assessments, which were constructed at the
district level with constant participation on the part of the teachers, helping to create
buy-in for the program (Datnow et al., 2008). Forming a curriculum replete with
interval assessments that generate formative achievement data further strengthens the
link between instruction and assessment, supporting the culture of data use (Sharkey &
Murnane, 2006).
Creating norms and expectations for behavior at both the school and
organizational level, as well as ensuring mutual accountability in the school to home
office relationship, are key components of building a culture of inquiry (Datnow et al.,
2007). In setting these norms and building mutual accountability, behaviors and
practices are often centrally connected to process of inquiry as a whole (Mandinach et
al., 2006). Functioning inquiry models are rooted in a cyclical, recursive process. This
cycle of inquiry, existing in various forms throughout the literature (e.g. Datnow et al.,
2007; Knapp et al., 2006; Coburn, Honig & Stein, in press; Coburn & Talbert, 2006;
Mandinach et al., 2006; Wayman & Conoly, 2006; Copland, 2003; Jandris, 2001;
Lachat, 2001), informs the implementation process for data practices across
organizations. Knapp et al. (2006), provides a relatively simple visual representation of
this connection (figure 1). Centered on the concept of culture, this representation
illustrates “five phases of activity” that characterize a cycle of inquiry. Building an
organizational culture of inquiry that moves through a cycle like the one here, with clear
21
expectations for action and behavior, and ensuring the involvement of all stakeholders
in the process, requires careful planning on the district level (Earl & Katz, 2002).
High quality professional development is widely recognized as an essential
element in successfully transitioning to a system of data use (Datnow et al., 2007;
Young,
Figure 1. Cultures and Cycles of Inquiry (Knapp et al., 2006, p. 16)

2006; Symonds, 2004; Supovitz, 2003). While the format and source of professional
development varies across organizations, focus and applicability to current work is a
common trait in successful organizations (Datnow et al., 2007). Furthermore, the design
of these plans must recognize the political structures of the organization. Fullan (1994)
stated, “Neither top-down nor bottom-up strategies for educational reform work. What
22
is required is a more sophisticated blend of the two” (p. 1). Fullan further explained that
in an increasingly complex, changing world:
…top-down strategies result in conflict and/or superficial compliance. Expecting
local units to flourish through laissez-faire decentralization leads to drift, ad
hocness and/or inertia. Combined strategies which capitalize on the center's
strengths (to provide perspective direction, incentives, networking, and
retrospective monitoring) and local capacity (to learn, create, respond to, and
feed into overall directions) are more likely to achieve greater overall coherence.
Such systems also have greater accountability because the need to obtain
political support for ideas is built-in to the patterns of interaction (p.1)”(cited
from Datnow et al., 2008, p. 11).

Mindfulness of the interactions that new structures and initiatives necessitate is an
important component in the development of successful change (Sherry & Gibson,
2005). Balancing top-down support with bottom-up innovation is a critical issue for
CMOs, who often wish to preserve the creative spirit of charter schools yet offer
systemic supports without reverting to bureaucracy.
The process for developing a culture of inquiry begins, incidentally, with the
collection and analysis of data. Sherry & Gibson (2005) found that “successful reform
initiatives begin by recognizing the evolving culture of the educational institution,
investigating its needs, gathering resources and making them available to all
stakeholders, and developing a sense of common purpose that is inherently grounded in
the institution's generally accepted learning ethic” (Introduction section, ¶ 7). Jandris
(2001) presents a similar plan for constructing a professional development plan,
centered on building capacity for data use and assessment. The connection between
reform and institutional practice creates an important framing question for school
leaders. Developing a plan to incorporate data use in reform efforts doubles the need for
23
careful planning and implementation, which is widely recognized in the literature (e.g.
Datnow et al., 2007; Young, 2006; Symonds, 2004; Supovitz, 2003, Earl & Katz, 2002),
making it essential to work toward investing all stakeholders in the process from the
start (Earl & Katz, 2002).
Further development in the area of capacity building can be found in literature
focused on the concept of Distributed Leadership (e.g. Kerr et al., 2006; Knapp et al.,
2006; Sutherland; 2004; Copland, 2003). In support of the concept of capacity building
and school buy-in, distributed leadership has been credited as a significant element to
extended support of reform success, including in the case of data driven decision
making. Formal leadership adjustments that support the innovation or reform are
recognized as a catalyst for change, sending a clear message that this is not a passing
fad (Copland, 2003). Some of the more formal roles created in the distributed leadership
model include “a rotating lead teacher instead of a principal; two co-principals;
principal/reform coordinator partnerships; and, inter-school leadership structures and
strategies. Findings suggest that these new structures are linked with developing school
culture that supports inquiry-based change” (Copland, 2003). While Copland addresses
mostly school level change, this idea is echoed in Datnow et al. (2007) at the
organizational level as well, indicating that having personnel with experience working
in a school, now dedicated to directing data management efforts across the organization,
plays a critical role in program success.
A key tool for supporting the programs and practices described in this section is
a user-friendly information management system. This important tool enables the use of
24
data in a variety of ways, organizing vast information is user-friendly formats, and is
considered integral to becoming more data driven “User-friendly information
management systems are critical scaffolds for enabling teachers to access student data.
Personnel at the system and school levels who can assist teachers in the use of these
systems are also critical” (Datnow et al., 2008).
Successful outcomes resulting from deliberate development of a culture of
inquiry have more recently been described by research (e.g. Datnow et al., 2008;
Datnow et al., 2007; Wayman & Conoly, 2006; Sutherland, 2004; Copland, 2003).
Sutherland (2004) found that the Edison Project Schools more prescriptive format for
data use, with clear roles and responsibilities for all stakeholders delivered positive
outcomes all around. Results of the reform effort were striking, teachers saw
administrators acting on the immediate needs of the students and the school based on
data, and administrators found teachers collaborating to build a stronger curriculum.
The use of data became foundational practice as a result of the reform.
The work of supporting a culture of inquiry must continue to result in long-
terms success, but preliminary results “affirm Desimone’s (2002) finding that reform
efforts are most successful when they become part of professional and social norms
through teacher participation in decision-making, the development of genuine buy-in,
and through participation in networks and collaboration” (Sutherland, 2004, p. 290).
This work at the organizational level is intended to foster successful school level data
use practices. How the strategies and tools described in this section influence school-
level practices is important in overall program development.
25
Supporting Effective Data Use at the School Level
In the previous section, organizational strategies to build capacity for data use
and encourage educational program improvement, were discussed. The first of these
strategies, goal setting, is a significant foundational element that must also be created at
the school level. School-wide goals, as well as grade-level and content-area goals,
focused on student achievement and professional development, help pave the way for
success throughout the process (Datnow et al., 2007).
While support for data use at the organizational level has been determined to be
nearly essential, so too is support at the school level. Datnow et al. (2008) found that
administrators, particularly in small school, often play this role. In some models,
however, a teacher or other staff member takes the lead on data use at the site level
(Datnow et al., 2008). Copland’s (2003) work on distributed leadership emphasizes the
power of staff involvement in new program leadership to accelerate buy-in school-wide
and accelerate progress toward realizing the new model.
The type of data being provided, and the interaction among data types, weighs
heavily on the potential to affect change through data use. Organizational and school
leaders, as well as teachers, can call upon a wide variety of data to inform decisions
(Supovitz & Klein, 2003). Among these types of data are “leading” or immediate,
formative data, and “trailing” or culminating, summative data (Datnow et al. 2007).
While trailing data informs effectiveness, the use of leading data is credited with
influencing instruction in more meaningful ways, supporting immediate and informed
change (Black & Wiliam, 1998). A key area in supporting school improvement is
26
supporting teachers in understanding and developing meaningful formative assessment
tools, which can include more formal benchmark exams and more immediate in-class
check-ups and quizzes (Datnow et al., 2008; Mandinach et al., 2006). This process often
requires significant, often differentiated professional development to become common
practice (Black & Wiliam, 1998).
Providing personnel and professional development resources to support data use
at the school level are also important steps in the capacity building process.
Additionally, organizations need to provide time for in-school collaboration, as well as
the opportunity to share information across school sites (Datnow et al., 2007;
Mandinach et al., 2006). The importance of time allowance is critical. Earl and Fullan
(2003) exposed anxieties that arise when expectations are not accompanied with support
and planning time. In addition to time for teachers to collaborate within the school,
creating opportunities for teachers to work with their colleagues from other campuses
supported effective data use (Datnow et al., 2007). One model, Design Studio, creates
the opportunity for schools to improve best practices while simultaneously sharing ideas
across the organization (Copland, 2003).
The final strategy identified in Datnow et al. (2007) and Datnow et al. (2008) is
the actual analysis data and use of results to improve student achievement. While the
previously mentioned strategies support this process, action at the school level is most
significant in bringing about meaningful change (Black & Wiliam, 1998). This involves
providing teachers with tools to analyze the data, and action plans to monitor progress
of the resulting actions. “The challenge for a school is to take action to improve student
27
learning based on what the data reveal. Schools may change goals, instruction,
curriculum, technical structures, or materials; reallocate resources; create new
programs; or focus on particular grade level and subject…” (Jandris, 2001, p. 59).
Providing the proper tools to facilitate an appropriate decision-making process that
results in these sort of changes is vital. “Engaging in critical dialogue and reflection as
part of the data discussion was essential in order for the desired outcomes to be
achieved” (Datnow et al., 2008, p. 59). Engaging students in the process is also
recommended, and all stakeholder buy-in further supports the realization of data culture
(Datnow et al., 2008).
For organizations beginning to collect data for the purpose of 21st century
innovation, little research or experiential data exists. Instead, these programs can derive
from the work of existing innovative programs to begin the development.
Promising Practices and Programs
There is no need to start from scratch when planning curricular reform for the
21st century. The push for this innovation has already begun in a variety of ways
worldwide. Programs that build student capacity to enter college and the workplace
with these skills exist, but on a very limited basis. The following outlines a few of those
programs, and the work they are doing to support and further student achievement in the
21st century. These programs were selected by conducting a survey of available
research and press.
28
Programme for International Student Assessment (PISA)
At the international level, efforts to create comparable data on student
performance led to the creation of the PISA. Developed by the Organisation For
Economic Co-Operation And Development (OCED), which is composed of 30
countries including the United States, “PISA assesses how far students near the end of
compulsory education have acquired some of the knowledge and skills that are essential
for full participation in society. In all cycles, the domains of reading, mathematical and
scientific literacy are covered not merely in terms of mastery of the school curriculum,
but in terms of important knowledge and skills needed in adult life” (OCED, 2008, p.
16)
Developed in 1997, the first PISA survey, focusing on reading literacy, was conducted
in 2000. “PISA 2000 revealed wide differences in the extent to which countries
succeeded in enabling young adults to access, manage, integrate, evaluate and reflect on
written information in order to develop their potential and further expand their
horizons” (OCED, 2006, p. 3). The second and third assessments, mathematics in 2003
and science in 2006, have indicated similar, wide differences in student abilities
(OCED,
2006). This model of assessment approaches measurement of student learning in a new
and progressive manner.
What makes PISA unique is its broad focus on literacy, rather than specific
curricular knowledge” (Lyne, 2001, p. 1). Unlike NCLB’s current approach to
assessment of learning, PISA resists the tired model of curriculum testing;
instead, it accesses whether students nearing the end of compulsory education
have the knowledge and skills needed for full participation in society. This is
29
similar to what many have referred to as 21st-century skills (Schoen & Fusarelli,
2008, p. 184).

Criticism of the PISA suggests that differences in standards and expectations for
students in these various countries are valid, though not aligned with PISA (Brockmann,
Clark, & Winch, 2008). These attempts to explain poor performance on the exams may,
or may not be, valid (OCED, 2006). What can be learned from the PISA about teaching,
learning, and assessment is truly a lesson in worldwide best practice (Schoen &
Fusarelli, 2008). Through analyzing PISA results, benchmarking more successful
countries, and working to reform our own standards, the American system could
progress toward building an educational system that builds the “…capacity of students
to extrapolate from what they have learned, and to apply their knowledge in novel
settings, and students’ capacity to analyse, reason and communicate effectively as they
pose, solve and interpret problems in a variety of situations” (OCED, 2006, p. 32).
Central Educational Center (CEC)
Based in Coweta County, Georgia, the CEC was developed by the district
superintendent as a response to feedback from local industry about the poor working
skill of his graduates. A case study conducted at the school explains that employers had
noticed that the demand for a skilled workforce was increasing, while the number of
qualified, skilled employees was decreasing. While they recognized that needs were
changing, education was not (MacAllum & Johnson, 2002).
The development of a program to address this need resulted. Russ Moore, CEO, claims
that CES:
30
…seamlessly combines academics with career and technical education … high
school with college … and education with businesses. We are a charter school
serving grades 9-12, and our “team members” (CEC jargon for “students”)
attend voluntarily from our county’s three base high schools for all or part of
every school day. We designed our curriculum around the needs and
expectations of business, using a needs assessment. As a result, we focus as
much on soft skills (work ethic) as we do on courses, offering a work ethic grade
in every class in addition to a course grade. We provide work-based learning
opportunities to hundreds of team members each year, providing 470 internships
and apprenticeships last year to 185 local businesses. And we offer high school
students the chance to take dual-enrollment classes with the local technical
college (which is located on our campus), learning alongside adults (Moore, in
press).

The CES model combines what has traditionally been considered non-college bound
vocational education with the high expectations of workforce development in the 21st
century (MacAllum & Johnson, 2002).
The alignment of the CES programs with current labor market demands is
explicitly planned. “With respect to curriculum development, teachers and central office
curriculum developers sit down with representatives from business who serve as subject
matter experts (SMEs) to identify skills, knowledge, attitudes, and behaviors needed in
the workplace. The curriculum is built around those parameters” (MacAllum &
Johnson, 2002, p.12). CES is frequented by education professional, and sponsors a
development program in order to help others duplicate the program, CES does not reach
a wide number of students, and is not site based. Their practices, however, may
ultimately serve as a model for future, more site friendly, incarnations (MacAllum &
Johnson, 2002).

31
Center for Advanced Research and Technology (CART)
Located in Clovis, CA, CART is a charter school formed in partnership between
two school districts. Serving eleventh and 12th grade students from schools in the
Clovis and Fresno Unified School Districts, CART houses students “who want to
combine their traditional academic program with opportunities for challenging project-
based learning and access to the latest technology” (The Educational Innovator, 2005).
CART students attend their home school for part of the day and are bussed to CART for
half-day laboratory classes.
Recognized in the University of Southern California's Compendium of
Promising Practices for tying student learning to real life through technology,
specifically in the areas of math and science, CART’s program is purported to help
prepare students for real-world jobs (CEG, 2006). The program offers students four
different career clusters: Professional Sciences, Global Dynamics, Advanced
Communications, and Engineering and Design. “Depending on their career interests,
students choose one of the clusters and designate a sub-focus for the entire year. For
example, a student in the engineering cluster might select biomedical engineering as a
sub-focus and use the technology lab to study the impact of poor air quality on lung
capacity” (Center on Educational Governance, 2006).
Students at CART are earning college prep credit in English, Science, Math and
Technology. Courses are designed to meet A-G requirements for the University of
California and California state university systems. Course that offer college credit are
32
also available (The Educational Innovator, 2005). Their website indicates that 85% of
CART students continue their studies at postsecondary institutions (CART, in press).
Achieve, Inc. and The American Diploma Project
Created in 1996 by the nation's governors and business leaders, Achieve, Inc. “is a bi-
partisan, non-profit organization that helps states raise academic standards, improve
assessments and strengthen accountability to prepare all young people for
postsecondary education, work and citizenship, focuses its work specifically on helping
states address this issue and align instructional standards with those required by
postsecondary institutions” (Achieve, 2008). In 2005, Achieve, Inc. launched “The
American Diploma Project Network”, which included educational leaders from 32
states who agreed to work toward upholding a four-part policy that aims at closing the
expectations gap. This policy includes aligning high school standards with college and
career requirements, requiring completion of a college and career ready curriculum to
earn a diploma, building college and career ready assessment measures statewide, and
holding high schools and postsecondary institutions accountable for student preparation
(Achieve, 2008). At time of publication for their 2008 study, 19 states reported having
aligned standards. Achieve, Inc. formally verified 12 of these. The effect of this
program has created progress in the effort to bring students and schooling into the 21st
century. For example:
To earn a high school diploma in Rhode Island, all students must demonstrate
proficiency in applied learning skills — critical thinking, problem solving,
research, communication, decision-making, interpreting information, analytic
reasoning, and personal or social responsibility — in all six core content areas.
Students can demonstrate applied learning through portfolios, exhibition or
33
capstone projects or performances, end-of-course assessments, or certificate of
Initial mastery (Achieve, Inc, 2008, p.10).

Achieve, Inc. continues to support the work of aligning state standards for student
achievement with universities and careers. Acknowledgement that state policy changes
without accountability and assessment for students and schools creates no change is
evident in their work. Future work, and a forthcoming report with recommendations for
building “next-generation assessment and accountability systems” is a clear component
of their continued work (Achieve, Inc., 2008).
While cases study research on what works for students in 21st century schools
has not yet been made available, some indications of programmatic success based on
available data and observational evidence indicate that these programs are headed in the
right direction. The examples listed above, as well as others not included in this review,
can serve to inform decision-making for future program development, providing a base
of data to pull from to begin the curricular development process.
Summary of Literature Review
The unifying theme of this literature review is the trends and innovations being
exercised on and in schools and school systems. The review began with an overview of
the stated needs for students in the 21st century. Based in constructivist theories, 21st
century education focuses on students’ ability to use knowledge in a great many
contexts, and apply knowledge to real world situations (Wallis & Steptoe, 2006). While
not new, this concept challenges the thinking behind much of the established standards
students are currently being taught and the assessments they are measured by. Business
34
and education leaders have continually emphasized the need to infuse this philosophy
into the curriculum and expectations of K-12 education, but progress in this area has
moved
exceedingly slow. Little is known about what it really takes to prepare students for the
rigors of the 21st century.
The review continued with a look at Charter Management Organizations. These
new district-like organizations dwell is a similar, innovative realm, possessing the
capability to challenge the traditional structure of school systems and improve upon the
educational models that currently exist (Fusarelli, 1999). CMOs, functioning as a
unifying body for charter schools, are credited with the potential to facilitate system
wide instructional reforms, thus creating a precedent for other systems to follow. How
this is done, and with what result, has yet to be revealed in existing literature.
Data use in instructional decision making, the third section of this review,
further pushes the concept of innovation, providing a look at strategies for data use that
have been recognized as a more methodical and effective way to make decisions and
inform instruction (Coburn et al., in press). The literature indicates that creating a
culture of inquiry and supporting development of curricular programs through its use
results in positive outcomes at the school level. How data are used to create and inform
curricular development to improve student achievement, specifically curricula for the
21st century, can only be surmised from the existing literature.
Finally, this review concluded with a sampling of existing promising programs
in 21st century instruction. These programs provide important insight into the
35
development of future programs, informing the future by providing data and promising
practices for others to adopt. What influence these programs have in development of
curricula, and how they factor into the inquiry process, were considered in the following
study.
The wealth of resources and tools for schools and school districts to use in rebuilding
the curriculum and requirements for students is vast. How these elements converge, and
with what result, is the overall focus of the following study.
36
Chapter 3: Methodology
This qualitative study centered around instructional innovation for the 21st
century, and how charter schools supporting this work. Specifically, this study focused
on how innovation and accountability are integrated in charter schools, and how data
and research are used to drive this process. Two charter schools who claimed to be
developing a “21st century” instructional program at the high school level and using
data to inform decision making were studied in order to answer the research questions.
The overarching research question was: How do charter schools use data and research
to inform curriculum and instruction in high schools to prepare students for success in
the 21st century? To fully analyze this practice, the following sub-questions were
answered as well.
1. To what extent are instructional strategies and programs being used to prepare
students for the 21st century?
2. What types of evidence are being used to inform curriculum and instruction?
3. How are charter schools building the capacity of teachers to use research-based
instructional strategies and to be engaged in data-informed instruction?
4. According to teachers, what impact have these practices had on student preparedness
for the 21st century thus far?
A comparative case studies method was used for this qualitative, descriptive-
interpretive study. A descriptive approach was used to present “an account of the
phenomenon under study,” while interpretive work complemented this descriptive
component in order to “develop conceptual categories, or to illustrate, support, or
37
challenge theoretical assumptions held prior to the data gathering” (Merriam, 1998, p.
38). The case study format was appropriate for this study because the design “offers a
means of investigating complex social units consisting of multiple variables of potential
importance in understanding the phenomenon” (Merriam, 1998, p. 41). The complexity
of educational settings, and the immense possibility for variability, makes case study
research an important tool in analysis of programs. Merriam (1998) indicates that the
format is proven to be useful in studying educational innovations.
As stated previously, two charter schools were studied, creating the opportunity
to compare cases. Merriam (1998) states that “the more cases included in the case study,
and the greater the variation across the cases, the more compelling an interpretation is
likely to be” (p. 40). My decision to include two case studies allows for greater
perspective on the processes and practices supported by different organizations working
toward the goal of an innovative, standards-based academic program informed by the
use of data.
Sample and Population
This study focused on two charter schools in southern California. Both schools
have experienced some success in developing a program that includes 21st century
skills, and attribute some success to the use of data in informing program development.
Purposeful sampling was used to select organizations that meet the criteria in order to
achieve diversity amongst the students being served at the schools studied. This
heterogeneous, “maximum variation” approach to sampling, created the opportunity to
expose commonalities that inform a wider audience. Patton (2002) claims that “any
38
common patterns that emerge from great variation are of particular interest and value in
capturing the core experiences and central, shared dimensions of a setting or
phenomenon” (p. 235). While the shared dimensions provided a frame through which to
compare these two organizations and their structures, it also created an opportunity for
wider applicability.
Sunny Bay Charter High School
Opened in 2000 as a single charter school through the work of a coalition of
local industry leaders and educators, Sunny Bay Charter High School (SBCHS) is
designed to immerse students in a rigorous learning environment that engages their
interests in the fields of math, science, and engineering. The SBCHS family of schools
has grown to include five high schools, two middle schools, and one elementary school.
The central organization of these schools became incorporated as a charter management
organization (CMO) in 2004, and offices for organization leaders are found among
classrooms on the campus of SBCHS. In addition to CMO staff on campus, there is one
director and 27 teachers at the school.
The Academic Performance Index (API) score for 2007-2008 was 785.
Admission to SBCHS is granted by way of lottery, and is designed to admit students the
variety of neighborhoods that surround the school. Current enrollment is 516. At
present, the school’s population is approximately 45 percent White, 12 percent African
American, and 24 percent Latino. Other student ethnicities include Filipino, Asian, and
Pacific Islander, all making up less than 10 percent each of the student population.
Approximately 20 percent of the total enrollment qualifies for free or reduced lunch.
39
This study included 8 interviews from teachers and staff at SBCHS. I
interviewed four teachers from a variety of grade levels, one resource specialist, as well
as the school’s director. The CMO’s Chief Academic Officer and Director of Policy and
Research also participated. Selection of interview participants was made by the
recommendation from school leadership.
City View Charter High School
City View Charter High School (CVCHS) opened in 2003 as a result of calls
from the community to continue the work being done in the organization’s City View
Charter Middle School of provide a high quality charter schools in densely populated
urban communities where the traditional public schools are low performing and
overcrowded. CVCHS is run by Urban Public Schools, which became a CMO in 2004.
The organizations focus is to produce college graduates, and to uplift and revitalize the
communities they serve through the development of partnerships. The organization
currently manages one elementary school, five middle schools, and two high schools.
On campus at CVCHS, staff consists of 18 teachers, an instructional leader, and one
principal.
The Academic Performance Index (API) score for 2007-2008 was 723.
Admission to CVCHS is granted by way of lottery, and is open to students throughout
the state. Current enrollment is 310. At present, the school’s population is
approximately 95 percent Latino, the remaining students are African American, White,
and Asian. Approximately 89 percent of the total enrollment qualifies for free or
reduced lunch.
40
This study included six interviews from teachers and staff at CVCHS. I
interviewed four teachers from a variety of grade levels, one college counselor, as well
as the school’s Principal and Instructional Leader. Selection of interview participants
was made by the recommendation from the instructional leader.
Data Collection Procedures
The primary tool used in collecting data for this study was the interview. “The
purpose of qualitative interviewing is to capture how those being interviewed view the
world, to learn their terminology and judgments, and to capture the complexities of their
individual perceptions and experiences” (Patton, 2002, p. 348). A set of interview
protocols was assembled to guide this process. A semi-structured approach was taken
during the interview series. Using the research questions listed previously, an interview
guides was prepared with questions appropriate for the various stakeholders
interviewed. A combined method, fusing Patton’s (2002) “Interview Guide Approach”
with the “Standardized Open-Ended Interview” created an opportunity for consistency
from interview to interview in terms of what questions are asked, but allows some
freedom in the directions the interview takes by allowing more flexible probing
questions. Protocols for interviews at each of these levels have been included in
appendices A, B, and C.
In order to ensure validity, interviews were conducted at all levels within each
charter school. This approach will help me achieve a “’holistic understanding’ of the
situation…to construct ‘plausible explanations about the phenomena being studied’”
(Merriam, 1998, p. 204). Instructional leadership at the home office level, site
41
administration, department chairs, and teachers at each site will participate in individual
interviews lasting approximately 30 – 45 minutes. Teacher selection will occur in
collaboration with site administration and department chairs. Six to eight interviews
were conducted within each school. Notes were taken during the interview, and
reflective notes added after its conclusion. Each interview was recorded. The results of
these interviews were transcribed for analysis.
In addition to interviews, I conducted some document analysis, took a tour the
school and visited classrooms as part of the data collection procedure. The collection of
documents helped to inform interviews and confirm details discussed in interviews.
“Documents prove valuable not only because of what can be learned directly from
them, but also as stimulus for paths of inquiry that can be pursued only through direct
observation and interviewing” (Patton, 2002, p. 294). Those documents that exist at all
levels of the organization that deal with developing innovative instructional strategies
and using data to inform this curriculum were requested, including assessments, memos,
planning maps, and curriculum guides, although I found that most of what either school
was willing to share was posted on its website. By touring the site and visiting
classrooms, an orientation of the overall school set-up and operation was observed,
providing anecdotal supporting evidence to complement the interview data.
Data Analysis Procedures
The goal of this study was to investigate the developing practices in two charter
schools surrounding the development of 21 century skills curricula, and the use of data
in that process. This study examined how charter schools created these programs, and
42
how they monitor and adjust curriculum and instruction through use of data. The
interview questions were developed to help identify this process and understand these
practices to help guide this study in answering the research questions.
Interviews were taped and transcribed. This raw data from transcribed
interviews were coded, along with field notes, document contents, and classroom visit
information. I analyzed and classify this information into categories to identify
commonalities within the data, beginning initially according to the research questions I
have developed to determine what information is significant. I then prepared sub-
category codes within each of these question-based areas to further code the data. The
codes used to classify this information were: 21st Century Skills – student preparation,
21st Century Skills – teacher understanding, Curriculum – planning, Curriculum – use
of data and research, Data – capacity building, Data – use of research, Standards,
Impact on student achievement.
HyperResearch, a qualitative coding software, was used to aid in this process.
As this analysis continued, I will developed an outline for reporting the findings based
on identified themes, trends, and patterns in the data. Both summarized findings and
direct quotations were used to characterize the findings. A narrative report of these
findings is given in chapter four of this study.
Ethical Considerations
Institutional Review Board (IRB) approval was obtained through the University
of Southern California. This process serves to ensure that the study is ethical and does
not pose a risk to participants. All participants will give consent before participating in
43
interviews or observation as a part of this study. This consent was made official by way
of a form distributed to the staff, as well as verbal reminder before interviews begin.
The recording device was always be left in plain site during interview recording
sessions.
Limitations of the Study
This study was conducted over a very limited period of time, and serves as a
snapshot of current innovation and data use within two charter schools The study
gathered qualitative data about the current school and organizational setting, but cannot
reveal outcomes of the program as a whole. While these samples may not allow for
generalization across the educational field, these selections were purposefully with hope
that the data provides some relevance for future research in this area.
Researcher’s Subjectivity
My subjectivity in this area is somewhat partial. I have worked for two different
Los Angeles based charter school organizations, and currently work in the home office
of my current organization. I personally embrace the idea of innovation for schools,
though I see this as an issue of great difficulty in current educational landscape. I
believe that traditional practices in education are somewhat obsolete, and
reconsideration of the way we teach students is necessary if we are to continue being
competitive on the global level. During the study, I attempted to remain neutral, judging
the programs for weaknesses as well as strengths. My intent in choosing this particular
topic was to investigate what is being done in other places, in order to help guide
44
development of 21st century programs for a greater number of schools and
organizations, including my own.
45
Chapter 4: Data Analysis and Interpretation of Findings
Introduction
This chapter focuses on an analysis of the data collected in this qualitative study
of instructional programs that prepare students for success in the 21st century. The
purpose of this study is to examine how the skills labeled 21st century, being called for
by industry, are being translated into curriculum and instruction in two southern
California public charter high schools.
This chapter presents the data thematically in relationship to each of the research
questions. Through research, coding, and analysis of the data, several thematic strands
emerged within the framework of these questions; these strands will guide this analysis.
The overarching question guiding this study was: How do charter schools use
data and research to inform curriculum and instruction in high schools to prepare
students for success in the 21st century? To answer this question, several sub-questions
were developed.
1. To what extent are instructional strategies and programs being used to prepare
students for the 21st century?
2. What types of evidence are being used to inform curriculum and instruction?
3. How are charter schools building the capacity of teachers to use research-based
instructional strategies and to be engaged in data-informed instruction?
4. According to teachers, what impact have these practices had on student preparedness
for the 21st century thus far?
46
Several major themes emerged from this framework. Twenty-First Century
Preparedness, Curricular Planning and Instruction, Teacher Capacity for using Data and
Research, and Student Outcomes became the major thematic elements that connect to
answer the research questions. This chapter is organized around these themes to answer
each of the research questions. The first theme, 21st Century Preparedness, connects to
the overarching research question, providing details on what staff at each site
understood about what it means to be prepared for the 21st century. Curricular Planning
and Instruction provides the answer to sub-question one, while Teacher Capacity for
using Data and Research addresses sub-questions two and three. Sub-question four is
answered in the Student Outcomes section.
Participant Overview
As noted in chapter three, the schools that participated in this study are Sunny
Bay Charter High School and City View Charter High School. SBCHS was founded in
the fall of 2000, graduating their first class of students in 2004. The school describes
itself as a project-based learning environment, and the academic program is designed to
blend of technical and academic preparation, a format intended to prepare students for
success in their adult life. School leadership is very involved in the instructional
program, both the Director and Chief Academic Officer were teachers at the school
before moving into leadership roles. The school has a record of high achievement, and
boasts a college acceptance rate near 100 percent. Slightly younger than Sunny Bay,
CVCHS was founded in 2003 and has had two graduating classes. Organized around the
goal of high school graduation and success in college, the school was created as a
47
response to calls from the community to provide a rigorous learning environment at the
high school level. The school’s Director and Instructional Leader were also teachers at
the school before moving into leadership positions. CVCHS significantly outperforms
similar schools in the surrounding area, and has a college going-rate of approximately
98 percent. Because students are encouraged to take at least one college class each
semester while enrolled at CVCHS, students often enter college with an advantage over
other high school graduates.
21st Century Preparedness
Implicit in the research questions is the need to look at what teachers and administrators
understand about 21st century skills. This detail is essential in building an
understanding of why strategies are being used and what outcomes can be considered
successful at these schools. This section presents an analysis of what teachers and
administrators understood about what it means to be prepared for the 21st Century.
Each interview was asked to describe 21st century skills and what it meant to
them to be prepared for the 21st century. Reactions toward this notion ranged from
enthusiasm to skepticism, however, it was clear through the narrative provided that
instructional staff and leadership at both schools were aware of the skills students are
expected to have in order to be prepared for success in the 21st century, and had
embraced them as part of the instructional program. Commonly mentioned concepts
included meaningful group work, authentic problem solving, synthesis of knowledge,
and clear communication through a variety of mediums. After describing a skill set
48
similar to what is described above, an English teacher at SBCHS explained why she
believes they are important.
If you look at some of our big tech leaders like Google and Microsoft there is so
much obvious competition between them because they have so many talented
people working under their umbrella who are innovators and designers, and so
it’s not so much about the person who is leading the company such as it’s not
about me the teacher who leads the class, but it’s about really seeing what each
student can bring to the table.

Overwhelmingly, those interviewed recognized the importance of authentic learning,
connected to the real world in preparing students for the 21st century. Strategies like
rote memorization and task completion were perceived by most as antiquated and
disinteresting, and many interviewees described making specific efforts to avoid these
tasks whenever possible; instead, there was a genuine focus on teaching students how to
treat new information and connect it to what they have already learned in a variety on
ways. A student’s ability to integrate new information emerged as a significant
benchmark in the preparation for future success. One 12th grade teacher CVCHS said,
“students who have persistence and curiosity are more successful when they encounter
frustration because they will continue to seek avenues to be successful.” He credited
components of the 21st century skill set with cultivating these characteristics.
While it was obvious that staff at each school had a strong grasp on what students need
in terms of preparation for the 21st century, they did not often recognize them
specifically as such, and explained that we are simply adding to an already long list of
pre-existing need-to-know information. One staff member at Sunny Bay High School
said:
49
We still need quite a few skills left over from the 20th, 19th, and 18th centuries
that we certainly need in the 21st, and that is to be able to read, comprehend, and
analyze information, to be able to write with clarity and be able to express ideas
in a written format as well as a digital format. Also, to be able to verbally
express yourself and present in front of a group of people. Increasingly, that’s
how we’re expected to transmit information…

These sentiments were echoed similarly by instructors at both schools, as well as by
administrators. SBCHS’s Chief Academic Officer described his misgivings, saying
“I’m sure nothing magical happened on one day in the 21st century… sometimes I think
of this as softer skills, something about interaction skills and people skills and learning
how to learn more than just having a bunch of facts.” References to the abundance of
information available over the internet and the need for students to be able to work with
this material appropriately was also common, teaching students to develop their own
ideas based on the information they gathered appeared to be a significant focus of
classroom teachers. A science teacher at CVCHS said, “students need to know how to
use the technology that successful people are literate in. Mainly, I mean the new means
of communication through the internet, and being able to navigate that to find good
data.”
Only a few teachers mentioned technology immediately upon being asked about
skills for the 21st century, while the vast majority of those interviewed believed
technology to be a tool with which to hone the skills mentioned above, an essential
compliment to the 21st century skill set rather than the primary focus. One teacher
explained, “We believe technology is a tool. It’s not the be-all end-all, it’s just a tool
like anything else, like a pencil, to get the work done.” Much of what teachers revealed
about their understanding of what students need to be successful became evident during
50
discussions of curricular planning and instruction. While the teachers I interviewed at
both schools demonstrated a deep understanding of what it means to be prepared for the
21st century, the existence of a school-wide instructional program that embraced this
was not as prevalent at CVCHS as it was as SBCHS. Sunny Bay teachers and
administrators recognized the push to prepare students for the 21st century involved the
whole school and every teacher. At City View, many teachers have become involved,
but not everyone is at the same level. The School Counselor, speaking specifically
about use of technology, explained, “ I think our students in some areas, in some
contents, are becoming very savvy because their teachers are requiring it, but there are
still some teachers who are not at that level and so how can they even have that
expectation in their classroom.”
Overall, these schools are pointed in the same direction when it comes to
moving their students forward and preparing them for success in the 21st century. In the
next section, the way in which 21st century skills are taught will be illustrated further.
Curricular Planning and Instruction
Neither of the schools I studied had developed, or indicated there were any plans
to develop, specific curriculum for teachers to use. Instead, each school has developed
tools that they believe guide the development of quality instruction. While their
approaches appear markedly different from the surface, the intent and expected
outcomes of these programs held a great many similarities. In this section, a description
of the instructional program at each school will be presented. This discussion will be
51
focused specifically on elements of curricular design identified as essential to the way
students are prepared for the 21st century.
Sunny Bay Charter High School
SBCHS is clearly identified as a project-based learning school, both in
documentation provided on their website and in dialogue with staff. Completed student
projects can be seen along the hallways as well as in common areas, classrooms, and
offices around the school. Additional projects are displayed in each student’s digital
portfolio, accessed through the school’s website. Explaining how classroom structure is
different because of this format, one teacher said “you’ll see students working on
projects a lot more than you would in see in a traditional classroom where they are
doing something or making something… working in small groups and communicating
with each other in a more student-centered environment.”
Project-based learning is universally sited as the primary approach to
instruction, and curriculum is built by each teacher around the school’s Design
Principles, elements that help to provide structure and guidance for staff in building an
original curriculum. The theme of original design begins with the arrangement of
courses at the school, where traditional subjects and instructional order have been
reconsidered to better facilitate cross-curricular project based learning. “It’s an
integrated curriculum, we don’t try to separate out things that naturally belong together
like History and English, Math and Science. In the adult world, in the 21st century, as
we explore and grow deeper in our knowledge we find how interlinked everything
really is,” explains one administrator. To support this philosophy, a pair of teachers
52
partner to teach multiple subjects; in the 9th grade, for example one teacher focuses on
Humanities while the other is responsible for Math and Science. The content is
controlled entirely by the teaching team, and projects are often planned to incorporate
elements of each content area. A variety of supports exist to ensure the success of these
teams; this will be discussed further in a later section of this chapter.
SBCHS Design Principles
The Design Principles that guide development of curriculum and inform the
instructional approach are: Personalization, Adult World Connection, and Common
Intellectual Mission. These principals exist to inform the design of student projects as
well as day-to-day instruction. The depth to which they are embedded in everything the
school does to prepare students becomes obvious in conversations with staff. Each
person I interviewed at the school framed their conversation with me around how they
use these tools to guide the development of their instructional program.
Personalization was described to me in a wide variety of ways, each version
contributing to the overall feeling that students at SBCHS are connected to the school
and staff in a meaningful ways, and are engaged in a program that challenges them to
explore their individual interests through project work. While not specifically part of the
academic curriculum, the mentoring program appears to be a foundation upon which
this is built. Each student at SBCHS is assigned a mentor upon entering the school, who
gets to know the student and their family in an effort to ease the transition into the
SBCHS environment, one admittedly different from traditional schools. According to
the school director, this ensures that each student “is known and known well by at least
53
one adult in the building,” an element many sited as a major contributor to the success
of the school as a whole.
Personalization plays a major role in supporting students academically, as
SBCHS does not have AP/Honors or special education classes. All students in the
school take the same core classes each year without scheduling differentiation aside
from a few available elective courses. As a result, differentiation to support those who
struggle and challenge those who excel is an important aspect of this principle. Several
teachers explained to me that the project-based learning environment made
personalization possible in a way they had not seen while teaching in traditional
schools. “There’s really a lot of backup support this way,” explains one of the schools
first year teachers. “Once one teacher says a kid isn’t doing well, the other teacher is
already on it and you can get more people involved so the kid has lots of support all the
time.” Teacher teams also share groups of approximately 55 students. For those who
had taught in a traditional school before coming to SBCHS, this fact was identified
widely as significant in helping them be effective with the principle of personalization.
The second Design Principle, Adult World Connection, was most often
discussed as the component that most encourages student preparation for the 21st
century. “We try to make what we do at our school have a connection outside the
classroom,” explained a veteran teacher. She described past projects that included
displaying a historic ships AutoCAD project as an exhibit at the local maritime
museum, entering a student-designed and built submarine in a human-powered
submarine competition, and holding a public event to educate the community about key
54
issues surrounding the resent presidential election, including global climate change, the
war in Iraq, healthcare, and the economy. Examples like these were cited by every staff
member interviewed, including one who described a project where students worked
with architects to design a building for one of the organizations newest schools.
One school leader explained that they refer to the adult world as opposed to the
“real world” at the school because they do not want to imply that what is done in school
is fake. This work to connect learning to the community requires the staff to collaborate
and share resources in unusual ways, which has informed the way professional
development is run. As mentioned previously, professional development strategies will
be explored in detail later in this chapter.
The final design principle at SBCHS is Common Intellectual Mission. To define
this, the material available on the website explains that the school “makes no distinction
between “college prep” and “technical” education; the program qualifies all students for
college and success in the world of work.” The thrust of this principle reinforces the
idea of preparation for the 21st century, combining skills considered technical with the
academic to allow for well-rounded graduates. By doing that, explains the school’s
Director, “we teach our kids to be better thinkers, deeper thinkers… to approach things,
life, with a lens that is not commonly found at traditional schools.”
These design principles provide a general guideline for goal setting and
instructional planning in all facets of instruction at SBCHS, and provides an interesting
foundation upon which significant preparation for the 21st century is taking place.
55
City View Charter High School
CVCHS staff describes the school’s instructional program as one designed to
prepare students for success in college. The design of the instructional program at the
school is created by the teachers and guided by the common instructional strategies that
have been adopted by the organization as a whole. Teachers and administrators spoke of
the school’s common instructional strategies, eleven in total, that are used to guide
instructional design. These help, according to teachers, to keep their performance
expectations clear. Teachers believe that because of the common strategies, CVCHS
administration has obvious expectations for teachers in terms of what instruction should
look like and presents a clear philosophy about what they believe works best for their
students. Of the eleven strategies, those identified in interviews with staff as most
significant in influencing their instructional design included project-based learning, use
of the learning cycle, and cultivating collaborative skills. Specifics about the additional
strategies were mentioned infrequently, and some were never identified at all.
Documentation of these strategies was not made available to me, although their
existence was clear. To summarize some of the other strategies used at the school, the
instructional leader explained:
We have a criteria for do-nows or warm-ups, and a description for making every
minute count so there is no downtime in the classroom, no wasted time, we use a
tool called Mad Minutes which is to get the students into a transition or get their
attention…we have guided instruction that has 3 parts which is working with
students and helping them become independent learners, and a description for
differentiation.

56
Despite the lack of clarity in regard to all eleven instructional strategies, the influence of
project-based learning, the learning cycle, and cultivating collaborative skills weighs
heavy on the way teaching and learning takes place at CVCHS.
CVCHS Common Strategies
Projects at CVCHS include those that extend over the course of a unit or two in
a single class, or in collaboration with other content areas in the same grade level, and
focus on students working in groups or on their own to generate new learning to share
with classmates. One veteran science teacher explained that each unit in her class has an
ongoing unit project associated with it. She also collaborates with the history teacher in
the spring semester for a larger cross-curricular project. The use of project-based
learning is growing at the school, it has become more prominent over the past few years
as support has increased. The school’s Instructional Leader explained:
The current expectation is that each grade level at minimum of once a year, but
hopefully more than once a year, are doing interdisciplinary units with common
outcomes. We just spurred our 11th grade teachers into doing a whole 1920s
piece, so students will be learning what I call self-learning, researching in
different areas and then sharing it back to their classmates in a non-alcoholic
speakeasy.

The growth of this component as part of the instructional program was described by
another veteran teacher as a key element in boosting literacy skills for students. He
explains that having students do more writing in science and reading in math was part
of the reason for the push, and there has been improvement in this area over the past
several years as a result. He explained that the challenge of getting people to work
together and cross subject boundaries has been challenging, but as teachers have begun
to see the benefit, interest has grown. He said, “I’ve seen improvements in the past 4
57
years, people being more open to work together. I think it’s been encountered with
resistance in the beginning by a lot of subject area teachers because they just want to
teach their subject area, but I’ve had good experiences with interdisciplinary projects.”
The Learning Cycle is the model for instructional planning at CVCHS,
consisting of four steps: accessing prior knowledge, extending prior knowledge,
application, and reflection. This cycle guides the planning of instructional units as well
as individual lessons, and appears to inform practices at the school level as well as in
the classroom. Many of the details concerning the learning cycle deal directly with how
staff at CVCHS are trained to use data, a subject I examine later in this chapter. This
format, explains one teacher, helps ensure real learning: “They really have to
manipulate what they know and find connections between concepts, between ideas,
from one unit to the next.”
Using the learning cycle to inform curriculum and support student achievement
is something that teachers have found very valuable in improving their practices in the
classroom. A teacher new to the school explained that using the learning cycle has
helped her plan lessons that target exactly where her students are from day to day. She
explained that after teaching a lesson one day, reflection encourages revision of plans
for the following day. “I focus on student engagement (as an indicator)… if I see one
head turn, one pencil drop, one student disengage, I know I’ve got to switch gears.”
This practice, she claims, has been very effective in improving her performance in the
classroom. A veteran teacher shared similar interaction with the learning cycle:
One of the things I continue to work on is the reflection piece at the end of each
day or class period, just looking at what they are taking away from this little ball
58
of information, so each day I look at ‘did they get it’ or ‘how did they
understand’. One of the major components of my Biology class is the interactive
notebook… I look at that to see what holes I need to go back and fill.

A majority of teachers I spoke to referenced reflection as the piece of the learning cycle
they are currently focused on mastering, working to continue the cycle throughout the
lesson, unit, and year. This serves as further confirmation of the school-wide push to
incorporate the learning cycle into the practice of instructional planning.
Student’s ability to work with one another successfully was emphasized in a
majority of the interviews with staff. When I asked what I might see in classrooms to
demonstrate that students are being prepared to succeed in the 21st century, their ability
to work in groups was referenced often as a strong point of this program. “It’s a lot
more than just physically moving the desks into groups and giving them questions”
explains one teacher. “In my class, to get each task done, they need each other and they
need all the pieces so they can connect things together and finish,” she added. The
teacher’s role, explained a science teacher, is not to lead the class or tell students how to
approach the problem, but to be the guide:
When they are in groups, I watch them find their own way… it’s not a constant
me nagging or saying you really should ask someone else or let someone talk,
it’s kind of guiding the kids, like just dropping little questions and allowing
them to find their own way and to see growth over the course of the year.

As with many things, this is a constantly evolving process; teachers said that getting
students to work with other who they may not get along with requires patience and
encouragement. One teacher told me she used to hear the complaint that one student or
another was annoying, and that she encourages students to find ways to overcome these
59
obstacles, explaining with humor that “annoying people become professionals, too” so
they must figure out how to make it work.
At CVCHS, these three strategies help to provide guidance for teachers in
planning of curriculum and instruction, influencing the way students are taught and
helping to promote a program that prepares students for success in the 21st century.
Comparisons between the Schools
While each school approaches curriculum and instruction through significantly
different lenses, each brings to this study a perspective on student preparation for the
21st century valuable for consideration in the larger educational environment. SBCHS
provides loose design guidelines upon which curriculum is built, while CVCHS directs
teachers to use specific strategies. The schools serve highly different student
populations and are built on varied principles. However, their expected outcomes are
quite similar. Each school recognizes the need for students to enter college as critical
thinkers, able to collaborate and think creatively to problem solve. How this point is
reached, how data are used to inform instruction, and the way in which the process is
supported, is explored in the next section.
Teacher Capacity for using Data and Research
The two schools included in this study were excellent examples of different
ways data and research work to inform curriculum and instruction. Sunny Bay Charter
High School teachers worked with data and research often without fully recognizing
this, as administration has embedded this practice into professional development
structures, and City View Charter High School staff addressed data and research in
60
more directly, explaining how the use of specific tools informs practices both in and out
of the classroom. In this section, an analysis of how data and research are being used to
drive instruction at the school will be conducted, looking first at the administrative
approach to use of data and research, followed by an analysis of the ways they are
infused into professional development to build teacher capacity and the benefit of this
as seen by teachers.
Administrative Use of Data and Research
At each school in this study, support for the use of data and research was
apparent at the school level, as well at the organizational level. School administrators at
each site described their CMO’s support as very useful in enabling them to incorporate
data and research into professional development. For each school, a variety of data was
prepared and disaggregated into more user-friendly pieces of information at the CMO
level. Sources of data include standardized test scores such as the California Standards
Test (CST), SAT and ACT, as well as survey and observational data collected from a
variety of sources. At Sunny Bay Charter High School, the CMO employs a Director of
Policy and Research, who explained that her responsibilities regarding research and data
revolve primarily around bringing information to directors that connects with the
instructional program. She explains:
I put a lot of data together for teachers and let them ask the questions. I give it to
them in a way that is non-controversial, a way that they can understand it, and
allow them to draw inferences and ask questions and the primary thing is what
strikes you, what things do you want to improve upon, and those things come
from teachers looking at data.

61
She continued by explaining that instructional goals are not set by administration, that
the whole idea is to provoke thought and spur more intelligent planning by providing
interesting and relevant information with which to make decisions. Making it easy for
school directors to access quality research information or present achievement data for
staff consumption was described as a primary goal of her work, and the value of this
service was reflected in the way school leadership approached access to data. The
school director described how this data is used:
It’s all done for us…what I’ve had the CMO do is break (data) down for us by
teaching team… and so we have the numbers. Let’s say you’re a 9th grade
English teacher, we have three at SBCHS, we’ll have you and your colleagues
(listed) and the numbers next to you… you can make your own conclusions…
maybe it’s time to attend a little more professional development… maybe I need
to reach out to colleagues and see what they’re doing. We have these discussions
and conversations among ourselves to better ourselves.

While most teachers could not immediately describe the ways in which data and
research are included in the professional development program, they could explain how
this process had helped them identify something about their instructional program that
needed additional focus. A math teacher described the process. “The UC math test is a
placement test our students take…we do a pre and post test to gauge our math and talk
about it in math meetings. We use it on an individual teacher basis to see what we need
to focus on.”
The structures for supporting use of data at City View Charter High School are
very similar to that of Sunny Bay, but the processes are much more obvious to teachers.
The use of data and research is embedded in the schools Common Strategies and rooted
in the organization’s philosophical approach to supporting student achievement. Each of
62
the common strategies are known by staff to be research-based and data backed, and
mention of using new information to further the impact of their work was widespread in
interviews. Data, particularly state test scores and benchmark assessment data, are
disaggregated by the CMO and provided to site administrators for use in professional
development. On campus, the school’s instructional leader works to incorporate
additional data to present to teachers.
The philosophical approach to this nearly mirrors SBCHS in the way data is
presented. The Instructional Leader explains how data use works, saying “It’s really all
about how do you take the numbers and work it into statistics and work it into telling a
story or revising a story.” One teacher confirms this “We’re given the data and then
we’re more data analyzers, looking at what it means. We’ll go back and look and do
reflections upon what our units were for those, trying to figure out and analyze the data,
what seemed to work and why did kids struggle…” Decision-making and instructional
planning results from teacher findings at both schools; however, assessment of the goals
created as a result of the data findings appears to be equally important at CVCHS while
this piece is not a focus at SBCHS.
At SBCHS, the intent of providing data and discussing strategies to address
these findings is to provoke more thoughtful instructional planning. No formal goal
setting framework exists to measure success in identified areas of need; the expectation
from administration is that teachers use what they know to help students achieve. The
Chief Academic Officer described the philosophy behind this. “We have a real feeling
or organizational value, that complex instructions beget simple behaviors and simple
63
instructions beget complex behaviors, and so we have a real resistance to creating very
complicated schemes.”
Conversely, CVCHS has developed a goal setting protocol for teachers to use
when looking at data, which includes ongoing assessment and reflection; this tool is
intended to help teachers focus on achieving these goals over the course of a year.
While the tool is still evolving, the basic format is established. Each department at
CVCHS sets a goal based on what is revealed by the data, and then develops action
steps and key results to make progress over the course of the year more tangible. The
Instructional Leader described the framework:
We have a school-wide goal setting process, so in the summer we take our CST
scores and we look at what we have to have and what we do have… and then
around that we set up our angle for the year around CST… then we set up key
results and the key results are the steps leading to that angle, and then we have
our specific thing that we’re going to do to get there, action steps, so that will be
part of our professional development, to go over key results and this is what
we’ve said and these are things we’ve done, and what we need to do to get
where we want.

Departments like math and English use school benchmark exam data to inform their
goals, which have included a variety of issues ranging from vocabulary and reading
comprehension to fractions and formulas. One science teacher discussed one of the
current goals for his department: “some we made up this year were that students will get
80% proficient on 80% of their lab write ups.” Despite the lack of standardized testing
data in every content area, all departments create goals using the schools’ protocol. In
these areas, observational data and other resources are used to inform the process. For
every department, the goals are reviewed and revised as part of the schools monthly
64
professional development program and results are considered in staff evaluation each
year.
Another striking difference between the two schools is the type of data they
choose to emphasize. At SBCHS, standardized test scores were certainly part of the
conversation, but other measures of student success, both qualitative and quantitative,
were equally weighted. A few teachers explained that state standardized testing, the
CST, is not a focus at the school and that student performance data on projects, and
exams like the SAT are viewed as more relevant in informing the school’s instructional
objectives. One staff member explained:
There’s the student work, and that to me is more authentic data. Our teachers
and staff will look at student work constantly…what happens from looking at
our Presentations of Learning, for example, is that they refine their rubrics and
re-define their essential questions, they re-define how they are evaluating
students and what they are asking students to do… it’s happens all the time,
that’s something of a cultural thing, it’s very habitual, and that’s happening in
our school because we’re constantly looking at authentic student data.

Addressing this phenomenon, the Chief Academic Officer explains that the school has
traditionally drawn high performing students to the school, and that they have not been
as successful as they would like in drawing an more academically diverse population.
He understands that the good scores have been somewhat of a luxury that has afforded
SBCHS the ability to treat standards and state testing with less concern. This said, he
credits part of this success to the school culture, “There’s a culture here where they feel
important and bought into and respected, so when we say try your hardest, they don’t
just bubble in.” Some teachers posit that expectations for student achievement at the
school are very rigorous, and through preparing students to think creatively and use the
65
problem solving skills cultivated through the instructional design used at the school,
they are preparing students to be more successful on any exam.
CVCHS also uses both qualitative and quantitative data to shape the
instructional program, but the focus on preparation for success on standardized tests is
significantly more pronounced that at SBCHS. The reason for this deliberate focus,
explains one administrator, resulted from her experience as a teacher at the school.
“What I experienced is that we didn’t have any cohesion, so those of us who liked each
other would work together but there wasn’t any cohesion across the school. These goals
kind of help create that.” Through conversations with staff at CVCHS, the sense that
high CST scores are considered very valuable to the school. The focus on raising scores
was particularly evident in interviews with science teachers, whose personal efforts to
raise student achievement were discussed at length. One teacher shared, “For my last
year’s physics students, there was a clear trend that the unit on heat and
thermodynamics was the weakest strand, so this year I’m re-examining that portion of
the curriculum and finding new reading, looking for new labs, and I ordered some new
equipment.” The belief that the schools approach to data use and infusion of research
supports improved student performance on the CST was generally shared by each staff
member I spoke with.
The emphasis on data use takes two very different forms at these schools, with the
common link being use of data itself. Beginning in nearly the same place by presenting
data to staff for analysis and action, School leaders hold very different expectations for
teachers in regard to the actual process. Expectations for outcomes are general at
66
SBCHS, progress is the goal; CVCHS expects more specific achievement goals to be
reached.
Data and Research in Professional Development
Professional development is a regular feature of the daily practices at SBCHS.
Teachers meet and work together in the morning before classes four days a week. Twice
a week, the staff meets as a whole, and the other two days are devoted to small group
time, be it with the teaching teams working together or in content-alike groups. Among
the many activities conducted during whole staff time, teachers explain that they are
often looking at each other’s work to provide feedback and gather ideas for themselves.
As teachers are constantly developing new projects, presentations of teaching plans are
also regularly included in the planning time, which provides an opportunity to share
information and improve their instruction. A 12th grade Humanities teacher described
some of these activities:
We have staff meetings and they aren’t one person talking and you’re just sitting
there bored. We have staff meetings that focus on various elements. One of them
is adult team leaders where we talk about what we’re doing well in advisory,
how are we doing connecting to our lower (socioeconomic) students … our ESL
students, and what kind of resources we’re providing. Those are actually overall
things that I think as a faculty and staff we work on daily. We sit down in core
groups with our teaching partners… and we take time to pitch our project and
talk about our main concerns… we do a lot of constructive criticism colleague-
to-colleague with projects that are ongoing, we are staying fresh…

Many teachers said that looking at student work with their colleagues is an important
part of how they work to develop a quality instructional program. A new member of the
SBCHS teaching staff described the information she receives from fellow staff
67
members as among the most important pieces of becoming successful in creating
projects for her students. She said:
I meet with just the 9th grade math/physics team (at least once a week), which is
a dream because that never happens at other schools. I’m a new teacher, so I do
need the help, I say ‘this is what I’m thinking of doing, do you have ideas,
suggestions to make me more successful’ and you get to bounce ideas off people
and get help. The school is awesome about having people willing to help and
collaborate and meet with you, and the scheduling really allows that.

Without realizing the practice she was engaged in, she is using data to inform
instruction, gathering information from the experience of others to drive her decision-
making.
At CVCHS, professional development time is primarily centered around academic
planning through the use of data and research to inform practice and achieve goals. The
school holds weekly meetings, as well as ten day-long events over the course of the
school year. Teachers describe these meetings as focused almost exclusively on
reviewing and analyzing data, goal setting, and instructional planning. One teacher
explains, “We’re really preached to about the learning cycle… one of the steps is some
type of assessment and we are expected to look at whatever assessment, whether its
formal or comes from just walking around the classroom asking questions, and figure
out what it means. We do the same thing that we model in CST analysis.” Part of the
overall plan, according to the Instructional Leader, is to ensure that teachers are aware
of the data available to them and know how to use their findings to target student needs.
The school’s principal, who works primarily with operational aspects of the
school, makes every effort to keep “administrivia” out of staff meeting, communicating
with teachers primarily through memos and e-mails instead of during valuable
68
professional development time. She explained that professional development is
organized around instructional data collected during classroom observations and results
of the school’s benchmark testing program, validating what many teachers’ claim about
the focus of these meetings. She said:
In years prior, when the school just started, data did not drive our instructional
program, it was not being used readily by the administration nor by the teachers.
In the past two years what I’ve seen because of the transparency and the use of
data with staff is that it has allowed them to become more empowered and
proficient teachers.

In regard to the type of data they are looking at, teachers cited examples beyond test
scores, including observation data and survey feedback. Several teachers explained that
the focus on student engagement is a big deal to them. One History teacher explained,
“I really watch student engagement and watch what the students are engaging with and
use the strategies that they engage with the most in the room so that every moment of
every class period is filled with a really academically enriching experience.”
Student engagement data is a focus of professional development at CVCHS. The
principal described data reviewed at the most recent meeting, an overview of
classrooms observation data regarding student engagement.
Teachers did not feel good about what it looked like (during the first
observations in early October), but it was a transparency piece that needed to
happen… they have all made progress in infusing more activities to engage
students less teacher talk, and we can now celebrate that progress because we
gave them new data and we have seen progress.

The alignment between what teachers regard as their focus in the classroom and the
information being covered in professional development align at CVCHS in meaningful
69
ways, and have a clear link to data. From professional development to the use of data to
inform instruction, the process is clearly connected and explicit.
In terms of building capacity for data use, each school presented a model that
teachers have found useful in gathering data to inform curriculum and instruction. The
collaborative spirit obvious in both schools professional development programs
supports the success of this approach, as staff appear comfortable working with one
another and their administrators to review data and grow as professionals. Supports for
building teacher capacity in terms of using data and research range from simple idea
sharing among colleagues to group analysis of specific findings, and each provides a
unique benefit to teachers working to impact student achievement. How student
outcomes have been affected as a result of this work, is explored in the next section.
Student Outcomes
Staff members at each school in this study spoke proudly of the work their
students produce and the impact they perceive the instructional program has of them.
While a variety of skills they believe students have gained were identified during
interviews, teachers and administrators also mentioned a few concerns in terms of what
students are lacking. In particular, the youth of these two schools and the current
instructional programs have left little time to find hard evidence in regard to how the
program as a whole affects students, specifically in regard to preparation to the 21st
century. In this section, the perceptions of teachers and administrators in regard to
student outcomes is discussed and analyzed.
70
Sunny Bay Charter High School
Of the two schools, SBCHS has a little more knowledge of what students who
have graduated from the program have to say about their experience in college. The
Chief Academic Officer described what he has learned from those students:
Kids worry that we didn’t do enough traditional instruction; they say that we
didn’t make them read lots and write long papers, not enough math. However,
they also talk about working in groups in school and presenting work orally and
having relationships with teachers and how they’re in a class of 500 at a UC
school and they’re the only one who ever shows up to office hours because they
think they’re supposed to have a relationship with the teacher… they hang
around after class and talk to professors and get involved in research with their
professors and go out and find internships because they did them here.

This information was gathered as part of an alumni survey and used as part of staff
development early in the year to help inform instruction. Their foci, he believes, are
validated by this feedback and feels that the areas of criticism fall in line with some of
the things staff has looked at over time. In terms of 21st century skills, the emphasis on
students’ high level of comfort in collaborating and communication with others is key.
Another staff member noted that campus visitors and groups working with students on
projects often comment of the ability to communicate their ideas with clarity, agreeing
with the data they have received that indicates communication skills are a significant
area of strength for SBCHS students. In fact, communication skills were identified by
nearly all interviews as the one thing that really improves for students at SBCHS over
their four years at the school; Presentations of Learning and student internships are two
programs components that gain much of the recognition for cultivating this skill.
While the availability of data in regard to student outcomes after graduation is
very limited, teachers echoed much of what was shared by students as areas of strength,
71
adding that the availability of technology at the school was significant in affecting
student preparation for the 21st century. “You see students working on computers all
the time, using advanced software; you don’t really see that anywhere else,” said one
teacher. Their ability to apply technological skills in a variety of academic areas is
something teachers see as unique to their school and important in setting their students
apart. For now, however, findings in regard to the impact this instructional program in
the long term has had on student success remains much in the theoretical and
hypothetical.
City View Charter High School
Unlike SBCHS, City View’s instructional programs has grown and changed
dramatically in the past few years, including the placement of the current administrative
team two years ago. Much of what the school is currently doing to prepare students for
college and the 21st century is difficult to support with data as a result of the changes.
That said, teachers share a common sense of how the work they are doing is different
from the norm, and have expectations for how these practices will affect student
preparation. Current perceptions regarding the impact these practices are having on
students includes their ability to apply skills across content areas, use technology
meaningfully, and the growing focus on data use to improve personal achievement.
Each of these will be discussed below.
Stemming primarily from the schools work in developing project-based learning
opportunities for students and the focus on group work, teachers agree that students are
growing in their ability to apply skills across content areas, a skill recognized during
72
interviews as one that should serve students well in the 21st century. A complement to
this application of skills is students’ ability to use technology in a variety of ways.
Student access to technology, and the number of tech-savvy staff members was cited as
having been essential in building this skill for students. One veteran teacher, explaining
how the schools use of data has filtered to the student level, described how the students
are becoming increasingly aware of how to use testing results to improve their
performance in the future. “…there’s a point when they realize (I’m) not just giving
quizzes to hurt (their) grades… it shows me what I need to cover and re-teach but it also
shows (them) what to focus on. The balance of it’s not just a tool for teachers but a tool
for students, and (they) are able to use the data to guide their own learning.” This is an
idea the Instructional leader had described as a strategy the school beginning to work on
developing for school-wide implementation in the future.
Staff members did, however, note several areas of work. First, they said that
students needed additional skill building in communicating with adults, noting that
some students maintain a fear of speaking to adults in an academic context. The
school’s college counselor described that while some students can navigate the college
application process comfortably, many students are still afraid to pick up the phone and
ask for what they need.
If a student is going into business they need the technological and
communication skills which in this day and age sort of start to blend because so
much communication today is electronic… but there still needs to be the ability
to interface face-to face… I have students afraid to pick up the phone… because
they’re not really sure how to have that professional level conversation and
know how to express what they’re asking for; those are things we don’t always
prepare them for in the academic setting.

73
She hopes the current work in redeveloping the schools internship program will help
address this communication need.
Veteran teachers expressed a concern over the lack of textbook work, something
they understand students are often required to do in college. Teaching strategies to
overcome this, explained an AP teacher whose students are working heavily in a college
level textbook, is something he recently discovered as absolutely necessary. “A point of
contention for (a few) teachers is that we don’t have textbooks…” explaining that his
initial findings in speaking with graduated students is that working with text is all they
are doing in college, something they did not learn to do in high school.
Discussion of student outcomes for each school, while revealing little about the
actual results of the program, serves as an interesting tool to tie the questions asked in
this study back together, and demonstrate how the programs as a whole serve students
to prepare them for the 21st century. It appears that each school has made steps toward
moving students toward preparation for the 21st century by designing the way they
learn to be more collaborative and project based, and projections in regard to results are
informed by observations and information gleaned from results. Interestingly, concerns
from staff at both schools revealed the perceived need to continue instruction as it is
traditionally known, with essays and lectures and work in textbooks. Working to
identify this issue, CVCHS’s college counselor said, “I think there’s still a gap between
higher education and the skills that people need in the 21st century,” indicating that
while she believes that the work being done at the school to prepare students for the
21st century is a good thing, she worries that this is not what some may get in college,
74
depending on where they enroll. I had a brief encounter with the college counselor at
SBCHS, who described a similar concern. At both schools, considering the instructional
program of different colleges is something they see as an important element in the
college search process. This concept was not prevalent in the research data, but appears
to be an important element in the overall process of preparing students for the 21st
century.
Conclusion
This chapter presented an analysis of student preparation for the 21st century
and the use of data and research in informing that preparation in California Charter
schools. The research questions focused around the design of curriculum and
instruction, professional development, and the use of data and research in 21st century
preparation. After collecting and analyzing the data, themes emerged that were linked to
each of the research questions of this study. These themes were 21st Century
Preparedness, Curricular Planning and Instruction, Teacher Capacity for using Data and
Research, and Student Outcomes. In the analysis of each of these themes, each of the
research questions was addressed. 21st Century Preparedness addressed the
foundational concept of the overarching question, while Curricular Planning and
Instruction, Teacher Capacity for using Data and Research, and Student Outcomes
specifically addressed the sub-questions.
Awareness of the ideals embodied in the 21st century skill set was clear and
widespread. While the need to create a label for these skills was questioned by some,
acceptance of the skills themselves was ubiquitous, and teachers were able to articulate
75
how their school’s instructional program served to prepare students for the 21st century.
Each school had developed guides for the instructional program to inform instructional
design and guide the development of curriculum. Embedded in the principles and
strategies described in this chapter was a recognition by staff at each school of the
relationship to 21st century skills development these programs contained. While the
structures of these instructional programs as they exist could indicate that these schools
had designed them specifically in order to prepare students for the 21st century, there is
nothing in the interview data or documentation that indicates this is a specific focus of
either program. The attitude of most teachers and administrators toward these skills was
that they have long been important, and the emphasis on developing them for the 21st
century is a revision of things that have existed long before the turn of the century. That
said, the attitude toward a focus on these skills was overwhelmingly positive; as
evidenced by the curriculum of each school, these skills are thought to be significant in
cultivating student success.
Professional development programs at each of the schools in this study
approached the support of instructional planning and curricular design through a variety
of strategies that encouraged teachers to collaborate and analyze their classrooms
practice to improve instruction and affect student outcomes. Use of data and research
played a significant role at each site, although the way this information is used and
monitored is very different at the two schools. Regardless of the strategies used by
administration, teachers at each school felt challenged to continuously improve their
76
practice, while also feeling supported by administration and understanding the variety
of tools available to them in this process.
In regard to student outcomes, the willingness of each school to share what they
perceive to be programmatic weaknesses along with their areas of strength provides an
interesting insight into how assessment and data use had made its way into instructional
practices in authentic ways. Consciousness about the big picture outcomes in addition to
the skill specific tools they could use in their daily practice to support these outcomes
was a quality teachers shared at these schools, a supportive administrative team was
often credited as part of the reason for this success.
In summary, student preparation for the 21st century was an intrinsic part of the
curriculum at each of these schools, intertwined with the instructional program to a
degree that it is impossible to discuss the approach to student preparation for the 21st
century without discussing the program as a whole.
77
Chapter 5: Summary of Implications and Findings
Introduction
The push for students to be prepared for the 21st century, armed with tools and
skills that will enable them to adapt, collaborate, and think creatively in the global work
environment, has led to an increased emphasis on the preparation of students at the high
school level. While a wide variety of descriptive characterizations of these skills are
available, little to no guidance exists indicating which instructional strategies may
contribute to the development of skills specific to the 21st century. There is a growing
need to begin sharing instructional practices that promote student success in
demonstrating skills many consider mandatory for the 21st century.
The push to improve student preparedness for the 21st century workplace is
considerable. Business leaders and policy-makers have been joined by educational
leaders in calling for the better preparation of students in a set of skills determined as
requisite for success in the 21st century. Synthesizing the many versions of this request,
21st Century skills were defined for the purposes of this study as initiative, creativity,
flexibility, critical thinking, problem solving, communication and collaboration in a
constantly changing environment.
The use data and research in supporting the development of an instructional
program melds neatly into the 21st century skills arena. The role of data and research in
instructional decision-making has recently become more prominent in educational
practices universally, informing process and providing guidance for the continued
improvement of curricular and instructional practices. One educational environment in
78
which innovation can be readily found is that of charter schools. This study examined
how 21st century skills are being incorporated into charter school curricula, and the role
of data and research in informing curriculum and instruction. As mentioned previously,
there is a dearth of research in this area to guide implementation of instructional
practices that strengthen these skills. This study sought to begin a strand of investigation
in this area. These conditions raise questions about how this work is being done. How
do charter schools use data and research to inform curriculum and instruction in high
schools to prepare students for success in the 21st century? More specifically: To what
extent are instructional strategies and programs being used to prepare students for the
21st century? What types of evidence are being used to inform curriculum and
instruction? How are charter schools building the capacity of teachers to use research-
based instructional strategies and to be engaged in data-informed instruction?
According to teachers, what impact have these practices had on student preparedness
for the 21st century thus far? The findings of this study reveal the efforts of two charter
schools.
The method of data collection for this study was primarily through interviews
with teachers and administrators at two school sites. Document analysis and review of
interview notes were also included in this process. Sampling from charter schools that
claim to prepare students for success in the 21st century and use data and research to
develop the instructional program, this study provided some insight into the design of
curriculum and instruction that prepares students for the 21st century, the use of data
79
and research to inform its creation, and the role of professional development in
supporting data informed instructional planning.
Some of the major conclusions resulting from this study were that 21st century
skills are part of the instructional program at each school, although to differing degrees.
At each site, the use of data, and to a lesser extent the use of research, plays an
important role in the planning of instruction and curricular development. Administrators
at both schools use data as the basis for making decisions and provide many
opportunities for teachers to use (and learn to use) data during professional development
time. Approaches to using data varied significantly between the sites, but expectations
in terms of results were equally high for teacher performance. In regard to student
preparation for the 21st century, strategies to cultivate the appropriate skills in students
are a part of the curriculum at each campus, although neither school specifically
includes those strategies for the purpose of 21st century education.
In the next section, I will make connections between this study’s findings to the
literature review, summarize new findings, suggest implications for policy and make
recommendations for further research. In general, the findings of this study extend the
information presented in each area of the literature review and begin the work of
making connections between these areas, drawing together 21st century skills, charter
schools, and the use of data and research in planning curriculum and instruction.
Connections to Prior Research
The findings from this study can be connected to existing literature in a variety
of ways. Chapter two of this study examined the areas of research from which this study
80
was conceived: 21st century education, CMOs and charter schools, Data Use in
Innovation, and Promising Practices and Programs. I will briefly review the literature
presented in each of these sections and then analyze the connection to the findings of
this study.
21st Century Education
The 21st century education portion of the literature review delved into which
skills are considered essential for the 21st century. Suarez-Orozco (2005) wrote that
these include intellectual curiosity, cognitive flexibility, tolerance for ambiguity, the
ability to synthesize knowledge across disciplines, cultural sophistication, and the
ability to work collaboratively with a wide variety of people. Johnson (2006) and Gates
(2005) claim that traditional schools are not set up to prepare students this way,
suggesting that a change in structure is required to move toward 21st century education.
A significant body of research exists to indicate that traditional teaching and learning,
particularly in the United States, does not align with the needs of the modern world
(Wallis & Steptoe, 2006; Conley, Aspengren, Stout, & Veach, 2006; Schoen &
Fusarelli, 2008). Allen (2008) found that there exist innovative programs that prepare
students for the 21st century, but the exist mostly in isolation.
In this study, a majority of educators interviewed have a clear understanding of
the kinds of skills that should be required of the 21st century learner. Expectations for
student outcomes at both sites clearly included a majority of the skills indicated by
research as those suited for the 21st century environment. The interesting difference
between prior research and the findings of this study is that the schools, while working
81
toward what is being asked of them by research in terms of preparing students for the
21st century, were not explicit about this effort. The learning outcomes for students,
both those considered 21st century and others, were embedded together into the mission
of the schools. The significant difference between the existing research and the findings
of this study was primarily found in the amount of attention being paid to what the skills
are being called. While indeed engaging in the teaching of 21st century skills, the
schools in this study failed to recognize that this work was considered innovative.
Perhaps this finding is connected to the idea of a charter school in general, a concept
that, in education research, has been, itself, nearly synonymous with the idea of
innovation.
Charter Schools
Charter schools, particularly those run by Charter Management Organizations
(CMOs), are recognized in the literature as a forum for innovation in education (Lake,
One might imagine that since charter schools are created with a better sense of purpose
and the needs of their students (Fusarelli, 2001) then preparation for the 21st century
could be an inherent feature of a school’s mission. In exchange for lesser regulation
than traditional public schools, charter schools are often held to higher expectations for
student achievement (Crawford & Fusarelli, 2001). Research suggests that the potential
for change under these conditions is stronger in charter schools because of the
conditions that exist regarding performance expectations (Schoen & Fusarelli, 2008, p.
199).
82
Furthermore, schools that are part of CMOs are thought to be more efficient, and
innovation is more likely in this setting (NCSRP, 2007). However, the effort required to
operate a single charter school has been found to take up a significant amount of school
leaders effort (Griffin & Wohlstetter, 2001), potentially hindering the rate of innovation.
This study found that at least in the case of the two charter schools studied, they
do things differently than traditional public schools. For teachers who had taught in
other schools, recognition of the differences was prominent during interviews. Among
the differences described by teachers, the quality of professional development and
administrative support, expectations for student achievement, class size, and overall
organizational quality were identified as strengths. Teachers and administrators at both
schools validated what is suggested by the research, that the charter school environment
provides opportunity to build more effective programs for students.
In this study, the role of the CMO was raised in discussions regarding support
for instruction and the use of data and research. In both cases, the work done to
disaggregate data and identify relevant research at that level made it easy for school
leaders to incorporate the information into professional development. While the
governance structures of these organizations were not compared in this study,
similarities regarding their projected priorities emerged in the research. For example,
the respective curricular design principles and common instructional strategies
demonstrate a sense of alignment and coordination; this, in addition to the support
provided for data use, conveyed a sense of mission driven behavior. Thus, the findings
of this study contribute to a growing, but currently small, body of literature on the role
83
of the central CMO in providing support services to schools, particularly regarding
curricular decision making and the use of data and evidence. The role of data and
research in planning instruction is explored in more detail in the next section.
Data Use in Innovation
Use of data and research to inform innovation is widely recognized an effective
way to inform curricular planning and increase student achievement (Fusarelli, 2008;
Earl & Katz, 2002; Mandinach, Honey & Light, 2006; Black & Wiliam, 1998). A wide
variety of data and research can be used to inform decision-making (Supovitz & Klein,
2003). Knowledge regarding the use of data and research is recognized as a tool for
more effective instructional planning (Earl & Katz, 2002). To be successful in this,
prior research indicates that institutions must develop a culture around use of data and
research, both within the organization as a whole and at the school level (Datnow, Park
& Kennedy, 2008). System and school leadership can carefully facilitate the integration
of data and research into the culture of the organization (Earl & Katz, 2002; Datnow et
al., 2007; Mandinach et al., 2006). Setting appropriate goals is one of the foundational
aspects to successful development of data-driven school culture (Datnow et al., 2007;
Datnow et al., 2008). The process of inquiry is central to the creation of data driven
culture (Mandinach et al., 2006). The Distributed Leadership school model has been
identified as one that supports the change toward successful inquiry based school
culture (Copland, 2003). Meaningful professional development is required to ensure
these behaviors become common practice (Black & Wiliam, 1998).
84
This evidence collected for study revealed an existing culture of data use at both
schools. Teachers and staff at each school were able to explain strategies employed at
the school for assessing student learning that involved the collection and analysis of
data, including test scores and internal assessment information as well as more
formative methods that take place during instruction. At SBCHS, the project results
play a central role in informing instruction, and staff explains that looking at student
work is a practice they consider very beneficial. CVCHS looks at data in a variety of
ways and uses the information to construct and improve instructional plans for the year.
In both schools, external data is disaggregated within the CMO and by school
administration before begin presented to teachers. At SBCHS, data and research to
inform instruction were presented to staff as a part of the overall program, and the
process was not at all explicitly identified. While every teacher was able to explain how
data and research inform their practice, they did not associate the process used in
professional development with those terms. Some teachers seemed surprised to make a
connection between the behaviors of their organization and the formal process of data
use. Conversely, at CVCHS, explicit professional development in regard to data
analysis and the process of data driven decision-making was explicit and teachers were
highly aware of the process and understood that administration was providing them with
support to successfully integrate findings into their instructional program. The teachers
interviewed were very positive about the process and indicated that they believed it was
helping their school improve overall. Practices that support student improvement exist
85
across educational programs; the links between promising practices and programs from
the literature review and this study are discussed in the next section.
Promising Practices and Programs
Several practices have been identified as promising in terms of preparing
students for the 21st century. For example, the following characteristics have been
found in existing programs: Assessment of skills and knowledge rather than curricular
content and specific topics (OCED, 2008); vocational education as a supplement to
traditional classwork, a program which includes work-based learning opportunities via
internships (MacAllum & Johnson, 2002); a project-based learning environment to
supplement traditional classroom learning in which students choose a focused learning
cluster according to their interests (Center on Educational Governance, 2006).
Alignment of student achievement standards with those expected in college was also
identified as a promising program in preparing students for the 21st century (Achieve,
Inc., 2008).
The schools in this study exhibit some of these and not others. Neither school has the
capacity to assess students based on skills and knowledge in a way similar to the PISA
program, but the work toward using data and research to inform instruction at each site
could provide similar data over time. The current load of standardized exams for
students in the state of California may hinder adoption of this sort of assessment.
SBCHS is comparable to the vocational and project-based learning programs in
a variety of ways, but there is no distinction between “regular school” and the program,
unlike the examples described in chapter 2. Designed to combine vocational education
86
with academics, learning at SBCHS appears to be much more streamlined. CVCHS is
more traditional than SBCHS, but connections to the characteristics exist here as well.
Project based learning and internships are also a feature of the curriculum at CVCHS,
which are integrated into the instructional program at the school.
Graduation requirements for students at each school exceed those required for
admission to the University of California system. By holding students to high
expectations, these schools have been able to support student preparedness for college.
While the weaknesses of these programs were described in chapter 4, the significance of
the high expectations from graduation cannot be understated as a programmatic strength
for each of these schools.
The striking difference between the findings of this study and the practices
reviewed in chapter 2 is the integration of these strategies into traditional learning
environments. Unlike the school models described in chapter 2, both CVCHS and
SBCHS perform both functions on the same campus, drawing together the academic
and vocational. The existence of project based learning at both schools, although this
was not a criteria for selection, indicates that this strategy may be a best practice for
preparing students for the 21st century.
In sum, this study shares many connections with existing literature in a variety
of genres. By sharing so many connections, it brings together information in a new way,
creating a new branch for educational research. As a first step toward carving out a
niche in the area of how data use can inform student preparation for the 21st century,
this study begins to identify connections, creates opportunity for more questions.
87
Implications for Future Research
Meeting the needs of the 21st century world requires educators to begin
incorporating strategies that help develop the skills necessary for success. While the
requirements are well documented, little is known about how to provide this type of
preparation. This study sought to begin the work of identifying ways in which 21st
century skills are brought to students in charter schools, and the practices involving data
and research that serve to inform this practice. However, much more work must be done
to identify the impact of specific practices in the wider arena of public education. Future
research is needed to deepen understanding in the following areas:
- Evaluating the ways in which existing curriculum at a variety of schools lends
itself to preparing students for the 21st century. Extending the scope of this
study, future research is needed to take a broader look at how charter schools, as
well as traditional public schools, have approached the preparation of students
for the 21st century.
- Examining practices around use of data and research to inform skills
development at the classroom level. A deeper focus on the way application of
teacher’s data analysis impacts student learning.
- Analyzing how project based learning promotes 21st century skills. I suggest a
longitudinal study of the skills student develop as a result of this instructional
model.
88
- Evaluating student group-work models for results regarding skills development,
focused on how different approaches to group work affect student learning
across time.
- Examining the requirements places of students in college in regard to 21st
century skills, specifically looking at to the success of students who went to high
school in an innovative school environment versus those who were enrolled in a
more traditional program.
- Analyzing student outcomes in programs that claim to prepare students for
success in the 21st century.
- Examine the role of CMOs in a variety of CMOs, specifically in relation to how
they support curriculum and instruction.
- A wide variety of approaches can be taken in addressing these areas for future
research. As very little is known about these topics, both qualitative and
quantitative inquiries will be valuable.
Implications for Policy and Practice
This section will address implications for policy and practice. Recommendations
are geared toward policymakers, CMOs, school leaders, and teachers. Based on the
findings of this study, recommendations regarding 21st century skills, and the use of
data and research in schools include:
1) Policymakers:
89
A) Policymakers should acknowledge that, while 21st century skills are
important, we cannot abandon the teaching of traditional skills, specifically in
regard to literacy and writing.
B) Funding for instructional resources that help facilitate the development of
21st century skills should become more fully supported in policy structures.
Limited access to technology for all students presents a significant obstacle for
developing programs.
2) CMO Leaders:
A) CMO leaders should draw connections between 21st century skills and the
organizational activities and strategies that can be used to cultivate those skills.
Much more in the way of guidance in needed for teachers to understand how to
imbue students with the skills necessary for success in the future.
B) CMO leaders should create an organizational “Promising Practices” database,
specifically identifying strategies being used to support student success in
college and beyond. Simply outlining skills they should have is not enough to
ensure the expected outcome.
C) Strategies identified as successful should shared across schools; CMOs
should create the opportunity for campus visits and professional development
sessions to share this information throughout the year.

90
3) School Leaders:
A) School leaders should collaborate with CMO leadership to identify existing
successful strategies in their schools to develop a meaningful way to grow that
success.
B) School leaders should infuse professional development with meaningful
opportunities to collect and analyze student achievement data.
C) School leadership should provide ample professional development time for
focused data analysis and instructional planning.
4) Teachers:
A) Teachers should provide a balance of instructional activities, preparing
students for success in a variety of environments. Evidence in this study
suggests to narrow a focus could prevent success in more traditional college
environments.
B) Teachers would be wise to incorporate available tools for the use of data to
inform instruction into professional development practices at school sites. While
inclusion of these strategies appears beneficial, teachers at CVCHS appeared
much more confident and aware of how data can affect their instruction.
If these policies and practices are implemented at multiple levels, it is likely that the
teaching of 21st century skills will be infused more broadly across charter schools as
well as in traditional public schools.
91
Conclusion
Significant attention has been called to the idea of preparing students for the
21st century. As education continues to evolve, particularly in regard to the use of data
and research to inform instruction, significant opportunity exists to develop a cache of
strategies that support the development of strategies that effectively promote student
growth and preparation. Although this study provides insight into how this work is
being done at two charter schools, it does not provide information about what 21st
century instruction and the use of data and research to inform the process in education
as a whole. While the school in this study provide a model for where to begin the work
of instructional innovation for the 21st century, not enough information in known to
provide guidance for the creation of change on a larger scale. There is much more that
needs to be learned about the ways in which education needs to evolve to prepare our
students to be successful in the 21st century.
92
References
Arnold, J. (2007). Breaking new ground: Cultivating 21st Century skills in
secondary schools. Ed.D. dissertation, Indiana University, United States --
Indiana. Retrieved May 26, 2008, from Dissertations & Theses: Full Text
database. (Publication No. AAT 3299090).

Achieve, Inc. (2008). Closing the Expectations Gap 2008: An Annual State Progress
Report on the Alignment of High School Policies with the Demands of College
and Careers.

Allen, L. (2008). The Technology Implications Of A Nation at Risk. Phi Delta Kappan,
89(8), 608-610. Retrieved May 26, 2008, from Research Library Core database.
(Document ID: 1461479841).

Bill & Melinda Gates Foundation (2008) National Research Study on Charter
Management Organization Effectiveness to Be Led by Mathematica Policy
Research and Center on Reinventing Public Education Leading researchers to
assess impact of fast-growing type of school organizations on student outcomes
and identify factors and practices that drive success.
http://www.gatesfoundation.org/UnitedStates/
Education/Announcements/Announce-080317.htm

Black, P. & Wiliam, D. (1998). Inside the black box: Raising standards through
classroom assessment. Phi Delta Kappan 80(2), 139-144.

Brockmann, M., Clarke, L., & Winch, C. (2008) Can performance-related learning
outcomes have standards? London, UK: Journal of European Industrial
Training, 32(2.3) 99-113.

California State University. (2006) Proficiency Reports of Students Entering The CSU
System, http://www.asd.calstate.edu/performance/proficiency.shtml

Cech, S. J. (2008). Study to Probe Effect of Charter-Management Models. Education
Week, 27(31), 6. Retrieved May 26, 2008, from Research Library Core
database. (Document ID: 1461654051).

California Education Code § 47605 – Establishment of Charter Schools.
The Center on Advanced Research and Technology (in press)
http://www.cart.org/about/whatis.php

93
Center on Educational Governance. (2006) USC's Compendium of Promising Practices:
Innovations in Charter Schools.
http://www.usc.edu/dept/education/cegov/cs_prj_prj_ctr.html

Chatterji, M. (2002). Models and methods for examining standards-based reforms
and accountability initiatives: Have the tools of inquiry answered pressing
questions on improving schools? Review of Educational Research, 72(3), 345-
386. Retrieved May 26, 2008, from Research Library Core database.
(Document ID: 275050461).

Coburn, C. E., & Talbert, J. E. (2006). Conceptions of evidence use in school districts:
Mapping the terrain. American Journal of Education 112(4), 469-495.

Coburn, C. E., Honig, M. I., & Stein, M. K. (in press). What is the evidence on districts’
use of evidence? In J. Bransford, L., Gomez, D., Lam, N., Vye (Eds.) Research
and practice: Towards a reconciliation. Cambridge: Harvard Educational Press.

Conley, D. T., Aspengren, K., Stout, O., &Veach, D. (2006). College Board Advanced
Placement best practices course study report. Eugene, OR: Educational Policy
Improvement Center.

Conley, D. T., Barton, P. (2007). The Challenge of College Readiness. Educational
Leadership, 64(7), 23-29. Retrieved May 26, 2008, from Research Library
Core database. (Document ID: 1251646491).

Copeland, M. (2003). Leadership of inquiry: Building and sustaining capacity
for school improvement. Educational Evaluation and Policy Analysis
25(Winter), 375-395.

Consortium for Policy Research in Education (2007) Quality Counts: The Growth of
Charter School Management Organizations. National Charter School Research
Project.

Crawford, J. R., & Fusarelli, L. D. (2001). Introduction: Charter schools and the
accountability puzzle. Education and Urban Society, 33(2), 107-112.

Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with data: How high
performing districts use data to improve instruction for elementary school
students. Los Angeles, CA: Center on Educational Governance, USC Rossier
School of Education.
http://www.usc.edu/dept/education/cegov/reform_publications.html#data

94
Datnow,A. Park,V. & Kennedy, B. (2008). How Urban High Schools Use Data to
Improve Instruction. Los Angeles, CA: Center on Educational Governance, USC
Rossier School of Education.

Datnow, A., Borman, G. & Stringfield, S. (2000). School reform through a
highly specified curriculum: Implementation and effects of the core knowledge
sequence. The Elementary School Journal, 101(2), 167-191. Retrieved May 26,
2008, from Research Library Core database. (Document ID: 65139836).

Day, J.C. and Newburger, E.C. (2002). The Big Payoff: Educational Attainment
and Synthetic Estimates of Work-Life Earnings. Washington, D.C.: U.S. Census
Bureau.

Deal, T., & Hentschke, G. (2004). Adventures of Charter School Creators: Leading
from the Ground Up. Lanham, Scarecrow Education.

Diamond, J. B. (2007). Where the Rubber Meets the Road: Rethinking the
Connection Between High-Stakes Testing Policy and Classroom Instruction.
Sociology of Education, 80(4), 285-313. Retrieved May 26, 2008, from
Research Library Core database. (Document ID: 1400925181).

Earl, L., & Fullan, M. (2003). Using data in leadership for learning. Cambridge Journal
of Education, 33(3), 383-394. Retrieved November 4, 2007 from
http://education.vermont.gov/new/pdfdoc/pgm_assessment/summit/research_res
ources/using_data_leadership_learning.pdf

Earl, L., & Katz, S. (2002). Leading schools in a data rich world. In K. Leithwood, Pl.

The Educational Innovator (2005) Center for Advanced Research and Technology Puts
Students on a Fast Academic and Professional Track.
http://www.ed.gov/news/newsletters/innovator/2005/0928.html

Hallinger, G., Furman, P., Gronn, J., MacBeath, B., Mulforld & Riley, K. (Eds.), The
second international handbook of educational leadership and administration.
Dordrecht, Netherlands: Kluwer.

Education Evolving (2003) Evaluating Chartering: A Case for Assessing Separately the
Institutional Innovation.

Evans, R. (2005). Reframing the Achievement Gap. Phi Delta Kappan, 86(8), 582-
589. Retrieved May 26, 2008, from Research Library Core database.
(Document ID: 817444501).

95
Fusarelli, L.D. (1999). Reinventing urban education in Texas: Charter schools,
smaller schools, and the new institutionalism. Education and Urban Society,
31(2), 214-224. Retrieved May 26, 2008, from Education Module database.
(Document ID: 38950103).

Fusarelli, L. D. (2008). Flying (Partially) Blind: School Leaders' Use of Research In
Decision Making. Phi Delta Kappan, 89(5), 365-368. Retrieved May 26, 2008,
from Research Library Core database. (Document ID: 1427860251).

Garrison, L., Holifield, M. (2005). Are Charter Schools Effective? Planning and
Changing, 36(1/2), 90-103. Retrieved June 4, 2008, from Education Module
database. (Document ID: 877541691).

Gredler, M. E. (1997). Learning and instruction: Theory into practice (3rd ed). Upper
Saddle River, NJ: Prentice-Hall.

Greene, J., Forster, G., Winters, M. (2003) Apples to Apples: An Evaluation of Charter
Schools Serving General Student Populations. Progressive Policy Institute.

Hayasaki, E. (2005). L.A. Unified to Consider Mandatory College Track;
Plan would require all students to take courses needed for admission to state
universities :[HOME EDITION]. Los Angeles Times,p. B.1. Retrieved May 26,
2008, from Los Angeles Times database. (Document ID: 826919071).

Hentschke, G.C., & Wohlstetter, P. (2007) Conclusion: K-12 Education in a Broader
Privatization Context Educational Policy 21(1); 297-307

Heritage, M., & Yeagley. R. (2005). Data use and school improvement: Challenges and
prospects. In J.L. Herman & E. Haertel (Eds.), Uses and misuses of data for
educational accountability and improvement National Society for the Study of
Education Yearbook, Vol. 104, Issue 2, pp. 320-339 . Chicago: National Society
for the Study of Education. Distributed by Blackwell Publishing. Retrieved
April 26, 2006 from http://www.blackwell-
synergy.com/doi/pdf/10.1111/j.1744-7984.2005.00035.x

Jandris, T. P. (2002). Data-based decision-making: Essentials for principals.
Alexandria, VA: National Association of Elementary School Principals.

Johnson, Andrew P (2006). No Child Left Behind: Factory Models and Business
Paradigms. The Clearing House, 80(1), 34-36. Retrieved May 26, 2008, from
Education Module database. (Document ID: 1214903021).

96
Kagan,S.L., Stewart, V. (2005). Introduction: A New World View: Education
in a Global Era. Phi Delta Kappan, 87(3), 185-187. Retrieved May 26, 2008,
from Research Library Core database. (Document ID: 923875071).

Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006). Strategies
to promote data use for instructional improvement: Actions, outcomes, and
lessons from three urban districts. American Journal of Education 112(4), 496-
520.

Knapp, M. S., Swinnerton, J. A., Copland, M. A., et al. (2006). Data-informed
leadership in education. Seattle: Center for the Study of Teaching and Policy,
University of Washington.

Lachat, M.A. (2001) Data Driven High School Reform: The Breaking Ranks Model.
South Hampton: New Hampshire. The Center For Resource Management.

Lake, R. (2008) In the Eye of the Beholder: Charter Schools and Innovation. Journal of
School Choice 2(2), 115-127.

Lake, R. (2007) Hopes, Fears, & Reality: A balanced look at American Charter schools
in 2007. National Charter School Research Project Center on Reinventing Public
Education; Daniel J. Evans School of Public Affairs: University of Washington.

Levinson, Arlene (2001, March 18). Colleges Buff Up Freshman Remedial Classes;
Education: Math and basic writing courses blossom for students bright enough
to be admitted but deficient in some skills. Public schools are blamed :[Bulldog
Edition]. Los Angeles Times, p. A.4. Retrieved May 26, 2008, from Los
Angeles Times database. (Document ID: 69789121).

MacAllum, K., Johnson, A.B. (2002) Reconceptualizing Education as an Engine of
Economic Development: A Case Study of the Central Educational Center. Paper
presented at the Annual Conference of the Association for Career and Technical
Education (76th, Las Vegas, NV, December 12-15, 2002).

Madinach, E. B., Honey, M., & Light, D. (2006). A theoretical framework for data-
driven decision making. Paper presented at the annual meeting of the American
Educational Research Association, San Francisco, CA. Retrieved April 6, 2007
from http://cct.edc.org/admin/publications/speeches/DataFrame_AERA06.pdf

Mathematica Policy Research, Inc. (in press) Evaluating the Effectiveness of Charter
School Management Organizations http://www.mathematicampr.com/education/
cmo.asp

97
Merriam, S. B. (1998). Qualitative research and case study applications in education.
San Francisco: Jossey-Bass.

National Alliance Public Charter Schools (2006) Definition of a Charter School.
http://www.publiccharters.org/content/article/detail/633/

New School Venture Fund (2007) National Study of CMO Effectiveness - Initial
Request for Proposals.

New School Venture Fund, (2006) Charter Management Organizations:Toward Scale
with Quality. http://www.newschools.org/about/publications/charter-
management-organizations

Organisation for Economic Co-Operation and Development (2006) PISA 2006: Science
Competencies, Volume 1.

McGuinn, P. (2005). The National Schoolmarm: No Child Left Behind and the New
Educational Federalism. Publius, 35(1), 41-68,188-189. Retrieved May 26,
2008, from Social Science Module database. (Document ID: 897512491).

Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand
Oaks, CA: Sage.

Popham, J. (2004). America’s “Failing” Schools. New York: Routledge.

RAND (2003) Charter School Operations and Performance: Evidence from California.

Raymond, M. (2003). The Performance of California Charter Schools. CREDO/Hoover
Institution.

Renzulli, L., Roscigno, V. (2007). Charter Schools and the Public Good. Contexts, 6(1),
31. Retrieved May 26, 2008, from Social Science Module database. (Document
ID: 1213481711).

Roberts, J. (1998). Confederation of British Industry Qualification Framework.
Education and Training, 40(2/3) 122.

Sanoff, A.P. (2006, March). A Perception Gap Over Students' Preparation. The
Chronicle of Higher Education, 52(27), B9-B14. Retrieved May 26, 2008, from
Research Library Core database. (Document ID: 1020415191).

98
Schoen, L., Fusarelli, L. (2008). Innovation, NCLB, and the Fear Factor: The
Challenge of Leading 21st-Century Schools in an Era of Accountability. Policy,
22(1), 181. Retrieved May 26, 2008, from Education Module database.
(Document ID: 1419587741).

Scribner, J. P., Cockrell, K. S., Cockrell, D. H., & Valentine, J. W. (1999). Creating
professional communities in school through organizational learning: An
evaluation of a school improvement process. Educational Administration
Quarterly 35(1), 130–160.

Sharkey, N. S., & Murnane, R. J. (2006). Tough choices in designing a formative
assessment system. American Journal of Education 112(4), 572-588.

Sherry, L., Gibson, D. (2005). Responsive Dissemination: A Data-Driven Approach to
Change. Journal of Technology and Teacher Education, 13(1), 85-104.
Retrieved May 31, 2008, from Education Module database. (Document ID:
786235861).

Smith, J., Wohlstetter, P., Brewer, D. (2007) Under New Management: Are Charter
Schools Making the Most of New Governance Options? Published in Lake, R.
(2007) Hopes, Fears, & Reality: A balanced look at American Charter schools in
2007. National Charter School Research Project Center on Reinventing Public
Education; Daniel J. Evans School of Public Affairs: University of
Washington."

Smith, N. (2003). Catching the Wave: Lessons from California’s Charter Schools

Spellings, M. (2007) Speech presented at National Charter Schools Conference,
Albuquerque, NM.

Stiggins, R. (2006). Balanced assessment systems: Redefining excellence in assessment.
Princeton, NJ: Educational Testing Service.

Strickland, D. & Alvermann, D (2004) Bridging The Literacy Achievement Gap
Teachers College Press

Suárez-Orozco, Marcelo M (2005). Rethinking Education In the Global Era. Phi Delta
Kappan, 87(3), 209-212. Retrieved May 26, 2008, from Research Library Core
database. (Document ID: 923875101).

Supovitz, J. A. & Klein, V. (2003). Mapping a course for improved learning: How
innovative schools systematically use student performance data to guide
instruction. Philadelphia, PA: Consortium for Policy Research in Education.
Retrieved February 5, 2005, from http://www.cpre.org/Publications/AC-08.pdf
99
Sutherland, S. (2004). Creating a culture of data use for continuous improvement: A
case study of an Edison Project School. American Journal of Evaluation 25(3),
277-293. Retrieved March 2, 2007 from
http://aje.sagepub.com/cgi/reprint/25/3/277

Symonds, K. W. (2003). After the test: How schools are using data to close the
achievement gap. San Francisco: Bay Area School Reform Collaborative.

The Secretary of Education’s Commission on the Future of Higher Education (2006). A
Test Of Leadership: Charting the Future of U.S. Higher Education. U.S.
Department of Education

Viadero, D. (2008). High School Studies Eye Role of Charter Status, Teachers.
Education Week, 27(32), 8. Retrieved June 4, 2008, from Research Library
Core database. (Document ID: 1466691381).

Vygotsky, L. (1978). Interaction between Learning and Development (pp. 79-91). In
Mind in Society. (Trans. M. Cole). Cambridge, MA: Harvard University Press.

Wallis, C., Steptoe. S.. (2006, December). How to Bring Our Schools Out of the
20th Century. Time: Special Issue: The Best Photos of 2006, 168(25), 50.
Retrieved May 26, 2008, from ABI/INFORM Global database. (Document ID:
1177845641).

Wayman, J. C. & Conoly, K. (2006). Managing curriculum. Rapid implementation of a
districtwide data initiative. ERS Spectrum, 24(2), 4-8.

Whitlock,M.(in press) CEC website.
http://www.gacec.com/index.cfm?section=message

Young, V. M. (2006). Teacher's use of data: Loose coupling, agenda setting, and team
norms. American Journal of Education. 112(4), 521-548.

100
Appendices
Appendix A: Interview Protocol – Teachers
Before the interview begins, I will review the IRB permission slip, explain that the
interview is confidential, and obtain a signature from each interviewee. I will provide an
additional copy of the permission slip for their records.

I. Background/Context
Before we ask you specific questions, we like to start by asking you some general
questions about your school, its surrounding neighborhood, and the district in order to
gain a broader understanding of the context in which you work.

1. Could you tell me a little about the students and community that you serve?
How would you describe the general culture of your school/system?
2. Could you tell me a bit about the history of your school, focusing on the last five
years (e.g., particular reform initiatives, strong partnerships with external
groups, major structural changes, etc.)? (Q. 2/3)
3. How long have you been at this school/system? What is your prior experience
and training?
4. What are your duties and responsibilities, especially around curriculum design
and data use? (Q.4)
i. Probe: accessing, summarizing, interpreting, planning, monitoring

II. Innovation
Lets talk about the work your school has done to prepare students for success in the 21
st

century.
A. Goals
1. In your own words, What does it mean to be prepared for the 21
st
century?
(Q. 1)
a. Probe: for students, schools
2. In terms if instructional innovation for the 21
st
century, what skills does your
school/organization value most? (Q. 1)
3. How are these values reflected in your mission/vision? (Q. 1)
4. What does the focus on these elements stem from? (Q. 2)
a. How were they planned? How were they communicated?
5. To what extent have federal policies and state standards influenced your
performance goals for students? (Q. 3)
a. In what ways do standards based instruction and 21
st
century skills
compliment one another in the curriculum?
b. What has been difficult about incorporating these two elements?
B. Instruction
1. Describe how 21
st
century skills are being taught in the school. (Q. 2)
101
a. What type of activities are characteristic of this?
b. How frequently do these activities/strategies take place?
c. How important are they in terms of the big picture?
i. Probe: measurement, assessment, expectations, graduation
requirements
d. How would I know this exists as a classroom observer? (Q. 1)
2. What went into developing this program/curriculum? (Q. 3)
a. What support is provided by the CMO/district? (Q. 4)
i. What role do they play beyond planning?
ii. Do they support collaboration between different school sites?
I. How does this work?
b. What support is provided by site administration? (Q. 4)
i. What role do they play beyond planning?
ii. How do they inform continued development?
III. Data
A. Measurement
1. What performance goals exist? How were they/are they being established?
(Q. 1)
a. What tools were used to develop them?
i. Data? Research?
b. How frequently do you re-assess the goals? (Q. 3)
2. How is your progress in building this program/curriculum being measured?
(Q. 3)
a. How is student work used as an assessment? What about benchmarks,
etc.? (Q.3)
b. What data is collected other than student assessment results?
i. How is it being collected?
1. Probe: who handles, who checks, who’s accountable?
c. What opportunities exist for analyzing the data? (Q.4)
3. What are the expectations for teacher data use from the organization? (Q.4)
a. How are these communicated?
b. How is data use monitored?
c. How are these expectations supported?
4. In what ways has data been used to inform the program? (Q. 3)
a. What have you found so far?
b. What has changed as a result?

IIII. Impact
A. What changes have you noticed in instruction and student achievement since
this began? (Q. 4)
1. Has there been positive changes at any level?
2. What areas do you feel are lacking?
3. What work remains to be done?
Close: Is there anything else you’d like to tell me?
102
Appendix B: Interview Protocol – Site Administration
Before the interview begins, I will review the IRB permission slip, explain that the
interview is confidential, and obtain a signature from each interviewee. I will provide an
additional copy of the permission slip for their records.

I. Background/Context
Before I ask you specific questions, I’d like to start by asking you some general
questions about your school, its surrounding neighborhood, and the district in order to
gain a broader understanding of the context in which you work.

1. Could you tell me a little about the students and community that you serve?
How would you describe the general culture of your school/system?
2. Could you tell me a bit about the history of your school, focusing on the last five
years (e.g., particular reform initiatives, strong partnerships with external
groups, major structural changes, etc.)? (Q. 2/3)
3. How long have you been at this school/system? What is your prior experience
and training?
4. What are your duties and responsibilities, especially around curriculum design
and data use? (Q. 4)
j. Probe: accessing, summarizing, interpreting, planning, monitoring
II. Innovation
Lets talk about the work your school has done to prepare students for success in the 21
st

century.
C. Goals
1. In your own words, what does it mean to be prepared for the 21
st
century?
(Q. 1)
2. In terms if instructional innovation for the 21
st
century, what skills does your
school/organization value most? (Q. 1)
3. How are these values reflected in your mission/vision? (Q. 1)
4. What does the focus on these elements stem from? (Q. 2)
a. How were they planned? How were they communicated?
5. To what extent have federal policies and state standards influenced your
performance goals for students? (Q. 3)
a. In what ways do standards based instruction and 21
st
century skills
compliment one another in the curriculum?
b. What has been difficult about incorporating these two elements?
D. Instruction
1. Describe how 21
st
century skills are being taught in the school. (Q. 2)
a. What type of activities are characteristic of this?
b. How frequently do these activities/strategies take place?
c. How important are they in terms of the big picture?
103
i. Probe: measurement, assessment, expectations, graduation
requirements
d. How would I know this exists as a classroom observer? (Q. 1)

2. What went into developing this program/curriculum? (Q. 3)
a. What support is provided by the CMO/district? (Q. 4)
i. What role do they play beyond planning?
ii. Do they support collaboration between different school sites?
I. How does this work?
iii. IF NO: What is being done at the school level continue
building this component of your instructional program? (Q. 4)
I. Probe: Formal or informal groups, subject alike- grade
levels
III. Data
B. Measurement
1. What performance goals exist? How were they/are they being established?
(Q. 1)
a. What tools were used to develop them?
i. Data? Research?
b. How frequently do you re-assess the goals? (Q. 3)
2. How is your progress in building this program/curriculum being measured?
(Q. 3)
a. How is student work used as an assessment? What about benchmarks,
etc.? (Q.3)
b. What data is collected other than student assessment results?
i. How is it being collected?
1. Probe: who handles, who checks, who’s accountable?
c. What opportunities exist for analyzing the data? (Q.4)
3. What are the expectations for teacher data use from the organization? (Q.4)
a. How are these communicated?
b. How is data use monitored?
c. How are these expectations supported?
4. In what ways has data been used to inform the program? (Q. 3)
c. What have you found so far?
d. What has changed as a result?

IIII. Impact
A. What changes have you noticed in student achievement since this began? (Q. 4)
1. Has there been positive changes at any level?
2. What areas do you feel are lacking?
3. What work remains to be done?

Close: Is there anything else you’d like to tell me?
104
Appendix C: Interview Protocol – CMO Staff
Before the interview begins, I will review the IRB permission slip, explain that the
interview is confidential, and obtain a signature from each interviewee. I will provide an
additional copy of the permission slip for their records.

I. Background/Context
Before I ask you specific questions, I’d like to start by asking you some general
questions about your school, its surrounding neighborhood, and the district in order to
gain a broader understanding of the context in which you work.

1. Could you tell me a little about the students and community that you serve?
How would you describe the general culture of your school/system?
2. Could you tell us a bit about the history of your district, focusing on the last five
years (e.g., particular reform initiatives, strong partnerships w ith external
groups, major structural changes, etc.) (Q. 2/3)
3. How long have you been at this system? What is your prior experience and
training?
4. What are your duties and responsibilities, especially around curriculum design
and data use? (Probe: accessing, summarizing, interpreting, planning,
monitoring, etc.) (Q. 4)

II. Innovation
Lets talk about the work your district has done to prepare students for success in the 21
st

century.
A. Goals
1. In your own words, what does it mean to be prepared for the 21
st
century?
(Q. 1)
2. In terms if instructional innovation for the 21
st
century, what skills does your
organization value most? (Q. 1)
3. How are these values reflected in your mission/vision? (Q. 1)
4. What does the focus on these elements stem from? (Q. 2)
a. How were they planned? How were they communicated?
5. To what extent have federal policies and state standards influenced your
performance goals for students? (Q. 3)
a. In what ways do standards based instruction and 21
st
century skills
compliment one another in the curriculum?
b. What has been difficult about incorporating these two elements?
B. Instruction
1. Describe how 21
st
century skills are being included in the curriculum. (Q. 2)
a. What type of activities are characteristic of this?
b. How frequently do these activities/strategies take place?
c. How important are they in terms of the big picture?
105
i. Probe: measurement, assessment, expectations, graduation
requirements
d. How would I know this exists as a classroom observer? (Q. 1)
2. What went into developing this program/curriculum? (Q. 3)
a. What support is provided by the CMO/district? (Q. 4)
i. What role do you play beyond planning?
ii. How do you support collaboration between school sites?
iii. How do they inform continued development?
iv. Is there collaboration between different school sites? (Q. 4)
1. How does this work?
a. Probe: Formal or informal groups, subject
alike- grade levels
III. Data
C. Measurement
1. What performance goals exist? How were they/are they being established?
(Q. 1)
a. What tools were used to develop them?
i. Data? Research?
b. How frequently do you re-assess the goals? (Q. 3)
2. How is your progress in building this program/curriculum being measured?
(Q. 3)
a. How is student work used as an assessment? What about benchmarks,
etc.? (Q.3)
b. What data is collected other than student assessment results?
i. How is it being collected?
1. Probe: who handles, who checks, who’s accountable?
c. What opportunities exist for analyzing the data? (Q.4)
3. What are the expectations for teacher data use from the organization? (Q.4)
a. How are these communicated?
b. How is data use monitored?
c. How are these expectations supported?
4. In what ways has data been used to inform the program? (Q. 3)
e. What have you found so far?
f. What has changed as a result?

IIII. Impact
A. What changes have you noticed in student achievement since this began? (Q. 4)
1. Has there been positive changes at any level?
2. What areas do you feel are lacking?
3. What work remains to be done?

Close: Is there anything else you’d like to tell me?

106
Appendix D: List of Codes
- 21st Century Skills – student preparation
- 21st Century Skills – teacher understanding
- Curriculum – planning
- Curriculum – use of data and research
- Data – capacity building
- Data – use of research
- Standards
- Impact on student achievement.

Abstract (if available)

Abstract The significant advancement in technology, along with the continuing globalization of business and industry, has raised calls to change the way American students are being educated in order to better prepare them to excel in this constantly evolving 21st century environment. One method of creating change and informing direction is the practice of using data to inform decisions. Using this approach to innovate instruction for the 21st century requires thoughtful and conscientious leadership. In recent years, charter schools have been credited with the potential to innovate practices across schools more successfully that traditional school districts. This study investigated the ways in which data use is driving 21st century instructional innovation in charter schools. To explore these issues, qualitative case studies were conducted in two charter schools. Interviews with administrators as well as teachers and department chairs were conducted, and complementary document analysis was completed.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

School leaders' use of data-driven decision-making for school improvement: a study of promising practices in two California charter schools

PDF

Designing school systems to encourage data use and instructional improvement: a comparison of educational organizations

PDF

Preparing students for the future - 21st century skills

PDF

The key essentials for learning in the 21st century: programs and practices

PDF

Developing 21st century skills through effective professional development: a study of Jamestown Polytechnic Charter School Organization

PDF

Examining the applications of data-driven decision making on classroom instruction

PDF

Instructional leadership: the practices employed by elementary school principals to lead the Common Core State Standards and 21st century learning skills

PDF

The role of the school district toward preparing students for the 21st century

PDF

A community struggling to create a charter school: a rural case study

PDF

Allocation of educational resources to improve student learning: case studies of California schools

PDF

A school's use of data-driven decision making to affect gifted students' learning

PDF

21st century skills as equity pedagogy to teach academic content standards: a socioeconomically diverse district's implementation

PDF

The influence of leadership on the development of 21st century skills throughout Irish schools

PDF

Parent compacts in urban charter schools: an exploration of contents and processes

PDF

Strategies California superintendents use to implement 21st century skills programs

PDF

A study of California public school district superintendents and their implementation of 21st-century skills

PDF

Allocation of educational resources to improve student achievement: Case studies of four California charter schools

PDF

How urban school superintendents effectively use data-driven decision making to improve student achievement

PDF

Embracing globalization and 21st century skills in a dual language immersion school

PDF

A case study of 21st century skills programs and practices

Asset Metadata

Creator Butler, Lindsay Faye (author)

Core Title Charter schools, data use, and the 21st century: how charter schools use data to inform instruction that prepares students for the 21st century

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 04/08/2009

Defense Date 03/02/2009

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag 21st Century,charter schools,Curriculum and Instruction,data driven decision making,data use,High School,OAI-PMH Harvest

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Datnow, Amanda (committee chair), Hentschke, Guilbert C. (committee member), Wohlstetter, Priscilla (committee member)

Creator Email lbutler@icefla.org,lfbutler@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m2066

Unique identifier UC1412749

Identifier etd-Butler-2702 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-213772 (legacy record id),usctheses-m2066 (legacy record id)

Legacy Identifier etd-Butler-2702.pdf

Dmrecord 213772

Document Type Dissertation

Rights Butler, Lindsay Faye

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu