University of Southern California Dissertations and Theses

A longitudinal comparative study of the effects of charter schools on minority and low-SES students in California

(USC Thesis Other)

A longitudinal comparative study of the effects of charter schools on minority and low-SES students in California

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content A LONGITUDINAL COMPARATIVE STUDY OF THE EFFECTS OF CHARTER
SCHOOLS ON MINORITY AND LOW-SES STUDENTS IN CALIFORNIA
by
Christopher Alan Lund
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
August 2004
Copyright 2004 Christopher Alan Lund
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 3145241
Copyright 2004 by
Lund, Christopher Alan
All rights reserved.
INFORMATION TO USERS
The quality of this reproduction is dependent upon the quality of the copy
submitted. Broken or indistinct print, colored or poor quality illustrations and
photographs, print bleed-through, substandard margins, and improper
alignment can adversely affect reproduction.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
®
UMI
UMI Microform 3145241
Copyright 2004 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ii
ACKNOWLEDGMENTS
I dedicate this work to James and Pamela Lund, my father and mother, who
instilled in me an inherent desire to learn and grow and to Eli and Gabriela Lund, my
wife and daughter, who supported me through the long and challenging journey with
patience and love. I hope this text furthers our understanding of the charter school
movement as we move towards a high quality education for all children.
I would like to acknowledge the strong support and encouragement of my
dissertation committee, Dr. Stuart Gothold, Dr. Lawrence Picus, and Dr. Guilbert
Hentschke. Dr. Gothold, my dissertation chair, used his extensive interpersonal skills
to facilitate learning with kindness and support. Dr. Picus’ deep understanding of
quantitative analysis helped make the data compelling and meaningful and Dr.
Elentschke’s passion and understanding of charter schools helped broaden the charter
discussion, placing my study within the context of the charter school movement.
Finally, I would like to acknowledge the assistance provided by RAND,
especially Derrick Chau and Aimee Bower, who provided the matched school data
used in RAND’s California study of charter schools.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
iii
TABLE OF CONTENTS
Acknowledgments.......................................................................................................ii
List of Tables..............................................................................................................vi
Abstract.................................................................................................................... viii
CHAPTER 1. THE PROBLEM AND ITS UNDERLYING FRAMEWORK 1
History of Charter Schools: Creating a Context for Change........................ 2
Growth of Charter Schools................................................................................. 3
Charter Legislation.............................................................................................. 5
Goals of Charter Legislation...............................................................................8
Purpose of the Study......................................................................................... 13
Research Questions........................................................................................... 14
Hypotheses..........................................................................................................15
Significance of the Problem............................................................................. 15
Methodology.......................................................................................................16
Assumptions.......................................................................................................17
Limitations and Delimitations..........................................................................18
Definition of Terms........................................................................................... 20
Organization of the Study................................................................................. 21
CHAPTER 2. REVIEW OF THE LITERATURE...............................................23
Charter Legislation and At-risk Children........................................................23
Non-discrimination Legislation....................................................................... 24
Admission Policies............................................................................................ 26
Racial/Ethnic Balance Enrollment Requirements...........................................28
Summary of Charter Legislation with Respect to At-Risk Students........... 29
At-risk Student Enrollment in Charter Schools..............................................30
Study #1: Charter School Operations and Performance: Evidence
from California. Conducted by RAND by Zimmer, et. al., 2003 ......... 31
Study #2: The Pacific Research Institute Study by Pam Riley, 2000... 34
Study #3: The Hudson Institute Study by Finn, et. al., 1997................ 36
Study #4: Center for Washington Area Studies of Charter Schools
in D.C. by Henig, et. al., 1999 .................................................................. 36
Study #5: The State of Charter Schools. Fourth Year Report.
Conducted by RPP International with Nelson, et. al., 2000.................... 38
Study #6: UCLA Charter School Study by Amy Stuart Wells, 1999... 39
Study #7: Brookings Institute Study by Tom Loveless, 2002................42
Study #8: Rhetoric Versus Reality by Gill, et. al., 2001.........................42
Conclusions on At-Risk Enrolment in Charter Schools...........................44
Academic Achievement in Charter Schools................................................... 45
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
iv
Study #1: Charter School Operations and Performance: Evidence
from California. Conducted by RAND by Ron Zimmer, et. al., 2003.. 45
Study #2: Does Charter School Attendance Improve Test Scores?
The Arizona Results by Solomon, et. al., 2000........................................ 50
Study #3: Apples to Apples: An Evaluation of Charter Schools
Serving General Student Populations by Greene, et. al., 2003............... 52
Study #4: Rhetoric Versus Reality by Brian Gill, et. al., 2001............... 56
Study #5: The 2002 Brown Center Report on American Education:
How Well Are American Students Learning? By Tom Loveless at the
Brookings Institute, 2002........................................................................... 57
Study #6: Student Academic Achievement in Charter Schools:
What We Know and Why We Know So Little by Gary Miron and
Christopher Nelson, 2001 ........................................................................... 58
Study #7: California Charter Schools Serving Low SES Students by
Slovacek, et. al., 2002. Follow up analysis by David Rogosa, 2002..... 59
Study #8: Reducing the White-Minority Achievement Gap in North
Carolina by C.S. Bingham, P. Harman, P. Finney and A. Hood, 2001.. 61
Study #9: Pacific Research Institute Study by Pam Riley, 2000.......... 62
Study #10: Hudson Institute Study by Chester E. Finn, Bruno V.
Manno, Louann A. Bierlein, And Gregg Vanourek, 1997..................... 63
Conclusions on Academic Achievement in Charter Schools.................. 65
At-risk Student Achievement in Charter Schools...........................................66
Conclusions on At-Risk Student Performance in Charter Schools 70
Conclusions....................................................................................................... 71
Charter Legislation Targeting At-Risk Students............................................71
Conclusions on At-Risk Student Enrollment in Charter Schools................. 71
Conclusions on Academic Achievement in Charter Schools........................72
Conclusions on At-Risk Student Performance in Charter Schools...............73
Implications....................................................................................................... 73
CHAPTER 3. RESEARCH METHODOLOGY.................................................. 75
Research Questions........................................................................................... 76
Research Design.................................................................................................77
Population and Sample................................................................................ 80
Instrumentation............................................................................................ 82
Data Collection............................................................................................ 82
Validity and Reliability............................................................................... 85
Data Analysis................................................................................................88
CHAPTER 4. FINDINGS.......................................................................................93
Validating the Matched Schools...................................................................... 95
Elementary Hispanic Data Analysis............................................................... 99
Secondary Hispanic Data Analysis................................................................104
Elementary African-American Data Analysis.............................................. 107
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Secondary African-American Data Analysis............................................... I l l
Elementary SED Data Analysis..................................................................... 115
Secondary SED API Growth Data Analysis................................................ 119
Summary........................................................................................................... 123
CHAPTER 5. DISCUSSION................................................................................ 126
Discussion.........................................................................................................126
Discussion on Elementary At-Risk Student Achievement in Charter
Schools............................................................................................................. 128
Discussion on Secondary At-Risk Student Achievement in Charter
Schools............................................................................................................. 129
Conclusions..................................................................................................... 130
Conclusions on the hypotheses.......................................................................131
Recommendations............................................................................................133
REFERENCES........................................................................................................136
APPENDICES.........................................................................................................141
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
vi
LIST OF TABLES
Table 1: Charter School Laws, Schools and Students by Y ear....................................... 5
Table 2: Goals of the Charter Law......................................................................................9
Table 3: State-by-State Comparison................................................................................ 25
Table 4: Minority Enrollment Nationwide in Charter and Non-Charter Schools 39
Table 5: Demographic Mean Comparison 1999-2000....................................................96
Table 6: Demographic Mean Comparison 2000-2001....................................................97
Table 7: Demographic Mean Comparison 2001-2002....................................................98
Table 8: Elementary Hispanic API Growth Mean Comparison.................................... 99
Table 9: Pearson Correlation for elementary Hispanic API growth............................101
Table 10: Elementary Hispanic API Growth Regression Analysis.............................102
Table 11: Secondary Hispanic Mean Comparison....................................................... 104
Table 12: Secondary Hispanic Pearson Correlation..................................................... 105
Table 13: Secondary Hispanic API Growth Regression.............................................. 106
Table 14: Elementary African-American API Growth Mean Comparison.................108
Table 15: Elementary African-American API Growth Pearson Correlation............. 109
Table 16: Elementary African-American API Growth Regressions............................110
Table 17: Secondary African-American API Growth Mean Comparison..................111
Table 18: Secondary African-American API Growth Pearson Correlation................112
Table 19: Secondary African-American API Growth Regression............................... 114
Table 20: Elementary SED API Growth Mean Comparison........................................116
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
vii
Table 21: Elementary SED Pearson Correlation........................................................... 116
Table 22: Elementary SED API Growth Regressions..................................................118
Table 23: Secondary SED API Growth Mean Comparison..............................120
Table 24: Secondary SED API Growth Pearson Correlation............................121
Table 25: Secondary SED API Growth Regression Analysis...........................122
Table 26: Average Annual Growth in API for All California Schools.......................127
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ABSTRACT
Prior research indicates that charter schools, in general, are making significant
progress, but the results do not indicate that charter schools are performing
substantially better or worse than non-charter schools. The purpose of the study was to
build on prior research by comparing longitudinal student achievement in charter
schools and non-charter schools and allow for disaggregated data analysis of at-risk
student achievement.
The study was guided by two overarching questions examining the
performance of at-risk students, defined as Hispanic, African-American and low-SES
students, in charter schools: (a) How does the academic performance of at-risk
students in charter schools compare the to the performance of at-risk students in
comparable non-charter schools in California? and (b) How does the academic
performance of at-risk students vary across charter school type (start-up, conversion,
non-classroom based) in California?
Using a quasi-experimental design, the students in the charter school,
specifically the at-risk student population, represented the experimental group. The
charter model represented the treatment, and the students in the non-charter school,
specifically the at-risk student population, represented the control group.
The at-risk students in 161 California charter schools were compared with the
at-risk students in 161 matched California non-charter schools. Using all secondary
sources, the student performance in charter schools and non-charter schools was
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
compared longitudinally, examining four years of Academic Performance Index (API)
data, or more specifically, three years of API growth.
A series of statistical analyses were conducted on the six disaggregated,
matched data sets, which included elementary and secondary Hispanic, African-
American and low-SES students. Regressions generally indicated that the charter
effect was not statistically significant in regard to API growth among Hispanic,
African-American and SED students. When the charter effect was significant, it
generally had a negative effect on API growth, which seemed to be explained by the
inclusion of new start-up charter schools in the study. Despite the generally
insignificant charter effect and the few negative correlations to charter status, the API
growth means of Hispanic, African-American and low-SES students were generally
comparable among charter schools and non-charter schools. The study concludes with
the policy implications of the study and the recommendations for future research.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1
CHAPTER 1
THE PROBLEM AND ITS UNDERLYING FRAMEWORK
“The charter school movement has taken the country by storm” (Wells, et. al.,
1998, p. 305). Since the first charter school law in Minnesota, charter schools have
become the reform effort of choice for American education. Minnesota’s 1991 charter
school law culminated 6 years of reform legislation within the state and planted the
seed for change on a national level. The “storm”, perceived as a draught-quenching
rain by proponents and a catastrophic deluge by opponents, spread rapidly across the
country. With broad bipartisan support, charter school legislation expanded to 8 states
in just 3 years and to 40 states, Puerto Rico and the District of Colombia by 2003 (US
Department of Education, Charter Schools website).
Charter schools are perceived as a last-ditch effort to save public education
from vouchers, a decent into socioeconomic segregation, a return to local control, a
potential change agent for non-charter public schools, and a new model of
accountability. The goals identified by charter legislation are as varied as the
perceptions surrounding charter schools. Charter legislation is established to improve
student achievement, create new opportunities for specific populations of students,
offer diverse approaches to instruction, provide new professional opportunities for
teachers and encourage parent and community involvement (Hill and Lake, 2002). A
brief look at the historical context in education provides a window to the broad
perceptions and goals surrounding charter school legislation.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2
History of Charter Schools: Creating a Context for Change
Easton’s system theory proposes that there are identifiable stressors or triggers
to any new legislation (Wirt and Kirst, 1997). The impetus for charter legislation is no
exception. In 1980, a new president came to office clamoring for change in education
policy. President Reagan sought the elimination of the Department of Education,
proposed a cut in federal education dollars and even promoted an early version of a
voucher initiative in 1981. Although he failed to realize any of these changes,
President Reagan turned a spotlight on education issues. The severity of the problems
in education was revealed in 1983 by the report, A Nation at Risk. According to the
report, education was in a crisis. The system responded with such reform efforts as
site-based management, broad reading initiatives, expanded magnet schools and
additional categorical funding. A little known piece of failed California legislation in
1987 even attempted to permit teachers to run their own schools (Bierlien, 1997).
By 1990, the education system had failed to improve. In fact, the National
Assessment of Educational Progress, or NAEP, continued to indicate the poor
performance of United States’ schools in comparison to many industrialized nations
(NAEP Data, 2003). Even the head of the American Federation of Teachers, Albert
Shanker, proposed radical changes to the existing system. Drawing from a 1989
article by Ray Budde, Mr. Shanker proposed the creation of charter schools, inspiring
a Minnesota senator and bringing the charter school discussion to a national level
(Bierlien, 1997). In 1991, Minnesota passed the first charter school law, permitting
educators to create new schools or convert existing public or private nonsectarian
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3
schools into public charter schools. With the Minnesota law, charter schools were
bom, and like wildfire, the charter concept spread across the country. California
followed suit in 1992, followed by Colorado, Massachusetts, Michigan, Wisconsin,
New Mexico and Georgia in 1993. By 2002, charter schools were serving over
684,000 students in 2,695 schools (Center for Education Reform, 2003).
Growth of Charter Schools
Charter schools have been discussed regularly in State of the Union speeches,
gubernatorial and presidential stump speeches and within the halls of Capitol Hill and
state legislatures throughout the country. The broad bipartisan support has allowed
charter reform to flourish in the many diverse regions of the United States. The
Republican leadership has embraced charter legislation as a means of providing
parental choice and promoting local control and autonomy in school decision-making,
while Democrats uphold the public nature of charter schools as a means of
empowering teachers and parents and combating voucher initiatives (Bierlein, 1997).
Former President Clinton consistently used the bully pulpit to promote charter schools,
and charter funding grew from 6 million to 175 million dollars under his
administration, according to the US Charter School’s Website. President Bush, a
strong supporter of charter schools as governor of Texas, continues to support the
charter movement as a means of promoting accountability and choice in education,
and US Department of Education proposed funding of charter schools in 2003 has
ballooned to $300 million (Overview of Charter Schools, 2003).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4
“Some observers see charter schools as a sort of ‘middle ground’ between
creating options within the current public system and adopting the more radical choice
proposals that include nonpublic schools” (Metcalf, et. al, 2003, p. 543). Charter
schools are still public schools, despite the freedom from regulation allotted by the
legislation. They represent an extreme within public education, but they are not as
radical as voucher initiatives, which look beyond the public system to provide an
education to children.
There are several growth trends within the charter school movement.
Beginning in 1991, the number of states with charter school laws grew exponentially
until 1998. Despite this growth in legislation, the actual number of charter schools
and charter school students remained a very small percentage of the total school
population. In the last five years, there have been relatively few new charter school
laws (only five additional states), yet the number of total charter schools and charter
school students has increased significantly. While still representing less that 1% of the
total US school enrollment, the number of students enrolled in charter schools has
almost doubled every 2 years since 1998, serving more than 300,000 students in 2000
and more than 600,000 students in 2002 (See Table 1 for the number of charter school
laws, schools and enrollment from 1991-2002). Therefore, the growth of charter
schools can be seen in two stages: an explosion of charter school legislation in the first
half and a proliferation of charter schools and charter students in the second half.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5
Table 1: Charter School Laws, Schools and Students by Year
Year New Charter Laws # of Charter Schools
Nationwide
# of Charter
Students
Nationwide
1991 Minnesota None None
1992 California Not available Not available
1993 Colorado, Georgia,
Massachusetts, Michigan, New
Mexico, Wisconsin
Not available Not available
1994 Arizona, Hawaii, Kansas Not available Not available
1995 Alaska, Arkansas, Delaware, 98 (US Dept, of Ed. 17, 400 US Dept.
New Hampshire, Louisiana,
Rhode Island, Wyoming, Puerto
Rico
Year 1) of Ed, Year 1)
1996 Connecticut, District of 223 (US Dept, o f Ed, 58,620 (US Dept of
Columbia, Florida, Illinois, New
Jersey, North Carolina, South
Carolina, Texas
Year 1) Ed., Year 1)
1997 Mississippi, Nevada, Ohio, 428 (US Dept, of Ed, 110,112 (US Dept.
Pennsylvania Year 2) o f Ed., Year 2)
1998 Idaho, Missouri, Virginia, Utah, 678 (US Dept, of Ed, 162,130 (US Dept.
New York Year 3) o f Ed., Year 3)
1999 Oregon, Oklahoma 1,010 (US Dept, of
Ed., Year 4)
252,009 (US Dept,
o f Ed., Year 4)
2000 1,735(US Charter
Schools website)
350,000(US Dept,
of Ed., Charter
Schools website)
2001 Indiana 2,212(US Charter
Schools website)
514,000(US Dept,
of Ed., Charter
Schools website)
2002 Iowa, Tennessee 2695 (US Charter
Schools website)
684,000(US Dept,
of Ed., Charter
Schools website)
2003 Maryland 2799 (US Charter
Schools website)
685,000(US Dept,
of Ed., Charter
Schools website)
Charter Legislation
There is a tendency among researchers, politicians and the general public to
group charter schools as one large entity. Charter schools share a common definition
as public schools that have a “contract with the state or local agency that provides
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6
them with public funds for a specific time period” (Berman, et. al., 1998, p. 1).
According to the U.S. Department of Education website, this contract or charter
“states the terms under which the school can be held accountable for improving
student performance and achieving goals set out in the charter,.. .[freeing] charter
developers from a number of regulations that otherwise apply to public schools.”
According to the US Charter Schools Website’s Overview of Charter Schools, most
charter school legislation intends to: (a) provide choice within the public school
education system, (b) deregulate the schools to promote innovation and creativity, (c)
create a new system of accountability in public schools, (d) improve the public
education system, (e) promote autonomy through teacher and parent empowerment in
schools, and (f) create new professional development opportunities for teachers.
These similarities create the perception that all charter schools and all charter
legislation are equal, yet nothing could be further from the truth. As stated by Wayne
Jennings and fellow researchers, “each state approaches charter school legislation in a
more or less unique way, so charter laws vary greatly from state to state” (1998, p. 4).
Jennings and his fellow researchers identified 42 dimensions in 7 areas within
charter legislation where the law varies across state lines. The study examined the
following major legal and policy areas: charter development, school status, fiscal
issues (funding and fiscal autonomy), students (admission/non-discrimination
policies), staffing and labor relations, instruction, and accountability.
There is as much similarity within the 7 areas of the legislation as there is
difference. Arizona, for instance, permits private schools to convert to public charter
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7
schools while most state do not permit such conversion. Arkansas charters need to be
renewed every 3 years, while most states renew every 5 years. Twenty-one states
mandate that charter school teachers be credentialed, and 9 states require that teachers
remain part of the collective bargaining unit. The differences across states are
significant and determine whether the legislation is classified as, what Lauren Bierlein
calls “strong” legislation, defined as charter school law that “challenges the status quo
aspects of the system” (1997, p. 43). According to Bierlein, strong legislation is that
which provides the greatest flexibility and autonomy to charter schools while “weak”
legislation limits the charter school’s ability to make decisions and challenge the status
quo. The distinction between “strong” and “weak” legislation greatly determines the
character and design of the charter schools within each state.
As if the legislation alone didn’t diversify the charter landscape enough, there
is even further diversification within each state. The charter agreement between each
charter school and its approving entity (local district, local school board, State Board
of Education, university) creates a unique contract for the school. Each charter
contract distinctly determines the financial and programmatic latitude allotted to the
charter school. As a result, the charter legislation becomes a palette, which charter
schools use to create their charter. In the end, a whole range of charter schools are
created. The literature is full of references to dependent charters, independent
charters, affiliated charters, in-district charters, out-of-district charters, quasi-charters
and charter-like schools. These labels most often indicate relationship between the
charter school and the local district. The greater the fiscal and organizational
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8
autonomy, the more “independent” the charter school becomes. The distinction
between charter schools doesn’t end with finances.
In addition, there are conversion charters, existing schools, in most cases
public, that have converted to charter status and start-up charter schools, which are
schools that have been created as charter schools. There are also charter schools run
by EMOs, Educational Management Organizations. Some EMOs are for-profit
companies, like the Edison Schools, while others are non-profit organizations like the
YMCA or the Boys and Girls Club. Some charters are not run by organizations, but
affiliated with them in order to share space and resources, while others have no
physical structure at all, providing distance-learning opportunities that support home
schooling programs. These labels/distinctions not only reflect the diversity of charter
schools, but also create a challenge to researchers to classify, examine and compare
charter schools within each state and throughout the country.
Goals of Charter Legislation
The three most often cited reasons to start a charter school are: (a) to realize an
educational vision, (b) to gain autonomy and (c) to serve a special population
(Overview of Charter Schools, 2003). The reasons behind starting a charter school
differ only in their simplicity when compared to the established goals of state charter
legislation. Paul T. Hill and Robin J. Lake, in their analysis of the first thirty-four
charter school laws, identify 12 purposes of charter legislation (2002). Table 2 lists
the purposes of the law and the number of states, including the District of Columbia,
which include the stated goal in their respective legislation.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9
Table 2: Goals o f the Charter Law
Purpose of the law # of laws
with said
purpose
Goal #1: Offer diverse approaches to teaching and learning 23
opportunity for innovation
Goal #2: Improve student achievement 22
Goal #3: Provide new professional opportunities for teachers 22
Goal #4: Create additional choices for parents and students 18
Goal #5: Develop new forms of accountability 19
Goal #6: Establish new tools for measuring student performance 11
Goal #7: Create new opportunities for special population, such as at- 10
risk or gifted
Goal #8: Create performance-based education programs in place of 8
rules and regulations
Goal #9: Encourage parent and community involvement 5
Goal #10: Create avenues for new providers to start school; create 5
alternatives to traditional public education
Goal #11: Establish school as the unit for improvement 6
Goal #12: Add deregulation—freedom from rules and regulations 5
As indicated in Table 2, some goals, like providing new professional
opportunities for teachers, are shared by many charter laws, while other goals, such as
providing alternatives to traditional public education, are unique to only several
charter laws. The goals reveal the similarities and differences among charter law.
Furthermore, they hint at the underlying philosophy or theory that support the
established goals.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
10
In their synthesis of the goals, Hill and Lake conclude that four general
strategies or theories guide the legislation. They are the innovative/experimentation
strategy, the standards-based reform strategy, the new supply of public schools
strategy and the competitive/market strategy. The innovative/experimentation
strategy, embraced by Georgia, promotes the idea that charter school laws are intended
to create schools that “serve as laboratories for successful teaching strategies” (Hill
and Lake, 2002, p. 17). Often, these laws do not represent a radical departure from the
role of the traditional public school. They create goals that promote experimentation
within the realm of curriculum, instruction and assessment, but most often leave the
management, organization and finances under the purview of the local school district
or board.
The standards-based reform strategy frees “schools from rules so they can meet
higher expectations” (Hill and Lake, 2002, p. 18). Charter school laws, like those in
California and Colorado, that embrace reform typically exempt charters from the rules
and regulations governing traditional public schools. In addition, they often target
specific populations of students that have not been well served historically in the
education system.
The new supply of public schools strategy attempts to “create an alternative
framework for providing public education” (Hill and Lake, 2002, p. 18). The charter
schools laws that promote a new supply strategy, like Michigan and Massachusetts,
often empower outside agencies (i.e., community groups, non-profits and/or for-profit
organizations) to design and run charter schools. The new supply strategy can also
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
11
target specific populations of students and often represents a more radical departure
from the traditional public school system.
The competitive/market strategy promotes parental choice as the driving force
behind charter legislation. The strategy asserts that charter schools, in a market-driven
economy, will compete directly with traditional public schools. Parents will choose
schools that provide the best education for their children and will direct state and
federal funding to those schools. The competitive/market strategy, embraced by
Arizona, creates laws that are broad in scope and authority, liberal in regulation and
driven by choice.
Although many states share similar strategies and general goals, the
implementation of the goals, in terms of charter policy, is often inconsistent. “Even
the most originally coherent laws become watered down, negotiated, and tinkered with
during deliberations” (Hill and Lake, 2002, p. 20). Charter laws are created within the
political frame, subject to the influences of lobbyist, special-interest groups, school
boards, and teacher’s unions. Therefore, despite clear goals for charter legislation, the
elements of the law don’t always align to the goals, further diversifying the charter
landscape nationwide.
As stated in Table 2 above, improving student achievement is a primary goal
of most charter legislation. How charter schools are meeting the achievement goal has
been an issue of debate since the onset of charter schools (Berman, 1998; Finn, et. al.,
1997; Gill, et. al., 2001; Miron and Nelson, 2001). For the longest time, there was
very little data available on charter school performance. Either there was not a critical
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
12
mass of charter schools in any particular state to make a valid, statistical comparison,
or there was not a consistent indicator or instrument used by both charter and non
charter schools. As a result, the initial studies of charter school performance measured
student achievement through the use of surveys: teacher surveys, parent-satisfaction
surveys and student-satisfaction surveys (Finn, et. al., 1997; Riley, 2000). Within the
past 3 years, there have been numerous statewide and national studies comparing
charter schools and non-charter schools, providing a broader picture of charter school
performance.
Statement of the Problem
Examining the studies that compare charter and non-charter school
performance, there is little evidence on the performance of at-risk students in charter
schools. The Hudson study (Finn, et. al., 1997) is the only national study that
disaggregates the data among minority and low-SES students, but the study is limited
in scope (only 17 schools across 10 states) and only indicates the perception of
minority and low-SES parents within charter schools. The study does not provide any
objective data on the academic achievement of the specific groups. In fact, none of
the national studies that use standardized testing data (Greene, et. al., 2003; Loveless,
2002; Gill, et. al., 2001; Miron and Nelson, 2001) disaggregate the data for at-risk
students. Only a few statewide studies from California, Arizona and North Carolina
provide any data on how charter schools are addressing minority and low-SES
achievement. The Slovacek (2002) and Rogosa (2002) studies from California
provide a limited picture of low-SES achievement, but the other studies either fail to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
13
disaggregate the data among minority groups or fail to draw any conclusions on the
achievement of minority groups or low-SES students within charter schools. As a
result, the only general indicator of the performance of at-risk students is based on the
statewide and national comparisons of similar charter and non-charter schools, a
potentially inaccurate indicator of the actual minority and low-SES achievement
results.
Purpose of the Study
The purpose of the study is to adequately compare longitudinal student
achievement in charter schools and non-charter schools and allow for disaggregated
data analysis of at-risk student achievement. Currently, there is only limited research
evaluating the performance of low-SES students in charter schools and no available
research evaluating the performance of minority students in charter schools.
Considering the extensive efforts made by legislators to address at-risk children (i.e.,
goals targeting at-risk children, non-discrimination and admission policies,
racial/ethnic requirements), this study will provide valuable information on how at-
risk children are performing in charter schools. Therefore, this study compares charter
and non-charter school performance, specifically addressing the issue of at-risk
student performance in charter schools.
Considering the extensive research that has already been done in California
(See Zimmer, et. al., 2003; Slovacek, et. al., 2002; Rogosa, 2002; Wells, et. al., 2000),
the purpose of this study is not to replicate, but to build on that research. The RAND
California study conducted by Zimmer and fellow researchers did not disaggregate the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
14
data to examine the comparative performance of any sub-groups of students (2003).
The Slovacek and Rogosa studies examined low-SES performance using the
Academic Performance Index (API), but compared ninety-three charter schools to all
non-charter schools in the state. Finally, Wells and fellow researchers did not use
standardized testing to examine student performance. Therefore, this study adds to
the body of research by examining minority and low-SES student achievement data in
similar charter and non-charter schools. The study is different from the Slovacek and
Rogosa studies because it: a) examines minority performance in charter schools, b)
examines 4 years of API data relating to minority and low-SES students and c)
compares charter and non-charter school performance more accurately by matching
similar charter and non-charter school data. The study also builds on the RAND
California study (Zimmer, et. al., 2003) by examining minority and low-SES student
performance in the same matched charter and non-charter schools identified in the
RAND California study. By using the same matched schools and the same or similar
formula for analyzing the results, the performance of minority and low-SES students
can be examined in relation to the overall student performance data presented in the
RAND California study.
Research Questions
The study is guided by two overarching questions examining the performance
of at-risk students, defined as minority and low-SES students, in charter schools:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1. How does the academic performance of at-risk students in charter schools
compare the to the performance of at-risk students in comparable non-charter
schools in California?
2. How does the academic performance of at-risk students vary across charter
school type (start-up, conversion, non-classroom based) in California?
Hypotheses
The current research on charter schools provides a general indication of how
at-risk students are performing in charter schools. Given that, (a) most studies indicate
that students in charter schools are performing on a comparable level with non-charter
school students and (b) the RAND study indicated that charter school students in non
classroom based charter schools underperformed students in start-up and conversion
charter schools and start-up charter school students slightly outperformed conversion
charter school students, the hypothesis is as follows:
1. At-risk student performance is comparable in charter and non-charter schools.
2. At-risk student performance in non-classroom based charter schools is below
at-risk student performance in start-up and conversion schools, and at-risk
students in start-up charter schools outperform at-risk students in conversion
schools.
Significance of the Problem
Since charter schools have been identified as an essential means of addressing
at-risk students, the importance of the study is to provide much-needed research, on a
statewide level, evaluating the performance of minority and low-SES students in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
16
charter schools. The study is the first to use standardized testing data to measure the
longitudinal effect of charter schools on minority students. Furthermore, the study
provides additional data on the longitudinal effect of charter schools on low-SES
students. The data builds on the Rogosa’s (2002) and Slovacek’s (2002) studies of
low-SES students by providing a more accurate indication of charter performance by
comparing low-SES student performance in charter schools to similar non-charter
schools instead of all non-charter schools. Finally, the study builds on the research
conducted by Zimmer and fellow researchers (2003) by disaggregating minority and
low-SES performance among the various types of charter schools. In this way, the
conclusions drawn by Zimmer, et. al., on the effects of start-up, conversion and non
classroom based charters as a whole can be compared to the effects of each charter
type on minority and low-SES performance.
Methodology
Campbell and Stanley Design 10 is the most common quasi-experimental
design (Michael, W.B. & Benson, J., 1994). The design is most frequently used in
non-laboratory situations or field studies because it is not dependent upon the
randomization of participants. As a result, the study allows for analysis of treatment
effect in an existing environment or situation. Since charter schools are schools of
choice, randomization is not possible within the experimental design. Furthermore,
the study will examine existing data in a fieldwork study (i.e., API growth data). As a
result, the quasi-experimental Design 10 is the most relevant design to compare
longitudinal student achievement in charter and non-charter schools and the design
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
17
used by the majority of statewide and national studies on charter school achievement
(Zimmer, et. al., 2003; Loveless, 2002; Solomon, et. al., 2000; Slovacek, et. al., 2002;
Rogosa, 2002).
Within the structure of Design 10, the students in the charter school,
specifically the at-risk student population, will represent the experimental group. The
charter model will represent the treatment and the students in the non-charter school,
specifically the at-risk student population, will represent the control group. The at-risk
student population will be defined as minority students and low-SES students, low-
SES being measured by student participation in the school lunch program. The charter
school “treatment” is recognized as being a broad area of study, given the diversity of
charter schools throughout the state. In order to narrow the charter treatment, the
study will disaggregate the data according to charter school type (start-up, conversion,
and non-classroom based). By examining the charter school type, the study not only
better defines the charter effect, but also allows at-risk student achievement to be
compared with the general results of the California study (Zimmer, et. al., 2003).
Assumptions
The study assumes that the Academic Performance Index (API), an index score
that combines language arts and math scores using the norm-referenced Stanford Test,
Ninth Edition and the criterion-referenced California Standards Test (CST), is a valid
measure of student performance (i.e. students exerted a maximum effort on the
Stanford Nine and CST to adequately reflect ability). Furthermore, the study assumes
that the API score provides a valid measure of the “charter effect” on student
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
18
achievement (i.e. the differentiated program created by charter schools as a result of
their charter status). Understanding that the “charter effect” is complex, there is an
assumption that the total charter effect can be isolated and measured by controlling
significant variables in the comparison of similar charter and non-charter schools.
Finally, the study assumes that the research design, instrumentation, data analysis
procedures and control measures are appropriate.
Limitations and Delimitations
The study is limited to at-risk achievement in charter schools within California.
The at-risk student population, comprised of traditionally under-performing students,
is limited to the categories used by the California Department of Education to group
minority students (i.e., black, Asian, Pacific-Islander, American-Indian and Hispanic)
and low-SES students (i.e., student who participate in free and reduced lunch
programs). Data is collected on each minority group, but the research findings are
limited to the two largest minority groups, black and Hispanic students. These groups
represent the largest minority groups in the state and, as a result, there is significant
data available within schools on these groups. The smaller minority groups, typically,
do not have API scores available, since, according to state policy, the specific minority
group population must be greater than 10% of the total school population to trigger a
score. As a result, the charter and conventional schools with small minority
populations do not have disaggregated scores. In terms of SES, the study recognizes
that student participation in the free or reduced lunch program is not the most accurate
indicator of SES, but it is the only indicator used by the state. The indicator also has a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
greater limitation on charter schools, where students qualify for free or reduced lunch
based on family income, but the school often does not offer a free or reduced lunch
program (Zimmer, et. al., 2003).
The study uses the API growth as a measure of at-risk achievement, which has
several limitations. First, the API has been calculated differently every year. The
SAT9 scores, and most recently California Standards Test (CST) scores, have been
weighted differently every year. Fortunately, the formula used to calculate the API
has, at least, been applied evenly across the state. So, even though the scores reflect
different values from year to year, they at least reflect the same, different values across
all elementary schools and across all secondary schools. Furthermore, by comparing
the API growth between charter and non-charter schools, the weighted difference in
the formula is diminished. The growth calculation provides a measure of school
improvement instead of strict school achievement.
The second problem with the API involves the formula used to calculate the
score. The formula is very aggregated, incorporating SAT9 and CST scores across
grade levels and across subjects. As a result, the API provides a general and imprecise
measure of varying student performance according to grade level and curricular
subject area (Zimmer, et. al, 2003). The API doesn’t control for matched groups nor
does it match individual student scores. Unfortunately, the state of California does not
have any mechanism to match individual student scores over time, so the API cannot
control for matched student groups.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
20
The third limitation of the API is that API scores are not calculated for every
school. The API was “not computed for 25 percent of charter schools whereas it was
not computed for only 6 percent of conventional schools” (Zimmer, et. al., 2003, p.
41). New schools do not have API growth scores since they need two years of testing
data to trigger a growth score. Also, the state does not compute API scores for schools
with excessive parent waivers (over 10%) that exempt children from taking the test.
The limitations to the study are a result of state policies and procedures. The
study design, which examines longitudinal data comparing matched schools, serves to
mitigate the limitations. The longitudinal aspect of the study provides a general
indication of student achievement, as measured by changes in performance over 4
years. Furthermore, the matched design of the study, comparing similar charter and
non-charter schools, compensates for the state’s lack of individual student data. Also,
by examining 352 charter schools, even if 25% of the schools have to be excluded due
to missing API data, there is still a significant number of charter schools (roughly 260)
to draw a significant conclusion.
Definition of Terms
1. Academic Performance Index (Abbreviated API): The aggregated index
score used by the California Department of Education that combines
language arts (reading, language, spelling) and math scores using the norm-
referenced Stanford Test, Ninth Edition (SAT9) and the criterion-referenced
California Standards Test (CST).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
21
2. At-risk: Students pertaining to a minority group or socioeconomic class that,
historically, has underperformed the national norm.
3. Charter School: A public school that operates under an established
agreement or charter defined by state charter legislation.
4. Conversion Charter: A traditional public school that operates as a charter
school.
5. Conventional School: A traditional public school that operates within a
district structure under state education code.
6. Low-socioeconomic status (low-SES): Students whose parents qualify for
the federal free or reduced lunch program.
7. Minority: Students whose parents have indicated on school enrollment
documents an affiliation to any, non-Caucasian racial or ethnic group.
8. Non-classroom Based Charter: A charter school that operates outside of
traditional classroom settings, where in-seat attendance is not measured in
classroom attendance.
9. Start-up Charter: A new charter school that have been created as a result of
state charter legislation.
Organization of the Study
Chapter 1 of the study has presented the introduction, the background of the
problem, the statement of the problem, the purpose of the study, the questions to be
answered, the research hypotheses, the significance of the study, a brief description of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
22
the methodology, the assumptions, limitations, delimitations, and the definitions of
terms.
Chapter 2 is a review of relevant literature. It addresses the following topics:
(a) charter legislation and at-risk children, (b) at-risk student enrollment in charter
schools, (c) academic achievement in charter schools, and (d) at-risk student
achievement in charter schools.
Chapter 3 presents the methodology used in the study, including the research
design; population and sampling procedure; and the instruments and their selection or
development, together with information on validity and reliability. Each of these
sections concludes with a rationale, including strengths and limitations of the design
elements. The chapter goes on to describe the procedures for data collection and the
plan for data analysis.
Chapter 4 presents the results of the study. Chapter 5 discusses and analyzes
the results, culminating in conclusions and recommendations.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
23
CHAPTER 2
REVIEW OF THE LITERATURE
Examining the charter effect on at-risk children has implications beyond
academic performance. First, as noted above, charter legislation is diverse in character
and purpose. As a result, there is a significant difference in how charter legislation
directly or indirectly impacts at-risk children. Second, the charter effect on at-risk
children is only significant if charter schools are actually servicing at-risk children.
An examination of the research on at-risk student enrollment in charter schools creates
a context for the academic performance of at-risk children. Third, the academic
performance of at-risk children is a subset of the overall achievement of students in
charter schools. By examining the growing body of research on charter school
performance, general conclusions can be drawn on how charter schools are meeting
the academic needs of students. In addition, comparisons can be made on how charter
schools are performing in relation to conventional or non-charter schools. Finally,
based on the studies of academic performance in charter schools, conclusions can be
drawn on what is currently known about the performance of at-risk students in charter
schools.
Charter Legislation and At-risk Children
“Charters have the potential to extend choice to low-income families that
presently lack options” (Gill, et. al., 2001, p. 139). As indicated above in Table 2, ten
states include specific goals in their charter laws that address providing opportunities
for specific populations, including at-risk children. Those states include California,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
24
Colorado, Florida, Illinois, Kansas, Michigan, Nevada, New Jersey, North Carolina
and Rhode Island. How those goals translate into specific policy requirements varies
across state lines. In addition, states without a specific goal to create charter schools
that provide greater opportunities for specific populations can also include policies
that support at-risk students. Wayne Jennings and fellow researchers looked at
specific policies within charter law involving students (1998). The policies regarding
non-discrimination, admission, and racial/ethnic balance have a direct or indirect
effect on how charter schools provide an education for at-risk students.
Non-discrimination Legislation
Non-discrimination policies can have a direct effect on which students get
admitted to and maintain enrollment in charter schools. According to Jennings’ study,
all states are required to abide by federal non-discrimination statutes (1998). As a
result, charter schools cannot discriminate on the basis of race, color, creed, gender,
ethnicity, or disability. Furthermore, many states incorporate separate language in
their charter laws to prohibit discrimination (See Table 3 for specific state-by-state
information). Of the ten states with goals to provide opportunities to specific
populations, eight of them include additional language on discrimination. In all, over
half the states (20 out of 34 laws) specifically prohibit discrimination based on race,
color, creed, ethnicity, national origin, gender or disability.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
25
Table 3: State-by-State Comparison
State Non-Discrim ination Policy Admission
Policy
R acial/Ethnic
R equirem ent
Alaska State and Federal law None-Open None
Arizona Ethnicity, national origin, gender, income level, disability,
English language proficiency, athletic ability
District
preference
None
Arkansas State and Federal law None-Open None
California Ethnicity, national origin, gender, disability Conversion-local
attendance
Reflect district
Colorado Disability, race, creed, color, gender, national origin, religion,
ancestry, need for special ed.
District only None
Connecticut Race, color national origin, gender, religion, disability, athletic
performance, English language
None-Open Yes; also
preference to
75% minority
Delaware Race, creed, color, sex, handicap, national origin Local preference None
District of
Colombia
Race, color, religion, national origin, language, intellectual or
athletic ability, special needs status
None-Open None
Florida Disabilities, LEP students District
preference
Reflect
community
Georgia State and Federal law District only None
Hawaii State and Federal law None-Open None
Idaho Religion, race, gender Local preference None
Illinois State and Federal law District only None
Kansas State and Federal law District only Reflect district
Massachusetts Race, national origin, creed, sex, ethnicity, sexual orientation,
language, disability, intellectual/academic
Local preference None
Louisiana Race, religion, ethnicity, national origin, intellectual ability,
disability
Attendance area
specified-charter
None
Michigan State and Federal law Local preference None
Minnesota Intellectual ability, achievement, aptitude, athletics Open Reflect
community
Mississippi Race, color, creed, national origin None-Open None
Missouri Race, ethnicity, national origin, disability, gender, income
level, EL, athletic ability
Open or specific
by charter
None
Nevada Race, gender, religion, ethnicity, disability None-Open Reflect district
New Hampshire Disabled student Open
Conversion-local
None
New Jersey Intellectual, athletic, achievement, aptitude, handicap, EL District
preference
Reflect
community
New Mexico State and Federal law District only None
North Carolina Ethnicity, national origin, gender, disability Open
Conversion-local
Reflect district
Ohio Race, creed, color, handicap, gender, intellectual ability,
aptitude, athletic ability
District only Reflect
community
Pennsylvania Intellectual ability District pref. None
Rhode Island State and Federal law None-same as
non-charters
Reflect district
South Carolina Disability, race, creed, color, gender, national origin, religion,
ancestry
Open Reflect district
Texas National origin, race, religion, disability Local preference None
Utah State and Federal law District pref. None
Virginia Disability, race, creed, color, gender, national
origin, religion, ancestry
District only None
Wisconsin Sex, race, religion, national origin, ancestry, pregnancy,
marital or parental status, sexual orientation, disability
Open
Conversion-local
Reflect district
Wyoming Ethnicity, national origin, gender, disability Open
Conversion-local
Reflect district
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
26
Several states are even more detailed in their anti-discrimination policy. Nine
states prohibit discrimination on the basis of religion, and eight states prohibit
discrimination on the basis of athletic and intellectual ability. Seven states prohibit
discrimination on the basis of English-language status, which, with the exception of
Florida, are not the large states with high EL populations. Two states reference socio
economic status and ancestry as criteria for non-discrimination, and Wisconsin even
includes language prohibiting discrimination based on sexual orientation and the
parent’s marital status. In summary, at-risk students that represent a minority group or
have a disability are well protected by the legislation. English-leamers and low-
performing students are not specifically referenced in the majority of anti-
discrimination charter policies and socio-economic status is only included in two
charter laws regarding discrimination.
Admission Policies
In regard to admission, the Jennings’ study examined whether the charter law
required admission based on the location of residence (1998). The implication for at-
risk students, including minority students, low SES (socio-economic status) students,
English learners (EL students) and students with disabilities is how residence, which is
a strong indicator of SES, is connected with charter school admission. Since choosing
a district in which to live is a choice made more frequently by middle- and upper-
income families than low-income families, where charter schools open and who they
admit has repercussions for at-risk students (Gill, et. al., 2001). States are divided into
three categories: (a) states that require admission based on a defined attendance
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
27
boundary, (b) states that require admission or preference from within the district, and
(c) states that require an open enrollment policy (See Table 3 for specific state-by-state
information).
First, nine states require charter schools to admit students based on a defined
attendance boundary (Jennings, et. al., 1998). Seven additional states require
conversion schools, traditional public schools that convert to charter status, to admit
students under their traditional attendance boundary. States that require charter
admission based on a local attendance boundary are required to accept students within
a specific area, which defines at-risk enrollment geographically. Under such a system,
charter schools that open in “at-risk communities” would thereby enroll a greater
number of at-risk students. The opposite would also hold true; charter schools that
open in middle/high SES communities would therefore service very few at-risk
students. Attendance is by choice, but the location of the charter school, from a policy
perspective, determines if the charter school matriculates a significant number of at-
risk students.
Second, ten charter laws require that charter schools admit or give preference
to students from within the sponsoring district. Under such a system, charter school
enrollment would potentially be based the demographics of the sponsoring district.
There are many mitigating factors to at-risk enrollment under such a policy. First,
without the benefits of busing, charter school enrollment may be limited
geographically in large districts. Larger districts, serving extensive geographic areas,
may preclude charter attendance simply based on the distance from home to school.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
28
Second, the diversity of the sponsoring district would determine the potential
enrollment of at-risk students. Conversion or start-up charter schools that open in
districts with high percentages of at-risk children, potentially, would have a significant
enrollment of at-risk students. Third, other admission policies from the state could
determine or influence charter enrollment. States that require charter schools to
maintain a racial or ethnic balance would define the charter enrollment more
specifically.
Finally, fifteen out of thirty-four state laws are either not specific or require an
open-admissions policy. Charter laws that are vague or mandate an open-admissions
policy would create charter schools that admit students regardless of residence.
Although school location, school mission, and other state admission policies may
influence enrollment, the open-enrollment policy is the least restrictive and provides
the charter school with greatest flexibility for enrollment.
Racial/Ethnic Balance Enrollment Requirements
According to Jennings’ study, thirteen of out thirty-four charter laws require
the charter school to attain some form of racial or ethnic balance (See Table 3 above
for specific state-by-state information). Eight states determine that the charter school
enrollment should reasonably reflect the enrollment of the sponsoring district while
four states determine that the charter enrollment should reasonably reflect the
enrollment of the school’s community. Connecticut’s law states that charter schools
must describe admission criteria and procedures to promote a diverse student body.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
29
Connecticut also gives preference to charter approval if the charter school enrolls
more than 75% minority students.
Whether charter school enrollment must reflect the enrollment of the district or
the community has implications for states with large districts. A charter school’s
ability to replicate district enrollment is significantly diminished in large, diverse
districts. Furthermore, attendance area admission policies can complicate racial/ethnic
enrollment requirements and vice versa. Nine of the thirteen states with racial/ethnic
requirements have open enrollment policies, yet the charter school’s enrollment must
reflect that of the district or community. Among the ten states with the goal to provide
opportunities for specific populations, seven include specific policy language to
promote a racial/ethnic enrollment balance with the district or community.
Summary o f Charter Legislation with Respect to At-Risk Students
There are various policy methods used within state legislation to address at-
risk students. Some, like non-discrimination policies and racial/ethnic enrollment
requirements, are direct means of addressing how charter schools will serve at-risk
students. Others, like admission policies based on location of residence, provide
indirect policy tools that impact at-risk enrollment in charter schools.
Non-discrimination policies are well written to protect at-risk students that
represent a minority group or have a disability. On the other hand, most policies do
not specifically protect English-leamers, low-performing or low socio-economic
students in the majority of charter laws. In regard to admission, attendance at charter
schools is voluntary, but guided by a variety of attendance-boundary policies that
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
30
could impact at-risk enrollment. Finally, about a third of the charter laws use
racial/ethnic balance requirements to define at-risk enrollment at charter schools.
As mentioned above in Table 2, ten states include a specific goal to provide
opportunities to at risk students. The goal, in most states, translates into specific
policies to support at-risk students. There are also many states, like Connecticut, that
include strong policies to support at-risk students even though the state does not have
an explicit goal for targeted populations. The goals and policies establish legislative
means to support at-risk students, but do not guarantee that at-risk students are being
served in charter schools. How charter legislation actually translates into at-risk
charter enrollment is a different area of study, one that has been explored in depth
since the beginning of charter school legislation.
At-risk Student Enrollment in Charter Schools
As indicated above, various state laws include non-discrimination policies and
racial/ethnic requirements that directly support serving at-risk children in charter
schools. Admission policies also have the potential to encourage charter schools to
reach out to at-risk children when they are used in conjunction with racial/ethnic
requirements or when they mandate an attendance boundary that directly targets an at-
risk student population. These policies provide a legislative tool for charter-approving
agencies to use as a means to address typically underserved populations. Greene and
fellow researchers found in their surveys that “many charter schools serve targeted
populations [because] the procedures by which new charter schools are created often
encourage such targeting” (2003, p. 7). Charter approving agencies, like local school
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
31
boards, use these procedures to encourage charter schools to serve at-risk children.
The targeted population also provides political benefit to the local school board since
the charter school that serves an at-risk population doesn’t compete directly with
traditional public schools and serves a student population that “regular public schools
find less desirable because they are more likely to cause trouble, require extra
academic help, or in some other way be difficult and expensive to educate” (Greene,
et. al., 2003, p. 7).
Since the first charter school law in Minnesota, the topic of at-risk student
enrollment has been at the forefront of charter school implementation. As a result, at-
risk student enrollment in charter schools has been measured from the onset of the
Minnesota legislation. The data is abundant and fairly conclusive; charter schools are
serving a comparable percentage of at-risk children nationwide and within individual
states as compared to the at-risk student enrollment in public schools.
Study #1: Charter School Operations and Performance: Evidence from California.
Conducted by RAND by Zimmer, et. al., 2003
The RAND California study looked at student enrollment in charter schools
from a policy and admission perspective. The researchers examined the issues of
access, admission, focus of school services and the characteristics of students. The
California law specifically states that charter schools describe in their charters “the
means by which the school will achieve a racial and ethnic balance among its pupils
that is reflective of the general population residing within the territorial jurisdiction of
the school district to which the charter petition is submitted.” The researchers point
out the potentially contradictory language of the law. Charter schools that target a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
32
specific at-risk population will not have an enrollment that reflects the “territorial
jurisdiction of the school district to which the charter petition is submitted”.
Furthermore, charter schools that petition an outside agency (i.e., county or state) are
being asked to describe how the charter enrollment reflects a district or region that
may be nothing like the community of the charter school. Recognizing the potential
conflict of the law, the researchers used surveys, student enrollment data, and
comparative analyses of charter and non-charter schools in order to draw some
conclusions on how charter schools are serving an at-risk student population.
First, in terms of access, the researchers examined the admission process used
by charter and non-charter schools. According to surveys of school principals, the
admission process used by charter and matched, non-charter schools were very
similar. Disaggregating by charter type, start-up charter schools were much more
likely to use achievement tests and personal interviews in the admission process, but
only 3.8 percent of start-up schools actually used the results as enrollment criteria.
Second, the researchers examined the difference between charter and non
charter schools in terms of promoting a school focus that targets a specific population
of students. Using the surveys, the study found that charter schools are much more
likely than conventional schools to use a school focus to target a specific student
population. Thirty-three percent of charter school principals indicated that they
focused their services on a specific population in comparison to only twenty-one
percent of conventional schools. Among the charter schools, start-up charters were
more likely than conversion schools to use a targeted focus on enrollment.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
33
Finally, the study examined the characteristics of students being served by
charter and conventional schools. The researchers looked at statewide and district-
level data to compare charter and non-charter enrollment. In a straight statewide
comparison of total students, the study found that charter schools serve a greater
percentage of white and black students and a lesser percentage of Hispanic and Asian
students. Among charter school types, start-up charter schools serve an even higher
number of white and black children, while conversion charter schools serve a
significantly smaller percentage of white students and a comparable percentage of
Hispanic students. Since the legislation requires charter schools to indicate how they
are achieving a racial balance with the student population in the district, the
researchers compared the average racial makeup of charter students to that of
conventional public school students within the same school district. Controlling for
district heterogeneity, they found that “charter school students are more likely to be
black and less likely to be Hispanic or Asian, but no more or less likely to be white”
(Zimmer, et. al., 2003, p. 36).
The conclusions drawn by the researchers provide a thorough snapshot of the
students being served in California charter schools. The data collected by the
researchers supports the conclusion that California charter schools serve a greater
percentage of black students and a lesser percentage of Hispanic and Asian students.
The actual difference between percentage of students served in charter and non-charter
schools is less than ten percent among each racial category, except Hispanic. There
also seems to be a notable difference between start-up and conversion schools, which
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
merits further study on a national level. The study did not disaggregate the data
according to low-SES student populations, but the researchers did note that, “many
charter schools do not participate in free and reduced lunch programs”, even though
the students are eligible (Zimmer, 2003, p. 192). This discovery raises an important
question on research methodology; Looking at schools that provide free and reduce
lunch does not necessarily indicate that the students are not low-SES.
Study #2: The Pacific Research Institute (PRI) Study by Pam Riley, 2000
The PRI study surveyed 100 charter schools in California with a total
enrollment of 41, 531 students. The study collected surveys from 100% of the
principals, 40% of the teachers (totaling roughly 1,000 teachers) and 40% of parents
(roughly 15,200 parents). The PRI study concluded that, “students in charter schools
share demographic characteristics with students in all public schools” (2000). The
conclusions were drawn from the following data collected: (a) thirty-five percent of
the students in charter schools are white compared to 40% of students enrolled in
California’s public schools, (b) forty-four percent of charter school students are
Hispanic while 40% of California’s public school children are Hispanic, (c) six
percent of charter school students are African-American while 10% of California’s
public school children are African-American, (d) seven percent of charter school
students are Asian/Pacific Islander while 10% of California’s public school children
are Asian/Pacific Islander, (e) four percent of charter school students are American
Indian while 1% of California’s public school children are American Indian, and (f)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
35
charter schools enrolled slightly more Hispanic and American Indian students and
slightly less African-American and Asian students (Riley, 2000).
The conclusions drawn by the study correlate with the data collected in the
study. In addition, they have greater credibility given the large number of parents
surveyed. The study explicitly asked the parents, “What is the ethnicity of the
child/children attending the charter school?” (Riley, 2000, p. 38). In addition, 4,000
surveys were translated into Spanish for Spanish-speaking parents. As a result, the
ethnicity percentages reported in the study are based directly on the surveys collected
from parents, but the accuracy of the data can be put into question.
First, the conclusions obviously reflect the minority enrollment of students in
California. Although the study doesn’t generalize the findings, they are, by nature,
state specific. Second, some discrepancy may exist in the data based on the random
selection of parents who received and completed the surveys. Also, the surveys were
not translated into any other minority language, so they may be somewhat skewed
towards English-speaking and Spanish-speaking parents, which could, theoretically,
affect the percentage of surveys collected from non-English speaking and non-Spanish
speaking parents. In addition, the survey didn’t ask the teachers or principals to
include any information on the ethnicity of the students at the school site, nor did the
study make any reference to consulting any other state collected data on ethnicity, so
there was no means of comparing the parent responses to any other data.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
36
Study #3: The Hudson Institute Study by Finn, et. al., 1997
The Hudson Institute Study, like the PRI study, was based on surveys sent to
principals, teachers and parents at charter schools. The Hudson study surveyed 50
charter schools enrolling 16,000 children across 10 states. One thousand, three
hundred surveys were collected. The Hudson study concluded that: (a) half of US
charter pupils belong to minority groups compared with a third in conventional
schools, and (b) between 34 and 41 percent of charter school children come from low-
income families (Finn, et. al. 1997). Given this data, the authors concluded, “One
might suppose that the ‘creaming’ allegation could now be laid to rest. Put simply,
one-third of public school students nationally are minorities, while half of charter
school students nationally are minorities” (Finn, et. al. 1997).
The study draws information from 10 different states and, therefore, can be
generalized more easily, but as with the PRI study, some questions can be made on
generalizing the entire student population based on a limited number of parent
surveys. Also, no indication was made whether the surveys were translated in any
other languages.
Study #4: Center fo r Washington Area Studies o f Charter Schools in D. C. by Henig,
et. al., 1999
The Center for Washington Area Studies looked at 18 charter schools in the
District of Colombia during their first full year of operation. The study used semi
structured interviews, site visits and data collected from secondary sources to analyze
issues of school financing, facilities, governance and curriculum. First, the researchers
concluded that, “for those that fear that charters will cater to an already-favored elite,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
37
the evidence from DC to date is generally reassuring” (Henig, et. al., 1999). This
conclusion was based on the following data: (a) eighty percent of charter school
students in DC are black compared with 85% of public school students in DC, (b)
sixteen percent of charter school students in DC are Hispanic compared with 9% of
public school students in DC, (c) one percent of charter school students in DC are
white compared with 4% of public school students in DC, and (d) thirteen percent of
charter school students in DC are limited English students compared with 7% of
public school students (Henig, et. al., 1999).
The data collected support the conclusion drawn in the study. The researchers
are careful not to generalize their conclusion and the research supports the conclusions
of the PRI study and the Hudson study in that charter school enrollment reflects the
enrollment of the general population.
The second conclusion drawn by the study was that, “while DC charters are
serving many low-income DC students, some low-income wards may be under
served” (Henig, et. al., 1999, p 29). The second conclusion of the study is based on
the following data: (a) wards 1, 2, and 6, which are racially and economically mixed,
all had four charter schools open within their wards, (b) wards 4 and 5, which are
predominantly black and middle SES, had 2 and 4 charter schools open respectively,
(c) ward 3, which is predominantly white, didn’t have any charter school open within
the ward, and (d) wards 7 and 8, which are predominantly black and low SES, didn’t
have any charter schools open within their wards (Henig, et. al., 1999).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
38
While the research collected does support the conclusion drawn in the study,
the researchers are careful to state that the data collected is only on the first year of
charter implementation and that “whether this is a long-term trend or not is uncertain”
(Henig, et. al. 1999, p. 29). They also note that selection of the location of charter
schools seems to be based on pragmatic considerations of building availability and
access rather than on race.
The second conclusion of the study is a unique one within the body of charter
school research. No other study noted within this paper mentioned any analysis of the
location of charter schools within urban settings. Studies have looked at whether
charter schools are proportionally represented within a state in urban, suburban and
rural communities, but no other studies have mentioned a microanalysis of charter
school representation within urban, suburban or rural communities. Perhaps, it is an
area worth further study on a nationwide level.
Study #5: The State o f Charter Schools. Fourth Year Report. Conducted by RPP
International with Nelson, et. al., 2000.
The US Department of Education study, conducted by RPP International, is a
4-year study with yearly reports issued on the status of charter schools in the United
States. The study is also done in survey format, but it is conducted via telephone with
every charter school operating in the US during the year of the study. The results cited
here are from the final installment of the four-year study by Research, Policy and
Practice, International, funded by the Office of Educational Research and
Improvement of the U.S. Department of Education. The study looked at 1,957 charter
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
39
schools operating in the U.S. from 1995-1999. See Table 4 for the findings on
minority enrollment in charter schools nationwide.
Table 4: Minority Enrollment Nationwide in Charter and Non-Charter Schools
Blacks, Not
Hispanic
Hispanic Asian Native American
Charter schools 27% 16% 2.4% 3.5%
Non-charter 17% 16% 3.4% 1.5%
A conclusion of the study was that minority students represented 48% of the
student population in charter schools, as compared with 38% of non-charter schools
nationwide. The data, although a few years old, represents conclusive information on
charter school enrollment on a national level.
Study #6: UCLA Charter School Study by Amy Stuart Wells, 1999
The UCLA Charter School Study, by Amy Stuart Wells, looked at 17 schools
in California selected with specific intent to show the difference between charter
schools in the state (1999). The study is “not about the average experience of charter
schools in this state, but rather [on] the range of experiences that have come to define
a diverse movement in a diverse state” (Wells, 1999, p. 15). The study looked at the
schools over a 2 1/2 year period by conducting 462 semi-structured interviews,
observations of meetings and analyzing charter documents. It took issue with the fact
that charter schools, according to California law, “will achieve a racial and ethnic
balance among its pupils that is reflective of the general population residing within the
territorial jurisdiction of the school district to which the charter petition is submitted”
(2000, p. 2). As a result, the study drew three conclusions on ethnicity in charter
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
40
schools: (a) the study questioned whether all families have access to increased
educational choice through charter reform, (b) charter school’s racial/ethnic
requirements have not been enforced, and (c) start-up charter schools differ
significantly in minority enrollment than conversion charter schools.
The study drew the first conclusion based on the fact that students at the 17
schools selected can be denied enrollment based on whether the parents at the schools
can meet the parent volunteer requirements of the school. The study raised this issue
as a concern even though there was no data cited in the study on how many students
have been denied enrollment based on the volunteer requirement. There was no
information indicating how this policy of charter schools affects the enrollment of
students or how it may affect minority parents with multiple jobs more harshly than
white parents. The concern may be legitimate, but without any evidence, it is
speculative and alarmist.
The second and third conclusion drawn from the study was based on the
following collected information: (a) in 10 of the 17 schools, at least one racial or
ethnic group was over- or under-represented by 15 percent or more in comparison to
their district’s racial make up, (b) five of the 17 schools, all of them start up schools,
didn’t receive any free/reduced lunch funding, (c) eight out of the 17 schools had
similar percentages of limited English students as their districts, (d) five out of the 17
schools, all start up schools, served percentages of LEP students that were at least 15
percent less than their district averages (Wells, 1999).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
41
The second conclusion of the study is based on a strict reading of the
California law. When following such a strict reading of the law, the data supports the
conclusion of the study. One question not raised by the study was whether every
charter school’s enrollment should reflect the entire district’s enrollment. According
to the law, charter schools would have to be similar to magnet schools, which are
required to maintain a percentage of white and minority enrollment, but charter
schools don’t have the assistance of busing programs. Another issue not raised by the
study is the other requirement of conversion charter schools in California to give
preference to students from their local community first. If the charter school is
required to serve students from the local community, how can the school truly reflect
the enrollment of the district? Also, if a charter school were targeting a specific
population of students, the racial/ethnic requirement would be skewed. As a result,
the conclusion that the racial/ethnic requirement has not been enforced is inconclusive.
The third conclusion involving the differences between start-up charters and
conversion charters is an interesting one. For clarification purposes, a start-up charter
school is one that creates a school where none existed previously whereas a
conversion school is one that originally was a local district public school that voted to
become a charter school. Given the fact that the racial make-up of start-up and
conversion schools was so divergent, further study is required, especially since the
conclusions were drawn from such a small representative sampling of schools. (It
should be noted that the RAND study, conducted by Zimmer and fellow researchers,
specifically addressed this issue—see Study #1 above).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
42
Study #7: Brookings Institute Study by Tom Loveless, 2002
The Brookings Institute Study from Brown University looked at 638 schools
across 10 states and primarily looked at test data in order to analyze student
performance. In addition, researchers collected demographic data in order to compare
charter and non-charter schools. The charter school demographics were then
compared with national statistics on public school demographics. The researchers
found that “the charters serve a higher proportion of black students (23% vs. 15 %
nationally and Hispanic students (18% vs. 15% nationally) than the average public
school” (Loveless, 2002, p. 31).
The representative sample of the study is quite large (638 schools) and spans
10 states, which allows for the study to be generalized more reliably. The comparison
of the charter school demographics to the national demographics only indicated that
these identified schools collectively serve a greater percentage of minority students
than the national average. The study does not compare charter demographics with
non-charter demographics within the identified 10 states, which would have been a
better indication of whether the charter schools served a greater percentage of minority
students than non-charter schools. The general conclusion that charter schools are
nonetheless serving a significant percentage of minority students can be made.
Study #8: Rhetoric Versus Reality by Gill, et. al., 2001
Brian Gill and fellow researchers examined existing research on charter
schools from 27 states. The researchers only considered those studies that they
deemed to meet high standards of research protocol. The issue of minority enrollment
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
43
in charter schools was examined under the umbrella of minority access to charter
schools. Access has been promoted as a viable goal within the charter school
movement. Brian Gill and his fellow researchers indicate that, “charter schools, too,
have been supported in part to promote choices for low-income parents” (2001, p.
141). In referencing the arguments of Coons, Sugarman and Joe Nathan, charter
schools provide valuable options for low-income, predominantly minority parents,
who otherwise, do not have the options availed to middle and upper-income families
(Gill, et. al., 2001).
With the issue of access in mind, the study examined charter legislation and
prior research to draw some conclusions on access. According to the study,
“aggregated nationally, the proportion of charter-school students whose family income
qualifies them for a free or reduced lunch is nearly identical to the proportion in
conventional public schools: 39 percent of charter-school students were eligible in
1998-99, compared with 37 percent of public-school students” (Gill, et. al. 2001, p.
153). Furthermore, the study found that charter schools, nationally, enroll slightly
more minority children that non-charter schools. When the study examined state-by-
state data, there were greater disparities in minority and low-income student
enrollment among the charter and non-charter schools. Eight of 13 states with at least
20 charter schools had at least 10 percent more minority students than the
conventional public schools, while two of the thirteen states had at least 10 percent
less minority students than the conventional public schools. It should be noted that
one of the two states with 10 percent less minority enrollment was California.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
44
According to the most recent data available on the California Department of Education
website, this disparity is no longer greater than ten percent. Currently, California
charter schools have a total minority enrollment of 60.1%, while California non
charter schools serve 66.3% minority students (California Department of Education,
2003).
In terms of low-income students, 11 out of 27 states had a proportion of
students eligible for free or reduced lunch of at 10 percent higher in charter schools
than non-charter schools. Among the 27 states, six had ten percent fewer students
receiving free or reduced lunch. The conclusion drawn from the data is that, in many
cases, charter schools do provide “new options for low-income and minority students
who might otherwise lack a choice” (Gill, et. al, 2001, p. 155).
Conclusions on At-Risk Enrolment in Charter Schools
The eight studies cited above provide compelling data on a national scale that
minority students are well represented in charter schools. Furthermore, in the majority
of states with charter programs, the charter school minority and low-income
enrollment meet or exceed statewide averages. As the California study indicates, there
may be discrepancy in terms of which racial/ethnic group is better represented in
charter schools, but there is no doubt that charter schools are serving a comparable
percentage of minority students nationally and within most states. Furthermore, the
California study suggests that start-up and conversion charters schools may serve
difference populations of students, which has not been examined on a national level.
Finally, as the Washington D.C. study insinuates, there may be greater disparity in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
45
charter school representation within an urban setting (i.e., by community, ward or
barrio), but there is not significant research on a national level to draw such general
conclusions.
Academic Achievement in Charter Schools
As stated in Table 1, twenty-two charter laws proclaim that charter schools are
created to improve student achievement (Hill and Lake, 2002). How charter schools
are meeting the academic needs of students has been an issue of debate since the onset
of charter schools (Berman, 1998; Finn, et. al., 1997; Gill, et. al., 2001; Miron and
Nelson, 2001). For the longest time, there was very little data available on charter
school performance. Either there wasn’t a critical mass of charter schools in any
particular state to make a valid, statistical comparison, or there wasn’t a consistent
indicator or instrument used by both charter and non-charter schools. As a result, the
initial studies of charter school performance measured student achievement through
the use of surveys: teacher surveys, parent-satisfaction surveys and student-
satisfaction surveys (Finn, et. al., 1997; Riley, 2000). Recently, several statewide and
multi-state studies have used standardized tests to compare student achievement in
charter and non-charter schools. By analyzing the research methods, depth and scope
of the studies, some general conclusions can be drawn on how charter school
performance compares with non-charter school performance.
Study #1: Charter School Operations and Performance: Evidence from California.
Conducted by RAND by Ron Zimmer, et. al., 2003
Student achievement was one area of study within the RAND study comparing
California’s charter schools and conventional schools conducted by Ron Zimmer and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
46
fellow researchers (2003). Under the area of achievement, the study examined three
indicators of student performance available in both the charter schools and
conventional schools: Academic Performance Index (API) growth data, school-wide
Stanford Achievement Test Nine (SAT9) data on individual students, and individually
matched, longitudinal student data from six districts throughout the state. Charter
schools, as a group, were compared with similar conventional schools. In addition,
charter schools were disaggregated into conversion, start-up and non-classroom based
schools to determine if academic performance varies across the different types of
charter schools. The study recognized some important limitations to the analysis of
academic performance data. First, the researchers recognize that there is no single
charter school effect. Charter schools vary greatly across the state, not just in terms of
classification (i.e., conversion, start-up, non-classroom based), but also in terms of
individual charter school identity. Second, the researchers recognize that naturally
occurring randomization of students is not possible in charter schools, given the fact
that parents have a choice to matriculate their children in charter schools.
Furthermore, the study recognized that individually matched, longitudinal student
testing data is not available statewide, reducing the “ability to control for unobservable
differences among individual students” (Zimmer, et. al., 2003, p. xxii).
The study compared 352 charter schools with 245 matched conventional
schools. The charter schools and conventional schools were matched using a
propensity score. The propensity score was determined for each charter and non
charter school according to the eight levels of students served (elementary schools,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
47
middle schools, high schools, county schools, continuation schools, juvenile hall
schools, special education schools, and alternative education schools) and according to
the demographics of the schools using ethnicity, socioeconomic status and percentage
English Learners. Mobility rates, class size, year-round calendar operation and
percentage of non-credentialed teachers were eliminated as matched indicators given
the fact that the philosophy or mission of many charter schools incorporate these
indicators as essential policy components. The final result of the comparison allowed
multiple charter schools to be matched to a single conventional school, thus the
difference in total numbers (352 charter schools vs. 245 conventional schools). Based
on the final matching, RAND sought permission from the identified schools to be part
of the study. One hundred and sixty one charter schools and 161 conventional schools
were included in the final study.
The first analysis of the data involved the use of the Academic Performance
Index (API) growth scores. The study examined API using all the schools in the state,
not the matched schools identified for the study. By comparing year-to-year changes
in API scores in both charter and conventional schools, a broad picture of student
performance is created. The data indicated “no statistically significant difference in
test scores between charter and conventional public schools. However, the
aggregation of a composite score at the school level masks variations in important
characteristics within schools and distorts linkages between student characteristics and
student outcomes” (Zimmer, et. al., 2003, p. xxii). Since the API aggregates so many
different scores, which have been computed in different ways each year from 1999 to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
48
2002, the comparison fails to indicate why outcome results vary across schools. In
addition, the API fails to provide scores for a significant number of charter schools,
since 25 percent of all charter schools were in the first or second year of operation and
didn’t have two consecutive years of data. Given the concerns with the API data, the
researchers analyzed SAT9 scores to reveal a more complete picture of student
performance.
The SAT9 data, consistent from 1998 to 2002, allows for a longitudinal
analysis of student performance. Furthermore, the data links performance and
demographic information, allowing for a more accurate and detailed comparison of
schools. The SAT9 test scores indicate that charter schools, in general, have
comparable or slightly lower results than non-charter schools. Furthermore, the study
draws some conclusions on student performance among the various types of charter
schools. Conversion schools have mixed results, performing better, the same and
worse than matched conventional schools. Start-up schools do better than
conventional schools in all areas, except elementary mathematics where the scores are
lower than matched conventional schools. Finally, charter schools with non
classroom based instruction have lower scores across all levels of programs. The
researchers note that schools with non-classroom based instruction include missions to
serve a traditionally underserved population of students. Also, the researchers are
careful to note that, without individually matched student data that would track
individual student progress from year to year, there is not a clear indicator of baseline
performance for the students. Without an accurate indicator of baseline performance,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
49
the achievement background of students is not known, complicating the issue of
selection bias in the comparison of charter and conventional schools. As a result, the
researchers created a third comparison of achievement.
Despite the lack of individually matched student data on a statewide level, the
researchers analyzed individually matched student data available from six school
districts. The longitudinal analysis allowed researchers to identify the level of
baseline performance of the students and provides the “best estimates of the effects of
charter schools” (Zimmer, et. al., 2003, p. 53). The six districts provided significant
data on elementary charter schools, but only two districts provided substantial data for
secondary charter schools students. Furthermore, the same two districts were the only
ones with data to compare start-up and conversion schools, and there was insufficient
data to compare non-classroom based charter schools. Despite the limited data set, the
results indicated that charter school students had comparable results in elementary and
secondary reading and math scores. “The district-level results suggest that much of
the difference in achievement from statewide individual data was related to
unobserved student factors” (Zimmer, et. al., 2003, p. 56).
The multifaceted approach by the researchers to the data creates a strong
indication of the academic performance of charter schools within the state of
California. The conclusions drawn by the researchers that: (a) charter schools are
performing as well as conventional schools and (b) non-classroom based charter
schools score significantly lower than conventional schools are reasonable. The
individually matched student data, although limited in size and scope, provides the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
50
most accurate indication of charter school performance relative to conventional
schools. The researchers are making a generalization. The suggestion that six,
predominantly urban and unified school districts represent the performance of charter
schools and conventional schools in the entire state is exaggerated, but when viewed
in relation to the analyses of the larger API and SAT9 data sets, the conclusions
appear more reasonable. The statistical analysis of the study is strong and the
comparison schools seem to be well identified. The researchers identified a major
limitation of the study—the lack of individually matched longitudinal data on a
statewide level. In addition, the study does not consider the years of operation of the
charter school as a factor, one which has been identified as a significant factor in
previous studies (See Gill, et. al., 2001 and Loveless, 2002), nor do the researchers
disaggregate the performance data by grade level, ethnicity, community or
socioeconomic status. Despite these limitations, the study represents the most
thorough analysis of charter schools within the state of California to date.
Study #2: Does Charter School Attendance Improve Test Scores? The Arizona
Results by Solomon, et. al., 2000
Lewis Solomon, Kem Paark and David Garcia compared charter and non
charter schools within the state of Arizona. The longitudinal study analyzed student
achievement data based on Standard Achievement Test, Ninth Edition (SAT9) from
1997 to 1999. Arizona administers the SAT9 to all students in grades 3-12, matching
student data for 40,305 students, 13.6 percent of whom attended charter schools.
Furthermore, the state data provides a means to track individual students allowing the
researchers to a) identify a baseline performance score for individual students, b)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
51
measure the “charter effect” over time, and c) determine the effect of mobility
between charter and non-charter schools.
The regression analysis of individually matched student data over time, using
fixed effects and random effects models as well as models with time and without time
effects, creates a comprehensive, statistical study recognized for its strengths by many
researchers (see Gill, et. al., 2001 and Miron and Nelson, 2001). The mobility factor,
tracking students from traditional public schools to charter schools and vice-versa, is
well analyzed and described in the study. Also, the study controls for individual
student traits, such as race, years in district, absences, primary language, special
education status, gifted status and grade level, by using a covariance method that
includes student traits as a regression covariate.
The researchers drew several conclusions from their research. First, in the area
of reading, charter school students show a significant decline in performance during
the first year of attendance, but by the third year, have made greater progress in
reading than traditional, non-charter school students (Solomon, et. al., 2000). In the
area of mathematics, there is a similar decline in the first year of charter school
attendance, a slight improvement in performance during the second and an
insignificant decline during the third year compared to traditional public schools. The
researchers indicate that the first year decline in charter school performance may be
due to the effects of mobility, which “generally results in a decline in test scores
during the first year in a new school” (Solomon, et. al., 2000, p. 4). The researchers
further analyzed the effects of mobility by examining changes in achievement as
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
52
students moved from traditional to charter schools and from charter to traditional
schools over the three years. The researchers found that “movement of all types had a
negative effect on scores and the positive effect of charter schools outweighed the
negative effect of moving (Solomon, et. al„ 2000, p. 20).
The study carefully addressed the issues of selection bias and mobility. Unlike
the RAND’s California study (Study #1 above), individually matched data was
available in Arizona statewide and allowed for a thorough analysis of the charter effect
and the mobility effect of students moving between charter and traditional schools.
Unlike the California study, the researchers did not break down the data according to
the type of charter school. Since Arizona has a “strong” charter school law, there is
high degree of diversity of charter schools within the state and the charter type may
affect performance, as it did in the California study (Bierlein, 1997; Jennings, et. al.,
1998). Although the study controlled for race, ethnicity, EL status, special education
status, gender and grade level, the data wasn’t disaggregated using those indicators.
Despite the limitations, the Arizona study remains on of the most thorough
comparisons of charter and non-charter school performance.
Study #3: Apples to Apples: An Evaluation o f Charter Schools Serving General
Student Populations by Greene, et. al., 2003
Greene, Forster and Winters examined the performance of charter schools
students in comparison to non-charter schools students in eleven states with a
significant number of charter schools and sufficient achievement data: Arizona,
California, Florida, Texas, Michigan, New Jersey, Ohio, Colorado, North Carolina,
Minnesota, and Pennsylvania. The researchers eliminated from the comparison any
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
53
school that targeted a specific population of students and all schools without two
consecutive years of test score data. Conversion schools and non-classroom based
schools were also eliminated from the list because they, respectively, do not work
under open-enrollment policies and do not have schools of comparison. Phone
surveys were done to verify the student population served by the schools. Comparison
schools were identified geographically, meaning that the traditional school of
comparison would be the traditional school located closest to the charter school,
making the assumption that geography is a strong indication of demography.
Standardized test scores were used to compare the performance of charter and
non-charter schools. By identifying grade levels within the schools that had available
longitudinal data, the researchers compared the growth of charter and non-charter
schools (for example, the change in third grade scores from 2001 to 2002). A
regression analysis was computed controlling for school type, race (i.e., the percentage
of white students) and grade level. The researchers found that “charter schools
serving the general student population outperformed nearby regular public schools on
math tests by 0.08 standard deviations, equivalent to a benefit of 3 percentile points
for a student starting at the 50th percentile. These charter schools also outperformed
nearby regular public schools on reading tests by 0.04 standard deviations, equal to a
benefit of 2 percentile points for a student starting at the 50th percentile” (Greene, et.
al., 2003, p. 1). The greatest growth, statistically significant at the <.05 level, came
from Florida and Texas.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
54
The researchers should be commended for recognizing an important distinction
between charter schools and traditional schools—the population of student served. By
limiting the comparison to schools that do not target a specific population, ideally, the
study compares like groups. Also, by comparing schools geographically, there is a
greater potential of comparing like populations. Furthermore, the study measures
academic achievement in terms of longitudinal growth over two years, potentially
limiting the effects of selection bias. Unfortunately, there are numerous
inconsistencies and limitations to the study.
First, the study doesn’t look at individually matched student data. As a result,
there are no controls for mobility and calls into question the issue of selection bias,
since no baseline performance can be established. Even though individual matched
student data wasn’t used, the study could have been improved by measuring potential
matched scores within a school. So, instead of comparing third grade performance in
2001 to third grade performance in 2002, the study could have better measured growth
by comparing third grade performance in 2001 to fourth grade performance in 2002,
potentially comparing the academic growth of the same students.
Second, despite attempts to create fair comparison groups by using geography
as the bell weather of similarity, there are potential problems that are not addressed by
the researchers. The only criterion used in the comparison was whether the nearby
traditional school served an untargeted population (i.e., not a magnet school or school
for juvenile delinquents) and had longitudinal test scores available. The researchers
did not look to demographic data to verify that the school did, in fact, serve a similar
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
55
population of students, nor did the study consider the size of the school as a
comparison factor. In rural districts, the closest traditional school could be
significantly distant from the charter school, potentially serving a different population
of students.
Furthermore, the study examined charter schools with open-enrollment
policies, excluding conversion schools. Since traditional, non-charter schools, in most
cases, accepts students based on a defined attendance boundary or zone, how is the
comparison accurate if the charter schools and non-charter schools have different
attendance policies and potentially serve distinct populations? Without the additional
demographic data, the comparison remains vague, providing only an indicator of
competition between two schools that geographically may be similar, but
demographically, may be different. At least conversion schools frequently have the
same attendance policies as traditional public schools, but they were excluded from
the study. Furthermore, the study doesn’t consider the state policies that can affect
attendance. Although the researchers claim that most states require charter schools to
have open-enrollment policies, among the eleven states studied, the research by
Jennings, et. al., indicates that only one of the states has a true open enrollment policy
(3 with the exclusion of conversion charter schools) while 8 have some form of
attendance requirement, either requiring preferences for district or local students
(1998). In addition, six of the eleven states have racial/ethnic requirements that may
impact enrollment (Jennings, et. al., 1998).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
56
Finally, the general conclusion drawn on the national performance of charter
schools is skewed by the strong performance of two states within the study: Texas and
Florida. These states had statistically significant performance data, yet the majority of
the states had scores that were positive, but not statistically significant. Without the
Texas and Florida scores, was the national data still statistically significant? Given the
numerous questions and limitations to the study, the results should be considered with
a high degree of caution.
Study #4: Rhetoric Versus Reality by Brian Gill, et. al., 2001
Brian Gill, Michael Timpane, Karen Ross, and Dominic Brewer, looked at
studies of academic achievement results in 3 states: Arizona, Texas and Michigan.
The studies were chosen based on methodological reliability factors, taking into
consideration issues of longitudinal data, selection bias, randomization and statistical
analysis. Given the limited data set from only 3 states, the researchers are hesitant to
generalize their conclusions and recognize that further studies need to be conducted.
The findings from the 3 statewide studies on achievement are mixed. The
studies indicate that charter school students have test scores below those of public
school comparison groups. Furthermore, the studies suggest that the academic
performance of charter school students improves after the first year of operation. In
conclusion, none of the studies suggest that academic achievement is “dramatically
better or worse on average than those of conventional public schools” (Gill, et. al.,
2001, p. xiv).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
57
Study #5: The 2002 Brown Center Report on American Education: How Well Are
American Students Learning? By Tom Loveless at the Brookings Institute, 2002
The Brookings Institute study looked at 3 years of achievement data on 376
charter schools operating in 10 states. The study recognized selection bias as a
validity factor and took into consideration a weighted mean in order to compensate for
school size. Z-scores were computed for regular and charter schools in each state and
compared individually and as a whole.
The study found that, “charter schools score significantly below regular public
schools on achievement tests, about .24 z-scores below average” (Loveless, 2002, p.
32). Looking at the individual state results, charter schools in 4 states (Massachusetts,
Michigan, Minnesota, Texas) had scores significantly below state averages, and the
other 6 states (California, Colorado, Florida, Wisconsin, Arizona, Pennsylvania) had
results essentially indistinguishable from the state average. When the data was
disaggregated by community type (urban, suburban, rural), school size and infancy of
the charter school, the researchers found that charter schools in urban communities
performed better, by comparison, than suburban and rural communities. Furthermore,
large schools tended to perform better than small schools and new charter schools
tended to have depressed scores for the first two years of operation. The study states
that, “charter schools are scoring below average on tests of academic achievement, but
why they do so remains a question. It could be because charters offer an inferior
education, or it could be because charters attract students who are low achieving in the
first place” (Loveless, 2002, p. 35).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
58
The Brookings study is one of the largest studies that examine charter
performance across the country. The study is strengthened by disaggregated data that
examines the effects of community type, school size and years of operation. The
conclusions of the study are overly generalized. Claiming that charter schools have
below average test scores ignores the researchers own findings that a majority of the
states (6 out of 10) had charter test scores that paralleled non-charter schools.
Furthermore, the study compared a select group of charter schools to all the non
charter schools in the state, ignoring the important distinction between the students
served by charter and non-charter schools. Interestingly, three out of the four states
referenced with depressed test scores (Minnesota, Michigan, and Texas), each have a
charter population that serves a greater percentage of minority and at-risk students (see
Nathan and Boyd 2003; Horn and Miron, 2001; and Greene, et. al., 2003 respectively).
Furthermore, the study did not disaggregate the data for minority or at-risk students
nor did the study differentiate according to the different types of charter schools. In
conclusion, the study provides a broad picture of charter performance across 10 states
and raises some important points on the performance within urban, rural and suburban
communities and the possible effect of school size and years of operation of a charter
school.
Study #6: Student Academic Achievement in Charter Schools: What We Know and
Why We Know So Little by Gary Miron and Christopher Nelson, 2001
Gary Miron and Christopher Nelson examined existing large-scale studies that
compared charter school performance to non-charter school performance using
standardized tests (2001). The studies were weighted based on the methodology of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
59
each and their ability to address validity and design concerns. Furthermore, state
legislation was taken into consideration in order to identify policy-related conditions
that may affect charter school achievement. Fifteen studies from 7 states and the
District of Colombia were found to meet the comprehensive criteria of the researchers.
The first conclusion of the study is that there is very little adequate data on
student achievement in charter schools. The implication is made that generalizations
on charter schools are difficult to make without a comprehensive data set from
multiple studies. Recognizing this limitation, Miron and Nelson draw a restrained
second conclusion that “the existing body of research reveals a mixed picture, with
studies from some states suggesting a positive impact, studies from other states
suggesting negative impact, and some providing evidence of positive and negative
impacts. Overall, the charter impact on student achievement appears to be mixed or
very slightly positive” (2001, p. i). Among the strongest methodological studies, the
greatest positive impact is seen in Arizona, while the greatest negative impact is found
in Michigan, suggesting further analysis of policy conditions that may impact student
performance in charter schools.
Study #7: California Charter Schools Serving Low SES Students by Slovacek, et. al.,
2002. Follow up analysis by David Rogosa, 2002
Slovacek, Kunnan and Kim conducted a longitudinal study comparing the
performance of charter schools in California with regular, public, non-charter schools.
The study analyzed the Academic Performance Index (API) data for 3 year for 93
charter schools and compared the API growth of the charter school students, the socio
economically disadvantaged students and socio-economically disadvantaged schools
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
60
with their like counterparts in regular public schools statewide. A follow-up analysis
by David Rogosa critiqued the study, questioned the process and results and offered
alternative conclusions.
Slovacek, et. al. (2002) made two findings related to minority performance
from the data collected: (a)“Califomia charter schools are doing a better job of
improving the academic performance (as measured by API) of California’s most at-
risk students” (p. ii), and (b) students achievement (as measured by API) in
California’s low-income charter schools is, on average, improving at a faster rate than
in similar non-charter schools” (p. ii). Slovacek’s data indicated that charter school
API growth was 13.1% as compared to the 11.9% growth of non-charter schools.
Furthermore, charter schools serving low-income students grew 22.6% as compared
with the 19.4% growth in non-charter schools serving low-income students.
Rogosa (2002) took issue with Slovacek’s findings. Rogosa corrected the data
set and analysis strategy used by Slovacek and refuted the conclusions drawn in the
study. The corrections made by Rogosa to the data set appear to be sound and the
results are presented in a more comprehensive manner. Rogosa’s findings support
Slovacek’s assertion that “charter schools are doing an effective job of improving the
academic performance of low-income students” (2002, p. 11).
Both studies look at statewide data and compare the performance on a
statewide level. Such a broad analysis provides a general comparison of the academic
performance of charter schools with all schools. Although students were matched by
grade in the Rogosa study, race, ethnicity, English learner designation and learning
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
disabilities were not disaggregated in either study. Furthermore, the statewide
comparison didn’t consider the varying levels of student achievement within rural,
suburban and urban communities across the state. As a result, charter school students
and schools are not compared in like communities.
Study #8: Reducing the White-Minority Achievement Gap in North Carolina by C.S.
Bingham, P. Harman, P. Finney and A. Hood, 2001
The North Carolina Study (Bingham, et. al, 2001) looked at the achievement
gap between white and minority students in charter schools and non-charter schools.
The study compared charter school white-minority achievement gap with the white-
minority gap of schools within the same district. Standardized test scores were used to
measure the white-minority gap. The study considered only those charter schools that
had been functioning for at least two years and test results from only the 1999-2000
school year were used, indicating that comparison, not longitudinal improvement, was
the focus of the study.
Although 6 charter schools had a lesser gap and 27 charter schools had a
greater gap than the schools in their like districts, the researchers were cautious to
draw any conclusion from the study. In fact, Bingham states that, “although it is
tempting to conclude that North Carolina charter schools generally fail to reduce the
white-minority achievement gap (and, in fact, appear to increase the gap), the analyses
simply do not support such an assertion” (2001, p. 8). Furthermore, Bingham
recognizes that other factors such as poverty and learning disabilities and specific
location were not controlled for in the study (2001).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
62
The North Carolina study reveals a more complex picture than a statewide
study by showing the breakdown within each district. The study shows a snapshot of
performance in charter schools and non-charter schools within each district in the
state. The one-year snapshot indicates that charter schools have a larger white-
minority achievement gap within most districts, but since the study is not longitudinal
and doesn’t disaggregate student performance data, the lack of resolution by the
researchers is fair and accurate.
Study #9: Pacific Research Institute Study by Pam Riley, 2000
Since the PRI study was based on a comprehensive survey conducted at 100
charter schools in California, the research findings use the survey results of parents to
demonstrate academic success. The study came up with the following results: (a)
sixty-six percent of parents were very satisfied with their child’s charter school, (b)
twenty-eight percent of parents were somewhat satisfied, (c) thirty-four percent of
parents perceived that their child was above grade level compared to only 24% who
said they were above grade level at their previous public school, and (d) fifty-five
percent of parents perceived that their child was on grade level compared to only 39%
who said they were on grade level at their previous public school (Riley, 2000).
The data certainly indicates that most parents (94%) are somewhat or very
satisfied with their child’s charter school. In addition, the data indicates that parents
perceive that their child is doing better at the charter school than at the previous public
school. The PRI study (2000) draws the conclusions that “parents report a high level
of satisfaction with their child’s experience in the charter school” (p. 16) and that
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
63
“parents say that academic performance improves when students enroll in a charter
school” (p. 18).
These two conclusions are reasonable given the extensive quantity of surveys
conducted, although the results are specific to California. Given the fact that charter
laws differ substantially between states, the conclusions are relevant to California
charter schools, but not to US charter schools as a whole. In addition, there were no
qualifiers for the parents to gauge their responses. They didn’t need to say that the
children were receiving better grades or performing better on standardized tests.
Unfortunately, no other conclusions can be drawn from this study on the
academic performance of children in charter schools. The survey didn’t ask the
principals or the teachers to comment on the academic performance of children, nor
was the school asked to provide any data on student performance, so the only indicator
in the PRI study of student achievement, including the academic progress of minority
children, is the parent’s perception of academic growth.
Study #10: Hudson Institute Study by Chester E. Finn, Bruno V . Manno, Louann A.
Bierlein, And Gregg Vanourek, 1997
The Hudson Study, like the PRI study, used surveys to judge academic
performance (Finn, et. al., 1997). The study looked at 17 schools across 10 states over
a two-year period. The study drew the following conclusions: (a) the number of
students doing ‘excellent’ or ‘good’ work rose 23.4% for African-Americans and
21.8% for Hispanics after enrolling in charter schools, (b) the students and their
parents, of all ages, races and genders, limited English proficiency, and other special
needs and family income, report that they are doing better than in their previous
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
64
schools, (c) among students performing “poorly” in their previous school (as judged
by their parents), nearly half are now doing “excellent “ or “above average” work”, (d)
it is unclear how well students in charter schools are learning or whether their
academic achievement will surpass that of similar youngsters enrolled in more
conventional schools, and (e) early signs on the achievement front are promising
(though comparable test scores are not yet available). For example, about a dozen
charter schools have had their charters renewed, suggesting that they are meeting their
student achievement goals (Finn, et. al., 1997).
The conclusions drawn by the Hudson study demonstrate that, like the PRI
study, parents perceive that their children are doing better at the charter school. In
addition, the Hudson study surveyed the children at charter schools and also found that
the children perceived themselves performing better at the charter school. Unlike the
PRI study, the Hudson study breaks down the data into demographic categories
demonstrating that over 20% of parents of Hispanic and African-American children
perceive that their children are now doing “excellent” or “good” work. In fact, the
Hudson study broke down the surveys in order to consider socioeconomic status
(SES), limited English students (LEP), racial/ethnic groups and special education.
Within every area, over 93% of parents and children perceive improved academic
performance. Because the Hudson study disaggregates the data and included charter
schools from 10 different states, more general conclusions can be drawn from the
Hudson study about parent perception and student perception of academic
performance.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
65
The Hudson study also draw the important conclusion that “we do not yet
know how much and how well students in charter schools are learning or whether their
academic achievement will surpass that of similar youngsters enrolled in more
conventional schools” (Finn, 1997, p. 3). The authors of the study recognize that
parent and student perception are not direct indicators of student performance.
Without any objective data using some norm-referenced or criterion-referenced test, it
is difficult to draw any significant conclusions regarding academic performance. The
study does draw a tentative conclusion on performance by suggesting that charter
schools that have had their charter plans renewed may indicate that schools and
children are meeting their performance goals. Given the embryonic state of charter
schools at the time and the lack of measures and procedures to renew certification of
charter schools, this conclusion is indeed tentative and most likely, not an accurate
indicator of student performance (see Wohlstetter, 1998 or Hill and Lake, 2002).
Conclusions on Academic Achievement in Charter Schools
By looking at the numerous studies that have measured academic achievement
via statewide tests and surveys, a broad picture is formed on charter schools as a
whole. First, parents, in general, are very satisfied with the education being received
at charter schools and perceive that their children are performing better at the charter
school. Given that parents have a choice to send their children to charter schools, this
conclusion is not surprising. Second, performance data indicates that charter schools,
in general, are making significant progress, but the results do not indicate that charter
schools are performing substantially better or worse than non-charter schools. There is
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
66
limited data available on the different types of charter schools. Only RAND’s
California study, conducted by Zimmer and fellow researchers, indicates that non
classroom based charter schools, which often serve a distinct at-risk population, don’t
perform as well as conversion or start-up charter schools and that start-up schools
slightly outperformed conversion schools in California. Furthermore, there seems to
be a strong indication that first year charter schools have depressed achievement
scores, which may be correlated to mobility effects.
At-risk Student Achievement in Charter Schools
Even though (a) there are various legislative measures to support at-risk
students in charter schools, (b) many charter schools target at-risk student populations,
(c) charter schools serve a comparative percentage of at-risk children and (d) charter
schools are making academic progress in comparison to non-charter schools, there is
very little known on how well at-risk students are performing within charter schools.
The temptation may be to say that since charter schools are making comparable
progress to non-charter schools, then the at-risk children within charter schools are
also making comparable progress. Given the different charter schools types, the
diversity of charter laws and the unique character individual charter schools that may
target specific populations of students, this conclusion cannot be made without
adequate disaggregation of the data. So, what does the existing research say about
how minority and low SES students are performing in charter schools?
Two of the studies (Zimmer and fellow researchers’ California study and
Solomon and fellow researchers’ Arizona study) use race and SES as statistical
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
67
regressors in the research. In the California study (2003), charter schools are
compared to similar conventional schools using race and SES as delineators for the
comparison. In addition, in Appendix C of RAND’s California study, the performance
data using the API and the SAT9 is disaggregated for racial/ethnic groups, EL students
and school lunch program as an indicator of SES, but no conclusions are drawn from
the disaggregated data (Zimmer, et. al., 2003). Although there are no conclusions
drawn on individual student groups, the study demonstrates the importance of
disaggregation. By disaggregating the performance data among the types of charter
schools (i.e., conversion, start-up, non-classroom based charters), the study
highlighted the difference in performance among the groups, a difference that may
have remained obscured. As a result, even though charter schools are performing as
well as conventional schools, the performance of individual groups of minority and
low-SES students remains unknown. Furthermore, there isn’t any indication of how
conversion schools, which service a greater percentage of minority students, meet the
needs of individual minority groups in comparison to start-up schools.
In the Arizona study (Solomon, 2000), individually matched study scores are
used to compare charter and non-charter school performance. The study controlled for
individual student traits, such as race, years in district, absences, primary language,
special education status, gifted status and grade level, by using a covariance method
that includes student traits as a regression covariate. Therefore, race, but not SES, was
used as an indicator of comparison, allowing for an accurate, but general comparison
of charter and non-charter schools. The data was not disaggregated for groups of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
68
students. As a result, the conclusion can made that charter schools are doing as well
as non-charter schools in terms of educating minority students as an entire group (as
measured by test scores), but there is no indication on how well low-SES and
individual minority groups actually perform in either charter or non-charter schools.
Furthermore, since the study did not differentiate between the different types of
charter schools, there is no means of determining whether conversion or start-up
schools, for instance, have an effect on minority or socioeconomic performance.
Among the remaining studies, only two disaggregate student performance
using racial or SES characteristics, the California study conducted by Slovacek, et. al,
and analyzed by Rogosa, and the North Carolina study conducted by Bingham, et. al.
In the California study (2002), the API data was disaggregated according to schools
that serve at least 50 percent low SES students and schools that serve at least 75
percent low SES students (Slovacek, et. al.; Rogosa, 2002). Examining the findings,
Rogosa concurs with Slovacek’s assertion that “charter schools are doing an effective
job of improving the academic performance of low-income students” (2002, p. 11).
The study drew the conclusion based on a limited data set (41 schools with at least
50% low SES and 25 schools with at least 75% low SES) and only used the API as an
indicator of performance. Although Zimmer and fellow researchers (2003) indicated
that the API fails to differentiate performance within a school, Rogosa used grade
level API data to more accurately measure student achievement. The study provides a
limited picture on how a small number of charter schools in California are improving
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
69
test scores for low-SES students, but does not provide any breakdown on minority
performance within those schools.
In the North Carolina study (2001), conducted by C.S. Bingham, P. Harman, P.
Finney and A. Hood, the white-minority achievement gap was measured as a means of
comparing charter and non-charter school performance. The one-year study
specifically addressed minority students as a group and examined the difference in
achievement compared to white students within each district in North Carolina.
Although 6 charter schools had a lesser white-minority gap and 27 charter schools had
a greater gap than the schools in their like districts, the researchers were cautious to
draw any conclusion from the study. Given the limited longitudinal scope of the
study, which doesn’t control for selection bias or measure the disaggregated
performance growth of each minority group, the study doesn’t provide any clear
insight into how charter schools in North Carolina are addressing minority
achievement. Furthermore, the study doesn’t examine SES as a factor in achievement.
Finally, only one national study, the Hudson study (1997) conducted by Finn,
et. al., examined minority performance using surveys. The Hudson study, examining
17 schools across 10 states, breaks down the data into demographic categories
demonstrating that over 20% of parents of Hispanic and African-American children
perceive that their children are now doing “excellent” or “good” work. In fact, the
Hudson study broke down the surveys in order to consider socioeconomic status
(SES), limited English students (LEP), racial/ethnic groups and special education.
Within every area, over 93% of parents and children perceive improved academic
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
70
performance. Because the Hudson study disaggregates the data and included charter
schools from 10 different states, more general conclusions can be drawn from the
Hudson study about parent perception and student perception of academic
performance. As indicated previously, the surveys conducted within the Hudson
study, only provide an indication of perception, not actual standardized performance.
As a result, the study is limited in its value toward creating an accurate picture of how
charter schools are addressing minority and low-SES achievement.
Conclusions on At-Risk Student Performance in Charter Schools
Examining the studies that compare charter and non-charter school
performance, there is little evidence on the performance of minority and low-SES
students in charter schools. The Hudson study (Finn, et. al., 1997) is the only national
study that disaggregates the data among minority and low-SES students, but the study
is limited in scope (only 17 schools across 10 states) and only indicates the perception
of minority and low-SES parents within charter schools. The study doesn’t provide
any objective data on the academic achievement of the specific groups. In fact, none
of the national studies that use standardized testing data (Greene, et. al., 2003;
Loveless, 2002; Gill, et. al., 2001; Miron and Nelson, 2001) disaggregate the data for
at-risk students. Only the few statewide studies from California, Arizona and North
Carolina provide any data on how charter schools are addressing minority and low-
SES achievement. The Slovacek (2002) and Rogosa (2002) studies provide a limited
picture of low-SES achievement, yet the other studies either fail to disaggregate the
data among minority groups or fail to draw any conclusions on the achievement of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
71
minority groups or low-SES students within charter schools. As a result, the only
general indicator of the performance of at-risk students is based on the statewide and
national comparisons of similar charter and non-charter schools, a potentially
inaccurate indicator of the actual minority and low-SES achievement results.
Conclusions
Charter Legislation Targeting At-Risk Students
Ten states include a specific goal to provide opportunities to at risk students.
The goal, in most states, translates into specific policies to support at-risk students.
Some, like non-discrimination policies and racial/ethnic enrollment requirements, are
direct means of addressing how charter schools will serve at-risk students. Others,
like admission policies based on location of residence, provide indirect policy tools
that impact at-risk enrollment in charter schools.
Non-discrimination policies are well written to protect at-risk students that
represent a minority group or have a disability. On the other hand, most policies do
not specifically protect English-leamers, low performing or low socio-economic
students in the majority of charter laws. In regard to admission, attendance at charter
schools is voluntary, but guided by a variety of attendance-boundary policies that
could impact at-risk enrollment. Finally, about a third of the charter laws use
racial/ethnic balance requirements to define at-risk enrollment at charter schools.
Conclusions on At-Risk Student Enrollment in Charter Schools
The numerous studies on at-risk enrollment in charter schools provide
compelling data on a national scale that minority students are well represented in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
72
charter schools. Furthermore, in the majority of states with charter programs, the
charter school minority and low-income enrollment meet or exceed statewide
averages. As RAND’s California study indicates, there may be discrepancy in terms of
which racial/ethnic group is better represented in charter schools, but there is no doubt
that charter schools are serving a comparable percentage of minority students
nationally and within most states. In addition, RAND’s California study suggests that
start-up and conversion charters schools may serve difference populations of students,
but this speculation has not been examined on a national level. Furthermore, as the
Washington D.C. study insinuates, there may be greater disparity in charter school
representation within an urban setting (i.e., by community, ward or barrio), but there is
not significant research on a national level to draw such general conclusions either.
Finally, in terms of charter legislation that addresses at-risk students (i.e., non
discrimination policies, attendance policies and racial/ethnic requirements), the
research, currently, does not indicate the actual effect of such policies. No study has
determined how attendance policies, for instance, actually impact at-risk enrollment.
Conclusions on Academic Achievement in Charter Schools
In terms of overall academic achievement in charter schools, performance data
indicates that charter schools, in general, are making significant progress, but the
results do not indicate that charter schools are performing substantially better or worse
than non-charter schools. There is limited data available on the different types of
charter schools. Only RAND’s California study, conducted by Zimmer and fellow
researchers, indicates that non-classroom based charter schools, which often serve a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
73
distinct at-risk population, don’t perform as well as conversion or start-up charter
schools and that start-up schools slightly outperform conversion schools in California.
Furthermore, there seems to be a strong indication that first year charter schools have
depressed achievement scores, which may be correlated to mobility effects.
Conclusions on At-Risk Student Performance in Charter Schools
Examining the studies that compare charter and non-charter school
performance, there is little evidence on the performance of minority and low-SES
students in charter schools. None of the national studies that use standardized testing
data (Greene, et. al., 2003; Loveless, 2002; Gill, et. al., 2001; Miron and Nelson,
2001) disaggregate the data for at-risk students. Only the few statewide studies from
California, Arizona and North Carolina provide any data on how charter schools are
addressing minority and low-SES achievement. The Slovacek (2002) and Rogosa
(2002) studies provide a limited picture of low-SES achievement, but the other studies
either fail to disaggregate the data among minority groups or fail to draw any
conclusions on the achievement of minority groups or low-SES students within charter
schools. As a result, the only general indicator of the performance of at-risk students
is based on the statewide and national comparisons of similar charter and non-charter
schools, a potentially inaccurate indicator of the actual minority and low-SES
achievement results.
Implications
There are several implications, based on the research, necessitating further
study. The first implication, addressed in this study, is to examine how at-risk
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
74
students are performing in charter schools. The most compelling research looks at
standardized, longitudinal data on a broad scale and compares like student groups (see
Zimmer, et. al, 2003 and Solomon, 2000). As a result, this study will replicate the
methodological design used in the RAND study by Zimmer and fellow researchers in
order to examine how at-risk students in California charter schools performed on 4
years of statewide tests in comparison to similar students in traditional or conventional
schools.
Second, not to be addressed in this study, is to conduct a microanalysis of at-
risk student enrollment in charter schools. Such a study should examine charter
school enrollment nationally according to charter school type (start-up, conversion,
non-classroom based), according to charter school concentration within urban settings
(wards, communities, barrios) and according to the impact of legislative policies (non
discrimination, attendance, racial/ethnic requirements) on charter school enrollment.
Results of such a study could guide legislators and charter-approving agencies in
servicing at-risk children in charter schools.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
75
CHAPTER 3
RESEARCH METHODOLOGY
The purpose of the study was to compare longitudinal student achievement in
charter schools and non-charter schools and allow for disaggregated data analysis of
at-risk student achievement. Currently, there is only limited research evaluating the
performance of low-SES students in charter schools and no available research
evaluating the performance of minority students in charter schools. Considering the
extensive efforts made by legislators to address at-risk children (i.e., goals targeting at-
risk children, non-discrimination and admission policies, racial/ethnic requirements),
this study provides valuable information on how at-risk children are performing in
charter schools. Therefore, this study compared charter and non-charter school
performance, specifically addressing the issue of at-risk student performance in charter
schools. Considering the extensive research that has already been done in California
(See Zimmer, et. al., 2003; Slovacek, et. al., 2002; Rogosa, 2002; Wells, et. al., 2000),
the purpose of this study was not to replicate, but to build on that research. The
California study conducted by Zimmer and fellow researchers did not disaggregate the
data to examine the comparative performance of any sub-groups of students (2003).
The Slovacek and Rogosa studies examined low-SES performance using the
Academic Performance Index (API), but compared ninety-three charter schools to all
non-charter schools in the state. Finally, Wells and fellow researchers did not use
standardized testing to examine student performance. Therefore, this study adds to
the body of research by examining minority and low-SES student achievement data in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
76
similar charter and non-charter schools. The study was different from the Slovacek
and Rogosa studies because it: a) examined minority performance in charter schools,
b) examined 4 years of API data relating to minority and low-SES students and c)
compared charter and non-charter school performance more accurately by matching
similar charter and non-charter school data. The study also built on RAND’s
California study (Zimmer, et. al., 2003) by examining minority and low-SES student
performance in the same matched charter and non-charter schools identified in
RAND’s study. By using the same matched schools and the same or similar formula
for analyzing the results, the performance of minority and low-SES students were
examined in relation to the overall student performance data presented in the
California study.
Research Questions
The study was guided by two overarching questions examining the
performance of at-risk students, defined as minority and low-SES students, in charter
schools:
1. How does the academic performance of at-risk students in charter schools
compare the to the performance of at-risk students in comparable non
charter schools in California?
2. How does the academic performance of at-risk students vary across charter
school type (start-up, conversion, non-classroom based) in California?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
77
Research Design
Campbell and Stanley Design 10 is the most common quasi-experimental
design (Michael, W.B. & Benson, J., 1994). The design is most frequently used in
non-laboratory situations or field studies because it is not dependent upon the
randomization of participants. As a result, the study used Campbell and Stanley
Design 10 to allow for analysis of treatment effect in an existing environment or
situation. Since charter schools are schools of choice, randomization was not possible
within the experimental design. Furthermore, the study examined existing data in a
fieldwork study (i.e., existing API results). As a result, the quasi-experimental Design
10 was the most relevant design to compare longitudinal student achievement in
charter and non-charter schools and the design used by the majority of statewide and
national studies on charter school achievement (Zimmer, et. al., 2003; Loveless, 2002;
Solomon, et. al., 2000; Slovacek, et. al., 2002; Rogosa, 2002).
Within the structure of Design 10, the students in the charter school,
specifically the at-risk student population, represented the experimental group. The
charter model represented the treatment, and the students in the non-charter school,
specifically the at-risk student population, represented the control group. The at-risk
student population was defined as minority students and low-SES students, measured
by student participation in the school lunch program. The charter school “treatment”
was recognized as being a broad area of study, given the diversity of charter schools
throughout the state. In order to narrow the charter treatment, the study disaggregated
the data according to charter school type (start-up, conversion, and non-classroom
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
78
based). By examining the charter school type, the study not only better defined the
charter effect, but also allowed at-risk student achievement to be compared with the
general results of the California study (Zimmer, et. al., 2003).
The student performance in charter schools and non-charter schools was
compared longitudinally, examining four years of API data, more specifically, three
years of API growth. The first year data, during the 1998-99 school year, represented
the pre-test, and the subsequent three years of API scores was used to measure the
charter treatment. The achievement of at-risk students in charter schools was
compared with the achievement of at-risk students in comparable non-charter schools.
By disaggregating the achievement results of at-risk students using the API, the results
of the study were compared and analyzed with respect to the California study
(Zimmer, et. al., 2003) results and the Slovacek/Rogosa results (2002), both of which
analyzed overall student performance using the API. Furthermore, even though the
Slovacek study and follow-up study by Rogosa considered SES as a factor, the study
did not compare charter schools to matched non-charter schools. Therefore, this study
analyzed low-SES achievement more specifically using matched non-charter schools
as a means of comparison.
One option for the study was to use the norm-referenced data as a means of
comparison. Wolf suggests that norm-referenced data can be used to represent a
control group (Michael, W.B. & Benson, J., 1994). Since this study looked at API
scores, calculated using Stanford 9 results, the option of using the norm-referenced
data as a control group was available, but not very practical. The norm-referenced
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
79
data available through Harcourt-Brace (producers of the SAT9) is not very relevant to
California. The Stanford 9 data was normed nationally with a representative sample
of only 2% English Learners (August, D. & Hakuta, K, 1997). Since California has
roughly 27% English Learners and charter school enrollment parallels statewide
enrollment, the national norms are not very accurate for the California student
population or a charter school study. That is not to say that the SAT9 used to calculate
the API does not provide accurate data on performance, but it did not provide a
valuable comparison benchmark for a quasi-experimental nonequivalent control group
design. The SAT9 data used to calculate the API represented an indicator of
performance and was used to compare schools of like populations within California.
The SAT9 did not represent an accurate control group, but it did represent a
satisfactory tool for comparing similar schools. Therefore, in order to build on prior
research, the comparable conventional (non-charter) schools identified in the
California study (Zimmer, et. al., 2003) were used as the control group in this study to
compare at-risk student achievement.
When randomization is not possible, Wolf suggests using a neighboring
institution as a comparison group (Michael, W.B. & Benson, J., 1994). The challenge
is to make sure that the two groups are comparable on all dimensions. The California
study (Zimmer, et. al., 2003) met the challenge. The study identified 352 charter
schools with 245 matched conventional schools and finally compared 161 charter
schools and 161 matched conventional schools. The charter schools and conventional
schools were matched using a propensity score. The propensity score was determined
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
80
for each charter and non-charter school according to the eight levels of students served
(elementary schools, middle schools, high schools, county schools, continuation
schools, juvenile hall schools, special education schools, and alternative education
schools) and according to the demographics of the schools using ethnicity,
socioeconomic status and percentage English Learners. Propensity scores were not
used to identify matched schools for county schools, continuation schools, juvenile
hall schools, special education schools, and alternative education schools since the
sample size was too small. Mobility rates, class size, year-round calendar operation
and percentage of non-credentialed teachers were eliminated as matched indicators
given the fact that the philosophy or mission of many charter schools incorporate these
indicators as essential policy components. Of the 352 charter schools and 245
conventional schools matched by RAND, 161 charter schools and 161 conventional
schools were included within the RAND study. As a result, 322 matched schools were
examined within this study.
By using the same matched schools identified by Zimmer and fellow
researchers (2003), this study was able to compare at-risk student achievement in
charter and non-charter schools with the general achievement comparisons drawn in
the California study. In this way, the study built on prior research and framed the two
research questions within the context of prior findings.
Population and Sample
The boundaries of the study and methods of sampling were defined by the
research questions. The research questions specifically directed the researcher
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
81
towards: (a) identifying a means of comparing charter and non-charter schools, (b)
identifying an instrument to measure at-risk student achievement in the identified
charter and non-charter schools, (c) defining the at-risk student group and (d) sorting
the charter schools into start-up, conversion and non-classroom based schools for
subsequent comparison. The criteria used to compare the charter and non-charter
schools were based on the matched schools identified by the California study
(Zimmer, et. al, 2003). The final list of matched schools included 161 charter schools
and 161 conventional schools. The API was the instrument used in the study to
measure achievement. The API data was collected on the at-risk students in each of
the schools. The at-risk student population was defined as minority students (black,
Asian, Hispanic, Native-American, and Pacific-Islander) and low-SES students
(measured by student participation in the school lunch program).
For each school, the following data was collected for the four years of API
scores, 1998-99, 1999-00, 2000-01, and 2001-02: (a) school level, (b) conventional
school or charter school type, (c) percent free or reduced lunch, (d) percent EL, (e)
parent education level, (f) percentage of white students, (g) percentage of black
students, (h) percentage of Asian students, (i) percentage of Hispanic students, (j)
percentage of Pacific-Islander, (k) percentage of Filipino, (1) percentage of American-
Indian, (m) total tested, (n) total API, (o) black student API, (p) Asian student API, (q)
Hispanic student API, (r) Caucasian student API, (s) American-Indian API, (t) Pacific-
Islander API, (u) Filipino API and (v) SED (socio-economically disadvantaged) API.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
82
Instrumentation
The two research questions specifically addressed the issue of at-risk student
achievement. As a result, there were several instruments that could measure student
achievement, including the API, SAT9 scores, other norm-referenced or criterion-
referenced tests, performance-based assessments, and more. An instrument had to be
identified that would be consistent for both the charter school and the matched non
charter school. Also, to address the validity concerns of the quasi-experimental
design, there was a need for longitudinal data. Given the limited manpower and
budget of the study, performance based assessments and other norm-referenced tests
were eliminated as options since they would take years to field-test and administer.
Both API and SAT9 data were available for charter and non-charter schools over a
four-year span, and both have been used in previous studies on academic achievement.
Recognizing that both sets of data provided only a limited perspective on achievement,
the API data was more succinct and easier to use. Parents, policy makers and
laypersons easily identify the API score as a general indicator of performance.
Furthermore, the state calculates disaggregated API scores for minority groups and
low-SES students, making it a valuable instrument to measure at-risk student
achievement, despite the limitations of the instrument indicated above.
Data Collection
The study was quantitative and used existing data available from secondary
sources. Three secondary sources were used to compile the data: (a) the California
Department of Education website on the API, (b) the California Department of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
83
Education website on charter schools, and (c) RAND’s identification of the matched
schools used in their California study using the propensity scores.
First, the California Department of Education website, www.cde.ca. gov/api.
provided a thorough database on the API data from the 1998-99 school year to the
present. The API score ranges from 200 to 1,000 and is determined using the school’s
performance on standardized tests across subjects. The API dataset included API
scores, growth targets, the number of students tested, percent English Learners, and
parent education level. The data was disaggregated by race, ethnicity and socio
economic status (i.e. students identified as socio-economically disadvantaged, or
SED). Data was provided for each group of students that represent at least 10% of the
school’s population. Therefore, each disaggregated group included the number of
students tested, the percent of that group in comparison to the entire school population,
an API score for the group and an API target for the group.
The data was available in two formats, API base data and API growth data.
The API base data represented API scores calculated in the fall using the revised
formula for each school year while the API growth data represented two years of
revised scores calculated using the same formula. As indicated above, the formula
identified by the State had changed each year to incorporate different norm-referenced
and criterion-referenced tests (i.e., Stanford Achievement Test, Ninth Edition and the
California Standards Test, respectively). During the four years of data pertaining to
this study, the formula changed each year. As a result, the base API most often
differed from the growth API. In order to maintain accuracy in instrumentation, only
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
84
one API score was used in this study: the growth API. The growth API measured the
growth within a two-year span using the same formula. As a result, there was
consistency within the data for the two-year growth cycle. This consistency did not
exist with the API base scores.
In the study, the API growth data was downloaded from the CDE website for
the 1999-2000 school year, the 2000-2001 school year and the 2001-2002 school year.
The growth data represented four years of API scores or three years of API growth.
Once the data was downloaded into a spreadsheet, the lists had to be reconciled. The
2001-2002 school year included roughly 200 additional schools than the 2000-2001
school year and roughly 600 additional schools than the 1999-2000 school year’s list.
Therefore, using the three separate spreadsheets, the lists of schools were reconciled
so that each list included the exact same schools, using the CDS codes to assure
consistency and accuracy of listing. Once the spreadsheets were consistent, they were
combined into one database including API growth scores from 1999-2002.
Using the information available on the California Department of Education’s
website on charter schools, www.cde.ca.gov/charters. the database was amended to
include charter school information. The CDE website on charter schools lists data on
each charter school within the state, including the start date, the type of charter school
(Conversion, Start-up) and if the charter school is site-based, independent study or a
combination of the two. Matching the schools by CDS (County-District-School)
code, the new information was added to the database, identifying the charter schools,
the charter type, and whether the site type (site-based or non-classroom based).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
85
Once the complete database was established, it was sent to RAND with a
listing of surrogate ID numbers (Zimmer, et. al., 2003). Since RAND maintains a
confidentiality agreement with the schools in its California study, the only way of
identifying the matched schools that were part of its study was through the use of
surrogate ID numbers. RAND reviewed the complete database, replaced the CDS
codes and names with surrogate ID numbers and identified the 161 charter schools and
161 conventional schools that were matched using a propensity score within its study.
The database was then returned for review and data analysis.
Validity and Reliability
With the quasi-experimental nonequivalent control group design, there were
several threats to validity that were a concern. The major threat to Design 10 was
caused by the absence of randomization. Without randomization, the design was open
to multiple threats to internal validity including: statistical regression, differential
selection and the interaction effects of selection and maturation, history and testing.
Furthermore, the randomization issue was complicated further by the fact that many
charter school students attend the charter school because their parents chose to do so.
As a result, there was a greater risk that the charter school students represented a
different group of students. In many cases, the charter school parents had volunteered
to send their children to the school, further complicating the threat of differential
selection and the interaction effects of selection and maturation, history and testing.
Looking first at statistical regression, the experimental group and the control
group testing data from 1998-1999 represented the pretest data for the two groups.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
86
The means and standard deviations of the two groups were determined. If the two
groups had significantly different API means, then the threat of statistical regression
was greater. When this occurs, the following year’s data may have been artificially
inflated or deflated as each group regressed towards the mean. So, in order to control
for this threat, three years of follow-up data were used. The three years of data helped
control for the regression effect. Also, since the charter schools were matched to
demographically similar non-charter schools and since the California study concluded
that, generally, charter schools were performing similarly to conventional schools,
there was less of a chance that the initial API results of the charter schools would
differ significantly from the non-charter schools.
The threat of differential selection was viable in the experiment. It is possible
that one group had stronger skills in reading or math than another group. This risk
was complicated by the fact that charter school parents choose for their children to
attend the charter school for a particular reason. Most parents are dissatisfied with the
non-charter school. This may indicate that the school was not meeting the needs of
that particular group of students (i.e. a school problem) or perhaps, the students were
not being successful at the non-charter school and therefore the parents wanted an
alternative (i.e. a student problem). How the problem is defined sheds light on the
sociopolitical context. If the problem is determined to be a school problem, then the
non-charter school was perceived to be failing. If the problem is determined to be a
student problem, then the charter school ends up attracting low-performing students.
These two perspectives on the problem may just balance each other out. Since most
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
87
research has indicated that charter schools have not “creamed” off the best of the non
charter, public school students (See Zimmer, et. al., 2003; Solomon, 2000; Loveless,
2002), then there was greater likelihood that the two groups (experimental and control)
did indeed represent similar groups of students.
Looking at the interaction effects of selection and maturation, history and
testing, once again the primary concern was the motivation behind the parents to
choose a charter school over a non-charter school. The motivation of the parents, as a
means of volunteering for an alternative school, represented a threat to the interaction
of selection and testing. The motivation of the parents to choose a charter school may
have indicated that they were more involved in their child’s education. Furthermore,
such parent awareness or motivation may have indicated a higher level of parent
education.
Since parent education levels have been correlated to higher levels student
achievement (see Betts, Reuben and Dannenberg, 2000), parent motivation and
subsequent indicators of parent education may have been a factor affecting selection
and testing. So, in order to control for the motivation factor, parent education levels
were analyzed via the state data system. The state collects data on parent education in
conjunction with the administering of the SAT9 and CST, data used to calculate the
school’s API. Therefore, the parent education level was set up as a control variable to
determine if the education level represented a factor in charter and non-charter school
student achievement. Finally, as a note of reference, the research indicates that
parents choose for their child to attend a charter school based on a belief that the non
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
88
charter public school was not meeting their child’s needs. There is a perception that
the charter school is a better school, based on the surveys with parents (see Riley,
2000; Finn, et. al., 1997), so there may be a greater motivation to show that the school
is doing well.
The threats to validity to the study on charter schools were significant, but
were controlled for through the design of the study and through careful matching of
schools. Also, many of the threats were diminished through the use of 4 years of
testing data, or 3 years of API growth data.
Data Analysis
Before any regressions were run on the API growth data, the charter and
conventional school groups were compared. In order to validate the propensity score
matching used by RAND, the charter and conventional school data sets were
compared by identifying the mean of each demographic group. The matched results
provided a framework for the regression analysis. Since the two groups were chosen
to be similar according to certain variables (i.e., racial/ethnic percentages, English
learner percentage, SED percentage and first school percentage), these same variables
were, therefore, less likely to impact the API growth from year-to-year. As a result,
these variables were predominantly excluded from the regression analysis.
In order to build on the results of the California study (Zimmer, et. al., 2003),
this study employed a similar method of data analysis. The California study examined
the change in API growth, denoted as AAPIJt, where j denotes a school, t denotes a
two-year span and API is the actual growth score over the two-year period. This study
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
89
looked at the same API growth over the same growth periods (i.e., 1999-2000, 2000-
2001, 2001-2002). Instead of examining the overall API growth, this study examined
the API growth for the predominant demographic groups: Hispanic, black and socio
economically disadvantaged (SED). A series of analyses were conducted within each
demographic group. Furthermore, the analyses were conducted separately on
elementary and secondary groups, given the disparity in API growth between the two
levels.
First, a mean comparison was conducted for the four-year span, comparing the
growth of charter schools with the growth of conventional schools. The mean
comparison included a breakdown of all schools by academic level (elementary and
secondary) and charter schools by charter type (conventional and start-up). Separate
mean comparisons were conducted for the three demographic groups: Hispanic, black
and socio-economically disadvantaged (SED).
Second, Pearson correlations were calculated for the same periods of API
growth within each demographic group. The correlations were calculated in order to
identify the charter effect in isolation of all other variables. In other words, the
Pearson correlation indicated whether charter status and charter type had any direct
relationship to the API growth for each growth year. As with the mean comparison,
the correlations were conducted by academic level, charter status and charter type for
Hispanic, black and SED students.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
90
Finally, a series of regressions were calculated, measuring the change in API
score using the following formula:
AAPIjt=AxjtP + Aujt
Within the formula, j denotes a school, t denotes a two-year period and API is
the actual school growth score within the identified two-year period. The formula
incorporated the following: (a) a variable, x, relating to observable factors affecting
API (i.e., charter status, parent education level, percent of teachers with credentials,
percent of teachers on emergency credentials), (b) a variable, P, related to
unobservable factors or parameters and (c) a random error factor,u. The change in the
API score, or growth from year-to-year, was measured as a change in each of the
variables in order to determine if the charter school status, the treatment in this case,
had an effect on the API growth or if some other observable or unobservable factor
caused the change in growth.
The regression variable and standard deviation of charter schools was
calculated separately for each group of minority and low-SES students, with additional
regression variables and standard deviations calculated for the other observable and
unobservable factors (i.e., charter status, parent education level, percent of teachers
with credentials, percent of teachers on emergency credentials and the constant). In
order to validate the matched comparisons, separate regressions were calculated that
also included the following observable factors: % white, % black, % Hispanic, %
Asian, % other minority, % EL, and % SED. Separate regressions were done for
elementary and secondary schools and for the three types of charter schools. After all
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
the regressions were complete, the regression variables that were statistically
significant at the >0.05 level were highlighted. From the data analysis, general
findings were developed that address the two research questions.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
92
CHAPTER 4
The purpose of the study was to compare longitudinal student achievement in
charter schools and non-charter schools and allow for disaggregated data analysis of
at-risk student achievement. Considering the extensive efforts made by legislators to
address at-risk children (i.e., goals targeting at-risk children, non-discrimination and
admission policies, racial/ethnic requirements), this study provides valuable
information on how at-risk children are performing in charter schools. By using four
years of disaggregated API data and the matched schools list from RAND’s California
study (see Zimmer, et. al., 2003), separate data sets were compiled for elementary and
secondary Hispanic, black and socio-economically disadvantaged (SED) students,
creating six unique data sets. As indicated in Chapter 3, three sets of calculations were
completed on each of the six data sets: a means comparison, a Pearson correlation and
a statistical regression. Each calculation was designed to shed further light on the
charter effect (i.e., the effect of charter status on at-risk children).
The purpose of this chapter is to report the results of the statistical procedures.
First, an analysis of the research instruments is presented. Second, a description of the
statistics of the sample is outlined and finally, an analysis of the data based on the
three sets of calculations is presented relating to: (a) the charter effect on elementary
and secondary Hispanic, black and SED students and (b) the charter effect, according
to charter type, on elementary and secondary Hispanic, black and SED students.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
93
Findings
The data was compiled using all secondary sources: the API database
available on the California Department of Education testing website, the charter
school information available on California Department of Education charter schools
website, and the matched data set comparing charter and conventional schools
provided by RAND. Using the information provided, six separate data sets were
created: (a) elementary Hispanic data, (b) secondary Hispanic data, (c) elementary
African-American data, (d) secondary African-American data, (e) elementary SED
data, and (f) secondary SED data. The actual data included in each data set is
considered accurate, using only school performance and demographic information
available to the general public. The validity concerns i.e., statistical regression,
differential selection and the interaction effects of selection and maturation, history
and testing) expressed in Chapter 3 were well addressed within the study.
The validity concern of statistical regression was addressed by the longitudinal
nature of the study. Since four years of API data was used, the regression effect is
minimized. Furthermore, the charter effect was calculated using matched data, taking
into consideration distinct demographic make-up of each school, and a regression
formula that considered other observable and unobservable factors such as parent
education level, percent of teachers with credentials and the percent of teachers on
emergency credentials. These safeguards help isolate the charter effect and minimize
the validity concern of statistical regression.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
94
The validity concern of differential selection was also addressed within the
study. Differential selection deals with the concern that two groups may have
different abilities. The two groups in this study are the charter school group and the
conventional school (non-charter) group. First, the matched data set provide by
RAND, identifying 161 charter schools and 161 conventional schools, was verified
within the study, demonstrating that the two groups were highly similar. Second, the
study examined API growth, not API scores. By examining API growth, the effect of
differential selection is highly minimized; student ability is measured through dynamic
growth instead of a static score.
In regard to the interaction effects of selection and maturation, history and
testing, the study accounted for parent selection effect by including parent education
level within the regression analysis. By including the parent education level, the
selection effect is minimized. Finally, since the API data was calculated for all
students using the Stanford Test, Ninth Edition, the validity threat of instrumentation
is not a concern.
The only other validity concern that was not considered prior to the study is the
effect of sample size. Since the study originally was designed to include over 300
charter and non-charter schools, the data sets were anticipated to be much larger.
Since RAND identified only 161 charter schools and 161 conventional schools that
were included in the California study (see Zimmer, et. al., 2003), the data set became
much smaller. As a result, the size of the six disaggregated data sets used in the study
became a concern. Once the data was disaggregated by school level (elementary and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
95
secondary), the data sets were cut in half, leaving 168 secondary schools and 154
elementary schools. Second, as indicated in Chapter 3, the API is only calculated for
minority groups when they represent over 10% of the school population. As a result,
the data sets were further diminished by racial/ethnic limitations. Finally, the data sets
were further limited by charter school type (conversion and start-up). Therefore,
above all other concerns, the sample size is the greatest threat to the results and
conclusions of the study.
Validating the Matched Schools
As indicated above, RAND used a propensity score to match charter and non
charter schools. The propensity score was calculated based on the academic level of
the school and the demographic make-up of the school site. The matched set sent by
RAND included 161 charter schools and 161 conventional schools. In order to
validate the similarity of the two groups, a mean comparison was calculated for each
of the three API growth years: 1999-2000, 2000-2001, and 2001-2002. The mean
comparison examined the same variables used by RAND to calculate the propensity
scores for elementary and secondary schools. The results are indicated in Tables 5, 6
and 7. Within the tables, the data is marked with an asterisk (*) when the
demographic characteristics of the charter schools, or charter schools of a particular
type, were different from the conventional schools by more than 5%. There was the
strongest relationship between the matched groups during the 2001-2002 school year
and the weakest relationship between the matched schools during the 1999-2000
school year.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 5: Demographic Mean Comparison 1999-2000
Other
English
Learner
Black Asian Hispanic Minority White SED
E lem entary 99 00% 99 00% 99 00% 99 00% 99 00% 99 00% 99 00%
C onventional
Mean 15.23 4.67 35.30 .93 40.13 45.77 22.50
N 30 30 30 80 30 30 30
Std. Dev. 20.222 6.288 24.403 1.868 32.107 36.667 23.111
C harter
Mean 6.24* 4.40 34.68 1.08 50.32* 36.88* 19.76
N 25 25 25 71 25 25 25
Std. Dev. 7.617 4.555 28.501 4.500 30.113 30.699 22.921
Start U p
Mean 6.14* 5.43 35.43 .39 50.71* 32.86* 18.57
N 7 7 7 28 7 7 7
Std. Dev. 6.619 5.412 34.583 .956 33.520 31.593 23.179
C onversion
Mean 6.28* 4.00 34.39 1.53 50.17* 38.44* 20.22
N 18 18 18 43 18 18 18
Std. Dev. 8.152 4.284 26.914 5.713 29.725 31.128 23.479
Other
English
Learner
Black Asian Hispanic Minority White SED
S econdary 99 00% 99 00% 99 00% 99 00% 99 00% 99 00% 99 00%
C onventional
Mean 16.76 4.37 32.33 3.88 40.81 45.22 17.67
N 67 67 67 80 67 67 67
Std. Dev. 25.941 5.471 26.836 6.032 31.183 35.190 19.403
Charter Mean 11.55* 6.17 23.44* 4.07 53.00* 34.29* 12.88
N 77 77 77 87 77 77 77
Std. Dev. 22.321 9.935 21.804 7.835 30.946 31.480 16.345
Start Up Mean 13.58 5.44 23.58* 4.66 50.85* 36.12* 12.17*
N 59 59 59 67 59 59 59
Std. Dev. 24.425 7.740 20.448 8.790 31.275 31.096 14.410
Conversion Mean 5.12* 8.65 22.88* 2.21 59.71* 28.12* 15.59
N 17 17 17 19 17 17 17
Std. Dev. 11.741 15.592 27.239 2.043 30.461 33.891 22.486
Site Based Mean 9.94* 6.55 23.08* 4.77 54.24* 31.73* 12.90
N 51 51 51 53 51 51 51
Std. Dev. 20.007 10.737 20.566 8.450 28.859 30.458 16.419
Some Non-Site Mean 4.64* 6.82 26.82* 3.07 56.18* 37.82* 15.64
Based
N 11 11 11 14 11 11 1 1
Std. Dev. 4.273 11.444 21.274 2.947 29.735 27.795 18.996
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 6: Demographic Mean Comparison 2000-2001
Elem entary
Black
00 01%
Asian
00 01%
Hispanic
00 01%
O ther
M inority
00 01%
W hite
00 01%
SED
00 01%
English
L earner
00 01%
Conventional
Mean
10.71 3.55 32.61 1.96 47.98 43.29 17.76
N
51 51 51 80 51 51 51
Std.Dev.
16.559 4.738 27.262 2.931 33.273 36.975 23.385
Charter
Mean
13.96 3.09 29.33 1.78 49.80 37.04* 15.29
N
45 45 45 74 45 45 45
Std.Dev.
26.676 4.252 28.690 4.507 33.722 33.459 22.733
Start Up
Mean
12.88 4.12 31.12 1.48 48.63 34.50* 14.25
N
16 16 16 29 16 16 16
Std.Dev.
23.670 4.829 31.908 2.613 36.520 37.326 21.164
Conversion
Mean
14.55 2.52 28.34 1.98 50.45 38.45 15.86
N
29 29 29 45 29 29 29
Std.Dev.
28.584 3.869 27.299 5.408 32.730 31.735 23.900
Secondary
Black
00 01%
Asian
00 01%
Hispanic
00 01%
O ther
M inority
00 01%
W hite
00 01%
SED
00 01%
English
L earner
00 01%
Conventional
Mean
14.56 4.27 31.09 3.69 44.95 41.61 16.35
N
75 75 75 80 75 75 75
Std.Dev.
24.434 4.957 27.665 5.138 32.765 34.687 19.692
Charter
Mean
10.16 5.87 23.72* 4.00 54.71* 33.15* 13.00
N
86 86 86 87 86 86 86
Std.Dev.
20.689 9.818 21.466 6.002 30.801 30.917 16.117
Start Up
Mean
12.02 5.30 24.41* 4.51 52.08* 34.56* 12.95
N
66 66 66 67 66 66 66
Std.Dev.
22.763 7.874 20.181 6.659 30.927 29.544 14.754
Conversion
Mean
4.16* 7.84 21.26* 2.37 63.26* 28.47* 13.47
N
19 19 19 19 19 19 19
Std.Dev.
9.788 15.034 26.400 2.290 30.267 36.476 20.937
Site Based
Mean
9.55 6.57 23.26* 3.98 55.02* 31.66* 12.96
N
53 53 53 53 53 53 53
Std.Dev.
19.309 10.899 20.044 5.383 28.871 31.519 16.114
Some Non-
Site Based
Mean
4.07* 7.14 27.29 5.86 54.64* 32.64* 16.21
N
14 14 14 14 14 14 14
Std.Dev.
3.407 10.734 20.593 4.589 28.171 27.988 18.573
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 1: Demographic Mean Comparison 2001-2002
98
O ther English
Black Asian Hispanic M inority W hite SED L earner
E lem entary 01 02% 01 02% 01 02% 01 02% 01 02% 01 02% 01 02%
Conventional Mean
14.29 5.13 32.97 4.13 42.25 44.93 19.89
N 75 75 75 80 75 75 75
Std.
Dev.
22.942 6.546 26.612 5.927 31.144 34.852 20.630
Charter Mean
16.24 3.14 36.17 1.86 40.79 46.76 21.59
N 71 71 71 74 71 71 71
Std.
Dev.
28.272 4.900 30.582 2.002 33.429 35.364 25.178
StartU p Mean
15.27 2.15 28.08 2.03 49.00* 32.27 11.92*
N 26 26 26 29 26 26 26
Std.
Dev.
28.390 2.664 28.689 1.842 34.234 37.102 20.010
Conversion Mean
16.80 3.71 40.84* 1.76 36.04* 55.13 27.18*
N 45 45 45 45 45 45 45
Std.
Dev.
28.510 5.767 30.971 2.112 32.389 31.807 26.342
O ther English
Black Asian Hispanic M inority W hite SED L earner
Secondary 01 02% 01 02% 01 02% 01 02% 01 02% 01 02% 01 02%
Conventional Mean
5.89 5.72 24.03 4.85 57.56 25.60 9.83
N 75 75 75 80 75 75 75
Std.
Dev.
12.093 9.463 22.451 7.834 27.657 26.511 12.613
Charter Mean
11.03* 2.83 19.83 3.97 57.86 28.44 6.52
N 64 64 64 87 64 64 64
Std.
Dev.
18.164 4.053 19.867 6.929 29.763 28.446 14.044
Start Up Mean
9.57 2.80 17.02 4.93 60.88 24.47 4.61
N 51 51 51 67 51 51 51
Std.
Dev.
15.126 4.065 17.477 7.614 28.422 26.726 11.361
Conversion Mean
16.77* 2.92 30.85* .79 46.00* 44.00* 14.00
N 13 13 13 19 13 13 13
Std.
Dev.
27.093 4.173 25.189 1.357 33.071 30.706 20.530
Site Based Mean
13.86* 3.38 22.83 5.06 50.79 34.38 9.07
N 42 42 42 53 42 42 42
Std.
Dev.
21.515 4.515 21.651 8.545 31.804 31.580 16.506
Some Non-Site Mean
5.00 1.88 10.75* 2.50 71.50* 16.88* .88*
Based N 8 8 8 14 8 8 8
Std.
Dev.
3.703 1.959 8.067 3.006 13.352 14.875 .835
The greatest difference between the charter school and conventional school
predominantly occurred in the white population group, the SED population group and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
the English Learner population group, with the difference occurring more frequently
among the various charter school types. The implication for the research was that, in
general, the matched schools represented a comparably normed group, and that the
regression analysis needed to consider discrepant variables (i.e., white population,
SED population, English Learner), if necessary, that may affect the API growth data
and skew the charter effect.
Elementary Hispanic Data Analysis
As indicated above, three sets of calculations were conducted on each
demographic group data set. The first of the calculations involved a simple mean
comparison between the Hispanic API growth, by year, among the conventional
schools, charter schools, conversion charter schools and start-up charter schools.
Table 8 presents the Hispanic API growth mean, sample size (N), and standard
deviation for each of the three years as well as the combined 1999-2002 growth.
Table 8: Elementary Hispanic API Growth Mean Comparison
1999-2000 2000-2001 2001-2002 1999-2002
School Type Growth Growth Growth Growth
Conventional Mean 45.86 31.87 19.79 105.12
Schools N 44 46 47 42
Std. Deviation 32.954 35.004 31.525 44.260
Charter Mean 45.61 31.82 12.68 105.81
Schools N 28 33 38 27
Std. Deviation 33.628 34.790 29.874 58.979
Conversion Mean 42.92 32.57 18.36 103.12
Charters N 26 28 28 25
Std. Deviation 33.410 34.671 27.799 60.487
Start Up Mean 72.67 22.17 -1.18 122.67
Charters N 3 6 11 3
Std. Deviation 13.650 37.563 30.298 30.665
Table 8 indicates that there were, on average 80 elementary schools with
Hispanic API data. Given the aggregated nature of the API, this score includes, on
average, 18,000 students. More conventional schools generated a Hispanic API score
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
100
than did charter schools. Furthermore, there were fewer elementary start-up charter
schools than conversion charter schools with Hispanic API scores. The number of
start-up charter schools is extremely small, jeopardizing the validity of any
conclusions that can be drawn from the start-up charter school data on Hispanic
students. The actual Hispanic API growth mean scores for elementary students were
extremely similar. Among the school types (conventional, charter, conversion and
start-up), there was little difference in the growth means of any group over the 4 year
period. Only the start up charter schools had a significantly different mean than the
conventional schools for the 2001-2002 school year. As a final note, no calculations
were done at the elementary level to distinguish site-based charter schools and non
site-based charter schools given the fact that no Hispanic API data was available for
non site-based charter schools.
The second calculation conducted involving elementary Hispanic students was
a Pearson correlation. The Pearson correlation identifies the relationship, positive or
negative, between charter school type, as compared to conventional schools, and API
growth for each year. Table 9 below indicates the Pearson correlation between school
type and the Hispanic API growth for the 1999-2000, 2000-2001, 2001-2002 and the
1999-2002 combined school years. Using a dummy variable to identify each charter
schools, conversion schools and start-up schools, the Pearson correlation calculates the
relationship between the API growth and the charter school status, in relation to
conventional schools. The correlation also indicates whether the relationship was
significant at the <0.05 level.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 9: Pearson Correlation fo r elementary Hispanic API growth
101
Year Charter Conversion Start Up
1999-2000 Growth Pearson Correlation
-.004 -.043 .203
Sig. (2-tailed)
.975 .721 .172
N
72 70 47
2000-2001 Growth Pearson Correlation
-.001 .010 -.089
Sig. (2-tailed)
.995 .933 .529
N
79 74 52
2001-2002 Growth Pearson Correlation
-.115 -.023 -.258
Sig. (2-tailed)
.294 .843 .050*
N
85 75 58
1999-2002 Growth Pearson Correlation
.007 -.019 .102
Sig. (2-tailed)
.956 .877 .505
N
69 67 45
Note i: (*) indicates significance at the <0.05 level
Examining the data presented in Table 9, the correlation between Hispanic API
growth and charter status is generally small and insignificant. With the exception of
the 2001-2002 start-up data, the two-tailed significance tests reveal very high scores.
With scores of almost 1.0, there is almost no relationship between charter status,
conversion status and Hispanic API growth for the three growth years. Only the start
up charter schools reveal a significantly negative correlation between start-up charter
status and Hispanic API growth for the 2001-2002 school year. In order to test the
relationship considering other observable and unobservable traits, a third calculation
was conducted on elementary Hispanic API.
The third calculation involved a regression analysis of the data, listed in Table
10 below . The regression analysis included four observable characteristics.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
102
Table 10: Elementary Hispanic API Growth Regression Analysis
Model
U nstandardized Coefficients t score Significance
B Std. Error
Dependent Charter (Constant) 50.898 72.879 .698 .492
Variable:
Elementary
Hispanic
Growth
Schools
Charter 2.371 14.039 .169 .867
N=28
Parent Ed. Level -1.506 8.909 -.169 .867
R Square:
%Tchr. Cred. .154 .617 .249 .805
1999-2000 0.066 % Emer. Cred. -.401 .938 -.427 .673
Start Up (Constant) 14.824 101.657 .146 .886
Charter
Start Up Charter 20.008 34.095 .587 .566
N=19
Parent Ed. Level 8.333 11.877 .702 .494
R Square:
%Tchr. Cred. .247 .909 .272 .789
0.179 % Emer. Cred. -.353 1.264 -.279 .784
Conversion (Constant) 22.475 81.175 .277 .784
Charter
Conv. Charter .514 14.317 .036 .972
N=27
Parent Ed. Level 1.491 9.688 .154 .879
R Square:
%Tchr. Cred. .345 .663 .520 .608
0.071 % Emer. Cred. -.125 1.003 -.124 .902
Dependent Charter (Constant) 55.675 42.980 1.295 .202
Variable:
Elementary
Hispanic
Growth
Schools
Charter -6.490 11.154 -.582 .564
N=47
Parent Ed. Level -6.232 7.412 -.841 .405
R Square:
%Tchr. Cred. .052 .405 .130 .898
2000-2001 0.066 % Emer. Cred. -.317 .450 -.705 .485
StartU p (Constant) 27.899 58.481 .477 .637
Charter
Start Up Charter -3.833 26.082 -.147 .884
N=32
Parent Ed. Level -4.485 10.733 -.418 .679
R Square:
%Tchr. Cred. .276 .569 .485 .631
0.200 % Emer. Cred. .016 .635 .025 .980
Conversion (Constant) 30.905 46.650 .662 .511
Charter
Conv. Charter -9.284 11.679 -.795 .431
N=45
R Square:
Parent Ed. Level -3.587 7.625 -.470 .641
%Tchr. Cred. .224 .427 .524 .603
0.050 % Emer. Cred. -.085 .476 -.178 .859
Depend. Charter (Constant) 129.960 42.574 3.053 .003*
Variable:
Elementary
Hispanic
Growth
Schools
Charter -3.801 7.269 -.523 .603
N=72
Parent Ed. Level -9.864 6.441 -1.531 .130
R Square:
%Tchr. Cred. -.820 .394 -2.080 .041*
2001-2002 0.109 %Emer. Cred. -1.126 .468 -2.408 .019*
Start Up (Constant) 98.077 59.298 1.654 .105
Charter
Start Up Charter -18.721 12.287 -1.524 .135
N=48
R Square:
Parent Ed. Level -6.185 10.195 -.607 .547
%Tchr. Cred. -.599 .579 -1.034 .307
0.118 % Emer. Cred. -.780 .689 -1.133 .263
Conversion (Constant) 159.044 43.775 3.633 .001*
Charter
Conv.Charter 2.662 7.583 .351 .727
N=63
Parent Ed. Level -8.627 6.732 -1.282 .205
R Square:
%Tchr. Cred. -1.188 .419 -2.836 .006*
0.144 % Emer. Cred. -1.268 .481 -2.634 .011*
The four observable characteristics listed in Table 10 include: charter status,
parent education level, percent teacher credentialed, and percent emergency
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
103
credentialed. These four characteristics were chosen based on the fact that the
matched school data did not include these traits as matched indicators. As a result, the
regression analysis distinguishes the charter effect in conjunction with the other
observable traits. Since the matched data described above in Tables 5, 6 and 7
indicated that there were several traits that were not perfectly matched, a full
regression analysis including all the available indicators (i.e., charter status, % black,
% Hispanic, % Asian, % Other Minority, % White, % SED, % EL, % First School,
Parent education level, % Teacher credential, % Emergency credential) was conducted
for each growth year, but with no significant difference in findings (see Appendix A,
B and C).
Table 10 indicates the regression coefficient (B), the t-score, and the
significance for each variable. Furthermore, the R-squared value and number within
the sample (N) is indicated for each regression by growth year. Both the R-squared
values and N are relatively low. The R-squared values significantly improved with the
full regression with all the variables (See Appendix A, B and C), but there was not any
significant change in the t-score or significance level.
The variables significant at the >0.05% level are highlighted with an asterisk
(*). None of the charter variables (charter, conversion or start-up) are significant at
the 0.05 level. Even the regression coefficient for start-up charter schools for 2001-
2002 school year, a variable, which was deemed significant using the Pearson
correlation, was not significant at the 0.05 level within the regression.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
104
Secondary Hispanic Data Analysis
In regard to secondary Hispanic students, the same three calculations were
conducted. Table 11 lists the mean comparison of the API for secondary Hispanic
students in conventional schools, charter schools and the four charter school types.
Based on the mean comparison for the three years of API data, secondary Hispanic
students in conventional schools had a greater mean growth two out of the three years
than charter schools and all charter school types. The number of secondary schools
with Hispanic API data gradually increased from 1999-2000 to 2001-2002.
Furthermore, there were very few non site-based charter schools with Hispanic API
data, which correlates to the demographic data presented in Tables 5, 6 and 7 where
non site-based charter schools served a significantly lesser percentage of Hispanic
students.
Table 11: Secondary Hispanic Mean Comparison
1999-2000 2000-2001 2001-2002 1999-2002
School Type Growth Growth G rowth G rowth
Conventional Mean 21.62 9.40 7.00 49.1250
School N 34 42 42 32
Std. Deviation 36.217 28.946 20.881 40.68823
Charter School Mean 15.90 10.33 3.88 37.2222
N 10 15 17 9
Std. Deviation 29.023 36.252 36.017 23.62614
Conversion Mean 16.67 4.78 10.88 37.0000
Charters N 6 9 8 5
Std. Deviation 22.088 27.901 24.032 22.47221
Start Up Mean 14.75 18.67 -2.33 37.5000
Charters N 4 6 9 4
Std. Deviation 41.363 47.911 44.685 28.54820
Site Based Mean 13.25 11.83 3.15 33.4286
Charters N 8 12 13 7
Std. Deviation 22.952 35.169 29.100 24.65669
Some Non-Site Mean 69.00 31.00 -78.00 37.0000
Based Charters N 1 2 1 1
Std. Deviation 19.799
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The second calculation conducted on the secondary Hispanic data set was a
Pearson correlation, described in Table 12. The Pearson correlation tests the
relationship between the secondary Hispanic API growth and charter status. Overall,
the correlation values were very small and therefore, statistically insignificant based
on a two-tailed test. In regard to the sample size, the number of secondary schools
included in the study gradually increased from the 1999 to 2002. As a final note, non
site-based schools were excluded from the analysis, given the small number of charter
schools that posted Hispanic API data.
Table 12: Secondary Hispanic Pearson Correlation
C harter Conversion S tart Up Site-Based
1999-2000 Growth Pearson Correlation
-.070 -.052 -.059 -.098
Sig. (2-tailed)
.650 .749 .725 .538
N
44 40 38 42
2000-2001 Growth Pearson Correlation
.013 -.062 .099 .034
Sig. (2-tailed)
.921 .664 .505 .808
N
57 51 48 54
2001-2002 Growth Pearson Correlation
-.055 .068 -.137 -.072
Sig. (2-tailed)
.679 .640 .338 .600
N
59 50 51 55
1999-2002 Growth Pearson Correlation
-.132 -.109 -.094 -.158
Sig. (2-tailed)
.409 .523 .585 .335
N
41 37 36 39
Note ii: Non-site based charter schools were not included, given the small number o f schools.
The third calculation involving secondary Hispanic students was a regression
analysis including 4 variables: charter status, parent education level, percent teacher
credentialed and percent emergency credentialed. The variables were chosen, as with
the elementary data, based on the fact that the matched variables did not include these
factors. As a result, they would be more likely to differentiate student achievement.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 13: Secondary Hispanic API Growth Regression
106
t score Significance
Model B Std. Error
Dependent Charter Schools (Constantl 7 162.674 2.808 .009*
Variable: N=33 Charter 12.986 13.189 .985 .333
Secondary R Square: Parent Ed. Lvl -15.155 7.885 -1.922 .064
Hispanic 0.38 %Tchr. Cred. -3.687 1.649 -2.236 .033*
Growth % Emer. Cred. -6.047 1.933 -3.129 .004*
1999-2000 Start Up Charter (Constant! 530.220 169.968 3.120 .005*
N=27 SU Charter 32.470 20.508 1.583 .127
R Square: Parent Ed. Lvl -11.588 8.573 -1.352 .190
0.506 %Tchr. Cred. -4.466 1.741 -2.565 .017*
% Emer. Cred. -7.468 2.084 -3.583 .002*
Conv.Charter (Constant! 470.872 167.479 2.812 .009*
N=30 Conv. Charter 6.381 16.275 .392 .698
R Square: Parent Ed. Lvl -13.913 8.644 -1.610 .120
0.399 %Tchr. Cred. -3.866 1.724 -2.243 .034*
% Emer. Cred. -6.244 1.997 -3.127 .004*
Site-based (Constant! 471.151 157.339 2.994 .006*
Charters Site 6.086 13.331 .457 .652
N=32 Parent Ed. Lvl -13.815 7.655 -1.805 .082
R square .419 %Tchr. Cred. -3.871 1.596 -2.425 .022*
% Emer. Cred. -6.258 1.871 -3.345 .002*
Dependent Charter Schools (Constant! 46.386 79.605 .583 .563
Variable: N=52 Charter 8.737 9.646 .906 .370
Secondary R Square. 044 Parent Ed. Lvl -7.159 5.651 -1.267 .211
Hispanic %Tchr. Cred. -.186 .734 -.253 .801
Growth % Emer. Cred. -.343 1.282 -.268 .790
2000-2001 Start Up Charter (Constant! 46.857 82.529 .568 .573
N=43 SU Charter 13.900 13.763 1.010 .319
R Square:
0.046
Parent Ed. Lvl -5.227 6.149 -.850 .401
%Tchr. Cred. -.251 .759 -.330 .743
% Emer. Cred. -.428 1.340 -.320 .751
Conversion (Constant! -18.743 79.510 -.236 .815
Charter Conv.Charter 2.710 10.815 .251 .803
N=46 Parent Ed. Lvl -6.749 5.293 -1.275 .209
R square .057 %Tchr. Cred. .487 .738 .660 .513
% Emer. Cred. .277 1.257 .220 .827
Site-based (Constant! 21.894 79.809 .274 .785
Charters Site 10.124 9.978 1.015 .316
N=49 Parent Ed. Lvl -7.677 5.482 -1.400 .168
R square .057 %Tchr. Cred. .082 .741 .111 .912
% Emer. Cred. -.025 1.293 -.019 .985
Dependent Charter Schools (Constantl 1.383 36.607 .038 .970
Variable: N=57 Charter 1.283 7.723 .166 .869
Secondary R Square: Parent Ed. Lvl -11.196 6.648 -1.684 .098
Hispanic
Growth
0.134 %Tchr. Cred. .430 .263 1.637 .108
%Emer. Cred. .141 .347 .408 .685
2001-2002 Start Up (Constant! -.321 41.175 -.008 .994
Charter SU Charter -.843 10.947 -.077 .939
N=49 Parent Ed. Lvl -11.276 7.174 -1.572 .123
R Square: %Tchr. Cred. .445 .316 1.407 .166
0.133 % Emer. Cred. .194 .385 .503 .617
Conversion (Constantl 37.818 41.550 .910 .368
Charter Conv.Charter 3.351 8.267 .405 .687
N=48 Parent Ed. Lvl -11.474 5.783 -1.984 .050*
R Square:
0.091
%Tchr. Cred. .054 .365 .147 .884
% Emer. Cred. -.169 .466 -.363 .718
Site-based (Constantl 7.536 31.770 .237 .813
Charters Site -.852 6.976 -.122 .903
N=49 Parent Ed. Lvl -12.775 5.678 -2.250 .029*
R square .057 %Tchr. Cred. .424 .230 1.844 .071
% Emer. Cred. .058 .332 .173 .863
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
107
Table 13 describes the regression analysis conducted using the secondary
Hispanic data set. The variables used within each growth year represented the data
from the earlier year. For example, the parent education level data used for the 2001-
2002 regression was data from the 2001 school year. The table shows the R-squared
value and total number of schools included (N) for each regression. In addition,
coefficient values, t-scores and significance values are cited for each value.
The R-squared values for each regression vary greatly. The 1999-2000 data
includes strong R-squared values, given the fact that the variables in that growth year
were significant. The R-squared values for the other growth years were relatively
weak, indicating that other variables not identified in the regression were impacting
the secondary Hispanic API growth.
The regression identifies several coefficients that were significant at the 0.05
level (*), including percent teacher credentialed, percent emergency credentialed and
parent education level. Even though the mean comparison between conventional and
charter schools indicated a significant difference in growth for secondary Hispanic
students, during no growth year was the charter status or charter type significant
toward the secondary Hispanic growth results.
Elementary African-American Data Analysis
The African-American data set was divided into two separate data sets: one
for elementary and one for secondary. With the elementary data set, the same three
formulas were calculated to measure the academic progress of African-American
students within conventional schools and charter schools. The first analysis involved a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
108
mean comparison of African-American students in conventional schools, charter
schools and within the various charter school types. Table 14 below lists the mean,
number of schools (N) and the standard deviation for elementary African-
American students for each growth year in each school type. The data set for African-
American students is smaller than the data sets for Hispanic students. On average,
thirty schools were included in the mean comparison, representing roughly 6,000
students. First, the table indicates that fewer charter schools posted African-American
API scores than conventional schools. Second the elementary African-American
mean API growth score was higher in charter schools than conventional schools for
the three growth years. Conversion schools had a higher mean for two of the three
growth years, while start-up schools had a very limited data set (only 1 to 3 schools
per year), thus limiting any conclusions that can be drawn from the data. Finally, the
mean API growth over the three-year period, 1999-2002, was significantly higher
among charter schools.
Table 14: Elementary African-American API Growth Mean Comparison
1999-2000 2000-2001 2001-2002 1999-2002
School Type Growth Growth Growth Growth
Conventional Mean 38.71 15.19 15.67 70.8000
School N 17 16 18 15
Std. Deviation 40.755 20.932 34.200 39.04430
Charter School Mean 42.25 23.94 23.92 95.1111
N 12 16 13 9
Std. Deviation 31.660 38.715 39.683 56.97685
Conversion Mean 38.27 26.77 20.30 92.5000
Charters N 11 13 10 8
Std. Deviation 29.897 37.296 42.045 60.33241
Start Up Mean 86.00 11.67 36.00 116.0000
Charters N 1 3 3 1
Std. Deviation 51.160 34.771
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The second calculation computed for elementary African-American students
was a Pearson correlation. Table 15 below includes the outcomes of the Pearson
correlation. The correlations for each of the three growth years were low and
insignificant for elementary African-American students. The only correlation that
approached significance was the 1999-2002 three year growth span, but still well short
of the necessary 0.05 level to reject the null hypothesis.
Table 15: Elementary African-American API Growth Pearson Correlation
Year Charter Conversion Start Up
1999-2000 Growth Pearson Correlation
.048 -.006 .271
Sig. (2-tailed)
.803 .976 .276
N
29 28 18
2000-2001 Growth Pearson Correlation
.144 .199 -.051
Sig. (2-tailed)
.433 .300 .834
N
32 29 19
2001-2002 Growth Pearson Correlation
.114 .062 .213
Sig. (2-tailed) .540 .754 .353
N 31 28 21
1999-2002 Growth Pearson Correlation .256 .223 .287
Sig. (2-tailed) .227 .306 .281
N
24 23 16
The final analysis conducted on the elementary African-American API growth
data set was a series of regressions. As with the Hispanic data, a series of regressions
were conducted using four variables not included in the matching of the conventional
and charter schools. The regression results are included in Table 16. The table lists
the R-squared values for each regression as well as the number of schools (N)
included in each regress. Finally, the coefficient scores, t-scores and significance
value for each variable is listed.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 16: Elementary African-American API Growth Regressions
110
Model U nstandardized Coefficients t score Significance
B Std. Error
Dependent Charter (Constant) 167.336 97.727 1.712 .131
Variable:
Elementary
African-
American
Growth
Schools
Charter 24.565 19.756 1.243 .254
N =11
R Square:
0.589
Parent Ed. Level -14.447 9.738 -1.484 .181
%Tchr. Cred. -1.205 .928 -1.298 .236
% Emer. Cred. -.238 1.253 -.190 .855
1999-2000 Start Up (Constant) 173.314 83.821 2.068 .175
Charter
Start Up Charter 28.394 24.607 1.154 .368
N=6
R Square:
0.92.2
Parent Ed. Level -10.261 12.489 -.822 .498
%Tchr. Cred. -1.403 .769 -1.823 .210
% Emer. Cred. -.373 .974 -.383 .739
Conv. (Constant) 154.212 150.330 1.026 .345
Charter
Conv. 23.671 22.528 1.051 .334
N=10
R Square:
0.485
Parent Ed. Level -13.628 12.454 -1.094 .316
%Tchr. Cred. -1.092 1.361 -.802 .453
% Emer. Cred. -.112 1.698 -.066 .950
Dependent Charter (Constant) -11.303 40.102 -.282 .782
Variable: Schools
Charter 9.117 13.445 .678 .507
Elementary
African-
American
Growth
N=20
R Square:
0 079
Parent Ed. Level -.102 7.897 -.013 .990
%Tchr. Cred. .235 .323 .726 .479
% Emer. Cred. .027 .361 .075 .941
2000-2001
S tartu p (Constant) -1.682 52.532 -.032 .975
Charter
Start Up Charter 20.104 23.532 .854 .415
N=13
R Square:
0.101
Parent Ed. Level -9.474 10.904 -.869 .408
%Tchr. Cred. .371 .419 .885 .399
% Emer. Cred. .376 .508 .740 .478
Conv. (Constant) 66.776 48.507 1.377 .192
Charter
Conv. 19.780 13.165 1.502 .157
N=17
R Square:
0 2.58
Parent Ed. Level 2.732 7.518 .363 .722
%Tchr. Cred. -.684 .443 -1.543 .147
% Emer. Cred. -.772 .428 -1.801 .095
Depend. Charter (Constant) 12.492 46.201 .270 .789
Variable: Schools
Charter 12.577 15.229 .826 .416
Elementary
African-
American
Growth
N=30
R Square:
0 050
Parent Ed. Level -2.935 12.722 -.231 .819
%Tchr. Cred. .213 .423 .502 .620
%Emer. Cred. -.348 .580 -.599 .554
2001-2002
Start Up (Constant) -3.879 62.067 -.063 .951
Charter
Start Up Charter 27.798 31.071 .895 .384
N=20
R Square:
0.105
Parent Ed. Level .174 16.612 .010 .992
%Tchr. Cred. .301 .468 .643 .529
% Emer. Cred. -.226 .723 -.312 .759
Conv. (Constant) 107.016 56.989 1.878 .073
Charter
Conv.Charter 14.834 14.735 1.007 .325
N=56
R Square:
0.109....
Parent Ed. Level -8.968 13.984 -.641 .528
%Tchr. Cred. -.553 .584 -.947 .354
% Emer. Cred. -1.754 .778 -2.255 .034*
First, the regressions calculated for start-up charter schools are highly suspect,
given the extremely small sample size used. Furthermore, there is a significant
difference in the R-squared values for each regression, indicating that the variables
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
I l l
used in the regression have varying degrees of significance. Examining the
regression variables, almost none of the coefficients are significant at the 0.05 level.
Despite the difference in API growth mean between charter and conventional schools
presented in Table 14 for elementary African-American students, the charter variable
did not turn out significant within the regressions. In fact, running the regression
using all the variables, including the matched school variables, still none of the
variables resulted as statistically significant.
Secondary African-American Data Analysis
Examining the growth mean comparison of secondary African-American API,
presented in Table 17, African-American students in conventional schools
outperformed African-American students in charter schools two out of the three
growth years and the students in conversion charter schools all three years.
Table 11: Secondary African-American API Growth Mean Comparison
1999-2000 2000-2001 2001-2002 1999-2002
School Type Growth Growth Growth Growth
Conventional Mean 20.18 12.45 7.18 41.0000
School N 11 11 11 9
Std. Deviation 24.136 30.543 17.820 30.20348
Charter School Mean 13.20 23.00 -4.13 22.7500
N 5 8 8 4
Std. Deviation 21.982 44.954 22.421 21.54646
Conversion Mean 13.20 4.80 -2.00 22.7500
Charters N 5 5 4 4
Std. Deviation 21.982 16.037 23.480 21.54646
Start Up Mean 53.33 -6.25
Charters N 3 4
Std. Deviation 65.957 24.690
Site Based Mean 7.88 19.38 22.59 47.4286
Charters N 8 13 17 7
Std. Deviation 21.013 41.798 35.005 53.73037
Some Non-Site Mean -12.00 -82.00
Based Charters N 1 1
Std. Deviation
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
112
The number of schools (N) is very small in all the school type categories, a
result of the small number of schools within the 322 matched school list that had a
significant number of African-American students to generate an African-American
API score. As with secondary Hispanic students, there are almost no non-site based
schools that serve a significant number of secondary African-American students. As a
result, the follow-up analyses exclude the non-site based category from the
calculations.
Table 18 describes the Pearson correlation between secondary African-
American API growth and school type. The table indicates the strength of the
relationship, in isolation, between school type and the academic performance of
secondary African-American students. The data is disaggregated by growth year and
school type. The correlation, in general is negative for every category, but not at the
0.05 significance level.
Table 18: Secondary African-American API Growth Pearson Correlation
Year C harter Conversion S tart Up Site-Based
1999-2000 Growth Pearson Correlation
-.145 -.145 .(a) -.145
Sig. (2-tailed)
.591 .591 .591
N
16 16 11 16
2000-2001 Growth Pearson Correlation
.147 -.138 .423 .147
Sig. (2-tailed)
.549 .610 .131 .549
N
19 16 14 19
2001-2002 Growth Pearson Correlation
-.285 -.221 -.309 -.285
Sig. (2-tailed)
.237 .429 .262 .237
N
19 15 15 19
1999-2002 Growth Pearson Correlation
-.310 -.310 .(a) -.310
Sig. (2-tailed)
.303 .303 .303
N
13 13 9 13
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
113
The third set of calculations run on the secondary African-American data set
included a series of regression by growth year and school type. The same four
variables included in the earlier analyses were used in the secondary African-
American API growth regression. Table 19 below shows the results of the regression
calculations.
The R-squared values of the 1999-2000 growth year were very low, while the
R-squared values for the 2000-2001 and 2001-2002 growth years were strong. The
number of schools included in the regression were also higher for the 2000-2001 and
the 2001-2002 growth years. Those same years also had several variables that were
statistically significant, including several charter variables.
During the 2001-2002 growth year, the secondary African-American API
growth was highly dependent upon charter status. Furthermore, there was also a
statistically significant result for start-up charter schools and site-based charter schools
for the same growth year. Each of the three variables were significant at the >0.05
level. Furthermore, the coefficients of these variables were all negative. The charter
variable had a coefficient o f-18.5 and the start-up charter variable had a coefficient of
-26.7. Since all African-American API growth scores came from charter schools that
were site-based, the site-based variable also had a coefficient o f-18.5. The same
variables from the prior two growth years were not statistically significant.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
114
Table 19: Secondary African-American API Growth Regression
Model B Std. E rro r T-Score Significance
Dependent Charter Schools (Constant) 72.936 447.415 .163 .875
Variable: N =11 Charter -9.756 19.529 -.500 .633
Secondary R Square: Parent Ed. Lvl -1.174 13.639 -.086 .934
African- 0.044 %Tchr. Cred. -.427 4.344 -.098 .924
American Growth % Emer. Cred. -.682 5.112 -.134 .898
1999-2000 Start Up Charter (Constant) N/A N/A N/A N/A
N=0 SU Charter N/A N/A N/A N/A
R Square: Parent Ed. Lvl N/A N/A N/A N/A
N/A %Tchr. Cred. N/A N/A N/A N/A
% Emer. Cred. N/A N/A N/A N/A
Conv.Charter (Constant) 72.936 447.415 .163 .875
N =11 Conv. Charter -9.756 19.529 -.500 .633
R Square: Parent Ed. Lvl -1.174 13.639 -.086 .934
0.044 %Tchr. Cred. -.427 4.344 -.098 .924
% Emer. Cred. -.682 5.112 -.134 .898
Site-based (Constant) 72.936 447.415 .163 .875
Charters Site based -9.756 19.529 -.500 .633
N =11 Parent Ed. Lvl -1.174 13.639 -.086 .934
R square .044 %Tchr. Cred. -.427 4.344 -.098 .924
% Emer. Cred. -.682 5.112 -.134 .898
Dependent Charter Schools (Constant) 542.669 218.601 2.482 .027*
Variable: N=17 Charter 7.271 16.409 .443 .665
Secondary R Square. 401 Parent Ed. Lvl -14.410 12.849 -1.122 .282
African- %Tchr. Cred. -4.569 2.174 -2.101 .056*
American Growth % Emer. Cred. -8.559 3.134 -2.731 .017*
2000-2001 Start Up Charter (Constant) 468.679 255.657 1.833 .104
N=12 SU Charter 26.638 30.822 .864 .413
R Square: Parent Ed. Lvl -6.446 19.999 -.322 .755
0.489 %Tchr. Cred. -4.098 2.583 -1.587 .151
% Emer. Cred. -7.813 3.657 -2.137 .065
Conversion (Constant) 458.853 155.556 2.950 .015*
Charter Conv.Charter -8.375 13.465 -.622 .548
N=14 Parent Ed. Lvl 4.017 11.887 .338 .742
R square .504 %Tchr. Cred. -4.536 1.508 -3.008 .013*
% Emer. Cred. -6.541 2.348 -2.786 .019*
Site-based (Constant) 542.669 218.601 2.482 .027*
Charters Site based 7.271 16.409 .443 .665
N=17 Parent Ed. Lvl -14.410 12.849 -1.122 .282
R square .401 %Tchr. Cred. -4.569 2.174 -2.101 .056*
% Emer. Cred. -8.559 3.134 -2.731 .017*
Dependent Charter Schools (Constant) -32.805 67.087 -.489 .632
Variable: N=18 Charter -18.476 8.370 -2.207 .044*
Secondary R Square: Parent Ed. Lvl 12.578 10.772 1.168 .262
African- 0.430 %Tchr. Cred. -.105 .490 -.215 .833
American Growth %Emer. Cred. .620 .620 1.000 .334
2001-2002 Start Up (Constant) -60.381 72.925 -.828 .427
Charter SU Charter -26.680 11.033 -2.418 .036*
N=14 Parent Ed. Lvl 13.929 11.856 1.175 .267
R Square: %Tchr. Cred. .135 .511 .264 .797
0.498 % Emer. Cred. .914 .642 1.424 .185
Conversion (Constant) -74.524 110.610 -.674 .516
Charter Conv.Charter -9.029 11.463 -.788 .449
N=14 Parent Ed. Lvl 10.662 13.084 .815 .434
R Square: %Tchr. Cred. .304 1.030 .295 .774
0.322 % Emer. Cred. 1.508 1.681 .897 .391
Site-based (Constant) -32.805 67.087 -.489 .632
Charters Site based -18.476 8.370 -2.207 .044*
N=18 Parent Ed. Lvl 12.578 10.772 1.168 .262
R square .430 %Tchr. Cred. -.105 .490 -.215 .833
% Emer. Cred. .620 .620 1.000 .334
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
115
Elementary SED Data Analysis
The API data is disaggregated for socio-economically disadvantaged children
based on the student qualification for the federal free or reduced lunch program. The
lunch program is based directly on family income. As a result, the SED students
generate a separate API if the SED population represents more than 10% of the
student population. For the SED API data, the same sets of calculations were done.
Table 20 provides the results of the API growth mean comparison for
elementary SED students. The table includes the API growth for the three growth
years for each school type. The data indicates that there is a higher number (N) of
SED API scores among the 322 schools. On average, there are 90 schools included in
the comparison of conventional and charter schools. The mean scores comparing the
API growth in conventional schools and charter schools indicate that the charter
schools slightly outperformed the conventional schools two out-of-the three years.
During the 2001-2002 school year, conventional schools significantly outperformed
charter schools by an average of 13.12 API points. In regard to the charter types,
elementary SED students in conversion charter schools had higher mean scores than
the SED students in conventional schools for two growth years and SED students in
the start-up schools had lower growth means for two years. The elementary SED
students in start-up charter schools outperformed the SED students in conventional
schools with a higher growth mean during the 1999-2000 growth year, but the sample
size was very small, just two schools. The sample size for start-up charter schools
gradually increased over the 4 year period from two schools to ten schools.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
116
Table 20: Elementary SED API Growth Mean Comparison
School Type
1999-2000
Growth
2000-2001
G rowth
2001-2002
G rowth
1999-2002
Growth
Conventional Mean 42.76 25.17 18.04 92.74
School N 50 54 54 46
Std. Deviation 37.844 30.829 33.052 54.174
Charter School Mean 44.25 32.61 4.92 96.47
N 32 41 48 32
Std. Deviation 29.135 34.410 37.291 57.694
Conversion Mean 41.53 35.08 5.79 94.53
Charters N 30 36 38 30
Std. Deviation 28.021 32.999 35.664 59.075
Start Up Mean 85.00 14.80 1.60 125.50
Charters N 2 5 10 2
Std. Deviation 1.414 43.136 44.918 13.435
Table 21 indicates the Pearson correlation for the elementary SED students.
The data correlates the API growth by year with each school type. The sample sizes
(N) for the correlations were, on average, 90 for charter schools, 86 for conversion
schools and 58 for start-up charter schools. None of the correlations are significant at
the 0.05 level, although two of the statistics are very close. During the 2001-2002
growth years, both the elementary SED API growth scores in charter schools and in
conversion schools had correlations significant at the >0.10 level. Both of the
correlations were negative for the school types (-.185 and -.176 respectively).
Table 21: Elementary SED Pearson Correlation
Year C harter Conversion S tart Up
1999-2000 Growth Pearson Correlation .021 -.017 .216
Sig. (2-tailed) .850 .878 .124
N 82 80 52
2000-2001 Growth Pearson Correlation .114 .153 -.092
Sig. (2-tailed) .271 .150 .489
N 95 90 59
2001-2002 Growth Pearson Correlation -.185 -.176 -.171
Sig. (2-tailed) .062 .094 .178
N 102 92 64
1999-2002 Growth Pearson Correlation .033 .016 .124
Sig. (2-tailed) .772 .892 .402
N 78 76 48
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
117
The final calculation computed for elementary SED students was a regression
analysis, which included 4 variables (charter status/type, parent education level,
percent teacher credentialed and percent emergency credentialed). The results of the
regression are detailed in Table 22. The table includes the R-squared values, the
number of schools in each sample (N), the coefficients for regression variables and the
t-score and significance of each variable.
The R-squared values vary greatly depending on the targeted growth year and
the type of charter school. The R-squared value for regression on SED API growth in
conversion charter schools was only 0.033 for the 2000-2001 growth year, while the
R-squared value for the regression on SED API growth in start-up charter schools was
0.337 for the 1999-2000 growth year. The sample size also varied greatly by charter
type and steadily increased over the 4-year period.
The regression variables were generally insignificant within the regressions.
Only one charter variable, the 2001-2002 variable for all charter schools turned out to
be significant at the 0.05 level. The variable had a coefficient valued at -16.744.
Based on the coefficients of each variable, the conversion charter schools impacted the
charter variable to a much larger degree than the start-up charter variable.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 22: Elementary SED API Growth Regressions
Model U nstandardized t score Significance
B Std. Error
Dependent Charter (Constant) 49.352 59.682 .827 .415
Variable:
Elementary
SED
Growth
1999-2000
Schools
Charter -9.546 14.733 -.648 .522
N=34
R Square:
0 009
Parent Ed. Level 3.877 9.075 .427 .672
%Tchr. Cred. .047 .559 .084 .934
% Emer. Cred. -.660 .829 -.796 .432
S tartu p (Constant) -7.897 71.508 -.110 .913
Charter
Start Up Charter 63.013 43.548 1.447 .166
N=21
R Square:
0 337
Parent Ed. Level 31.399 14.504 2.165 .045*
%Tchr. Cred. -.172 .732 -.235 .817
% Emer. Cred. -.861 .957 -.899 .381
Conv. (Constant) 23.261 61.935 .376 .710
Charter
Conv. -14.607 15.004 -.974 .338
N=33
R Square:
0.101
Parent Ed. Level 7.875 9.425 .836 .410
%Tchr. Cred. .203 .563 .360 .722
% Emer. Cred. -.494 .827 -.598 .555
Dependent Charter (Constant) -8.227 28.393 -.290 .773
Variable:
Elementary
SED
Growth
2000-2001
Schools
Charter .746 8.836 .084 .933
N=56
R Square:
0.004
Parent Ed. Level .183 5.541 .033 .974
%Tchr. Cred. .419 .261 1.606 .114
% Emer. Cred. .212 .313 .677 .502
Start Up (Constant) -35.304 48.919 -.722 .476
Charter
Start Up Charter 6.195 33.018 .188 .852
N=35
R Square:
0 0X7
Parent Ed. Level 1.980 8.634 .229 .820
%Tchr. Cred. .638 .448 1.423 .165
% Emer. Cred. .534 .538 .993 .328
Conv. (Constant) 2.371 30.901 .077 .939
Charter
Conv. 1.428 8.837 .162 .872
N=54
R Square:
0.033
Parent Ed. Level 3.968 5.636 .704 .485
%Tchr. Cred. .185 .304 .607 .547
% Emer. Cred. .004 .344 .012 .990
Depend. Charter (Constant) 61.886 32.369 1.912 .059
Variable: Schools
Charter -16.744 7.344 -2.280 .025*
Elementary
SED
Growth
2001-2002
N=91
R Square:
0.121
Parent Ed. Level -15.484 6.383 -2.426 .017*
%Tchr. Cred. -.014 .311 -.044 .965
%Emer. Cred. .142 .360 .395 .694
Start Up (Constant) 53.109 37.911 1.401 .170
Charter
Start Up Charter -1.261 16.417 -.077 .939
N=38
R Square:
0.170
Parent Ed. Level 9.100 7.887 1.154 .257
%Tchr. Cred. -.724 .328 -2.206 .034*
% Emer. Cred. -.397 .405 -.980 .334
Conv. (Constant) 34.361 32.256 1.065 .292
Charter
Conv.Charter -13.638 9.032 -1.510 .137
N=56
R Square:
0.109
Parent Ed. Level 4.455 5.925 .752 .455
%Tchr. Cred. -.386 .318 -1.215 .230
% Emer. Cred. -.031 .352 -.087 .931
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
119
A complete regression was calculated for the elementary SED API growth data
including all of the variables (see Appendix D, E and F). In the 1999-2000 growth
year, Appendix D, the elementary SED API was much more dependent upon charter
schools when all the variables were included. The charter variables were all relatively
significant in each regression (0.059, 0.062, 0.045 for charter schools, start-up schools
and conversion schools respectively). The coefficients were negative for charter
schools and conversion charter schools (-26.350 and -28.912 respectively) and
positive for start-up charter schools (59.170). It should be noted that the sample size
for the start-up charter schools in that year was only two.
The 2000-2001 full regression for elementary SED students (See Appendix E)
was statistically insignificant for every variable while the regression using the 2001-
2002 data (See Appendix F) was significant for most of the charter variables. The
overall charter variable had an overall negative impact on elementary SED students
with a coefficient o f-15.850, significant at the >0.05 level. The start-up charter
school variable also had a large negative impact on the elementary SED student API
results with a coefficient o f-43.983, significant at the >0.01 level. The conversion
charter school variable was also negative, but not significant at the >0.05 level.
Secondary SED API Growth Data Analysis
In regard to secondary SED students, the same calculations were completed
(i.e., mean comparison, Pearson correlation, and regression analysis). The first test
run on the secondary SED API growth data set was a straight mean comparison
according to school type. The mean of secondary SED API growth was calculated for
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
120
each growth year and presented in Table 23. The table indicates that the secondary
students in charter schools outperformed the secondary students in conventional
schools two out of three years. Secondary SED students in conversion schools
underperformed the SED students in conventional schools for two out of three years
while the SED students in start-up charter schools had a higher mean growth than the
students in conventional schools for the same two growth years. The vast majority of
the secondary charter schools serving SED students were site-based, leaving the non
site based category almost non-existent. In regard to actual numbers of schools
included in the mean comparison, the number of conventional schools outnumbers the
number of charter schools with SED API data, although there is a steady increase in
the number of charter schools per growth year.
Table 23: Secondary SED API Growth Mean Comparison
School Type
1999-2000
Growth
2000-2001
Growth
2001-2002
Growth
1999-2002
Growth
Conventional
School
Mean
25.65 8.59 8.70 44.7857
N
31 44 43 28
Std. Deviation 32.861 36.744 28.286 33.25698
Charter School Mean
7.88 10.38 20.30 47.4286
N
8 16 20 7
Std. Deviation
21.013 42.491 41.160 53.73037
Conversion Mean
8.17 -1.56 17.44 28.2000
Charters
N
6 9 9 5
Std. Deviation
16.774 22.738 15.525 23.82646
Start Up Mean
7.00 25.71 22.64 95.5000
Charters
N
2 7 11 2
Std. Deviation
41.012 57.760 54.888 92.63099
Site Based
Charters
Mean
7.88 19.38 22.59 47.4286
N
8 13 17 7
Std. Deviation
21.013 41.798 35.005 53.73037
Some Non-Site Mean
-12.00 -82.00
Based Charters
N
1 1
Std. Deviation
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
121
Table 24 includes the Pearson correlation results for the secondary SED API
growth by year. The correlations define the relationship, in isolation, between the API
growth of SED students and school type. According to the data, none of the growth
years have correlation significant at the >0.05 level, although the start-up charter
schools do indicate a positive correlation with secondary SED growth at the >0.10
level for the 2001-2002 growth year.
Table 24: Secondary SED API Growth Pearson Correlation
G rowth Y ear C harter Conversion Start-U p Site
1999-2000 Growth Pearson Correlation
-.231 -.208 -.137 -.231
Sig. (2-tailed)
.156 .216 .447 .156
N
39 37 33 39
2000-2001 Growth Pearson Correlation
.021 -.111 .149 .121
Sig. (2-tailed)
.874 .431 .297 .371
N
60 53 51 57
2001-2002 Growth Pearson Correlation
.165 .126 .161 .206
Sig. (2-tailed)
.197 .375 .244 .115
N
63 52 54 60
1999-2002 Growth Pearson Correlation
.029 -.187 .333 .029
Sig. (2-tailed)
.870 .297 .072 .870
N
35 33 30 35
The final calculation completed on secondary SED API growth for the
regression analysis including the 4 excluded variable from the matched data: charter
status, parent education level, percent teacher credential and percent emergency
credential. The data is presented in Table 25. The table indicates the R-squared value,
number of schools included in each regression (N), and the coefficient, t-score and
significance of each regression variable.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
122
Table 25: Secondary SED API Growth Regression Analysis
Model B Std. E rro r t score Significance
Dependent Charter l Constant! 425.573 171.180 2.486 .020*
Variable: Schools Charter -16.598 14.499 -1.145 .263
Secondary N=30 Parent Ed. Lvl -5.655 9.751 -.580 .567
SED Growth R Square: %Tchr. Cred. -3.695 1.832 -2.017 .054*
1999-2000 0.313
% Emer. -5.191 2.117 -2.453 .021*
Start Up (Constant! 432.259 211.187 2.047 .054*
Charter SU Charter -14.189 33.543 -.423 .677
N=24 Parent Ed. Lvl -5.459 11.748 -.465 .647
R Square:
%Tchr. Cred. -3.741 2.258 -1.656 .113
0.287
% Emer. -5.512 2.589 -2.129 .046*
Conv.Charter (Constant! 416.317 195.247 2.132 .043*
N=28 Conv. Charter -14.667 16.390 -.895 .380
R Square: Parent Ed. Lvl -6.903 10.582 -.652 .520
0.269
%Tchr. Cred. -3.579 2.073 -1.726 .097
% Emer. -4.905 2.414 -2.032 .053*
Site-based (Constant! 425.573 171.180 2.486 .020*
Charters Site -16.598 14.499 -1.145 .263
N=30 Parent Ed. Lvl -5.655 9.751 -.580 .567
R square .313
%Tchr. Cred. -3.695 1.832 -2.017 .054*
% Emer. -5.191 2.117 -2.453 .021*
Dependent Charter (Constant! 6.130 103.810 .059 .953
Variable: Schools Charter 9.552 11.622 .822 .415
Secondary N=56 Parent Ed. Lvl -12.848 7.507 -1.711 .093
SED Growth R Square. 070
%Tchr. Cred. .420 .952 .441 .661
2000-2001
% Emer. .257 1.644 .156 .876
Start Up (Constant) -.024 112.693 .000 1.000
Charter SU Charter 19.869 16.466 1.207 .234
N=47 Parent Ed. Lvl -10.949 8.579 -1.276 .209
R Square:
%Tchr. Cred. .418 1.023 .409 .685
0.080
% Emer. .313 1.774 .176 .861
Conversion (Constant) -145.048 104.746 -1.385 .173
Charter Conv.Charter -5.636 13.433 -.420 .677
N=49 Parent Ed. Lvl -7.737 6.942 -1.114 .271
R square .124
%Tchr. Cred. 1.795 .969 1.853 .070
% Emer. 2.020 1.582 1.276 .208
Site-based (Constant) -25.332 103.915 -.244 .808
Charters Site 20.154 12.219 1.649 .105
N=53 Parent Ed. Lvl -14.486 7.343 -1.973 .054
R square .123
%Tchr. Cred. .811 .958 .846 .402
% Emer. .517 1.651 .313 .755
Dependent Charter (Constant) 95.951 48.362 1.984 .052*
Variable: Schools Charter 7.311 9.695 .754 .454
Secondary N=51 Parent Ed. Lvl -16.113 9.716 -1.658 .103
SED Growth R Square:
%Tchr. Cred. -.424 .317 -1.335 .187
2001-2002 0.086
%Emer. -.471 .428 -1.101 .276
Start Up (Constant) 116.647 58.452 1.996 .052*
Charter SU Charter 1.632 14.294 .114 .910
N=52 Parent Ed. Lvl -18.000 11.362 -1.584 .120
R Square:
%Tchr. Cred. -.591 .396 -1.491 .142
0.099
% Emer. -.572 .494 -1.156 .253
Conversion (Constant) 73.094 56.972 1.283 .206
Charter Conv.Charter 8.333 10.066 .828 .412
N=50 Parent Ed. Lvl -9.448 8.416 -1.123 .267
R Square:
%Tchr. Cred. -.398 .464 -.856 .396
0.060
% Emer. -.304 .609 -.499 .620
Site-based (Constant) 88.476 45.679 1.937 .058
Charters Site 9.671 9.240 1.047 .300
N=58 Parent Ed. Lvl -14.685 9.026 -1.627 .110
R square .107 %Tchr. Cred. -.394 .297 -1.327 .190
% Emer. -.394 .432 -.911 .367
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
123
The R-squared values for the regressions conducted on the data from the 1999-
2000 growth year were stronger that the regressions from the other two years. Many
of the variables turned out to be statistically significant during the 1999-2000 growth
year, although none of the charter variables were statistically significant. In fact, none
of the charter variables for any of the regressions turned out to have a statistically
significant impact on secondary SED API growth.
Summary
In order to summarize the data analysis, three sets of calculations, mean
comparison, Pearson correlation and regression analysis, were conducted on each of
six data sets. The six data sets included: elementary Hispanic API growth, secondary
Hispanic API growth, elementary African-American API growth, secondary African-
American API growth, elementary SED API growth and secondary SED API growth.
These data sets were chosen because they correlate to the purpose of the study, which
was to analyze the effect of charter schools on at-risk students. Other minority groups
were excluded from the analysis since there was not a significant amount of data
available for analysis, given the fact that an API score is only generated for a minority
group when they represent at least 10% of the school population.
The mean correlations indicated that the charter schools at the elementary level
generally had equivalent or higher API growth means for Hispanic, African-American
and SED students that the conventional schools. The mean correlations at the
secondary level indicated that the conventional schools generally had equivalent or
higher API growth means for the three groups than the charter schools. Looking at the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
124
various charter types, conversion charter schools tended to have higher means that
start-up charter schools at the elementary level, while start-up charter schools tended
to have higher means than the conversion schools at the secondary level. The mean
correlations also indicated that, despite working with a sample of 322 schools, the
numbers drop significantly by level (elementary vs. secondary) and drop considerably
for the schools that actually have a significant number of Hispanic, African-American,
and SED students to trigger a disaggregated API score. Furthermore, the data sets for
start-up charter schools tended to be quite small.
In regard to the Pearson correlations, almost none of the charter variables
(charter status, conversion charter or start-up charter) had significant correlations to
API growth for any of the disaggregated groups. Only one growth year had a
significantly positive correlation for secondary start-up charter schools (for SED
students) and one growth year had a significantly negative correlation for elementary
charter, conversion and start-up schools (for SED, SED and Hispanic students
respectively). Therefore, the majority of the Pearson correlations, which assesses the
relationship between API growth for each disaggregated group and the charter status
of the school failed to reject the null hypothesis.
The regression analysis provided the most comprehensive data on the charter
effect, in relation to the other variables that may affect student achievement. The
majority of the regression analyses included 4 variables impacting API growth for
each student group: charter status/type, parent education level, percent teacher
credential and percent emergency credential. More comprehensive regressions were
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
125
also conducted on the elementary data sets with, generally, little difference in
outcome.
The regression analyses indicated that the majority of the charter variables
failed to reject the null hypothesis. Only two of the secondary charter variables
accepted the research hypothesis, one for all secondary charter schools and one for
secondary start-up schools, which both had a negative correlation to African-American
API growth during the 2001-2002 growth year. Finally, there were several charter
variables in the elementary SED data set that also affirmed the research hypothesis,
mostly having a negative impact on elementary SED growth for two separate growth
years, 1999-2000 and 2001-2002.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 5
DISCUSSION
As the charter school movement continues to gain momentum, accountability
measures have focused attention on student achievement. Prior research indicates that
charter schools, in general, are making significant progress, but the results do not
indicate that charter schools are performing substantially better or worse than non
charter schools. The purpose of the study was to build on prior research by comparing
longitudinal student achievement in charter schools and non-charter schools and allow
for disaggregated data analysis of at-risk student achievement. Considering the
extensive efforts made by legislators to address at-risk children (i.e., goals targeting at-
risk children, non-discrimination and admission policies, racial/ethnic requirements),
this study provides valuable information on how at-risk children are performing in
charter schools, a component of student achievement that has not been addressed in
the research.
Discussion
In order to measure the charter effect on at-risk children, a set of matched
charter and non-charter schools was created. The matching effort was designed to
compare like groups and isolate the charter effect. The list of schools was created
based on RAND’s use of propensity scores to match schools according to the
dominant demographic characteristics of the schools. The matched data set was
analyzed and, in general, represented similar groups. A few of the variables had a
difference greater than 5% between charter and conventional schools, but rarely was
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
127
the difference greater than 10%. In theory, a matched data set would eliminate the
matched variables as factors in student achievement. The schools were matched based
on the following variables: percent black, percent Hispanic, percent Asian, percent
other minority, percent SED, and percent EL. As a result, these variables should not
be mitigating factors in the student achievement difference between charter and non
charter schools and between charter school types. In order to accommodate for the
slight difference in the matched variables, second regressions using all the variables
were calculated (see the appendix). In general, the second regressions rarely changed
the statistical significance of the charter variable.
The calculations conducted provided a varied perspective on the achievement
data. With the matched data set, the mean comparison provided an overview of the
longitudinal performance of each demographic group in charter schools, conventional
schools and each charter type. The results indicated that Hispanic, African-American
and SED students in both charter schools and conventional schools made significant
progress over the three-year period. The average growth for all schools in California
is reported in Table 26. Compared to the average annual growth, almost every
demographic group for charter schools, conventional schools and both charter types
outperformed the average.
Table 26: Average Annual Growth in A PIfor All California Schools
Year Elementary Secondary
1999-2000 38.4 18.0
2000-2001 20.6 8.6
2001-2002 14.9 5.1
Note iii: See Zimmer, et. al., 2003
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
128
Discussion on Elementary At-Risk Student Achievement in Charter Schools
In regard to the comparison between charter and non-charter schools, there is a
general trend, according to the mean comparison, based on school level. In the
elementary category, the at-risk children in charter schools scored at or above the at-
risk children in conventional schools most of the time. Based on charter school type,
the at-risk children in conversion charter schools tended to outperform the at-risk
children in start-up charter schools, although the number of start-up charter schools
within each demographic group was very low. As the number of start-up charter
schools included in the study increased each year, the start-up charter school’s API
growth scores significantly dropped, often below the state average. One explanation
for the drop in score may be that, according to research, new charter schools often
have depressed scores during the first year of operation due to mobility (See Gill, et.
al, 2001). Since several of the start-up schools did not have API scores for the 1999-
2000 or 2000-2001 years, this may indicate that the schools became operational in
2001, resulting in a drop in academic performance.
The Pearson correlations and regressions helped determine whether the mean
API growth comparison for each demographic group was actually a result of the
children attending a charter school. In most of the correlation and regression analyses,
charter status had almost no statistically significant impact on elementary student
achievement. Only within the category of socio-economically disadvantaged students
(SED) did the charter status impact the API growth in a significant way. Among
elementary SED students, the charter status had a negative impact on student
achievement for two of the three growth years, 1999-2000 and 2001-2002.
Interestingly, the charter school SED students actually had a higher mean API growth
score for the 1999-2000 growth year, but the regression including all the variables
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
129
indicated a negative coefficient for the charter variable significant at the 0.06 level.
The results correspond to the conversion charter school growth more than to the start
up charter school growth. During that same year, the start-up charter schools actually
had a positive impact on SED API growth while conversion schools had an equivalent
negative impact. Therefore, the negative impact of charter status on SED students for
the 1999-2000 school year is a result of the SED students in conversion charter
schools.
The SED students in charter schools also had a statistically significant decrease
in API growth in charter schools during the 2001-2002 school year. During that
growth period, the depressed growth seems to result from the start-up school students
more than the conversion school students. Even though both groups had a negative
coefficient, the start-up coefficient was negative and significant at the >0.01 level (see
Appendix F). As indicated above, the negative impact of the start-up charter schools
on API growth for SED students may be a result of the new operational status of the
charter school.
Discussion on Secondary At-Risk Student Achievement in Charter Schools
Among secondary schools, the growth mean comparison indicated that, in
general, the Hispanic and African-American secondary students in charter schools
tended to have smaller API growth than the Hispanic and African-American students
in conversion schools. Among the low socio-economic secondary students, the API
growth in charter schools was greater two out of the three growth periods.
In terms of the charter effect, the regressions indicated that the charter status
had almost no statistically significant effect on secondary API growth. Only among
African-American students did the charter status impact API growth in a statistically
significant way during one growth period. Only during the 2001-2002 growth period
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
130
did the charter status have a negative effect on the API growth of African-American
students. Although the coefficient was negative for both conversion and start-up
charter schools, the start-up coefficient was statistically significant while the
conversion charter school coefficient was not. Therefore, it is likely that the start-up
charter schools produced the negative impact more than the conversion schools. The
negative growth of the start-up schools, as mentioned above, may also be a result of
the young operational status of the schools. Of the 4 secondary start-up schools, none
of the schools had scores from the 1999 school year, perhaps indicating that the
schools were relatively new schools.
Conclusions
There are several implications of this research study. First, in terms of
methodology and research design, the matched school analysis greatly reduced the
sample size for many of the calculations. The number of schools included in several
calculations, especially amongst African-American students, was very small, greatly
impacting the reliability of the results. One recommendation would be to start with an
even larger sample group, which will be possible in future studies as more and more
charter schools open up in California. Despite the limitation in sample size, the
matched school design does have its methodological strengths. As the Greene study
metaphorically indicated, the design helps compare apples to apples (2003). When
examining schoolwide data, which the API value indicates, there is a need to compare
like schools. Since the study relied upon the API index as the sole measure of student
achievement, the design appropriately matched similar schools in order to isolate the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
131
charter effect. Unfortunately, in the majority of instances, the charter effect was not
significant.
Despite the longitudinal nature of the study, the charter effect was not
statistically significant in most analyses. There are two possible conclusions based on
the low level of significance. First, the highly aggregated nature of the API score
could have diminished or hid the charter effect. Since the API index score includes so
many factors, including weighted differentiation, numerous grade levels and various
curricular areas, the charter effect is more difficult to isolate. This could be
substantiated by the numerous regressions that were conducted that did not
demonstrate any variable as statistically significant. Second, there is the possibility
that charter status is just not having a significant effect on student achievement. Since
prior research has indicated that charter schools, in general, are performing no better
and no worse than non-charter schools, the actual charter effect may be minimal. If
this is the case, charter status may not be having a statistically significant schoolwide
effect on student achievement.
Conclusions on the hypotheses
The hypotheses proposed in the research were as follows:
1. At-risk student performance is comparable in charter and non-charter schools.
2. At-risk student performance in non-classroom based charter schools is below
at-risk student performance in start-up and conversion schools, and at-risk
students in start-up charter schools outperform at-risk students in conversion
schools.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
132
The first hypothesis is generally, but weakly, substantiated by the research.
Although most of the regressions did not find that the charter effect was significant in
regard to API growth among Hispanic, African-American and SED students, the API
growth means of each group were generally comparable among charter schools and
non-charter schools. There were slight variations between elementary and secondary
students, but the variations seemed to be explained by the inclusion of new start-up
charter schools in the study, which depressed growth scores for the 2001-2002 period.
The only result that cannot be explained by the new start-up school factor, is the
negative growth correlated with charter schools and SED students during the 1999-
2000 school year. This result did not come up in the regression with few variables,
only in the regression with all the available variables, perhaps indicating that the
negative coefficient of the charter variable may be highly correlated with one or
several of the other variables. This would be substantiated by the fact that neither the
Pearson correlation nor the lesser-variable regression indicated a statistically
significant effect of charter schools.
In regard to the second hypothesis, the research did not generally support the
proposed hypothesis. Since there were few, if any, non-classroom based schools
included in the data, no conclusions could be drawn upon how non-classroom based
schools are impacting at-risk student achievement. In some respect, the lack of data
indicates that fewer at-risk students are being serviced by non-classroom based
schools, which may be an area for future research. Furthermore, the second part of the
hypothesis involving conversion and start-up schools was not substantiated by the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
133
research. In fact, since several start-up charter variables had a statistically significant
negative impact on elementary SED students in 2001-2002 and on secondary African-
American students in 2001-2002, the research did not indicate that at-risk students in
start-up schools were outperforming at-risk students in conversion schools. The
proposed explanation for the start-up charter school effect is that many of the start-up
schools were new schools, which often depressed achievement scores for the first
year.
Recommendations
Based on the research and literature review, there are several recommendations
for policy makers and for researchers. First, in order to monitor student achievement
more effectively in California, there needs to be a mechanism to track individual
students throughout the state. If the State of California had a student identification
system, student achievement could be measured more effectively, providing more
accurate and specific feedback to decision-makers. Second, despite ten years of
charter policy in California, the recent growth of charter schools and the recent
emphasis on accountability still demonstrates the juvenile state of charters schools. In
order to effectively measure the effect of charter schools, in all their diversity, there
needs to be a critical mass of schools to draw meaningful conclusions. Furthermore,
there needs to be a consistent measure of achievement. As more charter schools
beginning producing accountability data, stronger conclusions can be drawn, but there
needs to be a consistent measure of student achievement. This research study used the
API results, which are based on the Stanford Test, Ninth Edition. California no longer
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
134
administers the Stanford test. In 2003, the state transitioned to the California
Standards Test, which has yet to be quantified, and the California Achievement Test,
Sixth Edition. So far, only the California Achievement Test is quantified, yet it is the
test that does not align to the state standards, leaving the system inherently flawed in
terms of accountability. Therefore, California needs to quantify the California
Standards Test and use it consistently over a significant number of years in order to
adequately measure longitudinal achievement.
In terms of recommendations for future research, the literature study indicated
one other area of concern not addressed within this study. Based on the policy
mechanisms used by many states and local districts to impact at-risk student
enrollment in charter schools (i.e., goals, non-discrimination policies, attendance
boundaries, and racial/ethnic balance requirements), it is recommended that further
research be conducted on how those mechanisms are actually impacting at-risk student
enrollment in charter schools. Since these mechanisms are well established nationally,
there needs to be some validation on how these mechanisms actually impact at-risk
enrollment and how they correlate to the goal established by ten states to use charter
schools as a means to serve at-risk students. Second, since this study was not able to
draw any strong conclusions on at-risk students achievement in charter schools or at-
risk student achievement by charter type, further research is merited to measure the
relationship between at-risk student achievement and charter schools. With America’s
population growing more and more diverse and given the various policy tools being
used to promote the servicing of at-risk students via charter schools, there needs to be
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
a greater emphasis placed on disaggregating the charter achievement data for at-risk
students. The extensive literature review conducted in this study demonstrates the
lack of data currently available on how at-risk students are performing in charter
schools. Without accurate and comprehensive data, policy makers cannot evaluate the
effectiveness of charter policy. With more and more charter schools opening up
across the country, states like California, with ten years of charter history, have the
potential to drive national policy with research-supported conclusions.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
136
REFERENCES
Anderson, L. & Marsh, J. (1998). Early Results of a Reform Experiment:
Charter Schools in California. Retrieved July 12, 2002 from
http :/7ww w .lao.ca. gov/sricharterschools 1297-part 1 .html
August, D. & Hakuta, K. (1997). Student assessment. In D. August & K.
Hakuta, (Eds.), Improving schooling for language minority children: A research
agenda. Washington, D.C.: National Academy Press.
Berman, Paul, et. al. (1998). A National Study of Charter Schools.
Emeryville, CA: RPP International, p. 1.
Bierlein, Louann A. (1997). The Charter School Movement. In Diane
Ravitch and Joseph P. Viteritti (Eds.), New Schools fo r a New Century: The
Redesign o f Urban. New Haven, Connecticut: Yale University Press.
Bingham, C. S., Harman, P., Finney, P., Hood, A. (2001). An Exploratory
Study of the Impact of Charter Schools on Reducing the White-Minority
Achievement Gap in North Carolina. Greensboro, NC: Program on Education
Leadership, SERVE, University of North Carolina. (ERIC Document
Reproduction Service No. ED456552)
Budde, R. (1988). Education by charter: Restructuring school districts.
Andover, MA: Regional Laboratory for Educational Improvement of the
Northeast and Islands.
California Charter School Law (2000). Retrieved July 12, 2002, from
http://www.cde.ca.gov/charter/regs/law 1 -14)0.html
California Department of Education website. Retrieved May 18, 2003
from: http://datal.cde.ca.gov/dataquest
Center for Education Reform. (2003). Charter School Highlights and
Statistics. Retrieved on June 22, 2003 from
http://edrelbrm.com/pubs/chglance.htm.
ECS StateNotes: Charter Schools. (2001). Charter School Teachers.
Retrieved July 12, 2002 from www.ecs.org
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
137
Ericson, J., Silverman, D., Berman, P., Nelson, B., & Solomon, D. (2001).
Challenge and opportunity: The impact o f charter schools on school districts. A
report of the National Study of Charter Schools. Retrieved July 12, 2002 from
http://www.ed.gov/pubs/chartimpact/
Finn, C E., Manno, B. V., Bierlein, L. A., & Vanourek, G. (1997). Charter
Schools as Seen by Those Who Know Them Best: Students, Teachers, and
Parents. Charter Schools in Action Project, Part I, Washington, D.C., Hudson
Institute.
Finn, C. E., & Kanstoroom, M. (2002). Do Charter Schools Do It Differently?
Phi Delta Kappan. September 2002, p. 59-62.
Gill, B. P., Timpane, M., Ross, K., & Brewer, D. J. (2001). Rhetoric Versus
Reality: What We Know and What We Need to Know About Vouchers and
Charter Schools. Available at: wmv.rand.org/publications.
Greene, J. P., Forster, G. & Winters, M. A. (2003). Apples to Apples: An
Evaluation of Charter Schools Serving General Student Populations. The
Manhattan Institute. Retrieved July 17, 2003 from http://www.manhattan-
institute.org/html/ewp 01 .htm.
Henig, J. R., Moser, M., Holyoke, T. T. & Lacireno-Paquet, N. (1999).
Making a Choice, Making a Difference? An Evaluation of Charter Schools in the
District of Columbia. George Washington University: The Center for Washington
Area Studies.
Hill, P. T. & Lake, R. (2002). Charter Schools and Accountability in Public
Education. The Brookings Institution, Washington, D.C.
Horn, J.& Miron, G. (2000). An Evaluation of the Michigan Charter School
Initiative: Performance, Accountability, and Impact. The Evaluation Center at
Western Michigan University. Retrieved March 18, 2003 from:
www. wmi ch.ed u/e val ctr.
Jennings, W., Premack, E., Adelmann,, A., & Solomon, D. (1998). A
Comparison of Charter School Legislation: Thirty-three States and the District of
Colombia Incorporating Legislative Changes Through October, 1998. Conducted
by RPP International for the US Department of Education.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
138
Johnson, S. M. & Landman, J. (2000). Sometimes Bureaucracy has its
Charms: Working Conditions of Teachers in Deregulated Schools. Teacher
College Record. Volume 102 Number 1, 2000, p. 85-124. Retrieved April 20,
2003 from http://www.tcrecord.org.
Kane, P.R. (1998). Charter schools: Paying attention to ancillary findings.
Education Week on the Web. Retrieved July 12, 2002 from
http://www.edweek.org/ew/vol-18/07kane.hl8
Loveless, T, (2002). The 2002 Brown Center Report on American Education: How
Well Are American Students Learning? With Sections on Arithmetic, High
School Culture, and Charter Schools. Retrieved February 20, 2003 from:
www.brookings.edu.
Manno, B, V., Finn, C. E. Jr., Bierlein, L.E. & Vanourek, G. (1998). Charter
Schools: Accomplishments and Dilemmas. Teacher College Record Volume 99
Number 3, 1998, p. 537-558.
Metcalf, K. K., Theobald, N. D., & Gonzalez, G. (2003). State University
Roles in the Charter School Movement. Phi Delta Kappan. March 2003, p. 542-
545.
Michael, W.B. and J. Benson. (1994). Evolution of evaluation design. In T.
Husen, & T. H. Postlethwaite (Eds.). International Encyclopedia o f Education
(2n d ed.). Oxford, England: Elsevier Science, Ltd.
Miron, G. & Nelson, C. (2001). Student Academic Achievement in Charter
Schools: What We Know and Why We Know So Little. Retrieved April 5, 2003
from: www.wmich.edu/evalctr/
Minnesota Education Association. (1991). Why MEA Opposes Chartered
Schools. Letter to State Senator Ember Reichgott Junge dated April 11, 1991.
Nathan, J. & Boyd, W. L. (2003). Lessons About School Choice From
Minnesota: Promise and Challenges. Phi Delta Kappan. January 2003, p. 350-
355.
NAEP Data. (2003). Retrieved July 17, 2003 from
http://nces.cd.gov/nation.sreportcard/naepdata/
Nelson, B., Berman, P., Ericson, J., Kamprath, N., Perry, R., Silverman, D., &
Solomon, D. (2000). The State of Charter Schools. Fourth Year Report. RPP
International. Retrieved April 5, 2003 from: www.ed.gov/pubs/charter4thvear/
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
139
Overview of Charter Schools (2003). Retrieved October 17, 2003 from:
http://www.uscharterschools.org/pub/uscs docs/gi/ovcrview.htm/
Podgursky, M. & Ballou, D. (2001). Personnel Policy in Charter Schools.
Washington D.C.: Thomas B. Fordham Foundation. Retrieved July 13, 2002
from: www.edexcellence.net.
Riley, P. (2000). A Charter School Survey: Parents, Teachers, and Principals
Speak Out. Pacific Research Institute. Retrieved from
www .pacifi c i nstitute. or g
Rogosa, D. (2002). A Further Examination of Student Progress in Charter
Schools Using the California API. Retrieved on July 12, 2002 from www.cse.org.
Schwartz, W.. (1996). How Well Are Charter Schools Serving Urban and
Minority Students? ERIC/CUE Digest, Number 119.
Slovacek, S., Kunnan, A., Hae-Jin K. (2002). California Charter Schools
Serving Low-SES Students: An Analysis of the Academic Performance Index.
Retrieved from http://www.calstatela.edu/academic/ccoe/c_perc/rptl.pdf
US Department of Education. (2001). Overview o f Charter Schools.
Retrieved July 12, 2002 from:
www.uscharterschools.org/pubs/uscs docs/gi/overview
Vergari, S. (2000). Charter Schools in California: A Bruising Campaign for
Public School Choice. The Charter School Landscape: Politics, Policies, and
Prospects. University at Albany, State University of New York.
Wells, A. S.,Artiles, L., Camochan, S., Wilson Cooper, C., Grutzik, C., Jellison
Holme, J. Lopez, A. Scott, J., Slayton, J.,& Vasudeva, A. (1999). Beyond the
Rhetoric of Charter School Reform: A Study of Ten California School Districts.
UCLA Charter School Study.
Wells, A. S. (1998). UCLA Charter School Study, Charter School Reform in
California: Does it Meet Expectations? Phi Delta Kappan, December, 1998.
Wirt, F. M. & Kirst, M. W. (1997). The Political Dynamics of American
Education. Berkeley, California: McCutchan Publishing Company.
Wohlstetter, P. & Griffin, N. (1998). Creating and Sustaining Learning
Communities: Early Research on Charter Schools. University of Pennsylvania:
Consortium for Policy and Research.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
140
Wohlstetter, P., Griffin, N. & Chau, D. (2002). Charter schools in
California: A bruising campaign for public school choice. In S. Vergari (Ed.),
The charter school landscape (pp. 33-53). Pittsburgh, PA: University of
Pittsburgh Press.
Zimmer, R., Buddin, R., Chau, D., Daley, D., Gill, B., Guarino, C.,
Hamilton, L., Krop, C., McCaffrey, D., Sandler, M. & Brewer, D. (2003).
Charter School Operations and Performance: Evidence from California. RAND
Education. Retrieved July 8, 2003 from www.rand.org
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
141
APPENDICES
Appendix A: Elementary Hispanic API Growth Regression 1999-2000, All Variables
Model U nstandardized Coefficients t score Significance
B Std. Error
Dependent Charter (Constant) -29.169 290.750 -.100 .921
Variable:
Elementary
Hispanic
Schools
Charter 5.225 15.982 .327 .748
N=28
%Black 1.256 3.364 .373 .714
%Asian -1.560 4.295 -.363 .721
1999-2000 R Square:
0.323
%Hispanic 2.604 3.528 .738 .471
%Other Minority 1.720 3.641 .472 .643
%White .876 3.187 .275 .787
%SED -.642 .596 -1.078 .297
%EL -1.753 1.141 -1.536 .144
%First School -.984 .903 -1.090 .292
Parent Ed. Level -10.382 18.824 -.552 .589
%Tchr. Cred. .515 .859 .599 .557
% Emer. Cred. .597 1.328 .449 .659
Start Up (Constant) 38.836 429.427 .090 .931
Charter
Start Up Charter 20.297 46.559 .436 .678
N=18
%Black 3.873 4.388 .883 .411
%Asian 3.971 5.588 .711 .504
R Square:
0.733
%Hispanic 3.046 4.215 .723 .497
%Other Minority -7.574 12.545 -.604 .568
%White 2.843 4.182 .680 .522
%SED -2.115 .973 -2.174 .073
%EL .758 2.104 .360 .731
%First School -.716 1.094 -.655 .537
Parent Ed. Level -21.649 28.201 -.768 .472
%Tchr. Cred. -1.069 1.662 -.643 .544
.% Emer. Cred. -1.720 2.488 -.691 .515
Conversion (Constant) -22.431 300.879 -.075 .942
Charter
Conv. Charter 4.900 16.523 .297 .771
N=27
%Black 1.107 3.518 .315 .757
%Asian -1.647 4.440 -.371 .716
R Square:
0.309
%Hispanic 2.322 3.807 .610 .551
%Other Minority 1.479 3.874 .382 .708
%White .684 3.373 .203 .842
%SED -.647 .614 -1.053 .309
%EL -1.649 1.246 -1.323 .206
%First School -.929 .956 -.972 .346
Parent Ed. Level -9.716 19.584 -.496 .627
%Tchr. Cred. .615 .973 .633 .537
.% Emer. Cred. .675 1.404 .481 .638
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
142
Appendix B: Elementary Hispanic API Growth Regression 2000-2001, All Variables
Model U nstandardized Coefficients t score Significance
B Std. Error
Dependent Charter (Constant) 29.005 147.392 .197 .845
Variable:
Elementary
Hispanic
Growth
2000-2001
Schools
Charter -2.324 13.061 -.178 .860
N=47
%Black -.553 1.018 -.543 .590
%Asian -.362 1.742 -.207 .837
R Square:
0.139
%Hispanic .070 1.256 .056 .956
%Other Minority -1.320 3.047 -.433 .668
%White -.096 1.036 -.093 .926
%SED .294 .315 .932 .357
%EL -.470 .733 -.641 .526
%First School .138 .398 .348 .730
Parent Ed. Level -.928 16.379 -.057 .955
%Tchr. Cred. .206 .463 .444 .660
% Emer. Cred. -.127 .531 -.239 .813
Start Up (Constant) 28.328 195.302 .145 .886
Charter
Start Up Charter 52.445 49.864 1.052 .306
N=31
%Black -1.518 1.360 -1.116 .278
%Asian -2.074 2.676 -.775 .448
R Square:
0.235
%Hispanic -1.704 1.929 -.884 .388
%Other Minority -4.518 4.581 -.986 .336
%White -1.291 1.388 -.930 .364
%SED .155 .417 .372 .714
%EL 1.073 1.226 .875 .393
%First School .999 .607 1.647 .116
Parent Ed. Level 31.481 27.685 1.137 .270
%Tchr. Cred. .166 .718 .231 .820
.% Emer. Cred. -.165 .771 -.214 .833
Conversion (Constant) 5.556 149.088 .037 .970
Charter
Conv. Charter -5.425 13.429 -.404 .689
N=45
%Black -.407 1.056 -.386 .702
%Asian .167 1.813 .092 .927
R Square:
0.127
%Hispanic .360 1.446 .249 .805
%Other Minority -.685 3.202 -.214 .832
%White .055 1.073 .051 .959
%SED .274 .317 .864 .394
%EL -.723 .912 -.793 .434
%First School .010 .451 .022 .982
Parent Ed. Level -1.931 16.541 -.117 .908
%Tchr. Cred. .339 .483 .700 .489
.% Emer. Cred. .128 .564 .227 .822
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Appendix C: Elementary Hispanic API Growth Regression 2001-2002, All Variables
Model U nstandardized t score Significance
B Std. Error
Dependent Charter (Constant) 228.111 227.755 1.002 .321
Variable:
Elementary
Hispanic
Growth
2001-2002
Schools
Charter -7.642 8.615 -.887 .379
N=72
%Black -.258 2.274 -.113 .910
%Asian .960 2.267 .423 .674
R Square:
0.162
%Hispanic -.410 2.203 -.186 .853
%Other Minority -2.533 2.842 -.891 .376
%White -.133 2.239 -.060 .953
%SED -.126 .324 -.390 .698
%EL .059 .291 .203 .840
%First School -.117 .428 -.273 .786
Parent Ed. Level -20.382 13.382 -1.523 .133
%Tchr. Cred. -1.150 .497 -2.315 .024
% Emer. Cred. -1.380 .545 -2.533 .014
Start Up (Constant) 318.487 269.988 1.180 .246
Charter
Start Up Charter -20.326 15.255 -1.332 .191
N=47
%Black -1.341 2.689 -.499 .621
%Asian 2.691 2.941 .915 .367
R Square:
0.279
%Hispanic -1.507 2.597 -.580 .566
%Other Minority -5.449 3.364 -1.620 .114
%White -1.026 2.644 -.388 .700
%SED -.102 .419 -.242 .810
%EL .202 .433 .467 .643
%First School .349 .506 .690 .495
Parent Ed. Level -18.061 21.389 -.844 .404
%Tchr. Cred. -1.307 .811 -1.612 .116
.% Emer. Cred. -1.409 .916 -1.539 .133
Conversion (Constant) 346.561 233.873 1.482 .145
Charter
Conv. Charter -.881 9.237 -.095 .924
N=63
%Black -1.947 2.374 -.820 .416
%Asian -.301 2.364 -.127 .899
R Square:
0.202
%Hispanic -1.581 2.273 -.696 .490
%Other Minority -3.273 2.814 -1.163 .250
%White -1.547 2.338 -.661 .511
%SED .077 .343 .224 .824
%EL .017 .300 .056 .955
%First School .213 .492 .433 .667
Parent Ed. Level -6.514 14.646 -.445 .658
%Tchr. Cred. -1.629 .540 -3.018 .004**
.% Emer. Cred. -1.575 .572 -2.753 .008**
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Appendix D: Elementary SED API Growth Regression, All Variables, 1999-2000
144
Model U nstandardized Coefficients t score Significance
B Std. Error
Dependent Charter (Constant) -243.794 205.225 -1.188 .248
Variable:
Elementary
Schools
Charter -26.350 13.227 -1.992 .059
N=34
%Black 2.818 2.541 1.109 .279
urow tn
1999-2000
%Asian .288 2.770 .104 .918
R Square:
0.536
%Hispanic 4.118 2.307 1.785 .088
%Other Minority 5.519 2.560 2.156 .042*
%White 3.761 2.248 1.673 .109
%SED -.509 .576 -.884 .386
%EL .081 .789 .102 .919
%First School -.689 .591 -1.164 .257
Parent Ed. Level 3.476 16.580 .210 .836
%Tchr. Cred. -.392 .568 -.690 .497
% Emer. Cred. -.269 .923 -.291 .774
Start Up (Constant) -887.654 213.689 -4.154 .002**
Charter
Start Up Charter 59.170 27.831 2.126 .062
N=21
%Black 7.912 2.152 3.677 .005**
%Asian 3.502 2.349 1.491 .170
R Square:
0.914
%Hispanic 7.571 1.948 3.887 .004**
%Other Minority 16.531 5.989 2.760 .022*
%White 8.073 1.901 4.246 .002**
%SED -.521 .515 -1.012 .338
%EL 1.998 .994 2.009 .075
%First School -.309 .515 -.600 .563
Parent Ed. Level 58.688 17.153 3.421 .008**
%Tchr. Cred. -.028 .589 -.048 .962
.% Emer. Cred. -.615 .783 -.785 .453
Conversion (Constant) -269.294 207.932 -1.295 .209
Charter
Conv. Charter -28.912 13.574 -2.130 .045*
N=33
%Black 3.234 2.592 1.248 .226
%Asian .655 2.810 .233 .818
R Square:
0.542
%Hispanic 4.252 2.321 1.832 .081
%Other Minority 5.817 2.591 2.245 .036*
%White 4.036 2.277 1.772 .091
%SED -.590 .585 -1.008 .325
%EL .210 .805 .261 .797
%First School -.674 .594 -1.135 .269
Parent Ed. Level 2.628 16.671 .158 .876
%Tchr. Cred. -.332 .574 -.579 .569
.% Emer. Cred. -.293 .927 -.316 .755
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
145
Appendix E: Elementary SED API Growth Regression, All Variables, 2000-2001
Model U nstandardized Coefficients t score Significance
B Std. Error
Dependent Charter (Constant) -91.329 109.038 -.838 .407
Variable:
Elementary
SED Growth
2000-2001
Schools
Charter 2.132 9.944 .214 .831
N=56
%Black .266 .835 .319 .751
%Asian .637 1.328 .479 .634
R Square:
0.162
%Hispanic .684 1.036 .661 .512
%Other Minority -1.424 2.482 -.574 .569
%White .806 .844 .955 .345
%SED .315 .246 1.282 .206
%EL -.136 .504 -.269 .789
%First School -.039 .268 -.146 .884
Parent Ed. Level 7.096 11.430 .621 .538
%Tchr. Cred. .294 .292 1.008 .319
% Emer. Cred. .244 .345 .708 .483
Start Up (Constant) -93.416 173.865 -.537 .596
Charter
Start Up Charter 17.293 38.773 .446 .660
N=35
%Black -.435 1.197 -.363 .720
%Asian .425 2.075 .205 .840
R Square:
0.211
%Hispanic .036 1.480 .024 .981
%Other Minority -2.958 3.761 -.786 .440
%White .375 1.174 .320 .752
%SED .335 .364 .920 .367
%EL .379 .776 .488 .630
%First School .179 .371 .481 .635
Parent Ed. Level 20.836 21.801 .956 .349
%Tchr. Cred. .284 .574 .496 .625
.% Emer. Cred. .238 .639 .373 .713
Conversion (Constant) -53.027 115.394 -.460 .648
Charter
Conv. Charter .446 10.129 .044 .965
N=54
%Black .281 .834 .337 .738
%Asian .548 1.340 .409 .685
R Square:
0.099
%Hispanic .428 1.051 .408 .686
%Other Minority -1.875 2.512 -.746 .460
%White .636 .858 .741 .463
%SED .215 .261 .821 .416
%EL -.055 .507 -.109 .914
%First School -.047 .268 -.175 .862
Parent Ed. Level 5.712 11.478 .498 .621
%Tchr. Cred. .163 .345 .473 .639
.% Emer. Cred. .064 .419 .153 .879
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
146
Appendix F: Elementary SED API Growth Regression, All Variables, 2001-2002
Model U nstandardized Coefficients t score Significance
B Std. Error
Dependent Charter (Constant) 101.919 194.404 .524 .602
Variable:
Elementary
SED Growth
2001-2002
Schools
Charter -15.850 7.735 -2.049 .044*
N=91
%Black -1.020 1.932 -.528 .599
%Asian -.233 2.005 -.116 .908
R Square:
0.201
%Hispanic -.854 1.918 -.445 .657
%Other Minority -.380 2.164 -.175 .861
%White -1.276 1.952 -.653 .515
%SED .233 .321 .726 .470
%EL -.099 .303 -.327 .744
%First School -.093 .423 -.220 .826
Parent Ed. Level -2.777 12.795 -.217 .829
%Tchr. Cred. .135 .336 .402 .689
% Emer. Cred. .144 .386 .373 .710
S tartu p (Constant) 292.648 239.799 1.220 .229
Charter
Start Up Charter -43.983 15.558 -2.827 .007**
N=56
%Black -4.061 2.464 -1.648 .106
%Asian -2.567 2.637 -.973 .336
R Square:
0.329
%Hispanic -4.033 2.418 -1.668 .102
%Other Minority -3.628 2.567 -1.413 .165
%White -4.305 2.424 -1.776 .083
%SED .402 .412 .977 .334
%EL .108 .375 .289 .774
%First School .167 .465 .360 .721
Parent Ed. Level 3.428 19.087 .180 .858
%Tchr. Cred. .882 .447 1.975 .050*
.% Emer. Cred. 1.307 .552 2.366 .022*
Conversion (Constant) 185.672 185.287 1.002 .320
Charter
Conversion -9.799 7.303 -1.342 .184
N=82
%Black -.989 1.829 -.541 .590
%Asian .288 1.906 .151 .880
R Square:
0.317
%Hispanic -.481 1.803 -.267 .790
%Other Minority -.259 1.999 -.130 .897
%White -1.119 1.846 -.606 .546
%SED .163 .317 .516 .607
%EL -.270 .292 -.924 .359
%First School -.347 .429 -.809 .421
Parent Ed. Level -3.468 13.023 -.266 .791
%Tchr. Cred. -.807 .395 -2.046 .045*
.% Emer. Cred. -.889 .443 -2.008 .048*
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Implementation of educational policy: Technology implementation in a California middle school

PDF

A longitudinal look at what's important in comprehensive reform: A case study

PDF

A comparative case study: Tutoring in reading in two settings

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Los Coyotes High School District

PDF

An analysis of the implementation of content standards in selected unified school districts of California

PDF

Academic, environmental and social integration variables that maximize transfer preparedness for Latino community college students: An application of academic success models to the study of Tran...

PDF

Chinese parents' attitudes toward parental involvement: A case study of the ABC Unified School District

PDF

Exploring a team-based model for organizing schools: The case of Fenton Avenue Charter School

PDF

An analysis of student -level resources at a California comprehensive high school

PDF

Factors influencing minority parents to place their children in private schools

PDF

A case study: An analysis of the adequacy of one school district's model of data use to raise student achievement

PDF

Implementation and levels of use of technology plans in a Southern California K--12 district and its middle school

PDF

Induction practices in four public school districts in northwest Washington State: Perceptions of new teachers, mentors, and principals

PDF

Effective practices of a Spanish -bilingual and -bicultural principal vs. a non -Spanish -bilingual and -bicultural principal and the effects of their language proficiency and cultural knowledge ...

PDF

A study of doctoral student -advisor satisfaction: Considering gender and ethnic grouping at a private research university

PDF

Cognitive and motivational factors that affect the achievement of elementary students on standardized tests

PDF

HP2 schools achieving in spite of: A case study looks at programs, culture, and leadership

PDF

Elementary administrators and teachers' perceptions of the teacher evaluation process in California's public schools

PDF

An analysis of the use of data to increase student achievement in public schools

PDF

Administrative evaluations and their impact on school leadership in public and private secondary schools: A comparison study of governance structure

Asset Metadata

Creator Lund, Christopher Alan (author)

Core Title A longitudinal comparative study of the effects of charter schools on minority and low-SES students in California

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag Black studies,education, bilingual and multicultural,education, curriculum and instruction,education, sociology of,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Gothold, Stuart (committee chair), Hentschke, Guilbert (committee member), Picus, Lawrence O. (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-408682

Unique identifier UC11335916

Identifier 3145241.pdf (filename),usctheses-c16-408682 (legacy record id)

Legacy Identifier 3145241.pdf

Dmrecord 408682

Document Type Dissertation

Rights Lund, Christopher Alan

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA