University of Southern California Dissertations and Theses

Effects of the STAR testing program on teachers and the curriculum

(USC Thesis Other)

Effects of the STAR testing program on teachers and the curriculum

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content INFORMATION TO USERS
This manuscript has been reproduced from the microfilm master. UMI films
the text directly from the original or copy submitted. Thus, some thesis and
dissertation copies are in typewriter face, while others may be from any type of
computer printer.
The quality of this reproduction is dependent upon the quality of the
copy submitted. Broken or indistinct print, colored or poor quality illustrations
and photographs, print bleedthrough, substandard margins, and improper
alignment can adversely affect reproduction.
In the unlikely event that the author did not send UM I a complete manuscript
and there are missing pages, these w ill be noted. Also, if unauthorized
copyright material had to be removed, a note w ill indicate the deletion.
Oversize materials (e.g., maps, drawings, charts) are reproduced by
sectioning the original, beginning at the upper left-hand comer and continuing
from left to right in equal sections with small overlaps.
Photographs included in the original manuscript have been reproduced
xerographically in this copy. Higher quality 6” x 9" black and white
photographic prints are available for any photographs or illustrations appearing
in this copy for an additional charge. Contact UM I directly to order.
ProQuest Information and Learning
300 North Zeeb Road. Ann Arbor, M l 48106-1346 USA
800-521-0600
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
EFFECTS OF THE STAR TESTING PROGRAM ON TEACHERS
AND THE CURRICULUM
by
Regina Kay Lane
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
August 2001
Copyright 2001 Regina Kay Lane
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 3054768
Copyright 2001 by
Lane, Regina Kay
All rights reserved.
__®
UMI
UMI Microform 3054768
Copyright 2002 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UNIVERSITY OF SOUTHERN CALIFORNIA
School of Education
Los Angeles, California 90089-0031
This dissertation, written by
___________ REGINA KAY LANE___________
under the direction o f hoi— Dissertation Committee, and
approved by all members o f the Committee, has been
presented to and accepted by the Faculty of the School
o f Education in partialfulfilm ent o f the requirements for
the degree of
Do c to r o f E d u c a t io n
T 5 S S T
tb\lU
Dissertation&SiHmittee . /
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ACKNOWLEDGMENTS
With sincere appreciation to all who gave me support in conducting my
study, I extend my thanks. I wish to especially thank the following persons:
Dr. Dennis Hocevar, Dr. Robert Baker and Dr. Mike McLaughlin who shared
their expertise with me; my husband Gary, and my children Jeffery and Malinda
whose patience and encouragement sustained me throughout the study.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
TABLE OF CONTENTS
Page
ACKNOWLEDGMENTS....................................................................................... ii
LIST OF TABLES....................................................................................................vi
LIST OF FIGURES................................................................................................ vii
ABSTRACT...........................................................................................................viii
CHAPTER
1. INTRODUCTION........................................................................................1
Professional Significance of the Study......................................................3
Research Questions...................................................................................... 4
Assumptions................................................................................................. 5
Limitations and Delimitations o f the Study.............................................. 5
Definitions.....................................................................................................6
Sum m ary.......................................................................................................7
2. REVIEW OF THE LITERATURE............................................................8
Accountability..............................................................................................12
System Design............................................................................................ 15
Fairness............................................................................................. 18
Comprehensiveness........................................................................ 22
Competitiveness...............................................................................25
Consequential Validity................................................................... 28
Standards and the SAT-9...........................................................................32
Summary......................................................................................................35
3. METHODOLOGY.................................................................................... 38
Sample Population..................................................................................... 39
Survey..........................................................................................................40
Statements.........................................................................................41
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Data Collection............................................................................................41
Analysis o f D ata..........................................................................................42
4. RESULTS.....................................................................................................44
Part I - Statement Results......................................................................... 45
Curriculum Alignment...................................................................45
Enhanced Teaching........................................................................ 52
Part 2: Correlation Between Low-Performing and
Not Low-Performing Schools................................................................55
Enhanced Teaching........................................................................ 55
Curriculum Alignment...................................................................56
Correlation of Low- and Not Low-Performing Schools
Related to Enhanced Teaching.....................................................57
Correlation o f Low- and Not Low-Performing Schools
Related to Curriculum Alignm ent............................................... 58
Correlation of Teaching Experience of 10 Years or Less
and More Than 10 Years
Related to Enhanced Teaching....................................................60
Part3: Strengths and Weaknesses........................................................... 62
Strengths..........................................................................................62
Weaknesses..................................................................................... 65
Recommended Improvements...................................................................68
5. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 76
Summ ary.......................................................................................................76
Purpose.............................................................................................. 76
Alignment with Curriculum and Standards.................................. 78
Enhancement of Teaching...............................................................79
Concerns, Strengths, and Weaknesses...........................................80
Conclusions...................................................................................................82
Recommendations........................................................................................86
Practical Implications...................................................................... 88
SELECTED REFERENCES..................................................................... 89
APPENDIX A INFORMED CONSENT FOR
NON-MEDICAL RESEARCH....................................................... 102
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX B EFFECTS OF THE STAR TESTING
PROGRAM ON ENGLISH TEACHERS AND
THE CURRICULUM...............................................
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
vi
LIST OF TABLES
Table Page
1. Summary: Curriculum Alignment............................................................50
2. Mean for Low- and Not Low-Performing Schools..................................57
3. Independent Samples t-test Output............................................................61
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
LIST OF FIGURES
Figure Page
1. Curriculum Alignment................................................................................45
2. Statement 2 Results...................................................................................... 46
3 Statement 3 Results...................................................................................... 47
4. Statement 4 Results...................................................................................... 47
5. Statement 5 Results...................................................................................... 48
6. Statement 6 Results...................................................................................... 49
7. Statement 12 Results.................................................................................... 49
8. Statement 1 Results.......................................................................................51
9. Statement 7 Results...................................................................................... 52
10. Statement 8 Results...................................................................................... 52
11. Statement 9 Results...................................................................................... 53
12. Statement 10 Results.................................................................................... 54
13. Statement 11 Results.................................................................................... 54
14. Statement 13 Results.................................................................................... 55
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
viii
ABSTRACT
A study was conducted to examine the effects o f the STAR Testing
Program on teachers and the curriculum. One hundred forty-nine
English/Language Arts teachers in the state o f California completed a self-report
survey which was developed from the review o f the literature. The study was
conducted to answer four research questions which included teacher’s
perceptions o f test alignment to the California State Standards, enhancement of
teaching due to the STAR Testing Program, strengths and weaknesses of the
program, and suggestions for improving the program. Also examined was the
correlation o f whether or not a school was designated as a low-performing
school, as determined by the California Academic Performance Index, and the
results of the survey. Finally, correlation between the number o f years of
teaching experience and the responses o f the respondents were computed.
The self-report survey yielded demographic information o f respondents
and their responses to the questions regarding the STAR Testing Program’s
effects on their teaching, and curriculum. Surveys were distributed to 500
randomly selected California high schools. One hundred forty-nine 10th grade
English/Language Arts teachers responded.
Findings gathered from the data in this study indicate that the majority of
respondents disagree that the STAR Testing Program is aligned with the state
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
standards for 10th grade English/Language Arts curriculum. Over 50% o f the
respondents estimated that the everyday English/Language Arts Curriculum is
not aligned with the English subsections o f the STAR Test (vocabulary, reading,
spelling, and language). This, coupled with other responses regarding curricular
alignment o f state standards in the STAR Testing Program, indicate a strong
teacher perception regarding an overall lack o f alignment o f the STAR Testing
Program with state standards.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1
CHAPTER 1
INTRODUCTION
Education, as a major focus in political and media circles, accountability
reporting by schools, districts, and states, is receiving expanded public attention
and consequently, California has embarked on a particularly ambitious
accountability agenda. Nationally, the concept o f accountability has evolved as
taxpayers and government legislatures demand to know what the public is
getting for its money (Picus & Wattenburger, 1996). At the federal level,
national leaders in the early 1990s called on states to develop standards for their
public schools. Through the implementation of laws such as Improving
America’s Schools Act (IASA) of 1994, Congress promoted the establishment
of standards-based assistance and performance goals for schools serving lovv-
income and other special-needs students.
The federal initiatives led California to embark on an ambitious
accountability agenda to align standards and assessments and guide district and
school improvement (O’Day & Smith, 1993). During the late 1990s a number
of researchers and writers documented information regarding reform movements
(Bunting, 1999; Resnick & Hall, 1998; Slavin, 1997). However, there continues
to be a lack o f clarity as to whether attaching high stakes to accountability
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2
through standards-based testing may drive teachers to “drill and practice”
techniques on a narrow set o f skills or to focus solely on standards-based skill
development.
Skepticism is not new to the accountability movement. Eisner (1994)
insists that top down solutions are ineffective and inappropriately matched to the
complexity o f teaching. He believes that a key and unanswered question lies in
the educational effects on students o f the characteristics, values, and interests of
teachers. Other researchers express concerns o f possible test bias due to the
consistently low test performances of lower economic and minority students in
comparison to their white, middle-class peers (Sattler, 1992; Woolfork, 1998).
Darling-Hammond (1997) expresses concerns regarding feelings of
victimization o f teachers and students. She reports that teacher anxiety has been
raised and the curriculum has been narrowed to drilling for tests. Hambleton,
Jaeger, Koretz, Linn, Millman, and Phillips (1995) indicated that increased
scores might result from better test taking skills, but not more knowledge.
Sykes (1999) asserts that both standardized and criterion referenced tests
do not address the kind o f complex learning goals which standards now require.
Frequent mismatches between tests and instructional aims, narrowed curriculum
offerings, lowered efforts at innovation, and restricted opportunity to learn in
poorer schools, result as teachers choose not to focus on learning but on doing
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3
whatever is needed to avoid low scores (Smith, Edelsky, Draper, Rottenberg, &
Cherland, 1990).
Professional Significance of the Study
In April o f 1999, California legislators and Governor Gray Davis passed
their version o f a high-stakes, incentives-based accountability system, the Public
Schools Accountability Act, 1999 California (PSAA). (Padia, 2000). The law-
required, for the first time, that schools be publicly ranked based on a
performance index with some schools facing serious consequences for continued
poor performance and others receiving rewards for demonstrating progress. The
SAT-9 (Stanford Achievement Test, Ninth Edition, Form T; published by
Harcourt Brace Educational Measurement) was selected as the centerpiece o f the
STAR Testing Program. The test selected as the centerpiece of the STAR
Testing Program was the SAT-9 (Stanford Achievement Test, Ninth Edition,
Form T), published by Harcourt Brace Education Measurement.
Although improved student achievement is the most important goal,
other outcomes, such as teacher morale, the curriculum and enhanced teacher
behavior also are important as they ultimately affect student achievement. As
California continues to embark on the accountability agenda, there is a need to
investigate the effects o f the STAR program on teachers and the curriculum.
The overall purpose o f this study was to acquire data regarding the effects o f the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4
STAR Testing Program on 10th grade English/Language Arts teachers and the
curriculum.
Research Questions
This study was designed to provide information regarding the responses
o f high school English/Language Arts teachers as to the effects of the STAR
Testing Program on teachers and the curriculum. One o f the goals of the STAR
Testing Program is to enhance the alignment o f standards and improve student
learning. After a review of literature, the following four research questions were
developed.
1. Do high school English/Language Arts teachers agree that the
STAR Testing Program is aligned with the state standards?
2. Do high school English/Language Arts teachers agree that the
STAR Testing Program enhanced their teaching?
3. What do high school English/Language Arts teachers perceive as
strengths of the STAR Testing Program?
4. What do high school English/Language Arts teachers perceive as
weaknesses or concerns o f the STAR Testing Program?
Although of secondary interest, two correlational research questions also
were examined:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5
1. Are teacher reports o f curriculum alignment correlated as to
whether they are in a low-performing school, or not in a low-
performing school?
2. Are teacher reports of curriculum alignment correlated with
teacher experience?
Assumptions
The participants in this study responded with honesty. They had
knowledge of the SAT-9, 10th grade English/Language Arts State Standards and
the correlation between the two. With the amount o f emphasis on accountability
and curriculum alignment with state standards and districts providing staff
development in these areas, it is reasonable to infer high school
English/Language .Arts teachers have knowledge o f the SAT-9. The SAT-9 is in
its fourth year o f implementation.
Limitations and Delimitations o f the Study
The study was conducted using randomly selected high schools in the
state o f California. The study was conducted using 500 randomly selected high
school listed in the California Public Schools Directory, 2000. The respondents
were high school English/Language Art teachers from 149 different high
schools. The data collected were limited to the 10th grade English/Language
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Arts curriculum, and for that reason, only English Language Arts teachers were
surveyed.
The quantitative survey instrument was a Likert scale, and mailed to 500
participants. There were possibilities that all high schools on the target list did
not receive the survey or someone other than an English/Language Arts teacher
completed the survey, but there is no evidence that this happened.
This study was conducted in Spring, 2001. Most teachers in 2001 had
only one or two year o f familiarity with the STAR Testing Program. The data
collected are from high school English/Language Arts teachers pertaining to the
10th grade English/Language Arts curriculum. It is very possible that math
teachers would have reported differing results.
Definitions
For the purpose of this study the following definitions are used.
Accountability: The purpose o f evaluating the effectiveness o f schools and
teachers by measuring their students’ learning (Millman, 1997).
Content Standards: What students should know and be able to do as designed
by the California Department o f Education (National Association of
State Directors of Special Education, NASDSE, 1998).
Performance Standards: The level o f performance which is acceptable by each
state (NASDE, 1998).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7
Standardized Test— any examination that is administered and scored in a
predetermined, standard manner (NASDSE, 1998).
Summary
Chapter 1 provides an introduction o f the problem, definitions and need
for the study. Chapter 2 reviews literature focusing on accountability of the
educational system. Current literature is reviewed in terms o f definitions and
critical issues on current accountability efforts such as politics, standards,
assessment, and components needed to maintain a successful accountability
system. Chapter 3 discusses the methodology for data gathering and the
analysis procedures used. Chapter 4 presents the findings from data gathered
and addresses research questions raised. Chapter 5 summarizes the findings,
conclusions, implications generated from the study, and recommendations for
further study.
Appendix A presents the Informed Consent and Appendix B presents the
instrument used to gather data from the high school English/Language Arts
teachers.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8
CHAPTER 2
REVIEW OF LITERATURE
Although the concept o f accountability is not new to education, it is a
major focus in political and media circles where the call continues for greater
accountability in education. This call for educational accountability has been
growing for several years and continues to press onward by overlapping
interrelated initiatives at federal, state, and local levels. The focus on
accountability includes the assessment of teacher and school effectiveness that
form the basis o f the accountability movement (Millman, 1997; Odden & Picus,
1992).
The five waves of educational reform during the past 50 years include
the role of tests in tracking and selection emphasized in the 1950s, the use of
tests for program accountability in the 1960s, minimum competency testing
programs o f the 1970s, school and district accountability o f the 1980s, and the
standards-based accountability systems of the 1990s.
In 1983 the National Commission on Excellence in Education
recommended that citizens hold educators responsible for schools’ success, with
the caveat that citizens themselves provide resources and stability necessary to
education. This opened discussions of national educational goals, curriculum,
and tests as the basis for school improvements and accountability (Ravitch,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9
1995). Governmental and civic members, recognizing the need for improved
student performance, determined that accountability was the way to achieve it.
At the federal level legislators called on states to create a voluntary
system of standards. Laws passed by Congress, such as Improving America’s
Schools Act (IASA) of 1994, promoted the establishment o f standards-based
assistance and performance goals for schools serving low-income and other
special-needs students. States were encouraged to create statewide
infrastructures of aligned standards and assessments to guide district and school
improvement (O’Day & Smith, 1993).
President Bush’s America 2000 strategy defined national education goals
and called for national and state report cards to track student performance.
Clinton’s Goals 2000 Educate America Act promoted content and performance
standards and student assessments to measure progress toward standards
(Adams & Kirst, 1999).
The performance-oriented "new educational accountability” introduced
several important shifts in the nature of public school accountability (Elmore,
Abelmann, & Fuhrmann, 1996). Educational accountability in the 1990s shifted
from compliance regarding inputs and practices to student performance, from
comparative performance to performance against a standard o f achievement
(Fuhrmann, 1994; Hansen, 1993; O’Reilly, 1996). Elmore, Abelmann and
Fuhrmann (1996) saw the change in the new educational accountability program
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
10
to be an emphasis on student performance as the touchstone for state and district
governance.
In 1994 many California schools and districts administered the
California Learning Assessment System (CLAS) (2000) Grade Level
Performance Assessments to fourth, eighth, and tenth graders which covered the
content areas o f Reading, Writing, and Mathematics; and to fifth graders in the
content areas o f History-Social Science and Science. This accountability system
provided information on assessment results, as well as student participation and
student background factors used to describe schools or districts. The School and
District Summary reports presented data at the school, district, county, and state
levels. Components used in reporting results included grade level;
socioeconomic index, which was an indicator o f the educational background of
the parents; and the percentage o f students at each grade level who were
designated Limited English Proficient (LEP). Limited English Proficient
students were required to be assessed if the student had been in U.S. schools for
10 or more school months, or were receiving instruction in primary language
other than English with concomitant alternative assessment. The CLAS test was
designed using the California Curriculum Frameworks and performance levels
were compared with similar schools based on four background factors: (a)
Socioeconomic status, (b) percent o f limited LEP students, (c) student mobility,
and (d) the percent o f families in the district receiving Aid to Families with
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
11
Dependent Children. This allowed comparisons o f schools and districts with
similar student populations.
In the PSAA program all districts are required to administer the same
nationally normed basic skills, standardized test. The SAT-9 (Stanford
Achievement Test, Ninth Edition, Form T; published by Harcourt Brace
Educational Measurement) was selected as the centerpiece o f the STAR Testing
Program. Under the STAR program, virtually all students are tested every
spring in grades 2-11, including students with limited English proficiency. As
the program evolved, second and third components were added. An
augmentation that was more aligned with the state standards was added and
given to some schools that were randomly selected to participate. If a school is
found to be underachieving (low performing) the state provides school
improvement funds and the assistance o f an external evaluator who works in
concert with a community school team. If growth targets are not met in twelve
months following the implementation of a school improvement plan, local
interventions, possibly including reassignment of school staff, will take place. If
no substantial progress is made by the second year, state interventions (including
the takeover o f the governance of the school by the state Superintendent o f
Instruction or some other entity) may occur.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
12
Accountability
While the term accountability has been used for many years in relation to
public education, the majority o f available resources are focused on one or a few
elements. Oustin, Fiddler, and Earley (1998, p. 111) note that while the term is
widely used as if it were straight forward, it must be used as vague. This view is
shared by Kuchapski (1998) who notes that the pervasiveness o f accountability
as a method for reforming education suggests that a high degree o f clarity
surrounds the term, but that is not so. Kuchapski proposes a framework to put
together the disparate pieces of the accountability puzzle, but the result is
limited to a focus on the political aspect. Newman, King and Rigdon (1997)
describe the historical concept o f accountability as a relationship between a
provider of a service and the agent who has the power to reward, punish, or
replace the provider. These writers’ select a definition that is an adaptation of
that proposed by other writers: “accountability is a process by which schools
and school systems meet their goals” (Newman, et al., p. 42). They conclude
that a complete school accountability system should include at least four parts:
information about performance, standards forjudging its success, significant
consequences, and designation o f an agent that does the judging and distribution
of consequences.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
13
Frymier (1996) anchors accountability to the concept o f evaluation using
the following reasoning as the basis for his definition: to be accountable means
to be responsible; assessing responsibility involves judging performance against
a criterion; judging performance against a criterion means to evaluate; therefore,
accountability requires evaluation. He concludes that in most o f its present
forms, accountability is an instrument of control rather than a vehicle for
improvement. MacPherson (1998) stresses that definitions o f accountability are
based on values, political ideologies, and epistemologies and focuses on
particularly difficult challenges for urban schools in terms o f ethical, economic,
and practical dilemmas in designing an accountability policy.
One result of a West Ed/MAP (1999) study of California schools which
aligned with those of Hill (1999) found that the problem o f accountability is
built into the basic arrangements used to run public education, such as new
initiatives that are designed to address problems, but that only weaken schools
by adding district-wide programs and mandates based on “one-size fits all”
beliefs.
Despite the efforts to define the term accountability, differences and
confusions continue to appear. However, accountability remains the rallying cry
o f the current educational reform movement. As Tacheny (1999) writes,
“Accountability might be the most overused sound bite in education today” (p.
62). Elmore (1997) stated, “Accountability for student performance is one of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
14
the two or three--if not the most— prominent issues in policy at the state and local
levels right now” (p. 8). Although accountability means different things to
different people, the most common conception is that accountability is
synonymous with assessment (Guth, Holtzman, Schnieder, Carlos, Smith,
Hayward, & Calvo, 1999).
Darling-Hammond (1997) takes another position as she sees
accountability systems to be for the benefit o f politicians with winners and
losers being created for short-term political purpose. If however, accountability
exists for the benefit o f the schools and students, then one must ask if the system
causes improvement or impairs teaching and learning. A successful
accountability system must include policies that help teachers and schools
improve their practices that result in higher achievement.
Odden (1995) contends that the key theme for the accountability
movement is to hold schools accountable for the educational outcomes o f their
children. Along the same lines, Rothman (1995) suggests accountability is a
process in which parents or other constituencies can be sure that the school
meets its goals. DeMoulin and Kendall (1993) define an accountability network
from a sociological perspective consisting o f community members, politicians,
university personnel, parents, teachers, administrators and students. Slotnik and
Gratz (1999) assert that true accountability is the result of using accountability
processes as a tool for improving student achievement as opposed to punishing
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
15
schools and staffs. Millman and Schalock (1997) conclude that educators
believe teacher knowledge and skills determine effectiveness while parents and
legislators prefer to use gains in student achievement as the test of teacher and
school effectiveness.
For accountability to be effective, it must be systemic and coherent in
nature, and it must be clearly communicated at all levels of the educational
system. It is not enough for individual pieces o f accountability to be present;
they must reinforce and align with one another (Hill, 1999; Guth, et al., 1999).
The development and implementation of an accountability system is by no
means a simple, straightforward task. Not only must the system be designed in
ways that are likely to raise student achievement but also it must be designed
and implemented with the constraints o f available time and resources. The
system must also be coordinated with other existing or emerging policies,
programs, and initiatives. Accountability systems, whatever the design, share
the purpose o f evaluating the effectiveness o f schools and teachers by measuring
their students’ learning (Millman, 1997).
System Design
Although the groundwork for a statewide system o f standards has been
laid, how well the state’s assessment and accountability system will act as a
lever for inducing districts and schools to implement such standards is still in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
16
question. Regardless o f the design o f accountability systems, its purpose is to
evaluate the effectiveness o f schools and teachers by measuring their students’
learning (Millman, 1997).
In the development o f accountability systems Brown (1990) identified
the following five components o f an accountability program:
1. Key actors need to be identified and what they are held
accountable for clearly defined.
2. Clear goals need to be stated.
3. Adequate resources as well as control over the resources at the
appropriate level need to be identified.
4. Predetermined standards need to be in place.
5. Sanctions and rewards should be in place.
Newman, King and Rigdon (1997) assert that a complete school
accountability system needs a minimum of four parts:
1. Information regarding school’s performance such as test scores
should be available.
2. Standards should exist which are used to judge the degree of
success of performance.
3. Significant consequences such as rewards or sanctions including
bonuses for teachers for success or failure in meeting the
standards in place.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
17
4. An agent such as the district or state is required. The agent
receives information on organizational performance, judges the
success or failure against the standards and then distributes
sanctions and rewards.
Fuhrmann (1999) identifies newer accountability approaches as an
addition to a focus on student performance, state accountability, systems focused
on continuous improvement strategies, new forms of inspection, reporting,
consequences, and a recognition of internal and external accountability.
In Millman’s (1977) book Grading Teachers. Grading Schools: Student
Achievement a Valid Evaluation Measure? he established four criteria to judge
how well accountability systems carry out the function o f evaluating the
effectiveness of teachers.
1. Measurement should be fair to teachers and schools.
2. The accountability system must be broad in its
comprehensiveness and include a wide range o f learning
outcomes.
3. The competitiveness of the measurement with other measures and
methods.
4. The accountability system must maintain consequential validity.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
18
The remainder o f this chapter provides a review o f literature that draws
on Millman’s (1977) four criteria o f judging how well accountability systems
evaluate the effectiveness o f teachers and schools.
Fairness
The problems involved in the selection and administration o f statewide
assessments to measure progress toward the achievement o f standards are
seemingly endless. With “ideas about how to stimulate and measure school
improvement” still in its infancy (Darling-Hammond & Ascher, 1991, p. 13)
there are concerns regarding attempts to determine a teacher’s effectiveness by
measuring student learning that include factors beyond the teachers control
which affect the amount students learn (Millman, 1997). Educators express
concern with factors such as student ability, resources, class size, and
socioeconomic levels. It is noted that data based on limited indicators are
insufficient to use as the basis o f a comprehensive education indicator
information system (Bradley, 1999).
The selection of indicators have a defining influence on accountability
systems: who will be held accountable, they should allow for fair comparisons,
and provide an integrated picture of the schooling environment (Guth, et al.,
1999).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
19
Literature regarding the fairness o f accountability provides perceptions
o f fairness expressed by educators, writers and researchers. Millman (1997)
expresses that fair accountability systems use the prior achievement of students
in the school to provide a range o f expected outcomes. Popham (1999, p. 15)
maintains that standardized tests do not measure educational quality, and he
recommends that educators carry out a campaign to ensure that everyone
concerned understands the shortcomings o f this type of measurement.
At the same time, newer forms of assessment that were introduced as
more educationally relevant, such as performance events and portfolios, have
also been widely criticized. Koretz (1998) discusses the reliability and validity
o f four examples o f large-scale assessments that use portfolios Vermont,
Kentucky, Pittsburgh, and the National ssessment of Educational Progress
(NAEP). He concluded that
Portfolio assessment has attributes that are is particularly
appealing to those who wish to use the assessment to encourage richer
instruction.. .But, some o f these attributes may undermine the ability of
the assessments to provide performance data of comparable meaning
across large numbers of schools. (Koretz, 1998, p. 334)
The Kentucky Education Reform Act (KERA) designed a program that
would
be more fair and equitable than most previous programs used to
make judgments about schools. Traditional programs have compared
schools on resources . . . or on academic success . . . without regard to
any history o f inequities in school funding or the financial means of the
local community and parents. KERA changed this by judging schools
based on their progress toward state-set goals that were based on the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
20
schools’ initial academic scores and thus eliminated many potential
sources o f inequity. (Millman, 1997, p. 197)
The Dallas accountability system “controls for preexisting student
differences in ethnicity, gender, language proficiency, and socio economic status
. . . and prior achievement levels. Additionally, the hierarchical linear m odel. . .
employed controls for school-level variables” (Millman, 1997, p. 82).
Oregon’s work sample methodology, “Pre- to post-instructional gain
scores are calculated on a student-by-student basis, with separate analysis
required for initially high- and low-scoring pupils . . . Descriptors o f classroom,
school, and community context variables accompany all measures of learning
gain” (Millman, 1997, p. 245).
Although researchers at the UCLA National Center for School
Evaluation, as well as American Educational Research Association (AERA)
claim that the use of single measure evaluation is unfair and ineffective in the
improvement o f education. PSAA currently uses the SAT-9 test as the only data
in evaluating California schools. In a study conducted by Rich (2000) concerns
were expressed by teachers, parents, and media reports regarding the unfairness
in using only one measurement o f student performance. Educational
accountability based on performance is seen as unfair as it places too much
pressure on educators to do what it takes to raise test scores.
“Far too many educators feel that accountability carries connotations of
power being exerted over them by external quality control offices. We are
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
21
reminded o f politicians, administrators, or demagogues promising to hold others
accountable for the results o f their performance” (Sagor, 1996, p. vii).
Wong and Moulton (1998) comment that student outcomes may serve as
a useful indicator, but accountability programs often fail to specify the critical
link between a wide array o f institutional actors and school performance. They
expand the scope of those to be held accountable to everyone who has a role that
affects the schools, starting with the governor.
To ensure fairness, the question o f who influences student achievement
and to what degree must be determined. “The extent that students themselves
are responsible for their own performance, then educational accountability
systems must either recognize performance factors that operate beyond
educators’ control, and adjust sanctions in kind, or affirm students as
accountable agents and craft incentives to promote their compliance” (Adams &
Kirst, 1999, p. 480).
Tests cannot distinguish educational quality from other factors such as
students’ background, and tests cannot simultaneously serve monitoring and
accountability goals.
Despite the elaborate system o f rules and regulations produced by
... various accountability mechanisms, there is little belief that
the outcomes o f education are under anyone’s control...
The sheer complexity o f the mission and the environment
imposes limits on our ability to hold public education
accountable. In education, the normal complexity of
a public agency is compounded by the fact that we
are trying to educate human beings with their
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
22
different skills, interests, and resources, instead o f making robots
or processing tax forms. (Brown, 1990, p. 2)
It is important to determine how to make the measurement fair for
teachers given different student populations, the amount o f improvement
required for an award, the levels and types o f awards, and how schools will be
enabled to help them produce expected improvement (Consortium for Policy
Research in Education).
Comprehensiveness
Curriculum and standards in public schools today are designed to cover a
wide range o f outcomes. Therefore, accountability systems must be broad in
their comprehensiveness and cover a wide range o f learning outcomes (Millman,
1997).
Accountability goals are more likely to be achieved through designs
based on valid causal theories about the problematic teaching or learning
behaviors that inhibit performance and about the relationship between
accountability instruments and these behaviors (Winters, 1990).
Of crucial importance is the idea of a cohesive comprehensive
accountability system. If an accountability system is to be truly effective, its
components must be coordinated so that they can work in concert with one
another to produce the desired outcomes. Assessment in and o f itself does not
constitute an accountability system (Adams & Kirst, 1998; Clotfelter & Ladd,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
23
1996; Elmore et al., 1996: Goertz, Floden & O’Day, 1995; Massell, Kist, &
Hoppe, 1997; Newman, King & Rigdon, 1997).
Comprehensiveness o f an accountability system must include
consequences and incentives. Accountability systems promote compliance with
incentives. In the educational accountability system compliance is a matter of
enhancing the motivation and capacity for principals, teachers, and students to
attain accountability goals (Adams & FCirst, 1999). Thus the challenge is to
structure incentives that effectively motivate school staff and students to pursue
accountability goals.
Hanushek (1996) and Elmore (1997) found that individuals do not
respond the same way to the same incentives. While rewards and extrinsic
reinforcement do not reduce one’s intrinsic motivation (Cameron & Pierce,
1994), individual merit and incentive pay programs were not found to work.
However, it was determined that competency-based pay and group performance
rewards can reinforce learning goals (Odden & Kelly, 1997).
In a study o f California’s accountability program conducted by West
Ed/MAP (1999) it was determined that for the system to be comprehensive it
should have included the following:
• Alignment o f state and local standards.
• Student performance standards and aligned assessments.
• Ongoing data analysis and reviews of school performance.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
24
• School improvement and intervention strategy.
• Stakeholder involvement and engagement.
• Continuous improvement o f an accountability system.
Every year since STAR’S inception in 1998 when it included just one
basic skills test, new components have been added. At the core o f the changes
are state leaders’ efforts to bring the testing system into alignment with
California’s academic content standards. EdSource (2001) reported the
following changes:
1998 SAT-9 tested Grades 2-8 on Reading, Language Arts, Math,
and Spelling.
SAT-9 tested Grades 9-11 on Reading, Language Arts,
Math, History/Social Studies, and Science using the SAT-9.
1998 SAT-9 as described above, plus these additions:
SABE/2 Grades 2-12 Spanish speakers with less than a
year in California public schools
Augmented: Grades 2-8 Language Arts and Math
Grades 9-11 Language Arts and Math (depending on the
course the student has taken).
1999 Augmented tests now called “Standards Tests.” The Math
test was increased by 50 items.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
25
2000 The addition o f the following tests:
Writing assessment in Grades 4 and 7.
History/Social Science Standards
Grade 9 Content standards for grades 4-8.
Grade 10 World History
Grade 11 U.S. History
Science Standards Test
2001 Grades 9-11 if applicable, depends on the course the student
is taking.
A class of students who will be graduating in 2004 took a
practice version o f the High School Exit Exam in the
Spring. The State Board o f Education has also
recommended that the Golden State Exam program be
integrated in the STAR Program by 2002-03 in those
subjects where both tests are given (EdSource, 2001, p. 1).
Competitiveness
The judging o f an accountability system is the competitiveness of the
measurement with other measure and methods. Measuring effectiveness with
students’ test scores is not perceived by teachers to be fair, but may be superior
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
26
to evaluation by the school principal which is viewed as less fair or less
desirable (Millman, 1997).
When the PSAA replaced the standards-based accountability system in
California, it was determined that the SAT-9 would be used to test all students.
Diane Eastin, California’s State Superintendent o f Public Education (1999)
declared that multiple measures were no longer required. However to add
competitiveness to the accountability system, the PSAA introduced a statewide
system o f ranking schools on the Academic Performance Index (API) and
provided interventions and incentives for schools.
In a study conducted by Rich (2000) it was found that educators
expressed a need for other criteria in addition to the API as it would be
important to the staff, parents, and community members in evaluating the
quality of schools. “They included climate at the school, condition of the
physical plant, student discipline, teachers staying late at school to work, and
diversity of the student population” (Rich, 2000, p. 403). It was also expressed
that the .API based solely on the SAT-9 test was too narrow and the ranking of
schools caused public humiliation for teachers and students.
It is true that the scores which are published are not an accurate
reflection of the quality of work o f the teaching staff in the schools.
Factors other than teacher work effect achievement. Therefore it is not
completely fair to hold schools accountable for matters which are outside
their control. (Rich, 2000, p. 418)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
For the first time ever, California is offering merit scholarships as
incentives to students in every public high school in the state. California has
joined 13 other states that offer some form o f merit-based scholarships to high
school students. The scholarships are designed not only to reward high
achievement but also to address socioeconomic equity issues by ensuring that
students from every high school in the state will be eligible. Whether they
attend small rural schools or huge inner-city campuses, high school students
who achieve top scores in the state’s STAR assessment are guaranteed access to
scholarship dollars (EdSource, 2001).
While California adds monetary rewards for successful schools, Oregon
and South Carolina eradicated funding o f such programs. West Ed/MAP (1999)
found that consequences and incentives were not a main focus of educators
when talking about accountability systems. However, on January 31, 2001
California’s State Superintendent o f Public Instruction announced that the
Governor’s Performance Awards would be distributed to 4,502 schools for
significantly improving test scores since last year. More than 67?/0 o f the state’s
8,000 public schools qualified for rewards by testing at least 95% o f the eligible
students and improving by a specified amount (California Department of
Education News Advisory, 2001; Record Searchlight, 2001; Sacramento Bee,
2001; San Francisco Chronicle, 2001). In addition to school awards the state
will divide $350 million among staff members at schools that also received the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
28
Governor’s Performance Awards. In contrast to what researchers have reported
that incentives are not o f importance to the raising o f student achievement and
better teaching (Cameron & Pierce, 1994; Elmore, 1996, Hanushek, 1996;
Odden & Kelly, 1997), one California superintendent was quoted as saying, “It’s
not about just whether a school gets X amount o f dollars, it’s symbolic of
whether the school is making progress.. . . The awards send a powerful
message that schools are on the right track academically” (Sacramento Bee,
2001, p. Bl).
Consequential Validity
Linking teacher effectiveness to student learning inevitably will create
some undesirable side effects (Millman, 1997). Standards and assessments
have, at best, a minimal chance o f influencing achievement when there are no
significant consequences involved. Current state policies on this aspect of
accountability systems are summarized in detail in Education Week “Quality
Counts: Rewarding Results, Punishing Failure” (Edwards, 1999):
• Forty-eight states now test their students and 36 publish annual report
cards on specific schools.
• Nineteen states have sanctions for chronically failing schools.
• Fourteen states provide monetary rewards for schools.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
29
• Nineteen states rate their schools on performance, but seven consider
only test scores, while the rest calculate in other factors such as
attendance or graduation rates.
The use o f rewards for teachers and schools is equally controversial.
The Kentucky accountability program is the most extensive implementation o f
such a system. Originally, schools that exceeded their target received a pool o f
reward funds that could be distributed by teachers for any purpose, including
salary bonuses (Kelly, 1998) but recent legislated revisions to the program put
limits on the use o f incentive awards (Edwards, 1999, p. 147). In Kelly’s (1998)
study o f the reward system in Kentucky, she concluded that teachers did not
report that financial reward was a motivator. Rather they cited the following
outcomes that provided meaningful incentives: fear o f negative publicity, a
desire for public positive recognition, the intrinsic reward of seeing students
achieve, and, for a small number, and fear o f a loss o f professional autonomy
Accountability systems can create unintended and corrupt consequences
such as narrowing the curriculum, teaching to the test, drill and practice
(Clotfelter & Ladd, 1996) and the validity of accountability measures which
affect teachers’ attitudes (Elmore, Abelmann & Fuhrmann, 1996).
Accountability contributes to the solution o f how teachers understand the
nature o f knowledge, the student’s role in learning, and how these ideas are
manifested in teaching and class work (Elmore, 1997) by focusing expectations,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
30
structuring incentives to reinforce those expectations, developing capacity, and
requiring an account (Adams & FCirst, 1999).
The mandatory, high-stakes test exerts considerable influence at the
school level, affecting what professional development opportunities are offered,
and how classroom time is spent on test preparation. West Ed/MAP (1999)
found that the influence o f the SAT-9 goes beyond preparing students for the
test and extends into the realm o f shaping the curriculum itself. Concerns that
the emphasis on the SAT-9 promotes “teaching to the test” and limiting the
curriculum to what is on the test have been expressed. West Ed/MAP (1999)
also reported positive effects resulting from the accountability system, especially
on curriculum, instruction and assessment practices.
The effects of an accountability program is to determine to what extent
the accountability data is analyzed and used to determine how well students
have performed, not only on an individual level but also on classroom, school,
subgroup, district levels, and can judgments be made about the adequacy of
performance at each level (Elmore et al., 1996; Clotfelter & Ladd, 1996;
Newman, King & Rigdon, 1997). Assessment results can be used to determine
areas o f strength and weakness in student performance, and their analysis then
can serve as one step in the process of providing curriculum and instruction to
target weaknesses (Guth, et al., 1999).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
31
West Ed/MAP (1999) reported that district personnel make greater use o f
data than do school-level personnel. About 78% o f survey respondents
indicated that district accountability and assessment personnel examine and
analyze student assessment data “to a great extent.” In contrast, only 33% of
respondents reported that teachers in their district examine and analyze data “to
a great extent.” Several of the districts recognize the data limitation at the
school level and are trying to overcome them by offering school-level personnel
professional development opportunities on how to use and analyze data more
effectively.
When presented with a number of possible reasons why a district might
collect and analyze student assessment data, and asked to rate the extent of data
use for each one, districts replied “to satisfy state and federal reporting
requirements” far more than any other listed purpose. Other reasons rated high
on the survey included the improvement o f instruction, identification of students
needing assistance, and gauging of student subgroup performance. Only a few
districts cited the use of data to identify teachers needing assistance (Guth, et al.,
1999).
The emphasis placed on accountability has led to the rapidly growing
choice options, and the intrusion of market forces into public schools, are a
direct result o f the current reform movement. Public opinion polls over the past
few years have indicated rising support for increased choice, although the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
32
majority do not favor the use of public tuition for placement in private schools
(Rose & Gallup, 1998). Two policy briefs by the Education Commission o f the
States (Anderson & Lewis, 1997; Ziebarth, 1998) examined state policies in this
area and reported that in 1998, a total o f 22 states had “academic bankruptcy”
laws in place.
Standards and the S AT-9
A look at the state standards for 10th grade English/Language Arts
provides a perspective o f what is expected o f the students. In the area of
writing, students are expected to demonstrate a command of standard American
English, research, organizational, and drafting strategies. Students must apply
their knowledge of word origins to determine the meaning o f new words
encountered in reading materials and use those words accurately. Students are
to read and understand grade-level-appropriate material, analyze the
organizational patterns, arguments, and positions o f authors. Students are to
write biographical or autobiographical narratives or short stories: responses to
literature; expository compositions; including analytical essays; research papers;
persuasive compositions; business letters, and technical documents.
The standards for written and oral English language are:
• Grammar and mechanics o f writing (e.g., main and subordinate).
• Phrases (e.g., gerund, infinitive, and participial).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
33
• Mechanics o f punctuation (e.g., semicolons, colons, ellipses,
hyphens).
• Understand sentence construction.
• Demonstrate an understanding o f proper English usage.
• Control o f grammar.
• Paragraph and sentence structure.
• Dictation, and syntax.
The standards for manuscript are:
• Legible work that shows accurate spelling and correct use o f the
conventions o f punctuation and capitalization.
• Reflect appropriate manuscript requirements, including title page
presentation, pagination, spacing and margins, and integration o f
source and support material (e.g., in-text citation, use of direct
quotations, paraphrasing) with appropriate citations.
Listening and speaking strategies require students to:
• Formulate adroit judgments about oral communication.
• Deliver focused and coherent presentations o f their own that convey
clear and distinct perspectives and solid reasoning use of gestures,
tone, and vocabulary tailored to the audience and purpose.
• Students are to deliver narrative presentation, expository
presentations, apply appropriate interviewing techniques, deliver oral
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
34
responses to literature, present persuasive arguments, and give
descriptive presentations.
The presentation of assessment used in the SAT-9 is one o f reading and
selecting the correct answer. In the area of spelling students are to read a
selection o f sentences and select the sentence with either the misspeiied, or
correctly spelled, word. Reading comprehension is selecting answers related to
article types, meaning of phrases, recall, inferences, main idea, vocabulary (e.g.,
word meanings) and prediction.
The Language section o f the assessment asks the student to select items
related to use o f resources (e.g., dictionary, thesaurus, almanac, atlas, and
encyclopedia). Students are to edit passages by selecting the correct or incorrect
use of vocabulary, punctuation, and sentence construction. Questions also
include selection o f answers regarding the organization, preparation of writing
and pronunciation o f phonetically spelled words.
According to the California State Auditor’s Report on the STAR
Program (2000, April) there are ongoing conflicts between the State Board of
Education and the Superintendent of Public Instruction as well as continued
errors that impede the programs’ success. The audit cites a lack o f follow
through o f both parties as to what was to be carried out in the implementation of
the STAR testing program.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
35
1. The superintendent had not developed an annual implementation
plan, as law requires.
2. Time pressures imposed by the Legislature made it difficult for
the board to thoroughly evaluate potential test publishers and
select a final test.
3. During the first two test cycles (Spring 1998 and Spring 1999),
the department did not closely monitor the performance of the
test publisher. The program has been plagued with missed
deadlines, unreliable data, and inaccurate reporting of
achievement test results.
4. The department must take further action to ensure the success of
the Public Schools Accountability Act o f 1999, such as pushing
for better test security.
The audit only addressed these issues and failed to address curriculum
alignment and teacher quality which would address the objective o f the
program.
Summary
A brief glance at both the state standards and a summary of the SAT-9
for 10th grade English/Language Arts in the State o f California indicate some
alignment. However, there appears to be a lack o f alignment in the type of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
36
activities required by the state and how they are assessed on the SAT-9. This,
coupled with the auditors’ findings, provides enough information to question the
validity o f the STAR Testing Program.
The synthesis of the literature on educational accountability reveals that
there is much unfinished business in the educational reform movement and gaps
in the research that need to be filled to assist in guiding that movement. Forsyth
and Tallerico (1998) note that a focus on outcomes alone--so prevalent in
accountability programs— will not adequately serve the complexities of
education. “Survey data from teachers and the public suggest that, at a symbolic
level, the idea is accepted. But there is considerably less agreement about its
operational meaning” (McDonnell, McLaughlin, & Morrison, 1997, p. 66).
There is a need to direct energy toward changing those factors that are
under the control of the school, and recognizing that the uncontrollable parts of
the system, such as demographics o f student population, should not be used as
excuses for inadequate student learning. It is increasingly problematic that
media coverage tends to focus almost exclusively on test results. The
communicating of educational results needs to be expressed in a broader, more
thorough manner to include all aspects o f the system in order to cover the full
range o f accountability. This goal can be accomplished only if the
accountability system includes all the relevant components of a comprehensive
approach from inputs and processes, through individual student learning, to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
37
overall system results. School systems and states need to attend to all o f these
elements to be able to respond to the demands for educational accountability that
characterize the current reform movement in American education.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
38
CHAPTER 3
METHODOLOGY
This dissertation is an assessment o f the effects o f the STAR Testing
Program on high school English/Language Arts teachers and the representation
o f the 10th grade English/Language Arts curriculum. To define views of the
participants the four research questions included:
1. Do high school English/Language Arts teachers agree that the
STAR Testing Program accurately assesses the 10th grade
English/Language Arts curriculum?
2. Do high school English/Language Arts teachers agree that the
STAR Testing Program enhances their teaching?
3. What do high school English/Language Arts teachers perceive as
strengths of the STAR Testing Program?
4. What do high school English/Language Arts teachers perceive as
weaknesses o f the STAR Testing Program?
The study used a self-report survey designed around questions developed
from a review o f literature. Data from the survey yielded demographic
information o f respondents and their responses to the questions regarding the
STAR Testing Program’s effects on their teaching, curriculum and morale.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
39
Results also were correlated with whether or not a school was designated as low
performing by the State o f California and with teacher experience.
In the SAT-9 instrument used to assess student knowledge in the STAR
Testing Program for writing, students are expected to demonstrate a command
of standard American English. The presentation o f the assessment in the SAT-9
is one of reading and selecting the correct answer, editing passages, and
selecting answers regarding the organization, preparation o f writing, and
pronunciation o f phonetically spelled words.
Sample Population
The unit o f analysis was high school English/Language Arts teachers
from randomly selected high schools in the State o f California. A total of 500
surveys were sent to randomly selected high schools and 149 respondents
completed and returned the survey. O f the 149 respondents .7% were 26 years
old or younger; 14.2% were 27-34 years old; 12.8% were 35-41 years old;
17.6% were 42-48 years old; and 54.7% were 49 years old or older. The
percentage o f males was 24.3% and the percentage for females was 75.7%. The
percentage for teaches with less than 10 years or less teaching experience was
38.6% and 61.4 % had more than 10 years of teaching experience.
The criteria for selecting participants in the study included high school
English/Language Arts teachers in the State o f California. Schools were
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
40
randomly selected (every third high school) from an alphabetical list o f schools
in the 2000-2001 California Directory o f Public Schools.
Survey
The survey design was based on the goals o f the STAR Testing Program
and review o f literature. The instrument is presented in the appendix.
Participants were asked to respond to 13 statements pertaining to the effects of
the STAR Testing Program on their teaching and the curriculum. Seven
statements pertained to the alignment of the curriculum i.e., the mandated STAR
Testing Program has been aligned with the state standards for English/Language
Arts; The mandated STAR Testing Program accurately assessed the 10th grade
English/Language Arts curriculum; and to what extent does the English
subsections o f the STAR Test (vocabulary, reading, spelling, and language) are
aligned with your everyday English/Language Arts curriculum. Seven
statements inquired as to how the Star Testing Program enhanced teaching i.e.,
the mandated STAR Testing Program has forced me to teach to the test; The
mandated STAR Testing Program has provided a reasonable assessment of
teacher performance. The choices included (1) strongly disagree, (2) disagree,
(3) agree, (4) strongly agree, and (5) don’t know. Three open-ended questions
followed the survey. These statements solicited information as to the strengths
and weaknesses of the STAR Testing Program and what the state could do to
improve the program.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
41
Statements
The statements are as follows.
The mandated STAR Testing Program has:
1. Provided feedback as to the quality of my teaching.
2. Been aligned with the state standards for English/Language Arts.
3. Enhanced reading across the curriculum.
4. Accurately assessed the 10th grade English/Language curriculum.
5. Established reasonable goals for increased student achievement.
6. Had a positive impact on my students’ overall English achievement.
7. Forced me to teach to the test.
8. Enhanced what I do in the classroom.
9. Motivated me to seek assistance for improving my teaching.
10. Helped teacher’s incorporate higher-order thinking skills.
11. Provided a reasonable assessment o f teacher performance.
12. Measured the wide range o f outcomes intended by the curriculum.
13. Taken away from the creative endeavors o f teachers.
Data Collection
The primary source of data collection was the survey mailed to 500
randomly selected high school English/Language Arts teachers in the state of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
42
California. The 149 returned survey responses were used to derive demographic
information and perceptions o f each participant. The survey was sent out with a
letter requesting the participation o f each participant, including a date for the
survey to be returned. Data were summarized and analyzed for patterns that
addressed the questions o f the study. Out o f the 500 hundred high school
English/Language Arts teachers who were sent surveys there were responses
from 149 participants.
Analysis of Data
Once the data were collected, it was tallied by questions according to the
responses o f the participants. For the purpose of analysis the results were
calculated as frequencies and percentages, categorized by age, gender, years
served as a teacher, and low-performing schools that received extra state
support, and those schools who did not receive extra state support. A table of
the results follows a narrative explanation of the findings. Responses to the
three open-ended statements/question are listed and grouped according to each
statement.
Low-performing groupings were determined by the Academic
Performance Index (API) reports presented on the Internet by the State of
California. Reports from 1999 and 2000 were used to determine if a school was
designated as low-performing.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The grouping o f teacher experience was determined by information
the self-reporting survey.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
44
CHAPTER 4
RESULTS
The purpose o f this study was to provide information regarding the
responses of English/Language Arts teachers as to the effects of the STAR
Testing Program on teachers and their curriculum. The goal o f the STAR
Testing Program is to enhance the alignment of standards and improve student
learning. This chapter presents the results of the survey as completed by 149
high school English/Language Arts teachers within the California Public School
System. The results are presented in three parts. Part 1 presents results as they
pertain to each question. Part 2 presents the correlation pertaining to low-
performing schools and not low-performing schools. This determination is
made by the state using the Academic Performance Index, and the schools’
relationship to curriculum. Also, the correlation o f responses pertaining to
curriculum alignment and enhanced instruction as it relates to years o f teaching
experience and responses are presented. Part 3 presents strengths and
weaknesses o f the STAR Testing Program as reported by respondents.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
45
Part 1 - Statement Results
Curriculum Alignment
There are four research questions in the present study, and 13 statements
were used to obtain data to answer the four research questions. The results are
reported by category i.e., curriculum alignment and enhanced learning.
Therefore the data is presented using statements that were related to the
category. The choices were: (1) strongly disagree, (2) disagree, (3) agree, (4)
strongly agree, and (5) don’t know.
In response to an estimation of the extent to which the English
subsections o f the STAR Test (vocabulary, reading, spelling and language) are
aligned with the everyday English/Language Arts curriculum Figure 1 shows
5.4% estimated 20% or less; 16.9% estimated 21% - 40%; 31.1% estimated4l%
- 60%; 23.6% estimated 61% - 80%; and 18.2% estimated 81% - 100%.
Curriculum Alignment
| □ Series 1
2 3
Percent Aligned
4 5
Figure 1 - Curriculum Alignment
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
46
Responses to statement 2, shown in Figure 2 indicate the following:
23.6% strongly disagree; 34.5% disagree; 31.7% agree; and 10.1% strongly
agree that the STAR Testing Program has been aligned with the state standards
for English/Language Arts.
Statement 2
Figure 2 - Statement 2 Results
Statement 3, shown in Figure 3 indicates 35.8% strongly disagree; 31.1%
disagree; 27% agree; and 5.4% strongly agree that the STAR Testing Program
has enhanced reading across the curriculum.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
47
Statement 3
Responses
□ Series 1
Figure 3 - Statement 3 Results
Statement 4, shown in figure 4 indicates 33.1% strongly disagree; 42.6%
disagree; 21% agree; and 3.4% strongly agree that the STAR testing program
accurately assess the 10th grade English/Language Arts curriculum.
Statement 4
Responses
□ Series 1
Figure 4 - Statement 4 Results
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
48
Statement 5, shown in Figure 5, indicates 25% strongly disagree; 30.4%
disagree; 34.5% agree; and 8.8% strongly agree that the STAR Testing Program
established reasonable goals for increased student achievement.
Statement 5
1 2 3 4
Responses
Figure 5 - Statement 5 Results
Statement 6, shown in Figure 6, indicates 35.8% strongly disagree;
36.5% disagree; 24.3% agree; and 3.4% strongly agree that the STAR Testing
Program had a positive impact on their students overall English achievement.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
49
40
35
30
25
20
15
10
w
«
in
c
o
a
in
o
X
a
a
a >
m
c
a >
s 5
a
a. „
Statement 6
pit
|p|
llfl
III
2 3
Responses
□ Series 1
Figure 6 - Statement 6 Results
Statement 12, shown in Figure 7, indicates 43.2% strongly disagree:
35.8% disagree; 17.6% agree; and 2% strongly agree that the STAR Testing
Program measured the wide range o f outcomes intended by the curriculum.
Statement 12
tv's*
® 10
□ Series 1
Figure 7 - Statement 7 Results
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
50
Table 1 summarizes the results for the six curriculum alignment items.
Because all items are stated in a positive way (e.g., STAR accurately assess the
curriculum), a strongly agree response was taken as an indication of teacher
endorsement. In contrast, a strongly disagree was taken as an indication of
teacher non-endorsement. The results in Table I suggest that teachers, by and
large, do not endorse the STAR Testing Program as a way to enhance the
curriculum. That is, only 10% or less of the teachers surveyed strongly agree
that STAR Testing Program is aligned, accurate, reasonable, etc. On the other
hand, approximately 25% of the teachers strongly disagreed with each statement
that endorsed the curricular effects of STAR Testing Program.
Table 1 Summary: Curriculum Alignment
Statement Not Endorsed Endorsed
2. Aligned with State 23.6% 10.1%
3. Enhanced Reading Curriculum 35.8% 5.4%
4. Accurately Assesses Curriculum 33.1% 3.4%
5. Reasonable Goals 25% 8.8%
6. Positive Impact on Achievement 35.8% 3.4%
12. Measure Wide Range of Outcomes 43.2% 2%
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
51
Enhanced Teaching
Responses to statement 1, as shown in Figure 8, show respondents
responses as 39.2% strongly disagree; 38.5% disagree; 15.5% agree; and 3.4%
strongly agree that the STAR Testing Program provided feedback as to the
quality of teaching.
Statement 1
1 2 3 4
Responses
Figure 8 - Statement 1 Results
Statement 7, shown in Figure 9, indicates 12.2% strongly disagree;
20.3% disagree; 39.2% agree; and 28.4% strongly agree that the STAR Testing
Program forced them to teach to the test.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
52
Statement 7
I 1 2 3 4
I
Responses I
I
Figure 9 - Statement 7 Results
Statement 8, shown in Figure 10, indicates 25.7% strongly disagree;
46.6% disagree; 20.3% agree; and 6.8% strongly agree that the STAR Testing
Program enhanced what they do in the classroom.
S tatem en t 8
1 2 3 4
Responses
Figure 10 - Statement 8 Results
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
53
Statement 9, shown in Figure 11, indicates 36.5% strongly disagree;
35.1% disagree; 19.6% agree; and 8.1% strongly agree that the STAR Testing
Program motivated them to seek assistance for improving their teaching.
Statement 9
1 2 3 4 |
Responses
I
Figure 11 - Statement 9 Results
Statement 10, shown in Figure 12, indicates 37.2% strongly disagree;
31.8% disagree; 27% agree; and 4.1% strongly agree that the STAR Testing
Program helped teachers incorporate higher-order thinking skills.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
54
Statement 10
1 2 3 4
Responses
Figure 12 - Statement 10 Results
Statement 11, shown in Figure 13, indicates 54.1% strongly disagree;
32.4% disagree; 10.8% agree; and 2% strongly agree that the STAR Testing
Program provided a reasonable assessment o f teacher performance.
Statement 11
1 2 3 4
Responses
Figure 13 - Statement 11 Results
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
55
Statement 13, shown in Figure 14, indicates 10.1% strongly disagree;
12.2% disagree; 39.2% agree; and 37.2% strongly agree that the STAR Testing
Program takes away from the creative endeavors of teachers.
Statement 13
1 2 3 4
Responses
Figure 14 - Statement 13 Results
Part 2: Correlation Between Low-Performing
and Not Low-Performing Schools
Enhanced Teaching
In this section results are provided for two different correlations. The
first is based on two categories: Low-performing schools and not low-
performing schools as reported by the Academic Performance Index. The
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
56
second correlation is also based on two categories: ten years or less o f teaching
experience and more than ten years of teaching experience. In this analysis, an
observed probability that equals .05 or lower was considered significant.
Curriculum Alignment
As indicated in Table 2, the mean response o f low-performing schools
for statement 2 is 2.27, while the mean response for not low-performing schools
is 2.13. The observed probability is .58. The mean response o f low-performing
schools for statement 3 is 2.0, while the mean response for not low-performing
schools is 2.0. The observed probability is .99. The mean response of worst
performing schools for statement 4 is 1.93, while the mean response for not low-
performing schools is 1.83. The observed probability is .52. The mean response
o f low-performing schools for statement 5 is 2.28, while the mean response for
not low-performing schools is 2.11. The observed probability is .47. The mean
response o f low-performing schools for statement 6 is 1.94, while the mean
response for not low-performing schools is 1.9. The observed probability is .81.
The mean response o f low-performing schools for statement 12 is 1.77, while
the mean response for not low-performing schools is 1.68. The observed
probability is .61.
All o f the observed probabilities in the curriculum alignment were above
.05 and therefore not significant. This lack o f any significant difference
conclusively indicates that respondents perceptions o f the effects o f the STAR
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
57
Testing Program on the curriculum alignment was not based on low or not low-
performing schools.
Table 2 Mean for Low- and Not Low-Performing Schools
Mean for Low-
Performing
School
Mean for Not
Low-performing
School
Observed
Probability
Statement 1 1.84 1.58 .136
Statement 2 2.27 2.13 .584
Statement 3 2.00 2.00 .987
Statement 4 1.93 1.83 .515
Statement 5 2.83 2.11 .470
Statement 6 1.94 1.90 .814
Statement 7 2.81 2.80 .951
Statement 8 2.06 2.20 .549
Statement 9 1.96 2.15 .476
Statement 10 1.93 2.20 .231
Statement 11 1.56 1.63 .714
Statement 12 1.77 1.68 .613
Statement 13 2.98 3.08 .654 I
Correlation of Low- and Not Low-Performing Schools
Related to Enhanced Teaching
As was shown in Table 1 the mean response of low-performing schools
for statement 1 is 1.84, while the mean response for not low-performing
schools is 1.58. The observed probability is .14. The mean response of low-
performing schools for statement 7 is 2.81, while the mean response for not
low-performing schools is 2.80. The observed probability is .95. The mean
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
58
response of low-performing schools for statement 8 is 2.06, while the mean
response for not low-performing schools is 2.20. The observed probability is
.55. The mean response o f low-performing schools for statement 9 is 1.96,
while the mean response for not low-performing schools is 2.15. The observed
probability is .48. The mean response o f low-performing schools for statement
10 is 1.93, while the mean response for not low-performing schools is 2.20.
The observed probability is .23. The mean response o f low-performing schools
for statement 11 is 1.56, while the mean response for not low-performing
schools is 1.63. The observed probability is .71. The mean response of low-
performing schools for statement 13 is 2.98, while the mean response for not
low-performing schools is 3.08. The observed probability is .65.
All of the observed probabilities in enhanced teaching were above .05
and, therefore, not significant. This lack of any significant difference
conclusively indicates that respondents perceptions of the effects of the STAR
Testing Program on enhanced teaching was not based on low or not low-
performing schools.
Correlation of Low- and Not Low-Performing Schools
Related to Curriculum Alignment
Table 2 presented mean responses o f respondents with ten years or less
o f teaching experience and teachers with more than ten years o f teaching
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
experience. Teachers with experience o f ten years or less had a mean response
for statement 2 o f 2.23, while teachers with more than ten years’ experience had
a mean response o f 2.26. The observed probability for statement 2 is .839.
Teachers with experience o f ten years or less had a mean response for statement
3 o f 1.88, while teachers with more than ten years’ experience had a mean
response o f 2.07. The observed probability for statement 3 is .19. Teachers
with experience o f ten years or less had a mean response for statement number 4
o fl.8 6 , while teachers with more than ten years’ experience had a mean
response o f 1.96. The observed probability is .43. Teachers with experience of
ten years or less experience for statement 5 had a mean response o f 2.19, while
teachers with more than ten years’ experience had a mean response o f 2.29. The
observed probability is .57. Teachers with experience often years or less
experience had a mean response for statement 6 of 1.79, while teachers with
more than ten years had a mean response of 2.02. The observed probability is
.11. Teachers with experience of ten years or less had a mean response for
statement 12 of 1.72, while teachers with more than ten years’ experience had a
mean response o f 1.78. The observed probability is .65.
All o f the observed probabilities in the curriculum alignment area were
greater than .05 and therefore not significant. This lack o f any significant
differences conclusively indicates that the respondents’ perceptions o f the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
60
effects o f STAR Testing Program on curriculum alignment was not based on
respondents’ number o f years o f teaching experience.
Correlation of Teaching Experience o f 10 Years or Less
and More Than 10 Years
Related to Enhanced Teaching
Table 3 presents mean responses o f respondents in low-performing
schools and respondents in not low-performing schools as determined by the
API for the schools in California. Responses are reported in two sections:
Curriculum Alignment and Enhanced Teaching.
For teachers with experience o f ten years or less, the mean response for
question number one is 1.71, while for teachers with more than ten years’
experience the mean response is 1.87. The observed probability for question
one is .27. For teachers with experience o f ten years or less, the mean response
for question 7 is 2.78, while for teachers with more than ten years’ experience,
the mean response is 2.82.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
61
Table 3. Independent Samples t-test Output
Mean for <
10 Years’
Experience
Mean for > 10
Years’
Experience t-value
Observed
Probability
Statement 1 1.71 1.87 -1.11 .27
Statement 2 2.23 2.26 -.20 .84
Statement 3 1.88 2.07 -1.30 .20
Statement 4 1.86 1.96 -.78 .43
Statement 5 2.19 2.29 -.65 .52
Statement 6 1.79 2.02 -1.63 .11
Statement 7 2.78 2.82 -.25 .80
Statement 8 2.11 2.07 .26 .79
Statement 9 2.13 1.90 1.47 .14
Statement 10 2.04 1.92 .79 .43
Statement 11 1.56 1.59 -.19 .85
Statement 12 1.72 1.78 -.46 .65
Statement 13 3.09 2.92 .98 .33
The observed probability is .8. Teachers with experience of ten years or
less had a mean response for statement 8 of 2.11, while teachers with more than
ten years’ experience had a mean response o f 2.07. The observed probability is
.79. Teachers with experience of ten years or less had a mean response for
statement 9 o f 2.13, while teachers with more than ten years of experience had a
mean response of 1.89. The observed probability is .14. Teachers with
experience o f ten years or less had a mean response for statement 10 o f 2.03,
while teachers with more than ten years had a mean response of 1.96. The
observed probability is .43. Teachers with experience o f ten years or less had a
mean response for statement 11 of 1.56, while teachers with more than ten years
had a mean response o f 1.58. The observed probability is .85. Teachers with
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
62
experience often years or less had a mean response for statement 13 o f 3.09,
while teachers with more than ten years’ experience had a mean response of
2.92. The observed probability is .33.
All o f the observed probabilities in the enhanced teaching area were
greater than .05 and, therefore, not significant. This lack o f any significant
differences conclusively indicates that the respondents’ perception’s o f the
effects o f the STAR Testing Program on enhanced teaching was not based on
respondents number years o f teaching experience.
Part 3: Strengths and Weaknesses
This section presents the comments and opinions from the respondents
regarding the strengths, weaknesses, and suggestions on how the State of
California can best improve the STAR Testing Program .
Strengths
It is forcing me to do more “bubble tests” and test prep types o f testing.
It focuses teachers who can’t stay awake.
Individual results for students.
Attempt at accountability.
Eventually, the augmented SAT-9 test might be worthwhile.
Mandates that teachers teach state standards to ensure student success.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
63
It raised the bar for both teachers and students.
It forces some students to focus on needed skills they had not yet
developed.
Emphasis moving toward higher expectations student/teacher
performance.
Forces teachers/school to review/work on curriculum.
Test taking skills, grammar and spelling.
“Forces” weak teachers to improve overall curriculum/teaching.
Causes the abolishment o f passing students who cannot do the work.
May get students’ attendance up because that has been the real killer. If they’re
not here we can’t teach them.
Normed comparisons help public relations for the district.
Provides more opportunity in expected outcomes.
Encourages reading across the curriculum.
That it is standardized.
Students and parents take it seriously.
Students are more concerned.
Helped us come up with alternate methods o f assessment to help our
students practice.
A good check to be sure. We are consistently teaching vocabulary,
grammar, and punctuation.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Attempts to highlight a wide array o f skills basic to success in real world.
It made us aware o f our problems, as does WASC.
Cheap to implement.
Creates an incentive for administrators to encourage higher standards.
By conforming every student to one standard, teachers must conform,
thereby eliminating “weak” teachers.
Reasonable questions.
Allows us to track data in order to look at a student’s skills on an annual
basis objectively.
Provides relative comparisons to other like-schools for an objective test.
Accountability and heightened awareness o f the importance o f
benchmarks, public dialogue and increased professional dialogue.
Yearly feedback; focused community on education.
Helps assess our curriculum— re-visit course descriptions, requirements,
etc.
Encourages motivation and check o f assessment.
Gives overall picture o f the school.
Helps as a standard to evaluate administrators.
It confirms that schools in well-funded districts perform better than
under-funded schools and districts.
A common measure.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
65
Getting grade-level scores on students is useful.
Shows students how to survive under high-pressure situations.
Weaknesses
Doesn’t test what is taught.
The norm referencing does not reflect our student population.
Does not align with state frameworks to which our curriculum is geared.
STAR does not align with the new High School Exit Exam.
Does not account for migrant population statistics; which is a different
population than the usual LEP student.
The pressure to “teach to the test” has cut out enrichment.
Too long, too limited.
Socio-economic/racial issues. Is the test fair for those kids?
Resources— we barely comprehend State Standards.
The rich are rewarded. The poor are punished. No effort has been made
to deal with demographics of seriously address differences in scores.
Too many ESL students at our school to ever get full, real, and authentic
assessment.
It doesn’t assess higher-order critical thinking that a research paper
would reach. W e’ve had to eliminate important curriculum.
Multiple choice is a weak/poor choice o f testing style.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
66
Does not assess what is taught.
Students don’t always take the test seriously. Thus scores don’t always
reflect instruction in the classroom.
It is socio-economically biased.
Not reflective o f student abilities. Reflects a snapshot o f one day in the
life of a kid.
It’s limited in scope, doesn’t assess group processes, and team skills,
which are also part of the state standards, not yet fully aligned.
That so much rides on a test for which students have little or no reason to
do well is frightening.
No direct link to state standards and too long— we test for five days.
No teacher input in design or implementation.
Writing is now de-emphasized.
Failure to address test anxiety.
Encourages teacher to “teach to the test.”
Socio economically and ethnically biased. Severe penalties when the
test isn’t aligned with anything nor is there data to support the validity o f what it
assesses.
Waste o f instructional time.
No feedback until September.
Not criterion referenced.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
67
No teeth or accountability for students— no incentive for tangible results.
It pits one school against another.
Not aligned with district standards.
Duplication o f efforts/time now with High School Exit Exam— no buy-in
from students— teachers get no personal feedback.
Every few years the test is reformatted so there is an artificial dip in
scores.
Students are primed for the test rather than prepared with critical
thinking skills.
Students who come in to high school at a 2.0 reading grade level can’t be
expected to get to a 9.8 so quickly.
How can high school teachers be judged on this?
High school teachers never see individual scores so have no idea where
students stand.
Providing money to teachers and schools based on results is wrong!
The standards-based section is too specific in questioning what might be
taught in more general fashion in the reading section.
Does not prepare students for the real world.
Veteran teachers, with dynamic lessons, lose valuable class time.
Used as a hammer by state; monetary rewards have caused stress, and
bad feelings.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
68
Does not measure what is good or effective in the classroom.
Too much political pressure with API rankings has made teachers “teach
to the test.”
We need more professional development time to work on aligning our
curriculum to the standards.
The STAR came much to soon after the new framework for Language
Arts. The new framework is a 180-degree change from the old literature-based
framework.
Long reading/few questions— need more questions on more meaningful
selections.
No way to judge literary analysis and writing.
Only one measure o f student performance— too much importance placed
on it.
Recommended Improvements
Incorporate a writing prompt.
Less emphasis on speed (timed).
Improve teacher training.
Stop graduating limited-proficient teachers!
Continue mastery testing for college graduation
Include an authentic assessment.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
69
Allow the High School Exit Exam to replace 10th and 11th grade tests.
Reduce time required for testing.
Reevaluate standards.
By making it an authentic assessment that actually pertains to what an
inspired classroom is about.
Before subjecting students to 11 hours o f testing, have some educational
research to substantiate that 11 hours of multiple choice testing will reveal
anything about what a 14 yr. old knows. Adults know those are horrible
circumstances we should know that it is the same for our students also.
Start over— Begin with the standards and design a test to assess student
performance on them. Involve teachers in test development. The High School
Exit Exam is flawed but is more useful than the SAT-9.
Vocabulary out of context should be replaced with vocabulary
definitions in context.
Stop linking results with dollars.
Make students accountable
Return results in early May so students and staff can base next year’s
programs on STAR results.
Abandon it in favor of one exam, the High School Exit Exam.
Eliminate it in its present form.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
70
You can’t— it is a political solution to an educational problem— if it does
not accurately assess schools. There are too many factors that are taken without
proper consideration.
Shorter sessions for testing.
Shorten it.
Give teachers a way to make students accountable in the classroom.
Fire politicians who mandate change in areas they know nothing about!
We need writing assessment.
Providing more funds to reduce class size!
Provide better teachers in the inner city.
Drop it bring back CL AS.
Scores should not be compared to socio-economic status (or in other
words, by parent education). Are you saying uneducated parents should expect
less?
Stop the competitive emphasis.
Make all data available electronically so schools can track progress
internally.
Provide incentives to students to do well— API money is a start— why
can’t scores be placed on transcripts?
Students need individual scores, otherwise the test means nothing to
them. It is difficult to motivate them to do their best.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
71
Provide feedback for teachers. Perhaps the district receives detailed
feedback, but I do not.
Align to standards so testing does not use so much class time.
It seems as if this is merely a political football. We should toss this test
and use a test that is aligned with California and Los Angles standards.
Involve teachers in the curriculum planning and testing.
Judge by improvement, not by level, until students who enter high school
have mastered skills for grades 1-8!
Make passing it a requirement for promotion. Until the students care, all
the teachers’ work will be pretty much wasted. The students despise
standardized tests and will “blow it o ff’ if they have no reason to pass!
Then teachers won’t be so frustrated to see their work has no effect
because students don’t care.
Re-evaluate and readopt when current commitment has expanded. It is
not in alignment in social standards.
Incorporate/revise augmentations to better align with content standards.
Drop it!
Use the tests as one evaluative tool. Writing portfolios and other
projects should be considered for a truly accurate representation o f knowledge
gained.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
72
Instead o f having adopted an off-the-shelf test and tweak it toward
standards, the state should design an entirely new test!
Give teachers many exercises in grammar and punctuation that align
with test questions.
Scrap the test and find another way to assess how schools are
performing!
Use more than a one-shot test as an assessment o f progress.
Give more support to low-income schools.
Review the curriculum across the state and develop the test based on
findings.
Promote state supported mandatory in-services.
Better feedback as to weaknesses.
It is an excellent measure as it now is.
Provide opportunities beyond multiple choice to assess student progress.
If you want to test on facts, give the teachers a clear list of what students
are expected to know and choose from that, or keep it to information that
students should be able to grasp from context— especially vocabulary.
Change scoring method. Include performance items like the CAP test
did in the late 1980s, early 1990s.
The test needs to consist of authentic assessments based on some o f the
other modalities we see in our student population.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
73
Norms need to be based on a demographic sampling representative o f the
entire state (this is information that should be available now that the census 2000
is complete).
Kill it! Make Golden State mandatory. Why have two?
Develop a 45-minute test for just reading and another 45-minute test for
math.
Do away with it and return quality control to local school districts.
These kinds of tests wreak havoc on students’ lives and on quality teaching.
Rating schools on the basis of test is evil.
Move away from rote-type context.
Offer grading sessions so that teachers can be involved. CAP testing
was the most effective. Golden State for College Prep students should be
voluntary. We need to cut down on testing whenever possible.
Take into consideration non-English speaking students or Limited
English Program students when factoring the API.
Test in real situations. Life’s answers are not labeled A, B, C, D, or E.
Work with individual school districts.
It can’t be improved~it should be replaced by GSE or other modes of
better testing— no single multiple choice test can measure learning.
Make it accessible to students in their native language.
Do not bait educators with Award Systems-M erit Pay!
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
74
Align all curriculum materials to any testing program first.
Use test scores as indicators, but only as one indicator, not the sole basis
for assessment and school/teacher performance.
Recognize that we are a very diverse community.
Group socio economic districts together. Do not expect one high school
to compete with another, particularly in diverse economically/social areas.
Require higher-level thinking skills.
Look into all scores from elementary through high school to note
improvements.
Provide more training in the areas o f reading across the curriculum; we
need funds for resources and smaller classrooms.
Set goals for achievement, not a Bell-curve API.
The STAR is not the appropriate test to rank students in the state. The
science and history don’t match.
Set a score a student needs to achieve, not a norm-referenced test.
Get rid o f it~Focus on the California High School Exit Exam. To base
bonuses on STAR criteria and meeting arbitrary goals is absolutely ludicrous.
Statistics can be manipulated too easily. Obviously, we should have all done
poorly the first go-around.
Try to eliminate some o f the redundancy. Why does this test have to be
such an endurance contest (3 weeks!)?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Get rid o f it and the teacher bonus. Teacher bonuses based on student
scores does not reflect good teaching by every teacher on that campus-nor does
it consider that the clientele-poor and unmotivated students outnumber their
counterparts at some schools with wonderful teachers! They will almost never
qualify for a bonus.
Respondents’ recommendations for improvement of the STAR Testing
Program vary based on needs, frustration, and lack o f knowledge. Responses
range from “get rid of the test” to “provide staff development,” reevaluate the
reason for the program, and is the test providing appropriate information that is
enhancing the education of children. Other suggestions concerned the length of
the test, the requirement o f higher-order thinking skills, and the evaluation of the
incentive program.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
76
CHAPTER 5
SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS
This chapter summarizes the study, draws conclusions from the findings,
and makes recommendations based upon the conclusions.
Summary
Purpose
The purpose of this study was to assess the effects of the STAR Testing
Program on high school English/Language Arts teachers and the representation
of the 10th grade English/Language Arts curriculum. The participants were high
school English /Language Arts teachers from randomly selected high schools in
the State o f California.
A self-report survey was designed using questions developed from a
review of literature. Data from the survey yielded demographic information
about the respondents and their responses to the questions regarding the STAR
Testing Program’s effects on their teaching and curriculum. Results also were
correlated with whether or not a school was designated as a low-performing
school or a not low-performing school by the State of California and with
teacher experience.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
77
The following research questions were investigated:
1. Do high school English/Language Arts teachers agree that the
STAR Testing Program is aligned with the state standards?
2. Do high School English/Language Arts teachers agree that the
STAR Testing Program enhanced their teaching?
3. What do high school English/Language Arts teachers perceive as
strengths of the STAR Testing Program?
4. What do high school English/Language Arts teachers perceive as
weaknesses or concerns of the STAR Testing Program?
Although of secondary interest, two correlational research questions
were examined.
1. Are teachers’ reports of curriculum alignment correlated as to
whether or not they are in a low-performing school?
2. Are teacher reports of curriculum alignment correlated with teacher
experience?
The data were collected and tallied by statements according to the
responses of the participants. For the purpose of analysis the results were
calculated as frequencies and percentages, categorized by age, gender, years
served as a teacher, and low-performing schools that received extra state support
and those schools who did not receive extra state support. Responses to the
three open-ended questions are listed and grouped according to each question.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
78
The analysis o f the findings focused on the following areas:
1. Alignment of the STAR Testing Program with the 10th grade
English/Language Arts curriculum.
2. Effects o f the STAR Testing Program on the enhancement o f
teaching the 10th grade English/Language Arts curriculum
3. The strengths, weaknesses, concerns, and suggestions for
improvement o f the STAR Testing Program.
Alignment with Curriculum and Standards
The self-report survey indicated that the majority o f respondents disagree
that the STAR Testing Program is aligned with the state standards for 10th grade
English/Language Arts curriculum. Over 50% o f the respondents estimated that
the everyday English/Language Arts curriculum is not aligned with the English
subsections o f the STAR test (vocabulary, reading, spelling and language).
This, coupled with other responses regarding curricular alignment o f state
standards in the STAR Testing Program, indicate a strong teacher perception
regarding the lack o f alignment o f the STAR Testing Program. With the
Language Arts curriculum, this is growing more problematic as districts and
schools are changing curriculum and instruction to align with standards that are
not aligned with the SAT-9, and the SAT-9 is the basis for all major
accountability decisions.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
79
Enhancement o f Teaching
Responses to statements 1 ,8 ,9 , 10, and 11 on the survey indicate that
the majority of respondents disagree that the STAR Testing Program has
enhanced their teaching. On statement 7, “The mandated STAR Testing
Program forced me to teach to the test,” over 50% o f the respondents agreed.
Responses to statement 13, “The mandated STAR Testing Program has taken
away from the creative endeavors o f teachers,” over 77% of respondents agreed.
Although the STAR Testing Program has had an effect on teaching, the data
reveals that the respondent in this study do not feel that the STAR Testing
Program enhances what they do.
The information reported above reiterates reports referenced in the
review o f the literature that when emphasis is placed on the results o f one
assessment instrument such as the SAT-9, which is the sole assessment
instrument in the STAR testing program, it promotes “teaching to the test” and
that it limits the curriculum to what is on the test. Although there is observable
evidence that districts are structuring staff development to align curriculum with
the state standards, i.e., some districts have used a multitude of staff
development days to improve instruction and align curriculum with the
standards. Results from the respondents in this study report that the STAR
Testing Program has not enhanced, motivated, or provided feedback or
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
80
reasonable assessment o f their teaching. This is contrary to research that
indicates the accountability program has made a positive impact on curriculum
and teaching as reported in West Ed/MAP (1999). The research indicated that
the implementation of accountability programs caused districts and schools to
modify instruction based upon the results of the assessment to enhance quality
teaching.
Is it possible that respondents are not familiar with state standards and
the types o f knowledge and skills assessed on the SAT-9 test? The correlation of
responses with years of teaching experience did not show a significant
difference between those with 10 years or less and those with more than 10
years o f teaching experience. Also there were no significant differences in
responses from low performing and not low-performing schools. Therefore, it
could be concluded that the respondents are correct in their assessment of
curriculum and that state standards lack of alignment with the STAR Testing
Program.
Concerns. Strengths, and Weaknesses
The respondents in this study expressed several concerns and
weaknesses regarding the STAR Testing Program. A summary of the concerns
and weakness are discussed in this section.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
81
Although each school site is permitted to develop their own test
schedule, many respondents shared that the implementation time varies from
five to ten days including make-ups for those students who were absent on the
day of the original test.
There is a strong concern regarding the lack of teacher and student ownership in
the assessment process. Many fear that the test, although not aligned with
standards, will drive the entire educational program. Although the Academic
Performance Index (API) established by the Public Schools Accountability Act
o f 1999 eventually will be composed of several measures, at present, the only
measure for which reliable and valid data are available is the STAR multiple
choice test (currently the SAT-9). Many teachers reported that tests are
meaningless to students, but the emphasis placed by the API and state was of
great importance to teachers and the school. This has brought about a great
concern over the unfairness o f the test, as it requires all students be tested in
English even if they are also taking a primary language Normed Referenced
Test. It is felt that the data used to determine the API should include attendance,
alternative assessments, socioeconomic levels, parent support, teacher turnover,
transience, student and teacher attendance, and the need to take into account the
differences in schools and students, resources, and teacher preparedness.
Strengths expressed by respondents indicated that public schools and
educators need to be held accountable. It was reported that many schools used
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
82
the information provided by the STAR Testing Program to evaluate curriculum
and programs in which they worked. It was expressed that the STAR Testing
Program helps students to focus on needed skills as it provides individual results
for students which are useful in determining their strengths and needs. Although
there are flaws in the present accountability system, the STAR Testing Program
is viewed as an attempt at accountability and providing guidelines for the
improvement of instruction. The STAR Testing Program provides leverage for
opening communications among schools and the community, because the testing
program is becoming readily available to the public.
Conclusions
Is the STAR Testing Program reaching the goals established by the State
of California? Does the STAR Testing Program assess student progress and
enhance instruction? Using the data obtained in this study several conclusions
can be drawn concerning the effects of the STAR Testing Program on
English/Language Arts teachers and the curriculum.
1. The data indicates that there is no significant difference among the
responses of teachers with 10 years or less of teaching experience
and those with more than 10 years o f teaching experience.
Therefore, the data questioning the enhancement o f teaching and
alignment of curriculum with the STAR Testing Program is not
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
83
effective as perceived by teachers regardless o f the number of years
o f teaching experience.
2. There is no significant difference between the responses of
respondents in low performing and not low-performing schools.
Therefore, the data indicates that the perceptions of respondents are
the same regardless o f whether they are in a low-performing or not
low-performing school.
3. Although research indicates for assessment to be successful and
useful, it must be aligned with what is being taught. The data
reported in this study indicates that state standards and the everyday
English/Language Arts lessons are not aligned with the STAR testing
program.
4. The lack of significant findings related to teacher experience
indicates that teachers’ opinions o f the STAR Testing Program are
very similar. Therefore, using the results o f this study is not valid in
any type o f prediction.
5. The lack o f significant findings related to low-performing and not
low-performing schools indicates that the opinions of respondents
from low-performing and not low-performing schools are very
similar. Therefore, using the results o f this study is not valid in any
type o f prediction.-
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Although one o f the goals o f the program is to effect the curriculum and
student learning, the data reported in this study overwhelmingly indicates a
failure to enhance teaching and student learning at least in the perception of
teachers. As reported in the literature, teachers agree that the accountability
system did not enhance their teaching but caused them to “teach to the test” and
limited curriculum and creative lessons to skills and knowledge being assessed.
The lack o f teacher enhancement could be tied to the lack o f analysis o f the data,
as reported by some respondents to assess curriculum, lessons and teacher
effectiveness.
The lack o f significant difference in low-performing and not low-
performing schools could be the result of a faulty system in comparing 100
similar schools. This also raises the question o f why there were no schools on
the low-performing reports for two consecutive years?
Strengths expressed by respondents indicated that public schools and
educators need to be held accountable. It was reported that many schools used
the information provided by the STAR Testing Program to evaluate their
curriculum and programs. It was expressed that the STAR Testing Program
helps students to focus on needed skills as it provides individual results for
students that are useful in determining their strengths and needs. Although there
are flaws in the present accountability system, the STAR Testing Program is
viewed as an attempt at accountability and providing guidelines for the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
85
improvement o f instruction. Because the STAR Testing Program is being made
public, it provides leverage for opening communications among schools and the
community.
Although each school site is permitted to develop their own test
schedule, many respondents shared that the implementation time varies from
five to ten days including make-ups for those students who were absent on the
day of the original test.
There is a strong concern regarding the lack o f teacher and student
ownership in the assessment process. Many fear that the test, although not
aligned with standards, will drive the entire educational program. Although the
Academic Performance Index (API) established by the Public Schools
Accountability Act o f 1999 eventually will be composed o f several measures, at
present the only measure for which reliable and valid data are available is the
STAR multiple choice test (currently the SAT-9). Many teachers reported that
tests are meaningless to students, but the emphasis placed by the API and state
was of great importance to teachers and the school. This has brought about a
great concern over the unfairness o f the test as it requires all students to be
tested in English even if they are also taking a primary language Normed
Referenced Test. It is felt that the data used to determine the API should include
attendance, alternative assessments, socioeconomic levels, parent support,
teacher turnover, transience, student and teacher attendance, and the need to take
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8 6
into account the differences in schools and students, resources, and teacher
preparedness.
Recommendations
This study has served as a foundation for the development o f an
understanding o f the effects o f the STAR Testing Program on teachers and the
curriculum. If the state is consistent in the implementation o f the STAR Testing
Program, it will build longitudinal data that may be used to determine the
evolution of alignment between state standards and the SAT-9 test.
There is a need to provide training in the use of data obtained from the
STAR Testing Program to enhance instruction and align curriculum with state
standards.
There is a need to include a number of variables in determining the API such as
staff development, attendance rates, length o f school year, and transience to
name just a few.
Recommendations for areas o f future research include: (a) a longitudinal
study o f an identified low-performing school and the effects o f outside
assistance from the state; (b) a study of how the STAR Testing Program effects
the morale of teachers, correlated with teaching experience and staff
development; (c) an attitudinal study of student motivation and engagement in
the STAR Testing Program since the implementation o f monetary rewards for
exceptional students performance; (d) a qualitative study as to the accuracy of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
87
curriculum alignment and state standards with the STAR Testing Program; (e) a
study that accurately assesses student reaction to the STAR exam, especially in
light o f their role as the major stakeholders.
The STAR Testing Program legislation is up for review during the 2001-
2002 school year. Results of the suggested research studies could assist
educators in the evolution of the STAR Testing Program and its effects on
students, teachers, and the curriculum. The STAR Testing Program continues to
add new components to its procedures each year. The State Board of Education
has recommended the Golden State Exam be incorporated in the STAR Testing
Program by 2002-2003 in subjects covered in both tests.
If California is to have a comprehensive accountability system that is
effective in enhancing instruction and student performance, there is a need to be
a direct correlation in the assessments, use of data, staff development, and state
standards. The STAR Testing Program includes many o f the elements of a valid
accountability program such as student performance standards, ongoing data
analysis, school improvement intervention strategies, and competitiveness.
However, some o f the main criteria established by Millman (1997) and others
continue to be lacking, such as fairness and comprehensiveness.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8 8
Practical Implications
The data obtained in this study could be used to evaluate whether the
STAR Testing Program is meeting its objectives of improving education for all
students. Review o f the results indicate that teachers perceive the program to be
a hindrance to their teaching o f higher standards and creativeness. Results also
indicate the failure of the program to align the SAT-9 assessment instrument
with the curriculum standards established by the State of California.
The overwhelming responses to the strengths, weaknesses, and
recommendations for improvement would be useful when evaluating and
amending the programs. These comments are made by those who are in the
trenches, using the instruments and working daily with students who to be
assessed.
Evaluators of the STAR Testing Program need to use the criteria of an
effective accountability program, as established by researchers, when evaluating
and revising the STAR Testing Program i.e., accountability, system design,
fairness, comprehensiveness, competitiveness, and consequential validity.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
89
SELECTED REFERENCES
Adams, J., & Kirst, M. (1999). New Demands and Concepts for Educational
Accountability: Striving for Results in an Era of Excellence. Handbook
o f Research in Education Administration. Washington. D.C.: American
Association o f Educational Researchers.
Agee, J.L. (Ed). (1992). Second to None: A Vision of the New California High
School. Sacramento, C A: California Department o f Education.
Albert, D. (1998). Standardized Testing and Reporting Program.. Office o f
Legislative Counsel, Sacramento, CA. Available online:
http :/www.cde.ca.gov.
Anderson, A.B. & Lewis, A.C. (1997). Academic Bankruptcy Policy Brief,
Denver, CO: Education Commission o f the States. Available online:
www.ecs.org.
Anrig, G. (1992, September). Testing and Accountability. The American
School Board Journal. 170.34-36.
Apple, M.W. (1996). Cultural Politics and Education. New York: Teachers
College Press.
Asimov, N., (2001, February 2). Schools Elated at Testing Rewards. San
Francisco Chronicle. San Francisco, CA., p. A-25.
Associated Press (2001, April 17). Test Preparation: A Concern, Record
Searchlight. Redding, CA, p. A4.
Association o f California School Administrators (ACSA), (2001, March).
Assessment Confusion? Here’s the Solution. Sacramento, CA: EdCal. .
Bamburg, J & Medina, E. (1993). Analyzing Student Achievement: Using
Standardized Tests as the First Step. In J. Bamburg (Ed.), Assessment:
How Do We Know What They Know? (pp. 43-64). Dubuque, LA:
Kendall-Hunt.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
90
Bandura, A. (1997). Self-Efficacv: The Exercise o f Control. New York:
Freeman.
Bennis, W. & Goldsmith, J. (1997). Learning to Lead. Reading, MA: Addison-
Wesley.
Berk, R.A (1988). Fifty Reasons Why Student Achievement Gain Does Not
Mean Teacher Effectiveness. Journal o f Personnel Evaluation in
Education. 1. 345-363.
Bemauer, J.A. & Cress K. (1997, September). How School Communities Can
Help Redefine Accountability Assessment. Phi Delta Kappan. 79.71-75.
Bolman, L.G. & Deal, T.E. (1995). Leading With Soul. San Francisco: Jossey-
Bass.
Boyd, W.L. (1982, Summer). The Political Economy o f Public Schools,
Educational Administration Quarterly. 18 (3). 111-130.
Bradley, A. (1999). Zeroing in on Teachers. Education Week. 18 (17). 46-52.
Brophy, J.E. & Good, T. (1986). Teacher Behavior and Student Achievement.
In M. Wittrock (Eds.), Handbook o f Research on Teaching (3rd . ed. (pp
328-375). New York: MacMillan.
Brown, P.R. (1990). Accountability in Public Education. (Policy Briefs No. 14)
San Francisco: Far West Laboratory.
Bunting, C. (1999). School Reform: Does it Really Matter? The
Clearinghouse. 72 (4), 213-216.
Bums, J.M. (1978). Leadership. New York: Harper & Row.
Burtless, G. (Ed.). (1990). A Future of Lousv Jobs: The Changing Structure of
US Wages. Washington D.C.: The Brookings Institution.
Byrk, A.S., Lee, V.E., & Smith, J.B. (1990). High School Organization and Its
Effects On Teachers and Students: An Interpretive Summary of
Research. In W. J. Clune & J. F. Witte (Eds.), Choice and Control in
American Schools. Vol. 1 (pp. 135-1226). Philadelphia: The Falmer
Press.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
91
California Department o f Education (1998). English-Language Arts Content
Standards for California Public School: Kindergarten Through Grade
Twelve. Sacramento, CA: California Department o f Education.
California Department o f Education (1998, Spring). Frequently Asked
Questions About Standards-based Accountability. Sacramento. CA:
California Department o f Education.
California Department o f Education (1999). 1999 Academic Performance Index
(API) Base Report.Policv and Evaluation Division. Available online:
http .//data I .cde.ca.gov.
California Department o f Education (2000) 2000 Academic Performance Index
(API) Base Report. Policy and Evaluation Division. Available online:
http://datal .cde.ca.gov.
California Department o f Education (2000, April). STAR Program: Ongoing
Conflicts Between the State Board of Education and the Superintendent
of Public Instruction as Well as Continued Errors Impede the Program’s
Success. Sacramento, CA: California Department o f Education.
California Department o f Education (2000, June). How to Use STAR Test
Results to Improve Student Learning. Sacramento, CA: Standards and
Assessment Division, California Department o f Education.
California Learning Assessment System (2000). News: Facing Deadline:
California is Locked In Battle Over How To Teach Math.
http:// www. edweek. o rg/e w/vo 1-16/24math. h 16.
California State Auditor (2000, April). STAR Program: Ongoing Conflicts
Between the State Board o f Education and the Superintendent of Public
Instruction as Well as Continued Errors Impede the Program’s Success.
Sacramento, CA.
Cameron J. & Pierce, VV.D. (1994). Reinforcement, Reward, and Intrinsic
Motivation: A Meta-Analysis. Review of Educational Research. 64.
363-423.
Cawelti, G. (July, 1999). Improving Achievement: Finding Research-Based
Practices and Programs That Boost Student Achievement. The
American School Board Journal. 186. 34-37.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
92
Clotfelter, C.T., & Ladd, H. (1996). Recognizing and Rewarding Success in
Public Schools: Holding Schools Accountable. Washington. D.C.: The
Brookings Institution.
Council of Chief State School Officers (CCSSO). (1998). State Education
Accountability Reports and Indicator Reports: Status o f Reports Across
the States 1998. Washington, D.C.: CCSSO http://www.ccsso.org.
Darling-Hammond, L. (1988). Accountability and Teacher Professionalism.
American Educator. 12 (4):8-13, 38-43.
Darling-Hammond, L. (1994). National Standards and Assessments: Will They
Improve Education? American Journal of Education. 102. 478-510.
Darling-Hammond, L. (1997). The Evaluation of Student Learning for the
Improvement of Teaching. In J. Millman (Ed.), Grading Teachers.
Grading Schools (pp. 241-261). Thousand Oaks, CA: Corwin Press.
Darling-Hammond, L., & Ascher, C. (1991). Creating Accountability in Big
City School Systems. New York: National Center for Restructuring
Education, Schools, & Teaching. (ERIC Document Reproduction
Service NO. ED pp 334-339.).
Darling-Hammond, L. & Goodwin, A.L. (1993). Progress Toward
Professionalism in Teaching. In Challenges and Achievements o f
American Education. (The 1993 ASCD Yearbook) (pp. 21-33).
Alexandria, VA: Association of Supervision and Curriculum
Development.
Davie, G & Silva, E.E. (1999, January). The Politics o f Accountability. Thrust
for Educational Leadership. 28. 6-9.
DeMoulin, D.F. & Kendall, R (1993). The Administrative Role in an
Accountability Network: A Developmental Conceptualization. Journal
of School Leadership, 3, 688-698.
District and School Support Division, California Department o f Education
(1998). Standards Survey Report of Results. Sacramento, CA: author.
District and School Support Division, California Department o f Education
(1999). Standards Survey Report o f Results. Sacramento, CA: author.
District and School Support Division, California Department o f Education
(2000). Standards Survey Report o f Results. Sacramento, CA: author.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
93
Eastin, D (2001, January). State School Chief Eastin Announces Distribution o f
Governor’s Performance Award Funds To Schools. News Advisory.
Sacramento, CA: California Department o f Education.
Eastin, D. (2001, February). 2000 Academic Performance Reported For
California Schools. News Advisory. Sacramento, CA: California
Department of Education.
Eastin, D. (1999, August). School Accountability and Reporting Requirements
(Memo to District and County superintendents, et al.). Sacramento, CA:
California Department o f Education.
EdSource. (1998, June). Report: Shifting the Focus to Learning: California’s
Accountability Debates. Palo Alto, CA, author.
EdSource. (1999, June). Report: What to Expect From California’s New
Accountability Law. Palo alto, CA: author.
EdSource. (2001, Winter). California Focuses on Student Assessment. Palo
Alto, CA: author.
Edwards, V.B. (Ed.). (1999). Quality Counts: Rewarding Results, Punishing
Failure (Special Issue). Education Week. 18 (17). 1-45.
Eisner, E.W\ (1994). The Educational Imagination. On the Design and
Evaluation of School Programs. Upper Saddle River, NJ: Prentice Hall.
Elmore, R.F. (1997). Accountability in Local School Districts: Learning to Do
the Right Things. Advances in Educational Administration. 59-82.
Elmore, R.F., Abelmann, C.H., & Fuhrmann, S.H. (1996). The New
Accountability in State Education Reform: From Process to
Performance. In Brookings Institute (Ed.), Holding Schools
Accountable (pp. 15-28). Washington, D.C.: The Brookings Institution.
Fausset, L. (1998, July 20). Guidance on Standards-based Accountability
(Memo to District County Superintendents, et al.). Sacramento CA:
Educational Policy, Curriculum, and Department Management,
California Department o f Education.
Forsyth, Patrick B. & Tallerico, M. (1998, August). Accountability and City
School Leadership, Education and Urban Society. 30 (4), 546-555.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
94
Frymier, J. (1996). Accountability in Education: Still and Evolving Concept,
Phi Delta Kappa Educational Foundation. Bloomington, IN: Phi Delta
Kappan.
Fullan, M. (1993). Change Forces. New York: Falmer Press.
Fuhrmann, S.H. (1994). Evaluation o f Performance in The United States:
Changes in Accountability. New Brunswick, NJ: Rutgers University,
Consortium for Policy Research in Education.
Glass, G.V. (1990). Using Student Test Scores to Evaluate Teachers. InJ.
Millman & L. Darling-Hammond (Eds.), The New Handbook o f Teacher
Evaluation (pp 229-240). Newbury Park, CA: Sage.
Goertz, M. E., Floden, R.E., & O’Day, J.A. (1995, July). Studies o f Education
Reform: Systemic Reform. Volumes I, II, III. Philadelphia, PA:
Consortium for Policy Research in Education (CPRE), University of
Pennsylvania, http:www. upenn. edu/gse/cpre/.
Grissmer, D., & Flanagan, A. (November, 1998). Exploring Rapid
Achievement Gains in North Carolina and Texas. National Education
Goals Panel. http//:www.negp.gov/publications/negpdocs/negpdocs/
negprep/gissmerpaper/part 1 .doc.
Guth, G.P., Holtzman, D.J., Schnieder, S.A., Carlos, L., Smith, J.R., Hayward,
G.C. & Calvo, N. (1999). Evaluation of California’s Standards Based
Accountability System: Final Report, http://www.wested.org.
Hambleton, R., Jaeger, R., Koretz, D., Linn, R., Millman, J. & Phillips, S.
(1995). Review o f the Measurement Quality o f the Kentucky
Instructional Results Information System 1991-1994. Frankfort, KY:
Office o f Educational Accountability.
Hammond, J. (November, 1995). Managing For High Performance: A
Superintendent’s Nine Rules for a Results Oriented School System. The
School Administrator, pp 12-16.
Hansen, J.B. (1993). Is Educational Reform Through Mandated Accountability
an Oxymoron? Measurement and Evaluation in Counseling and
Development. 16. 11-21.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
95
Hanushek, E.A. (1996). Comments on Chapters Two, Three, and Four. InH.F.
Ladd (Ed.), Holding Schools Accountability: Performance-based
Reform in Education ( d p . 128-136). Washington, D.C.: The Brookings
Institution.
Harris, M.B. (1995). Basic Statistics for Behavioral Science Research.
Needham Heights, MA: Allyn & Bacon.
Hill, P.T. (1999). Supplying Effective Public Schools in Big Cities. Available
online: www.crpe.org.
Hill, P.T., & Lake, R.J. (1997). Toward a K-12 Educational Accountability
System in Washington State. Seattle. WA: Washington University,
Center on Reinventing Public Education.
http :/7www. gspa.washington.edu/CRPE/crpehome.html.
Johnson, S.M. (1990). Teachers at Work: Achieving Success in Our Schools.
New York: Basic Books, Inc.
Kelley, C. (1998, May). The Kentucky School Based Performance Award
Program: School-Level Effects. Educational Policy. 12 (3), 305-324.
Kirst, M. (1990). Accountability: Implications For State and Local
Policymakers. Washington D.C.: US Department of Education, Office
of Educational Research and Improvement, Information Services.
Kirst, M. & Mazzeo, C. (1997, March). The Rise. Fall and Rise o f State
Assessment in California. 1993-1996. Paper presented at the annual
conference of the American Educational Research Association, New
York, NY.
Kortez, D.M. (1995). Sometimes a Cigar is Only a Cigar, and Often a Test is
Only a Test. In D. Ravitch (Ed.), Debating the Future o f American
Education (pp. 154-166). Washington, D.C: The Brookings Institution.
Kortez, D. (1998). Large Scale Portfolio Assessments in the US: Evidence
Pertaining to the Quality o f Measurement. Assessment in Education. 5
(3), 309-334.
Kuchapski, R. (1998). Conceptualizing Accountability: A Liberal Framework.
Educational Policy. 12. 191-202.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
96
Lee, V.E., Bryk, A.S., & Smith, J.B. (1993). The Organization o f Effective
Secondary Schools. In L. D. Hammond (Ed.), Review of Research in
Education (pp. 171-268). Washington D.C.: American Educational
Research Association.
Leithwood, K.A. (1992). The Move Toward Transformational Leadership.
Educational Leadership. 49 (5), 8-12.
Linn, R.L., & Baker, E.L. (1999, Spring). Standards-based Accountability
Systems’ Adequate Yearly Progress: Absolutes, Wishful Thinking, and
Norms. The CRESST Line. Special Issue. Los Angeles, CA: National
Center for Research on Evaluation, Standards, and Student Testing.
MacPherson, R.J.S. (1998). Accountability in City Schools: Theory and
Practice in Urban Educational Administration. Educational and Urban
Society. 30 (4), 443-458.
Madaus, G.F. (1999). The Influence o f Testing on the Curriculum. InM . J.
Early & K. J. Rehage (Eds.), Issues In Curriculum: A Selection o f
Chapters From Past MSSE Yearbooks (pp 73-111). Chicago:
University of Chicago Press.
Maeroff, G.I. (1988). The Empowerment o f Teachers: Overcoming the Crisis
of Confidence. New York: Teachers College Press.
Magnone, S. (2001, April). Standards, Assessment and Support: An Update on
the California Assessment System: Paper Presented at the ACSA
Conference in Lake Tahoe, Nevada.
Marsh, D. (1997). Educational Leadership for the 2U1 Century: Integrating
Three Emerging Perspectives. A paper presented at the annual meeting
ofAERA, Chicago, 1997.
Massell, D., Kirst, M. & Hoppe, M. (1997, March). Persistence and Change:
Standards-based Systemic Reform in Nine States. CPRE Policy Briefs.
Philadelphia, PA: Consortium for Policy Research in Education
(CPRE). University of Pennsylvania, http:www.upenn.edu/gse/cpre.
McDonnell, L.M., McLaughlin, M.J. & Morrison, P. (Eds.) (1997). Educating
One and All. Washington D.C.: National Academy Press.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
97
McKenna, R.A. (1997, June 30). Standards-based Accountability (Memo to
District and County Superintendents, et al.). Sacramento, CA: Policy,
Curriculum and Department Management, California Department of
Education.
McKenna, R.A. (1998, April 15). Standards-based Accountability (Memo to
District and County Superintendents, et al.). Sacramento, CA: Policy,
Curriculum and Department Management, California Department of
Education.
Millman, J. (Ed.). (1997). Grading Teachers. Grading Schools. Is Student
Achievement a Valid Evaluation Measure? Thousand Oaks, CA:
Corwin Press.
Millman, J. & Schalock, H.D. (1997). Beginnings and Introduction in Millman.
J. (Ed.), Grading Teachers. Grading Schools. Is Student Achievement a
Valid Evaluation Measure? Thousand Oaks, CA: Corwin Press.
Millman, J. & Sykes, G. (1992). The Assessment o f Teaching Based on
Evidence o f Student Learning: An Analysis. (Research Monograph No.
2). Washington D.C.: National Board for Professional Teaching
Standards.
Monk, D. (1990). Educational Finance: An Economic Approach. New York:
McGraw Hill.
Murphy, J. (1988). The Instructional Leadership Role o f the School Principal,
Educational Evaluation and Policy Analysis. 10(2), 117-139.
Murphy, J. & Hallinger, P. (1993). Restructuring Schooling: Learning From
Theory and Practice. Thousand Oaks, CA: Sage.
National Association o f State Directors o f Special Education (1998). Tasks and
Guidelines for Implementing the Inputs and Processes Component of an
Inclusive Accountability System. Alexandria, VA: Author.
Newman, F.M., King, M.B., & Rigdon, M. (1997, Spring). Accountability and
School Performance: Implications for Restructuring School. Harvard
Educational Review. 67 (1). 41-74. .
O ’Day, J.A., & Smith, M.S. (1993). Systemic Reform and Educational
Opportunity. In S.H. Fuhrmann (Ed.), Designing Coherent Educational
Policy (pp. 52-75). San Francisco, CA: Jossey-Bass Publishers, Inc.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
98
O ’Reilly, F.E. (1996). Educational Accountability: Current Practices and
Theories In Use. Cambridge. MA: Harvard University, Consortium for
Policy Research in Education.
Odden, A.R. (1995). Educational Leadership for America’s Schools. New
York: McGrawHill.
Odden, A. & Kelley, C. (1997). Paving Teachers for What They Know and Do.
Thousand Oaks, CA: Corwin Press.
Odden, A. & Marsh, D. (1988). How Comprehensive State Education Reform
Can Improve Secondary Schools. Phi Delta Kappan. 69 (8), 593-598.
Odden, A.R. & Picus, L.O. (1992). School Finance: A Policy Perspective.
Boston: McGraw Hill.
Office o f Policy and Evaluation, California Department o f Education. (1997).
Steering Bv Results: A High-stakes Rewards and Interventions Program
for California Schools and Students. Sacramento, CA: Report o f the
California Rewards and Interventions Advisory Committee.
Olsen, L. (1998, Feb 11). The Push for Accountability Gathers Steam.
Education Week. On the Web http://www.edweek.org/sreports/
qco 1/articles/gco lstorv.cfm?slug=intro.h20
Olsen, L. (2001). Finding The Right Mix. Education Week. On the Web
http://www.edweek.org/sreports/qcol/articles/gcolstorv.cfin7slugHntro.
h20.
Oustin, J., Fiddler, B. & Earley, P. (1998). The Educational Accountability of
Schools in England and Wales. Educational Policy. 12 (1-2), 111-123.
Padia, W. (2000). Public Schools Accountability Act. Policy and Evaluation.
Sacramento, CA: California Department of Education.
Picus, L.O. & Wattenburger, J.L. (1995). Where Does the Money Go?
Resource Allocation in Elementary and Secondary Schools. Thousand
Oaks, CA: Corwin Press.
Popham, W. J. (1997). The Moth and the Flame: Student Learning as a
Criterion o f Instructional Competence. In J. Millman (Ed), Grading
Teachers. Grading Schools. Is Student Achievement a Valid Evaluation
Measure? (pp. 46-56). Thousand Oaks, CA: Corwin Press.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
99
Popham, W.J. (1999). Why Standardized Tests Don’t Measure Educational
Quality. Educational Leadership. 56 (6) 8-15.
Purkey, S.C., & Smith, M.S. (1983). Synthesis o f Research on Effective
Schools. Educational Leadership. 40 (3) 64-69.
Ravitch, D. (Ed.). (1995). Debating the Future o f American Education: Do We
Need National Standards and Assessments? Washington, D.C: The
Brookings Institution.
Resnick, L. (1992-93). Standards, Assessment and Educational Quality.
Stanford Journal o f Law and Policy. 4, 53-59.
Resnick, L. & Hall, M.W. (1998). Learning Organizations for Sustainable
Education Reform. Daedalus. 127 (4), 89-118.
Rich, W.M. (2000) Accountability: A Case Studv. Unpublished doctoral
dissertation, University o f Southern California. Los Angeles, CA.
Rose, L.C. &Gallup, A.M. (1998). The 30th Annual Phi Delta Kappa/Gallop
Poll in the Public’s Attitude Toward the Public School. Phi Delta
Kappan. 80 (1), 41-56.
Rothman, R. (1995). Measuring Up. Standards. Assessment and School
Reform. San Francisco: Jossey-Bass.
Ryan, R.M. & Deci, E.L. (1996). When Paradigms Clash: Comments on
Cameron and Pierce’s Claim That Rewards Do Not Undermine Intrinsic
Motivation. Review of Educational Research. 66. 33-38.
Sagor, R. (1996). Local Control & Accountability: How to Get it. Keep It. and
Improve School Performance. Thousand Oaks, CA: Corwin Press.
Sattler, J. (1992). Assessment o f Children (3rd ed. revised). San Diego: Author.
Schomoker, M. (1996). Results. Alexandria, Va: Association for Supervision
and Curriculum Development.
Sergiovanni, T.J. (1987). The Principalship: A Reflective Practice Perspective.
Boston: Allyn & Bacon.
Sergiovanni, T.J. (1992). Moral Leadership: Getting to the Heart o f School
Improvement. San Francisco: Jossey-Bass.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 0 0
Sizer, T.R. (1984). Horace’s Compromise: The Dilemma o f the American High
School. Boston: Houghton Mifflin.
Slavin, R. (1989). Achievement Effects o f Substantial Reductions in Class Size.
In R. E. Slavin (Ed.), School and Classroom Organization (pp. 247-257).
Hillsdale, N.J.: Erlbaum.
Slavin, R. (Ed.). (1989). School and Classroom Oreanization. Hillsdale. N.J.:
Erlbaum.
Slavin, R.E. (1997). Sand. Bricks and Seeds: School Chanee Strategies and
Readiness for Reform. John Hopkins University, Center for Research on
Education o f Students Placed At Risk. Available online:
http://successforall.net/sandbrick.htm.
Slotnik, W.J. & Gratz, D.B. (January, 1999). Guiding Improvement. Thrust for
Educational Leadership. 28. 10-12.
Smith, M.L., Edelsky, C., Draper, K., Rottenberg, C., & Cherland, M. (1990).
The Role of Testing in Elementary Schools. Los Angeles: University of
California Los Angeles, Center for Research on Educational Standards
and Student Tests, Graduate School o f Education.
Standardized Testing and Reporting Program (2001). Standford Test o f
Academic Skills, fourth ed. Sacramento: Harcourt Brace: CA
Stigler, J.W. & Hiebert, J. (1997, September). Understanding and Improving
Classroom Mathematics Instruction. Phi Delta Kappan. 79.14-21.
Sykes, G. (1999) The “New Professionalism” in Education: An Appraisal. In J.
Murphy & K. Seashore-Louis (Eds.), Handbook of Research on
Educational Administration. San Francisco, CA: Jossey-Bass.
Tacheny, S. (1999). If We Build It, Will they Come? Educational Leadership.
56, (6) 62-65.
Theobald, P. & Mills, E. (1995, February). Accountability and the Struggle
Over What Counts. Phi Delta Kappan. 76. 34-38.
Toppo, G. (2001, April 5) Test Rank World’s 8th Graders. San Francisco
Chronicle, p. A-3, San Francisco, CA.
Travers, R.M.W. (1981). Criteria o f Good Teaching. In J. Millman (Ed.),
Handbook of Teacher Evaluation (pp. 14-22). Beverly Hills, CA: Sage.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 0 1
Tucker, M.S. & Codding, J.B. (1998). Standards for Our Schools: How to Set
Them. Measure Them, and Research Them. San Francisco, Jossey-Bass.
Wendling, W. & Cohen, J. (1981). Educational Resources and Student
Achievement: Good News for Schools. Journal o f Education Finance.
7(1), 44-63.
WestEd/MAP (1999) Accountability Evaluation: WestEd/MAP: Menlo Park,
CA. Available online: http://www.wested/MAP.
Williams, A. (2001, March 5) Testing Idea Doesn’t Impress Officials. Record
Searchlight, p. A-l. Redding, CA.
Winters, S. (1990). Integrating Implementation Research. In D.J. Palumbo &
D.J. Calista (Eds.), Implementation and the Policy Process: Opening Up
the Black Box (pp. 19-38). New York: Greenwood Press.
Wong, K. K. & Moulton, M.H. (1998). Governance Report Cards:
Accountability in the Chicago Public School System. Education and
Urban Society. 30 (4), 459-478.
Woolfolk, A.E. (1998). Educational Psychology. 17th Ed). Boston: Allyn &
Bacon.
Ziebarth, T. (1998). State Takeovers and Reconstitutions Policy Brief. Denver,
CO: Education Commissioner of the States. Available online:
www.ecs.org.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 0 2
APPENDIX A
INFORMED CONSENT FOR NON-MEDICAL RESEARCH
CONSENT TO PARTICIPATE IN RESEARCH
Effects of the STAR Testing Program on
Teachers and the Curriculum
You are asked to participate in a research study concducted by Dr.
Dennis Hocevar Ed.D. and Regina K. Lane M.A./ Ed.D. candidate, from
the Rossier School o f Education at the University o f Southern California.
The results o f this research will contribute to the dissertation in partial
fulfillment o f the Ed.D Program. You were selected as a possible
participant in this study because o f your experience as a 10th grade
English/Language Arts teacher. A total o f 500 subjects will be selected
from 10th gade English/Language Arts teachers in the State o f California
to participate. You participation is voluntary. Completion and return
of the survey indicates your consent to participate.
PURPOSE OF THE STUDY
The purpose o f the study is to acquire data as to the effects o f the STAR
Testing Program on 10th grade English.Language Arts teachers and their
curriculum.
PROCEDURES
If you volunteer to participate in this study, we would ask you to do the
following things:
Complete the survey and return it in the self-addressed stamped envelope
to Regina K. Lane. Return o f the questionnaire indicates your consent to
participate.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
103
POTENTIAL RISKS AND DISCOMFORTS
The information obtained by your completion o f the survey will be
anonymous, as you are not asked to identify yourself in any way.
POTENTIAL BENEFITS TO SUBJECTS AND/OR TO SOCIETY
As an individual you will not receive any benefit from participating in
this study. The data will provide information that will assist us in
determining the effect the STAR Testing Program is having on teachers
and their curriculum.
PAYMENT FOR PARTICIPATION
Your participation in this study is strictly voluntary. Therefore, you will
not receive any monetary reward.
CONFIDENTIALITY
Any information that is obtained in connection with this study will be
kept confidential. Your personal identification is not requested in this
study. However, the name o f your district and school is requested. The
identification of the school and district will not be used in the report of
the findings. Dr. Dennis Hocevar and Regina K. Lane will be the only
researchers that will have access to the completed surveys. The
completed and returned surveys will be kept in a locked file cabinet in
the office o f Regina K. Lane. Upon the completion o f the dissertation
shredding will destroy the surveys.
When the results o f the research are published or discussed in
conferences, no information will be included that would reveal your
identity or the identity o f your school or district. Demographic
information will be used only to compare and categorize data.
PARTICIPATION AND WITHDRAWAL
You can choose whether to be in this study or not. If you volunteer to be
in this study, you may withdraw at any time without consequences of
any kind. You may also refuse to answer any questions you don’t want
to answer and still remain in the study. The investigator may withdraw
you form this research if circumstances arise which warrant doing so.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
104
IDENTIFICATION OF INVESTIGATORS
If you have any questions or concerns about the research, please feel free
to contact Dr. Dennis Hocevar Principal Investigator, Faculty Sponsor
(831) 624-3766 or (213) 740-2368 Rossier School o f Education at
University o f Southern California, Los Angeles, California and/or
Regina K. Lane, student (530) 246-4050 work (530)221-1990 home,
4140 Alta Mesa Dr. Redding, CA 96002
RIGHTS OF RESEARCH SUBJECTS
You may withdraw your consent at any time and discontinue
participation without penalty. You are not waiving any legal claims,
rights or remedies because o f your participation in this research study. If
you have questions regarding your rights as a research subject, contact
the University Park IRB, Office o f the Vice Provost for Research,
Bovard Administration Building, Room 300, Los Angeles, CA 90089-
4019, (213) 740-6709 or upirb@usc.edu.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
105
APPENDIX B
EFFECTS OF THE STAR TESTING PROGRAM ON
ENGLISH TEACHERS AND THE CURRICULUM
PART 1: Demographic Information
1. Gender Male Female
2. Age: 26 years or younger 27-34 35-41 42-48 49 +
3. Name of District_________________ Name of school__________________
4. Years as a high school teacher 2 or le s s 3-6 7-10 11-14__ 15 +
5. Estimate the extent to which the English subsections of the STAR TEST
(vocabulary, reading, spelling and language) are aligned with your everyday
English/Language Arts curriculum.
20% or less 21%-40% 4I% -60% 61-80% 81%-100%
PART 2: This section contains statements regarding the effects o f the STAR
Testing Program on teachers and the curriculum. There are five possible
responses to each statement: 1 = Strongly Disagree, 2 = Disagree, 3 = Agree,
4 = Strongly Agree, 5 = D on’t Know. For each statement, circle the ONE
response that best reflects your opinion. There are no right or wrong responses.
The mandated STAR Testing Program has:
1 . Provided feedback as to the quality o f my teaching.
1 2 3 4 5
2. Been aligned with the state standards for English/Language Arts.
1 2 3 4 5
3. Enhanced reading across the curriculum.
1 2 3 4 5
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
106
4. Accurately assessed the 10th grade English/Language curriculum.
1 2 3 4 5
5. Established reasonable goals for increased student achievement.
1 2 3 4 5
6. Had a positive impact on my students overall English achievement.
1 2 3 4 5
7. Forced me to teach to the test.
1 2 3 4 5
8. Enhanced what I do in the classroom.
1 2 3 4 5
9. Motivated me to seek assistance for improving my teaching.
1 2 3 4 5
10. Helped teacher’s incorporate higher order thinking skills.
1 2 3 4 5
11. Provided a reasonable assessment of teacher performance.
1 2 3 4 5
12. Measured the wide range o f outcomes intended by the curriculum.
1 2 3 4 5
13. Taken away from the creative endeavors o f teachers.
1 2 3 4 5
PART 3: Please provide information to the following questions.
14. Strengths o f the STAR Testing Program.
15. Weaknesses or concerns regarding the STAR Testing Program.
16. How can the State best improve the STAR Testing Program?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

A quantitative and qualitative study of computer technology and student achievement in mathematics and reading at the second- and third-grade levels: A comparison of high versus limited technolo...

PDF

Cognitive coaching training for master teachers and its effects on student teachers' ability to reflect on practice

PDF

Evaluation of the effects of a continuous improvement program on special education student achievement

PDF

An evaluation of the REMEDY project with an emphasis on sustainability: A Safe Schools /Healthy Students federal grant

PDF

Evaluation of the effects of longitudinal tracking of student achievement to assess school quality

PDF

An evaluation of probation supervision and its role in performance of students identified as living in poverty in Shasta County Court and Community Schools

PDF

Institutional factors affecting academic persistence of underprepared community college freshmen

PDF

A case study of the California teacher evaluation system and its impact upon teacher practice in an alternative education high -performing urban high school

PDF

Family group conference: An innovative approach to truancy in schools. A case study

PDF

Correlates of job satisfaction among California school principals

PDF

Into, Through and Beyond: Evaluation of migrant education reading tutor program innovations

PDF

Evaluation of a two -period double math program on the academic achievement of underperforming seventh grade math students

PDF

Effects of an integrated content and methods course on preservice teachers' beliefs and efficacy toward mathematics

PDF

A comparative study of the role of the principal in conventional public schools and in charter schools

PDF

China's critical educational access demand and United States higher education distance learning curriculum: An answer?

PDF

Differential characteristics of beginning teachers

PDF

Comparison of the predictive validity of a written test, an integrity test, a conscientiousness questionnaire, a structured behavioral interview and a personality inventory in the assessment of j...

PDF

Faculty perception of stress in a non-profit business and technology college

PDF

High school teachers' and administrators' perceptions of the teacher evaluation process in California's public schools

PDF

A case study of instructional supervision, including teacher evaluation, and the impact on teacher practice

Asset Metadata

Creator Lane, Regina Kay (author)

Core Title Effects of the STAR testing program on teachers and the curriculum

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, administration,Education, Secondary,education, technology of,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Hocevar, Dennis (committee chair), Baker, Robert (committee member), McLaughlin, Michael (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-155080

Unique identifier UC11334597

Identifier 3054768.pdf (filename),usctheses-c16-155080 (legacy record id)

Legacy Identifier 3054768-0.pdf

Dmrecord 155080

Document Type Dissertation

Rights Lane, Regina Kay

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA