University of Southern California Dissertations and Theses

Design, implementation and adequacy of data utilization in schools: A case study

(USC Thesis Other)

Design, implementation and adequacy of data utilization in schools: A case study

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content DESIGN, IMPLEMENTATION AND ADEQUACY OF DATA
UTILIZATION IN SCHOOLS: A CASE STUDY
by
Jean Cantor
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
December 2003
Copyright 2003 Jean Cantor
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 3133245
Copyright 2003 by
Cantor, Jean
All rights reserved.
INFORMATION TO USERS
The quality of this reproduction is dependent upon the quality of the copy
submitted. Broken or indistinct print, colored or poor quality illustrations and
photographs, print bleed-through, substandard margins, and improper
alignment can adversely affect reproduction.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
®
UMI
UMI Microform 3133245
Copyright 2004 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UNIVERSITY OF SOUTHERN CALIFORNIA
School of Education
Los Angeles, California 90089-0031
This dissertation, written by
______Jean Louise Cantor
under the direction of h ^^Dissertation Committee, and
approved by all members of the Committee, has been
presented to and accepted by the Faculty of the School
of Education in partialfulfillment of the requirements for
the degree of
D o c t o r o f E d u c a t io n
December 17, 2003
Date
Dissertation Committee
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ACKNOWLEDGMENTS
ii
I wish to thank my family for the encouragement each member gave me to
pursue my dissertation:
To Mom, for always letting me know that there was a lot to ieam in the
world.
To my son, Bryan, who was patient and supportive while I went to school,
studied and was always “busy.”
To my sister, Diane, who sat with me and edited word-by-word, concept-by-
concept, page-by-page.
To my cousin, Caryn, who never gave up on the final product.
I love you all. Thank you.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
TABLE OF CONTENTS
ACKNOWLEDGMENTS ii
LIST OF TABLES v
LIST OF FIGURES vi
ABSTRACT ' vil
PAGE
OVERVIEW OF THE STUDY 1
Introduction 1
Statement of the Problem 12
Purpose of the Study 13
Research Questions 14
Importance of the Study 14
Limitations 16
Delimitations 16
Definition of Terms 16
LITERATURE REVIEW 20
Introduction 20
Synthesize of the Literature 21
Conclusion 46
METHODOLOGY 48
Introduction 48
Sample and Population 50
Instrumentation 54
Data Collection 56
Data Analysis 57
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
iv
4 THE FINDINGS 60
Introduction 60
Findings by Research Question 63
Discussion 98
Conclusion 124
5 SUMMARY, CONCLUSIONS, AND IMPLICATIONS 126
Summary 126
Purpose and Problem 126
Methodology 127
Findings 128
Conclusions 130
Implications 131
REFERENCES 135
APPENDICES 143
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
V
LIST OF TABLES
TABLE PAGE
1 Economic and Ethnic Representation of District/
School Student Population 52
2 Credentials held by the Teaching Staff at Del
Sol Elementary School 52
3 Ethnicity of Teacher Staff 53
4 Demographics of Teachers Interviewed 53
5 Highest Average Scores Across all Six Stages
O f Concern 93
6 Stages of Concern Average Score by Teacher 93
7 Highest Stages of Concern Scores for Stage 0-4 94
8 Number of Stages of Concern with averages of at
Least 3.5 95
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
LIST OF FIGURES
vi
FIGURE PAGE
1 District-Wide Language Arts Essential Agreements
And Assessment Timeline 65
2 District-Wide Writing Mechanics and Assessment
Timeline 67
3 Mapping of Data Flow 91
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
vii
ABSTRACT
This study is a qualitative single-case design study to examine the use o f data
in one elementary school The purpose of this study is to determine, if an elementary
(K-8) school district has created a design for the collection of data. If a design was
developed what are the elements of the design? The study examines the degree the
elements of the design are being implemented in one school, specifically in six
diverse classrooms to determine the extent of improved student performance in the
area of language arts. The study also examines the degree of adequacy o f thedata
collection design.
The district studied was purposively sampled and nominated based. The
urban fringed district has challenges including, diverse ethnicities, languages and
economic ranges.
Data were collected through formal and informal interviews, A Teacher
Questionnaire, and a Stages of Concern Questionnaire. Observations, document
reviews, and artifact analysis were also conducted.
Results indicated that the district capitalized on the national state policy
trends and needs to create its design for data use to improve instruction.
Implementation of the design was found in the school and the six classrooms studied
at varying degrees.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1
CHAPTER 1
OVERVIEW OF THE STUDY
Introduction
In every state of the nation, it is clear that there is a tremendous focus on
educational accountability. This accountability has been evolving since Spudnik
in 1957. The Russian space program, Spudnik, spurred the United States to question
the effectiveness and accuracy of the curriculum and teaching of science and
mathematics. The nation’s math and science programs were altered significantly as a
result.
Educational accountability grew again when, in 1983, A Nation At Risk was
reported and published. A Nation at Risk reported that America’s high school
students were unprepared to graduate and enter the work force with appropriate skills
and knowledge. Their academic performance was compared to their counterparts in
international comparisons. The results of the comparisons indicate that the academic
performance of American students was poor.
The status of student performance (especially in math and literacy) in the
United States and California compared to other states and nations is a high priority
(Donahue, Voelkl, Campbell & Mazzeio, 1999; National Center for Education
Statistics, 1999; Reese, Miller, Mazzeo & Dossey, 1997). Multiple studies and
reports from The Third International Mathematics and Science Study (TIMSS); The
National Assessment of Educational Performance (NAEP); National Center for
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2
Education Statistics, 2000; Harcourt Brace, 1998, 1999, 2000, 2001; and California
Department of Education, 1998,1999,2001,2002 demonstrate that the performance
of students in the United States has fallen slightly above the international average in
science and below the international average in mathematics and literacy. In
California the performance scores lag far below the national average.
In addition, the research on the distribution of the performance on all students
show that cohorts of black and Hispanic students tend to lag behind their white and
Asian classmates. Paul Barton in his Executive Summary, Raising Achievement and
Reducing Gaps; Reporting Progress toward Goals for Achievement (Barton, 2001)
noted that states have not generally reduced the achievement gap between top and
bottom quartiles or between white and minority students (Barton, 2001).
Essentially, these reports are consistent reminders that American students are
performing with poor or average skills in mathematics, science, geography, history
civics, literature, writing and reading skills and the arts when compared with other
students in modem nations (Grissmer & Flanagan, 2001). Results indicate that all
children need to receive the instruction necessary to be viable citizens and workers in
the newly emerging, economically competitive world.
There are long-term consequences of low student performance, both of the
students themselves and for society. Most immediate is the concern regarding the
eighth grade results on the state assessments. They are troubling because those
students will have to pass the California High School Exit Exam to graduate.
Without a high school diploma, the access to employment diminishes, as does the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3
entrance to a middle class life style and opportunities for meaningful social
inclusiveness (Marshall, Ray, Tucker & Marc, 1992; Mumane, Richard & Levy,
1996; NICEE, 1990).
The most pressing social issue of our day is the increasing dramatic disparity
between the skills children are acquiring in our school system and the skills they will
need to obtain good jobs in an increasingly selective, globally influenced job market
(Mumane & Levy, 1996). Although our educational system is undergoing
incremental improvement, the business environment is growing in exponential rates.
If our students continue to improve at the rate they are, fewer and fewer students are
being prepared for a global economy.
In 1990, The National Center on Education and the Economy, a not-for profit
organization reported that a problem was emerging within the United States. In the
Executive Summary of America’s Choice: High Skills or Low Wages!, states that the
problem facing the nation is that since 1969, real average weekly earnings in the
United States have fallen by more than 12% (NCEE, 1990). The work force was
predicted to slow in the 1990’s because productivity growth has decreased. “If
productivity continues to falter, we can expect either the top 30% of our population
to grow wealthier while the bottom 70% become progressively poorer” (NCEE,
1990). The future belongs to societies that organize themselves for learning, where
all institutions are organized to leam and to act on what they learn (Mumane &
Levy, 1996).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4
The results of national, state and local assessments demonstrate a national
problem in American education and that the national problems require a national
solution. The standards-based comprehensive school reform in the United States is a
response to the outcry for more accountability within the educational system.
Momentum and support for the standards movement began in 1989 with the
Governor’s Conference on Education.
Comprehensive school reform in the United States is based in standards-
based accountability. Standards-based accountability provides (1) indicators and
measurement of student performance, (2) strategies to improve student performance
(3) professional development to meet and maintain the standards, and (4) monitoring
systems of the results.
The national and local conceptual framework on comprehensive school
reform began in 1989. IAS A in 1994 required all states to test students and the
Obey-Porter legislation of 1999 set aside funds to encourage states to engage in
Comprehensive School Reform. In 2002, President George W. Bush further refined
school reform with his Leave No Child Behind legislation. This act contains
increased emphasis on academic achievement and accountability and stipulates
specifically the resources and research required to advance the reform movement.
The standards-based reform helps mitigate the “significant achievement gaps
that exists between rich and poor, and among ethnic and cultural groups” (West-Ed,
2000). Full implementation of standards-based instruction and assessment was the
most important factor identified by the Education Trust in 366 schools with poverty
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5
level less then 50% that were high performing or most improvement in achievement
tests in 21 states. “These schools have become places where all classroom activity is
aligned with state standards” (EdTrust, 1999).
Educational and business literature and empirical studies support standards-
based reform to push our nation to a competitive position with other high performing
countries such as Japan and Germany (Marshall & Tucker, 1992). Other advanced
nations have stringent performance standards that directly affect the employment
prospects of the student (NCEE, 1990).
The elements of standards-based reform evolved from the research
recognized by the federal government, and from a synthesis of findings on high-
performing, high poverty schools and districts (Johnson, Haycock, Carter, Reeves,
Supovitz, Pogllinco & Snyder, 2001; Statewide System of School Support, CDE,
2000). These 9 elements are:
1. A focus on academic achievement as the highest priority and mission of the
district and school
2. High quality teaching and learning of rigorous standards
3. Collegial and collaborative faculty, teamwork - engagement in self-study and
reflection
4. High expectations of student achievement
5. Staff development linked to teaching of standards
6. Use of state and local standards and or accountability system
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6
7. Parent involvement - actively engaging the parents, community and other
stakeholders in the process
8. Use of research and or data to promote continuous improvement
9. Safe, orderly school climate conducive to learning
Additionally there are four organizational features that will be needed to make
such reforms successful including:
1. Governance - Systemic and flexible decision making structures and practices
that are responsive to the identified needs from student achievement.
2. Organizational Arrangements - Collaborative district and school environments
focused on analysis of student performance and instruction.
3. Leadership - A clear focus about promoting the advancement of student
achievement at all levels.
4. Financial arrangements - Realigning funding sources to support reform measures
(Clune, 1998). However, Allen Odden (2000) reports that research results are
unclear as to whether reallocating existing funds can support financial support of
reforms or whether additional funds are required. Although Odden supports the
educational reform elements he states that the fiscal aspects of implementing
school-wide reform must be considered as well.
The intent of the role that data/information about student performance and
related matters in comprehensive school reform is to provide accountability,
feedback, improving instruction on a broad scale and increasing student teaming for
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7
sustainable success. Data and information are used with the following intentions
(Holcomb, 1999; Reeves, 1999, 2000).
a. Reporting and disaggregating student performance
b. Holding schools, districts and students accountable through the state defined
accountability systems, and locally defined accountability systems
c. Setting performance targets and goals for schools
d. Defining progress
e. Creating Consequences for Performance for the school, the district and the
students
f. Assisting low performing schools
Across the United States, policymakers and educational leaders are
implementing accountability and assessment measures in response to increasing
concerns about improving student performance. All 50 states have embarked on
educational initiatives related to high standards and challenging content.
The role of data in the California State expectations for schools is multi
dimensional as evidenced by the current and emerging trends. These trends can be
best explained by California Legislative platform. The accountability platform
includes the SAT 9, Standards-Based test, California English Language
Development Test (CELDT), California High School Exit Exam (CAHSEE), and the
rewards and sanctions that are connected to achievement results.
The emerging, long-term plan for the California State Assessment System
revolves around six guiding principles: (1) State tests will focus instruction on the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8
mastery of state content standards, (2) all state assessments will conform to rigorous
technical standards (3) STAR standards tests will be the “core” of the state testing
system, (4) a nationally norm-referenced test will provide national comparisons data
on student achievement, and (5) STAR will be integrated with other state
assessment.
Currently, what is provided to schools by the State Department of Education
and the State Board of Education are state content standards in mathematics,
language arts, social science and science; individual and group scores from the states
STAR program.
There are three expectations of the schools. The first is to ensure there are
increases in student achievement to reach and exceed API target growth. The second
is to provide on-going assessment and intervention. The third expectation is to
provide on-going sustaining professional development aligned to the standards and
instruction.
Assessment and accountability are two of the most powerful policy
initiatives on the American education landscape. State efforts continues to align
California’s assessment system with content standards and will continue until at least
2003 (EdSource on-line, 2001). This alignment in principle allocates authority,
provides incentives, builds capacity, measures progress, and reports results and
enforces consequences, all of which are centered on student performance (The
Progress of Education Reform 1999-2001).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9
The scant research of sustained, district-wide academic success for children
of color and children from low-income homes have appeared in states such as New
York, North Carolina, and Texas. These states have highly developed and stable
state accountability systems, (Skrla, Scheurich, Johnson, Hogan, Koschoreck &
Smith, 2000). The school districts have created conditions in which school success,
for all children is a reality, (Elmore, 1997; North Carolina Department of Public
Instruction, 2000; Ragland, Asera & Johnson, 1999), The major findings of the
study reveal the following types and purposes for the uses of information/data in
schools: (1) align the curriculum, (2) develop coherent practices for the delivery of
instruction within the curriculum through benchmark targets (p. 24), (3) acquire,
allocate, and align fiscal, human and material resources, (4) collect, interpret, use
data and monitor results, (5) supervise, evaluate, and hold people accountable,
(6) refocus energies, refine efforts and ensure continuous performance, and
(7) create and nurture alliances.
Kate Jamentz (1999) Douglas B. Reeves (2000), and Richard Stigler (2001)
each articulate that sustained, focused examination of student work, will generate a
culture of reflection that is crucial to the analysis of information. The current uses of
data scores from state test determine school performance and growth under
California high stakes accountability program.
The explanation/speculations for such patterns of use is best articulated in the
results of a three-phase, multi-year study conducted by the 10 United States Regional
educational laboratories between 1998 and 2000 (United States Office of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
10
Educational Research and Improvement, 2000). The use of data across all levels of
the system are the following: (1) Student learning standards as a foundation for
reform work, (2) teachers’ capacity to provide effective instruction, (3) meaningful
local and classroom assessment of student achievement, and (4) data that includes
statistics on attendance, grades, referrals, retention and standardized test results. The
data can be disaggregated to reveal unrecognized patterns and may suggest areas in
need o f improvement (Johnson. 1997).
There are four explanation/speculations for the lack of use of data. The first
is that data analysis has never been viewed as a high priority. Until recently, state
education departments put little emphasis on schools gathering data, and thus
provided little incentives to devote time, money and staff resources to using data in
new ways (Bernhardt, 2000). Edie Holcomb in her book Getting Excited About
Data, identified a continuum of intrinsic to extrinsic motivating factors for why
teachers use data. The most intrinsic, being able to satisfy ones own drive for
excellence. The most extrinsic being, able to respond to public press for
accountability (e.g., federal, state, business sector).
The second speculations for the lack of data use is fear. The fear of data and
the analysis of it has gotten in the way of using the information that has been so
readily available. There is the fear, also, that the data analysis will turn up
something they do not wish to have revealed. Third, some districts and schools have
antiquated jobs and technologies to allow the analysis of data to be a routine practice.
There are too few people trained at the school level to collect and maintain
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
11
databases. Finally, there are not enough good examples of schools gathering,
maintaining and benefiting from the use of data.
We are now entering a new era of thinking and research about how
information and data could be used in schools and other similar organizations. Data
is more than standardized test scores and rankings. Data now includes instruction
and non-instruction measures of student performance. Instructional data are grades,
norm referenced test results, criterion referenced test results, and samples of student
work.
Non-instructional data are statistics on attendance (students and staff),
referrals to Student Study Teams and other intervention programs. Retention and
graduation rates are non-instructional data. The data can be disaggregated to reveal
unrecognized patterns and may suggest areas in need of improvement, (Johnson,
1997).
We are now entering a new era of thinking and research about how
information and data could be used in schools and other similar organizations. The
most promising examples of new ways to use information in schools to support
strong student performance involves: (i) a standards-based focus at the district,
school and classroom levels, (2) processes that provides student-specific, valid,
reliable information about student performance visa vie the standards, (3) both
system and student incentives and accountability, (4) powerful process of school and
teacher/reflection/problem-solving and culture, (5) strong instructional improvement,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
12
and (6) the enhancement of public understanding, engagement and support of school
reform.
Promising strategies for policymakers are as follows: (1) increasing
participation of minority students in challenging academic courses, (2) investing in
teacher professional development, (3) lowering class size in high-minority schools,
(4) providing extended learning time and intensive supports for students who are
having difficulty, (5) strengthening parent and community support for learning, (6)
ensuring adequate supply of well-qualified teachers in high-minority schools, and (7)
addressing disparities in curriculum, instruction, and facilities between high minority
and low minority schools (Williams, 1996).
Statement of the Problem
There is an increase alarm for attention to improving educational conditions
in California’s urban schools. Short-term alarm centers on the complexities of
diverse urban schools. Complexities such as increasing student achievement on
current state assessments and creating local criterion referenced assessments to
measure more precisely and specifically how students are performing on state
standards are compelling issues.
The long-term alarm for improving the educational conditions for students in
California and the nation is centered around consequences of low student
performance. These consequences are for both students themselves and for society,
and are serious. Nationally, 30 million workers can scarcely read and write, and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
13
another 40 million lack the basic skills to get by in a technological and rapidly
changing world.
There is little research on a school districts design for data use, the schools
implementation of the design. Research about the current use of information and data
in schools shows that little has been done on large-scale school success.
Additionally, scant research exists on the effects of school district data use design
and the use of the design in the classroom.
Purpose of the Study
This study is one of five parallel qualitative case study dissertations to
examine the use of data in elementary schools. The purpose of this study is to
determine, if the district has created a design for the collection of data, what are the
elements of the design. This study will also examine how and if the design is being
implemented in a school site and specifically in six diverse classrooms to determine
the extent of improved student performance in the area of language arts. The study
will also examine the degree of adequacy of the data collection design. Instructional
type data such as SAT 9, augmented SAT 9, and other district determined
assessments will be examined. Non-instructional type data such as attendance,
demographic and socio-economic status will be examined. Teacher and administrator
data collection and use will be examined.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
14
Research Questions
1. W hat is the district design for using data regarding student performance, and how
is that design linked to the current and the emerging state context for assessing
student performance?
2. To what extent has the district design actually been implemented at the district,
school and individual teacher level?
3. To what extent is the district design a good one? (use multiple criteria)
Importance of the Study
The collection of student academic achievement data, instructional practices
related to those data, and non-instructional data are of importance in the study of
school improvement and legislatively driven restructuring efforts. What is not known
to be of importance is the school districts’ involvement in the design of a data
collection strategies. This study will determine, if the district has created a data
collection design, what are the elements of the design, how and if the design is being
implemented in a school site and specifically in six diverse classrooms. The findings
of the districts’ data collection design and the school/classroom use of the data will
shed important light on current efforts to provide strategies for emerging trends in
California’s reform efforts.
In the absence of comprehensive evaluations, state policymakers lack
empirical evidence about the effects of reforms on student achievement. Districts
and schools have been vulnerable to constantly changing trends. Academically,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
15
students’ success falter somewhere in the middle and upper grades. The United
States seem to be the only country in the world whose children fall farther behind the
longer they stay in school
This study will add to the educational literature and assist district and school
level leaders in identifying the adequacy of a district design to address the current
and emerging state context for assessing student performance to improve student
achievement. This study will provide knowledge about the extent that such a design
is implemented in a school site. District and school site leaders may find the study of
value in their development of a data collection and implementation plan. The
conceptual frameworks for collecting student achievement results from instructional
and non-instructional sources of data may be useful for developing a district design
for data collection.
Teachers may find the results of the study of value in their school and
classroom data collection efforts to improve instruction for the purpose of increasing
students’ academic achievement as measured by a multitude of norm and criterion
referenced assessments.
Although the focus will remain on districts and schools in California,
policymakers and scholars may find this study useful in identifying designs for
effective collections of data. A deeper understanding will be gained about how state
and local assessment results and other school documents have combined to provide
sources of information about student achievement.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
16
Limitations
The use of a qualitative case study approach may limit the study through
subjective interpretations of observations and interviews. The researcher cannot
control the participants’ willingness to complete and return surveys and
questionnaires. The uniqueness of the individual school district and school may
make findings and conclusions and recommendations difficult to generalize to other
districts and schools.
Delimitations
The data for this qualitative case study will be collected in one kindergarten
through eighth grade urban school district in Southern California. Data from the
central office and one elementary school will be collected to answer the three
research questions. The data will be collected in the fall and winter of the 2001-2002
school year and will not reflect changes that occur after that period. The district and
school were not randomly selected, but were purposely chosen to represent best
practices in data use and student achievement gains.
Definitions of Terms
For the purposes of this study, key terms have been defined to provide a
common theoretical understanding.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
17
Academic Performance Index (API): A single number to rate the performance of
schools in terms of their student academic achievement.
Accountability: The notion that individuals and or organizations are held responsible
for improving student achievement.
Adequacy: An approach to school funding, school accountability and instructional
practices that are targeted to achieving the states educational goals of increasing
student academic achievement.
Alignment: The degree to which assessments, curriculum, instruction, textbooks and
other instructional materials, teacher preparation and professional development, and
systems of accountability all reflect and reinforce the educational program’s
objectives and standards.
Assessment: A tool, instrument, process or exhibition composed of a systematic
sampling of behavior measuring competence, knowledge, skills or behavior.
Benchmarks: Checkpoints along a continuum to monitor progress towards
performance target or standard.
Conceptual Frameworks: A theory or set of theories by which a design can be
viewed, discussed and data can be gathered and analyzed.
Content Standard: Stated expectations of what should be known and demonstrated in
particular subjects, skill.
Curriculum: The course of study offered by a school or district.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
18
Data: Information including statistics on attendance, grades, referrals, retention and
standardized test results. The information can be disaggregated to reveal
unrecognized patterns and may suggest areas in need of improvement (Johnson,
1997).
Equity: The equalization of students’ access to educational opportunities
High Student Performance: Explicit grade level achievement that aligned to and
exceeds the state standards.
Immediate Intervention/Underperforming Schools Program (II/USP): A California
legislation that provide for additional resources and supports for schools performing
below the 50th percentile on STAR.
Interventions: Instructional strategies such as extended learning programs, tutoring,
change in assignments, and other improved services that are incorporated into the
curriculum to prevent students from failure (Rudolph, 1999).
Multiple Measures: Relying on more that one Indicator to measure assessment of a
student’s academic strengths and weaknesses.
Norm Referenced Testing: Assessment in which individual scores are compared to
the scores earned by the group used in the norming process (Gall, Borg & Gall,
1996).
Performance Standards: Explicit grade level achievement indicators that
demonstrate a specific level of academic proficiency (Linn, 2000).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
19
Professional Development: Programs that allow teachers or administrators to
acquire knowledge and skills needed to perform their jobs successfully.
Public Schools Accountability Act: A collection of related programs and
assessments designed to help state leaders, educators, and the public evaluate the
quality and progress of public schools in California.
Reliable: A measure of consistency in assessment.
Retention: The act or policy of holding students back from advancing to the next
grade level if they do not meet established standards.
Standardized Testing: An approach to assessment in which all assessments and
scored in a consistent way (Gall et al., 1996).
Standard: Statement of expectations for student learning (Linn, 2000).
Standards-Based Education Reform: Consistent interlinked elements that promote
effective use of standards for the improvement of teaching and learning (Association
of California School Administrators Task Force, 1996).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 2
LITERATURE REVIEW
Introduction
Across the United States, policymakers and educational leaders are
implementing accountability and assessment measures in response to increasing
concerns about improving student performance. The states are interpreting the call
for accountability in different ways. Although their policies vary, states face similar
challenges in creating their accountability systems. Individual school districts are
also interpreting the state’s call for accountability. Schools and teachers also are
interpreting their role in the data collection process.
A central focus of these efforts has been the establishment of a common set
of academic standards for all students, the assessments that measure student
performance, and accountability systems that are focused on student outcomes.
This chapter will provide a review of the literature that has led to the current
focus on accountability. A review of the status of student performance in math and
literacy in the United States and specifically in California will be explored. The
nature of standards-based comprehensive school reform in the United States will be
discussed. The role and the intent o f data in school reform will be reviewed. Review
of the literature will encompass the current use of data in schools, both at the
individual teacher and school-wide levels.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2 1
Synthesize of the Literature
The status of student performance, especially in math and literacy in California
and the United States, California is comparatively low when compared to other states
and nations (Donahue, Voelkl, Campbell & Mazzeio, 1999; National Center for
Education Statistics, 1999; Reese, Miller, Mazzeo & Dossey, 1997). The Third
International Mathematics and Science Study (TIMSS), the largest international
comparative study of educational achievement assessed the performance in
mathematics and science for more that 500,000 students in 41 countries in 1996 and
in 1999. O f these 500,000 students, 33,000 were American. The results indicated
that students performed above the average in mathematics in only two other
countries. Overarching results from TIMSS indicate that we need to ensure that all
children receive the instruction necessary to be viable citizens and workers in the
newly emerging, economically competitive world.
The National Assessment of Educational Performance (NAEP) compares
student performance within the United States. This assessment is administered every
four years, since 1969, to randomly selected fourth and seventh grade students
throughout the nation. It has been the sole, ongoing national indicator of what
American students know and can do in major academic subjects. The results are
reported as average performance scores by state. Students are scored and ranked on
a Standard of Proficiency, "Advanced,” “Proficient,” “Basic,” or “Below Basic.”
Over 2/3 of students tested performed at the “Proficient” level. Long-term trend data
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
22
shows only small Improvement in student math performance over the past quarter
century (National Center on Education and Statistics, 2001; Reese et ah, 1997).
The research on the distribution of the performance on all students show that
cohorts of black and Hispanic students tend to lag behind their white and Asian
classmates. Beginning in the early 1990's, Gissmer and Flanagan (2000) analyzed
the NAEP data and began to provide more detail about differences in trends among
blacks, Hispanic and white students; differences in trends for lower and higher-
scoring students; differences by age; and particularly differences by entire cohorts.
Paul Barton in his Executive Summary, Raising Achievement and Reducing
Gaps; Reporting Progress toward Goals for Achievement, analyzed and found that
states have not generally reduced the achievement gap between top and bottom
quartiles or between white and minority students (Barton, 2001). However, it is
important to note that the primary objective of the analysis of the scores from NAEP
has always been to monitor trends in achievement rather than explain those trends
(Grissmer & Flanagan, 2000). To explain these trends, we must look closer at
individual states, local districts, schools and classrooms.
To add to the complexity of the TIMSS and NAEP results, California's
student population is more diverse than other states. Delaine Eastin, State
Superintendent of Public Instruction says, “that the reason California's score lag far
below the national average is that 27% of the state's fourth graders and 19% of its
eighth graders are not fluent speakers of English.” Nationally, those figures are 6%
and 4% for the two grades, respectively (Colvin, 2001).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
23
A glimpse at the state of California’s children is offered to provide light on
the challenges faced by school systems. California is the most populous state and the
seventh largest economic power in the world. Its response to the 21st century
problems will be closely watched because California is, according to the State
Librarian, Kevin Starr, “a prism through which the United States is glimpsing its
future” (CDE, on-line July 2001). California’s population growth has been dramatic
and diverse. In 1950, one out of every 15 people in the United States lived in
California. According to the 2000 Census, one out of every eight people in the
United States lived in California. There are 5.7 million students in the state. Based
on the 1999-2000 school enrollment, Hispanic students comprise 42.2% of the
school population. White students make up 36.9%, Asian and Pacific Islanders 11 %,
African American 8.6 %, and American Indian .9 %. 1.48 million students were
identified as English Language Learners. Schools enroll 2,271,283 students with a
native language other than English, or 38% of California students. Forty-seven
percent of the students come from low-income backgrounds based on their
enrollment in free or reduced price meal programs. The population is expected to
continue to grow.
The California assessment program referred to as STAR, Standardized
Testing and Reporting program, has been designed to measure the achievement
levels of students in California. It is comprised on four elements: (1) a nationally
norm-referenced (NRT) achievement test in English, the SAT 9 (2) The California
Standards Test, (3) California English Language Development Test (CELDT), (4)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
24
The California High School Exit Exam (CABSEE). Using the scores of the SAT
9 and the CELDT, and the state department calculates an Academic Performance
Index (API) for individual schools.
The Stanford 9, known as the SAT 9 test has been administered each Spring,
since 1998 to all students in grades two through 11. The results from these tests
show the distribution of performance for all students. The most recent results from
the 2000-01 school year shows that the overall scores are up. Solid gains were
shown over last year in the elementary grades, where most of Governor Davis’
education reforms have been focused. In grades two through six, the percentage of
students who scored above the national average in math rose by three points, and in
reading, by 1.6 points.
Over the last three years, the percentage of students in grades two through 11
scoring at or above the national average in math has increased by 6.2 points and in
reading by 3.5 points. The greatest gains were in third and fifth grade math, where
students' average scores have moved from the 36th percentile to the 61st percentile
and the 40th percentile to the 55th percentile, respectively, in three years (CDE on
line, August 2001).
A deeper examination of the SAT 9 scores demonstrate that the gap between
English proficient students and English learners has widened in some instances over
the past three years (CDE on-line, August 2001).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
25
The California Standards Test assesses student mastery of California’s
academic language arts and math standards. It includes a writing assessment
administered to students in grades four and seven. This test is referred to as the
STAR Augment Test. Students in grades four and seven were also administered the
California writing assessment, which is a newly created standards-based criterion
referenced exam.
Students in grades two through nine who are designated English Language
Learners (ELL) are administered the English Language Development Test
(CELDT). ELL students are given the CELDT until the student passes the
intermediate level of fluency or after completion of two years of instruction in the
United States’ schools, whichever comes sooner. The students then take the
California Standards Assessment.
In 2001, the California High School Exit Exam (CAHSEE) was administered
for the first time. Approximately 378,000 ninth graders out of approximately
480,000 took the test in Spring 2001. The exit exam consists of English-Language
Arts and math tests designed to assess whether high school students have mastered a
basic set of skills to participate fully in civic life and to succeed in the workplace
(California State Department of Education, 2001). It is intended to improve student
achievement and to restore meaning to the high school diploma awarded in
California. Under current law, starting with the high school class of 2004, students
must pass the exam in addition to meeting other criteria to earn a high school
diploma.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
26
The results of the test reveal that approximately 65% of the students met or
exceeded the passing score of 60% for the English Language Arts section. Forty-
five percent of the students met or exceeded the passing score of 55% for the
mathematics section. Less than 45% of ninth graders passed both sections and less
than 25% of Hispanic and African American students passed both sections (CDE on
line, August 2001).
The purpose of the API is to measure the academic performance and progress
of schools. It is a numeric index that ranges from a low of 200 to a high of 1,000.
The 2001 API established a baseline for a school’s academic performance and sets
an annual target for growth. This target assists the schools in designing programs,
strategies and interventions to improve student performance.
The long-term consequences of low student performance are politically and
socio-economically damaging to our nation. Without a high school diploma the
access to employment diminishes, as does the entrance to a middle class life style
and opportunities for meaningful social inclusiveness.
The consequences are far more reaching than students not receiving a high
school diploma or attending college. A look at how society has changed over the
past 30 years is a compelling reason to take stock. From the 1950’s to the 1970’s,
America’s productivity grew at a healthy pace. The nation was getting richer, and
workers lived better on what they earned.
During the 1980’s, a higher percentage of Americans were working than at
any time in this century. The “baby boom” generation came into the workforce and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
27
many women went to work to maintain family incomes. Therefore, the American
economy was growing (MCEE, 1990). Over the past 15 years, the earning gap
between white collar professionals and skilled trades-people has gone from 12% to
37%; the gap between professionals and clerical workers has gone from 47% to 86%.
In the 1980’s, earnings of college educated males increased by 10%.
Earnings of those with only high-school diplomas declined by 9%. Those in the
workforce who did not hold high-school diplomas saw their real incomes drop by
12% (NCEE, 1990).
In our new global economy 50% of high school graduates are unprepared for
the challenges to support a family and become responsible citizens. Almost 4
million adult Americans cannot read, sign their names, or perform simple addition
and subtraction. Thirty million workers can scarcely read and write, and another 40
million (40,000,000) lack the basic skills to get by in a technological and rapidly
changing world (Lloyd et a l, 1998). The most pressing social issue is the increasing
dramatic disparity between the skills children are acquiring in our school system and
the skills they will need to obtain good jobs in an increasingly selective, globally
influenced job market (Mumane & Levy, 1996). Although our educational system is
undergoing incremental improvement, the business environment is growing In
exponential rates. If our students continue to improve at the rate they are, fewer and
fewer students are being prepared for a global economy.
Standards-based comprehensive school reform in the United States has been
emerging since A Nation at Risk was published in 1983. This report focused on
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
28
American high schools and their unprepared graduates. The national and local
conceptual framework on comprehensive school reform began in 1989. The then
governor of Arkansas, Bill Clinton, led a summit between the then President George
H.W. Bush and the nations’ governors. They bucked the nation’s long-standing
trend o f local control in public schooling and established as a national “goal” that
students in grades four, eight, and 12 demonstrate competency in challenging
subjects such as English and Math. Since then, ratcheting up standards has become a
cornerstone of school reform at the state level (Toch, 2001).
The Federal Improving America’s Schools Act (IASA) of 1994 required that
states test students at least once in each grade span; third-to-fifth, sixth-to-ninth and
10th-to-12th (CPRE, May 2001). In 2002, President George W. Bush enacted the
latest educational reform for the nation, No Child Left Behind, which replaces the
IASA of 1994. This new national reform legislation requires yearly assessment and
focuses on research, based practices for teaching.
The conceptual frameworks on comprehensive school reform have been
emerging to increase the accountability and effectiveness of public education in
America. These frameworks are described as four phases or waves of school reform
at the local, state and national levels. Kent Lloyd and Diane Ramsey describe these
waves in “Reclaiming our Nation at Risk, ” 1998. The first wave of reform came
from outside the education community. Governors took a leadership role and other
elected officials advocated an agenda for action that was regulatory-laws, mandates,
and detailed rule-making. The second wave of reform was led by educators. The
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
29
focus was on studying effective school principals and teachers who were flexible,
innovative and committed to their work and their students. Later, the improvements
focused on parent involvement and external collaborations. By the mid 1980’s, the
reform movement focused on local school sites and their ability to reorganize the
instructional procedures such as strategies, working conditions and external business
and industry partnerships. The third wave of reform is characterized by national
leadership. It was becoming clear that there was a linkage between American
education crises and our declining economic competitiveness (Mumame & Levy,
1996). National education goals were established through President Clinton’s Goals
2000: Educate America Act legislation. This act encouraged local community-based
actions to meet the educational needs, help more students achieve higher standards,
increase parental partnership, and improve teaching. There was a dramatic change in
curriculum, instruction, and assessment that focused on student achievement of
standards. The fourth wave of reform is what is called the grassroots public
engagement--a nationwide, school-site, community-based, reform movement driven
by parents, teachers, and citizens including governors, business leaders, presidents,
and educators.
The most compelling framework, however, is the current focus on measuring
student achievement against a set of standards. The Obey-Porter legislation of 1999,
set aside nearly 50 million dollars in federal funds to encourage states, districts, and
schools to engage in comprehensive school reform. Comprehensive school reform
seeks to have schools create, adopt, or adapt an educational approach that bring all
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
30
students and programs Into a cohesive, school-wide educational strategy. There Is
a curriculum that sets high standards for all students and doesn’t “water down”
material for those in categorical programs, but makes use of appropriate instructional
strategies that provide extra help for students who must straggle to master the regular
curriculum. The reform legislation also addresses the grouping of students for
different subjects, the scheduling of instruction and planning time for teachers, pupil
support and home outreach strategies, professional development and the use of
computer technologies (Odden, 2000).
Educational and business literature and empirical studies support standards-
based reform to push our nation to a competitive position with other high performing
countries such as Japan and Germany (Marshall & Tucker, 1992).
The standards-based reform helps mitigate the “significant achievement gaps
that exist between rich and poor, and among ethnic and cultural groups” (West-Ed,
2000). Full implementation of standards-based instruction and assessment was the
most important factor identified by the Education Trust in 366 schools with poverty
level less than 50% that were high performing or most improved in achievement tests
in 21 states. “These schools have become places where all classroom activity is
aligned with state standards” (Ed Trust, 1999).
But how good are the state standards, how are the standards matched with
strong school accountability? These questions have been addressed by the Fordham
Foundation in its State o f the State Standards, 2000 Report (EdExcellence 2000).
The report indicated that in 1997, 42 of the 50 states held mediocre or inferior
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
31
expectations for their kindergarten through 12th grade students, at least on most
subjects. In the same year that the National Education Goals were to be met, most
states still had not successfully completed the first step of standards-based reform
(EdExcellence 2000). Again in 1999, the Fordham Foundation re-evaluated the
standards of the states because many states had changed their standards since 1997.
Overall there was mild improvement in the quality of academic standards and the
number of states, including California earning “honors” rose. The trends that
emerged were: (1) State standards are becoming more specific and measurable, (2)
content is making a comeback and (3) states are less enamored o f national standards
promoted by professional organizations.
The lessons learned from the Fordham Foundation evaluation are: (1) Most
states cannot legitimately claim to embrace standards-based reform (2) states must
improve their academic standards at the same time they are installing tougher
accountability systems (3) since standards-based reform is so elusive, it should not
be used as the sole “silver bullet” for revitalizing America’s kindergarten through
12th grade education. Other reform strategies should be embraced as well.
A growing number of states are getting serious about school accountability,
attaching assessments and consequences to their academic standards (Achieve,
2000). Data on standards from the Fordham report and data on school accountability
from Quality Counts ’99 depict the state of standards-based reform in America. The
data reveals that (1) Five states with California being one of them and four of them
in the south combine solid standards with strong accountability, (2) 30 states display
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
32
a combination of mediocre or no standards and weak accountability, and (3) 12
states rest high stakes accountability systems upon mediocre or inferior standards.
The nine key elements of standards-based reform evolved from the research on
the comprehensive school reform models recognized by the federal government, and
from a synthesis of findings from the research on high-performing, high poverty
schools and districts (Johnson, Haycock, Carter, Reeves, Supovitz, Pogllinco &
Snyder, 2001; Statewide System of School Support, CDE, 2000). The elements are:
1. A focus on academic achievement as the highest priority and mission of the
district and school. Structures at all levels that guide, examine and offer
consequences for student achievement results. The district office’s accountability
contains three levels: (1) common goals, (2) department goals, and (3) narrative
descriptions of the performance of the departments. The schools accountability
contains two levels: (1) using student learning standards as a foundation, and (2)
enhancing teachers’ capacity to provide effective instruction toward meeting those
standards.
2. High quality teaching and learning of rigorous standards through the use of
staff development linked to the teaching of standards. This must be done through
sustaining on-going professional development aligned to instruction and
assessment.
3. Support within the school for collegial and collaborative faculty and teamwork
toward engagement in self-study and reflection.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
33
4. High expectations of student achievement through the use of measurable goals
and benchmarks.
5. Use of state and local standards and accountability system to drive school
improvement efforts and students’ academic achievement.
6. Parent involvement that actively engaging the parents, community and other
stakeholders in the process.
7. Evaluation strategies and the use of research and or data to promote continuous
improvement and provides meaningful assessment of student achievement
through the following venues: intervention for the students who are not
achieving the standards, measurement of student achievement against a set of
standards, on-going formative and multiple assessment on student progress, and
other strategies for the struggling learners
8. External Technical support and assistance.
9. Safe, orderly school climate conducive to learning which focus on self-discipline
and self-motivation.
Additionally, the literature indicates there are four organizational features
that will be needed to make such reforms successful (Supovitz, Pogllinco & Snyder,
2001):
1. Governance for a systemic and flexible decision-making structures and practices
that is responsive to the identified needs for student achievement.
2. Organizational arrangements that support collaborative district and school
analyses of student performance and instruction.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
34
3. Leadership with clear focus on promoting the advancement of student
achievement at all levels.
4. Financial arrangements to realign funding sources for support of reform
measures (Clune, 1998). Odden (2000) contend that collecting resource data by
school level and curriculum content areas is important for both state and
nationwide education policy. “It would be helpful to know how much is spent on
the content areas as compared to other expenditures” during the shifting times of
policy focus.
The intention to use student performance data is to provide accountability,
feedback, improvement of instruction on a broad scale and to increase student
learning for sustainable success (Asp, 2000). Specifically data and information can
be used in the following ways (Glickman, 2000; Holcomb, 1999; Reeves, 1999,
2000):
1. Reporting and disaggregating student performance
2. Holding schools, districts and students accountable through the state defined
accountability systems, and locally defined accountability systems
3. Setting performance targets and goals for schools
4. Defining progress
5. Creating consequences for performance for the school, the district and the
students
6. Assisting low performing schools
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
35
More specifically, the intentions of data use in comprehensive school
reform builds on the following four pillars to monitor student achievement gains
(NCEE, 2001).
1. Using learning standards as a foundation for reform work and the district, school,
and classroom levels. The data to support a standards-based focus at the district,
school and classroom level would include the assessment results that target critical
skills and concepts aligned with the standards. Additionally, providing student-
specific, valid and reliable information about student performance vis-a-vis the
standard using formative benchmark assessment results that are collected and
profiled every six to 12 weeks.
2. Enhancing teachers’ capacity to provide effective instruction toward the standards
through professional development. This includes assessment training and support for
cross grade articulation to support strong instructional improvement and reflects a
collaborative effort to reinforce consensus decision-making and a culture of
reflection and problem solving.
3. Actively engaging the parents, community and other stakeholders in the process
to enhance public understanding, engagement and support of school reform. The data
used for this focuses on the communication to the public about the progress the
school is making.
4. Provide meaningful, on-going assessment of student achievement for instructional
accountability. There are four types of data to support strong instructional
improvement. The first is disaggregated data at the individual student level in a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
36
usable format. The second is data that identifies students that do not meet grade
level expectations. The third types of data are the summative results that are
disaggregated and used to address instructional weaknesses. The fourth type of data
are the use of the results to inform instructional decisions and evaluate program
effectiveness including data that determines if schools meet targeted API growth,
including each sub-group.
With accurate information, educators are better able to make effective
decisions (Johnson, 1997). If knowledge is power, then studying the data about the
abilities, skills, attitudes, and learning styles of students empowers educators to
adjust the curriculum to achieve whatever goals the school, district and state have
chosen. Doug Reeves (2000) states, “that with a high degree of confidence, there are
consistent associations between classroom strategies and student achievement in a
wide variety of tests and subjects.”
McLean (1995) contends that, “implementation o f a complete program of
data collection and use can lead to the improvement of education as has no other
educational innovation of the last century.” Traditionally, data collection in schools
has been used to assess student performance, only. However, with a focus on
accountability, educators are more likely to rely on data to help them make better
choices and uncover a better way of serving students and the community.
The future belongs to societies that organize themselves for learning, where
all institutions are organized to learn and to act on what they learn (Mumane &
Levy, 1996). School improvement happens when a school develops a professional
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
37
learning community that focuses on data about student work and adjusts teaching
strategies based on that data. Any school that is trying to improve has to think of
professional development as a cornerstone strategy to build the needed skills,
capacities, and relationships that foster reform and accountability.
Each level of the educational system has a distinct role in gathering data to
support the implementation of reform. States often provide technical assistance for
schools, especially low performing schools. Districts frequently develop standard-
aligned curricula and assessments and set designs for systemic reform. Schools use
student achievement data to develop evaluation and improvement plans such as state
comprehensive school-wide plans, Program Quality Reviews and Western
Association of Schools and Colleges Accreditation. Also, using the states middle
and high school reform documents, Taking Center Stage, 2001 and Aiming High,
2001, respectively, requires attention to data related to systemic alignment,
collaborative relationships, leadership efforts, and on-going review of progress.
Scholars and policymakers have been coming together for the past several
decades to understand the complexities of our educational system. Most recently
however, their intent and the high-stakes associated with them have been anted up as
a result of increased public and business attention to education. The intention to
gather information to align all components leading to student achievement as
measured against standards cannot be underscored.
A fundamental aspect of our educational system is fairness and the
opportunity to learn for all students. Opportunity to Learn, describes the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
38
circumstances and conditions provided to ensure that each student has the quality
of personnel, courses, curriculum, materials, technologies, instructional time,
working space, financing, procedures for placement, provisions for special aid, and
other services necessary to achieve content and student performance standards
(Council of Chief State School Officers, 1995). The data gathered should be able to
demonstrate an alignment between assessment, instruction and access. Even if an
assessment may be high in its content toward the standards, the assessment would be
unfamiliar at least to students if instruction is poorly aligned with the standards.
Students should not be expected to demonstrate proficiency when there are
insufficient opportunities to leam. To this end, educators can work to ensure that all
students have the opportunity to leam, not just those students who have the supports
and resources readily available to them (Coladarci, Johnson, Beaudry. Cormier,
Ervin, Rossenblum & Silvemail, 2000).
In the absence of comprehensive evaluations, state policymakers lack
empirical evidence about the effects of reforms on student achievement. State
education policy is therefore vulnerable to constantly changing trends, which
contribute to public skepticism about reform (Education Commission of the States,
1996). Some say that assessment and accountability are two of the most powerful,
most controversial and most damaging policy initiatives on the American
educational landscape (Pearson, Vyas, Sensale & Kim, 2001). Others conclude that
people are empowered by access to data and belief is replaced by knowledge. The
American Educational Research Association (1999) in the Standards fo r Educational
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
39
and Psychological Testing suggests that California policymakers include multiple
opportunities to pass exams, have test-taking strategy preparation, and provisions for
appropriate support systems that afford students the opportunity to leam the content
and skills that are tested. It has become apparent in the literature that the
implementation of education reform requires tremendous attention, activity,
resources, and coordination across all levels of the system.
In this study, the construct of data means a collection of things known or
assumed: facts, figures, evidence, records, statistics-information that describes or
from which conclusions can be inferred. A piece of data can be number
(statistical/quantitative data) or a paragraph of text (textual data). The data may be
classified into three groups: (1) data about people includes demographic
characteristics, such as age, race, gender, income or grade level, (2) data about
events includes attendance, graduation rates, referrals, retention rates, retention, and
(3) data about things or processes such as norm and criteria- referenced tests scores,
and specific instructional or intervention programs (Calhoun, 1995; Ma, Erickson &
Thomas-Strong, 1995; Reeves 2001). The importance of collecting such data is that
it can be disaggregated to reveal previously unrecognized patterns and may suggest
areas in need of improvement (Johnson, 1997).
The role of data in California is multi-dimensional as evidenced by the
current and emerging trends. These trends can be best explained by California
legislative platform that includes assessments, and the rewards and sanctions that are
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
40
connected to them. There will be increasing accountability for improving student
performance for pupils, teachers, administrators, and school districts.
The emerging, long-term proposal for the California State Assessment
System will include individual student scores for analysis. In grades two through
eight, the California Standards Assessment will be given. The test Includes the core
subjects (English/language Arts, Math, Science, Social Science, and History).
The Writing Assessment will continue to be given to students In fourth and
seventh grades. Beginning in the ninth grade, the California High School
Assessment System including writing samples will be implemented. The California
High School Assessment System includes the end of course standards-based exams
in the core subjects and the CHSEE. In order to facilitate an articulated assessment
system, the state is asked to adopt a reasonable alternative for college admission and
course placement assessment that is consistent with California’s academic standards
and eliminate duplication (The Ed Coalition, on-line, July 2001).
Additional student evaluation information will be gleaned from school
districts utilization of multiple measures which include student work, coursework
grades, end of course standards based exams, district-wide assessments, California
High School Assessment.
The Academic Performance Index (API) will also include multiple measures.
These multiple measures will be: California Standards Assessment, California High
School Exit Exam, survey version of nationally norm test, attendance rates, CELDT
results, California High School Assessment System and graduation rates.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
41
Additionally, efforts to align curriculum and assessment can be discussed
from the perspective of the legislative action known as the Public Schools
Accountability Act (PSAA) of 1999. This act has focused educators’ attention and
expertise on the classroom, school, and district (Brownell, 2001). This statewide
measure is designed to effect student achievement through the use of sanctions and
incentives (Duffy, Goertiz & Le Floch, 2001). The PSAA is California’s
accountability system that is comprised of three sections: (1) the Academic
Performance Index (API), (2) the Immediate Intervention/ Under-performing
Schools Program (II/USP) and, (3) the Governor’s Awards program (Brownell &
Hart, 2001).
Currently what is provided to schools by the State Department of Education
are state content standards in mathematics, language arts, social science, and science;
individual and group scores on SAT 9; school API targets and statewide ranking and
individual norm referenced scores. Using the scores of the STAR, the state
department calculates an Academic Performance Index (API) for individual schools.
The API is used to rank schools according to its’ students achievement. The data is
used to determine participation rates for awards eligibility (95% elementary and
middle schools, 90% high schools).
The expectations of the schools are: increase student achievement to reach
and exceed API target growth, provide on-going assessment and intervention, and
provide on-going sustaining professional development aligned to the standards and
instruction.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
42
Assessment and accountability are two of the most powerful policy
initiatives on the American educational landscape. Work continues to align
California’s assessment system with content standards and will continue until at least
2003 (EdSource on-line, 2001). This alignment, in principle allocates authority,
provides incentives, builds capacity, measures progress, reports results and enforces
consequences, all of which are centered on student performance (The Progress of
Education Reform 1999-2001).
In addition to the statewide achievement test, students in grades four, eight
and 12 also take the National Assessment of Educational Progress test which
assesses and provides a national ranking score of each state.
These scores, and the scores of the other criterion local assessments add to
the multitude of data that we have on student performance. These scores are the first
that give us implicit results of how well students are individually meeting the content
standards.
Efforts to align the SAT 9 Augmented test with all of California’s standards
are fast-tracked. The California Standards Assessment will provide individual
student scores and a standard based report for each student in grades two through
nine. The writing Assessment at grades four, seven, and 10, (with 10th grade as part
of the High School Exit Exam) is also given. The California Standards Assessment
will include items to help evaluate educational programs over time. Standards-based
test preparation materials will be made available to educators, students and parents to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
43
bolster test taking skills and familiarity with the California Standards Assessment
and High School Assessment System.
The High School Exit Exam Assessment System will provide individual
scores for grades 10 through 12. This system will include the recently developed
High School Exit Exam and other possible alternatives like the end of course subject
specific standards based exams and Golden State Exam. The English Language
Development Test will continue to be used to determine English proficiency.
The research about the current use of information and data in schools shows
that little has been done about large-scale school success. Additionally, scant
research exists on the effects of school district (as opposed to school campus),
organization, operation, and leadership on student achievement (Berry & Achilles,
1999).
The scant research of sustained, district-wide academic success for children
of color and children from low-income homes has been done in New York, North
Carolina, and Texas. These states have highly developed and stable state
accountability systems (Skrla, Scheurich, Johnson, Hogan, Koschoreck & Smith,
2000). The school districts have created conditions in which school success, for all
children is a reality (Elmore, 1997; North Carolina Department of Public Instruction,
2000; Ragland, Asera & Johnson, 1999).
The Charles A. Dana Center, at the University of Texas at Austin, reports on
the findings from four districts deemed “best of the best” in terms of widespread
academic success for children from low-income homes and children of color (Skrla
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
44
et al., 2000). The districts demonstrated impressive gains in the passing rates of
all state tests (TASS) and widespread improvement in academic performance for ail
students over six years. The major findings of the study reveal the following types
and purposes for the uses of information/data in schools: (1) Align the curriculum,
(2) develop coherent practices for the delivery of instruction within the curriculum
through benchmark targets (p. 24), (3) acquire, allocate, and align fiscal, human and
material resources, (4) collect, interpret, use data and monitor results, (5) supervise,
evaluate, and hold people accountable, (6) refocus energies, refine efforts and ensure
continuous performance, and (7) create and nurture alliances.
The use of data to align the curriculum and develop coherent practices was
successful in the Brazosport School District, one of the four high performing, low
poverty districts in Texas. The district identified eight steps to the use of data for
teachers: (1) Establish baseline data on each child using identified test results, by
disaggregating all the data by subgroups, (2) focus on weak areas, (3) build to the
strongest areas, (4) maintain what has been taught, (5) develop an instructional focus,
(6) develop an instructional calendar, (7) assess and track each child to determine
how they are performing and (8) re-teach, and tutor. The spiraling process
continues. Linda Darling-Hammond (1999) also identified guidelines for the use of
data for raising standards of practice. They are: (1) To teach concepts in depth for
understanding, (2) to diagnosis, (3) to provide significant extra supports for students,
and (4) to provide longitudinal data to track student learning gains. The findings
from these studies coupled together offer a variety of purposes and uses for the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
45
teacher to collect data. All teachers have to assume the responsibility for
achieving equitable and excellent learning in their classrooms.
The school-wide leadership team, including the principal, use information to
practice “proactive redundancy.” The Dana Center describes “proactive
redundancy” as designing two or more “processes whose goal is to change a same
specific practice.” Attaining academic success with all children is important;
therefore, the schools proactively develop redundancies in their processes to ensure
that all students are learning. This process is used for the following purposes: (1) To
provide equitable, excellent learning, (2) to lead improvements in instruction and
thus student achievement, (3) to motivate positively and assist teachers to improve
instruction and thus learning, and (4) to assist teachers in being successful with all
students at equally high academic levels.
The use of coaches, interventions, vertical teaming, critical friends, and
mentors are examples of the processes of proactive redundancy to provide support to
teachers in order to succeed. Kate Jamentz (1999) and Douglas B. Reeves (2000)
each articulate the sustained, focused examination of student work, will generate a
culture of reflection that is crucial to the analysis of information.
There are four explanation/speculations for the lack of use of data. The first
is that data analysis has never been viewed as a high priority. Until recently, state
education departments put little emphasis on schools to gather data, and thus
provided little incentives to devote time, money, and staff resources into using data
in new ways (Bernhardt, 2000). Edie Holcomb in her book Getting Excited About
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
46
Data, identified a continuum of intrinsic to extrinsic motivating factors for why
teachers use data. The most intrinsic, being able to satisfy ones own drive for
excellence. The most extrinsic, being able to respond to the federal, state, and
business sectors press for accountability. These motivating factors are important
considerations for moving classrooms and schools to embrace data. The second
speculations for the lack of data use is fear, the fear of data and the analysis of it has
gotten in the way of using the information that has been so readily available. There
is the fear, also, that the data analysis will turn up something educators do not wish
to have revealed. Third, some districts and schools have antiquated jobs and
technologies to allow the analysis of data to be a routine practice. There are too few
people trained at the school level to collect and maintain databases. Finally, there
are not enough good examples of schools gathering, maintaining and benefiting from
the use of data.
Conclusion
The literature is contributing to the study by building a foundation of context
to be examined. The context has five components that will build upon the already
existing research on the design, implementation and adequacy of data utilization in
schools.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
47
The five components of the context that will contribute to the study are:
1. The emphasize on the examination of performance for all students
2. The key elements of school reform and the conceptual and theoretical
frameworks that support them.
3. The intended role of data and information in school reform efforts.
4. The expected and actualized role of data use in California.
5. The promising perspectives of data utilization to improve student
achievement.
The study focuses on the examination of data collection, and implementation
in one elementary school. The five components discussed above are found in the
literature and serve as the conceptual framework for the study. The components have
been synthesized to create three research questions to analyze the District’s design
for data use to make instructional decisions to improve the achievement results of the
students. The schools implementation of the data use design will be analyzed. The
adequacy of the design and data use will also be analyzed based on foundational
context found in the literature.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
48
CHAPTER 3
METHODOLOGY
Introduction
This study will add to the educational literature and assist district and school-
level leaders in determining the value and usefulness o f a district design to address
the current and emerging state context for assessing and improving student
performance. The study analyzed the use of data to improve instruction to increase
achievement in one elementary school within the district, and specifically in six
classrooms. The degree to which the design is adequate for implementing data
collection and analysis to improve student achievement also was studied.
The focus of the study was on one school district, The Bandaras Heights City
School District, in an Urban Fringe (large city) in Southern California with a high
percent of children from poverty. The school site that was studied is the Del Sol
Elementary School, a kindergarten through grade five school. Six of its classrooms
were studied. One teacher from first grade, second grade, third grade, fourth grade,
and one teacher with a third through fifth grade special education class were studied.
National policymakers may find the study of value in identifying policy
designs for data collection at the district, school and the classroom levels. District
and school site leaders may also gain insight into the adequacy of collecting
information on students and it’s relationship to student achievement. Teachers and
other school site staff may gain insight into the process of analyzing student
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
49'
performance data to improve student achievement as measured on norm
referenced assessments as well as state and local performance criteria-assessments.
The multi-method qualitative approach was utilized to answer the following
three research questions:
1. What is the district design for using data regarding student performance, and how
is that design linked to the current and the emerging state context for assessing
student performance?
2. To what extent has the district design actually been implemented at the district,
school and individual teacher level?
3. To what extent is the district design a good one?
The research focused on the use of data collection and disaggregation
strategies to improve student performance at the Del Sol Elementary School that has
been designed by the Bandaras Heights City School District. Because the study
focused on a specific district and school, a case study is most appropriate. A case
study is defined as the in-depth study of instances of a phenomenon in its natural
context and from the perspective of the participants in the phenomenon or case (Gall,
Borg & Gall, 1996).
The phenomenon or case of this study has two aspects: This study used a
multi-method qualitative approach. There were two aspects of the study. The first is
the focus of analysis. The focus of analysis is on the collection of data in an
emerging time of assessment analysis. The second aspect was on the collection of
data at the school district and at the school site.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
50
Data were collected to inform the researcher of the school districts design
for using data regarding student performance. Additional data were collected to
determine how the design is linked to the current and the emerging state context for
assessing student performance.
Data were collected to determine to what extent the district design has
actually been implemented at the district, school and individual teacher level.
Additionally, data will be collected to determine to what extent is the district design
a good one.
This is a single-case design. This design is appropriate because the district
studied had a history over the past three years of innovations, specifically those
related to the improvement of student performance and student achievement results
in writing as indicated on state norm-referenced tests. The extent to which this
district created, aligned and implemented a data driven design is of interest to this
researcher.
Sample and Population
Five criteria were used to select the sample for study:
1. Purposive Sample - The district and school were selected because they were
likely to be “information-rich” with respect to the purpose of this qualitative
research study.
2. District and school was not always high performing. The school is this study had
a three-year history of continuous improvement in student achievement gains as
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
51
compared to the schools of similar characteristics within the district. The
performance indicators for the selection of the district were that the SAT 9 scores
have been increasing and that the district appears to use standards-based criteria-
assessments, and other local multiple measures to assess and explain student
performance.
3. Nominated - The number one stated priority in the district is the continuous
improvement of student achievement. Resources had been allocated in the area of
professional development, interventions for under-achieving students, and
English Language Learners (ELL). ELLs’ participated in one of three district
programs: the mainstream program, the structured English immersion program or
the alternative program.
4. The district and school were selected because of they used cutting edge data
practices. The district appeared to be using current and emerging best practices in
light of state context for assessing student performance. The school participated
in the collection of data about student achievement, including state test scores,
local criteria-assessments and a variety of other measures.
5. The district had a diverse student population with lots of urban challenges.
District student population is 7,116. There are 11 elementary schools and two
middle schools. The school in the study has an enrollment of 526 students.
Table 1 illustrates the economic and ethnic representation of the district and
school student population.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
52
Table 1
Economic and Ethnic Representation of District/School Student Population
Population District % School %
Free/Reduced Price Meals 60.1% 57.8%
Families receiving CalWorks
(California work opportunity program for
parents.)
15.8% 14.7%
Hispanic 86% 86%
White 11% 11%
Asian, Pacific islander 2% 2%
Other 1% 1%
English Language Learner 30.2% 27%
Table 2 represents the credentials held of the teaching staff at Del Sol
Elementary School.
Table 2
Credentials held by the Teaching Staff at Del Sol Elementary School
Credentials Held Number of Teachers
N=25
Professional Clear 17
Emergency 7
CLAD 3
Bilingual 1
Special Education 3
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
53
The ethnicity of the 25-member teaching staff is represented in Table 3.
Table 3
Ethnicity of Teacher Staff - N=25
White Hispanic Asian Unknown/
undeclared
13 10 1 1
The range of experience teaching within profession is 1-41 years. The
number of years in the current profession range from 1st year to 10th year. There are
24 females and 1 male teacher in the school.
Table 4 for represents the demographics of the six teachers interviewed in
this study.
Table 4
Demographics of Teachers Interviewed
ID
#
Credentials held Grade Teaching Years Teaching Years in Current
Assignment
1 Elementary 3rd 6 6
2 Elementary 1st
(Alternative)
6 6
3 NY Elementary
C A Emergency
10 2.5
4 Emergency 5th 1.5 1.5
5 Elementary &
Special
Education
(Learning
Handicapped)
Learning
Handicapped 3-5
24 8
6 Elementary 4 20 8
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
54
Instrumentation
Three questions were asked to determine the district’s design, the degree of
implementation of the districts’ design and the level of adequacy of the design. Each
of the three questions and the data collection instruments designed for each question
is discussed below.
The first question asks, “What is the district design for using data regarding
student performance, and how is that design linked to the current and emerging state
context for assessing student performance?” This question was developed to gamer
data descriptions of data use policies and strategies. There were five areas of focus
for data collection. They are:
1. Student performance assessed in the context of current and emerging
instruments.
2. Overview of the elements of district design of data use to improve student
performance.
3. District decisions and rulings that support use of district design.
4. Intend results of design plans to improve student performance.
5. Data use policy and strategy funding.
The data collection instruments developed to answer this question were
created as a Case Study guide. The Case Study Guide includes:
a) interviews with a district administrator, a site administrator, six teachers (made
up of grade levels and special needs),
b) mapping of data flow at district and school site,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
55
c) artifact analysis and collection including: district policies and procedures;
archival records and information from district, school and classroom; school and
class policies and procedures, memos, bulletins, handbooks, and student work.
d) quantitative data including: SAT 9 district, school and student reports, 1998,
1999, 2000, 2001; Standards Test in Language Arts, 2000, 2001; API Report
1999, 2000, 2001; District baseline and benchmark assessment data
1997,1998,1999,2000,2001, 2002.
The second question asks, “To what extent has the district design actually
been implemented at the district, school and individual teacher level?” This question
was developed to collect evidence about the implementation of data use policy and
strategy in practice. There were five areas of focus for the data collection. They
were:
1. Degree of design implementation.
2. Implementation of current data practices.
3. Implementation of emerging state data practices.
4. Accountability for data use at the district, school and individual level.
5. Improving student achievement through implementation of data use.
The data collection instruments developed to answer this questions were:
a) The Case Study Guide
b) situated interviews with the same teachers interviewed for research
question 1
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
56
c) Teacher Questionnaire
d) Stages of Concern Questionnaire (SOC)
The third question asks, “To what extent is the district design a good one?”
This question was developed to determine how adequate the district design was to
improve student performance as demonstrated in standardized and criterion
assessment results. To answer this question, four areas of foci were developed.
1. The district support of standards-based instruction and assessment: The intended
district result and the observed impact on the school site.
2. The district and school accountability to standards-based curriculum: Student
performance data forwarded to the school and how the school is utilizing student
data to impact student performance.
3. Determination of “high” student performance.
The data collection instruments developed to answer this question are:
a) Case Study Guide
b) Researcher Rating Form was completed at the conclusion of the data
collection and analysis by this researcher to determine the designs’ adequacy.
Data Collection
During data collection, the conceptual framework and the guiding interview
questions served as the focus when conducting formal and informal interviews and
while observing at the district and school site levels.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
57
The data collection occurred in two rounds. The days for each round were
scheduled four to six weeks apart. The schedule is as follows:
Round 1
Day 1 District level interview (Assistant Superintendent) Collected artifacts and
quantitative data.
Day 2 Administered and collected Stages of Concern Questionnaires and teacher
Questionnaires from all teachers.
Day 3 Site level interviews with teachers (case study oriented and situated
interviews) and administrator. Collected artifacts and quantitative data.
Break for preliminary data analysis and data mapping
Round 2
Day 4 Completed any unfinished teacher interviews
Day 5 Completed interview with site principal
Day 6 Completed collection of artifacts and quantitative data
Data Analysis
Data analysis was conducted using the literature and the conceptual
frameworks as guides. Each research question was analyzed from the data collected
to answer the element in the conceptual frameworks.
Analysis of research question one included a comparison of responses from
the interview questions asked of the assistant superintendent, principal and the
six teachers. Comparison of interview responses to the types of artifacts and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
58
quantitative data collected also were conducted. The results of the analysis
lead to the creation of the Mapping of the Data Flow at the district and school site.
Analysis of research question two compared the responses from the interview
questions asked of the assistant superintendent, principal and the six teachers.
Comparisons were made between the types of artifacts, documents, and quantitative
data collected. Individual responses from the Stages of Concern Questionnaire were
tabulated using a Stages of Concerns Scoring Matrix. Average scores for each of the
stages were generated. An analysis and interpretation of the profiles of the following
was conducted:
1. The highest average score across all six stages of concern for each teacher.
2. The highest score for stages 0-4.
3. The number of stages per teacher (for stages 0-4) that had at least averages of
3.5.
In addition, the distribution of scores using frequency counts were used to
identify trends and patterns across the six stages.
Results from the Teacher Questionnaire were analyzed to determine teacher
perceptions o f Research Question #2. Average scores using a 5 point Likert Scale,
0=Don’t know, ^D isagree Strongly, 2=Disagree Somewhat, 3 - Agree Somewhat,
and 4=Agree Strongly, determined the range of teacher perception o f the degree
of implementation, the range of accountability for data use and improvement of
student achievement through implementation of data use.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
59
The analysis of Research Question #3 included the comparisons of
responses from the interview questions asked of the assistant superintendent,
principal, and the six teachers. Comparison of interview responses to the types of
artifacts and quantitative data collected was also be conducted. A Researcher
Rating Form was completed at the conclusion of the data analyses by this researcher
to determine the designs’ adequacy. The Researcher Rating Form was a five-point
Rating Matrix. The Rating Matrix had been created to provide a scoring guideline to
accurately assess the three areas of focus.
Problems that arose during the data collection phrase and that were
anticipated include: scheduling concerns or conflicts that caused a change in the data
collection schedule, mismatch of information revealed during interviews and artifact
analysis and mismatch of statements among participants.
Problems that arose and that the researcher was unable to anticipate or work
with include possible inaccurate responses on the Stages of Concern Questionnaire,
possible inaccurate responses on the Teacher Questionnaire, possible inaccurate
responses during interviews.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
60
CHAPTER 4
THE FINDINGS
Introduction
The purpose of this study was to examine the use of data to improve student
achievement in one elementary school. The school district’s design for data use was
analyzed for its policies, strategies and involvement in creating a design or plan for
data utilization. It was examined to determine its value, utility, and adequacy. The
elementary school and six of its classrooms were studied to determine the degree of
implementation of the district design and the extent of improved student performance
in the area of language arts/writing.
The focus of the study was on Bandaras Heights City School District, an
elementary district with a culturally diversed student population of 7, 200 in an
Urban Fringe (large city) with a high percentage of children from poverty. The
school site studied was Del Sol Elementary School. The student enrollment during
the 2001-02 school year was 527.
The data were collected to inform the researcher of the district’s design for
data use and the degree of the design’s implementation in a school and specifically in
six classrooms. Additionally, data were collected to determine to what extent the
district design was a good one. The data were collected through formal interviews
with the district Assistant Superintendent of Instruction, the school site principal, and
with six classroom teachers from the selected school site. Situated interviews were
conducted with the six classroom teachers. A Teacher Questionnaire and a Stages of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
61
Concern Questionnaire were given to all teachers to determine the degree of
knowledge and usage of the district’s design.
Observations, document reviews, and artifact analyses were also conducted at
the district, school, and classroom levels to gamer additional data for this study. At
the conclusion of the data collection and analyses, A Researcher Rating Form was
completed by this researcher to determine the design’s adequacy.
Three questions were asked to determine the district’s design, the degree of
implementation of the district’s design, and the level of adequacy of the design. Each
of the three questions is discussed below:
The first question asks, “What is the district design for using data regarding
student performance and how is that design linked to the current and emerging state
context for assessing student performance?” This question was developed to gamer
descriptions of data use policies and strategies. There were five areas of focus for
data collection. They are:
1. Student performance assessed in the context of current and emerging
instruments.
2. Overview of the elements of district design of data use to improve student
performance.
3. District decisions and mlings that support use of district design.
4. Intended results of the design to improve student performance,
5. Data use policy and strategy funding.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
62
The second question asks, “To what extent has the district design actually-
been implemented at the district, school, and individual teacher level?” This question
was developed to collect evidence about the implementation of data use policy and
strategy in practice. There were five areas of focus for the data collection. They
were:
1. Degree of design implementation.
2. Implementation of current data practices.
3. Implementation of emerging state data practices.
4. Accountability for data use at the district, school, and individual level.
5. Improving student achievement through implementation of data use.
The third question asks, “To what extent is the district design a good one?”
This question was developed to determine how effective the district design was to
improve student performance as demonstrated in standardized assessment results. To
answer this question four areas of foci were developed.
1. District support for standards-based instruction and assessment.
2. District and school accountability to standards-based Curriculum.
3.Determination of “High” Student Performance.
4. Determination of degree to which the high performance is communicated to
teachers, students, and parents.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
63
Findings by Research Question
The findings from interviews, observations, and artifact analyses for the first
question, “What is the district design for using data regarding student performance
and how is that design linked to the current and emerging state context for assessing
student performance?” are described under five separate heading representing the
areas o f focus.
Student Performance. Assessed in the..Context ofCurrent and Emerging Instruments
Current. The district and the school are focusing on improving SAT 9 scores
and API ratings as defined by the state. The California English Development
Language Test (CELDT) was administered throughout the district but the data from
the tests is not being utilized at the time of this study.
Technology is used to a large extent for the state data, but systems are not
currently in-place at the district or the school level to work with other data sources
electronically.
Emerging. The emerging trends to assess with authentic or performance
assessments that are linked to the state standards are used. Authentic assessments
such as writing prompts in language arts that are linked to the state standards are
used to guide instruction and make curricular decisions. The district and school use
the California Standards in Language Arts, Math, Science, Social Science, Visual
and Performing Arts, and English Language Development to improve student
performance. The district has developed research-based, developmentally
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
64
appropriate baseline and benchmark assessments that are administered three to
four times a year. According to the assistant superintendent, “There is a greater
reliance on standards-based awareness and not much reliance on SAT 9.”
Bandaras Heights City School District uses criterion-referenced tests, as
indicated in the Essential Agreements in each of the grade levels. The Essential
Agreements are provided to each school to be used as on-going assessments. Each
grade has specified time allocation by subject for baseline assessments and
benchmark assessments. These criterion-referenced tests provide the district, the
school, and the teachers with the levels of proficiency for each student. In addition,
teachers are encouraged to assess more frequently to further determine the
instructional needs of their students. Instructional pacing, program acceleration, and
individual interventions are important decisions that arise from the analyses of the
criterion-referenced tests. Figure 1 summarizes the Language Arts Essential
Agreements by grade-level.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
65
Figure 1
District-Wide Language Arts Essential Agreements and Assessment Timeline
Grade Baseline
Assessments
Timeline Benchmark Assessments Timeline
Kindergarten Administer to all students:
Letter/sound identification,
concepts about print,
optional writing prompt
Administer to all students
Running Record,
Comprehension
Aug.-Sept.
November
Administer individually to
all students:
Letter /sound
identification, running
record/comprehension.
Administered in small
groups: writing prompts
February
and May
1st Review Kindergarten student
assessment Portfolio for:
Letter/sound identification
Writing Prompt “Tell me
something about yourself’
Optional: DRA Running
Record
Administer individually to
all students:
Running Record
Administer in small
groups
Writing Prompts
November,
February,
May
November,
February,
May
2nd Administered to all students
Running Record and
Comprehension Response,
Writing Prompt.
Between
September
and
October 15
Administered individually
to all students:
Running Record &
Comprehension
CORE One Minute
Fluency Test
Administered
independently or in small
groups to all students:
Writing Prompts
November,
February,
May
November,
May
November,
February,
May
3 rd Administered to all
students:
DRA beginning with
Level 24 to determine
appropriate level
Baseline Writing
Assessment provided by
teacher
Between
September
and
October
15
Administered
individually to all
students:
Running record,
Independent Reading,
Comprehension and
Response for Reading
Benchmark 1,2,3,
CORE
One minute fluency
test.
Administered to whole
class:
November,
February,
May
Writing Benchmark
1,2,3, District writing
Prompt.
4th
Reading-Running Record
&comprehension tests
September Administer CORE one
minute fluency to all
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
66
Figure 1 (con’t)
from Spring Plus
Placement, Writing
Prompt (“One Summer
Experience”)
students individually
Administer Reading
Comprehension Test
Administer Writing
Prompt to whole class
November,
February,
May
gth
Students read orally, “Do
Animals Think?” for
fluency
Administer Scholastic
Placement test for
Comprehension
Administer Writing
Prompt, (“All About Me”)
Administer Independent
reading
Comprehension,
Writing Prompts
CORE One Minute
Fluency
November,
February,
May*
November,
May*
*May
assess
ments go
into cum
folder
6 -8 th Linguistic Class: Pre-
Test, CORE Spelling
Inventory/San Diego
Quick, Fluency Test
(Placement in Language!)
ALL: Initial Writing
assignment (teacher
determined)
SAT 9 results from
previous year
Reading Performance
Assessment
(Informational Text)
and Connected Writing
McDougal-Little
Reading Selection Test
and Connected Writing,
On the Spot Writing,
Language!
Assessments, Writing
Portfolios
Baseline assessments are expected to be administered in the early Fall of each
year to every student. Beginning in kindergarten, students are administered
letter/sound identification and concepts about print. An optional writing prompt has
been included. Running Records are administered in November to assess specific
decoding and fluency skills.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
67
Benchmark assessments and Timelines also are included in the Essential
Agreements. These are administered to all students in kindergarten twice a year and
in all other grades three times a year. In kindergarten, the Benchmark Assessments
are administered in February and May to determine student skills In letter/sound
identification and comprehension. Writing prompts also given at this time.
The District Wide Writing Mechanics and Assessment Timelines are
summarized in Figure 2 and show the delineation of skills assessed.
Figure 2
District-Wide Writing Mechanics and Assessment Timeline
Grade level Mechanics Assessment Timeline
Kindergarten 1. Focus/organization
2. Content materials specific to
topic
3. Written conventions and
Penmanship
February and May
1st 1. Organization/Focus
2. Content
3. Written conventions and
Penmanship
November - Benchmark
#1
February - Benchmark #2
2n d
Grade
4 Organization/Focus
5 Written Conventions and
Penmanship
Mechanics
November-Benchmark #1
Assessment Timeline
Assessments
□ Spelling
□ Penmanship
□ Capitalization
□ Sentence
□ Punctuation
□ Sequence
February -Benchmark #2
□ Narrative-based on
their experience
May-Benchmark # 3
Friendly Letter
3rd 1. Organization/Focus
2. Writing Conventions and
Penmanship
Between Sept. and Oct. 15
□ Baseline Assessments
November-Benchmark # 1
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
68
Figure 2 (con’t)
□ One paragraph
Description
FebraaryBenchmark#2
□ Personal Narrative
4th 1. Organization/Focus
2. Writing Conventions and
Penmanship
A
5th 1. Organization/Focus
2. Writing Conventions
November-Benchmark #1
□ Persuasive Writing
February-Benchmark #2
□ Response to Literature
May-Benchmark' #3
Expository
6-7-8* Expository (Content)
1. Writing Application-explain
main idea with supporting
details
2. Writing Strategies/Focus
(Organization)
3. Written Conventions
Response to Literature
1. Writing
Applications/Response to
Literature
2. Writing Strategies/Focus
(Organization)
3. Written Conventions
Persuasive Essay
Baseline Assessments
Linguistic class students:
□ Pre-Test, CORE
Spelling Inventory/San
Diego Quick, Fluency
Test (Placement in
Language!)
All Students:
Initial writing assignment
(teacher determined)
SAT-9 results from
previous year
Benchmark Assessments
□ Reading Performance
Assessment
(Informational Text)
and Connected Writing
□ McDougal-Little
Reading Selection Test
and Connected Writing
□ On the Spot Writing
□ Language! •
Assessments
□ Writing Portfolios
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
69
The teaching of the mechanics of writing is intended to begin in
kindergarten and continues into 8th grade. Focus, organization, content, written
conventions, and penmanship are assessed through grade four. Writing conventions
begin in grade four and continue through middle school. The assessments used are
linked to the curriculum and are scored using a rubric.
Although some of the schools in the district have participated in the NAEP,
few comparisons are made between the performance of students in this district and
the performance of students nationally.
Overview of the Elements of District Design of Data Use to Improve Student
Performance
The Bandaras Heights design of data use to improve student performance
begins with the district’s Belief Statement:
1. All students have the right to reach their full achievement potential.
2. Education is the shared responsibility of the individual, family, school and
community.
3. All individuals bring a wealth of personal and cultural experiences that deserve
to be repeated and possess inherent talent and abilities which merit recognition.
4. All individuals are entitled to feel safe and secure.
5. Decisions and practices should be based on honesty, fairness, and integrity
6. A spirit of teamwork enhances the educational process.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
70
7. Learning is an essential life-long process which enriches the quality of life.
8. Education should be an enjoyable journey.
The design elements are based on Four Key Questions and Four Key
Assumptions: The four Key Questions are:
1. Is there a relationship between what occurs in the classroom and student
learning?
2. Is there a relationship between quality of program and student achievement?
3. How do we know it?
4. Where is the evidence?
The four Key Assumptions are:
1. Student Learning is related to program quality.
2. Variables unrelated to program quality also can affect student learning.
3. Effective use of information can improve program quality control for other
variables.
4. Improvement does not equal remediation.
Types of Data. Assessment, and Timelines. Bandaras Heights City School
District began its development of the design in 1997 and was based on the legislation
SB IX. The data at that time included SAT 9 test scores only. Each year, additional
requirements are added via legislation and are included in the design. Non-legislative
archival data such as attendance, demographic characteristics, gender, parental
education level, referral for Student Study Teams and special education also are
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
71
collected. The evolving nature of the federal and state legislation causes the
district to re-focus and re-shape the types of assessment data it collects and the
timelines for the assessment implementation. STAR achievement data are collected
and compared to other academic achievement data.
Multiple assessment data are collected in reading, writing, and math. Some
of these assessments are associated with publishers and some have been created by
the school district, such as the benchmark writing assessments. The district has
determined that it does not want to make tests, as they are a challenge to make.
Beginning in Kindergarten, all students are to be assessed in Language Arts
using multiple assessments throughout the school year. Benchmark assessments are
to be administered individually to all K-3 students. These assessments include
Running Record, Comprehension Responses, and writing prompts. Benchmark
assessments are to be administered to the whole class and individually in grades four
through eight. These assessments include Running Records, Spring Plus Placement,
oral reading fluency, comprehension tests and writing prompts. The assessments are
developmentally aligned to the grade levels and to the curriculum.
The schools also collect and assess data for opportunities to learn; the
conditions that provide and ensure that each student has the services and provisions
to achieve standards. Data are collected on program quality. The program quality
data are gathered through the Analysis of Process developed by J.K. Associates. This
process is an instrument to determine perceptual data of teachers about the materials,
activities, and programs that directly affect the quality of instruction.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
72
Methods Used to Analyze and Disseminate Data. A District Assessment
Team was created in 1997, a year after the new superintendent was appointed and a
new assistant superintendent of instruction was hired. The newly appointed leaders
understood that a major new focus in education was emerging; the standards-based
accountability movement. Their insight and leadership create a District Assessment
Team.
The purpose of the Team was to train two to four identified school site
classroom leaders and each site principal in the strategies of data collection,
analyses, and instructional improvement. Teams were created to enable them to take
back expertise to the schools to ensure every teacher’s ability to understand and use
data.
The team is the vehicle for the information to get into the hands o f the
classroom teachers. The Assessment Team meets at least four times a year for
training and data study. Similarities and differences between STAR results and other
data are disaggregated and analyzed. Patterns and trends in the data highlight student
needs. The student needs are examined to glean insight into instructional
improvement. Strategies and action plans are then created to address the identified
instructional improvement. Additionally, the team works with the benchmark data
during the school year and makes comparisons of single year district and school
cohort-matched cases.
The local assessments and classroom level cohorts are analyzed at grad-level
meetings at least bi-monthly. Banked school days are used to analysis the student
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
73
data. Informal teacher-to-teacher conferences, and individual teacher conferences
with the principal all are opportunities to analyze student work and assessment data.
The results of the data analysis affect how students are grouped (flexible grouping)
and how interventions are implemented.
Rationale for Method of Analysis. The rationale for the district plan of
analysis has been guided by the reputable research on Performance Assessment and
The Backward Design Process by Grant Wiggins and Jay Tiege, the criteria
referenced early literacy research; TIMMS reports by James Stigler; and the
Concerns Based Adoption Model (CBAM) by Gene Hall and Shirley Hord. The final
elements of the plan were developed in collaboration with J.K. and Associates.
Using Data to Improve Student Performance. The district’s design to use data
to improve student performance utilizes five steps:
1. Structuring the analysis/planning activity; creating an environment in which
staff knows and believes this is important work
2. Summarizing and analyzing the data; establishing student needs.
3. Attributing cause (Pollutants):
a. Physical environment during testing
b. Attitudes of teachers and students toward the testing program
c. Test taking skills
d. Alignment of test with curriculum
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
74
4. Quality of instructional Program using Analysis of Process to determine the
materials, activities and programs that are deterrents and supports to student
achievement, and
5. Creating the improvement plan
Training/Professional Development. The district design is focused on
improving student achievement through valid measures that include private and
county consultants providing professional development. Professional development
and one-to-one conferences with the principal are utilized to further train and
sharpen teachers data use abilities. Building teacher expertise is the most significant
need according to the assistant superintendent.
The stated purpose of the Assessment Team training is to build
knowledgeable, purposeful site support and leadership for the effective use of
information to improve student learning. The leadership team is responsible for
taking the skills and knowledge from their trainings back to the school sites to build
the capacity of the teaching staff.
The areas of identified needs provide the purpose for additional training.
More specifically, the areas of need have been in Standards-Based instruction with
an emphasis in writing, support for current focus in reading intervention for grades
4-8 (Language! Program), focus on grade-level articulation within the schools to
make instructional decisions, and English Language Development.
Influences That Affect Policy. There are six outside and six inside influences
that affect the policy and strategy design. The six outside influences are: ranking of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
75
schools, the political climate, a highly educated board that does not use
ultimatums to increase student test scores, a positive community, promotion and
retention, and the unintended resource issues of class-size reduction. The four inside
influences that affect policy and strategy design are: employee negotiations, the data
comparisons that cause the district to be dissatisfied with the results, the expertise
and experience of the teachers, and the competing needs of students within the
community and the additional expectations to address the needs of the whole child,
the expectations of special interests groups within the district such as GATE, fine
arts, and special education.
District Decisions and Rulings that Support Use of District Design
Board and Stakeholder Involvement. District decisions and rulings that
support use of district design include a variety of stakeholders in its decision-making
process to improve student achievement. The board is highly responsible and there is
a high degree of trust, stated the assistant superintendent. The board has supported
the district’s leadership in guiding the district’s design for data utilization. The
stakeholders involved with influencing the board are: the legislation, and the
teacher’s union. The board is informed of the leadership decisions through written
communication and through the regulations of the formal board meetings.
Additionally, a district-wide Community Advisory Committee, Curriculum
Committee, Principals’ Committee, and school site based group committees are
informed of the proposed changes. Due to the high stakes nature of assessment
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
76
driven accountability, “there is no room for creativity, there must be consistency,”
stated the assistant superintendent.
Intended District Results of The Design to Improve Student Performance
District. At this level, the intended result of the district design to improve
student performance is singular, “to increase the percentage of students achieving,”
stated the assistant superintendent and the principal. Using data from state and local
assessments, the district design is to increase the percentage of students scoring at or
above the state standard on grade-level tests. The Essential Agreements are an
integral component of the district’s design to improve student achievement.
Analyses of student work and research led the district to establish high
performance goals in writing for all grades. Writing was determined to be the area of
focus for student achievement because the process of writing includes essential skills
that are applicable across the content areas. The interviews with the assistant
superintendent and the principal indicated that research shows a link between the
process o f writing and the increase in student performance in reading and
comprehension.
School. At this level, the intended result of the district design to improve the
percentage of students achieving is divided into two steps. The first step is the
selection of standards from among those the students need to know. The second step
is to design assessments through which students will have an opportunity to
demonstrate knowledge of the selected standards. The school uses multiple data
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
77
sources to meet this intended result. Instructional data are used directly to drive all
curricular, intervention, and budgetary decisions at the site.
There is on-going effort to improve the principal’s and the teachers’ ability to
use data to increase student performance through a variety of methods. For example,
once a year, the J.K. and Associates Analysis of Process is used to determine newly
revealed supports and deterrents to student learning and to gauge the effectiveness
of the instructional programs. Another example is the administration of the
assessments embedded in the Essential Agreements at the prescribed times.
Textbook chapter and unit tests also are given to students. The teachers at Del Sol
Elementary School meet informally to discuss student data and make adjustments to
instruction based on the results of the assessments.
Parents are informed of student learning performance and how they can assist
their students’ improvement through standards-based report cards, and timely parent
conferences. Additionally, special nights are provided during the year for families to
introduce them to the curriculum. Such special nights include activities in math,
writing and reading.
Classroom. Although grade-level meetings, according to the teachers, are the
most valued method for analyzing student performance data, the intended result
includes three standards-based practices within the classroom. The three practices the
teachers are expected to following are:
1. Decide what learning opportunities students will need for them to learn the
standards.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
78
2. Plan instructional opportunities to assure that each student has adequate
opportunities to leam.
3. Use data from assessment to give feedback, to re-teach or move to the next level.
The classroom teachers analyze the data to drive decisions including
interventions within the instructional day although their ability varies from
classroom to classroom.
A four-point rubric: below basic, basic, proficient, and advanced has been
established to guide conversations and teaching strategies leading to improved
student achievement. Specified mechanics of writing are analyzed by teachers
within grade and cross grade-level meetings.
Evidence from interviews, artifact analyses, and observations reveal that the
teachers ask themselves the Four Questions and examine the Four Assumptions from
the district design. Additionally, learning time is given to students to provide
intensive supports for those who are having difficulty.
There is evidence within the classrooms and within the teacher grade books
that a high priority is placed on improving student performance. Working samples of
rubrics show evidence that teachers are engaged in using them and that they are
based on the California Language Arts Standards, particularly in the area of writing.
The standards-based curriculum used in the classroom is driven by the choices
provided by State Department of Education. Student work samples are available in
student portfolios, on bulletin boards and in folders to take home.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
79
Parents are provided with opportunities through the year to participate in
their children’s school experiences. The district is supportive of a strong relationship
between the school and home and A Del Sol Elementary School Student-Parent
Teacher Compact is provided for every student and his/her parents. The compact has
two to three agreements for the student, the parent, and the teacher. The agreements
are for the students to:
1. Come to school prepared to learn and with assigned homework completed.
2. Be responsible at school for their own behavior.
3. Deliver all school and classroom information to their parent(s) or guardian.
The agreements are for the parents to:
1. See to it that their child is at school and on time.
2. Check daily homework assignments/contracts, especially daily reading
requirements.
The teachers’ agreements are to:
1. Communicate with parents in a timely manner so student academic needs can be
addressed.
2. Provide an environment that supports student learning and self respect.
The report cards are grade-level specific and include a rubric for achievement.
The descriptive and numeric rubric levels are:
Advanced (4), Performance exceeds standard;
Proficient (3), Performance meets standard;
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
80
Basic (2), Performance approaches standard; and
Below Basic (1), Performance does not meet standards.
The report cards list the Language Arts and Mathematics Standards.
Additional subjects in the elementary grades are marked for effort only. Data on
Running Records and Alternative Assessments also are indicated on the report cards.
Data Use Policy and Strategy Funding (District and School Site)
Budget Control. Fiscal and funding issues cause competition between
materials, staff development, intervention, and time for people to meet. “Reform is
resource hungry, the greatest resource is time,” stated the assistant superintendent.
The data use implementation is funded through the Bandaras Heights general funds.
General and categorical funds are allocated to the schools from the district office.
The school then determines how it will use the funds to implement the district design
for increasing student achievement.
Funding Decisions at the School Site. The accountability for the
implementation of the district plan is at the school level and, therefore, the school is
accountable for all budgeting and funding of the programs and strategies. Each
school has input into the budgeting and funding through the School Site Council.
Results from the Teacher Questionnaire indicate that the majority o f teachers are
unaware of how the implementation of data use is funded. However, the six teachers
interviewed indicated their awareness that funding is based on student needs. One
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
81
teacher explained that the amount of books or Apple Smarts programs depend on
what the data indicate.
The theory, policy, and structure of the data design are fundamental to
instruction. The district is aware that there is a gap between funding and curriculum
*
decisions. The school understands this discrepancy between what funds are available
and what can be purchased. The less money that is available for the school, the fewer
choices the teachers feel they have in making curricular and instructional decisions.
The funding drives decisions; the more funding there is, the more programs there are
available for the students.
Additional Funding Needs. Interviews with the assistant superintendent and
principal reveal that additional funding could provide support to teachers that will
affect the quality of their programming. Additional funding also would allow the
delivery of differentiated instruction. All participants interviewed indicated that
training, coaching, and modeling could be enhanced as would more opportunities for
site based staff development with additional funds.
Teacher interviews revealed additional funding is needed across the board to
expand the curriculum programs and expand the 20:1 class-size reduction program.
Additional funding is needed to support the quality of programming that has been
identified through the Analysis of Process.
The district design for data use to increase student achievement is detailed in
its development, its elements, and intended use. Further elaboration of the district
design is discussed in the findings for the second question: “To what extent has the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
82
district design actually been implemented at the district, school, and individual
teacher level?” The findings are based on interviews with the assistant
superintendent, the principal, and the six teachers. Results from the Stages of
Concern Questionnaire and the Teacher Questionnaire also were analyzed. The
findings are described under three separate headings or areas of focus.
Degree of Design Implementation in the Current and Emerging Contexts
Knowledge of Policies. Strategies and Timelines. The interviews and results of
the questionnaires indicate that the school has knowledge of the district design
strategies. O f the 25 teachers who completed the Teacher Questionnaire, the
majority indicated they Agree Strongly or Agree Somewhat that they are aware of
the design for using data.
In 1999, the school began to implement the Analysis of Process, and reviews
yearly. The teachers are aware of the timeline in the district design. Each teacher has
a copy of the state assessment timelines. The additional timelines have been added to
the design to determine student performance on district math, language arts, and
English Language Development benchmark assessments that are administered
quarterly.
Factors for Implementation. Increasing student achievement is the primary
factor for implementation of the district design for data use. The assistant
superintendent stated that the factors she considered for the implementation were the
knowledge base of the leadership teachers and the principals. The principals are
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
83
given the job of implementing the design at the school. Del Sol Elementary
School’s principal embraced the design because of her belief in its structure and her
continual refinement of data analysis. Three factors considered by the teachers for
implementation are the availability of:
1. Assessments
2. Instructional material
3. Training in data utilization
There have been some discomforts in the implementation of the design
because of its evolving nature. The assistant superintendent stated that although there
are discomforts, the leadership of the Assessment Team, principals, and some
teachers are motivated to work through the discomforts.
The interviews revealed several roadblocks to the implementation of the
design for data use. Anxiety, fear of being judged, lack of comfort with multiple
measures, and books not delivered on time by the vendors were identified as the
roadblocks.
Current and Emerging Data Collection Practices. The use of the SAT 9
assessment data has been a practice since 1998. Using SAT 9, California Standards
test scores, and benchmark assessments the teachers, under the guidance of the
principal analyze student work to determine grade-level instructional strategies and
processes to improve student writing.
O f the 25 teachers who completed the Teacher Questionnaire, the majority
Strongly Agree that they collect and use data, including test scores on a weekly basis
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
84
to monitor student progress and guide instruction. The tests are in the textbooks
and others are prescribed in the Essential Agreements for each subject. Benchmark
assessments are given twice a year in kindergarten and first grade and three times a
year in third through eighth grades.
The school has taken the initiative to create a writing process that meets the
needs o f the students. Data from student work samples on the mechanics of writing
to guide instruction is an emerging data practice being implemented.
The teachers Strongly Agree that the principal collects data. One teacher
referred to the principal as the “teachers teacher because of her involvement in
assessment and instructional decisions.”
Accountability for Data Use at District. School and Individual Level
Monitoring and Storage of Data. The California assessment data are
currently stored with the assistant superintendent. The school completes reports of
data implementation for the district database. Plans are in place to store the data
electronically with the Director of Informational Services. The new database with
have more capacity. Data Works, an outsource company, also stores the state
assessments for the district. The benchmark assessment data are stored with the
assessment team members and with the teachers.
The majority of teachers Strongly Agree and Agree Somewhat that they
monitor student progress. Each teacher creates assessment data files that best meets
her individual needs. Assessment portfolios on each student are kept in classrooms.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
85
One teacher keeps daily data on mis-read words from the reading programs. The
teachers Strongly Agree and Agree Somewhat that the administrators monitor their
utilization of new data practices.
The majority of teachers Strongly Agree and Agree Somewhat that they
monitor student progress. Each teacher creates assessment data files that best meet
their individual needs. Assessment portfolios on each student are kept in the
teacher’s classrooms. One teacher keeps data daily on students missed words for the
reading programs. The teachers Strongly Agree and Agree Somewhat that the
administrators monitor their utilization of new data practices.
Responsibility for the Data. Instructional Practices and Student Achievement.
There are seven guidelines that are implemented to govern the responsibilities for
instructional practices and student achievement. They are:
1. Principals are responsible for incorporating the Essential Agreements into the
formal supervision process.
2. They are expected to visit classrooms as part of the informal supervision process.
3. The Superintendent and Assistant Superintendent of Instruction and the
Instructional Services staff are expected to visit each classroom in the Fall.
4. Continued classroom visits are scheduled by the Instructional Services staff.
5. Benchmark assessments are collected three times a year at the district level.
6. Teacher training days include sharing of student progress, and articulation across
grade-levels.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
86
7. The Site Assessment Teams keep staff and parent groups informed regarding
student programs.
The responsibilities of classroom teachers are to analyze student data to
improve classroom instruction, to link instructional practices to the standards-based
practices to meet student needs, and to offer intervention. The Teacher Questionnaire
indicated that all but one teacher Agree Somewhat or Agree Strongly that
intervention is more easily employed through the utilization of data. One teacher
responded that she did not know if interventions were employed through data use.
All teachers make a commitment to work together. They meet as a school
team and analysis the data from state and local assessments and classroom
assignments. One teacher stated in her interview that the work is “never individual,
it’s grade-level responsibility.” Since 2000, the teachers have been responsible for
evaluating the writing products of the students collaboratively using the four-point
scoring rubric.
The majority of teachers who completed the Teacher Questionnaire indicate
they hold students accountable for improved performance through the use of data. In
addition to the student contract, students are responsible to do their homework, pay
attention, follow the rules and take information home to parents. The standards are
posted in classrooms and students are accountable for knowing them.
The involvement and the commitments from the superintendent and the
assistant superintendent are evidenced in their visits to classrooms during the year.
The superintendent attends Open House, Back to School Night, and other events
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
87
during the year when data are discussed and shared with the community.
According to reports made during teacher interviews, the assistant superintendent
visits classrooms frequently to observe instruction.
Accountability for data use is written into the principal’s performance
evaluation and teachers are held accountable for standards-based instruction through
their performance evaluations. Review of a weekly school bulletin indicated
teachers “were expected to discuss how students are progressing toward
benchmarks” for the final teacher evaluation. The majority of teachers Agree
Strongly that the state holds the district, the district holds the school, and the school
holds them accountable for data utilization. The majority Agree Strongly that their
salary and promotion within the school are dependent upon utilization of data
practices. Interviews with the teachers revealed that they felt accountable regardless
of their use of data, however, their individual commitment to accountability has been
enhanced by data practices.
Data Disaggregation and Use. The structure of the district’s design for data
collection and use is fundamental to instructional practices. The emphasis on the data
keeps the focus on student achievement and helps the schools set explicit guidelines
for instruction. In order to accommodate the structure of the design, five district
wide grade-level meetings are held during the year, in addition to the weekly
meetings that are held at the school.
The state assessments are disaggregated by Data Works and are presented in
binders to each school. The data are disaggregated by gender, socio-economic
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
88
status, grade level, classroom, language proficiency, parent education levels, and
special needs. Matched scores are disaggregated to reveal trends and patterns in
student achievement over time.
Benchmark assessments are disaggregated at the school to reveal
comparisons between baseline and baseline assessments individually, by class, and
by grade.
The majority of teachers Agree Strongly that they use data to compare the
past and present performance of an individual student and they Agree Somewhat that
they compare the students across the school in the same grade. They discuss new
data practices with each other regardless of level of experience. Seven of the 25
teachers Disagree Somewhat with the practices of grade-level comparisons.
The principal and the teachers work to set instructional goals and make
decisions about how to deliver lessons and provide interventions after they have
analyzed student work.
Improving Student Achievement through Implementation of Data Use.
The assistant superintendent indicated in her interview that she would be interested
to know if the school can link the improved student achievement to the
implementation o f data use. Teacher interviews revealed several case studies of how
student achievement improved as a result of using data. During her interview, one
teacher told a story that in 2000 there was a marked discrepancy in the boy and girl
performances in reading scores on the SAT 9 and benchmark assessments. A survey
was created to determine what books and authors the boys like to read. After review
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
89
of the surveys, the School Improvement Program (SIP) purchased the desired
books for the school library. The boys were encouraged to check out and read the
books, which they did. Data on both benchmark assessments and SAT 9 in 2001
showed marked improvements in the reading scores o f the boys.
Another teacher discussed that the math scores increased as a results of using
data. Another teacher indicated that the increase in the API is evidence that the use
of data increases student achievement. This was the first year that writing was
assessed using standardized benchmarks. Evidence shows that the instructional
process being developed by the staff is enhancing the writing skills of students in all
grades. The results of the questionnaire indicates that the majority of teachers Agree
Somewhat and Agree Strongly that they have seen student achievement visibly
improve when data are used as a benchmark for students to reach. Four teachers did
not know if using benchmarks data made a difference. The majority also indicated
that they Agree Somewhat or Agree Strongly that student achievement visibly
improves when they use data to modify their teaching. Three teachers either did not
know, or Disagree Strongly, or Disagree Somewhat with seeing visible improvement
in student achievement when using data to modify teaching.
Another example of how disaggregated data was used for math instruction
was given during an interview with a teacher. It was discovered that students’ scores
in math computation were low. Observation of student performance and benchmark
assessments informed the staff that the students were not able to calculate quickly, a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
90
skill required for SAT 9. Five-minute time tests on computational skills were
created for each grade level. Teachers administered the tests daily. The scores on
both benchmark assessments and the 2001 SAT 9 improved.
Effective Elements of Data Use. Two elements of data use that are the
most effective for teachers in bringing about student achievement gains are the (a)
standardized assessment and (b) the analysis of district, school, and classroom data.
To illustrate their effectiveness, it was discovered that the high achieving students
were not performing as well as predicted. Based on the data analysis, differentiated
instruction was employed to enhance and enrich instruction. The SAT 9 and
benchmark assessment data for the high achieving students’ scores rose.
Teacher responses ranged from Disagree Strongly to Agree Strongly on the
Teacher Questionnaire asking if student motivation increases through the
dissemination of data to parents. Interviews with teachers indicated that when the
parents were given the data about their child’s performance in understandable terms,
they were more able to support the school and support their child’s efforts.
Tracking Student Growth. There are no mandates on how the student
growth should be tracked and a variety of methods were observed. Such methods
include personal record keeping with simple to complex charts and graphs, using
portfolios, comparing growth and making changes, using benchmarks and daily
charts, and the use of journals. The Teacher Questionnaire results indicate that 22 of
the 25 teachers who responded use data to compare the past and present performance
of an individual student. Nineteen of the teachers use data to compare students
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
91
within their class. Figure 3 represents a schematic representation of the Bandaras
Heights Mapping of Data Flow.
Figure 3
Mapping of Data Flow
4 Key Questions
1. Is there a relationship between what occurs in the classroom and student learning?
2.1s there a relationship between quality of program and student achievement?
3.How do we know it?
4.Where is the evidence?
California Assessment Data (IX) <==> District Assessment Data (4x)
API, SAT 9, Standards Based Assessments, CELDT Essential Agreements
■ U Writing Benchmarks
Data Works (analysis of above)
u
District Assessment Team
Purpose: To build knowledgeable, purposeful site support and leadership for effective use of
information to improve student learning
Professional Development aligned to assessments and standards
U
School Assessment Teams
(Principal, 3 teachers)
Parents < = > - H o Students
District Web-site Grade level — Teams Individual scores
Establish cause, attitudes, curriculum alignment, Quality of Program
U
Grade/cross grade-level informal articulation
Develop/share materials lessons, interventions, instructional strategies
Interventions < = > ft f!
Parents < = > Classroom
Chapter tests, daily/weekly assessments=>instructional decisions
U f t u
Student
Student contract
h ft
Improved Student Achievement
4 Key Assumptions
1. Student Learning is related to program quality
2. Variables unrelated to program quality also can affect student learning
3. Effective use of information can improve program quality control for other variables
4. Improvement does not equal remediation
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
92
The flow begins with the California state and district driven assessments.
The state assessments are analyzed by Data Works, an independent data analysis
company. Once disaggregated, the data are shared and explained to the District
Assessment Team that then takes the information to the school for further analyses
and discussion. Further assessments are given and analyzed at Del Sol.
Parents and students are informed of the data results. Grade/cross grade-level
articulation occurs weekly to assist teachers in understanding the data and making
instructional decisions to improve student achievement.
In addition to the findings from the interviews, observations, Teacher
Questionnaire, and artifact analyses, a Stages of Concern Questionnaire was
administered. This questionnaire was designed to find the pattern of teachers’
concern about the district’s design to use student data to improve student
performance The findings were gathered and analyzed and are represented in the
following section. Table 5 represents the highest average score across all six stages
of concern for each teacher, the strongest Stage of Concern for each teacher.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
93
Table 5
Highest Average Scores Across all Six Stages of Concern
Teacher
Stage 0
Awareness
Stage 1
Informational
Stage
2
Personal
Stage 3
Management
Stage 4
Consequences
Stage 5
Collaboration
Stage 6
Refocusing
1 3.8
2 3.2
3 4.0
4 4.8
5 4.2
6 4.8
7 5.6
8 4.6
9 6.0
10 5.2 2.4
11
12 5.4
13 5.6
14 2.6
15 5.6
16 5.0 5.0
17 4.2
18 4.6
19 5.8
20 4.0
21 5.0
22 4.0
23 4.0
24 3.6
25 2.8 2.8
The pattern reflected in Table 5 indicates that most of the teachers have their
strongest concern in Stage 2, the Personal Stage. These findings also are supported
by the results of Table 6, Stages of Concern Average Scores by Teacher.
Table 6
Stages of Concern Average Score by Teacher
Stage 0 Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 Stage 6
Awareness Information Personal M anagement Consequence Collaboration Refocusing
1 2 12 5 2 3 0
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
94
Further analysis of the Stages of Concern Questionnaire is represented in
Table 7, the highest score for stages 0 through 4.
Table 7
Highest Stages of Concern Scores for Stages 0-4
Stage 0
Awareness
Stage 1
Information
Stage 2
Personal
Stage 3
Management
Stage 4
Consequence
5.6 6.0 5.8 5.5 5.5
This data was analyzed and revealed that Stage 1, Information, is the highest
within the first four stages. However, across the stages, it appears that the scores for
each stage are relatively high.
The level of intensity of concern per teacher is represented by an average
score of 3.5 for stages 0-4 as indicated in Table 8. Analysis of this chart reveals that
five teachers have levels of concern below 3.4, a low level of intensity of concern.
Six teachers have scores in only one stage. Two of these teachers scored in the
Information Stage, three teachers scored in the Personal Stage, and two teachers
scored in the Management Stage. Five teachers scored with at least 3.5 in three
stages. Two teachers had scores in the three stages of Information, Personal, and
Management. Two teachers had scores in the three stages of Information and
Personal and Consequence. Two teachers had scores in four stages. One teacher’s
four highest scores began at Awareness and continued into Management. Another
teacher’s four highest scores began at Information and continued into Consequence.
Two teachers had scores of at least 3.5 in all six Stages.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
95
Table 8
Number of Stages of Concern with averages of at least 3.5
Teacher Awareness Information Personal Management Consequence # of
stages
1 0
2 0
3 3.8 4.0 2
4 3.8 1
5 4.2 1
6
O©
4.4 4.2 3.8 4
7 5.6 5.2 4.0 3
8 3.8 4.0 2
9 6.0 5.6 4.2 3
10 3.6 4.2 5.2 4.6 4
11 0
12 4.0 5.4 2
13 4.6 5.6 4.2 3
14 0
15 5.6 3.6 4.6 5.2 4.4 5
16 4.8 5.0 5.0 3
17 4.2 3.6 2
18 3.8 4.6 4.2 3
19 4.2 4.4 5.8 5.2 5.4 5
20 4.0 1
21 4.6 1
22 4.0 1
23 4.0 1
24 3.6 3.6 2
25 0
The findings of Bandaras Heights City School District’s data use design and
the implementation of the data use at Del Sol Elementary School have been reported.
The third question, “To what extent is the district design a good one?” was
developed to determine how effective the Bandaras Heights City School District
design was to improve student performance as demonstrated in assessment results. In
other words, the degree of adequacy of the design was measured.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
96
Evidence from interviews, questionnaires, observations, artifact analysis,
and document reviews were scored using a five-point scoring rubric to answer this
question. The findings are represented under three separate headings or areas of
focus.
District Support for Standards-based Instruction and Assessment
Intended District Impact. Evidence from interviews, questionnaires,
observations, artifact analysis and document reviews reflect that there are clear
performance goals that are aligned to high standards. There are disaggregated
standards-based assessment data. In-services are provided at least five times a year
on how to use data. The majority of teachers have attended professional development
training in the past six months related to new data practices. The district prepares the
schools for the emerging state assessments through the District-Wide Assessment
Teams.
Observed Impact. The district plan is implemented in the school. The district
is in the process of developing a plan for addressing emerging state assessments.
The intended impact has propelled Del Sol School to extend the use of assessment
into the writing process.
There are clear performance goals that are aligned to the state goals
(standards) and in writing the goals are higher. Teachers, the principal, and most
students are aware of student performance goals. The teachers have seen student
achievement visibly improve when data are used as a benchmark for students to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
97
reach. Additional days are used at Del Sol School to assist teachers to examine
student data and make instructional decisions and adjustments.
District and School Accountability to Standards-Based Curriculum
Student Performance Data. The motivation to use standards-based
assessment and instruction at the school site is evident through the principal and
teachers support of their use. The majority of teachers and the leadership team
actively support standards-based instruction and assessment. There is insufficient
information to determine the level of support from parents and students.
Collaborative grade-level and whole school teams meet weekly to monitor
and assess student work. The instructional strategies are individualized to promote
learning,
How the School is Utilizing Student Data. Student performance is assessed
weekly and collaborative grade-level meetings occur weekly. The instructional
strategies are formally individualized weekly and in some classrooms, daily.
Determination of “High” Student Performance
The degree to which high student performance is aligned to standards. The
majority of teachers, leadership team and students at Del Sol School understand high
performance rubrics, and standards-based curriculum. There is insufficient data to
determine the degree to which parents and students understand standards-based
curriculum.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
98
The degree to which high student performance is communicated to
teachers, students and parents. Report Cards are aligned to state standards in
Language Arts and Mathematics in the elementary grades and are continual works in
progress. All students and parents/guardians are provided with the Student-Parent-
Teacher Contract. The majority of students and student sub-groups are progressing
toward high-performance goals.
Discussion
The discussion of the findings for each research question is guided by the
researcher’s professional reflections on the study findings and from the perspective
of the literature on school reform. Each research question will be discussed in the
same format as the findings were reported, by area o f focus. The discussion section
will conclude with a summary.
Research Question #1, “What is the District Design for using data regarding
student performance and how is that design linked to the current and emerging state
context for assessing student performance?”
Student Performance Assessed in the Context of Current and Emerging Instruments
Current. The Bandaras Heights City School District follows the guidelines
set forth in the California STAR program which has been designed to measure the
achievement levels of all students in California. It is comprised of three elements:
(1) a nationally norm-referenced (NRT) achievement test in English, The Stanford 9,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
99
known as the SAT 9 is administered each Spring. The results from these tests
show the distribution of performance, not average performance, for all students. (2)
The California Standards Test, and (3) The Spanish-language norm referenced test.
Using the scores of the STAR, the state department calculates an Academic
Performance Index (API) for individual schools. Each of these assessments and their
results are a part of the Bandaras Heights City School District data design.
Emerging. Although examination of the SAT 9 scores in the state
demonstrate that the gap between English proficient students and English learners
has widened in some instances over the past three years, this has not been the case in
the Del Sol Elementary School. Students in all subgroups have made significant
improvements. Additionally, the Essential Agreements baseline and benchmark
assessments for math and language arts demonstrate that Bandaras Height City
School District is using criterion-referenced assessments. There is no evidence that
the district design measures student performance against international performance
standards.
With the outside assistance of Data Works, the district is capable of viewing
state student assessment results in a variety of disaggregated perspectives.
Technology is used to a large extent for the state data, but systems are not
currently in place at the district or the school level to work with other data sources
electronically. This does not seem to be interfering with the school’s ability to
examine criterion-referenced data and make instructional and programmatic
decisions based on the data.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
100
Overview of the Elements of the District Design of Data Use to Improve Student
Performance
Types of Data. Assessment and Timelines. The educational literature on
comprehensive school reform seeks to have schools create, adopt, or adapt an
educational approach that bring all students and programs into a cohesive, school-
wide educational strategy to improve student performance (Fullan, 1996; Holcomb,
1999; Reeves, 2001; Schmoker, 2001; Wiggins, 1998). The design elements are
coherent with the literature. All students are expected to be assessed for baseline data
with follow-up benchmark assessments. Coherent and clear assessment instruments
that are administered at specified times create an educational strategy for all to
follow.
The matched scores provide a powerful tool for teachers to see clearly the
patterns and trends of cohorts of students and provide meaningful interventions and
enhancements to the instructional program.
The district is well informed of the intentions to provide accountability,
feedback, and improve instruction on a broad scale to increase student learning for
sustainable success (Asp, 2000). The use of the J.K. and Associates Analysis of
Process is a measurement to determine opportunities to learn for all students. That
the district is using this process in conjunction with test scores may be an indication
of their steady improvement in student performance.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
10!
Methods used to Analyze and Disseminate Data. Efforts are in place to
use and disseminate the multiplicity of available data. The disaggregation of the
state assessment results by Data Works helps alleviate the district from this task.
There is inconsistent teacher use of the Data Works reports. The creation and
utilization of the computerized assessment instruments will assist in getting
information about student performance results to the schools more directly and
expediently.
The analysis at the school, grade, and classroom level is more meaningful
for the teachers perhaps because of the connection of the assessment to the
instruction. The district design is coherent with the literature of Carl Glickman, Edie
Holcomb, and Douglas Reeves in terms of the processes for collection and
disaggregation. The district has elected to focus on the mechanics and conventions of
writing as a measure of student achievement. The teachers at Del Sol are tackling
ways of improving students writing assessments by disaggregating the data of
student writing samples, using and refining rubrics, and continually finding solutions
to increase their instructional practices. The methods of analysis o f the data are
collaborative.
The district has established a comprehensive design for data collection. The
teachers and principal at Del Sol Elementary School have taken ownership of the
data of their students over whom they have direct influence. The systematic strategy
for using data to improve instruction is visible at the school.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
102
Rationale for Method of Analysis. Stephen Covey (1991) illustrates the
rationale of Bandaras Heights City Schools District data design model in the
following simple paraphrase. Beginning with the end in mind means starting with a
clear understanding of your destination. Knowing where you’re going means you
understand where you are now and the steps you take are always in the right
direction (Covey, 1991).
It appears from the findings that the district’s rationale for using the
Backward Design Process by Grant Wiggins is to avoid inadequate assessment and
instructional planning and to foster an engagement of inquiry for the staff and for the
students themselves. The model allows for a continuum of methods for
appropriately assessing the degree of student understanding and allows for a
comprehensive focus on student achievement. The Backward Design is a purposeful
task analysis that is seen repeatedly through the Bandaras Heights City School
District design. It also requires that teachers . .think like an assessor” (Wiggins,
1998) about evidence of learning to clarify their goals and more sharply define
teaching and learning targets to allow students to perform better and to know their
goals.
Not only is data collected on student achievement, it is also collected on the
climate and conditions of the district and the school. The human elements of
schooling are considered important contributing factors to student academic success.
The human element of education is a perspective that currently is not receiving much
attention. However, one of the major goals of humanistic education is to foster the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
103
acquisition of skills and competencies (e.g., academic, personal, interpersonal,
communicative, and economic) for living in a multi-cultural society (Dembo, 1994).
Using Data to Improve Student Performance. The design allows for the
continual monitoring of student achievement gains by having in place the four
assurances as stated by the National Center on Education and the Economy. The first
two steps in the district’s design are supported in the literature by Doug Reeves,
Richard Elmore, and Edie Holcomb. The first step, structuring activities for
analysis, must be planned and presented in an understandable, friendly manner. An
environment that fosters the importance of improving student achievement through
the study of data must be established. The second step, summarizing and analyzing
the data, and establishing student needs is a spiral process.
The teachers are clearly aligned in their beliefs that using data improves
student performance and includes not only quantitative data, but data that will ensure
the students have opportunities to learn. The school’s focus on student writing is a
compelling goal because the complex elements of writing are essential for school
success (Cunningham & Allington, 1994; Hiebert & Raphael, 1998; Raphael &
Hiebert, 1996). Many schools tackle only one or two achievement goals annually to
prevent the overload that is so clearly the enemy of improvement. Just as overload is
the enemy o f improvement, the use of data becomes the enemy o f overload (Fullan
& Hargreaves, 1996). Del Sol Elementary School tackles only a few achievement
goals annually, those focused on improving writing.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
104
Training/Professional Development. The single distinguishing
characteristic of the best professional in any field is that they consistently strive for
better results, and are always learning to become more effective. This characteristic
is evident at Del Sol Elementary School. All teachers know and understand the
importance of data in making instructional decisions to improve student
achievement. Teacher development and student development are closely linked. The
value o f teacher development and teacher collaboration must ultimately be judged by
whether these changes make teachers better for their students in ways that teachers
themselves can see.
Additionally, the staff at Del Sol Elementary School developed a school-
wide focus on writing with grade-level articulation. All teachers work together with a
single focus. Another contributing factor to the professional development at the
school is the awareness of the need for students to master strong writing abilities.
The professional development that is in place at Del Sol Elementary School is
simple, direct, and meaningful.
The training provided by the Bandaras Heights City School District is based
on current literature that advocates the proactive approach to data utilization
(Holcomb, 1999; Reeves, 2001; Wiggins, 1998). Training teachers using the
Backward Design facilitates the examination of evidence of student performances
called for by the state standards. The training also helps teachers adjust teaching and
learning to equip students to increase their performance results.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
105
One element of comprehensive school reform that the district is still
developing for use is the creation of computer technologies that will assist the
schools in accessing student information in a variety of formats for disaggregation
and analysis.
Influences that Affect Policy. The district’s design for data use is an example
of the fourth wave of reform or the grassroots public engagement wave described by
Kent Lloyd and Diane Ramsey in their book Reclaiming our Nation at Risk (1998).
It is a national standards-based reform movement that is driven by parents, teachers,
and citizens including governors, business leaders, presidents and educators. Over
the past 15 years, Standards-based reform has become a fundamental part of the
architecture of policy and governance in American education (Elmore, 2001).
Former United States President Bill Clinton’s Goals 2000 Act has influenced the
district’s design for data use. The act encouraged local community-based actions to
meet the educational needs and help more students aspire to higher standards.
The politics surrounding policies are becoming more and more visible.
Richard Elmore elaborates in his book, Building a New Structure fo r School Leaders,
that standards-based reform represents a fundamental shift in the relationship
between policy and institutional practice. The very foundation on which public
education was founded has shifted as a result of standards-based reform. The reform
policies of the past two decades no longer are effective in the new area of standards-
based reform. The political climate, as mentioned by the assistant superintendent, is
now a much stronger influence affecting policy. The pending Assembly Bill 2160,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
106
the California Teachers Association sponsored collective bargaining bill, would
authorize teacher unions to negotiate the curricular choices and other policies that
directly affect classroom practices. This would have a direct influence on the
employee negotiations within the district.
The high rate of economically disadvantaged children and families within the
community requires additional attention. Kati Haycock, Director of Education Trust,
states that money for urban schools serving poor and minority youth must rise to the
level of that spent on suburban middle-class schools, with higher teacher pay and
smaller classes. Low income children also need more social support, along with
better health care, stable housing, and good pre-schools (New York Times,
4/ 11/02).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
107
District Decisions and Rulings that Support the Use of District Design
Board and Stakeholder Involvement. According to Carl Glickman,
The only legitimate role of school boards and districts is to ensure that students are
being educated to become productive citizens of the larger society. The school
board's role is one of setting broad policies and providing resources that support
schools’ ability to accomplish that goal (Glickman, 2002, p. 112).
The Bandaras Heights City School District engaged a number o f significant
stakeholders who directly influenced the policy and strategy of the design.
Trust, being a major component of the relationship between the district leadership,
the board and stakeholders has played a significant role in the district development of
the design.
When parties consider each other’s facts to be accurate and information to be
clearly and thoroughly presented with sound reasoning, agreements can often be
reached expeditiously and in a harmonious fashion (Loofbourrow, 1996). The
emerging trend of standards-based reform and policy reaches, at least in theory,
directly into the instructional core of schools, making what actually gets taught a
matter of public policy and open political discourse. The district has been fortunate
to have a board and other stakeholders that have supported the policies and strategies
for data utilization.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
108
Intended Results of Design to Improve Student Performance
District. The literature on change is replete with articles that stress the
importance of an organization having a statement or document that articulates its
“Passion” or the core values and purposes that guide it (Holcomb, 1999, p. 2). The
intended result of the Bandaras Heights City School Districts design, which is to
increase the percentage of students achieving, is well articulated. The literature
(Glickman et ah, 2002) supports the district’s intention of using student learning
standards as a foundation, enhancing teachers’ capacity to provide effective
instruction toward the standards through professional development, actively
engaging parents, communities, and other stakeholders in the process and providing
meaningful, on-going assessment of student achievement.
School. The design’s intended result enhances the teachers skills by using
the data to make the instructional decisions required to provide students the optimum
resources for learning. The annual Analysis of Process by J.K. and Associates has
been in place for several years and allows the teachers and principal an opportunity
to reflect on what program elements are supports or deterrents to their efforts.
Teachers are supported and encouraged to meet as a school and in grade-level
groups to analyze data. This creates an atmosphere of collaboration and teamwork.
The on-going, persistent involvement in structured, goal-oriented teamwork allows
teachers to see that they have something to learn from their colleagues (Reeves,
2001). Although all teachers use data, they do it together as a team. The most
important effect of teacher collaboration is its impact on the uncertainty of the job
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
109
which, when faced alone, can undermine a teacher’s sense of confidence (Fullan
et al., 1992). In effective schools, collaboration is linked with norms and with
opportunities for continuous improvement and learning (Reeves, 2001; Rosenholt,
1989; Schmoker, 1996).
The schools intended result to improve student achievement is supported by
the literature on comprehensive school reform which seeks to have schools create,
adopt, or adapt an educational approach that brings students and programs into a
cohesive, school-wide educational strategy (NCEE).
Classroom. The standards-based practices ensure that there is a curriculum
that sets high standards for all students and doesn’t “water down” material for those
in categorical programs, but makes use of appropriate instructional strategies that
provide extra help for students who must struggle to master the regular curriculum.
The consistent use of the rubric guides teachers to focus on the student work
samples to refine instruction. The intended result of the district design allows the
teachers to self- reflect. Using the Essential Questions and Assumptions, teachers
can address the grouping of students for different subjects, the scheduling of
instruction, planning time, pupil support, home outreach strategies, and professional
development. The structures of the design give the teachers the support to look at
data and opportunities to apply the results to their teaching.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
110
Data Use Policy and Strategy Funding (District. School Site)
Budget Control. Funding of the data use implementation at the district and
school level is allocated from general funds and is monitored by the district.
Funding Decisions at the School Site. Site based decision-making is an
essential element to the districts overall structure and budgetary decisions are
deferred to the individual school sites. Decisions are based on the student data that is
analyzed.
Additional Funding Needs. Odden and Picus (2000) report that the general
revenues support the regular or base education programs and is an issue of concern
across districts. There was little discussion at the Del Sol School about the specifics
of the allocation of funds. Although this study did not delve deeply into the funding
issues for the district design and its implementation, further examination would be a
worthy venture. A connection between schooling and long-term outcomes could be a
further research topic. As the connections are developed, analysis of the outcomes
and their links to the distribution of school fiscal resources could be included (Odden
et al., 1999).
Research Question #2, “To What extent has the district design actually been
implemented at the district, school and individual teacher level?
Degree of Design Implementation
Knowledge of Policies. Strategies and Timelines. The teachers at Del Sol
Elementary School implement the Bandaras Design by collecting the evidence of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Ill
student performance from a range of assessments required by the state and from
the Essential Agreements. The expectations are clear that teachers will know and
adhere to the design.
Factors for Implementation. The district design has been embraced by the
principal and a majority of the staff at Del Sol Elementary School. Although
discomforts, anxiety, and fear of judgment are factors in the implementation, the
staff is fully aware of the reality of the evolving purposes and needs for data analysis
to improve student achievement. The reality is that passing exams required for
student advancement and graduation is the most important factor. Classrooms and
schools are held accountable and receive public rewards and sanctions based on how
well students do in absolute terms of mastery and in comparison with other
classrooms, schools, and districts (Glickman, 2002).
Current and Emerging Data Collection Practices. The assessments are
collected and analyzed weekly and includes teacher observations of students, the
school/classroom environments, textbook quizzes, state assessments, and district and
school level assessments. The scope of the assessments vary from short-term to long
range. These assessment practices are supported in the literature by Wiggin and
Tiege (1998), Reeves (2001) and Holcomb (1999).
Implementation of Current and Emerging Data Practices. The Bandaras City
School District and Del Sol Elementary School are focused on implementing the
current data practices as discussed in the literature. The school has implemented all
the data practices mandated through the STAR program. In addition, the school has
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
112
worked with the district and on its own to develop and refine assessments in
writing for all elementary grade levels.
Accountability for Data Use at the District. School, and Individual Level
Monitoring and Storage of Data. There is a vast amount of data that is
available to the staff at Bandaras Heights City School District. Data Works has
provided the district with disaggregated data through the dissemination of binders.
Additional data such as the baseline and benchmark assessments are formatted for
easy disaggregation and are stored with the Assessment Team members. The district
office is creating ways to store and monitor data electronically. Once this occurs, the
access to that data for schools and classrooms will provide opportunity to manipulate
the data and further improve instruction.
Responsibility for Data.Instructional Practices and Student Achievement
Reauthorization of federal funds and President George W. Bush’s No Child
Left Behind Legislation emphasizes the need to use assessments and instruction to
document improvements in student achievement. The district’s seriousness about
data use is well documented.
The findings demonstrate that the principal is a driving force in the use of
data, in instructional practices, and in increasing student achievement. The teachers
work well as a group with data through grade and cross grade-level meetings and
other collaborative efforts. However, the Stages of Concern results indicate that
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
113
their highest concern for data use was at the personal level. This implies that
they have concerns about their own capacities to work independently with data.
Interviews and the Teacher Questionnaire revealed that the teachers believe all levels
of the educational system, from the state to the classroom, are held accountable for
data utilization. This reinforces their concern about how the principal, the central
office, and other authorities will perceive their use of data. They Strongly Agreed on
the Teacher Questionnaire that their salary and promotion within the school is
dependent upon utilization of data practices. The findings indicate that their feelings
are valid as district policy states that teachers are evaluated on their data use
practices and are held accountable for student achievement. The perception of the
assistant superintendent, who stated that there are roadblocks to implementation,
indicates that she is aware of the high stakes nature of classroom data utilization. The
district leadership can capitalize on the teachers’ strong collaborative working
relationships to build and reinforce their individual data use capacities because
improvement inside the classroom depends on improvements outside the classroom
(Fullan et al., 1996) and, as Carl Glickman (2002) states, there is an overwhelming
need for greater involvement of teachers in educational reform, outside as well as
inside their own classrooms.
The students’ responsibility for their behavior is an integral part of growth
and development; they dramatically affect the academic climate of a school (Blase,
1986; Likona, 1991; Me Pherson, 1972). The concept of responsibility can help
structure and assist students in creating their own clear, understandable goals for
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
114
their schooling and their personal life. Whether this is a conscious by-product of
the teachers is not clear, but further investigation warrants an answer.
improving Student Achievement through Implementation of Data Use. Linda
Darling-Hammond (1997, p. 106) pointed out in her book, The Right to Learn, that
one key to student development is providing clear standards and criteria for
performance on specific tasks, linked to lots of feedback about work progress, and
continual opportunities for students to revise their work in response to this feedback.
Most marked results are found with the increase use of baseline and benchmark data
to improve writing instruction for students. The teachers at Del Sol Elementary
School are using the evidence of the writing assessments to change the strategies in
their classroom instruction as described by Darling-Hammond.
Perhaps the nature of the questions asked during interviews and the focus
on academic student performance data limited the elaboration of other possible
causes of improvement in student achievement. Further investigation of observations
and document reviews reveal that the environment of the school, teacher attitude and
relationships, and the use of non-instructional data may be contributing factors to the
improvement in student achievement. This use of two or more processes to change
a specific practice is referred to as “proactive redundancy.” Developing redundant
processes ensures that all students are learning and provides equitability that leads to
improvements in instruction and student achievement. It also assists teachers in
being successful with all students at equally high academic levels. This was found to
be the case in one of the four under-performing schools in Texas.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
115
Evidence shows that better outcomes are achieved by “personal-
communal” school models, fostering common learning experiences, opportunities for
cooperative work, and continual relationships and greater participation of parents,
teachers and students (Lee, Bryk & Smith, 1993).
Effective Elements of Data Use. The two effective elements of data use
identified by the teachers interviewed are directly connected to the intended use of
the district design, to bring about student achievement gains in the standardized
assessments and the analysis of the district, school, and classroom data. The seven
indicators for successful implementation of the use of data in the literature are
included in the design. The fifth indicator, Supports for Strong Instructional
Improvement, specifies the four types of data to be used. They are:
1. Disaggregated data down to the individual student level in a format that is
useable to improve instruction.
2. Data that can be identified by the students that do not meet grade-level
expectancy for intervention purposes.
3. Summative results that are disaggregated and used to address instructional
weaknesses.
4. Data that is used to inform instructional decisions and evaluate program
effectiveness.
These indicators provide accurate information and assist educators to be able to
make effective decisions (Johnson, 1997).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
116
Stages of Concern scores reveal that the teachers, as a group, are
performing mostly in Stage 2, personal stage, with the highest number of averages of
at least 3.5. Three teachers had an average of at least 3.5 in Stage 2 only. Eight
teachers had at least a score of 3.5 in a minimum of two other stages. The remaining
teachers had scores of at least 3.5 in more than three stages. This indicates a wide
range o f individual capacities and it is difficult to determine from these results which
of the seven indicators the teachers are using well. It is important to note here that
knowledge of the particular type and quantity of information/data that is provided to
teachers for student improvement should be addressed to move the process forward
(Hall et al., 1986). Interventions from the data teams and district and site leadership
may be designed to address the levels of concern. Further collaboration, one-to-one
analysis of data teacher to teacher, and teacher to principal would pinpoint more
specifically, the depth of the effective elements of data use.
Tracking Student Growth
The principal is ultimately held responsible for the implementation of the
district’s design improve student achievement through the use of data at the school
site, but each teacher and each child also is held responsible for improving student
achievement.
The research from the Charles A. Dana Center at the University of Texas in
Austin found that four districts that had wide-spread improvements in all academic
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
117
performances identified eight accountability steps for the use of data for teachers.
The findings from this study revealed that Del Sol Elementary School uses the steps.
1. Establish baseline data on each child using identified test results, by
disaggregating all the data by subgroups,
2. Focus on weak areas
3. Build to the strongest areas
4. Maintain what has been taught
5. Develop an instructional focus
6. Develop an instructional calendar
7. Assess and track each child to determine how he is performing
8. Re-teach, and tutor
This spiraling process continues, as all teachers have to assume the responsibility for
achieving equitable and excellent teaching and learning in their classrooms.
Organizations and individuals who do not think and act in terms of feedback
spirals remain oblivious to the data coming to them. They are insensitive to
feedback. They resist or avoid considering and reflecting on incoming data, and they
lack the inclination for self- modification. Spiraling is crucial to the process of
renewal (Costa & Kallick, 1995).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
118
Improving Student achievement through Implementation of Data Use
Nine teachers scored at least 3.5 for Stage four, Consequences on the Stages
of Concern Questionnaire. This indicates an arousal of the impact of the district
design to use student data to Improve student performance.
Further patterns of teacher concerns from the results of the Stages of Concern
Questionnaire indicate that the intensity of concerns about the implementation of the
district’s design for data use are varied across the faculty. Although 11 teachers had
intense concerns in none of the stages or only in one stage (Information or Personal),
the remainder of the staff had two or more intense concerns ranging from
Information into Consequence. Two teachers had intense concerns in all six Stages.
This data indicates that although the staff is using data, there are personal concerns
about their capacity to function with the data. These results may also indicate that
there are some uncertainties and ambiguities associated with the data implementation
design.
Evidence from observation, interviews and artifact analysis exemplifies the
literature on improving student achievement through data practices which are
described as four pillars by the National Center on Education and the Economy:
1. Using student learning standards as a foundation for reform work.
2. Enhancing teachers’ capacity to provide effective instruction toward the standard
through professional development.
3. Engaging the parents, community, and other stakeholders in the process.
4. Providing meaningful, on-going assessment of student achievement.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
119
Assurances that they are in place allows for the continual monitoring for student
achievement gains (National Center on Education and the Economy, 2001).
The findings of the districts data use policies and strategies, its plan and the
implementation of the data use policy and strategy in practice have been discussed in
depth. The findings for the third and final research question, “To What extent is the
district design a good one?” are discussed under four headings. The headings are:
1. District Support for Standards-based Instruction and Assessments: The Intended
District Impact and the Observed Impact on the School Site.
2. District and School Accountability to Standards-Based Curriculum: Student
Performance Data Forwarded to School and How School is Utilizing Student
Data to Impact Student Performance.
3. Determination of “High” Student Performance Aligned to Standards.
4. Degree to which the High Performance is Communicated to Teachers, Students,
and Parents.
District Support for Standards-Based Instruction and Assessments
Intended District Impact. The literature on the intended use of data about
student performance contends that its purpose is to provide accountability and
feedback, improve instruction on a broad scale, and increase student learning for
sustainable success (Glickman, 1993). The Bandaras Heights City School District
intentions are in alignment with these purposes. The district began its refocusing
efforts to standards-based accountability in 1996 when the new superintendent was
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
120
appointed. Within the standards-based accountability act, instructional decisions
based on assessment results were growing in magnitude. The district office started
its data-driven efforts by providing the schools with training and timelines for school
staff engagement.
The district chose a highly visible approach to standards-based instruction
and assessment to ensure everyone was aware that using data is an important focus.
It also wanted to convey the expectations that all staff will support and participate in
using data. Additionally, the district wanted to ensure that staff members receive
common information about why, what will happen, and how it will affect them. The
district leadership is well informed and all indications reveal that it is serious about
conditions of and long-term consequences for low student performance. The
consequences of low student performance would weaken our declining
compositeness and economic stability. Almost 4 million American Adults cannot
read, sign their names, or perform simple addition and subtraction. Mumane and
Levy (p. 18) reported in Teaching the New Basic Skills, that he most pressing social
issue is the increasing dramatic disparity between the skills children are acquiring in
the school system and the skills they will need in order to obtain good jobs in an
increasingly selective, globally influenced job market.
Observed Impact. The observed impact at the school site reflects the
district’s support for data driven practices. Although teachers at Del Sol Elementary
School are at varying degrees of data use proficiency, all teachers use data to drive
instructional decisions and understand its importance in increasing student
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
121
achievement. This approach has been cited in the literature (Holcomb, 1999) as
an effective approach to change that is both visible and large in its magnitude.
The district design also aligns with the literature on comprehensive school
Reform models that were developed for the Obbey-Porter Legislation in 1999. It
includes the following nine components:
1. A focus on academic achievement as the highest priority and mission of the
district and school
2. High quality teaching and learning of rigorous standards
3. Collegial and collaborative faculty, teamwork-engagement in self-study and
reflection
4. High expectations of student achievement
5. Staff development linked to teaching o f standards
6. Use of state and local standards and accountability system
7. Parent involvement
8. Use of research and data to promote continuous improvement.
9. Safe, orderly school climate conducive to learning
District and School Accountability to Standards-Based Curriculum
Student Performance Data. The Bandaras Heights City School District has
established a district and school accountability system to standards-based
curriculum. It is aligned with the state’s accountability efforts and with the literature
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
122
that describes the success of widespread academic achievement for children from
low-income homes and children of color (Skrla et a l, 2000).
How the School is Utilizing Student Data. The school has shown increased
gains in the state tests and local assessment for student improvement in writing
performances as a result of using student data. The literature reinforces the school
uses o f four major purposes for using student data. They are to:
1. Develop coherent practices for the delivery of instruction within the curriculum
through benchmark targets.
2. Collect, interpret, use data, and monitor results
3. Supervise, evaluate, and hold people accountable
4. Refocus energies, refine efforts, and ensure continuous performance
Determination of “High” Student Performance
The Degree to which High Student Performance is Aligned to Standards.
The literature on determining high student success is in place at Del Sol Elementary
School. The following processes, as described in a Texas study, ensure that all
students are learning (Johnson et al., 1998):
1. Provide equitable, excellent learning
2. Lead improvements in instruction and thus student achievement
3. Motivate positively and assist teachers to improve instruction and thus learning.
4. Assist teachers in being successful with all students at equally high academic
levels.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
123
Degree to Which the High Performance is Communicated to Teachers. Students and
Parents
Teachers. Communication to teachers and other staff about the expectations
for high performance is evident from interviews, observations, and artifact analysis.
School visits from the district superintendent and assistant superintendent reinforce
the high performance expectations. The principal and the teacher evaluations are
crafted based on the high performance expectations and the use of data to support
those expectations.
Students. High performance expectations are communicated to students
through classroom assignments, homework assignments, and rubric scoring. The
assignments are based on the state standards, which drive the curricular choices. The
rubrics were developed with the standards as the framework. The rubrics define what
the students should know and be able to do. Assessments of the assignments are
provided daily or weekly.
Parents. Parents are routinely informed of student performance goal through
progress reports and standards-based report cards. At Del Sol Elementary School
Parent and Family nights throughout the year offer additional opportunities for the
district and school to share with the community the high expectations for student
performance and achievement.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
124
Conclusion
This study was done to add to the educational literature on the adequacy of
a district design for data use and the design’s implementation at a school site.
The focus was on an urban elementary district and one of its school sites.
Findings and the discussion of this case study provide a thoughtful perspective of
one district’s intended result to increase the percentage of students achieving through
the use of multiple data sources.
Bandaras Heights City School District has capitalized on the national and
state trends and needs for data to create its design for data use. The district chose an
approach to data use that is comprehensive and inclusive, one with reputable
research from multiple sources. The involvement of all key stakeholders, including a
supportive board, in the creation and implementation of the data use design pushed
the importance of such an effort to the forefront of the district.
Continual renewal of the design and its elements allows the school and the
teachers to stay current with the state trends and make adjustments to their
instructional strategies. The power of such a design is that old piecemeal initiatives
give way to focused, carefully planned strategies and policies that are explicitly
grounded in standard and research practices.
Technology utilization at the school and classroom level would perhaps
assist the teachers in making faster decisions that impact instruction and student
learning.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
125
The intended result set out by the district in its policies and strategies is
clearly articulated and understood to have one purpose, to increase the percentage of
students achieving. It is apparent that there are efforts to find all appropriate avenues
to reach this intended result. All teachers use data to guide instruction and
communicate in various ways to students and to their parents. The principal plays a
crucial and guiding role in the schools use of data.
The general fund has provided the largest allocation for data use
implementation. Once the funds are released to the school, the School Site
Council and the staff use data to fund programs and other strategies that affect
curriculum and instructional decisions to increase student achievement.
The principal is ultimately held responsible for the implementation of the
district’s design to improve student achievement through the use of data at the school
site, but each teacher and child also is held responsible for improving student
achievement.
The adequacy of the district design in this study has been ranked between
four- and five-point Scoring Rubric in the four areas it was assessed.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 5
SUMMARY, CONCLUSIONS, AND IMPLICATIONS
Summary
Purpose and Problem
This case study was conducted to add to the educational literature on whether
the use of student performance data improves student achievement. Several factors
arose as the study was being created. The first factor was whether the district’s
creation of a plan to use student data to improve instruction and increase student
achievement was valuable or useful. A second factor was whether the design was
actually implemented at a school site and, if so, to what degree. Review of the
literature indicated that there was little research that clearly explained whether a
school district’s design was of value. Further, there was no research that connected
the implementation of a district’s design at a school site. Additionally, there was
little research that explained whether the use of data in a school actually improved
student achievement. And lastly, if a district design was created and used in a school,
how good might it be? These factors were then re-shaped into the three research
questions for the study. The doctoral program under which this study was conducted
focused on the challenges and needs for urban schools. Therefore, this study focused
on one diverse urban elementary district and one of its school sites. The content area
under investigation in this study was Language Arts/writing.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
127
The findings and discussion of this case study provide a thoughtful
perspective of one district’s intended results to increase student achievement through
the use of multiple data sources.
Methodology
Data were collected to determine first if the district created a design for data
use, and second, if a design was created, what were the policies and strategies of the
design. Data were collected to determine the extent of implementation of the design
in one school and specifically in six classrooms. Data also were collected to
determine the degree of adequacy of the district design.
The district and school were not randomly selected. They were purposively
sampled and nominated based on the facts that they were not always high
performing, but they made progress and appeared to be implementing cutting edge,
current, and emerging data use research-based best practices. The district was
nominated also because of its urban challenges with a diverse student population.
A multi-method qualitative approach was used for this study. Formal
interviews with the Assistant Superintendent of Instruction, the principal and six
teachers were conducted. Situated interviews were also conducted with the teachers
to gamer anecdotal accounts and stories for the case study. A Teacher Questionnaire
and a Stages of Concern Questionnaire were administered to all 25 teachers in the
school. Observations, document reviews and artifact analyses were conducted.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
128
Analysis of each research question included comparisons of responses
from the formal and situated interviews, observations, archival data, quantitative
data, and artifact analysis. Individual scores on the Teacher Questionnaire and the
Stages of Concern Questionnaire were analyzed.
Results of the findings were discussed and are summarized in the ensuing
section.
Findings
Bandaras Heights City School District has capitalized on the national and
state policy trends and needs to create its design for data use to improve instruction.
The district chose a comprehensive and inclusive approach to data use, one that
includes reputable research from multiple sources to improve student achievement
performance. The involvement of all key stakeholders, including a supportive board,
in the creation and implementation of the data use design pushed the importance of
such an effort to the forefront of the district. There is a District Assessment Team
comprised of two to four teachers and the principal from each of the schools who
guided the staff to follow the design strategies.
The design includes all data from state mandated tests for elementary grades
and its own baseline and benchmark assessments and timelines referred to as
Essential Agreements are used. Comparisons of student performances are conducted
within the district, within the school, and within grade levels. Comparisons are
conducted also between current individual student work and their prior year scores.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
129
The district design does not analyze student performance against international
comparisons.
The intended result of the design is clearly articulated and understood; to
increase the percentage of students achieving. Continual renewal of the design
allows the school and the teachers to remain current with the state trends. The
Backward Mapping Process by Grant Wiggins has been incorporated into the district
design. The general funds provide the largest allocation for data use implementation.
The School Site Council and the staff use data to recommend disbursement of the
funds to support programs and strategies to increase student achievement.
Technology utilization at the school and classroom levels would perhaps
assist the teachers in making faster decisions that impact instruction and student
learning.
The intended result is also evident in Del Sol Elementary School. The
principal played a crucial role in guiding the school’s use of data, particularly in the
area of Language Arts with an emphasis on writing. Teacher perceptual data, student
attendance, participation and attitude, referral rates for Student Study Team and
special education, and behavior intervention data were collected and added to the
data analysis process to discover trends and patterns in school performance that
might not be uncovered with quantitative data only.
The two elements of data use that were found most effective for teachers
were standardized assessments and the analysis of district, school, and classroom
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
130
data. When the data were shared with the students and parents in understandable
terms, the parents were more able to support the school and their own child’s efforts.
The principal was ultimately held responsible for the implementation of the
district’s design to improve student achievement through the use of data at the school
site. Accountability expectations were written into the principal’s evaluations. Each
teacher and child also was held responsible for improving student achievement.
Teachers were held responsible to provide evidence of student work progress during
their performance appraisal meetings. Students were held responsible to adhere to
their commitments on the Student-Parent-Teacher Contracts. Teacher perception
indicates that the majority believe that there is a direct link between accountability
and their promotion.
Collaborative teaming to disaggregate school and classroom data through
grade levels meetings and informal gatherings has been valued by the teachers.
The adequacy of the district design in this study has been rated using a
Research Rating five-point Scoring Rubric in the four areas it assessed.
Conclusions
This case study was able to demonstrate that an elementary school district
could create a design to use data to improve instruction and increase student
achievement. The results of this study indicate that the elements of the district
design are worthy of consideration when researching best practices for the creation
and implementation of data use in schools. The design had research-based elements
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
131
that were proven to assist schools In assessing the needs of student to Improve
their achievement levels. The design was implemented in one school within the
district. The school was able to Implement the district design successfully and
propel its own efforts beyond the district’s design.
Implications
For Research
Based on the findings and the discussion in the current study, four areas
emerged for further investigation. The first area was for a more in-depth look at the
technology. The second was the relationship between fiscal adequacy and district
efforts to improve student achievement through data use and assessment analysis.
The third area was that of examining accountability practices, specifically as they
relate to teacher promotion and salary advancement at the district and school site
level. The fourth area for further investigation that emerged was the comparison
between the individual teacher Stages of Concern for data use and the improvement
of student achievement.
Further research investigating the role of state-of-the-art technology used to
address current and emerging student assessments on the state, district and school
levels would add to the literature about the multiple uses of student data to improve
student achievement. Although this study aimed at addressing the role of state-of-
the-art technology, the district out-sourced its state assessments and did not yet have
the capacity to maintain and analyze the data itself. An additional area of technology
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
132
to investigate would be to examine the benefits of out-sourcing data for
disaggregation versus the district taking full responsibility for all areas of data
collection and analysis.
The second area that emerged, the relationship between fiscal adequacy and
district efforts to improve student achievement through data use and assessment
analysis is another area touched upon in this study. Time and resources did not
permit enough in-depth discussion and analysis to gamer sufficient information in
this important area of educational reform. Further investigation into the alignment of
the fiscal resources allocated to the district design to improve student achievement
could yield helpful data to local, state and national policymakers.
Although this study did not delve deeply into the funding issues for the
district design and its implementation, further examination would be a worthy
venture. A connection between schooling and long-term outcomes could be a further
research topic. As the connections are developed, analyses of the outcomes and their
links to the distribution of school fiscal resources could be included. (Odden, 1999).
The third area that emerged, that of examining accountability practices,
specifically, as they relate to teacher promotion and salary advancement at the
district and school site level.
The fourth area for further investigation that emerged was the comparison
between levels of use of data by teachers and the levels of student achievement. This
study included the Stages of Concern Questionnaire by Hall and Hord. The
information the questionnaire revealed was related directly to their affect, perception,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
133
and feelings of the district’s design to use student data to improve student
performance. What people feel and how they perceive a situation can be quite
different from what they actually do.
The second diagnostic dimension of the Concern Based Adoption Model
(CBAM), the Levels of Use of the Innovation, could serve as a valuable diagnostic
tool for planning and facilitating the change process and could be a useful guide in
offering differentiated professional development.
For Policymakers
Policymakers will find this study useful for identifying a design for effective
data collection. Further, a deeper understanding of how assessment legislation, local
district assessments, and classroom assessments link to provide a comprehensive
study of student learning. The adequacy of the design is an impressive measure for
policymakers to consider when searching for data use design models.
For District and School Leaders
District leaders will find this study of value as a model for a data collection
design strategy. Key elements of the Bandaras Heights School District’s design are
research based and have broad appeal for that reason. The replication of the design
that was implemented at the De Sol School should be done with caution. The school
used the design as its framework and structure. However, the creativity, knowledge,
and passion of the school principal were a determined and driven force in the actual
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
134
implementation of data use. The elements from the conceptual frameworks for
this study will help guide district and school leaders in the development of their own
district/school custom design for data use. The elements of the design analyzed and
discussed in this study should be considered when creating a new design.
For Teachers
Caution is warranted because of the nature of this study. Case studies are not
designed for replication; however, the results from this study will assist teachers in
four ways. One way is that they may use the results to understand that the use of well
developed multiple measures that are aligned to standards can be analyzed to
improve instruction and student achievement. Another way that teachers can learn
from this case study is to understand that the use of standardized assessments have
their role in providing specific information about each student as compared to other
students. Thirdly, teachers may discover that the routine analysis of criterion-
referenced student assessment and student work samples provide a perspective on
student learning and achievement that cannot be found from standardized tests. The
fourth way that this study will assist teachers is to highlight that data collection and
analysis of student work is a laborious process but, without the skills to use specific
data to guide instruction, teachers may fail to take students to the levels necessary to
compete in a global work force market. The job as educators is to prepare students to
enter an ever-changing world. The students’, the state’s, and the nation’s futures are
at risk.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
135
REFERENCES
Alpert, S. (1999). Senate Bill IX: Public School Accountability Act 1999.
Sacramento, CA. Author. [On-Line]. Available:
www.cde.ca. gov/psaa/sb 1 x.htm.
Asp, E. (2000). Assessment in Education: Where have we been? Where are we
headed? Education in a New Era, ASCD Yearbook. Alexandria, YA: Association
for Supervision and Curriculum Development, 123-158.
Baker, E., Brown R. & Herman, J. (2000). Student assessment and student
achievement in the California Public School System, CSE Technical Report
519, Center for the Study of Evaluation National Center for Research on
Evaluation Standards and Student Testing.
Berlinger, D. (1993, February). Educational reform in an era of disinformation.
Education Policy Analysis Archives. 1(2): 1-35. Retrieved July 6.
Bernhardt,V. (1994). The School Portfolio: New Jersey: Eye on Education.
Blank, R. & Wagner, L. (2001). Understanding NAEP and TIMMS results: Three
types of analyses useful to educators. ERS Spectrum. 19(1): 23-33.
Blase, J. (1986). A Qualitative Analysis of Teacher Stress: Consequences for
Performance. American Educational Research Journal. 23: 13-40.
Boese. L., Burkhardt. R., Carstens. L ., Devine. M., Gaffney. T. & Just. A. (2001).
Public School Accountability (1999-2000) Immediate
Intervention/Underperforming Schools Program (II/USP): How Low
Performing Schools in California Are Facing the Challenge of Improving
Student Achievement-.Sacramento, CA. California Department of Education,
Division of Policy and Evaluation. [On-Line]. Available:
www.cde.ca.gov/psaa/PSAA%20Yr%201.pdf
Brownell, S. & Hart, G. (2001). An era of accountability in California. The Clearing
House, 74(4): 183-186.
Calvo, N., Carlos, L., Guth, G., Hayward, G., Holtzman, D. & Schneider, S. (1999).
Evaluation of California's Standards Based Accountability System: Final
Report November 1999. California Department of Education: WestEd.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
136
Carr, J.F. & Harris, D. (2001). Succeeding with standards linking curriculum,
assessment and action planning. Alexandria, VA: Association for Supervision
and Curriculum Development.
Charles A. Dana Center, University of Texas at Austin. (1999). Hope for Urban
Education: A Study of Nine High Performing, High-Poverty, Urban Elementary
Schools. Washington, D.C.: U.S. Department of Education, Planning and
Evaluation Services.
Charles A. Dana Center. (2000). Equity-driven achievement-focused school
districts: a report on systemic school success in four Texas school districts
serving diverse student populations. University of Texas at Austin.
Christie, K. (2000, September). Monitoring what matters. Phi Delta Kappan. 82(1):
5-6.
Clemments, B. & Blank, R. (1997). What do we know about education in the states:
Education indicators in the state reports. Paper presented at the Annual Meeting
o f the American Educational Research Association, Chicago, IL.
Consortium for Policy Research in Education (CPRE). (2000). Assessment and
accountability in the fifty states: 1999-2000. Available: www.cde.ca.gov
Conzemius, A. (2000, Winter). Framework. Journal of Staff Development. 21(11:
38-41.
Costa, A.L. & Kallick, B. (1995). Assessment in the learning organization shifting
the paradigm. Alexandria, VA: Association for Supervision and Curriculum
Development.
Covey, S. (1991). The seven habits o f highly effective people. New York: Simon &
Schuster.
Cunningham, P.M. & Allington, R.L. (1994). Classrooms that Work They Can All
Read and Write. New York: HarperCollins College Publishers.
Darling-Hammond, L. (1991). The implications of testing policy for quality and
equality. Phi Delta Kappan. 73(31: 220-225.
Darling-Hammond, L. (1997). The right to learn: a blue print fo r creating schools
that work. New York: Jossey-Bass.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
137
DeSchryver, D. (1999, June). Truth in spending: The cost of not educating our
children. The Center for Education Reform. 1-11. Retrieved July 6 from
www.edreform.com/pubs/tnith.htm .
Duffy, M., Goertz, M. & Le Floch, K. (2001). Assessment and Accountability
Systems in the 50 States: 1999-2000. Consortium for Policy Research in
Education.
EdSource. (2001). California's new academic standards take hold. On-line.
Education Week & Pew Charitable Trust. (2001). Quality Counts 2001: A Better
Balance. Bethesda, MD: Education Week & Pew Charitable Trust. [On-Line].
Available:
www.edweek.com/sreports/qc01/articles/qc01story.cfm7siug-17exec__sum.h2Q
Fierros et al. (1997). Using multiple methods of assessment to promote district level
reflection about instructional improvement. Chicago, IL: Paper presented at the
annual meeting of the American Educational Research Association.
Frey, S. (2000). National accountability movement offers lessons for California.
Palo Alto, CA: EdSource.
Frye, B., Fugerer, B., Harvey, S., McKay, L. & Robinson, O. (1999). SAM: A
student achievement model designed to empower teachers and increase student
achievement through action research. ERIC Document Reproduction Service
No. ED 430 015.
Fullan, M. & Hargreaves, A. (1996). What’ s worth fighting fo r in your school? New
York: Teachers College Press.
Fullan, M. & Miles, M. (1992, June). Getting reform right: What works and what
doesn't? Phi Delta Kappan. 745-52.
Glickman, C.D. (1993/ Renewing Am erica’ s schools a guide fo r school-based
action. New York: Jossey-Bass.
Glickman, C.D. (2002). Leadership fo r Learning How to Help Teachers Succeed.
Alexandria, VA: Association for Supervision and Curriculum Development.
Goertz, M. (1997). The challenges of collecting school-based data. Journal of
Education Finance. 22: 291-302.
Grissmer, D. & Ross, M. (2000). Analytic issues in the assessment of student
achievement). Jessup, MD: NCES Publication No. 2000-050.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
138
Guth, C., Holtzman, D., Schneider, S., Carols L., Smith, J., Hayward, G. & Calvo, N.
(1999},, Evaluation o f California's Standards-Based Accountability System.
Final Report. West Ed, San Francisco, CA. [On-Line]. Available:
www.wested.org/online___pubs/accoimtabiiity/iiidex.html
Hall, G.E. & Hord, S.M. (1986). Change in Schools Facilitating the Process. New
York: State University Press.
Hart, G. & Brownell, N. (2001). An era of educational accountability in California.
The Clearinghouse. 74141 183-186.
Heck, R. (2000). Examining the impact of school quality on school outcomes and
improvement: A value-added approach. Educational Administration Quarterly
36(4): 512-552.
Hiebert, E.H. & Raphael, T.E. (1998). Early Literacy Instruction. Florida: Holt
Reinhart and Winston.
Holcomb, E. (1999). Getting excited about data: How to combine people,_passion,
and proof Thousand Oaks, CA: Corwin Press.
Johnson, E. & Siegendorf, A. (1998). U.S. Department of Education. National Center
for Educational Statistics. Linking the National Assessment of Educational
Progress (NAEP) and the Third International Mathematics and Science Study
(TIMSS): Eighth-Grade Results, NCES 98-500, Washington.
Johnson, J. (1997). Data driven school improvement. ERIC Clearinghouse on
Educational Management Digest #109. Eugene, OR: University of Oregon.
Jones, R. (2000, September). What researchers say will make standards work. The
American School Board Journal. 187: 13-19.
Kharnia, R., Trousdale, D., Pennell, W. & Kell, J. (1999). Supporting data use among
administrators: Results from a data planning model. Montreal, Quebec, Canada.
Paper presented at the annual meeting of the American Educational Research
Association.
Kieff, J. (2001). Small schools: great strides: A study of new small schools in
Chicago: CMljlMsd Education, 77(4): 253-254.
Killion, J. & Bellamy, G. (2000, Winter). On the job: data analysts focus school
improvement efforts. Journal of Staff Development, 21(1). Retrieved from the
www.nsdc.org/library/isd/killion211 .html
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
139
Larson, K., Guidera, A. & Smith, N. (1998). The formula for success: A business
leader's guide to supporting math and science achievement (Publication No.
SAI-98-3028). Washington, D.C. (ERIC Document Reproduction Service No.
ED 419 914).
Levesque, K. (1996). Using data for program improvement: how do we encourage
schools to do it? National Center for Research in Vocational Education. 12: 1-6.
Levine, R., Rathbun, A., Seldon, R. & Davis, A. (1998). NAEP's constituents: What
do they want? Report of the NAEP Constituents' Survey and Focus Groups.
Washington, DC: National Center for Education Statistics.
Lickona, R. (1991). Educating fo r Character. New York: Bantam.
Linn, R. (2000). Assessment and accountability. Educational Researcher. 29(2): 4-
16. [On-Line]. Available: www.aera.net/pubs/er/arts/29-02/linn 15.htm
Lipsitz, J., Mizell, H., Jackson, A. & Austin, L. (1997). Speaking with one voice.
Phi Delta Kappan. 78: 533-541.
L oofbourrow , S. (1996). M aximizing School Board Leadership, West Sacramento,
CA: California School Boards Association.
Lloyd, K., Ramsey, D., Bell, & Terrel, H. (1998). Reclaiming our nation at risk,
lessons learned: reforming our public schools. The Terrel H. Bell Knowledge
Network for Education Reform.
Making standards matter 1999: Executive summary. (1999). Available:
www.AFT.org.edissues/standards99/Index.htm
Marshall, R. & Tucker, M. (1992). Thinking fo r a living Education and the wealth o f
nations. New York: Basic Books.
Marzano, R. (2000). Transforming classroom grading. Alexandria, VA: Association
for Supervision and Curriculum Development.
Marx, G., Hunter, D. & Johnson, C. (1997). Increasing student achievement: an
urban district's search for success. Urban Education. 31(3): 529-544.
McCary, M., Peel, J. & McColskey, W. (1997). Using accountability as a lever for
changing the culture of the schools: examining district strategies. Greensboro,
North Carolina: SERVE, 1997. 69 pages. ED 408 697.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
140
McPherson, E. (1972). Small Town Teacher. Cambridge: Harvard University
Press.
Merrow, J. (2001, May). Undermining standards. Phi Delta Kappan. 653-59.
Meyer, I. & Baker, D. (1996). Forming American educational policy with
international data: Lessons from the sociology of education. Sociology of
Education (Extra Issue). 123-129
Mumane, R. & Levy, F. (1996). Teaching the new basic skills principles for
education children to thrive in a changing economy. New York: The Free Press.
National Education Goals Panel. (2001). Promising practices: Progress toward the
Goals, 2000. Available www.negp.gov: Author. The National Education Goals
Panel. (2001). Raising achievement and reducing gaps: Reporting progress
toward goals for academic achievement. Available www.negp.gov: Author
National Education Association. (2000, July). Data-driven decision making and
student achievement. Available: www.nea.org/issues/high-stakes/dddm.html
Noyce, P., Perda, D. & Traver, R. (2000). Creating data-driven schools. Educational
Leadership 57(5): 52-6. Alexandria, VA: The Association for Supervision and
Curriculum Development,
O' Donovan, E. (2001). Dealing with information overload. Phi Delta Kappan
21(6): 9-11.
Odden, A. (1999). Making better use of resources for educational reform. In D.D.
Marsh (Ed.), 1999 ASCD Yearbook: Preparing our schools for the 21st century.
Alexandria, VA: Association for Supervision and Curriculum Development.
Odden, A. (2000, February). The costs of sustaining educational change through
comprehensive School reform. Phi Delta Kappan 81. (6): 433-438.
Ogawa, R. & Collom, E. (2000). Using performance indicators to hold schools
accountable: Implicit assumptions and inherent tensions. Peabody Journal of
Education. 75(4): 200-215.
Olson, L. (1999). Rating the standards: Quality counts ’99. Education Week 18. 8-
10.
Pardini, P. (2000). Well Done. Journal of Staff Development. 9(1): 12-19.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
141
Raphael, T. & Hiebert, E.H. (1996). Creating an integrated approach to literacy
instruction. Florida: Holt, Rinehart and Winston.
Reeves, D.B. (2001). Accountability in action, as blueprint fo r learning
organizations. Denver, Colorado: Advanced Learning Press.
Robinson, V. & Timperley, H. (2000). The link between accountability and
improvement: The case of reporting to parents. Peabody Journal of Education.
75(4): 66-89.
Rosenholtz, S. (1989). Teachers’ workplace: the social Organization o f schools:
New York: Longman.
Schmoker, M.J. (1996). Results: The key to continuous school improvement.
Alexandria, Virginia: Association for Supervision and Curriculum
Development.
Schmoker, M.J. (2001). The results fieldbook: practical strategies from dramatically
improved schools: Alexandria, VA: Association for Supervision and Curriculum
Development.
Shepard, L. (1995). Educational Leadership: Vol. 5. Using Assessment to Improve
Learning, 5(5). Available:
www.. ascd.org/readingropm/edlead/9502/Shepard.html
Sparks, D.& Hirsh, S. (1997). A new vision fo r staff development. Alexandria, VA:
Association for Supervision and Curriculum Development.
Strong, R.W., Silver, H.F., & Perini, M.J. (2001). Teaching What Matters Most,
Standards and strategies fo r raising student achievement. Alexandria, VA:
Association for Supervision and Curriculum Development.
Thomas B. Fordham Foundation. (1999). Better by design? A consumer's guide to
schoolwide reform (Executive Summary 2001). Available
www.edexcellence.net:
Traub, J. Thomas B. Fordham Foundation. (2001). A TIMMS primer. Available
www.edexcellence.net: Stevenson, H.
Thomas, D. & Bainbridge, W. (2001, January). All children can learn: facts and
fallacies. Phi Delta Kappan. 660-662.
Tucker, M. & Codding, J. (1998). Standards fo r our schools: How to set them,
measure them, and reach them. San Francisco, CA: Jossey Bass.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
142
Wiggins, G. & McTighe, J. (1998). Understanding by design. Alexandria, VA:
Association for Supervision and Curriculum Development.
Yinger, R. & Hendricks-Lee, M. (2000). The language of standards and teacher
education reform. Education Policy. 14(1): 94-106.
Zacfa, T., Zon, C. & Ackley, D. (1998). Standards for improvement. Thrust for
Educational Leadership. California: Association for California School
Administrators, 21l 14-18
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDICES
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
144
OBSERVATION GUIDE #3
j'Jpmfv Datfv Room* School ■
Clearly Evident Somewhat Evident Not at a il Evtde
During the Observation, watch for the following: 5 4 3 2 l
a. To what extent does the district design support a
standards-based focus at the district, school and
classroom levels?
□ □ □ □ □
1. How do the assessment results target critical skills
and concepts aligned with the standards? Q □ □ O Q
b. To what extent does the district design provide
student specific, valid, reliable information about
student performance visa vie the standards?
□ □ □ □ □
1. Are formative benchmark assessment results
collected and profiled every 6-12 weeks to monitor
student progress?
□ □ □ □ □
c. To what extent does the district design support both
system and student incentives and accountability? □ □ □ □ □
1. Are the schools meeting their targeted API growth to
include each subgroup? □ □ □ - □ O
2. Is the needed data being disseminated to all levels?
□ □ □ □ □
d. To what extent does the district design support
powerful process o f school reflection and teacher
reflection/problem solving and culture?
□ □ □ □ □
1. Is there a collaborative effort that reinforces
consensus decision-making? □ □ □ Q □
2. Is it required cross grade articulation assessments?
□ Q □ Q □
3. Is assessment training and support provided to
teachers and administrators? □ □ □ □ □
e. To what extent does the district design support
strong instructional improvement? □ O □ □ □
1. Is the data disaggregated down to individual student
level in a format that is useable to improve
instruction?
□ □ □ □ □
2. Can staff identify by the students that do not meet
grade level expectancy for interventions purposes? □
□
□ □
□
3. Is there summative results that are disaggregated and
used to address instructional weaknesses? Q □
□
□ □
4. Are results used to inform instructional decisions
and evaluate program effectiveness? □ □ □
□
□
f. To what extent does the district design enhance
public understanding, engagement? □ □
□
□
1. How does the district communicate to the public
their progress o f school’s strategic plan? □ □ O □ □
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
SURVEY #3
Data Source: ____________________________ Date: ____________________
Please rate each o f these questions on a scale o f 1-5
To a Great Extent T o Som e Extent N ot at All
5 4 3 2 1
a. To what extent does the district design support a standards-based focus at the
district, school and classroom levels?
1. The assessment results target critical skills and concepts aligned with the standards?
b. To what extent does the district design provide student specific, valid reliable
information about student performance visa vie the standards
1. Are formative benchmark assessment results collected and profiled every 6-12
weeks to monitor student progress?
c. To what extent does the district design support both system and student incentives
and accountability?
1. Are the schools meeting their targeted API growth to include each subgroup?
2. Is the needed data being disseminated to all levels?
d. To what extent does the district design support powerful process o f school reflection
and teacher reflection/problem solving and culture?
1. Is there a collaborative effort that reinforces consensus decision-making?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
146
SU R V EY #3
Page 2
2. Is it required cross grade articulation assessments?
3. Is assessment training and support provided to teachers and administrators?
e. To what extent does the district design support strong instructional improvement?
1. Is the data disaggregated down to individual student level in a format that is useable
to improve instruction?
2. Can staff identify by the students that do not meet grade level expectancy for
intervention purposes:
3. Is there summative results that are disaggregated and used to address instructional
weaknesses?
4. Are results used to inform instructional decisions and evaluate program
effectiveness?
f . To what extent does the district design enhance public understanding, engagement?
1. How does the district communicate to the public their progress o f school’s strategic
plan?
7/31/0!
JC :ae
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Interview Guide #3
Data Source:____________________________________ Date:______________________
a. To what extent does the district design support a standards-based focus at the district,
school and classroom levels?
1. How do the assessment results target critical skills and concepts aligned with the
standards?
b. To what extent does the district design provide student specific, valid reliable. .
information about student performance visa vie the standards?
1. Are formative benchmark assessment results collected and profiled every 6-12
weeks to monitor student progress?
c. To what extent does the district design support both system and student incentives
and accountability?
1. Are the schools meeting their targeted API growth to include each subgroup?
2. Is the needed data being disseminated to all levels?
d. To what extent does the district design support powerful process o f school reflection
and teacher reflection/problem solving and culture?
1. Is there a collaborative effort that reinforces consensus decision-making?
2. Is it required cross grade articulation assessments?
3. Is assessment training and support provided to teachers and administrators?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
e. To what extent does the district design support strong instructional improvement
1. Is the data disaggregated down to individual student level in a format that is
useable to improve instruction?
2. Can staff identify by the students that do not meet grade level expectancy for
intervention purposes:
3 Is there summative results that are disaggregated and used to address
instructional weaknesses?
4. Are results used to inform instructional decisions and evaluate program
effectiveness?
f. To what extent does the district design enhance public understanding, engagement?
1. How is does the district communicate to the public their progress o f school’s
strategic plan?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
149
Data Use Study
Conceptual Framework A
Description of Date Use Poides and Strategies: The Design
This conceptual framework addresses research question 1: What is the district design fo r ■ .
using data regarding student performance, and how is that design linked to the current
and the emerging state contextfor assessing student performance?
i. StodfisttM ffiro^^
Current
• Are the district and school focusing on irajrovmg Stanford 9 scores and API ratings
as defined by the State?
• Are the district and school using fee California Content Standards to improve student
■ performance?
• Are authentic assessments linked to state standards throughout fee district, school,
and classrooms?
• Are noun-referenced assessments used?
• Axe interventions wife students linked to performance assessments? -
• Is there awareness on fee district and school levels o f fee Stanford 9, API scores, etc?
• Is state-of-the-art technology used to address current and emerging student
assessments on fee state, district, and school levels? '
Tfrnwi>m»
• Is there awareness on the district and school levels o f emerging state assessments?
• Is there preparation for fee High School Exit Exam (Sr. High only)?
• Is there prq>aration for fee CaJi&mia English Language Test (CELDT)?
• Is there evidence o f district planning to prepare students for emerging assessments?
• Are there examples o f fee use o f criteria-refermced tests on fee district and school
levels? .
• Is there measurement o f student performance against international performance
standards?
i
9» Overview o f fee Elements of District Design ofD ata Use to Improve Student.. •
Performance..
• What types o f data are collected in fee district design o f data use?
• What is fee timeline for fee receipt and use o f data in fee district? '
• What assessment instruments is fee district using to collect data? .
• What is the rationale for collecting these instruments?
• What methods are used to analyze these data?
• What is the rationale for selecting these methods o f analysis?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
150
• Did reputable research guide the district’s design o f data use?
• What methods does the district have for disseminating data?
• What training does the district provide for imjrovmg/modifymg data?
• Are multiple stakeholders involved in tie decision-making process within the district?
• What steps do district and schools take to use data to improve gtndant performance?
• What outside influences affect policy and strategy design?
• What inside influences affect policy and strategy design?
• What rote does fiscal and funding issues play in the design o f the policies md
strategies? .
• What state legislation was considered in the design?
• What ament data practises are included in die policy?
• What emerging data practices are instated in the policy?
3- District Deaagpg and M ingsJgL SB asriP « QfPgsfcLBssm
• What board rulings directly support the district design?
• ■ To what degree are multiple stakeholders involved in establishing/ mftafinemg
boards?
• Are research studies guiding the board support of the district design?
• Does the board consider State legislation? . ‘ '
• What is the timeline to implement the district design?
• Is there board mandated training for die district design o f the use o f data?
• What is the process for establishing district-wide high-performance goals?
• Is there money allotted to develop and implement die district design?
• Is technology designated to be used in die implementation o f the design?
• Are outside sources used to imptemmt the design? ■
'I. Intended Results ofD esian Plans to Improve Student Performance (District School.'
and n«ssroom l
District
• What are die intended results of district dssigs? ’
• Does the district want to use data to increase student performance* m l is there an
increased focus on improving student performance?
• Was there a process created to establish district-wide high-performance goals? •
• Does the district support schools’ efforts to improve student performance fey
providing student data?
• Does fee district use multiple measures to increase performance arid guide
instruction?
• Does the district provide schools with the results o f these multiple measures? ■
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
151
School
• What is the school doing to use data (directly or indirectly) to promote student
learning?
• Are there any schoolwide plans/ efforts to improve how data are gathered, analyzed,
and used?
• Does fee school try to improve administrator’s and teachers5 ability to.use date to
increase student performance through training, etc.?
• Does the school try to improve administrator’s and teachers’ ability to analyze-data
effectively to guide instruction?
• I s fee school trying to raise shMmt paforamce expectations with teachess and
parents?
• Does the school administrators) support school-wide implementation o f staadwds-
based cuniculnm to improve student performance? •
. . • Does the school regularly inform the students, parents, and community o f student
performance? ■ .
• Was flie effectiveness of Instructional programs evaluated?.
• 'Were there any roadblocks or challenges to making these improvements?
gagsasm
• Are teachers able to effectively analyze performance data and other forms of data?
• Are schools training teachers to improve their ability to use data?
• Are administrators and teaches using standards and rubrics to improve student
■performance? Are teachers executing high-level staadards-based cumcotam?
• Are classroom teachers using data to evaluate the effectiveness of instructional
programs? ' ’
• Are teachers placing a high priority on improving student performance?
• Are students expected to raise fhdr ability and achieve M gfa pafannaaee goals?
• Are classroom teadrersregalarly informing students, parents and community of
student learning performance and how they can assist student improvement?-
D A .U gg.M lcy and Strategy F ir o S ig iP is tic t and School Site)
• How is data use implementation fended?
• ' Does the district control fee budgeting and funding of the programs and strategies, or
do schools have a role? •
® Do all grade levels and subject areas receive equal shares of fending?
• How is the fending affecting curricular and instructional decisions?
• Where is additional funding needed?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
152
Use o f Data in School Study
Stages o f Concern,. Question #1
(Teachers)
Name (optional) ___________________________
In order to identify these data, please give us the last four digits of your Social Security
number:____________________
This is a questionnaire about the district’s design to use student data to improve student
performance. The purpose of this questionnaire is to determine what teachers who are using or thinking
about using the district’s design to use data to improve student learning are concerned about at various
times during the innovation adoption process. A good pan of the items or. this questionnaire may appear
to be of little relevance or irrelevant to you at this time. For the completely irrelevant items, please circle
“O ” cm the stale. O ther items wili represent those concerns you do have, in varying degrees of intensity,
and should be marked higher cm the scale.
Please respond to the items in terms of your present concerns, or how you feel about your
involvement or potential involvement with the district’s design to use data to improve student learning.
We do not hold to any one definition of this innovation, so please think of it in terms of vour perception of
what it involves. Remember to respond to each item In terms of vour present ccocems about your
involvement or potential involvement with the district’s design to use data to improve student learning.
Thank you for taking time to complete this questionnaire.
Please circle the number that best reflects your response to each statement based cm the following rating
scale:
0 1 2 3 4 5 6 7
Irrelevant Not true for me Somewhat true for me Very true for me now
1.1 am concerned about student attitudes toward the district’s design to use student data to
improve student performance.
0 1 2 3 4 5 6 7
2.1 now know o f some other approaches that might work better than the district’s design to
use student data to improve student performance.
' 0 1 2 3 4 5 6 7
3.1 don’t even know what the district’s design to use student data to improve student
performance is.
0 1 2 3 4 5 6 7
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
153
4.1 am concerned about not having enough time to organize myself each day because o f
the district’s design to use student data to improve student performance.
0 1 2 3 4 5 6 7
5.1 would like to help other faculty in their use o f the district’s design to use student data to
improve student performance.
0 1 2 3 4 5 6 7
6.1 have a very limited knowledge about the district’s design to use student data to improve
student performance.
0 1 2 3 4 5 6 7
7.1 would like to know how the implementation o f the district’s design to use student data to
improve student performance would affect my classroom, my position at my school and my
future professional status.
0 1 2 3 4 5 6 7 -
8.1 am concerned about conflict between my interests and responsibilities with respect to
implementation o f the district’s design to use student data to improve student performance.
0 1 2 3 4 5 6 7
9 .1 am concerned about revising my use o f the district’s design to use student data to improve
student performance.
0 1 2 3 4 5 6 7
10.1 would like to develop working relationships with both our faculty and outside faculty
while implementing the district’s design to use student data to improve student performs ace.
0 1 2 3 4 5 6 7
11.1 am concerned about how the district’s design to use student data to improve student
performance affects students.
0 1 2 3 4 5 6 7
12.1 am not concerned about the district’s design to use student data to improve student
performance.
0 1 2 3 4 5 6 7
13.1 would like to know who will make the decisions in the district’s new design to as*
student data to improve student performance.
0 1 2 3 4 5 6 7
2
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
154
34.1 w ould like to discuss the possibility o f using the district’s design to use student data to
improve student performance.
0 1 2 3 4 5 6 7
15.1 would like to know what resources are available to assist us in implementing the
district’s design to use student date to improve student performance.
0 1 2 3 4 5 6 7
16.1 am concerned about my inability to manage all that is required by the district’s design
to use student data to improve student performance.
0 1 2 3 4 5 6 7
17.1 would like to know how my teaching or administration is supposed to change with
the implementation o f the district’s design to use student data to improve student performance.
0 1 2 3 4 5 6 7
18.1 would like to familiarize other departments or people with the progress o f this new
approach to use district’s design to use student data to improve student performance.
0 1 2 3 4 5 6 7
19.1 am concerned about evaluating my impact on students in relation to the district’s
design to use student data to improve student performance.
0 1 2 3 4 5 6 7
20.1 would like to revise the district’s design to use student data to improve student performance.
0 1 2 3 4 5 6 7
21.1 am completely occupied with other things besides the district’s design to use student date
to improve student performance.
0 1 2 3 4 5 6 7
22.1 would like to modify our use o f the district’s design to use student data to improve student
performance.
0 1 2 3 4 5 6 7
3
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
155
23 .Although I don’t know about the district’s design to use student data to improve student
performance, I am concerned about aspects o f the district’s design.
0 1 2 3 4 5 6 7
24.3 would like to excite my students about their part in the district’s use of student data to
improve student performance.
0 1 2 3 4 5 6 7
25.1 am concerned about time spent working with nonacademic problems related to tfafe
district’s design to use student data to improve student performance
0 1 2 3 4 5 6 7
26.1 would like to know what the use o f the district’s design to use student data to improve
student performance.
0 1 2 3 4 5 6 7 . .
27.1 would like to coordinate my effort with others to maximized the effects o f the
district’s design to use student data to improve student performance.
0 1 2 3 4 5 6 7
28.1 would like to have more information on time and energy commitments required by
the district’s design to use student data to improve student performance.
0 1 2 3 4 5 6 7
29.1 would like to know what other faculty are doing in the area o f implementing the
district’s design to nse student data to improve student performance.
0 1 2 3 4 5 6 7
30.At this time, I am not interested in learning about the district’s design to use student data to
improve student performance.
0 1 2 3 4 5 6 7
31.1 would like to determine how to supplement, enhance, or replace the district’s design to
use student data to improve student performance.
0 1 2 3 4 5 6 7
4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
156
32.1 would like to use feedback from students to change the district’s design to use student
data to improve student performance.
0 1 2 3 4 5 6 7
33.1 would like to know how my role will change when I am using the district’s design to
use student data to improve student performance.
0 1 2 3 4 5 6 7
34.Coordination o f tasks and people in relation to the district’s design to use student data to
improve student performance is taking too much o f my time.
0 1 2 3 4 5 6 7
35.1 would like to know how the district’s design to use student data to improve student
performance is better than what we have now.
0 1 2 3 4 5 6 7
36.1 am concerned about how the district’s design to use student data to improve student
performance affects students.
0 1 2 3 4 5 6 7
5
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
157
Data Study
Teacher Questionnaire
Thank you for agreeing to participate in this study. We ensure complete
confidentiality of your valued contribution. As part of this study, your
responses will provide educators and policy-makers in California with much
needed information. Accordingly, please take time to answer each question
carefully and completely by circling the number that best corresponds to
your view.
We would appreciate it if you would provide the following demographic
data for purposes of the study only. Again, complete confidentiality will be
maintained. -
Credential(s)
(Indicate if it is an emergency
credential)
Years o f Experience
Years in current position
Grade level(s) currently teaching
Courses Currently Teaching
(By department only)
Gender
Ethnicity
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Don’t
Know
0
Disagree
Strongly
1
Disagree
Somewhat
2 '
Agree
Somewhat
3
Agree
Strongly
4
Degree o f design implementation of current data practices
1. 1 am aware of the
design for using data.
0 1 2 3 4
2 .1 use data in my classes
on a weekly basis.
0 1 2 3 4
3.1 collect data on a
weekly basis.
0 1 2 3 4
4 . 1 use data to monitor
student progress.
0 1 2 3 4
5 .1 use data to guide my
instruction.
0 1 2 3 4
6 .1 use data to improve
student outcomes.
0 1 2 3 4
7 .1 collect data on test
scores.
0 1 2 3 4
8 .1 collect data on class
participation.
0
1
2 3 4
9 . 1 collect data on student
attitudes.
0 1 2 3 4
10. I collect data. 0 1 2 3 ■ 4
11. My department head
collects data.
0 1 2
" h
4
12.1 use data to compare
the past and present
performance o f an
individual student.
0 1 2 3 4
13.1 use data to compare
students within my class.
0 1 2
n
J 4
14.1 use data to compare
students across the school
in the same grade.
0 1 2 3 4
15. Reports are sent to
parents on a regular basis
(about once a month).
0 1 2 3 4
16. The school completes
reports o f data
implementation for district
databases.
0 1 2 3 4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Don’t
Know
Disagree
Strongly
Disagree
Somewhat
Agree
Somewhat
Agree
Strongly
Degree of design implementation of emerging state data practices
17. The school offers
frequent professional
development to raise
awareness o f new data
practices.
0 1 2 3 4
18.1 have attended
professional development
training in the past six'
months related to new data
practices.
0 1 2 3 4
19.1 frequently discuss
new data practices with
teachers who are about as
experienced as I.
0 1 2 3 4
20.1 frequently discuss
new data practices with
teachers who are more or
less experienced than I
(mentor/ mentee format).
0 1 2 3 4
21.1 frequently discuss
data practices with
teachers in different
disciplines from mine
0 1 2 3 4
22. School administrators
have assisted me in
implementing new data
practices.
0 1 2 3 4
23. School administrators
have monitored my
utilization o f new data
practices.
0 1 2 3 4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Don’t
Know
Disagree
Strongly
Disagree
Somewhat
Agree
Somewhat
Agree
Strongly
Accountability for data use at district, school, and individual level
24.1 think that the state
holds the district
accountable for data
utilization.
0 1 2 3 4
25. I think that the district
holds the school
accountable for data
utilization.
0 1 2 3 4
26. My school holds me
accountable for data
utilization.
0 1 2 3 4
27.1 hold my students
accountable for improved
performance through the
use o f data.
0 1 ■ 2
3 4
28. My salary 1s
dependent upon utilization
o f data practices.
0 I 2 3 ' 4
29. My promotion within
the school is dependent
upon utilization o f data
practices.
0 1 2 3 4
■ Improving student achievement through implementation of date use
30.1 have seen student
achievement visibly
improve when data is used
as a benchmark for
students to reach.
0 1 2 3 4
31.1 have seen student
achievement visibly
improve when I use data
to inform my teaching.
0 1 2 3 4
32. Intervention is more
easily employed through
the utilization o f data.
0 1 2 3 4
33. Student motivation
increases when data is
present in my classroom.
0 1 2 3 4
34. Student motivation
increases through the
dissemination o f data to
parents.
0 1 2 3 4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Intended District Impact:
Observed Impact on School Site:
Researcher’ s rating on how effectively district design improves student performance as
demonstrated in standardized assessment results.
1= not effective 2= somewhat effective 3= unclear 4= effective 5= very effective
Researcher’s Rating:
Use of D ata in Schools Study
Researcher’s Observations and Rating Form, Question #3
District and school accountability to standards-based curriculum
Student Performance Data forwarded to School:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
162
How School is Utilizing Student Data to Impact Student Performance:
Researcher's rating on degree that district provided student data is used by school.
1= not effective 2= somewhat effective 3= unclear 4= effective 5= very effective
Researcher’s Rating:
Use o f Data in Schools Study
Researcher’s Observation and Rating Form, Question #3
Degree to which High Student Performance is aligned to Standards and Communicated
to Teachers, Students and Parents:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Researcher’ s rating on how effectively high student performance is developed throughout
the school learning community.
1= not effective 2 - somewhat effective 3= unclear 4= effective 5 - very effective
Researcher’s Rating: 1 2 3 4 5
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
164
D istrict Interview Guide, Question #3
CFa
• Describe inservices developed by the district to promote the successful
implementation of standards-based instruction.
• What inservices were developed by the district to assist school staff in the use of
student data?
• What standards-based assessment data was forwarded to the school?
• How was student performance data disaggregated for the schools?
• How many of the assessments were based on state standards?
• In what ways is the district helping schools prepare for future changes in state
assessments?
CFb
• Describe state and district incentive plans that are in place to support high-quality
standards-based student performance?
• How often do district personnel review student performance data?
• What corrective actions are taken by the district when student data indicates that
student achievement is not at an acceptable level? Give examples, (vignettes)
• What district strategies are used to determine possible strategies for improving
student performance on assessment instruments?
CFc
• What are the district’s “high” performance goals for students?
• How do district performance standards correspond to state standards?
• Describe district strategies for ensuring that all students master the state standards
for their grade-level.
• In what way does the district communicate to parents about standards-based
curriculum and student assessment results?
• Describe the district process for determining what student data is to be disseminated
to schools.
• What is your overall assessment of student achievement as it relates to state-based
standards.
9
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
165
School Interview Guide-Principal, Question #3
CFa
8 What standardized assessments were forwarded to the school site?
• Which o f the assessments are aligned to state standards?
• What assessments disaggregated student performance data?
• What strategies do you use to assist your teachers in understanding the information in
the assessment data?
• What reflective structures do you promote among your teachers to enhance necessary
changes to instruction based on student performance data? Give examples,
(vignettes)
CFb
• How do you motivate teachers to maintain high energy and enthusiasm towards the
improvement of student performance? (Give examples, (vignettes)
• What methods and instruments do you use to evaluate student performance?
• How frequently does the school evaluate student performance?
CFc
• How are teachers, student and parents made aware of the learning goals o f standards-
based assessments?
• Describe high performance by students as it relates to state standards.
• What strategies are used at your school to ensure that all students achieve levels of
high performance. Give examples, (vignettes)
• What are some of the barriers you perceive to developing a high level o f student
performance in a standards-based environment?
• Describe ways that these barriers can be reduced or eliminated.
School Interview Guide-Leadership Team, Question #3
10
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
166
CFa
• Describe the inservices that were provided to allow you to effectively use student data
as a team.
0 What did the data tell you about how well all student groups are performing?
• Describe the relative effectiveness of individually reviewing student data as compared
with evaluating it in a collaborative team environment.
• How often do you work in collaborative units to evaluate student performance?
CFb
• How do you know how to alter instruction to improve student performance? Give an
example, (vignette)
• How do you determine that your instruction is promoting strong student
performance? Give examples, (vignettes)
CFc '
• What standardized assessments do you use to evaluate student performance?
“ How do you ensure that parents understand standards-based instruction and _
assessments?
• ■ What are the barriers, from a teaching perspective, to making sure that all students
perform at a high level? - •
• What changes would have to made to reduce these barriers?
® How are your report cards based on state standards?
• Describe how you know when a student is performing at a “high” level.
Document Classification, Question #3
II
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
(CFa) Support of Standards-Based Curriculum and Assessment:
District School
(CFb) Student Performance Data Documents:
Standards-based Disaggregated Non/partially Standards-based
* * *
{CFa) Incentive Programs:
District School
* »
(CFd) Reflective Practices:
District School
* *
(CFe) Support for Strong Instructional Improvement:
■School
s is
(CFf) Standards-based instruction o f Parents, Community Members and Students:
Parents CpmnrositJl S c e n ts
District-
School-
12
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Design, implementation and adequacy of using student performance data and the design's link to state context for assessing student performance: A case study

PDF

How districts and schools utilize data to improve the delivery of instruction and student performance: A case study

PDF

A case study: An analysis of the adequacy of one school district's model of data use to raise student achievement

PDF

An analysis of the impact a district design on the use of data has on student performance

PDF

An analysis in the use of student performance data in schools

PDF

An analysis of the elementary principal's role in implementing school accountability within California's high -priority school: A case study

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Los Coyotes High School District

PDF

An analysis of the elementary principal's role in implementing school accountability within California's High Priority School: A case study

PDF

Data: Policies, strategies, and utilizations in public schools

PDF

Implementation of educational policy: Technology implementation in a California middle school

PDF

How effective schools use data to improve student achievement

PDF

Implementation and levels of use of technology plans in a Southern California K--12 district and its middle school

PDF

Implementation challenges of a school district's technology plan at the middle school level

PDF

An analysis of the use of data to increase student achievement in public schools

PDF

How classroom teachers react to and implement California's Beginning Teacher Support and Assessment educational reform policy

PDF

A case study of social promotion and retention policies, strategies, and programs

PDF

An analysis of student -level resources at a California comprehensive high school

PDF

A case study of the role of high school booster organizations in fund raising, governance issues, and personnel decisions

PDF

Districtwide instructional improvement: A case study of Algiers Elementary School in the Orchard Unified School District

PDF

Barriers to implementing distance education: A case study in the community colleges

Asset Metadata

Creator Cantor, Jean (author)

Core Title Design, implementation and adequacy of data utilization in schools: A case study

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, administration,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Marsh, David D. (committee chair), [illegible] (committee member), Picus, Lawrence O. (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-476574

Unique identifier UC11340070

Identifier 3133245.pdf (filename),usctheses-c16-476574 (legacy record id)

Legacy Identifier 3133245.pdf

Dmrecord 476574

Document Type Dissertation

Rights Cantor, Jean

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA