University of Southern California Dissertations and Theses

Evaluation of the effects of a continuous improvement program on special education student achievement

(USC Thesis Other)

Evaluation of the effects of a continuous improvement program on special education student achievement

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content EVALUATION OF THE EFFECTS OF A CONTINUOUS
IMPROVEMENT PROGRAM ON SPECIAL
EDUCATION STUDENT ACHIEVEMENT
by
Cindi Kehoe
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2005
Copyright 2005 Cindi Kehoe
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 3180492
Copyright 2005 by
Kehoe, Cindi
All rights reserved.
INFORMATION TO USERS
The quality of this reproduction is dependent upon the quality of the copy
submitted. Broken or indistinct print, colored or poor quality illustrations and
photographs, print bleed-through, substandard margins, and improper
alignment can adversely affect reproduction.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
®
UMI
UMI Microform 3180492
Copyright 2005 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Signature Page
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ii
ACKNOWLEDGEMENTS
Writing a doctoral dissertation was a personal test o f commitment and
perseverance. I am forever indebted to a significant number of people for seeing me
through this challenging endeavor.
First, I would like to express my great appreciation and thanks to Dr.
Hocevar, my dissertation chair, for nudging me along the way. I will never forget
his great words— “you are finished, send it to Evelyn”—music to my ears. Dr.
Hocevar is an incredibly intelligent, talented man with an uncanny knack for getting
to the point in such an understandable way. I could have never done this project
without his humble guidance.
Next, I would like to thank Evelyn Gray, my wonderful editor for her
assistance in making this paper fit for publication. Evelyn was unfailing in her
attempts to keep me moving toward timelines on this project.
I would also like to thank the other members o f my committee— Dr. Mike
McLaughlin and Dr. Carl Cohen who have given me much inspiration and positive
feedback throughout this process.
Great thanks to all my Redding Cohort pals who prodded and nagged me on a
daily basis to stay focused and get finished. I also want to thank my best friend,
Kathryn, who encouraged me all the way through this project and who continues to
support me in all that I do. And thanks to all my friends who have come and gone
throughout this project whom also encouraged and supported me along the way.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Last but not least, I would like to thank my children, Jami and Robby, for
their encouraging words and moral support. These two have seen me through all my
educational triumphs and have endured a mother glued to her computer keyboard.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
iv
TABLE OF CONTENTS
ACKNOWLEDGEMENTS..................................................................................... ii
LIST OF TABLES..................................................................................................... vi
ABSTRACT................................................................................................................ vii
Chapter Page
1. INTRODUCTION........................................................................................ 1
Problem Analysis................................................................................... 4
Teacher Knowledge and Skills............................................................. 5
T eacher Motivation................................................................................ 11
School Level Factors.............................................................................. 12
Summary.................................................................................................. 16
2. REVIEW OF LITERATURE..................................................................... 18
Stepl: Test Score Disaggregation....................................................... 36
Step 2: Time Line Development......................................................... 36
Step 3: Instructional Focus................................................................... 37
Step 4: Assessment................................................................................ 37
Steps 5 and 6: Tutorials and Enrichment........................................... 37
Step 7: Maintenance.............................................................................. 38
Step 8: M onitoring................................................................................ 38
3. METHODOLOGY........................................................................................ 44
Design Summary.................................................................................... 44
Participants: Sam pling......................................................................... 45
Intervention Description........................................................................ 45
Instrumentation....................................................................................... 46
Quantitative....................................................................................... 46
Qualitative......................................................................................... 48
Procedural Timeline........................................................................ 49
Data A nalysis.......................................................................................... 50
Quantitative Analysis...................................................................... 50
Qualitative Analysis........................................................................ 50
Delimitations of the Study.................................................................... 51
4. RESULTS...................................................................................................... 54
Introduction............................................................................................. 54
Analysis of Mean Differences............................................................... 54
Value Added Analysis............................................................................ 57
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
V
Chapter Page
Qualitative R esults................................................................................. 60
Summary of Interviews......................................................................... 67
5. DISCUSSION..................................................................................................... 72
Summary.................................................................................................. 72
Theoretical Implications of the Study................................................. 76
Practical Implications of the Study...................................................... 78
Further Recommendations.................................................................... 79
Limitations of the Study........................................................................ 81
REFERENCES........................................................................................................... 82
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
vi
LIST OF TABLES
Table Page
1. CST-English/Language Arts Scores (2002-2003).................................... 3
2. Mathematics Scores (2002-2003)............................................................... 4
3. Descriptive Statistics for California Standards Test: English
Language.................................................................................................. 55
4. Tests o f Between-Subjects Effects: California Standards Test:
English Language A rts........................................................................... 56
5. Descriptive Statistics for California Standards Test: M ath.................... 57
6. Tests o f Between-Subjects Effects: California Standards Test:
M ath......................................................................................................... 58
7. Descriptive Statistics— California Standards Test: English
Language A rts......................................................................................... 58
8. Multivariate Tests Effect for the California Standards Test:
English Language A rts........................................................................... 59
9. Descriptive Statistics— California Standards Test: M ath ....................... 59
10. Multivariate Tests Effect for the California Standards Test: M ath 60
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
vii
ABSTRACT
The purpose of this study was to evaluate the effects of a district-wide 8-Step
Continuous Improvement Program designed by Davenport and Anderson (2002) on
special education student achievement in School District A. School District A is a
K-8 school district consisting of one 6th-8th grade middle school, two K-8
elementary schools, and four K-5 elementary schools. Approximately 170 special
education students are served within the district.
California Standards Test scores in the areas of English Language Arts and
Math were compared on the special education subgroup for the years 2002-2003
and 2003-2004 in order to determine if any differences existed as a result of
implementation of the continuous improvement model. Results from that
comparison, indicated no significant differences in CST scores in the area o f English
Language Arts for the special education subgroup from 2002-2003 to 2003-2004.
Results were similar in the area of Math as well, with no significant differences
noted from 2002-2003 to 2003-2004. In general, this study found that the continuous
improvement model had no significant impact on the special education subgroup
during the first year o f implementation.
Additional statistical procedures in the form of a repeated measures analysis
were used to determine if individual students made significant gains on the CST for
English Language Arts and Math between the 2002-2003 and 2003-2004 school
years. Results of that analysis again indicated that no significant gains were made by
individual students in any of the grade levels examined by this study (Grades 2-7).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
viii
In the case of Math, students in Grades 2 and 4 actually posted negative gains,
though they were not statistically significant.
A qualitative analysis was also conducted in the form of informal interviews
with site level administrators in order to determine levels and quality o f implement-
tation and inclusion of special education students at individual school sites. Results
of those interviews indicated that all schools within School District A are
implementing the 8-Step Continuous Improvement Model in some form, though
implementation is not uniform across the district.
Findings of this study indicated that the continuous improvement program at
its current level of implementation, had no effect on special education student
achievement from the 2002-2003 to 2003-2004 school year. Findings further
indicate that the district should develop a well-defined protocol to follow for
implementation of each step of the improvement process. Additionally, it is
recommended that the district consider increasing inclusion of students with mild
disabilities in the general education classroom during standards-based instruction.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1
CHAPTER 1
INTRODUCTION
The school accountability movement generated by the No Child Left Behind
(NCLB) legislation has brought forth many problems facing the current system of
public education. As schools disaggregate data generated by standardized testing,
achievement gaps are being identified between specific groups of student popula
tions. Achievement gaps exist between ethnic groups, as well as gaps between
specific ethnic groups and students of low socioeconomic status. To further compli
cate matters, No Child Left Behind legislation mandates that special education
students be held to the same standards as students without disabilities, and that they
too become proficient on standards set by individual states.
School districts nationwide are rising to the challenge o f NCLB by imple-
mentting various forms of programs designed to improve student learning and
achievement. This dissertation project will be an evaluation of one such program’s
effectiveness in terms of special education student achievement in one specific
school district. School District A recognizes the challenges resulting from compli
ance to NCLB, and has taken proactive measures to ensure that its schools are
equipped with the knowledge and skills necessary to improve student learning.
School District A is a K-8 school district consisting of one 6th-8th grade
middle school, two K-8 elementary schools, and four K-5 elementary schools. The
district has a student population of approximately 3,800 students, and special
education students comprise nearly 10% o f that population. Special education
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2
students are served in a number of ways in the district, including full inclusion of
students with mild disabilities, pull-out resource services, and special day classes.
The district has a resource program at each school site, and the middle school
employs one resource teacher for each of the three grade levels. Resource programs
vary from school-to-school, and all provide some degree of pull-out services for
students. Special Day Classes exist on four of the district school sites, including the
middle school, one K-8 school, and two K-5 elementary schools. The typical Special
Day Class has a student population of no more than 16, and disabilities range from
mild to moderate in severity. Severely handicapped students are served in county
programs.
School District A recently provided a district-wide in-service to all district
schools outlining an 8-Step Continuous Improvement Program designed by
Davenport and Anderson (2002) to improve student performance on state and
federally mandated assessments as required by NCLB. All schools within the district
are in the beginning phases of implementing continuous improvement plans, and all
are using the model as presented in the recent in-service. The 8-Step Continuous
Improvement Program consists of the following (Davenport & Anderson, 2002):
1. Disaggregation of state standards data in order to identify specific
instructional groups.
2. Development of an instructional timeline to align curriculum with
state performance standards.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3
3. Development of “instructional focus” to guide classroom instruction
in critical state standard areas.
4. Frequent, ongoing assessment to determine effectiveness of
instruction.
5. Provision of tutorials to students failing to master standards through
classroom instruction.
6. Provision of enrichment activities for students who quickly master
standards to provide academic challenge.
7. Reinforcement of previously learned materials.
8. Monitoring of the effectiveness of instruction by the school leader.
Problem Description
A preliminary analysis of disaggregated data on special education students
indicates that many schools in the district report “below basic” or “far below basic”
performance on the California Standards Test (CST). The Tables 1 and 2 illustrate
the current status of performance for each grade level (2n d through 8th):
Table 1. CST-English/Language Arts Scores (2002-2003)
Grade 2 3 4 5 6 7 8
# tested 25 35 38 53 42 37 39
Mean scaled score 302.3 283.2 327.1 310.9 322.1 296.0 290.2
% advanced 8 0 8 4 7 3 3
% proficient 8 6 11 11 15 5 3
% basic 16 23 13 19 15 8 23
% below basic 32 20 16 21 24 32 31
% far below basic 36 51 53 45 39 51 41
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4
Table 2. Mathematics Scores (2002-2003)
Grade 2 3 4 5 6 7 8
# tested 25 35 38 53 42 37 37
Mean scaled score 340.7 291.1 320.7 290.9 321.3 287.0 285.8
% advanced 20 6 13 2 7 5 0
% proficient 12 6 5 11 17 3 5
% basic 24 29 16 15 15 14 22
% below basic 28 17 29 38 37 32 32
% far below basic 16 43 37 34 24 46 41
An evaluation of the effectiveness of the 8-Step Continuous Improvement
Program, is necessary in order to determine if positive outcomes result for all
students. Evaluation of the impact of this program on special education student
achievement, is critical in order to ensure that positive growth is made towards
compliance to mandates specified in NCLB in regard to the special education
subgroup. Ultimately, movement of special education students from the “Below”
and “Far Below Basic” categories to “Basic” or above, would suggest that improve
ment strategies are indeed effective for this specific subgroup. Results from this
evaluation, will be useful to school leaders in order to determine if general improve
ment strategies are effective for special education students, or if additional strategies
are needed to increase learning for this subgroup of students.
Problem Analysis
Many factors contribute to the performance gap between general education
and special education students. An analysis of the gap can be performed through
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5
evaluation of three important factors: (a) teacher knowledge and skills, (b) teacher
motivation, and (c) organizational factors.
The responsibility of general education classroom teachers in the education
of special education students in general curriculum areas is becoming increasingly
important in the age of accountability. Marzano (2003) cites research supporting
the fact that the classroom teacher is by far the most critical factor in student
achievement (pp. 72-75). Given this fact, it seems worthy to evaluate the special
education student achievement gap in terms of the classroom teacher, considering
both the knowledge and skills necessary to teaching special needs children, as well
as teacher motivation to take responsibility for the learning of special needs children.
Teacher Knowledge and Skills
Marzano (2003) discusses teacher knowledge and skills in terms of three
main characteristics, “instructional strategies, classroom management, and classroom
curriculum design” (p. 71). Marzano further asserts that teachers who are competent
in these three areas, even if merely “average,” can have a significant positive effect
on student learning (p. 75). A preliminary analysis of the problem will be made in
light of these three areas, beginning with classroom instructional strategies.
Marzano (2003) identifies nine specific instructional strategies proven
through research to have positive effects on student learning. In School District A,
many teachers may effectively use these strategies with their general education
students. It is not clear, however, that teachers understand that these strategies must
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6
be used more explicitly with special education students, and that specific modifica
tions to the strategies may be necessary.
Marzano’s (2003, p. 80) nine strategies include the following:
1. Identifying similarities and differences
2. Summarizing and note taking
3. Reinforcing effort and providing recognition
4. Homework and practice
5. Nonlinguistic representations
6. Cooperative learning
7. Setting objectives and providing feedback
8. Generating and testing hypotheses
9. Questions, cues, and advance organizers
Through observation and interview of site leaders, special education teachers,
and general education teachers, several themes emerge in terms of teacher use of
effective instructional strategies with special education students. First, special
education teachers are far more experienced in modifying instruction for special
education students than are general education teachers. This creates quite a dilemma
in terms of achievement for special education students, as a great deal of time is
spent in the general education classroom with the expectation that they will learn
from the same instructional strategies that are used with students without learning
problems.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7
Another problem involves the fact that a general assumption is made that
most, if not all teachers know about and are competent in using all of the nine
strategies as defined by Marzano (2003). At this point, School District A has not yet
analyzed the degree to which teachers understand and effectively use proven
instructional strategies. Many teachers may show competency in using one or
several of the strategies, but use of all nine in a systematic, consistent way cannot be
verified at this time. Observation of classroom teaching, as well as comments made
by teachers in student study team meetings, certainly suggests that many teachers in
the district lack knowledge of what is proven to work, and that many teach using
styles that feel “comfortable” or familiar to them.
The most critical lack of knowledge and skill is evident in the absence of
knowledge on the part of classroom teachers in understanding individual difference
in learning, specifically when dealing with a student with special needs. Individual
differences require that instruction be individualized to meet the specific needs of
that individual. It is one thing to tell a classroom teacher that he or she must
individualize for a particular student, and another thing altogether to assume that that
teacher understands how to do it. Judging from the resistance many teachers display
in response to seemingly helpful suggestions from support staff (i.e., special
education teachers, speech therapists, occupational therapists, and psychologists), it
seems likely that teachers are not at all clear as to what they are suppose to do
differently.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8
In summary, special education teachers appear to have the knowledge and
skill required to deliver adequate instruction to special education students. However,
special education teachers in School District A generally do not deliver the entire
academic curriculum to special education students. Students spend a great deal of
time in the general education environment with teachers who may not necessarily
understand the learning needs of the student nor have the skill to individualize
instruction to meet those needs. The 8-Step Continuous Improvement Program
implemented in School District A provides teachers with specific instructional
strategies which should provide useful to general education teachers required to
teach standards-based curriculum to special education students. This program
evaluation may be useful in determining if the use of proven strategies is effective
for the special education population.
The second area of teacher knowledge and skill focuses on the use of
effective classroom management strategies. Marzano (2003) suggests that there are
four areas of classroom management that are essential to teacher knowledge and skill
as follows:
1. Establishing and enforcing rules and procedures
2. Carrying out disciplinary actions
3. Teacher and student relationships
4. Maintaining an appropriate mental set
School District A has implemented a positive discipline program district-
wide emphasizing consistent classroom and school-wide rules focused on “respect,”
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9
“responsibility,” and “safety.” Staff are in process of being trained to use positive
discipline strategies, including teaching rules systematically and explicitly, providing
a great deal of positive rewards for appropriate behavior, and structuring the class to
facilitate positive relationships between teachers and students.
Further, School District A utilizes positive behavior support plans for special
education students who display significant behavior problems that interfere with
learning. A critical component of the support plan involves the use of teaching
strategies and environmental supports designed to improve the learning environment
in order to support the student in using more appropriate behaviors. Within the area
of environmental support, students are provided certain modifications to the environ
ment specific to their needs. For example, a student may be acting out in response to
an inability to complete a difficult assignment in the amount of time given. This
student may have a specific modification to the length of time given in order to
reduce or eliminate the student’s need to avoid the assignment. These two compon
ents are used effectively in all schools within the district and appear to meet the
criteria for classroom management as defined by Marzano (2003).
Classroom curriculum design is the final area Marzano (2003) refers to as
essential to teacher knowledge and skill. Marzano speaks of the “sequencing and
pacing” of curriculum to enhance the learning of students (p. 107). He identifies
three specific “principles” essential in the effective curriculum design as follows
(pp. 109-116):
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
10
1. Learning is enhanced when a teacher identifies specific types of
knowledge that are the focus of a unit or lesson.
2. Learning requires engagement in tasks that are structured or are
sufficiently similar to allow for effective transfer of knowledge.
3. Learning requires multiple exposure to, and complex interactions with
knowledge.
Currently, School District A has adopted a specific language arts and
mathematics curriculum and mandated their use in every classroom. A preliminary
analysis o f teacher use o f the curriculum reveals inconsistent use as well as
inconsistent delivery of lessons. Teachers who do use the curriculum, do so in
varied ways. For example, some teachers follow the sequence of the lessons strictly,
with no deviation. This type of curriculum sequencing is often too fast paced for
special education students, leaving them with gaps in learning, and little or no time
to practice learning concepts to mastery. Teachers share that they feel pressured to
cover all curriculum, as “that is what the students are being tested on,” and are not
necessarily knowledgeable about how to align the curriculum to the essential
standards covered on the California Standards Test.
The instructional calendar and focus lessons provided in the 8-Step
Continuous Improvement Program assists teachers in aligning district-adopted
curriculum to state standards. The instructional calendar also allows teachers to
sequence and pace curriculum as well. This program evaluation will be useful in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
11
determining if the sequencing and pacing provided to general education students is
also effective for the special education population.
Teacher Motivation
Motivation in terms of the willingness of classroom teachers to take
responsibility for teaching children with special needs also deserves an analysis in
regard to the achievement gap between general education and special education
students. Clark and Estes (2002, p. 80) discuss the issue of motivation in depth and
identify three “motivational processes” likely at work within the learning
environment:
1. Active choice to pursue a goal
2. Persistence at a specific goal, without being distracted by less
important goals
3. Mental effort put forth to achieve a goal
In School District A, classroom teachers vary in their motivational levels
when it comes to the issue of including special education students in the general
education classroom. The issue of motivation, or “buy-in,” permeates the district,
not only in relationship to the achievement gap of special education students, but the
motivation to embrace the changes in educational practice that are a result of NCLB.
Veteran teachers insist that the accountability movement is yet another “fad,” likely
to disappear in a few years. These teachers pay lip service to mandated curriculum,
and continue using curriculum they have deemed viable due to their “observation”
that students have learned just fine in the past.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
12
In terms of inclusion of special education students, teachers commonly
comment that they do not have the time to slow down curriculum, or that it is the
responsibility of the special education teacher to make sure special needs students
achieve. In other words, many teachers within School District A have not yet bought
in to the idea that improving student achievement means the achievement of every
student, including those with special needs.
Two specific problems in regard to motivation will be addressed in this
program evaluation. Informal interviews with site level principals will address
teacher’s willingness to implement the 8-Step Continuous Improvement Program, as
well as the extent to which teachers include special education students in general
curriculum instruction.
School Level Factors
Marzano (2003) identifies five school level factors critical to successful
program implementation. Three of these areas will be analyzed in order to determine
strengths and weaknesses of the organization. The first school level factor important
to analyze is the notion of a “guaranteed and viable curriculum” as defined by
Marzano (p. 22-34). Marzano cites various researchers who found that there is a
“discrepancy” between the curriculum “intended” for implementation and the
curriculum that is actually implemented (p. 23). In School District A, this is indeed
an issue, as inconsistencies have been identified across the school district in teacher
use of the mandated curriculum. A strength of School District A is represented by
the fact that School District A has adopted and mandated the use of a standards-
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
13
based curriculum district-wide. Inconsistent use of the curriculum across the district
on the other-hand represents a critical weakness for the organization. The 8th step of
the Continuous Improvement Program requires that site leaders monitor teachers in
order to ensure that standards-based lessons are being taught in the classroom. An
element of this program evaluation will be to determine if site level leadership is
following this component of this program.
Another aspect of the curriculum area involves the amount of time allowed in
a day to cover the intended curriculum. Schools vary in their use of block time to
cover essential standards, and the variance is even more pronounced when compare-
ing specific classrooms within a school. An analysis of School District A reveals
that some schools have developed “intervention” periods within the school day to
allow extra time to teach students who are lagging behind in core curriculum areas.
Other schools have implemented after school tutoring programs to assist students
who are in need of additional instructional time. Intervention blocks and after school
tutoring programs are both areas of strength for the organization, yet problems do
exist in these models, mainly when addressing the issue of after school programs.
Often parents are required to transport students to these programs, which at times can
be logistically impossible for the family. One way that some schools have dealt with
this problem is to add the tutoring program to the after school program, which allows
parents to access day care as well as provide extra instructional time to the student.
Again the weakness of the organization is represented by the fact that not all schools
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
14
utilize after school programs in a manner, which provides additional instructional
time to struggling students.
“Challenging goals and effective feedback” is another factor important to a
successful school. At this time, it is unclear that all schools within School District A
have specific learning goals that are aligned to the standards and curriculum, or that
there is consistent method for providing feedback to the students on their progress
toward the goals. One strength that the district displays is the fact that some schools
have begun the process of implementing a continuous improvement plan, and have
developed instructional calendars that specify what standards are being taught and
when. Again the weakness of the organization is in the inconsistencies across
schools in buying in and implementing a school-wide calendar, leaving the goals and
feedback mostly to individual teachers to develop and implement.
The final area o f analysis involves the “collegiality and professionalism” that
exists within the school district (Marzano, 2003, p. 60). Marzano defines collegiality
as “the manner in which teachers interact with one another” (p. 61). Marzano further
cites research that shows positive correlations between teacher interaction and
student achievement when teachers engage in discussion and advice giving. Friend
ship on the other hand had little or no impact on student learning (p. 62). One
strength of School District A is the recent addition of the Professional Development
Center. This center has allowed teachers to engage in professional dialog and to
share with one another successful teaching strategies and monitoring techniques.
The extent to which teachers utilize the center is not well documented, and may
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
15
deserve closer examination. Again, within school sites, collegiality varies, some
schools having successful grade-level teams and others having little or no sharing of
knowledge between teachers.
Leadership is another important factor contributing to a climate of college-
ality and professionalism within the school. In order to foster collegiality and
professionalism among the staff, the leader must provide time for discussion and
sharing, and allow the staff to actively participate in instructional planning.
Instructional planning time and collegial staff meetings are an area of strength for
School District A in several ways. First of all, one school provides a weekly
minimum day in order to provide staff time to plan and collaborate. Other schools
have weekly or biweekly grade-level meetings where staff is expected to collaborate
on curriculum and instruction. External threats exist in providing extra time within
an instructional day for staff to collaborate. For example, parents are often opposed
to minimum days, as it causes childcare problems. Another threat is the reluctance
of teachers to give up lesson planning time in order to collaborate with others on a
weekly or biweekly basis.
Many factors impact the achievement of special education students in School
District A. Teachers lack the knowledge and skill in instructional strategies indi
vidualized to meet the learning needs o f special education students, and many lack
the motivation to include these students in the general curriculum instructional
program. Organizational factors contribute to the gap as well, including inconsistent
use of standards-based curriculum across the district, and a variance in the amount of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
16
time special needs students are given to access and master the curriculum. Leader
ship and the provision o f time for collaboration is also a contributing factor to the
achievement gap of special education students.
Summary
In summary, special education students in School District A perform within
the “below basic” to “far below basic” levels on the California Standards Test.
Many factors appear to contribute to the achievement gap between general and
special education students in the district. Factors include the lack of teacher
knowledge and skill in understanding the needs of special education students as well
as the lack of consistency in the use of proven teaching strategies. Teacher motive-
tion is also a factor in the achievement of special education students, as teacher “buy-
in” of the 8-Step Continuous Improvement Program is not yet realized in the district.
Overall, consistent use of standards-based curriculum as well as adherence to
instructional timelines is at issue in the district as well.
The goal of this program evaluation is to determine the impact of the 8-Step
Continuous Improvement Program on achievement of the special education popu
lation in School District A. It would seem likely that if individual learning needs are
being addressed through instructional strategies and tutorials provided by the pro
gram, the impact could well be positive for that population of students. In order to
determine the effectiveness of the program, the following research question will be
addressed:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
17
Does the 8-Step Continuous Improvement Program currently in use in School
District A significantly improve special education student achievement?
Results generated from this study, will provide useful information to site-
level leaders in regard to the impact of the Continuous Improvement Program on
special education subgroups. Results from this study, may also assist leaders in
assessing their school’s level of implementation and degree to which special
education students are included in critical standards-based curricular lessons in the
classroom.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
18
CHAPTER 2
REVIEW OF LITERATURE
Continuous improvement is an important topic in education given the current
focus of student achievement on standards-based assessments. General education
student populations are typically the target of continuous improvement programs,
and much research exists to support elements critical to improving academic perfor
mance in the mainstream classroom. While the effects of improvement programs on
the general student population is important, there also exists the need to evaluate the
effectiveness of these programs on special needs populations as well.
According to a publication by the American Youth Policy Forum (Kober,
2002), there is both good and bad news in regard to the current status of special
education student achievement. Kober asserts that there are three overarching areas
of positive change that has happened over the past 25 years serving to improve the
education of special needs students. First, special education students now realize
adequate access to the public education system. More special education students are
being educated in classrooms with nondisabled peers, and the advent of early identi
fication and intervention programs has greatly improved disabled student academic
success (p. 3).
Kober (2002) further asserts that while IDEA has been the catalyst for
inclusion of special populations (making inclusion legally mandatory), a general
“change in attitude” by parents and teachers alike has evolved (p. 3). As the
awareness of disabilities and what the rights of students with disabilities are has
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
19
risen, educational personnel have become more willing to include students with
disabilities and to design programs aimed at improving academic performance for
these students as well.
Kober (2002) makes a final point that school personnel now have more skill
and knowledge in regard to how to teach students with disabilities, and that special
education teachers are now “better integrated into school operations” as well (p. 3).
School leaders have also increased their knowledge in regard to the educational
rights of students with disabilities, which creates the potential for a cultural shift in
school-wide acceptance of the responsibility of educating special needs populations.
While the advances made to improving educational access o f special needs
students is impressive, much work remains in the area of academic improvement for
special education populations. Kober (2002) identified three main themes in regard
to the future direction of educational improvements for special needs students. First,
given the fact that many special needs students are now included in general educa
tion classrooms, it has become critical to examine the quality of academic instruction
that these students receive from mainstream teachers. Kober also makes an
important point that effective education for special needs students will ensure that
students graduate from high school and that they are well-prepared to attain higher
education or seek adequate employment as adults (p. 7).
Kober (2002, p. 7) further notes the impact of high-stakes testing on the
education of special needs students. No Child Left Behind legislation mandates that
all students be held to the same level of accountability on standards-based
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
20
assessments regardless of disabilities or other disadvantages experienced by many
public education students (including poverty and ethnicity). In order for special
education students to achieve at high levels, an inordinate amount o f intervention and
support may be necessary to ensure that these students master the same content as
their non-disabled peers.
A final point made by Kober (2002), focuses on the political aspects of the
federal mandates put forth by IDEA. Kober notes that the legal requirements of
IDEA place an extra burden on educators in the form o f “paperwork” and time (p. 7).
Consequently, this takes away from time that could be spent planning and training
teachers to better educate students with disabilities.
The issue of mainstreaming or inclusion of special education students in the
general education classroom has been the topic of conversation and research since
the revision of Public Law 94-142 in 1991 (subsequently renamed Individuals with
Disabilities Act—IDEA). Tompkins and Deloney (1995) noted that the definition of
inclusion varies widely, as educators have different notions of what that looks like
depending on their areas of expertise. Originally, the term “mainstreaming” was
used interchangeably with the notion of inclusion, and carried with it the connotation
that disabled students could “mingle” with general education students during non-
academic learning times.
Eventually the term “integration” grew out o f the notion of mainstreaming,
and suggests something different entirely than merely being around non-disabled
peers. Tompkins and Deloney (1995) point out that the term “integration” has its
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
21
roots in “civil rights/desegregation legislation of the 1960s and before.” Thus, the
authors assert, the term is tied more to legal requirements as defined by civil rights
law as opposed to being primarily a philosophical notion. Integration subsumes that
students with disabilities should not only be exposed to non-disabled peers in a
“physical” sense, but that they also be included in academic and social interactions
as well.
In both of the above definitions, students with disabilities are primarily the
responsibility o f special education teachers in the areas of academic learning. While
students may be mainstreamed in general education settings or participate with non
disabled peers in certain academic subjects, the responsibility for remediation or
modification of the general curriculum rests primarily with the special education
teacher mainly through the resource program or self-contained classrooms. The term
“inclusion” suggests something entirely different, however, and shifts more
responsibility to the general education teacher in terms of educating the included
special education student.
Tompkins and Deloney (1995) make several important points in relationship
to the definition of “inclusion.” First, the authors note the importance of bringing
services to the child rather than pulling the child from the natural setting in order to
provide services. Secondly, the issues of assessment and collaboration become
important. Tompkins and Deloney note the responsibility of the general education
teacher in knowing where each child performs academically, regardless of whether
that child has a disability or not. Further, in order to design adequate academic
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
22
programs, the general education and special education teachers must work cooper
atively in the academic interest o f the student.
The term “full inclusion” is likely the point of most controversy between
parents and general and special education teachers. Tompkins and Deloney (1995)
note that there are differing opinions and definitions of “full inclusion,” just as
mainstream, integration, and inclusion carry with it different definitions. The
authors note the perspective of “absolutists” who unequivocally believe that all
disabled students must be included in the general education environment for all
activities, academic and social. The underlying assumption given the absolutist
perspective is that adequate academic and social supports already exist in the general
education environment in order to ensure success for the student. Unfortunately, this
is a huge assumption that is not always realized due to many factors, including
budget constraints, lack o f skill, and lack of knowledge of disabilities and how
disabled students learn. Tompkins and Deloney note that nonabsolutists contend that
all factors must be analyzed prior to making a blanket decision to fully include a
special needs student.
Tompkins and Deloney (1995) cite research by Jay Heubert, which asserts
agreement on several issues surrounding inclusion. Both sides agree that students
can be successfully included given adequate staff development, collaborative support
between general and special educators, and adequate administrative support.
Teacher perception is also an important factor in successful inclusion practice.
Tompkins and Deloney cite research from the American Federation of Teachers
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
23
indicating that the majority of teachers polled, do not believe that students receive
full educational benefit from full inclusion practices. Tomkins and Deloney (1995)
note the main concerns expressed by teachers as follows:
1. Increased focus on a few
2. Increase in behavioral difficulties in the classroom
3. Lack of appropriate training and support
4. Hidden motivation due to funding issues
The notion of inclusion brings with it much disagreement, yet special educa
tion law makes inclusion mandatory for most special needs students. Access to core
curriculum is inherent in the mandates of IDEA, and student IEP’s must reflect to
what degree this is realized by each student. Agran, Alper, and Wehmeyer (2002)
designed a study to assess the extent to which special education students actually had
access to core curriculum. Teachers of moderately to severely handicapped students
were polled to determine the “type and degree of access to the general education
curriculum” for these students. Within this population, the majority o f students were
integrated with non-disabled peers 2-3 days per week or less. Further, students
worked mainly with a paraprofessional rather than participating with peers within the
general education classroom (p. 127).
O f greatest concern in the Agran et al. (2002) study was the finding that
many districts “had no clear plan for ensuring access to the general education
curriculum for students with moderate to severe disabilities” (p. 128). Further,
significant roadblocks were identified in the form of “resistance from general
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
24
educators and students’ challenging behaviors” (p. 129). The Agran et al. study
indicated that while many moderately to severe students participate with non
disabled peers at least on a weekly basis, no specific goals were outlined to deter
mine how each student will access core curriculum. Teachers in this study also
overwhelmingly indicated that access to core curriculum is most important for
students with mild disabilities (p. 129).
Significant advances have been made over time in the provision of adequate
education for students with special needs. Special education students now partici
pate alongside their non-disabled peers in many aspects of the educational setting.
Not only are special needs students included in general education class-rooms, but
they are also required to participate in the standards-based assessments currently
used to evaluate student progress and the effectiveness of our schools. Given this
fact, it becomes important that educators pay close attention to the individual needs
of these students in order to provide instruction and learning experiences that will
enhance performance on standards-based measures.
Effective instructional strategies are a critical piece in educating all children,
and are even more critical when considering the unique learning needs of special
education students. In his book, Classroom Instruction That Works, Marzano (2001)
identifies nine specific instructional strategies that have been proven effective
through research. It stands to reason that these strategies may be effective for special
populations as well, and should prove useful to general education teachers when
delivering standards-focused lessons class wide.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
25
Marzano (2001) states that at the “core of all learning” is the ability of
students to be able to identify similarities and differences in materials they are
required to learn (p. 14). Marzano cites research showing significant effect sizes for
academic success in students who have been taught to identify similarities and
differences in a variety of ways. Four specific ways of teaching this skill were
identified by Marzano (2001, pp. 15-16) as follows:
1. Direct teach similarities and differences in relevant materials
2. Encourage students to identify similarities and differences on their
own
3. Use of “graphic or symbolic” representations of similarities and
differences (p. 16)
4. Use of variety of forms, including metaphors and analogies
Marzano (2001) identified “summarizing and note taking” as the second skill
necessary for learning mastery (p. 29). Marzano cites research supporting this
technique and describes three main themes relevant to the summarizing and note
taking strategy. When students are presented with novel material that they are
required to master, Marzano suggests that in order to organize the material and make
sense of it, one may “delete some information, substitute some information, and keep
some information” (p. 30). This technique can be directly taught and illustrated to
students regardless of their learning style or level of cognitive functioning. Another
critical element of this strategy is the necessity to “analyze the information at a fairly
deep level” (p. 31). Marzano cites research suggesting the ability to analyze material
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
26
at a deep level resulted in the largest effect size in studies of effectiveness of
summarizing (p. 32). The final area important to summarizing is the student’s
ability to understand the structure of the written work (i.e., introduction, body, and
conclusion) (p. 32).
The third strategy important to effective instruction involves the issue of
reinforcement and feedback. Marzano (2001) explains that “effort” is the critical
element in student achievement, and that most students do not understand the
concept of effort, nor do they understand that they have the power to change their
level of effort (p. 50). Another important factor in effort is the ability of students to
understand the “connection between effort and achievement” (p. 52). Marzano
suggests direct teaching of the concept of effort as well as providing extrinsic and
symbolic reinforcement in order to assist students in making the connection between
effort and achievement (p. 52).
A fourth element in effective instruction is the issue of practice. Marzano
(2001) uses the example of homework as an effective means of providing additional
practice to students to solidify learning. Marzano (2001) cites research by Cooper
(1989) suggesting that the impact of homework on academic success increases with
grade level. Elementary grades showed the lowest effect size (.15) compared to the
highest in high school (.64) (p. 61).
The fifth element of effective instruction can be defined as the use of visual
representations of linguistic materials. Marzano (2001) refers to the use of visual
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
27
representation as “nonlinguistic representations” (p. 73). Two specific areas are
important for teachers to consider when using nonlinguistic modes o f teaching:
1. Use of a variety of activities (i.e., graphic representations, physical
models, visualization, pictures, and kinesthetic activities) (pp. 73-74)
2. Use of nonlinguistic representations to “elaborate on knowledge”
rather than to teach new knowledge (p. 74)
Goal setting is another key element in effective instruction according to
Marzano (2001). Marzano cites three studies showing significant effect sizes of the
impact of goal setting on student achievement (p. 93). Marzano explains that there
are three key elements that contribute to the effectiveness of goal setting on
academic performance. First, Marzano suggests that “instructional goals narrow
what students focus on,” making it easier for students to determine what is important
to know when learning new concepts (p. 94). Secondly, Marzano cautions that goals
not be “too specific” as to inhibit “the individual and constructivist nature of the
learning process” (p. 94). Finally, Marzano states the importance of students making
the goals their own by “personalizing” goals identified by the teacher (pp. 94-95).
As students master instructional strategies at the basic level, movement into
higher cognitive levels of knowledge construction becomes necessary. Marzano
(2001) states that “generating and testing hypotheses” can be described as the “most
powerful and analytic of cognitive operations” (p. 104). This method of teaching is
used to apply previously gained knowledge in order to elaborate or expound upon
current knowledge. Marzano (2001) describes two critical elements involved in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
28
teaching students to generate and test hypotheses. First, Marzano states that the use
of “deductive” and “inductive” thinking is important (p. 104). Deductive thinking
requires the student to make predictions based on what he or she may already know
about some concept. Inductive thinking is a process by which a student may gain
new knowledge about a particular concept already understood in another way
(pp. 104-105).
The final strategy discussed by Marzano (2001) is the use of questioning by
the teacher in order to stimulate critical or creative thinking in students (p. 113).
Marzano identifies four elements important to effective use of cues or questions by
the teacher:
1. Questions should be relevant to what is being taught
2. Use of “higher-level” questioning will stimulate “deeper learning”
(p. 113)
3. Effective use of “wait-time” prior to expecting responses from
students will lead to deeper thought on the part of students
4. Use questions prior to teaching a concept in order to “front-load”
learning (p. 113)
All of the above techniques outlined by Marzano (2001) are effective
instructional strategies when used with all types of learners. To effectively use these
strategies with special needs students, modifications may be necessary to adequately
accommodate for student learning strengths and weaknesses. Coyne, Kame’enui
and Simmons (2004) note, “In both cases of inclusion and of standard-based
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
29
accountability, efforts designed to benefit all students have had the effect of
jeopardizing the fundamental mission of special education” (p. 232). The authors
assert the importance of early intervention programs, particularly in the area of
reading in order to reduce the incidence of reading disabilities. The authors further
suggest a “school-wide reading improvement program” with emphasis on all
students, but also “providing individualized instruction for students with reading
disabilities and students at risk for reading failure” (pp. 233-234).
Components of an effective school-wide plan to improve early reading skills
are outlined by Coyne, Kame’enui, and Simmons (2004), and begin with “long-term
reading goals and intermediate performance benchmarks for all students” (p. 234).
Assessment of student progress toward benchmark goals allows educators to
distinguish between students with strong reading skills and those who are at risk of
failure in the early academic years (Kindergarten to 1st grade).
Coyne et al. (2004) identifies assessment as the second important component
of a school-wide improvement plan. The authors suggest use o f a “school-wide
assessment system that monitors both student and school performance in beginning
reading” (p. 234). By assessing student progress at intervals throughout the school
year, educators are able to “identify the strengths and weaknesses of their school-
wide reading practices,” as well as “allow schools to respond to the shifting and
evolving needs of their student population” (p. 234). Analysis of assessment data
also assists teachers in deciding interventions appropriate to strengthen weak areas of
performance in their students.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
30
Instruction is the third component identified by Coyne et al. (2004) as
“critical” to a school-wide improvement program (p. 236). Instruction must be in the
form of a “continuum,” supporting students of all performance levels (p. 236). The
authors note that a core curriculum in reading, is necessary to address the needs of all
students, and that “supplemental programs extend the core program by providing
students with additional instruction in identified strategic skill areas” (p. 236).
Coyne et al. further defines the use of “intervention” programs as those that “provide
intensive support for students who are struggling to acquire certain beginning
reading skills” (p. 236). In the model proposed by Coyne et al., all students are
exposed to core curriculum, and additional supports are provided to students in need
of specific skill building or more intensive intervention.
Another important component imbedded in the instructional piece involves
the issue of scheduling. Coyne et al. (2004) suggest that schools need to set aside a
specific block o f time to focus primarily on reading, which will allow adequate
instructional and practice time. They further assert that scheduling should occur on a
school-wide basis “to ensure efficient use of staff and resources” as well as to allow
for “creative, flexible, and effective grouping practices” (p. 236).
Once the first three components are in place, “ongoing progress monitoring”
becomes important (Coyne et al., 2004, p. 236). Use o f a school-wide assessment
screening tool allows for early identification of reading difficulties. Coyne et al.
notes that without the use of a screening device, teachers are left to notice the
struggling readers after the problems are already set in motion (p. 236). By catching
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
31
difficulties early, teachers have the ability to apply more intensive interventions and
instruction to struggling readers, improving chances for later reading success.
Coyne et al. (2004) suggest another important element in ongoing program
monitoring as allowing teachers to monitor not only class wide how students are
performing, but to track individual student progress as well (p. 236). The authors
also suggest using ongoing performance data to compare individual student progress
to the group as a whole. Comparisons such as these allow for early identification of
students who fail to make adequate progress toward the benchmark for the norm of
the group (p. 236).
Coyne et al. (2004) make several important points on the matter of reading
instruction for learning disabled students. First, the authors cite research that
supports the notion that “students with reading disabilities do not require reading
instruction that is qualitatively different from effective reading instruction for
students without disabilities” (p. 237). Rather these students “require instruction that
is significantly more intensive, systematic, and sustained” (p. 237), suggesting that
instruction be skill specific and time intensive. Therefore, scheduling, and allowing
adequate time for explicit instruction as well as student practice becomes paramount.
Coyne et al. (2004) suggest a variety of models be used to provide the extra
instruction and practice to struggling readers. Pull-out programs certainly serve as
one avenue for providing extra support to students with disabilities, however, with
the emphasis on inclusion, service delivery in the classroom may also be advanta
geous. One such example can be found in a study conducted by Me Donnell,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
32
Johnson, Polychronis, and Risen (2002) where students benefited significantly from
in-class service delivery.
McDonnell et al. (2002) found that the use of “embedded instruction”
improved student learning in a specific population of special needs students (p. 353).
The authors explain that embedded instruction refers to the delivery of “systematic”
instruction within the context of the natural setting within the general education
classroom. In other words, rather than deliver specific instruction in an artificial
environment, the student is provided explicit instruction as the need occurs within
the natural confines of the classroom. In order to achieve this, the use of a parapro-
fessional is necessary in order to provide the instruction as needed by the student
when the opportunity arises (p. 363).
In summary, Coyne et al. (2004) outline a school-wide continuous improve
ment program designed to improve reading performance in early elementary age
students. The important components of an effective program involve long-term and
short-term (benchmark) goals, assessment, instruction, and ongoing monitoring of
progress. Coyne et al. can be used as a model for an effective continuous improve
ment program implemented on a school-wide basis.
The 8-step Continuous Improvement Program designed by Barksdale and
Davenport (2003), has many of the same elements as the program recommended by
Coyne et al. (2004). In their book, Closing the Achievement Gap: No Excuses,
Davenport and Anderson (2002) share the motivation for developing a continuous
improvement program for the Brazosport Independent School District in Brazosport,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
33
Texas. The district recognized a significant achievement gap between schools with
large poverty populations and those with wealthy populations. State accountability
reform made the gap more glaring, and soon gained the attention of parents and
community members. In response to parental and public pressure, the district
embarked on a transformational journey, resulting in significant reductions and
elimination of the achievement gap between poor and wealthy schools.
The Brazosport Independent School District blended the theory of Total
Quality Management developed by W. Edward Deming with effective school
research to develop a focus and plan for reforms at Brazosport. Before beginning,
staff within the district attended training in Total Quality Management provided by
the Dow Chemical Corporation. Davenport and Anderson (2002) noted that they
were at first skeptical about blending business tactics with education, but soon
discovered that the two were not exclusively different (p. 21). Fourteen points are
identified as essential “to improve quality, productivity, and competitive position”
(P- 31):
1. Create constancy of purpose
2. Adopt the new philosophy (mission)
3. Cease reliance on mass inspection
4. End the practice of doing business on price along
5. Improve constantly and forever the system of production and service
6. Institute training on the job
7. Institute leadership
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
34
8. Drive out fear
9. Break down barriers between staff areas
10. Eliminate slogans and targets for the work force
11. Eliminate numerical quotas
12. Remove barriers to pride and joy of workmanship
13. Institute a vigorous program of education and retraining
14. Take action to accomplish the transformation
Davenport and Anderson (2002) further discuss research conducted in the
1960s at Michigan State University where researchers found that schools with low-
to-middle income students performing at high achievement levels shared five
common characteristics (p. 26):
1. Strong instructional leadership
2. High expectations of student achievement
3. Pervasive and broadly understood instructional focus
4. Safe and orderly school climate conducive to teaching and learning
5. Measures o f pupil achievement as an indicator o f program success
Based on Deming’s (cited in Davenport & Anderson, 2002) 14 points of
Total Quality Management, Brazosport School District first needed to identify a
vision and define it through a mission statement. Davenport and Anderson (2002)
noted that a change was needed in the way staff viewed student ability in terms of a
belief that all students can learn regardless of all the barriers that seemed to exist
(p. 37). Working together with school staff, parents, and community members, the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
35
district created a vision statement based on a 3-5 year plan for improvement:
“Brazosport Independent School District 2000: An Exemplary School District”
(p. 40). Short-term goals were also written as follows:
1. Raise test scores to award-winning levels
2. Create an environment that rewards performance
3. Produce graduates who are successful
4. Increase opportunities for choice
5. Win— develop criteria for ranking schools compared to each other
and to other districts
6. Plan— coordinate planning efforts in district to ensure consistency of
common purposes, goals, visions, etc.
7. Increase parental participation
8. Integrate technology
9. Improve/Add facilities
The Brazosport model began much in the same way that Coyne et al. (2004)
described as the beginning step in creating an effective improvement program.
Brazosport set both long-term goals for improvement, as well as developed short
term benchmark goals for reaching the ultimate goal in 5 years. Brazosport’s next
step involved a close look at the data gained from the Texas Assessment of Aca
demic Skills (TAAS). As a result o f the disaggregation of data, several trends were
identified by the Brazosport leadership team. Davenport and Anderson (2002) noted
they found “a number of teachers on both the district’s south and north sides whose
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
36
students scored high on the TAAS” (p. 45). They further found that certain teachers
in the higher socioeconomic areas had students of low socioeconomic status who
scored low on the TAAS. One teacher, Mary Barksdale, was found to have a high
success rate on the TAAS: “94 percent of her students were considered ‘at risk,’
virtually all of them had mastered each section of the TAAS” (p. 45).
The Brazosport leadership team analyzed this teacher’s (Mary Barksdale)
method of teaching and determined that at the core of her success was “a dynamic
process of continuous assessment and reteaching for those students not up to
mastery” (p. 45). Out of this simple strategy, the 8-Step process was created and
applied school-wide with excellent success. The 8-Step Continuous Improvement
Plan contains the following elements (Davenport & Anderson, 2002, pp. 46-49):
Step 1: Test Score Disaggregation
Test data taken from criterion-referenced tests (such as the California
Standards Test) is analyzed by grade level teams and used to identify instructional
groups. Data is also used to identify areas of the curriculum that students perform
well in and to target areas where more instruction is needed. In order to do this, data
is ranked by mastered skills from weakest to strongest, resulting in specific
instructional goals for improving student performance.
Step 2: Time Line Development
Time lines are developed prior to the beginning of the school year, and
outline the sequence of instruction for the content standards throughout the year.
Within the time line is also an emphasis on skill areas identified as weak in the Step
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
37
1 process. Again, this is done collaboratively in grade level teams, and teachers set
“weekly objectives” that are used school-wide (p. 46). This time line serves as a
guide for all school staff, allowing reinforcement of learning across environments of
the school site.
Step 3: Instructional Focus
Teachers instruct students using the adopted time line to ensure that essential
content areas are covered. In this phase, all teachers are expected to teach the
“target” skill areas at the same time. Support is provided to the teachers in the form
of lesson plans that include instructional strategies proven effective to student
learning. Teachers have the option of utilizing the lessons and strategies provided or
creating their own. The key is that all teachers deliver lessons according to the time
line, and all teachers deliver the target lessons as well.
Step 4: Assessment
Assessment is conducted at the end of each teaching unit, and tests are
generally short (4 questions) and specific. All teachers conduct assessment at the
same time, and results are used to determine if students have attained mastery.
According to Davenport and Anderson (2002), mastery is determined by 80% of
students scoring at least 3 out of 4 on the unit tests. If mastery is attained, the class
moves on to the next unit.
Step 5 and 6: Tutorials and Enrichment
Tutorial and enrichment time is built into the instructional day and typically
consists of a 30-60 minute time block. During the time block, students who have not
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
38
yet mastered a skill receive instruction again in a new way. The authors assert that
this time block is used in a “positive light” and students are treated with respect and
support (p. 48). Students who have mastered skills receive a higher level of
instruction based on the Blooms Taxonomy where students are challenged to use
higher level thinking skills when interacting with the same content.
Step 7: Maintenance
This step is aimed at reinforcing previously learned skills in order to ensure
that students do not forget what they have learned over time. The author asserts that
these lessons are especially important as the state testing deadlines approach, and
maintain that lessons should be designed to be “lively” and “interactive” (p. 49).
Maintenance activities are embedded in the instructional time line calendar and are
performed by all teachers at the same time throughout the year.
Step 8: Monitoring
Leadership plays an important role in this step of the process, as it is the site
administrator’s role to monitor teachers in following the instructional time line and
that students are improving school wide. Monitoring is in the form of classroom
visits as well as meetings with individual teachers, students, and grade level teams.
It is the leader’s job to ensure that the academic goals and priorities are being carried
out, and that enthusiasm for student learning stays constant.
According to Davenport and Anderson (2002), 9 years after beginning the
transformation to a continuous improvement system, Brazosport accomplished the
following (p. 10):
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
39
1. Students in low SES areas performed as well or better than those in
higher SES areas regardless of race, gender, or socioeconomic status
2. District drop-out rate declined from about 6% to one-tenth of 1 %
3. Several prestigious awards, including recognition of exemplary
business accomplishments as well as educational distinctions
Another important component in the 8-Step process involves the concept
created by Deming referred to by Davenport and Anderson (2002) as the “Plan, Do,
Check, Act” cycle (p.32). The basic cycle is as follows (p. 32):
1. Plan a change aimed at improvement, collect data, and establish a
timetable
2. Do implement the change on a small scale
3. Check the results of your actions. What did you learn?
4. Act, apply the change or abandon it, or run through the cycle again
under different conditions
Following with Deming’s (as cited in Davenport & Anderson, 2002) theory
on the improvement cycle, the 8-Step process fits as follows (p. 60):
1. Plan. Data Disaggregation (Step 1) and Calendar Development
(Step 2)
2. Do. Direct Instructional Focus (Step 3)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
40
3. Check. Assessment (Step 4), Maintenance (Step 7), and Monitoring
(Step 8)
4. Act. Tutorials and Enrichment (Steps 6 and 7)
The Brazosport model clearly fits with the four components identified by
Coyne et al. (2004) as critical to a school-wide program (p. 236). The initial stage of
the Brazosport model involves a preliminary analysis of data to determine instruct-
tional goals and objectives. An instructional calendar is then created to target a
timeline for benchmark goals to be achieved. This process follows the same
principles outlined by Coyne et al. (2004), as goals are set according to the strengths
and weaknesses of assessment data. Short-term goals and objectives are accom
plished through the development of the instructional calendar.
The second criteria of an effective continuous improvement program as
defined by Coyne et al. (2004) is the use of ongoing assessment to determine
progress and guide programmatic changes (p. 234). The Brazosport model uses
assessment throughout the instructional cycle to identify students who have attained
mastery and those in need of additional teaching. Brazosport also incorporates a
monitoring system by which administrators can assess teachers in their effectiveness
of instruction and use of curriculum. Results of student and teacher assessment are
then used to improve the system in order to achieve at higher levels.
The instructional piece identified by Coyne et al. (2004) is also present in the
Brazosport model. Brazosport has several components that involve instruction,
including a direct instructional focus, tutorials, enrichments, and maintenance
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
41
lessons (Davenport & Anderson, 2002, pp. 46-49). Brazosport also uses the same
components identified by Coyne et al. (2004), as instruction is tailored first to all
students, then additional instruction provided to individual students based on
assessed need. Scheduling is also addressed by Davenport and Anderson (2002),
another critical element related to the instructional piece as noted by Coyne et al.
(2004). In the Brazosport model, 90 minute blocks of time were allotted on a daily
basis for teaching reading and math in order to allow for adequate instructional and
practice time (Davenport & Anderson, p. 79). The authors note that other subjects
were taught on alternating days in the same time blocks. They further note that most
schools in the district agreed that daily teaching of math and reading in 90-minute
time blocks was necessary for students to attain adequate mastery of skills (p. 79).
Progress monitoring is also present in the Brazosport model, and is used in a
similar fashion as that recommended by Coyne et al. (2004). In the Brazosport
model, assessment is used within the instructional focus lessons, and teachers have
instant access to identification of students who reach mastery and those who do not
(Davenport & Anderson, 2002, p. 79). The data gained from class-wide assessments
can also be compared between individuals to determine gaps in learning. Also
included in the Brazosport model is on-going monitoring which allows students to
be given support immediately through re-teaching, tutorials or provided additional
enrichment activities if mastery is achieved (p. 48).
Teaching strategies are also addressed within the 8-Step model developed by
Brazosport. Strategies are not defined to the degree that Marzano (2001) presents,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
42
yet some similarities exist. In the 8-Step model, the following teaching strategies are
recommended (pp. 81-82):
1. Have consistent, high expectations
2. Model the way to think about an issue
3. Provide essential vocabulary as you teach math, reading, and writing
4. Make learning real
5. Use cooperative-learning techniques
6. Incorporate drill
7. Teach test-taking strategies
8. Celebrate mastery of skills and knowledge
Three specific teaching strategies included in the Brazosport model directly
mirror those recommended by Marzano (2001): (a) consistent, (b) high expectations,
(c) cooperative learning techniques, and (d) celebration of mastery of skills and
knowledge. Other strategies infer good teaching as defined by Marzano, such as the
concept of making learning “real” for students. In Marzano’s recommendations, this
may involve the use of graphic and symbolic representations when teaching
important concepts to students (Marzano, 2001, p. 16).
Not included in the Marzano (2001) literature is the issue of teaching test-
taking strategies to students. Davenport and Anderson (2002) assert that students
learn important skills when taught how to take tests, including “problem-solving,
process thinking, hi-order skills, organization, and handling pressure and stress”
(p. 82). It would also seem logical that students should practice in the format that
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
43
they will be tested in as well in order to become more familiar and comfortable with
the testing environment prior to formalized assessment.
Much progress has been made over the years in the inclusion of special
education students in the general education environment. Research suggests that
many strategies aimed at improving education within the general population can also
be effective with special education populations as well. Coyne et al. (2004) clearly
states that special needs populations do not necessarily require specialized curricu
lum, but more opportunity for direct instruction and practice in curricular areas of
weakness (p. 237). An evaluation of the effects of the 8-step process on special
education student achievement seems worthy of study given the fact that it contains
many elements considered effective by Coyne et al. (2004) when intervening with
students who have disabilities.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
44
CHAPTER 3
METHODOLOGY
Design Summary
A quasi-experimental design was used for the purpose of this study, with the
experimental group consisting of the School District A special education subgroup
receiving the 8-Step Continuous Improvement Program as an intervention. The
following design format was used:
Experimental Group Pre X Post
Pretest: 2003 California Standards Test
Treatment: 8-Step Continuous Improvement Program
Posttest: 2004 California Standards Test
Results o f the 2003 CST language arts and mathematics scores for the special
education subgroup in School District A was compared to 2004 CST results for the
same group in order to determine general growth trends for that population.
Additionally, pre-post CST scores were compared on individual students who were
continuously enrolled in special education in School District A in a effort to deter
mine individual student growth for that population.
A qualitative component was also used in this study in the form of informal,
open-ended interviews with site-level principals in School District A. Principals
were interviewed early in the 2004/2005 school year and asked to describe the level
of implementation attained at each site, as well as asked an open-ended question
about inclusion o f special education students in general curriculum instruction.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
45
Participants: Sampling
Participants in this study consisted of the subgroup population of special
education students participating in the STAR assessments in School District A.
Subgroup data was reviewed for Grades 2 through 8 in School District A, and mean
scaled scores as well as percentages of Advanced, Proficient, Basic, Below Basic,
and Far Below Basic were also reviewed the district. Subgroup results for the
California Standards Tests for the 2003 and 2004 spring reports were attained by
accessing the STAR website provided by the California Department of Education.
Additional participants consisted of five site-level principals who agreed to
participate in informal interviews to attain information pertaining to the levels of
implementation of the 8-Step program and the degree of inclusion of special
education students in individual schools within the School District A. Results of
interviews were not used for data comparisons, but to provide the district with
practical information that could prove useful in improving implementation and/or
inclusion practices district-wide.
Intervention Description
The treatment intervention provided to the experimental group consisted of
varying degrees o f implementation of the 8-Step Continuous Improvement Program
developed by Davenport and Anderson (2002). School District A began to imple
ment the 8-Step program midway through the 2003-2004 school year and contains
the following steps:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
46
1. Disaggregation of state standards data in order to identify specific
instructional groups
2. Development of an instructional timeline to align curriculum with
state performance standards
3. Development of “instructional focus” to guide classroom instruction
in critical state standard areas
4. Frequent, ongoing assessment to determine effectiveness of
instruction
5. Provision of tutorials to students failing to master standards through
classroom instruction
6. Provision of enrichment activities for students who quickly master
standards to provide academic challenge
7. Reinforcement of previously learned materials
8. Monitoring of the effectiveness of instruction by the school leader
Instrumentation
Quantitative
Quantitative data was collected by reviewing existing data as provided by the
STAR reports on the California Department of Education Website ( < Standardized
Testing and Reporting Program, 2003). Only the California Standards Test (CST)
results was reviewed for the subgroup population of special education students
within School District A. Additionally, only the English-language arts and
mathematics portions of the CST were used, as these are areas tested consistently in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
47
the 2n d through 8th grade levels. According to the information provided by the
California Standardized Testing and Reporting Program (STAR) as retrieved on
August 1, 2004, the English Language Arts testing is as follows:
The Grade 2 and 3 California English-language arts standards tests each have
65 questions. The Grade 4 and 7 tests each have 93 points comprised of 75
multiple-choice questions and an 8-point writing assessment. The tests for
Grades 5, 6, and 8 through 11 each have 75 questions. During 2003, the
Grade 4 and 7 California English Language Arts Standards Test included a
writing sample in addition to the multiple-choice questions. . . . Two readers
using a 4 point scoring guide score the students’ papers independently. The
two readers’ scores are added to the 75 multiple-choice questions resulting in
a maximum score of 83 points for these two grades.
The same website describes the California Mathematics Standards Tests as
well, and basically, notes that for Grades 2 through 7, 65 standards based questions
are administered based on grade level (.Standardized Testing and Reporting
Program, 2003). The website information adds that:
1. All students in Grades 8 and 9 who had not yet completed or were not
enrolled in discipline specific, standards based math courses, or who were enrolled in
the first year of a 2-year Algebra I course were required to take a General Math Test
based on California Math Standards for Grades 6 and 7.
2. Students in Grades 8 through 11 who had completed or were enrolled
in discipline specific standards based math courses too California Mathematics
Standards Tests in either Algebra I, Geometry, Algebra II, or 1st, 2nd, or 3 rd year
Integrated Mathematics.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
48
Another section of the STAR website explains how the scores for the CST
are reported on the state website as follows {Standardized Testing and Reporting
Program, 2003):
The following types of scores are reported by grade level and content area
for each school, district, county, and the state:
1. Mean scaled score is the group average scaled score for each grade
and content area. The California Standards Test scaled scores range from approx
imately 150 to 600. Scores between 300 and 349 are at the Basic Performance
Standard (Academic Achievement Standard), and scores of 350 or higher are at or
above the Proficient Performance Standard.
2. The % Advanced, % Proficient, % Basic, % Below Basic and % Far
Below Basic, is the percentage of students in the group whose scores were at this
performance standard. The state target is for every student to score at the Proficient
or Advanced Performance Standard.
Qualitative
Qualitative data collection was collected through an informal interview
format delivered face-to-face with site administrators. Interviews were used to
provide information to site administrators regarding implementation levels and
integration of special education students in core curriculum instruction. The inter
views were conducted by the evaluator, and consisted of the following open-ended
and/or yes/no questions:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
49
1. Describe your school’s procedures for data disaggregation.
2. Describe how your school developed its instructional calendar
(sequence of standards-based instruction).
3. Do your teachers use “focus lessons” to teach target standards-based
skills, and are focus lessons synchronized across grade levels? If not, how are target
skills taught in your classrooms?
4. Do all teachers in all grade levels use assessments at the end of each
teaching unit to determine mastery? If no, how are students assessed on mastery?
5. Does your school offer tutorials and enrichment blocks within the
instructional day for all grade levels? If not, describe what your school does offer
students in the form of tutorials and enrichment.
6. Describe how maintenance lessons are taught at your school site.
7. Do you (site administrator) monitor teachers to ensure that the
instructional time lines are followed school-wide? If not, describe how you deal
with monitoring issues at your school.
8. Describe how content standards lessons are delivered to your special
education population in your school.
Procedural Timeline
Specific time lines were followed in order to complete this study within the
time limitations. The time-line for data collection and interviews was as follows:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
50
October 2004. Quantitative data was recorded from the CST special
education subgroup for School District A. Interviews were scheduled for site
administrators within the School District A.
November 2004. Data analysis was run using SPSS on results of CST scores
collected in October 2004. Interviews were conducted with five of the seven district
administrators in School District A.
December 2004. Data analysis was interpreted, and interview results were
compiled and analyzed.
Data Analysis
Quantitative Analysis
A factoral analysis of variance (ANOVA) was used to compare pre-post
means for the special education subgroup in School District A. Additionally, a
repeated measures using the general linear model was conducted to determine
individual student academic growth on students continuously enrolled in School
District A in both the 2002-2003 and 2003-2004 school years.
Qualitative Analysis
Qualitative information generated by the informal interviews was analyzed
using six steps as recommended by Cresswell (2003) in his book, Research Design:
Qualitative, Quantitative, and Mixed Methods Approaches as follows:
Step 1. Interviews will be transcribed.
Step 2. Record general ideas generated by interview questions.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
51
Step 3. Organize information provided by interview questions into
categories by “chunking” information (Cresswell, 2003, p. 192).
Step 4. Identify themes generated by Step 3.
Step 5. Provide a detailed description of themes arising from interview
questions.
Step 6. Provide a personal interpretation of themes.
Delimitations o f the Study
The focus of this study was to determine whether or not a district-wide
continuous improvement program has a positive impact on special education student
achievement. Many other aspects of the continuous improvement program are
worthy of study, particularly those related to implementation issues. Issues related to
implementation stages and practices were not included in this study due to several
limiting factors.
First, it is already a known fact that the school district is in the beginning
stages of implementation, and that many improvements to the program will evolve
over the next several years. This study purports merely to examine the effect of the
existing implementation regardless of how far along each school may be in the
process. Second, time constrains exist in the context of the current study, as results
must be reported by April 2005. For the purpose of this study and in keeping within
time constraints, pre-post data was only reviewed using the Spring 2003 and 2004
STAR results. Follow-up data comparisons should be made in the years following
the completion of this study in order to determine gains in special education student
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
achievement as implementation becomes more solidified. If at that time gains are
not realized, further examination of implementation aspects would certainly be
warranted.
Inclusion aspects were also limited within this study. The School District A
currently uses a variety of service delivery models for mild to moderate disabilities.
This study focused mainly on generalities in special education student achievement
gains as opposed to looking at gains as they relate to level o f inclusion again for
reasons of time constraints as restricted by the study timelines.
For information regarding implementation and inclusion, this study utilized
an informal interview process in order to lay groundwork for future examination of
these issues if needed. Interviews were used merely to gather information at each
school site to better understand the degree of implementation of the 8-Step process as
well as to determine each particular school’s culture and practices related to inclu
sion practices. This information may prove useful to site-level administrators as a
result of this study, but will not contain specific information as to how implementa
tion levels or inclusion practices currently affect special education student achieve
ment specifically.
This study did not use written surveys or questionnaires, as informal
interview results were utilized specifically for informational purposes and future
study possibilities only. Additionally, this study did not include any observational
data due to time constraints and the limits of the study to achievement score data.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Limitations such as these are bound to occur within the educational environ
ment due to the complexity of a naturalistic setting such as this. Much more time
would be needed in order to examine both the implementation factors and the inclu
sion factors, and idealistically would occur longitudinally over time rather than in a
1-year time frame,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
54
CHAPTER 4
RESULTS
Introduction
The purpose of this study was to evaluate the effects of a district-wide
continuous improvement program on special education student achievement.
Analysis was based on both quantitative and qualitative data. Quantitative data
analysis was used to compare special education student achievement on the
California Standards Test for the 2002-2003 and 2003-2004 school years. A
factorial Analysis of Variance (ANOVA) was used to compare pre-post means for
the special education subgroup in School District A for the 2002-2003 and 2003-
2004 school years. Additionally, a repeated measures using the general linear model
was conducted to determine individual student growth on students continuously
enrolled in School District A in both the 2002-2003 and 2003-2004 school years.
A qualitative analysis was also conducted in the form of informal interviews
with site level administrators. This analysis was conducted in order to determine
levels and quality of implementation and inclusion of special education students at
individual school sites. Results of those interviews are reported in transcription
format, and an analysis of the general themes derived from those interviews reported.
Analysis o f Mean Differences
In order to compare growth on the California Standards Test (CST) for the
special education subgroup, a factorial Analysis of Variance was conducted to
compare means on the 2002-2003 and 2003-2004 CST for both English/Language
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
55
Arts and Math. Descriptive statistics are reported for the CST—English Language
Arts for Grades 2-8 as follows in Table 3.
Table 3. Descriptive Statistics for California Standards
Test: English Language Arts
Grade Year Mean SD N
2 2002 269.00 29.13 13
2003 266.50 41.48 10
Total 267.91 34.18 23
3 2002 257.80 37.99 20
2003 263.08 23.10 13
Total 259.88 32.62 33
4 2002 289.18 30.86 22
2003 266.11 64.12 18
Total 278.80 49.40 40
5 2002 286.64 34.40 25
2003 286.87 40.91 23
Total 286.75 37.26 48
6 2002 276.19 44.76 26
2003 276.96 24.39 26
Total 276.58 35.69 52
7 2002 284.50 35.91 34
2003 274.06 40.59 34
Total 279.28 38.40 68
8 2002 280.82 29.65 34
2003 267.65 32.74 40
Total 273.70 31.84 74
Total 2002 279.21 35.91 174
2003 272.55 39.02 164
Total 275.98 37.54 338
Aw A N OVA was conducted to determine if there was a difference by grade or
year. Grade and year effects as well as a grade by year interaction are reported in
Table 4.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
56
Table 4. Tests o f Between-Subjects Effects— California Standards Test: English
Language Arts
Source
Sum of
squares
d f
Mean
square F Sig.
Grade 15795.21 6 2632.54 1.907 .079
Year 2713.72 1 2713.72 1.966 .162
Grade* 6165.97 6 1027.66 .744 .614
Year
The grade effect resulted in an F equal to 1.907, which is at the .079
significance level, indicating no significant difference by grade. The year effect
resulted in an F equal to 1.966, which is at the .162 significance level, indicating no
significant difference by grade. The grade by year interaction effect resulted in an F
of .744, which is at the .614 significance level, indicating no significant difference
for some grade levels over others.
Descriptive statistics are reported for the CST—Math for Grades 2-8 as
follows in Table 5.
An ANOVA was conducted to determine if there was a difference by grade
or year. Grade and year effects as well as a grade by year interaction are reported in
Table 6.
The grade effect resulted in an F equal to 2.796, which is at the .012
significance level, indicating no significant difference by grade. The year effect
resulted in an F equal to .007, which is at the .932 significance level, indicating no
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
57
Table 5. Descriptive Statistics for California Standards Test: Math
Grade Year Mean SD N
2 2002 296.46 80.91 13
2003 267.78 86.75 9
Total 284.73 82.56 22
3 2002 257.65 66.23 20
2003 262.67 34.21 12
Total 259.53 55.77 32
4 2002 250.95 32.75 22
2003 281.87 44.54 15
Total 263.24 40.31 37
5 2002 247.24 44.31 25
2003 243.61 33.26 23
Total 245.50 39.04 48
6 2002 270.78 29.14 27
2003 266.81 20.14 26
Total 268.83 24.97 53
7 2002 270.59 39.26 34
2003 274.26 38.33 34
Total 272.43 38.55 68
8 2002 271.33 41.50 24
2003 271.78 37.83 27
Total 271.57 37.83 51
Total 2002 265.04 46.87 165
2003 267.01 44.27 146
Total 265.97 44.27 311
significant difference by grade. The grade by year interaction effect resulted in an F
of 1.162, which is at the .326 significance level, indicating no significant difference
for some grade levels over others.
Value Added Analysis
A crude value added analysis of student growth was conducted. Initially,
student scores were standardized within grades to make the scores from one grade to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
58
Table 6. Tests o f Between-Subjects Effects— California Standards Test: Math
Source
Sum of
squares d f
Mean
square F Sig.
Grade 31618.70 6 5269.78 2.796 .012
Year 13.72 1 13.72 .007 .932
Grade* 13146.26 6 2191.04 1.162 .326
Year
the next somewhat comparable. Gains from one grade to the next could then be
computed by subtracting 2003-2003 Z scores from the 2003-2004 scores. A repeated
measures analysis using the general lineal model was conducted in order to deter
mine if individual students made significant growth on the CST for English
Language Arts and Math between the 2002-2003 and 2003-2004 school years.
Descriptive statistics for the CST English Language Arts are reported in Table 7.
Table 7. Descriptive Statistics— California Standards Test: English Language Arts
2002/2003 2003/2004
Grade N Mean (SD) N Mean (SD)
2 7 .002(1.120) 1 .071 (.765)
3 8 .180(1.377) 8 .072(1.33)
4 15 .262 (.6700) 15 .021 (.984)
5 18 .047 (.6090) 18 .042 (.638)
6 23 .149(1.030) 23 .079 (.997)
7 26 .104 (.8550) 26 .307 (.934)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
59
A multivariate tests effect was conducted in order to determine if significant
gains were made in English Language Arts by individual students from the 2002/
2003 to the 2003-2004 school years using the Wilks’ Lambda method. Results are
reported in Table 8.
Table 8. Multivariate Tests Effect for the California Standards Test:
English Language Arts
Factor Value F
Hypothesis
d f
Error
d f
Sig.
Gain .989 1.026 1.00 91.00 .314
Gain* .954 .878 5.00 91.00 .499
Grade
Multivaiate analysis indicated no significant difference in student achieve
ment in English Language Arts from 2002-2003 to 2003-2004 (F= 1.026; Sig. =
.324). Analysis of gain by grade also indicated no significant difference in student
achievement in some grade level over others (F= .878; Sig. = .499). Descriptive
statistics for the CST Math are reported in Table 9.
Table 9. Descriptive Statistics— California Standards Test: Math
Grade N Mean (SD) N Mean (SD)
2 6 -.364 (0.735) 6 -.111 (.7140)
3 7 .572(1.325) 7 .823 (1.123)
4 15 -.140 (0.844) 15 .098 (.8840)
5 18 .245 (1.092) 18 .092 (.8360)
6 23 .029(1.117) 23 .036 (.8930)
7 18 .009(1.236) 18 .023(1.208)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
60
A multivariate tests effect was conducted in order to determine if significant
gains in math CST scores were made by individual students from 2002/2003 to
2003/2004 using the Wilks’ Lambda method. Results are reported in Table 10.
Table 10. Multivariate Tests Effect for the California Standards Test: Math
Factor Value F
Hypothesis
d f
Error
d f Sig.
Gain .997 .247 1.00 81.00 .621
Gain* .954 .786 5.00 81.00 .563
Grade
Multivariate analysis indicated no significant difference in student achieve
ment in math from 2002/03 to 2003/04 (F= .247; Sig. = .621). Analysis of gain by
grade also indicated no significant difference in student achievement in some grade
level over others ( F - .786; Sig. = .563).
Qualitative Results
Interviews with five principals within the School District A resulted in the
following responses:
1. Describe your school's procedures fo r data disaggregation.
The CST group report for the school, district, and grade level are used by
individual teachers. Teachers also use multiple measures, such as the Basic
Phonics Skills Test, Spelling Inventory, and Sight Words as a baseline
measure. Students are grouped according to scores on either the CST or
Multiple Measures (this varies by teacher).
Staff meet prior to students beginning school to review data. Data review
includes teachers analysis of the CAT scores of their previous year’s students
in addition to reviewing the scores of their incoming students with whom
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
61
they will be teaching during the current school year. EduSoft program is
utilized as data is segregated and reported in applicable methods. Teaching
staff and administrators identify strengths and needs based on data patterns.
Goals for the school year are determined based on results from the process.
Data is disaggregated throughout the year based on student data resulting
from on-going School District A and classroom assessments including
Running Records, Basic Phonics Skills Test, etc.
We look at all forms of data, including STAR, local assessments and
teacher developed assessments, and compile on a data spreadsheet for every
teacher/student. This data is compiled at the beginning of the year (for entry
level data) and the end of each trimester. Teachers record raw scores and
our Project Tech compiles the spreadsheets. We are beginning the use of
EduSoft as a tool to help us analyze data other than STAR. We use EduSoft
exclusively to access and analyze STAR data—current results and past year
trends. Teachers meet in grade level meetings, every other week collabora
tion time, semiannual Teacher Support Team meetings and at other times as
well.
District provides printouts of our multiple measures and CST scores from
the previous year. Each teacher is given a binder showing their last year’s
class and this year’s class list and their score. Teachers meet at grade level
meetings and during August SIP days to discuss overall strengths and weak
nesses for their students. Jointly they develop strategies to work on the areas
of emphasis. They develop an instructional focus calendar that reflects all the
standards for their grade level. Throughout the year the grade level teachers
refine and reflect on progress and standards.
We use EduSoft, and STAR reporting. Our project tech identifies each
child that is basic and below so that they may receive tutoring.
2. Describe how your school developed its instructional calendar
(sequence o f standards-based instruction).
By grade level. We also used another school’s calendar that had been
previously developed.
We created our instructional calendar from a program called “Standards
Plus,” and are also using a calendar created by the previous principal.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
62
Our school calendar is based on the sequence o f textbooks for classroom
lessons along with recommended by Standards Plus for calendaring focus
lessons.
The calendar (one for LA and one for Math) was developed by the 8-Step
leadership team comprised on teachers and myself (principal). This task was
completed in the Spring of 2004 and shared with staff prior to the close of
the school year. We looked at calendar development along other system
components of our school including daily schedules, length of the day,
instructional minutes, lunch/recess schedules, school-wide intervention, use
of all available resources including Title I aides, librarians and after school
personnel. The entire “system” revamp was presented alongside the instruct-
tional focus calendars. The entire system is dependent upon each individual
component factor—we wanted the rest of the staff to “see” the new systemic
changes we were proposing. The Instructional Focus calendars are revised in
the Fall of 2004 to reflect new information that came forward regarding the
Instructional Focus lesson materials from Standards Plus. The L. A. instruct-
tional focus remained the constant, school-wide focus, that is published and
shared each day. The Mathematics instructional focus varies by grade level
and is published on separate calendar and shared at the individual classroom
level. We publicize the LA Instructional Focus through our web page, front
marquee, morning message and newsletters.
One to two teachers from each grade level were involved in the Patricia
Davenport training. Using samples from her materials and the Standards Plus
calendar materials, teachers were given time during the year and during the
summer to work collaboratively to develop a grade level and subject specific
instructional focus calendar based on the state standards. In August we
dedicated one of our SIP days to focus on developing and refining the
instructional calendar. During monthly grade level and department meetings,
teachers meet to adjust and modify the instructional calendars. The leader
ship team meets throughout the year to reflect and refine the instructional
calendar for their department.
3. Do your teachers use “ f ocus lessons ” to teach target standards-based
skills, and are focus lessons synchronized across grade levels? I f not, how are target
skills taught in your classrooms?
Yes, our teachers all use ‘focus lessons’ in their classrooms. These
lessons are used in all classrooms on a daily basis and are synchronized
across grade levels.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
63
Our teachers use the Standards Plus curriculum for Language Arts as
focus lessons in the classroom. One teacher is piloting the Standards Plus
Math on a daily basis as focus lessons as well. Another teacher uses the
Standards Plus lessons to support daily curriculum lessons. Our teachers are
very consistent in teaching focus lessons on a daily basis (99% of the time).
Our school utilizes “focus lessons” to teach target standards-based skills
and focused lessons are synchronized across grade levels.
Yes, the focus lessons are used to teach “targeted” skills based on the
California State Standards for LA and Math. They are not the only source of
curriculum/materials used for this purpose. We also utilize the core program
materials (Saxon and Houghton/Mifflin) to teach students targeted skills.
Additionally, students participate in school-wide intervention 4 days per
week in every grade level. During the intervention time, students are taught
targeted skills in LA based on their levels of expertise/performance in com
prehension, fluency, etc. We are just beginning a school-wide mathematics
intervention which will focus on students in need of remediation. All
students have access to grade level core materials and instruction on a daily
basis in addition to any intervention/remediation programs offered.
The instructional focus lessons are conducted mostly through using the
Standards-Plus materials. Most teachers are using them in Language Arts
and Mathematics. Teachers try to correlate the lessons with the current day’s
focus.
4. Do all teachers in all grade levels use assessments at the end o f each
teaching unit to determine mastery? I f no, how are students assessed on mastery?
Yes but they are not formal assessments. Assessments take various forms
including: written products, presentations, portfolios, projects, chapter tests,
teacher made tests, or group projects. Some teachers use commercial made
tests. District multiple measures are used annually. These include: Gates-
Macgintie Reading and Vocabulary Assessment, Direct Writing Prompt, and
Math Criterion Test. In additional all students are given the California State
Standardized tests.
Yes, a variety of assessments are used including the Instructional Focus
unit assessments from Standards Plus as well and Houghton/Mifflin and
Saxon unit tests. Also, students are assessed on a frequent basis to determine
levels of reading proficiency (fluency/comprehension). We are working
towards using our data as a basis in all instructional decisions.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
64
All teachers in all grade levels use assessments at the end of teaching
units to determine mastery as provided by state approved and state adopted
texts and programs.
Assessments to determine mastery are always used in our intervention
groups to drive placement. Assessments for mastery are also given at the end
of each Standards Plus lesson.
Yes, mastery is assessed by teachers using a variety of assessments,
including teacher made tests, and unit tests in the general curriculum texts.
5. Does your school offer tutorials and enrichment blocks within the
instructional day fo r all grade levels? I f not, describe what your school does offer
students in the form o f tutorials and enrichment.
Yes, tutorials and enrichment is provided within the instructional day by
each classroom teacher within their classroom. Our resource teacher also
provides additional support to the classroom teacher, and after school tutoring
programs are in place as well.
We provide a half-hour per day of intervention/enrichment for our
students. There is also an afternoon math pullout program provided by
AmeriCorp workers. We are also starting an after school tutorial program in
January, 2005.
Tutorials are offered within the instructional day for all grade levels
through teachers teaching in small groups, support via the Resource
Specialist Program and small groups of students who meet with the
Intervention teacher.
We offer our tutorials and enrichment during the school day, and we
also include remediation and enrichment in our after school program. Our
primary focus, of course, is to serve students while they are here during the
regular school day and not make needed support dependent on transportation.
Teachers schedule study halls before school, after school and sometimes
during lunch. Our after school program provides daily homework assistance
time for students attending. We also provide an after school intervention
program manned by a credentialed teacher. Academic Saturday School is
provided for at-risk eighth grade students beginning the end of the first
quarter of school. In terms of enrichment, we provide a weekly GATE class
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
65
period each Wednesday. In addition, we offer enrichment experiences in
mathematics through leveled classes. Our elective programs also provide
enrichment opportunities.
6. Describe how maintenance lessons are taught at your school site.
Maintenance lessons are spiraled throughout each lesson or unit.
Teachers include regular maintenance activities as part of their instructional
calendar. The Standards Plus materials also have maintenance activities built
in their lessons.
Maintenance lessons are a part of the Instructional Focus lesson calendar
and are held periodically throughout each trimester. Since this is our first
year of implementation, we are looking at our success/feedback from this
year to determine if the timing of maintenance/review lessons is working to
best benefit our students. We continue to refine based upon feedback. We
announce maintenance/review focus just as we would the other weekly focus
lessons.
Maintenance lessons are taught at our school using the same methods
provided in Standards Plus that are utilized in focus lessons, support through
the after school program through the utilization of KAMICO standards-based
learning activities and wrap-around lessons provided in state adopted texts
and programs.
Maintenance lessons are provided through the Standards Plus Focus
Lesson calendar. We also review skills before testing through periodic
maintenance lessons. Maintenance lessons are built into the Saxon math
curriculum.
Our resource teacher and the classroom teachers teach the maintenance
lessons using general curriculum materials.
7. Do you (site administrator) monitor teachers to ensure that the
instructional time lines are followed school-wide? I f not, describe how you deal with
monitoring issues at your school.
Yes, monitoring is provided through observation and interview with
individual teachers.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
66
Yes, monitoring is provided through observation and the use of reflective
questions.
Administrators monitor teachers to ensure that instructional time lines are
followed school-wide through application of walk-throughs, summary notes
written by teacher grade-level teams as Wednesday minimum day meetings,
informal teacher chats and sharing during staff meetings.
Yes, through walk-through observations and ongoing dialogue, we follow
through on timelines. Certainly we have those staff members who have
willingly embraced the process and others that have been more reluctant.
Since we have school-wide intervention and school-wide announcements of
Instructional Focus, staff has been more cooperative than not in the full
implementation of the Plan-Do-Check-Act cycle.
The principal and vice principal have both been trained in a process
called ‘walk-through’ training. This is helpful to assessing the classroom
instruction that takes place. Both the principal and vice principal serve on the
Patricia Davenport site committee and conduct monthly meetings with the
Plan-Do-Check-Act committee. Team leaders/counselors attend monthly
grade level and department meetings to monitor progress as well.
8. Describe how content standards lessons are delivered to your special
education population in your school.
At our school we have developed an intervention program that supports
identified special needs students as well as school-based students. Our
resource teachers work collaboratively with the student’s regular classroom
teachers to support the classroom instruction and mastery o f the grade level
standards. Services are provided in class as well as pull-out for all academic
subjects. Our Special Day Class is primarily self-contained. Students are
mainstreamed for P.E. and an elective period. IEP goals are set based on the
grade level standards for each student. The classroom teacher modifies
instruction so the student can master the content standards.
All students participate in the Instructional Focus lessons— including our
SDC, RSP and any other special needs student. The Special Day Class is a
K-2 class and the teacher has elected to present the Kindergarten Instructional
Focus lessons in both LA and Mathematics. All other students participate in
their regular classroom and their progress is monitored with the rest of the
general education students. No student is left out of learning opportunities.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
67
All students benefit from a standards-based research-based, enriching
curriculum.
Our school has a resource specialist program and two special day classes
in which special education students’ needs are met. The resource specialist
program targets students with special needs in small, pull-out groups and in-
class math support. Special day class teachers teach through the same state-
adopted texts and Standards Plus Focus Lessons as used in general education
classrooms. Mainstreaming and reverse mainstreaming offer alternative
environments and social environments for our special education students.
Lesson accommodations and modifications are implemented as needed.
Special education and general education teachers meet together to discuss
how everyone can collaborate to meet the needs of special education
students.
All our special education students are mainstreamed and participate in our
daily intervention block. Pullout assistance is also provided per individual
student IEP.
For the most part, the content standards lessons are delivered in the
classroom, however, the resource teacher is actively involved in making sure
the students are learning as they should. Our resource teacher also monitors
the tutoring program at our school, so she is quite aware of the IEP students’
progress.
Summary o f Interviews
1. Describe your school’ s procedures fo r data disaggregation. All
schools are using some form of data disaggregation including:
a. California Standards Test
b. Multiple Measures in the form of district assessments
c. EduSoft Program to organize data
d. Teacher and administrators work together to disaggregate data
e. District provides assistance/data as needed
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
68
2. Describe how your school developed its instructional calendar
(sequence o f standards-based instruction).
a. Calendar developed by grade level
b. Calendar is sequenced with curriculum
c. Standards Plus curriculum incorporated into calendar
d. Leadership teams develop calendars based on multiple factors
(length of day, daily schedules, instructional time, school-wide interventions)
e. Language Arts is the most consistent instructional focus with
some schools adding math
f. Calendar is developed collaboratively within grade levels and
leadership teams
3. Do your teachers use ' focus lessons ” to teach target standards-based
skills, and are focus lessons synchronized across grade levels? I f not, how are target
skills taught in your classrooms?
a. All schools are using focus lessons in Language Arts
b. Focus lessons teach target skills based on State Standards
c. All school use Standards Plus as part of their focus lessons
d. Some teachers deliver focus lessons through general
instruction in the classroom
e. Some focus lessons are delivered during intervention blocks
within the instructional day
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
69
4. Do all teachers in all grade levels use assessments at the end o f each
teaching unit to determine mastery? I f no, how are students assessed on mastery?
a. Teachers assess using informal measures that are teacher-
made
b. Some formal assessments are used including unit tests from
State approved textbooks (Houghton-Mifflin, Saxon) and Standards Plus
curriculum
5. Does your school offer tutorials and enrichment blocks within the
instructional day fo r all grade levels? I f not, describe what your school does offer
students in the form o f tutorials and enrichment.
a. All schools are provided some form of tutorial or intervention
programs
b. Tutorials and intervention are provided by classroom teacher
within the regular instructional day
c. Intervention is provided by resource/intervention teachers
through pullout programs
d. Tutorials delivered within intervention blocks during
instructional day
e. Intervention provided in small groups with classroom or
resource teacher
f. Intervention provided in after school programs
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
70
g. Intervention provided in the form of study halls (before and
after school)
6. Describe how maintenance lessons are taught at your school site.
a. All schools provide maintenance lessons to students
b. Maintenance lessons are delivered within standard curriculum
and instructional day
c. Many use Standards Plus curriculum for maintenance lessons
d. Some maintenance lessons provided in after school programs
using the Kamico curriculum materials
e. Maintenance lessons are used as a review before state testing
7. Do you (site administrator) monitor teachers to ensure that the
instructional time lines are followed school-wide? I f not, describe how you deal with
monitoring issues at your school. All administrators monitor teachers using:
a. Observation and Interview
b. Reflective questions
c. Walk-throughs
d. Meetings, informal notes
e. Informal teacher sharing
8. Describe how content standards lessons are delivered to your special
education population in your school. All schools deliver content standards lessons to
special education students by any of the following methods:
a. Intervention programs/block time
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
b. In-class and pullout programs
c. Resource students receive content standards lessons in main
stream classrooms
d. Special Day Class students are taught standards in self-
contained special education classroom
e. Resource students are monitored by the resource teacher to
ensure target skills are mastered
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
72
CHAPTER 5
DISCUSSION
Summary
The purpose of this study was to evaluate the effects of a district-wide
continuous improvement program on special education student achievement in
School District A. The continuous improvement program was designed by
Davenport and Anderson (2002) to improve student performance on state and
federally mandated assessments as required by NCLB. The 8-Step Program consists
of the following:
1. Disaggregation of state standards data in order to identify specific
instructional groups.
2. Development of an instructional timeline to align curriculum with
state performance standards.
3. Development of “instructional focus” to guide classroom instruction
in critical state standard areas.
4. Frequent, ongoing assessment to determine effectiveness of
instruction.
5. Provision of tutorials to students failing to master standards through
classroom instruction.
6. Provision of enrichment activities for students who quickly master
standards to provide academic challenge.
7. Reinforcement of previously learned materials.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
73
8. Monitoring of the effectiveness of instruction by the school leader.
A preliminary analysis of disaggregated special education student subgroup
data indicated that many schools in the district report “below basic” or “far below
basic” performance on the California Standards Test (CST). Components of the
continuous improvement program included identification o f skill deficits through
data disagregration and focused instruction aimed at teaching state standards as
assessed by the CST. While students in general education classrooms are the main
target of the continuous improvement model, its effect on special education students
is also important, as this subgroup tends to perform significantly below the proficient
level as mandated by state and federal guidelines.
California Standards Test scores in the areas of English Language Arts and
Math were compared on the special education subgroup for the years 2002-2003
and 2003-2004 in order to determine if any differences existed as a result of
implementation of the continuous improvement model. Results from that
comparison indicated no significant differences in CST scores in the area of English
Language Arts for the special education subgroup from 2002-2003 to 2003-2004, nor
was there a significant difference for this subgroup when considering each grade
level individually. Results were similar in the area of Math as well, with no
significant differences noted from 2002-2003 to 2003-2004 either by the subgroup as
a whole or by individual grade level. In general, one can deduce that the continuous
improvement model had no significant impact on the special education subgroup
during the first year of implementation.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
74
Additional statistical procedures in the form of a repeated measures analysis
were used to determine if individual students made significant gains on the CST for
English Language Arts and Math between the 2002-2003 and 2003-2004 school
years. Results of that analysis again indicated that no significant gains were made by
individual students in any of the grade levels examined by this study (Grades 2-7).
In the case of Math, students in Grades 2 and 4 actually posted negative gains,
though they were not statistically significant.
In addition to quantitative comparisons of CST scores to analyze academic
growth, a qualitative analysis was also conducted in the form of informal interviews
with site level administrators. This analysis was conducted in order to determine
levels and quality of implementation and inclusion of special education students at
individual school sites. Results of those interviews indicated that all schools within
School District A are implementing the 8-Step Continuous Improvement Model in
some form, though implementation is not uniform across the district. Furthermore,
while all schools within the district use some form of data disagregation, not all
schools use the same tests for this procedure. All schools use the California
Standards Test results from the previous year to set the tone for an instructional
calendar. Individual school sites and teachers differ in the use o f additional student
data (including both standardized tests, district-made, and teacher-made tests) in
order to develop skill-specific instruction in individual classrooms.
All schools within the district have some form of instructional calendar in
place, and all are using the Standards Plus curriculum as part of the instructional
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
75
focus and calendar. Currently, School District A is using these components
primarily in the area o f English Language Arts, though individual teachers are
moving toward focus lessons in math as well. While all schools report use of
assessment to determine mastery, it is unclear as to whether adequate interventions
exist to address students who fail to meet mastery on any standard. Much diversity
in the form of interventions available to students exists in School District A.
Interventions ranged from in-class groupings to after school intervention programs.
No specific means for matching individual students to an appropriate intervention
was noted by any of the schools participating in the study.
Site administrators indicated that maintenance lessons were used within their
schools, but delivery of those lessons was not well delineated. Some reported that
they use maintenance lessons, with no specific examples given over and above the
general curriculum materials. Further, maintenance, as defined by some
administrators, meant reteaching a few key concepts close to the testing deadline.
Monitoring of individual teacher’s use of the components of the continuous
improvement program was mainly done through informal interviews and
observations performed by the site administrator.
Questions relating to inclusion of special education students in content
standards lessons revealed diverse findings as well. While all schools reported that
special education populations received instruction in the standards, the method for
delivery varied widely by school. Many reported delivery through pullout programs
(such as groups served in a special education setting either through a school resource
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
76
program or self-contained special education classroom), while several schools
successfully mainstreamed their students into general education classrooms for this
type of instruction.
Theoretical Implications o f the Study
Quantitative analysis of special education student achievement data indicated
no significant impact of the 8-Step Continuous Improvement Program on special
education student achievement. Several factors may have contributed to the lack of
educational benefit for special education students, and these factors will be
considered in light of the literature cited earlier in this study.
Kober (2002, p. 7) noted that the quality of instruction that is delivered to
special education students is critical to student learning. Given this fact, it is
possible that the instructional focus lessons being taught in the general education
classroom are not explicit enough, or intensive enough for the special education
population. Further, teacher ownership of special education students continues to be
problematic. As noted by most site administrators interviewed for this study, special
education teachers continue to be responsible for monitoring special education
students’ progress toward standards-based skill mastery. Additionally, mainstream
teachers may lack knowledge of the learning needs of the special needs students in
their classrooms and may not have an adequate baseline of their special needs
student’s skill mastery or deficit. Lack of collaboration between general education
and special education teachers may also contribute to this lack of knowledge.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
77
As noted in the literature by Marzano (2001), use of research-proven
instructional strategies is critical to individual student learning. Given the lack of
positive results reported in this study, it stands to reason that teachers may lack skill
in one or more of the nine key instructional strategies outlined by Marzano in his
book, Classroom Instruction that Works (2001). More analysis of teacher skill and
use of critical instructional strategies in relationship to special education student
inclusion in general instruction may be warranted. Results o f this type of analysis
would likely prove useful in guiding staff development offerings to improve
teachers’ instructional skills.
In Thurlow’s (2004) article, Biting the Bullet: Including Special Needs
Students in Accountability Systems, the issue of testing accommodations and
modifications and their impact on test performance is addressed. Thurlow notes that
“there are hundreds more accommodations that students may use as they participate
in assessments,” and that “it is usually assumed that there should be a connection
between the accommodations that a student uses during instruction and those used
during assessments” (p. 122). In light of this statement, the possibility exists that
there is a disconnect between the accommodations/modifications allowed a particular
student on assessments and the types of modifications received in classroom
instruction. Again, lack of communication between general education and special
education teachers may contribute to this breakdown.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
78
Practical Implications o f the Study
Qualitative analysis provided by site administrator’s interviews resulted in
several themes that may also contribute to the lack of impact of the 8-Step process on
special education student achievement. First of all, teachers were introduced to the
8-Step process late in the 2002-2003 instructional year, and many classrooms did not
begin to use the process until the beginning of the 2003-2004 year. In essence,
implementation of the program was in its start-up stages at the beginning of this
study. It is likely that not enough time had been allowed for implementation to be
realized to such a degree as to have an impact on student achievement (both for
general education and special education students) at the time of this study.
Further, not all teachers at all schools participating in this study have bought-
in to the program or are using the process at its optimum level. As indicated by
interview results, not all teachers currently use the same assessment tools to
determine students’ learning needs. Assessments vary from teacher-made tests to
standard testing materials embedded within state-adopted textbooks. It is likely that
a more uniform method of identification and ongoing assessment of skill mastery is
needed within the district in order to individualize instruction effectively for all
students.
Intervention is also an ongoing work-in-progress for the district. Interview
results indicated that a variety of intervention methods are in use, ranging from pull-
out programs to within instructional day intervention blocks to after-school tutoring
programs. It is unclear at this time whether or not the intervention programs
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
79
currently in use in the district are designed to target individual student’s skill deficits.
More study in this area is needed to determine the effectiveness of these intervention
programs as well as to assist in designing adequate methods for maintaining skills
once they are mastered by an individual student.
Interview results also indicate that inclusion could also contribute to the lack
of achievement on the CST for special education students as well. Analysis of
responses by site administrators on the subject of inclusion indicated that most
schools within the district, continue to rely on the special education teachers to both
deliver key standards instruction and to monitor student progress on the standards.
Additionally, most schools continue to provide instruction to special education
students through pullout programs, thus removing special education students from
instruction in the general education classrooms. More evaluation in this area will be
needed in order to determine if special education students receive adequate
instruction in the key academic standards within these pullout programs.
Further Recommendations
Findings from this study indicate that implementation of the 8-Step process is
still in the formulative stages within School District A. A major recommendation of
this study would be to develop a well-defined protocol to follow for implementation
of each step of the improvement process. Standard criterion for each step of the
process is critical, such as use of uniform assessments district-wide when
disaggregating achievement data. More study is also recommended in the area of
intervention strategies with focus on research-based instructional programs and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
80
methods. Additionally, some form of monitoring system is needed to ensure that
teachers have the necessary instructional skill for delivery o f standards-based
materials to special education students.
Analysis of inclusion practices within School District A indicates that there is
a need for more emphasis on inclusion of students with mild-disabilities in the
general education classroom during standards-based instruction. In addition, more
collaboration time is needed for special education and general education teachers in
order to ensure that general education teachers are knowledgeable about their
student’s learning needs. In regard to pull-out instructional programs, it is clear that
additional evaluation of the content lessons being used in these programs is needed
to ensure that key standards are being taught.
Based on the literature provided by Thurlow (2004), more awareness is
needed in the area of accommodations and modifications provided to special
education students, both in classroom instruction and in the testing situation. Special
education teachers first need to identify appropriate accommodations and modifi
cations for individual students, then communication of those individual needs to the
general education teacher must occur. Some form of ongoing assessment of the
effectiveness of those modifications and consistency of use in both the classroom
and testing environment would also seem beneficial to student performance.
Finally, ongoing evaluation of the 8-Step process in terms of implementation
and inclusion is recommended. As the district moves toward higher levels of
implementation, it is likely that the impact of the program on special education
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
81
student achievement may be more readily identified. Recommendations are for both
statistical evaluation of achievement gains in the special education subgroup as well
as evaluation of the levels of implementation at each school site.
Limitations o f the Study
Two major limitations existed in this study including:
1. Lack of a control group in order to allow for comparison of growth
scores—this study was not able to procure a suitable control group due to a lack of
cooperation from like districts in use of individual student achievement data. Future
study in this area may benefit from a control group for comparison purposes.
2. Subjective nature of the interview questions— more input from
individual teachers in the form of a survey may be more useful to a study such as
this, as it would allow more insight to the actual practices that occur within the
classroom in terms of instructional content and delivery.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
REFERENCES
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
83
REFERENCES
Agran, M., Alper, S., & Wehmeyer, M. (2002). Access to the general curriculum
for students with significant disabilities: What it means to teachers
Education and Training in Mental Retardation and Developmental.
Disabilities, 3(2), 123-133.
Clark, R. E., & Estes, F. (2002). Turning research into results: A guide to selecting
the right performance solutions. Atlanta, GA: Center for Effective
Performance, Inc.
Coyne, M. D., Kame’enui, E. J., & Simmons, D. C. (2004). Improving beginning
reading instruction and intervention for students with LD: Reconciling “all”
with “each.” Journal o f Learning Disabilities, 37(3), 231-239.
Davenport, P., & Anderson, G. (2002). Closing the achievement gap: No excuses.
Houston, TX: American Productivity & Quality Center.
Kober, N. (2002). Twenty-five years o f educating children with disabilities: The
good news and the work ahead. Washington, DC: Center on Education
Policy.
Marzano, R. J. (2001). Classroom instruction that works: Research-based
strategies fo r increasing student achievement. Alexandria, V A: Association
for Supervision and Curriculum Development.
Marzano, R. J. (2003). What works in schools: Translating research into action.
Alexandria, VA: Association for Supervision and Curriculum Development.
McDonnell, J., Johnson, J. W., Polychronis, S., & Risen, T. (2002). Effects of
embedded instruction on students with moderate disabilities enrolled in
general education classes. [Electronic version]. Education and Training in
Mental Retardation and Developmental Disabilities, 37(4), 363-377.
Standardized testing and Reporting Program. (2003). Retrieved August 1, 2004,
from http://star.cde.ca.gov/star2003/about STAR, asp
Standardized Testing and Reporting Program. (2003). Retrieved January 25, 2004,
from http://star.cde.ca.gov/star2003/about STAR about scores.asp
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
84
Thurlow, M .L. (2004). Biting the bullet: Including special needs students in
accountability systems. In S. H. Fuhrman & R. F. Elmore (Eds.),
Redesigning accountability systems fo r education (pp. 115-137). New York,
NY: Teachers College Press.
Tomkins, R., & Deloney, P. (1995). Inclusion: The pros and cons. Issues. . .
About Change, 4(3). Retrieved May 4, 2003, from
http://www. sedl. org/change/issues/issues43. html
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Evaluation of the effects of longitudinal tracking of student achievement to assess school quality

PDF

An evaluation of the REMEDY project with an emphasis on sustainability: A Safe Schools /Healthy Students federal grant

PDF

A quantitative and qualitative study of computer technology and student achievement in mathematics and reading at the second- and third-grade levels: A comparison of high versus limited technolo...

PDF

Evaluation of a two -period double math program on the academic achievement of underperforming seventh grade math students

PDF

An evaluation of probation supervision and its role in performance of students identified as living in poverty in Shasta County Court and Community Schools

PDF

Effects of the STAR testing program on teachers and the curriculum

PDF

Cognitive coaching training for master teachers and its effects on student teachers' ability to reflect on practice

PDF

Adult school student achievement on the California High School Exit Examination: Are adult schools ready for the challenge?

PDF

Into, Through and Beyond: Evaluation of migrant education reading tutor program innovations

PDF

Effects of an integrated content and methods course on preservice teachers' beliefs and efficacy toward mathematics

PDF

Implementing computer assisted instruction in a multilevel -multigraded classroom evaluation /action plan

PDF

Improving student achievement: An urban success story

PDF

Improving math and science education in charter secondary schools through the use of technology

PDF

Evaluation of CLAD training in northern California

PDF

A case study of the Math Matters professional development program in one elementary school

PDF

Corrective action plan for program improvement of a middle school

PDF

GEAR UP and TRIO: Redirecting the fight to preserve access and opportunity in the Higher Education Act

PDF

Increasing student retention through benchmarking and organizational improvement

PDF

A study of the relationship between student achievement and mathematics program congruence in select secondary schools of the Archdiocese of Los Angeles

PDF

Elementary administrators and teachers' perceptions of the teacher evaluation process in California's public schools

Asset Metadata

Creator Kehoe, Cindi (author)

Core Title Evaluation of the effects of a continuous improvement program on special education student achievement

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, curriculum and instruction,education, special,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Hocevar, Dennis (committee chair), Cohn, Carl (committee member), McLaughlin, Michael (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-388028

Unique identifier UC11340212

Identifier 3180492.pdf (filename),usctheses-c16-388028 (legacy record id)

Legacy Identifier 3180492.pdf

Dmrecord 388028

Document Type Dissertation

Rights Kehoe, Cindi

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA