Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Alternatives for achievement: a mathematics intervention for English learners
(USC Thesis Other)
Alternatives for achievement: a mathematics intervention for English learners
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
ALTERNATIVES FOR ACHIEVEMENT:
A MATHEMATICS INTERVENTION FOR ENGLISH LEARNERS
by
Jill Michelle Manning
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2007
Copyright 2007 Jill Michelle Manning
ii
Dedication
For my mother, Helen Elizabeth Morrissey, who showed me that
education provides you with the keys to unlock all doors and my daughter,
Bea Ivy Dyball, who I hope will never stop trying to open them.
iii
Acknowledgements
With much gratitude to Dr. Dennis Hocevar for guiding me through this
process and the members of my thematic dissertation group, William
Mannion, Damita Myers-Miller, and Virginia Yee for sticking together so that
we could call ourselves members of the Class of 2007. Many thanks also go
to Dr. Richard Brown and Dr. Elyse Sullivan for agreeing to serve on my
committee
I would also like to thank Cheryl Yamashita and Brenda Lara for
encouraging me to begin and complete my doctoral studies, even when I
thought I was not capable of doing either; Marc Chun for providing his
perspective on the process of becoming a doctor of education: and Jane
Ching Fung for serving as a role model for the educator that I have always
wanted to be and that every child deserves.
Finally, I wish to thank my dedicated editor, Dr. Ramona Maile Cutri,
who made sure that all my writing passed muster and gave me the courage
to pursue a doctorate through her example – not bad for two Title I kids.
iv
Table of Contents
Dedication ii
Acknowledgements iii
List of Tables v
Abstract vi
Chapter 1: Introduction 1
Chapter 2: Literature Review 20
Chapter 3: Methodology 38
Chapter 4: Findings 47
Chapter 5: Discussion 60
References 76
v
List of Tables
Table 1 Leadership Correlations with Student
Academic Achievement 28
Table 2 Mean Gains by Grade Level 47
Table 3 Mann-Whitney U Test Results by Grade Level 48
Table 4 Effect Size Estimates 49
Table 5 Performance Bands by Grade Level 50
Table 6 Proficiency Rates by Grade Level 50
Table 7 Total Sample Proficiency Rates for Pre- and Post-Test
Data 51
Table 8 Basic and Above Rates by Grade Level 52
Table 9 Rates for Total Students Scoring “Basic or Above” 52
Table 10 Between-Subjects Effects 52
Table 11 Second Grade Means 54
Table 12 Second Grade ANOVA 54
Table 13 Third Grade Means 55
Table 14 Third Grade ANOVA 56
Table 15 Fourth Grade Means 57
Table 16 Fourth Grade ANOVA 57
Table 17 Fifth Grade Means 58
Table 18 Fifth Grade ANOVA 59
vi
Abstract
The purpose of this study is to examine the impact of a standards-
based, locally designed intervention, The Alternative Guide, on the academic
achievement of English Learners (EL) on the mathematics portion of the
California Standards Test (CST). A nonequivalent control groups design with
one dependent variable and two matched control groups was used. This
quasi-experimental design consisted of two control groups that were not
randomly assigned. The Alternative Guide treatment was administered in the
experimental school and pre and post intervention comparisons were made
between the experimental school and the two control schools.
Participants in this study consisted of approximately 700 students
enrolled in grades 2 – 5 at an elementary school. Out of this population,
575 are identified as EL students. The Alternative Guide intervention began
in January 2005 and continued until June 2006. The primary student
outcome, student performance on the mathematics section of the California
Standards Test, was measured in April 2006. Performance level data were
coded on a 0-4 scale and the analysis was limited to grades 2-5.
Qualitative findings reflect that the intervention was implemented
differently in each grade level so fidelity of implementation was limited.
Teachers and administrators noted that there was an increase in the level of
performance in mathematics as a result of using the intervention.
vii
In the experimental school, improvement on the mathematics section
of the CST was uniformly positive at all grade levels with differences ranging
from +.145 to +.601 of a proficiency band and +.329 when all of the grade
levels studied were combined. Effect sizes ranged from +.137 to +.553 and
the findings were statistically, as well as, practically significant.
Quantitative findings regarding the comparison of the experimental
school in conjunction with the two matched control schools had different
results for each grade level. Statistically significant differences favoring the
experimental group were found for grades two and five but were not found for
grades three and four.
1
Chapter 1: Introduction
Problem Description
Eric Carle Elementary School is a Title I school in the MacArthur
Union area of Los Angeles with a student population of 1,000, where 95% of
the students are Latino and 82% are identified as English Language
Learners. District reporting data indicates that the parent education level is
low and that the average parent at our school is not a high school graduate.
School records show that more than 70% of the students are the first people
in their family to attend school in the United States and many of these
children are, quite literally, the family’s first chance at having someone
graduate from high school and potentially help the help the family move out
of poverty. The community holds the school in high esteem, and parents
believe that their child’s success at school is crucial if they are to realize their
full potential, be it academic or economic.
State test data shows that the vast majority of students at Eric Carle
are headed for quite a different future than either they or their parents have
imagined. Since 2002, Carle has achieved poorly on the California
Standards Test (CST). Specifically in mathematics, student scores in grades
2 -5 have been abysmal. In 2002, 83.75% of students scored below
Proficient, the state’s minimum standard, in 2003, 76.25% scored below
Proficient, and in 2004, 77.25% scored below Proficient. If one were to look
2
at students who scored at the Basic level or higher in mathematics for this
same time frame, we would see better student performance: in 2002, 39% of
students scored at Basic or higher, in 2003, 50% of students scored at Basic
or higher, and in 2004, 48% of students scored at Basic or higher. Scores for
Language Arts during the same period show that 86.25% scored below
Proficient in 2002, in 2003, 82.5% scored below Proficient, and in 2004,
86.25% scored below Proficient. This poor achievement in elementary
school mathematics leads to problems in middle and high school. District
data indicates that 50% of students who enroll in Algebra I in eighth grade do
not pass the course on the first try and repeat it in ninth grade. Clearly,
something has to be done and soon, if our students are going to meet the
academic challenges that lay ahead.
Grade level breakdown of the mathematics test score information
sheds a little more light on what is happening at Eric Carle. In 2002, 20% of
second grade students scored at Proficient and above, in 2003, 30% of
second graders scored at Proficient or above and in 2004, 25% of the
students in second grade scored at Proficient or above. The overall growth
rate of 5% for these three schools years does not keep pace with the
requirements of No Child Left Behind and is cause for concern.
In third grade, 16% of the students scored at Proficient or above, in
2003, 25% scored at Proficient or above and in 2004, 29% of the students
scored at Proficient or above. Again the overall increase of 8% in students’
3
tests scores over three school years is not sufficient to keep pace with
federal requirements for student progress.
Student progress for fourth grade was slower than for grades two and
three with an overall average growth of 6.5%. In 2002, 16% of fourth graders
scored at Proficient or above, in 2003, 17% of fourth graders scored at
Proficient or above and in 2004, 26% of fourth graders scored at Proficient or
above.
Finally, fifth grade students showed the least progress of all grade
levels with 13% scoring Proficient or above in 2002, 23% scoring Proficient
or above in 2003, and 11% scoring Proficient or above in 2004 for an overall
average growth of 5%.
The school’s poor performance on the CST has resulted in the school
being ranked as a I (one) on California’s Academic Performance Index (API).
The school is also in its second year on the Program Improvement (PI)
Watch list and has been informed that failure to meet the 2006 Adequate
Yearly Progress (AYP) goals set by the federal government under the No
Child Left Behind (NCLB) act will lead to sanctions against the school in the
2006 – 2007 school year.
The overall picture for Eric Carle students is bleak and the majority of
our students will probably be unable to pass the California High School Exit
Exam (CAHSEE) and graduate from high school. The two high schools
which Eric Carle students attend, have dropout rates of 48.4% and 45.5%
4
respectively, compared to the district overall dropout rate of 33.1%, the
county dropout rate of 18.9% and the state dropout rate of 13.1%.
Redesignation rates for English Learners also are low, with 3.23% of
students redesignating in 2002, 1.78% redesignating in 2003, and 4.60%
redesignating in 2004. Students who do not redesignate prior to attending
middle school do not receive the standard curriculum that would allow them
to graduate nor do they receive appropriate instruction to prepare them for
the CAHSEE. At Eric Carle, the vast majority of students have attended
school in the United States, if not the Los Angeles Unified School District,
since kindergarten so the inability to reach proficiency in English is a major
concern and an obstacle to achieving academic success.
Overall, Carle Elementary is not preparing its students to be
successful academically as mandated by Part A of NCLB which specifically
states, “children who are limited English proficient, including immigrant
children and youth, attain English proficiency; develop high levels of
academic attainment in English; and meet the same challenging standards
as all children are expected to meet.” (No Child Left Behind Act of 2001).
The purpose of the study is to conduct action research to solve a
problem of practice around mathematics instruction in grades two through
five at Carle Elementary School. If Carle does not improve its performance
on the spring 2006 California Standards Test (CST), then it will be placed in
Program Improvement (PI) status and be subject to numerous interventions
5
that the State deems to be applicable to the school and its student
population. This study will evaluate the effectiveness of a local intervention
in mathematics for grades 2 – 5 and its effects on CST scores.
Problem Analysis
There are three main variables involved with poor student
achievement at Eric Carle Elementary School : the students who attend the
school, the teachers who teach at Carle, and the school itself. Of these three
factors, students are the variable that is easiest to deal with because they
come to school and want to learn. Overall school attendance rates are
nearly 97%, which is four percent higher than the district average. For the
purposes of this study, students are seen as positive force within the school
and able to achieve given the opportunity, as evidenced by the 11% increase
in mathematics test scores on the CST for the 2005 school year. The fact
that students come from low socioeconomic backgrounds and are classified
as English Learners is not a factor that Carle Elementary School can control
for.
Teacher Level Factors
Teachers are the main issue that needs to be addressed in relation to
student achievement at Eric Carle Elementary, specifically, teacher
knowledge of mathematics subject matter and mathematics pedagogy as
6
well as their expectations for student success. Survey and observational
data collected over two schools years at Carle as part of a dissertation
project indicates that 25% of the teachers are unable to identify all the grade
level mathematics standards they are required to teach, 25% of teachers are
unable to plan appropriate standards-based lesson in mathematics, and 75%
of teachers are unable to teach in a manner that is consistent with high
student achievement. Marzano (2003) notes that, “all researchers agree that
the impact of decisions made by individual teachers is far greater than the
impact of decisions made at the school level.” (p. 71). Clearly teachers who
are not familiar with the content they teach, who can not plan effective
learning experiences, and who are unable to deliver effective instruction are
a huge obstacle to student achievement and make instructional decisions
that negatively affect their students.
These factual and procedural knowledge gaps require that teachers
learn the grade level standards they are required to teach. Clark and Estes
(2002) call this basic level of support information, “When we tell people
something about their jobs they need to know how to succeed on their
own…” (p. 58). However, teachers who are not successful on their own in
planning and delivering standards-based instruction tend to rely on the
textbook to support their planning and teaching and this does not translate
into high levels of student achievement, so teacher training is needed to help
resolve the situation. Clark and Estes found that this training should include
7
“information plus guided practice and corrective feedback.” (p.58). Guided
practice requires looking at the grade level standards and coming to a
consensus as to what it means to know each standard from a student
perspective, how students can demonstrate their knowledge, and what kind
of skills it would take to teach each standard for student mastery. Teachers
would then need to plan common learning experiences to help them deepen
their understanding of the content, use each other as sounding boards for
their ideas, and let the group knowledge guide the planning.
Teacher motivation is another issue that contributes to Carle’s poor
academic performance. Many teachers at Carle express experiencing low
job satisfaction which affects their motivation to refine their practice and
engage in teaching at high levels of competency. Another barrier to effective
mathematics instruction is teacher self-efficacy. “General self-confidence is
not as critical for work motivation as is task-specific confidence.” (Clark &
Estes, 2002, p. 90). Teacher surveys collected over two school years as part
of a dissertation project indicate that many teachers have limited confidence
in this content area and this lack of confidence affects their teaching and
student outcomes.
Teachers have also expressed feelings of frustration about teaching
math and investing the mental effort necessary to be effective. Clark and
Estes (2002) have found that, “Mental effort is determined, in large measure,
by our confidence. Those who lack confidence tend not to invest much
8
mental effort in a task. Why should people work hard when they believe they
will fail?” (p. 81).
Other teacher level factors that Marzano (2003) has identified are at
play at Carle. These include:poor instructional strategies, lack of clear goals,
poor monitoring of progress, lack of feedback, and poor classroom
curriculum design as evidenced by lack of engaging tasks that allow for
knowledge transfer. Poor instructional strategies are obvious by the lack of
coherence in the instructional program. Teachers are not helping students to
make connections across curricular areas or even within the area of
mathematics. Students tend to spend their time learning a strategy or
procedure, but not applying either one in problem solving tasks. Teachers
also fail to provide students with clear goals. Units of study are introduced,
but students are not made aware of what the desired outcomes should be
when the unit is finished.
Marzano (2003) notes that schools need to establish clear goals and
monitor progress so that students are aware of the progress they are making.
Currently, students are tested every two months on the mathematics
standards they have covered, but overall, student performance is poor and
the school wide average in grades 2 – 5 is about 50% on each quarterly
exam. Quarterly assessments are an opportunity for teachers to inform
students of their progress, but typically, the tests are given and the results
are not shared with students because teachers feel that the tests are unfair,
9
that students will not be able to understand the results, and, if they do, then
they will be demoralized by them, and that there is insufficient time to review
standards students have not performed well on so it is not a worthwhile
endeavor on their part.
Finally, classroom curriculum design plays a large role in student
failure at Carle. Marzano (2003) points out that, “Regardless of the direction
provided by the school (or district), individual teachers still need to make
decisions regarding curricular design at the classroom level given the unique
characteristics of their students.” (p. 106). Put frankly, teachers are unwilling
to invest the time required to make appropriate decisions about classroom
curriculum design, so they rely heavily on textbooks and worksheets to teach
mathematics to their students instead of engaging them through classroom
discussion and problem solving activities.
School Level Factors
There are many school level factors that affect teacher effectiveness
and student learning and achievement at Carle. Probably the most obvious
and crucial for Carle is what Marzano (2003) calls opportunity to learn (OTL).
Marzano has found that, “Opportunity to learn (OTL) has the strongest
relationship with student achievement of all school-level factors.” (p.22). If
English Learners are not given an opportunity to learn how will they achieve
at the level of their English speaking peers? Since, as Marzano notes,
10
“teachers commonly make independent and idiosyncratic decisions regarding
what should be covered and to what extent” (p.23) it is hard to know if
students are receiving instruction in all the standards that the state and
district require and how much of each standard students are learning.
Administrators, essentially supervisors of instruction, need to know the
curriculum in place at their schools and the learning outcomes that are
expected of every student at each grade level if they are going to be effective
in jobs and move underperforming schools forward. Unfortunately at Carle,
there are knowledge and organizational gaps at the administrative level that
impede student progress.
As mentioned in the previous section, throughout the school there is
an absence of clear learning goals that are articulated to students and poor
methods of providing students with feedback on their progress. Because the
school is underperforming, teachers and administration tend to rely on CST
results to guide school reform efforts, but this data comes too infrequently to
provide the school with any meaningful guidance. Teachers need to analyze
the needs of the students in their classrooms by implementing informal
assessment systems, such as observations which allow the opportunity for
immediate feedback on student work. Marzano (2003) suggests that schools
“establish an assessment system that provides feedback on specific
knowledge and skills at least every nine weeks.” (p. 39). Mathematics data is
provided every eight weeks, and fits the suggested time, but it is only data
11
since students typically are not given their test results and teachers do not
analyze the information to inform their instruction.
Finally, there is a lack of collegiality and professionalism at Carle.
Teacher interactions are based on friendship or similar lunch and recess
times as opposed to, “authentic interactions that are professional in nature.”
(Marzano, 2003, p. 61). It is rare, even in staff development, for teachers to
discuss student progress, pedagogy, or curriculum in any meaningful way.
Observational and survey data collected over the course of the study indicate
a reticence on the part of teachers to share student work for fear that it would
expose their classroom practice to their peers. Marzano’s research has
shown that, “the more friendship interactions, the lower students’ academic
achievement.” (p. 61) which goes a long way in explaining what has
happened at Carle Elementary School.
Of the factors that have been identified as contributing to the problem
of poor student achievement at Carle Elementary School, teachers’
mathematical knowledge, teachers’ motivation, opportunity to learn and
teacher professionalism are all factors which the school has differing levels of
control over and can work on in order to improve student achievement.
Data collection for this study took place over two school years at Eric
Carle Elementary School by the Mathematics Instructional Coach as part of
the requirements for their doctoral course work and dissertation. Data was
collected through surveys at professional development and grade level
12
meetings. Observation data was collected when the Mathematics
Instructional coach visited classrooms to observe teachers and when
delivering demonstration lessons. Informal conversations and interviews
were conducted with teachers during grade level meetings, lesson debriefs,
and whenever the need arose on the part of either the teacher or
Mathematics Instructional Coach.
Problem Solution
In order to improve English Learner (EL) student achievement, Carle
Elementary School has implemented a series of locally-designed, standards-
based mathematics lessons to supplement the district-adopted textbook
known as the Alternative Guide. In addition to the Alternative Guide,
teachers have received training on how to implement the lessons through
direct observation by the Mathematics Instructional Coach teaching the
lessons to students and professional development on a monthly basis in staff
meetings and at grade level meetings. Reeves (2004) notes that, “School
reform takes place through the actions of individual parents, teachers, and
administrators who are willing to change what happens in the classroom.” (p.
3) and the Alternative Guide is an attempt to make that happen.
The Alternative Guide was implemented in July 2004 in all grade
levels at the school. Teachers received the guide at a professional
development meeting and were given instructions about how to use the
13
guide in their classrooms which included how to plan for instruction, lesson
pacing, and how to use the quarterly concept lessons to support student
learning. Teachers were not required to use the guide but its use was
strongly suggested by the Mathematics Instructional Coach and the school’s
administration. This approach was deemed necessary in light of the
teachers’ unfavorable reaction to previously mandated curriculum in
language arts that, to date still faces implementation obstacles.
The guide provided teachers with eight week blocks of lesson plans
that were directly aligned to the district Mathematics Quarterly Assessments
and were written in a manner that was easily understood. Both Marzano and
Reeves stress the importance of current assessment data to guide
classroom instruction and enhance student learning. Marzano (2003)
suggests, “an assessment system that provides feedback on specific
knowledge and skills at least every nine weeks.” (p. 39) and Reeves (2004)
recommends that teachers, “create a few assessments – perhaps half a
dozen for each nine week quarter.” (p. 20). By breaking instruction into eight
week blocks that align to an assessment, teachers can make a better
connection to what they are teaching and how well students are learning the
material as evidenced by their performance on the assessment.
Teacher motivation to teach mathematics was also considered when
the Alternative Guide was created. Because teachers at Carle have
knowledge gaps about the mathematics they are teaching, each quarter
14
includes a “Things You Need To Know” section that helps teachers
understand why the standards are being taught in the sequence presented in
the Alternative Guide and how they relate to student overall understanding of
mathematics as a coherent set of ideas and provides teachers with
information about common misconceptions that students have and problem
areas they may encounter. By reducing the stress surrounding teaching
unfamiliar materials, the Alternative Guide increases the likelihood that
teachers will use the Alternative Guide and improve their mathematics
instruction.
Teacher expectations about student performance are also an issue
that the Alternative Guide strives to address. Wigfield, Galper, Denton, and
Seefeldt (1999) found that “Teachers rated White children significantly higher
than the Hispanic children and the African American children” (p.101) when
asked to rate students’ prospects for future academic success and since the
overwhelming majority of Carle’s students are Latino; this is an obstacle our
staff needs to overcome. Since the lessons in the Alternative Guide are
standards-based, there is no argument about the relevance for our student
population.
Teachers have come to consensus that standards are not going away
and that it is their job to help students meet the standards. By removing the
burden of lesson planning from teachers, teachers are able concentrate on
lesson delivery and improving their observational skills as they teach the
15
lesson. This is a crucial piece of the design because teachers need to
understand what their students know and can do in mathematics as well as
differentiate between student error and student misconception. The ultimate
goal of the invention is to raise teacher expectations for student performance
and as Trouilloud, Sarrazin, Martinek, and Guillet (2002) found, “the higher
the teacher expectations were for a student, the higher was the student
achievement.” (p. 591).
Supporting teachers through ongoing professional development and
observations is another reason why this intervention was chosen. Many
teachers at Carle indicated that they are not confident about teaching math
because they lack the content knowledge and have low self-efficacy because
of this knowledge gap. The Mathematics Instructional Coach provides
teachers with just in time assistance as gaps appear. Teachers are able to
meet with the coach individually to discuss concerns or bring them up at
grade level meetings to see if other teachers are having similar problems.
Professional development for the entire staff helps teachers to see
problems as they manifest themselves across the grade levels and try to
determine ways to identify and remediate these problems. At professional
development, all teachers engage in mathematics learning, regardless of
their grade level or the content they teach, and refine their presentation skills
by problem solving in small groups and presenting their solutions to their
peers. Teachers are also encouraged to observe their peers teaching
16
lessons that either they or their class have had difficulty with. Finally, the
Mathematics Instructional Coach models how to teach lessons from the
Alternative Guide in demonstration lessons and debriefs with teachers about
critical understandings for students and the teacher.
Administrators are also trained on the how to use the Alternative
Guide and what to look for when they observe classroom during mathematics
instruction. The Mathematics Instructional Coach meets weekly with
administration to update administrators about what concepts are being taught
in each grade level on each track and to help them better understand the
instruction techniques being used. Administrators also take part in grade
level meetings and professional development and are expected to learn
along with the teachers. To date, no administrator has taught a math lesson
in a classroom, but they have participated in demonstration lesson and are
becoming more fluent in understanding Quarterly Assessment data and how
it should be used to evaluate classroom progress.
This intervention has the best chance of being successful because it is
locally designed and teachers feel that they have the ability to ask for
changes to the guide if they think it is necessary. This element of control is
particularly important to the teaching staff of Carle Elementary School
because it maintains their sense of professionalism and reinforces the idea
that they are able to make decisions about what is best for their individual
classrooms and students. Administrators are supportive of the intervention
17
because they realize that the school needs to improve instruction to improve
test scores and that they too have a lack of understanding about the
curriculum that they are supervising. Administrators also recognize that the
Alternative Guide provides an option that lowers the school staff’s effective
filter and makes the work of mathematics accessible to all stakeholders at
Carle Elementary School.
Purpose, Design and Utility
The purpose of this study is to conduct a formative and summative
evaluation of the Alternative Guide Intervention for mathematics instruction at
Eric Carle Elementary School. The formative evaluation will involve looking
at how the Alternative Guide is implemented in different grade levels and
classrooms with a focus on improving classroom instruction and student
performance on the CST. The summative evaluation will involve looking at
quantitative data to see if the intervention was effective in helping to improve
student achievement on the 2006 CST.
The methodology of this study will be both quantitative and qualitative.
The quantitative aspect will involve an interrupted time series quasi-
experimental design with two control group focusing on the independent
variable of the Alternative Guide Intervention and the dependent variable of
the 2006 Math CST scores at all three schools. The qualitative aspect of this
study will involve interviewing teachers at the experimental site as well as
18
observing their implementation of the Alternative Guide, and document and
material analysis at the experimental site.
The unit of analysis for this study is two school years, from July 2004
to June 2006 in the experimental site during the regular school day from 8:00
a.m. to 2:30 p.m. and during professional development meetings and teacher
interviews that will take place after school hours.
The sample will consist of approximately 600 students in grades 2 – 5
and the test scores from the mathematics section of the CST available for the
California Department of Education and 26 teachers on staff at the
experimental site. Teachers will be observed as part the routine practice of
the Mathematics Instructional Coach who will also conduct unobtrusive
interviews and anonymous open-ended surveys.
This study is significant because it is analyzing a problem of practice.
The researcher is the mathematics instructional coach at the experimental
site and has been charged with raising student achievement at Eric Carle
Elementary School. If the school is unable to meet adequate yearly progress
targets, it will be placed in Program Improvement status and be forced to
deal with the sanctions that come with that status. Currently, over 60% of the
students at Eric Carle are not meeting the minimum performance standard of
Proficient on the mathematics section of the CST as set out by the California
Department of Education. Given the current level of mathematics
achievement, it is unlikely that the vast majority of students at Eric Carle will
19
be able to pass secondary mathematics courses or the CAHSEE if
mathematics instruction does not improve and finally, it is unlikely that the
school will be able to meet the NCLB goal of 100% by 2014.
20
Chapter 2 – Literature Review
Introduction
With an ever increasing number of students who do not speak English
as their primary language attending public schools in both California and the
United States, it is imperative that schools meet the academic needs of non-
English speakers so that they can achieve their full potential. Public Law
107, also known as The No Child Left Behind Act of 2001, requires that all
students meet “minimum, proficiency on challenging State academic
achievement standards and state academic assessments” (Public Law 107-
110, p. 1439) and strives to ensure that all students receive an equitable
education regardless of their language background.
In the United States, 19% of all school aged children speak a language
other than English at home and are classified as English Learners. In
California, 24.9% of all students who attend public schools are classified as
English Learners (EL). In the Los Angeles area, 43% of students attending
public schools are categorized as English Learners and at Eric Carle
Elementary School, 82% of the student body are ELs. According to Public
Law 107 – 110, also known as, The No Child Left Behind Act of 2001
(NCLB), an English Learner is any student who:
(a) being 3 to 21 years of age, (b) enrolled or preparing to enroll in
elementary or secondary school, (c) either not born in the United States
or speaking a language other than English, and (d) owing to difficulty in
speaking, reading, writing, or understanding English, not meeting the
state’s proficient level of achievement to successfully achieve in English-
only classrooms. (Abedi, 2004, p. 4).
21
The California Department of Education defines an English Learner as any
student, “who is not proficient in English”. (California Department of
Education, 2006).
Statewide, 28.5 % of English Learners in grades 2-5 scored at
Proficient or above on the math portion of the 2004 CST, compared to 17.5%
of English Learners at Eric Carle who score at Proficient or above. While
Adequate Yearly Progress (AYP) goals were met in 2004, 82.5% of EL
students did not perform at proficient levels. In order to achieve the No Child
Left Behind 2014 AYP goal of 100% proficiency, student achievement in
mathematics must increase. The purpose of this literature review is fourfold.
First, I will examine relevant literature regarding ELs and their academic
achievement. The second topic to be examined will be the use of local
interventions to increase EL student achievement. The third area will
examine literature focused on the performance of ELs on the CST. Finally,
the fourth section of this literature review will explore how these three factors
relate to and interact with each other to improve EL academic achievement.
This chapter will also look at how teacher and school level factors influence
student achievement. In each section, how teacher and school level factors
figure into English Learners’ achievement will be included where relevant.
.
22
Literature Regarding English Learners
Given immigration trends, all states have had to face the challenge of
educating students who do not speak English as their primary language, and
California has had the largest number of non-English speaking students to
educate. (Lachat, 2004). One of the significant characteristics of English
Learners is that, proportionally, they live in high-poverty areas and
segregated neighborhoods that have an array of socio-economic problems,
and they experience the prejudices that some school personnel feel toward
immigrants, ethnic minorities, and poor people. (Lachat). These
circumstances do not translate into the best learning environment for any
child and can be insurmountable for some children.
In addition to these challenges, “the greatest difference between
English Learners and their peers is the magnitude of learning expected of the
former.” (Lachat, 2004, p. 29). While all students must work toward
achieving academic standards, English Learners must do this while acquiring
a new language. To compound this problem, there is no fixed time frame for
how long it takes an English Learner to develop sufficient proficiency in
English to use it for academic purposes, with some researchers noting that it
can take five to seven years before EL students are able to use English in an
academic setting. (Lachat).
23
Abedi (2001) has also found that, “English Language Learner students
scored lower than students who are proficient in English on the standardized
tests of mathematics achievement in elementary school” (p. 220) that had a
language component. Computation is not the problem, rather understanding
what the question is asking serves as a barrier to EL student achievement.
Collier (1992) also makes note of the phenomenon, “language minority
students who are limited in English proficiency score at extremely low levels
on standardized tests in English normed on native speakers. These tests are
clearly not appropriate measure in the first couple of years of L2
development.” Essentially, all tests, regardless of content area, become
tests of language comprehension for English learners.
Teacher level factors that affect EL students’ achievement are varied.
Marzano (2003) notes that, “Effective teachers employ effective instructional
strategies, classroom management techniques, and classroom curriculum
design in a fluent, seamless fashion. (p. 77). Since the majority of ELs
attend schools in high poverty areas, it is unlikely that their teachers have
reached a level in their practice where they are employing these types of
effective strategies, which is the case at the experimental school in this
study. Marzano (2003) concludes, “The expert teacher has more strategies
at her disposal than the ineffective teacher.” (p.87).
Teachers can also consider their classroom curriculum design and
how it affects English Learners. Marzano (2003) suggests that, “When
24
students are first exposed to content, learning should ideally involve the use
of stories or other forms of dramatization along with the use of visual
representations of information.” (p. 114). This type of classroom practice
works for all students and is enormously helpful to ELs.
School level factors that affect EL students’ achievement on
standardized tests, include the school’s ability to intervene when problems,
are found. Marzano (2003) has found that, “schools provide interventions
that are designed to overcome student background characteristics that might
impede learning.” (p. 8), such as speaking a language other than English.
Schools can also ensure EL student achievement by guaranteeing
opportunity to learn. Of all the factors studied by Marzano (2003),
“Opportunity to learn (OTL) has the strongest relationship with student
achievement of all school-level factors.” (p. 22).
Another school level factor that must considered when
discussing English Learners is goal setting. Schools that focused on setting
academic goals had an estimated 21 percentage point gain on achievement
tests over schools that did not engage in this type of behavior (Marzano,
2003, p. 35). By having clear expectations for student academic behavior,
schools make it possible for all students to know what needs to be done in
the classroom in order to be successful.Literature Regarding Local
InterventionsIn his work on school leadership Robert Marzano (2005) has
found that there is direct correlation between school leadership and student
25
academic achievement, “An increase from in principal leadership behavior
from the 50
th
percentile to the 84
th
percentile is associated with a gain in
overall achievement of the school from the 50
th
percentile to the 60
th
percentile.” (p. 30) for an average correlation of .25.
The intervention that was chosen for the experimental school was
based on the needs that the students, teachers, and administrators exhibited.
Marzano (2005) has found that, “to design a site-specific intervention, a
school must begin with a model or framework of those factors that can be
altered in a school to enhance student achievement.” (p.81). At Eric Carle,
the factors that could be modified were the ability of the administrator
establish clear goals, to participate in the mathematics curriculum and be
aware of mathematics instruction in the classroom, to actively acquire and
cultivate knowledge about mathematics, monitor and evaluate student
progress and become more visible in classrooms.
The principal has the ability to keep a school focused on student
achievement and “the extent to which the leader establishes clear goals and
keeps those goals in the forefront of the school’s attention” (Marzano, 2005,
p. 50) can lead to greater student gains on standardized tests. To keep
teachers focused on the local intervention in mathematics used in this study,
the principal had the responsibility of continually referring to the mathematics
quarterly assessment data to track student achievement throughout the
school year and help teachers set new goals for struggling students. In order
26
to do this, the principal, in turn, had to be trained in the use of the
intervention and understand the rationale for using the intervention.
This type of involvement in curriculum, instruction and assessment is
key to helping a school move forward. Marzano (2005) has found the,
“knowledge of subject matter and pedagogy should be as important to
administrators as it is to teachers” (p. 53) and that, “an administrator’s ability
and willingness to provide input regarding classroom practices was one of
the most highly valued characteristics reported by teachers.” (p. 54). One of
the key variables to a successful implementation of the mathematics
intervention was the administrator’s willingness to become involved in the
daily workings of classrooms instruction and knowing what lessons would be
taught in any classroom on a given. Being able to provide teachers with
constructive and relevant feedback on their teaching helped to encourage
struggling teachers and lent credibility to the principal.
Another key aspect of the intervention was the amount of professional
development that was provided to the teachers to help them understand the
pedagogy involved in the intervention as well as the mathematics. Again, the
administrator played an important role by being an active learner and
receiving their own training on the intervention both with the staff and with
other administrators at principal’s meetings given by the local district. This
training helped the principal with the, “acquisition and cultivation of
27
knowledge” and become “aware of the best practices in the domain.”
(Marzano, 2005, p. 54).
Monitoring and evaluating student progress throughout the
intervention is one of the most difficult but important factors that the
administrator has control over. As Marzano notes, “the extent to which the
leader monitors the effectiveness of school practices in terms of their impact
on student achievement” (p. 55) is what moves a school forward. In fact, “in
the most effective schools ‘constant evaluation’ is a norm.” (Marzano, 2005,
p. 56) and for Eric Carle to continually improve student achievement and
meet federal goals, the culture of continuous improvement must prevail. The
administrator, along with the support of the Mathematics Instructional Coach,
kept quarterly assessment data, as well as informal assessment data, at the
forefront of staff and professional development meetings in order to foster a
climate of constant evaluation and reassess the effectiveness of the
intervention.
Finally, the visibility of the administrator at a school site can either
move a school forward or allow it to stagnant. By walking into classrooms
and participating in professional development, the administrator serves as a
role model for teacher behavior and Marzano (2005) notes that, “highly
effective principals are in classrooms on a routine basis.” (p. 61) because “it
communicates the message that the principal is interested and engaged in
the daily operations of the school (and) it provides opportunities for the
28
principal to interact with teachers and students regarding substantive issues.”
Given the achievement history at Eric Carle, this is a message that all staff
members needed to receive in order to improve their practice and increase
student academic achievement.
According to Marzano (2005), the following five responsibilities of a
school leader have actual correlations to student achievement as listed in the
table below:
Table 1 – Leadership Correlations with Student Academic Achievement
Correlation with Student
Achievement
Responsibility
.24 Focus
.25 Knowledge of Curriculum
.20 Involvement in Curriculum
.27 Monitoring/Evaluating
.20 Visibility
Administrators who actively engage in these five responsibilities can expect
to see an estimated improvement in student achievement of .23, or, as noted
at the beginning of this section, from the 50
th
percentile to nearly the 60
th
percentile.
Zmuda, Kuklis, and Kline (2004) also address the “transformational”,
or school site, factors which impact all student learning, but particularly
English Learners. Kline, et al., refer to the work of Linda Darling-Hammond
(1997) to illuminate the principle that effective and meaningful change must
occur at the site level, and be responsive to the individual and specific needs
of the “local context” if the change is going to be successful. It is the
29
responsibility of the local school site personnel to determine how they will
approach and sustain a “continuous improvement effort” to transform the
curricula and practice at their school.
If the purpose of every school is to optimize student achievement; “it is
the core beliefs that define achievement” (Zmuda, et. Al, 2004, p.57). As
Zmuda et. al note, the initial step for optimizing student achievement is
identifying and clarifying the core beliefs. This process requires that all all
stakeholders discuss the level of commitment required from the school
community in order to maintain and sustain a competent learning community
and requires that all individuals involved with the school have a say about
their responsibilities in moving the school forward.
As more and more schools and school systems enroll English
Learners, the development of an action plan to address these “learning
issues” becomes critical. The “essential question,” according to Zmuda,
Kline, & Kudklis, thus becomes making it happen. The principles for the
development of this action plan are:
1. Staff development must promote collective autonomy by embracing
teaching as a distributed quality of the school.
2. Planning must provide the clear, concise direction necessary for
systemic change while remaining flexible enough to accommodate
the “nonrational” life in schools.
30
3. Staff development must reflect the predictable stages of teacher
concern about the complexities of moving from new learning to
systemic consequences
The intervention from this study, addresses the issues of teacher autonomy,
as they decide how to implement in the intervention for their students;
flexibility by allowing teachers to change the sequence of the lessons to meet
student needs; and finally, professional development is an integral part of the
intervention to help teachers better understand the mathematics they are
required to teach.
One teacher level factors that affects the implementation and use of
local interventions is how fully the teacher embraces the change that the
intervention is introducing. Marzano (2003) has found that, “that the impact
of decisions made by individual teachers is far greater than the impact of
decisions made at the school level.” (p. 71). Teachers can be given an
intervention designed to meet the needs of their local student population, but
if they choose not to implement the intervention, then it will not bring about
the desired results in student achievement. This factor is probably the most
difficult to overcome in any school setting and requires leadership on the part
of the school administrator if any local intervention or curricular program is
going to be successful.
31
Another teacher level factor that needs to be considered is how the
teacher implements the intervention. “Regardless of the direction provided
by the school (or district), individual teachers still need to make decisions
regarding curricular design at the classroom level given the unique
characteristics of their students.” (Marzano, 2003, p. 106). Teachers need to
be aware of the needs of their students in order to sequence instruction to
meet those needs.
School level factors that affect the implementation of local
interventions again focus on the teacher and how the school admininstrator
monitors their use of curricular materials. Again, Marzano (2003),
underscores this finding, “even when highly structured textbooks are used as
the basis for curriculum, teacher commonly make independent and
idiosyncratic decisions regarding what should be covered and to what extent.
(p. 23). Given the achievement levels of ELs on standardized tests, teachers
can not afford to make decisions that hinder their students’ ability to learn.
One way schools can ameliorate this problem is to decrease the amount of
time not devoted to actual instruction (Marzano, 2003), but it is up to the site
administrator to both make and enforce this type of decision.
32
Literature Regarding English Learners’ Achievement on
Standardized Tests
In 1993, California had 1,151,819 English Learners enrolled in public
schools. As of 2002, the last year for which data is available, 1,511,299
English learners are enrolled in California public schools, an increase of over
350,000 students. Since 25% of the students enrolled in California schools
are classified as ELs, the accountability requirements of No Child Left Behind
(NCLB) call into question the validity of testing students who are required to
meet grade-level standards while they are acquiring English language skills.
Given the growth of the EL student population in California and the
focus on accountability for all students, regardless of their mastery of the
English language, California needs to examine the use of the California
Standards Test (CST) to measure the academic progress of EL students. In
2001, Butler and Stevens* found that prior to the No Child Left Behind Act,
there were no requirements for measuring the progress of EL students. In
fact, EL students did not participate in standardized testing, as a rule, and
therefore, schools were unable to measure their progress at all.
Educators have tried a variety of ways to address the question
of EL student progress through standardized testing by providing tests in
students’ primary language, offering testing accommodations, and excluding
or including students in standardized tests based on their level of proficiency
in English. These accommodations did not take into the account that EL
33
students were still expected to participate in standardized assessments that
were designed for their English-only peers or students who are proficient in
the English language, whether they have achieved a reasonable level of
English language skills and regardless of the length of time that they have
been living in the USA (Butler & Stevens, 2001).
Much research has been conducted about the discrepancies between
the performance of EL students on standardized test and their English
speaking peers. Abedi, Leon, and Mirocha (2005), note, that, “English
language learner (ELL) students generally perform lower than non-ELL
students in reading, science, math and other content areas - a strong
indication of the relationship of English proficiency with achievement
assessment.” (p. 2). Furthermore, studies have shown that the more
language skills a content area requires causes the gap between EL students
and non-EL students performance to widen. Abedi et al. (2005) consider this
“language load” (p. 2) as a threat to the validity of EL student achievement on
standardized tests and cite the example of mathematic computation as an
area where language is not a significant factor in EL performance because,
“The difference between ELL and non-ELL student performance becomes
the smallest in math, particularly on math items where language has less
impact, such as on math computation items” (p. 3).
In their 2005 study, Abedi et al. examined two data sites to compare
EL student performance. One site included data from a large public school
34
district for grades 2 through 8 during the 1999 school year for the reading
and mathematics subtests of the Iowa Tests of Basic Skills (ITBS).
Information was also available about students’ race, gender, birth date, and
participation in bilingual education. The second site that was studied
included data from a state department of education for students in grades 2
through 11 who were enrolled in public school during the 1997-98 school
year. This data included results from the Stanford Achievement Test Series,
Ninth Edition (Stanford 9). Results from this study were consistent with
previous studies, namely that the level of English proficiency of a student
directly correlates to their performance on standardized assessments.
The achievement gap between the performance of EL students and
non-EL student widens as the requirement of increased language proficiency
(“language load”) rises (Abedi et al., 2005, p. 39). Abedi et al. (2005) found
that language proficiency plays an integral part in the academic achievement
of EL students.
Butler and Castellon-Wellington (2005) have similar findings to Abedi
and note that EL students exhibit lower levels of performance in content-area
subject tests than their non-EL peers. In their study, Butler and Castellon-
Wellington, examined the scores of 778 third grade students and 184
eleventh grade students who participated in both the Stanford 9 tests and the
Reading/Writing Component of the Language Assessment Scales (LAS).
Their findings indicate that, “there is a strong relationship between the
35
English language proficiency of ELL students and their performance on a
content assessment.” (p. 75).
The use of standardized assessments to measure the academic
progress of EL students is problematic, as Butler and Stevens (2001) note in
their research, the vast majority if standardized assessments are designed
with English-only students therefore the utility of measuring the content
knowledge of English Learners in severly limited. Abedi, et al. (2005) have
also found that, “ELL students perform substantially lower than non-ELL
students particularly in content areas with more language load.” (p. 21). This
type of Assessments results do not inform classroom teachers about what
ELs need to learn in order to be successful academically. Ultimately, the use
of standardized tests to determine what EL students have learned in school
is a question that will remain unanswered until reliable data is available to
show what content has been mastered and not the amount of English a
student speaks.
Interactions between the Three Topics
English Learners are now, and will continue to be, an increasing part
of the educational landscape. Schools need to understand the unique needs
of English Learners and take measures to ensure that they meet these
needs. With the federal government’s focus on measuring achievement
through standardized test scores through the No Child Left Behind Act until
at least 2014, all schools need to consider how their English Learners will
36
fare in this high stakes environment. One way for schools to meet the needs
of English Learners and federal government requirements is to consider the
use of local interventions.
This study seeks to investigate how local interventions can help
English Learners to meet state and federal achievement goals and evaluate
if local interventions are the an effective means to close the achievement
gap.
Conclusion
Schools face many challenges in meeting federal and state guidelines
for student achievement and schools that serve large populations of English
Learners can find these challenges almost insurmountable given the amount
of time it takes for ELs to achieve proficiency in English. Despite this rather
grim prospect, it is possible for schools to meet and even exceed
achievement goals if they are willing to do the work. The use of local
interventions is one way for schools to make curricular decisions that meet
the needs of the students they actually serve versus using a standardized
curriculum that may not be embraced or implemented by the teaching staff.
By allowing teachers to have input on the intervention, teachers feel their
professionalism is respected and this creates more buy-in and use of the
intended intervention.
English Learners are particularly vulnerable when it comes to
opportunity to learn because they are required to learn English at the same
37
time they are acquiring content knowledge, it is very easy for teachers to
focus on English language development at the expense of all other curricular
areas. One way that local interventions, such as the one in this study,
address this issue is by providing instruction that is standards-based and
viable for the school site where it is being used.
38
Chapter 3: Methodology
Design Summary
This study utilized a pre-post design to test the efficacy of a local
mathematics intervention for English Learners (EL). The experimental group
included all students enrolled in grades 2 – 5 who were designated as ELs at
a large underperforming urban elementary school in the Los Angeles Unified
School District (LAUSD). The intervention was implemented during the
2004-2005 and 2005-2006 school years. The 2004 California Standards
Test (CST) data on the mathematics portion of the test served as the
baseline data to measure against students’ performance on the mathematics
portion of the 2006 CST. In order to provide a summative evaluation of the
intervention, this study used an O pre X O post design:
Pre-test Observation: Mathematics portion of the 2004
California Standards Test
Treatment (X): District 4 Mathematics Alternative Guide for
Mathematics
Post-test Observation: Mathematics portion of the 2006
California Standards Test
This study compared student performance on the mathematics portion
of the 2004 CST for English Learners to student performance on the
39
mathematics portion of the 2006 CST for English Learner in order to
determine the effectiveness of the intervention.
This study also utilized qualitative methods to evaluate the
intervention. Teachers who implemented the mathematics intervention
participated in informal and open-ended interview about their experiences
using the District 4 Mathematics Alternative Guide for Mathematics.
Teachers were interviewed throughout the school year on a quarterly basis to
get their feedback on the Alternative Guide and to address any concerns or
issues that arose in their classrooms. Specifically, the interviews were used
to judge the level to which teachers were implementing the intervention and
to determine how to assist them in fully implementing the Alternative Guide.
Participants/Sampling
Participants in this study included all students identified as English
Learners by the LAUSD Home Language Survey and student performance
on the California English Language Development Test (CELDT) in grades 2
– 5. Since the school has 100% participation in the STAR testing program,
all EL student test scores were included. Participant data included mean
scaled scores as well as percentages of students scoring in the Advanced,
Proficient, Basic, Below Basic, and Far Below Basic bands on the CST. This
data was gathered from the California Department of Education using the
40
STAR website. School and student level reports were accessed from the
school archives.
Intervention Description
The experimental group received the treatment intervention in their
classrooms throughout the school year during their mathematics instructional
block, which typically was one hour of mathematics instruction per day.
Local District 4 began implementing the Alternative Guide during the second
semester of the 2004 – 2005 school year and revised the Alternative Guide
so that it could be fully implemented during the 2005 – 2006 school year.
The Alternative Guide consists of the following instructional supports for
teachers and students:
1. A grade level specific instructional timeline that aligns to each of the
four quarterly assessments mandated by the school district and aligns
with the California Standards Test.
2. Daily standards-based lessons that include pedagogy to improve
mathematics instruction.
3. A mathematics background section to help teachers understand the
rationale for the mathematics activities presented in the Alternative
Guide.
4. Concept lessons that address a critical standard from the quarter and
help students to improve their mathematical reasoning skills.
41
5. Homework support materials that align with the lessons to further
student understanding of a concept.
In addition to these materials, teachers were provided with in classroom
support as needed and attended professional development focused on
mathematics content development and instructional strategies for a minimum
of 15 hours per teacher.
Instrumentation
Quantitative Instruments
The quantitative data for this study came from two sources: The 2005
and 2006 California Standards Test (CST) data for the mathematics portion
of the test was collected by reviewing existing data as provided by the STAR
reports on the California Department of Education Website (Standardized
Testing and Reporting Program) and individual student reports were
collected from the school’s archives. Since this study is concerned with EL
performance, this study will focus on the test scores of students identified as
ELs but the State of California does not provide STAR data for all five
performance bands for the EL subgroup, so reporting of this data in the study
consists of the each school’s entire student population, regardless of English
Language proficiency. The mathematics portion of the California Standards
Test has 65questions that test student knowledge of algebra, measurement
and geometry, statistics, data analysis and probability, and number sense as
42
well as students’ mathematical reasoning abilities in a multiple-choice format.
Score reports are broken down by grade level and content area for each
school, district, county and the state of California as a whole. Per No Child
Left Behind, California has set a goal of all students being proficient on the
CST. In order to determine proficiency, the state determines a mean scaled
score, which is the group average, scaled for the each grade level and for
each content area. Scores on the CST range from 150 to 600 and any
student scoring at 350 is considered proficient. In addition to students who
are proficient, the state also provides information on the number of students
who are considered Advanced, Basic, Below Basic and Far Below Basic for
the entire school population, but not by subgroup.
Qualitative Instruments
Qualitative data for this study was collected by using an informal
interview process with teachers as issues arose and at the beginning of
every quarter when the Alternative Guide was delivered. The interviews took
place individually, as needed, and with entire grade levels during grade level
meetings. The purpose of these interviews was to determine how to best
support teacher’s classroom implementation of the Alternative Guide and to
determine the efficacy of the implementation of the lessons by each teacher.
Interviews were conducted by the evaluator and consist of both opened-
43
ended and simple response questions. Some examples of questions asked
are as follows:
1. Do you use the Alternative Guide?
2. Describe your implementation of the Alternative Guide.
3. What features of the Alternative Guide are most/least useful to
you?
4. Is the pacing of the guide realistic for your classroom and
students?
5. If you taught this grade level before, how are your students
performing using the Alternative Guide versus the textbook?
6. What are your concerns about the Alternative Guide?
7. How can the math coach support your implementation of the
guide?
8. Do you have any questions about the Alternative Guide?
Procedural Timeline
In order to meet the specifications of this study, the following timeline
was established for collection of data. This timeline was completed as
follows:
July – December 2004: Alternative Guide version one being developed and
written in by local district math coaches.
44
January – June 2005: Alternative Guide version one distributed to teachers
for classroom implementation.
July - August 2005: Teachers received training on implementing the Quarter
1 Alternative Guide, version two. Interviews scheduled with teachers to
discuss implementation.
August 2005: Data analysis of spring 2005 CST math scores.
September - November 2005: Teachers received training on implementing
the Quarter 2 Alternative Guide, version two and interviews regarding
Quarter 1 implementation took place. Analysis of 2005 Math CST scores
shared with teachers and school administration.
December 2005 – February 2006: Teachers received training on
implementing the Quarter 3 Alternative Guide, version two and interviews
regarding Quarter 2 implementation took place.
February 2006 – May 2006: Teachers received training on implementing the
Quarter 4 Alternative Guide, version two and interviews regarding Quarter 3
implementation took place. Review of current year quarterly assessment
data in preparation for the 2006 CST.
May 2006 – June 2006: Interviews regarding Quarter 4 implementation took
place as well as review of 2005 – 2006 Quarterly Assessment data to set
specific goals for the 2006 – 2007 school year.
45
Quantitative Analysis
In order to test the efficacy of the intervention, the Mann-Whitney test
to compare posttest data of the experimental and control groups using the
2006 scores from the mathematics section of the CST was used. The Mann-
Whitney test was also used to compare performance band scores on the
experimental group using the pre data from the 2004 scores on the
mathematics section of the CST and the post data from the 2006 scores on
the mathematics section of the CST. Tiered effects displays were also used
to determine the effect size of the intervention. Performance band
information was used to display the size of the difference between the
experimental and control groups and the pre/post experimental gains and
losses.
Qualitative Analysis
Creswell suggests six steps to analyzing information from qualitative
research in his book, Research Design: Qualitative, quantitative, and mixed
methods approaches (2003). Following his suggestions, qualitative
information was analyzed as follows:
Step 1: Transcribe interviews.
Step 2: Record general ideas generated by interview questions.
Step 3: “Chunk” information from interview questions.
Step 4: Identify themes generated by Step 3.
46
Step 5: Provide a detailed description of themes arising from
interview questions.
Step 6: Provide a personal interpretation of themes.
Delimitations and Limitations of the Study
One of the limitations of this study was that is a quasi-experiment and
therefore had problems with internal validity. One way to maximize internal
validity was by using a non-equivalent control group study with two control
groups. External validity has been maximized by using disaggregated
English Learner data from the California Department of Education.
47
Chapter 4: Findings
Changes in CST Math Performance: 2004 versus 2006
In analyzing the test data the five CST performance bands were
converted to a 1-4 numerical scale where CST bands Far Below Basic,
Below Basic, Basic, Proficient and Advanced were given values of 0-4,
respectively (FBB=0; BB = 1, B = 2, PROF = 3 and ADV=4). Mean
performance differences for all students in the second through fifth grade at
Carle Elementary in the pre- and post-test years are shown in Table 2.
Table 2 Mean Difference by Grade Level
Grade
Level
2006 2004 Difference
Second 2.33
(n = 131)
1.73
(n = 194)
+ .601
Third 1.95
(n = 147)
1.80
(n = 197)
+.149
Fourth 1.74
(n = 168)
1.89
(n = 209)
+.145
Fifth 1.72
(n = 173)
1.18
( n = 186)
+.540
Total 1.95
(n = 619)
1.62
(n = 786)
+.329
Second and fifth grade had the highest mean differences for the pre-
and post-test years and third and fourth grades had smaller mean
differences. This would indicate that the intervention was more successful in
second and fifth grades, but when all grade levels are considered, the mean
difference is +.329 which would indicate the overall effectiveness of the
intervention in the four combined grade levels.
48
Using the Mann-Whitney U test to analyze these data shows that
there is a significant statistical difference between the pre- and post-test
performance on the mathematics section of the CST of the second and fifth
grade students at Carle Elementary. Mann-Whitney results for each grade
level are shown Table 3 below. The increase in CST performance from 2004
to 2006 was statistically significant (p = .001) for second and fifth grades, but
not for the third and fourth grades (see Table 1). The overall difference in the
mean test scores from 1.62 to 2.05 (Basic) in the total sample also was
statistically significant (p = .001).
Table 3 Mann-Whitney U Test Results by Grade Level
2
nd
Grade 3
rd
Grade 4
th
Grade 5
th
Grade
Mean Rank 2004 143.90 168.07 185.06 162.17
Mean Rank 2006 191.28 178.43 195.14 199.17
ZTest +4.593 +.985 +1.019 3.516
Observed Prob. .001 .325 .308 .001
Table 4 shows the effect size estimate for the means differences in
CST performance classifications. Effect size was estimated by dividing the
difference by the observed standard deviation of the CST Math 2004
classifications.
49
Table 4 Effect Size Estimates
2
nd
Grade 3
rd
Grade 4
th
Grade 5
th
Grade
Gain +.601 +.149 +.145 +.540
2004 SD 1.102 1.085 1.014 .975
Effect Size +.543 +.137 +.142 +.553
The effect size estimate was +.305 for the total sample.
Changes in CST Proficiency Rates: 2004 versus 2006.
The performance classifications shown in Table 5 below also
demonstrate student growth from 2004 to 2006. These same results are
reconfigured in Table 6 as student proficiency rates on the CST according to
the standards of the No Child Left Behind Act (percent of the students who
scored proficient or advanced). As shown in the Table 6 cross-tabulation,
student proficiency levels in second grade doubled from 25.3% in 2004 to
51.1% in 2006, and fifth grade student proficiency levels nearly tripled from
11.3% in 2004 to 31.2% in 2006. Both are significant increases (Fisher z
test, p = .001). While the proficiency increases in third and fourth grade were
not statistically significant the lack of statistical significance could be
attributed to the lessened effectiveness of the intervention at these two grade
levels and the implementation of the intervention (see Discussion chapter).
50
Table 5 Performance Bands by Grade Level
Grade FBB BB B P A N
2
nd
2004 12.9 33.0 28.9 19.1 6.2 194
2
nd
2006 7.6 19.1 22.1 35.1 16.0 131
3
rd
2004 10.2 36.0 24.9 21.8 7.1 197
3
rd
2006 15.6 26.5 21.1 21.1 15.6 147
4
th
2004 8.1 39.2 26.8 22.0 3.8 209
4
th
2006 13.1 31.0 22.0 22.0 11.9 168
5
th
2004 24.2 46.8 17.7 9.1 2.2 186
5
th
2006 22.0 30.1 16.8 16.2 15.0 173
Total 2004 13.6 38.7 24.7 18.2 4.8 786
Total 2006 15.0 27.1 20.4 22.9 14.5 619
Table 6 Proficiency Rates by Grade Level
2
nd
Grade 3
rd
Grade 4
th
Grade 5
th
Grade
Pass Rates
2004
25.3% 28.9% 25.8% 11.3%
Pass Rates
2006
51.1% 36.7% 33.9% 31.2%
+25.8% +7.8% +8.1% +19.9%
Fisher Exact p .001 .131 .090 .001
51
Table 7 Total Sample Proficiency Rates for Pre- and Post Test Data
Proficiency Fisher Exact
0 1
2004 77.0%
(n=605)
23.0%
(n=181)
2006 62.5%
(n=387)
37.5%
(n=232)
Total +14.5% p=.001
In Table 7, the Fisher Exact Test indicates that the increase in student
proficiency (23% to 37%) on the mathematics portion of the CST for the four
combined grade levels was not due to sampling error and the probability of
these changes (p = .001) in the proficiency rates would not occur by chance.
Changes in CST “Basic or Above” Rates: 2004 versus 2006.
Because the NCLB proficiency guidelines may hide important gains at
the lower levels (e.g., BB Basic), the proportions of students scoring
“Basic or Above” are shown in Tables 8 and 9. These results indicate that the
overall increase in the proportion of students achieving “Basic or Above”
status, although not recognized as indicating student proficiency under the
current NCLB guidelines, is encouraging, particularly in Grades 2 and 5.
Table 9 indicates that overall there has been about a 20% increase from
2004 to 2006 in the number of students classified as “Basic or Above” at Eric
Carle Elementary School.
52
Table 8 Basic or Above Rates by Grade Level
2
nd
Grade 3
rd
Grade 4
th
Grade 5
th
Grade
Basic & Above
Rates 2004
54.2%
(n=105)
53.8%
(n=106)
52.6%
(n=110)
29%
(n=54)
Basic & Above
Rates 2006
73.2%
(n=96)
57.8%
(n=85)
55.9%
(n=94)
48%
(n=83)
Fisher Exact p .001 .131 .090 .001
Table 9 Rates for Total Students Scoring “Basic or Above”
“Basic” of Higher Fisher Exact
0 1
2004 50.3%
(n=770)
49.7%
(n=762)
2006 40.9%
(n=508)
59.1%
(n=735)
Total -9.4% +9.4% p=.001
Changes in CST Math Performance:
Experimental versus Control Group Comparisons
Table 10 Between-Subjects Effects
Dependent Variable: Perform
Source
Type III Sum
of Squares df
Mean
Square F Sig.
Corrected Model
327.444(b) 23 14.237 11.491 .000
Intercept
11414.263 1 11414.263 9212.863 .000
Grade
172.011 3 57.337 46.279 .000
Year
37.546 1 37.546 30.305 .000
School
19.459 2 9.730 7.853 .000
Grade * Year
36.297 3 12.099 9.766 .000
Grade * School
55.729 6 9.288 7.497 .000
Year * School
15.096 2 7.548 6.092 .002
Grade * Year *
School
16.627 6 2.771 2.237 .037
Error
4660.925 3762 1.239
Total
16771.000 3786
Corrected Total
4988.369 3785
a Computed using alpha = .05
b R Squared = .066 (Adjusted R Squared = .060)
53
Table 10 shows that the most statistically significant factor (.001)
affecting the achievement were the year being studied, the school being
studied, the grade and the year being studied as well as the grade and the
school being studied.
When comparing Eric Carle (experimental (E) school 1) to the two
control (C) schools, Pine Elementary and Donne Elementary, to evaluate the
efficacy of the intervention, we need look at grade level separately. A 2 way
(year by school) ANOVA was used to analyze the results from each school
site.
Second Grade Findings
In second grade, the analysis of the performance band means (Table
11) showed that there were differences in achievement at all three schools
from 2004 to 2006 with Eric Carle (E school 1) and Pine Elementary (C
school 2) showing greater differences than Donne Elementary (C school 3).
Overall the difference from 2004 to 2006 in all three schools 1.73 to 2.27
(one-half a performance band) was statistically significant, F (1, 877) = 48.1,
p = .001 (Table 12). Table 12 shows that the most significant factor for
interpreting the differences between schools was the year being studied
followed by the school and then the school and year interaction.
54
Table 11 Second Grade Means
Descriptive Statistics
Dependent Variable: Perform
1.73 1.102 194
1.75 1.062 161
1.73 1.010 142
1.73 1.062 497
2.33 1.180 131
2.40 1.120 139
2.04 1.145 116
2.27 1.155 386
1.97 1.170 325
2.05 1.135 300
1.87 1.082 258
1.97 1.134 883
School
1
2
3
Total
1
2
3
Total
1
2
3
Total
Year
2004
2006
Total
Mean Std. Deviation N
Table 12 Second Grade ANOVA
Tests of Between-Subjects Effects
Dependent Variable: Perform
Source
Type III Sum
of Squares df
Mean
Square F Sig.
Corrected
Model
70.230(b) 5 14.046 11.579 .000
Intercept
3424.037 1 3424.037 2822.739 .000
Year
58.327 1 58.327 48.084 .000
School
4.937 2 2.469 2.035 .131
Year * School
4.548 2 2.274 1.875 .154
Error
1063.818 877 1.213
Total
4551.000 883
Corrected Total
1134.048 882
a Computed using alpha = .05
b R Squared = .062 (Adjusted R Squared = .057)
Third Grade Findings
For third grade, Table 13 shows the Pine Elementary (C school 2) had
the largest difference (+0.37) of the three schools, but that this interaction
55
was not significant statistically, F (2,936) = 2.11, p = .122 (Table 13). The
gain in all three schools from 2004 to 2006 (1.81 to 2.00) was statistically
significant, F (1, 936) = 4.21, p = .040. This suggests that all three schools
have made progress from 2004 to 2005. Looking at the third grade ANOVA
in table 14, the year being studied was the most significant factor, followed
by the school being studied and finally, the year and school interaction.
Table 13 Third Grade Means
Descriptive Statistics
Dependent Variable: Perform
1.80 1.111 197
1.65 1.092 198
2.07 .968 139
1.81 1.079 534
1.95 1.318 147
2.02 1.291 160
2.04 1.288 101
2.00 1.297 408
1.86 1.204 344
1.82 1.197 358
2.06 1.111 240
1.89 1.181 942
School
1
2
3
Total
1
2
3
Total
1
2
3
Total
Year
2004
2006
Total
Mean Std. Deviation N
56
Table 14 Third Grade ANOVA
Dependent Variable: Perform
Source
Type III
Sum of
Squares df
Mean
Square F Sig.
Corrected
Model
22.921(b) 5 4.584 3.325 .006
Intercept
3297.624 1 3297.624 2391.836 .000
Year
5.805 1 5.805 4.210 .040
School
7.429 2 3.715 2.694 .068
Year * School
5.813 2 2.906 2.108 .122
Error
1290.463 936 1.379
Total
4692.000 942
Corrected Total
1313.384 941
a Computed using alpha = .05
b R Squared = .017 (Adjusted R Squared = .012)
Fourth Grade Findings
Data analysis of the fourth grade (Tables 15 and 16) shows that Eric
Carle had a slight mean difference that was not statistically significant
(+0.15) while Pine (school 2) and Donne (school 3) had slight decreases in
their means, -0.13 and -0.14 respectively. This would indicate that other
factors may have caused the lack of student achievement gains in the fourth
grade in all three schools. These factors will be reviewed in the Discussion
chapter. Analysis of table 16 shows that the most significant factor at this
grade level was the school being studied followed by the year and school
interaction.
57
Table 15 Fourth Grade Means
Descriptive Statistics
Dependent Variable: Perform
1.74 1.014 209
1.84 1.073 182
1.70 .849 142
1.77 .994 533
1.89 1.235 168
1.71 1.119 150
1.56 1.098 116
1.74 1.165 434
1.81 1.119 377
1.78 1.094 332
1.64 .969 258
1.75 1.074 967
School
1
2
3
Total
1
2
3
Total
1
2
3
Total
Year
2004
2006
Total
Mean Std. Deviation N
Table 16 Fourth Grade ANOVA
Dependent Variable: Perform
Source
Type III Sum
of Squares df
Mean
Square F Sig.
Corrected
Model
9.316(b) 5 1.863 1.622 .151
Intercept
2830.353 1 2830.353 2463.503 .000
Year
.411 1 .411 .358 .550
School
5.309 2 2.655 2.311 .100
Year * School
4.464 2 2.232 1.943 .144
Error
1104.107 961 1.149
Total
4088.000 967
Corrected Total
1113.423 966
a Computed using alpha = .05
b R Squared = .008 (Adjusted R Squared = .003)
Fifth Grade Findings
Finally, analysis of fifth grade student achievement data indicates that
Eric Carle had a large performance band gain in this grade level (+.54 or
58
slightly over one-half a performance band), while Pine (C school 2) and
Donne (C school 3) had declines in their means of -0.02 and -0.33,
respectively. This statistical interaction was statistically significant,
F (2, 988) = 7.17, p = .001. Analysis of table 18 indicates that the most
significant factor at the fifth grade level was the school being studied followed
closely by the year and school interaction and finally the year being studied.
Table 17 Fifth Grade Means
Descriptive Statistics
Dependent Variable: Perform
Year School Mean
Std.
Deviation N
1 1.18 .975 186
2
1.75 1.117 205
3
1.11 .764 149
2004
Total
1.38 1.022 540
1
1.72 1.370 173
2
1.73 1.234 175
3
1.11 .969 106
2006
Total
1.58 1.258 454
1
1.44 1.210 359
2
1.74 1.170 380
3
1.11 .853 255
Total
Total
1.47 1.140 994
59
Table 18 Fifth Grade ANOVA
Tests of Between-Subjects Effects
Dependent Variable: Perform
Source
Type III Sum
of Squares df
Mean
Square F Sig.
Corrected
Model
87.117(b) 5 17.423 14.315 .000
Intercept
1953.039 1 1953.039 1604.610 .000
Year
7.137 1 7.137 5.864 .016
School
59.344 2 29.672 24.378 .000
Year * School
17.449 2 8.724 7.168 .001
Error
1202.537 988 1.217
Total
3440.000 994
Corrected Total
1289.654 993
a Computed using alpha = .05
b R Squared = .068 (Adjusted R Squared = .063)
API Findings
An analysis of the API growth for English Learners for the 2005 - 2006 school
year for show that ELs had 22 points of growth compared with the Eric
Carle’s overall growth of 27 points. Therefore, the EL API score would be
632 versus 637 for the entire school. Prior to the 2005-2006 school year, the
California Department of Education did not publish disaggregated data on EL
API growth.
60
Chapter 5 – Discussion
In analyzing the data from this study, it is important to consider the
affect that the intervention had on the teachers’ practice and student
performance in grades 2 - 5 at Eric Carle Elementary School. The primary
outcome, performance on the math CST was scaled as follows: 0 Far Below
Basic, 1 Below Basic, 2 Basic, 3 Proficient and 4 Advanced
The mean gain for Eric Carle for all grade levels was from 1.62 to 1.95
or one-third of a proficiency band. In terms of proficiency as per NCLB, the
rate increased from 23% to 37%. , Not only is the growth on the performance
bands and on the proficiency rate statistically significant (p=.001), it is
practically significant and the reason why Eric Carle met both the Adequate
Yearly Progress (AYP) goals and the state Academic Performance Index
(API) goals in mathematics. In fact, the percentage of students scoring at
Proficient and Above in mathematics (37.5%) exceeds the 2008 mathematics
AYP goal. The high percentage of students at Eric Carle scoring Basic or
Higher (59.1%) also is noteworthy.
The effect size estimate of the intervention for all grade levels was
+.305, nearly one-third of a standard deviation on the 0-4 performance band
score. The effect size is important because it is a measure of how well Eric
Carle performed in mathematics on the 2006 California Standards Test
(CST) using the mathematics intervention in comparison to the pre-test year
(2004).
61
In comparing Eric Carle to the two control schools in this study, Pine
Elementary School and Donne Elementary School, the results were mixed
but still interesting in terms of the efficacy of the mathematics intervention. In
second and fifth grades, Carle had statistically significant gains, while in third
and fourth grades, the gains were marginal. Overall, the analysis suggests
that the intervention was effective as evidenced by student test score
increases.
Second Grade Discussion
Second grade students at Eric Carle showed the greatest differences
of all grade levels with a mean proficiency band gain of +.601 on the
mathematics section of the CST from 2004 to 2006. This difference indicates
that the mathematics intervention was most successful at this grade level
and that the increased scores did not occur randomly, as evidenced by the
Fisher Exact Test results that indicated an observed probability of .001. As
noted in chapter one, students at Eric Carle are seen a positive force within
the school.
Teachers are responsible for the implementation of the intervention
and the student gains indicate that second grade teachers were able to
successfully implement the intervention. The effect size estimates for
second grade show student achievement increased by one-half of a standard
deviation. This type of gain on the CST in second grade means that more
students are prepared to meet grade level standards and achieve scores of
62
Proficient or above on the CST in subsequent years as past test performance
is the greatest indicator of future test performance.
In looking at the intervention in second grade, teacher interview and
researcher observations support the idea of high level implementation.
Teachers requested grade level meetings at the beginning of each quarter to
review the material that would be covered during the eight-week instructional
block and to plan ways to support student learning to prepare them for the
quarterly assessment. The Mathematics Instructional Coach facilitated these
grade level meetings to ensure that all relevant questions were answered
and to provide support as needed. The Mathematics Instructional Coach
also scheduled demonstration lessons and observations during these grade
level meetings to make sure that all classrooms had equal access to this type
of instructional support.
School administration also played a role in the success of second
grade by allowing teachers to determine the content of their grade-level
meetings and to meet during school hours. Administrators attended
approximately half of the grade level meetings that focused on mathematics
and played the role of observer, letting teachers set the tone for the
discussion that took place. This type of support by administration helped
teachers to feel that they had control over their professional development
time and made them more willing to accept the intervention and implement it.
63
Overall, 51% of second grade students scored Proficient or Advanced
on the mathematics section of the 2006 CST, a 25% increase over 2004.
While this increase is both impressive and statistically significant, if we were
to look at the numbers of students who scored Basic or above, this
percentage would increase to 73%. By focusing on this additional 22% of
students in the years to come, Eric Carle could see even greater increases in
grades 3, 4, and 5.
Third Grade Discussion
Results for third grade were not as promising as the second grade
results. However, results did show a mean proficiency band gain of +.149
which would indicate that the gains were similar to the growth that would be
expected over the course of one school year and do not support the efficacy
of the intervention at this grade level.
Analysis of the three schools in the study indicate that from 2004 to
2006 all three schools made gains in third grade but that these gains were
not statistically significant. For Eric Carle third grade teachers these findings
seem to indicate that the intervention was not fully implemented. Although
teachers are this grade level received training on a quarterly basis to
familiarize them with the Alternative Guide, they were not as likely to take
advantage of the Mathematics Instructional Coach’s expertise and schedule
classroom visitations and demonstration lessons.
64
Qualitative data from this grade level indicates that teachers felt
overwhelmed about teaching multiplication and division to students that they
felt were unprepared to learn these concepts. There was a great deal of
attention paid to the fact that students lacked automaticity of multiplication
facts for all of the times tables from 0 to 10 when student entered the grade
level despite the fact that the third grade level standards clearly state that
students learn these facts in third grade, not prior to third grade.
Observational data also indicates that third grade teachers spent more
time teaching procedural lessons, for example, how to set up a multi-digit
multiplication problem, than teaching for understanding, for example, the
meaning of multiplication and its relationship to division. This focus on
procedures can make it difficult for students to score well on the CST since
only a small percentage of the questions are strictly computational.
Administration could have focused more on this grade level by
attending grade level meetings on a regular basis. In general, third grade
had grade level meetings on Wednesdays which would coincide with district
principal meetings so that the principal or assistant principal would not
always be on campus. Still, it is crucial for administration for send a clear
message to the staff about the importance of the intervention if they want to
ensure its success.
Overall, 36.7% of third grade students scored Proficient or Advanced
on the mathematics section of the 2006 CST, a 7.8% increase over 2004.
65
While this increase exceeds the 2006 AYP goal it is not statistically
significant. Looking at the numbers of students who scored Basic or above,
this percentage would increase to 57.8%. Third grade teachers need to
reconsider their teaching methods and use of the intervention in order to
significantly increase third grade student achievement.
Fourth Grade Discussion
Results for fourth grade were similar to the results from third grade.
The mean proficiency band gain was +.145 which, again, indicates growth
similar to what would be expected in school year and does not support the
efficacy of the intervention at this grade level.
Data analysis of the fourth grade shows that Eric Carle had a slight
mean difference that was not statistically significant (+0.15) while the two
control schools had slight decreases in their means, -0.13 and -0.14
respectively. This would indicate that other factors may have caused the
lack of student achievement gains in the fourth grade in all three schools.
At Eric Carle one of the factors that affected student achievement
gains was the teachers themselves. Qualitative data from this grade level
indicates that fourth grade teachers were the least likely to use the
intervention on a regular basis and devote sufficient instructional time for
mathematics. Fourth grade teachers also requested the least amount of
support from the Mathematics Instructional Coach. The fourth grade
66
teachers were some of the most veteran teachers on the staff and most
resistant to making changes in their instructional practices. While teachers at
this grade level received the same kind of training on the instructional guide
as all other grade levels, they were the least receptive the training and on
several occasions throughout the study asked for a second copy of the
Alternative Guide because they had either lost of misplaced their original
copy.
Qualitative data from this grade level also indicates that teachers felt
overwhelmed by the number of students they had to teach, the different
ability levels of their students, and the number of standards they were
required to teach. While it is true that fourth grade has the most mathematics
standards in elementary school, the Alternative Guide was designed to
address this issue. Fourth grade is the first grade where class size
increases. At Eric Carle, students went from being one of twenty students in
a class to one of 30 to 33 students depending on which class and track
assignment. Teachers at this grade level also complained that students
entering their classrooms had not mastered basic multiplication facts, with
one teacher going so far as to state that they would not teach any other
concept until the entire class had mastered multiplication.
Observational data indicates that fourth grade teachers spent a larger
portion of their mathematics instructional time involved with classroom
management. Rather than teach math, teachers spend anywhere from 10 to
67
30 minutes of the instructional time asking students to sit down, get out
materials, or stop lessons to reprimand students who were off task. Clearly,
if a classroom is poorly managed, it is difficult to teach any concept and
losing such a significant amount of instructional time would hinder any
student’s ability to be successful.
Administrative support for this grade level was lacking. Sign in
records for grade level meetings indicate that an administrator had only three
grade level meetings during the 2005 – 2006 school year and that no
administrator attended any grade level meeting in the 2004 - 2005 school
year. It is possible that administrator did not feel their presence was
necessary since the teaching staff was experienced but this grade level
would have benefited greatly from the presence of an administrator at their
meetings to underscore the importance, not only of the Alternative Guide, but
of mathematics instruction in general.
Overall, 33.9% of fourth grade students scored Proficient or Advanced
on the mathematics section of the 2006 CST, an 8.1% increase over 2004.
While this increase also exceeds the 2006 AYP goal it is not statistically
significant. The numbers of students who scored Basic or above was 55.9%.
Fourth grade teachers have several needs when it comes to mathematics
instruction. They need to reconsider their teaching methods and, most
importantly, their relationship to their students and the expectations they
68
have for their students in terms of mathematics achievement if Eric Carle is
going to see an increase in student performance on the CST next year.
Fifth Grade Discussion
Fifth grade students at Eric Carle showed the second greatest gains of
all grade levels with a mean proficiency band difference of +.540 on the
mathematics section of the CST from 2004 to 2006. This difference indicates
that the mathematics intervention was successful at this grade level and that
the gain would not have occurred randomly, as evidenced by the Fisher
Exact Test results that indicated an observed probability of .001. The effect
size estimates for fifth grade show student achievement increased by one-
half of a standard deviation.
In looking at the intervention in fifth grade, teacher interview and
researcher observations support the idea of high level implementation. Fifth
grade teachers had the requisite grade level meetings at the beginning of
each quarter to review the material that would be covered and also asked for
additional meetings after school to discuss implementation of the Alternative
Guide. The Mathematics Instructional Coach facilitated both the grade level
and after school meetings to ensure that all questions were answered and to
sufficient support was provided to teachers. The Mathematics Instructional
Coach also scheduled the most demonstration lessons and observations for
69
this grade level. While fourth grade may have the most grade level
standards to cover, fifth grade has the most difficult standards to teach.
Fifth grade teachers sought the most support from the Mathematics
Instructional Coach. Review of school records indicate that the Mathematics
Instructional Coach was in at least one fifth grade classroom every week of
the school year either demonstrating lessons or observing classroom
teachers. The fifth grade teachers, while not necessarily the least
experienced on staff, were the least experienced in the grade level which
contributed to their help seeking behavior.
School administration was aware of the teachers’ experience in fifth
grade and paid close attention to the instruction at this grade level. It should
also be noted that two of the three school administrators had been fifth grade
teachers when they were in the classroom, so they may have been more
familiar with both the standards and instructional practices in this grade level.
An administrator was present at almost every grade level meeting and would
engage in discussion with the teachers when they had questions or
comments about the Alternative Guide. Again, his type of support by
administration helped teachers to feel that they had control over their
professional development time and made them more willing to accept the
intervention and implement it.
Overall, 31.2% of fifth grade students scored Proficient or Advanced
on the mathematics section of the 2006 CST, a nearly 20% increase over
70
2004. While this increase is both impressive and statistically significant, if we
were to look at the numbers of students who scored Basic or above, this
percentage would increase to 48%. Both of these increases are even more
significant given the difficulties encountered in third and fourth grade for
students at Eric Carle Elementary School.
Theoretical Implications
Given the importance of student achievement under the No Child Left
Behind Act, the overall growth at Eric Carle due the Alternative Guide local
intervention is practically significant. The Alternative Guide was conceived
and designed with English Learners in mind. Teacher needs were also
considered as well as the amount of time available to teach mathematics
given other curricular constraints. In this section, the theoretical implications
of these findings will be discussed.
By focusing on and providing ELs with what Marzano (2003)calls
opportunity to learn in the area of mathematics we are giving them the best
chance to be successful academically. Again, Marzano has found that,
“Opportunity to learn (OTL) has the strongest relationship with student
achievement of all school-level factors.” (p.22). If English Learners are not
given an opportunity to learn how will they achieve at the level of their
English speaking peers? Since, as Marzano notes, “teachers commonly
make independent and idiosyncratic decisions regarding what should be
71
covered and to what extent” (p.23) it is hard to know if students are receiving
instruction in all the standards that the state and district require and how
much of each standard students are learning. The pacing and clear
guidelines of the Alternative Guide leave nothing to chance when it comes to
mathematics instruction.
Teachers also need to feel autonomous and as if their input in
the process is valued. The Alternative Guide was never prescriptive, but
allowed teachers to make the choice about sequencing and delivering the
instruction. Zmuda, Kuklis, and Kline (2004) support this idea with their
findings that it is the responsibility of the local school site personnel to
determine how they will approach and sustain a “continuous improvement
effort” to transform the curricula and practice at their school. Darling-
Hammond (1997) also notes that effective and meaningful change must
occur at the site level, and be responsive to the individual and specific needs
of the “local context” if the change is going to be successful.
Administrative support of the intervention helped to make it successful
at Eric Carle, even though more could have been done on the part of
administration at specific grade levels. A school’s administration has the
ability to keep a school focused on student achievement and “the extent to
which the leader establishes clear goals and keeps those goals in the
forefront of the school’s attention” (Marzano, 2005, p. 50) can lead to greater
student gains on standardized tests. To keep teachers focused on the local
72
intervention in mathematics used in this study, the school administration had
the responsibility of continually referring to the mathematics quarterly
assessment data to track student achievement throughout the school year
and help teachers set new goals for struggling students.
Another key to the success of the intervention at Eric Carle
Elementary was the “administrator’s ability and willingness to provide input
regarding classroom practices was one of the most highly valued
characteristics reported by teachers.” (Marzano, 2005, p. 54). Because the
administration was willing to become involved in the daily workings of
classrooms instruction and knowing what lessons would be taught in any
classroom on a given day, they were able to provide teachers with
constructive and relevant feedback on their teaching helped to encourage
struggling teachers and students.
Standardized testing is integral to the current accountability system in
place for public schools. The success of this intervention clearly shows that
ELs can be successful on standardized test if they are given ample
opportunity to learn the material. As Lachat (2004) has found, there is no
fixed time frame for how long it takes an English Learner to develop sufficient
proficiency in English to use it for academic purposes, with some researchers
noting that it can take five to seven years before EL students are able to use
English in an academic setting. By focusing on language development
through the use of mathematical vocabulary, the intervention helped support
73
EL achievement as evidenced by the CST score gains over the course of this
study.
This study has internal validity issues because of its quasi-
experimental design. The use of a quasiexperimental design was an attempt
to maximize internal validity through the use of two control groups. Looking
at the data from the control schools, validates the student achievement in
grades 2 and 5 because the experimental school had such large growth that
can not be explained by sampling error and thus could reasonably be
attributed to the use of the intervention. However, without randomization, it is
important to recognize that other factors may explain why the intervention
was more effective in the 2
nd
and 5
th
grades.
External validity is also a concern for this study. It is unknown
whether the study’s intervention could be used in other schools with other
trainers, teachers, and students. The use of disaggregated data from the
California Department of Education is one attempt to maximize the external
validity by comparing the control schools growth as reported to the state.
However, the state’s current policy of not reporting grade level performance
band results for disaggregated groups, made it impossible to know if the
study results for ELL students were identical to those reported herein.
Nevertheless, this limitation is not a serious problem in this particular study
because of the high proportion of English Language Learners in Eric Carle
Elementary which is a Title I school in the MacArthur Union area of Los
74
Angeles with a student population of 1,000, where 95% of the students are
Latino and 82% are identified as English Language Learners.
Conclusion
This study’s intervention has had enormous impact on Eric Carle
Elementary. Teachers have begun to look at their teaching practice in
mathematics in a new way. Instead of seeing each school year as a series
of standards to be taught, they are thinking more about how the standards
relate across grade levels and how this year’s class performance affects next
year and, ultimately, student success in middle and senior high school.
Teachers have been encouraged by the test score gains and these gains
give them reason to continue to improve their practice in mathematics.
The success story of Eric Carle is one that can easily be repeated in
Los Angeles. The use of local intervention to increase student achievement
is an idea whose time has come. Schools can increase student achievement
by staying focused on a set of goals. School administration needs to state
these goals in a clear manner and commit to monitoring the goals throughout
the school year and by providing opportunity to learn. Teachers need to
commit to seeing the goals through to fruition by employing effective
instructional practices.
75
Recommendations from this study are as follows:
1) Schools create clear grade level learning goals for all students based
on state content standards
2) School administration monitor these goals on a bimonthly basis with
assessments to track student progress
3) Teachers commit to using effective instructional practices to increase
student understanding
4) Teachers monitor student progress with in class assessments on a
weekly basis
5) Teachers and administrators meet on a bimonthly basis to discuss
student progress and alter instruction and assessment practices as
needed
The process of continuous improvement is just that, continuous, and all
stakeholders at a school need to be vigilant and committed to student
academic success in order to meet state and federal guidelines, and the
potential of the students they serve.
76
References
Abedi, J., Leon, S., & Mirocha, J. (2000/2005). Examining ELL and non-ELL
student performance differences on their relationship to background
factors: Continued analyses of extant data. In The validity of administering
large-scale content assessments to English language learners: An
investigation from three perspectives (CSE Rep. No. 663, pp. 1- 45). Los
Angeles: University of California, National Center for Research on
Evaluation, Standards, and Student Testing (CRESST).
Abedi, J., Hofstetter, C. & Lord, C. (2004). Assessment accommodations for
English language learners: Implications for policy-based empirical
research. Review of Educational Research, 74(1), 1-28.
Abedi, J. & Lord, C. (2001). The Language Factor in Mathematics Tests.
Applied Measurement in Education, 14(3), 219-234.
Butler, F., & Castellon-Wellington, M. (2000/2005). Students’ concurrent
performance on tests of English language proficiency and academic
achievement. In The validity of administering large-scale content
assessments to English language learners: An investigation from three
perspectives (CSE Rep. No. 663, pp. 1- 45). Los Angeles: University of
California, National Center for Research on Evaluation, Standards, and
Student Testing (CRESST).
Butler, F.A. & Stevens, R. (2001). Standardized assessments of the content
knowledge of English language learners K – 12: Current trends and
old dilemmas. Language Testing 2001, 18 (4) 409-427.
California Department of Education (2005). 2005 Accountability Progress
Report. Retrieved October 5, 2006, from
http://www.cde.ca.gov/ta/ac/ay/documents/aypcriteria05.pdf
California Department of Education (2006). Student Profiles and Reports..
Retrieved February 16, 2007 from URL:http://www.ed-data.k12.ca.us
California Department of Education (2006). Item and time charts for the
CSTs, CAT/6 Survey, and CAPA. Retrieved October 5, 2006, from
http://www.cde.ca.gov/ta/tg/sr/documents/itemchartv4.doc.
Clark, R. & Estes, R. (2002). Turning research into results: A guide to
selecting the right performance solutions. Atlanta, GA, CEP Press.
77
Collier, V. (1992). A synthesis of studies examining long-term language
minority student data on academic achievement. Bilingual Research
Journal, 16(1&2), 187-212.
Creswell, J. (2003). Research design: Qualitative, quantitative and mixed
methods approaches. Sage Publications, Thousand Oaks, CA.
Lachat, M. (2004). Standards-based instruction and assessment for English
Lanaguage Learners. Corwin Press, Thousand Oaks, CA.
Marzano, R. (2005). School leadership that works: From research to results.
Association for Supervision and Curriculum Development, Alexandria, VA.
Marzano, R. (2003). What works in schools: Translating research into action.
Association for Supervision and Curriculum Development, Alexandria, VA.
Reeves, D. (2004). Making standards work: How to implement standards-
based assessments in the classroom, school, and district. Advanced
Learning Press, Englewood, CO.
Patton, M.Q. (2002). Qualitative research & evaluation methods. (3
rd
ed.).
Thousand Oaks, CA: Sage Publications.
Trouilloud, D., Sarrazin, P., Martinek, T. & Guillet, E. (2002). The influence of
teacher expectations on student achievement in physical education
classes: Pygmalion Revisited. European Journal of Social Psychology.
32, 591-607.
United States Code (2002). Public Law 107-110, also known as the No Child
Left Behind Act of 2001.
Wigfield, A., Galper, A., Denton, K. & Seefeldt, C. (1999). Teachers’ beliefs
about former Head Start and non-Head Start first-grade children’s
motivation, performance, and future educational prospects. Journal of
Educational Psychology, 91, 98-104.
Zmuda, A., Kuklis, R. & Kline, E. (2004). Transforming Schools: Creating a
culture of continuous improvement. Association for Supervision and
Curriculum Development, Alexandria, VA.
Abstract (if available)
Abstract
The purpose of this study is to examine the impact of a standards-based, locally designed intervention, The Alternative Guide, on the academic achievement of English Learners (EL) on the mathematics portion of the California Standards Test (CST). A nonequivalent control groups design with one dependent variable and two matched control groups was used. This quasi-experimental design consisted of two control groups that were not randomly assigned. The Alternative Guide treatment was administered in the experimental school and pre and post intervention comparisons were made between the experimental school and the two control schools.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
An evaluation of the impact of a standards-based intervention on the academic achievement of English language learners
PDF
The effect of a language arts intervention on the academic achievement of English language learners
PDF
English learners' performance on the California Standards Test at Aviles Elementary
PDF
An evaluation of the impact of a standards-based intervention on the academic achievement of algebra students
PDF
The block schedule and the English language learners: impact on academic performance and graduation rate at Oxbow High School
PDF
An evaluation of the impact of direct instruction intervention on the academic achievement of English language learners
PDF
Examining the effectiveness of the intervention programs for English learners at MFC intermediate school
PDF
Closing the science achievement gap for ninth grade English learners through standards- and inquiry-based science instruction
PDF
The implementation of strategies to minimize the achievement gap for African-American, Latino, English learners, and socio-economically disadvantaged students
PDF
The impact of professional learning communities and block scheduling on English language learner students at Oak Point Middle School
PDF
The effects of open enrollment, curriculum alignment, and data-driven instruction on the test performance of English language learners (ELLS) and re-designated fluent English proficient students ...
PDF
Evaluation of the progress of elementary English learners at Daisyville Unified School District
PDF
An analysis of the impact of the total educational support system direct-instruction model on the California standards test performance of English language learners at experimental elementary school
PDF
English language learners utilizing the accelerated reader program
PDF
The effect of site support teams on student achievement in seven northern California schools
PDF
The impact of restructuring the language arts intervention program and its effect on the academic achievement of English language learners
PDF
The effectiveness of the literacy for success intervention at Wilson Middle School
PDF
The implementation of studio intervention at a medical magnet high school
PDF
Status of teaching elementary science for English learners in science, mathematics, and technology centered magnet schools
PDF
The comparison of hybrid intervention and traditional intervention in increasing student achievement in middle school mathematics
Asset Metadata
Creator
Manning, Jill Michelle
(author)
Core Title
Alternatives for achievement: a mathematics intervention for English learners
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Education
Publication Date
04/20/2007
Defense Date
03/14/2007
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
elementary mathematics,English learners,OAI-PMH Harvest
Language
English
Advisor
Hocevar, Dennis (
committee chair
), Brown, Richard Sherdon (
committee member
), Sullivan, Elyse (
committee member
)
Creator Email
jmmannin@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-m428
Unique identifier
UC1227996
Identifier
etd-Manning-20070420 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-478955 (legacy record id),usctheses-m428 (legacy record id)
Legacy Identifier
etd-Manning-20070420.pdf
Dmrecord
478955
Document Type
Dissertation
Rights
Manning, Jill Michelle
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Repository Name
Libraries, University of Southern California
Repository Location
Los Angeles, California
Repository Email
cisadmin@lib.usc.edu
Tags
elementary mathematics
English learners